Skip to content

Lesson 6 of 6

Hand Over: Monitoring, Changing the Model Safely and the Runbook

Leave the customer able to see whether the system still works, change it without breaking it, and act when it fails, with an evaluation set that keeps growing.

5 minForward Deployed Engineering

After this lesson you will be able to hand an AI deployment over to a customer: with an honest measure of how well it works, a routine for watching it, a safe way to change it, and a runbook.

The honest number

In lesson 4 the system scored 92% on the evaluation set, and the lesson warned that the figure was flattering. The shadow week lets you check. Those thirty messages come with a person's answer, so they can be run as an evaluation set, through the model alone and without the guard.

python evaluate.py shadow_week.csv
30 cases, 22 right (73%)
queue      cases right
emergency      4     2
billing       10     9
meter          6     5
moving         5     3
complaint      5     3
Wrong:
  S09 meter -> billing (0.44): I need a new key for my prepayment meter.
  S14 moving -> billing (0.2): We bought a house. How do we take over the supply?
  S15 moving -> meter (0.62): Final meter reading for my old flat is 9912. I moved out on Sunday.
  S20 complaint -> billing (0.44): I am fed up with being sent bills for a flat I left in January.
  S21 complaint -> meter (0.8): Your engineer damaged my meter cupboard and nobody will take responsibility.
  S22 billing -> meter (0.62): I have been charged for a meter reading that is not mine.
  S24 emergency -> billing (0.2): There is smoke coming from the boiler.
  S25 emergency -> meter (0.8): My meter is hot and there is a burning smell.
emergency cases missed: 2 (must be 0)
pass mark 85%: not met
RESULT: FAIL

On messages it had never been adjusted against, the model alone is right 73% of the time, and it misses two of four emergencies.

Both numbers matter for the handover. The first says that the 92% belonged to the cases you had studied. The second says that the guard is carrying the hard rule in live use, and the model is not. The customer should hear both from you in plain words. The system as deployed is safe because of the rules around the model, and the triage desk is still needed for about a quarter of the messages.

Every mistake becomes a case

The shadow week also gave you eight cases the model gets wrong. Do not just fix them. Add them to the evaluation set first, with the answer the person chose, and then improve the system until the set passes again. The exercise does this for three of them.

Keep doing it for as long as the system runs. When the triage desk corrects a routing, or a customer complains about a slow answer, that message becomes a case. A set that grows this way comes to describe what this customer's messages are really like, and every future version has to pass it.

What to watch each week

Agree a short list of numbers with the head of customer service, and who looks at them.

  • The share of messages left for a person. If it climbs, the messages have changed or the model has.
  • The number of messages the guard sends to the emergency queue, and how many of those the emergency team says were false alarms.
  • A sample check. Each week somebody on the triage desk reads fifty messages that were routed automatically and marks the ones they would have sent elsewhere. This is the only way to see confident mistakes like S21, because nothing else flags them.
  • Messages moved from one queue to another by hand. Each one is a mistake somebody had to repair.

The sample check is the one that gets dropped when people are busy. Put it in somebody's diary by name.

Changing it safely

The system will change after you leave. The provider will retire the model and offer a newer one. Somebody will want to add a queue. The emergency team will ask for a new guard word.

Each of these is a change like the ones in lesson 4, and the same routine applies.

  • Make one change.
  • Run the whole evaluation set. Both rules must pass.
  • Compare with the last version and read the broken list.
  • For a large change, such as a different model, run a shadow week before switching.
  • Keep the results file, with a line saying what changed and who agreed.

A newer model is a large change, even when the provider describes it as an improvement. It may be better on average and worse on this customer's messages, and only the evaluation set can tell you.

The runbook

Write one page for the person on duty at Tessel who did not build the system.

# Runbook: Tessel message sorting

## What it does
Reads each written message and sends it to a queue. Messages with
a danger word always go to emergency. Messages the model is unsure
about go to the triage desk.

## Switch it off
Set AUTO_ROUTING to off. Every message except emergencies then goes
to the triage desk, as before the system existed. Anyone on the
triage desk may do this. Tell the head of customer service after.

## Signs of trouble
- The triage desk queue is much longer than usual.
- The emergency team reports many false alarms.
- A message about danger is found in another queue.
  Treat the last one as urgent: switch off, then call the owner.

## Every week
- Read 50 automatically routed messages. Count the wrong ones.
- Add each wrong one to the evaluation set with the right queue.

## Before any change
Run the evaluation set. Both rules must pass. Compare with the
last version. Nothing ships with a missed emergency.

## Who owns what
Head of customer service: the pass mark and the threshold.
Emergency team lead: the guard words.
Triage desk lead: the evaluation set and the weekly check.
Security lead: which model may be used.

What the customer owns when you leave

Hand over four things, and walk the customer through each one in use. The evaluation set, with its rules, and somebody who knows how to add a case. The scripts that run and compare it. The switch, tried once in front of the people who may use it. And the runbook.

Look back at what made this a deployment. The model was the smallest part, and in this course it was a stand-in. The rest was what the customer can rely on: an agreement about which mistake must never happen, a set of real cases to measure against, a comparison after every change, a guard that works without the model, a person behind every unsure answer, and a way to stop. Those are yours to build as an FDE, whichever model sits in the middle.

Key takeaways

  • Live messages are the honest test; expect a lower score than on the cases you tuned against.
  • Every mistake found in live use becomes a new case in the evaluation set.
  • Watch a few numbers every week, and have a person re-check a sample of automatic decisions.
  • Any change to the model, its instructions, the guard or the threshold goes through the evaluation first.
  • The customer owns the evaluation set, the switch and the runbook when you leave.

Try it

Add three shadow-week messages to messages.csv as new cases: S21 as M25, S24 as M26 and S25 as M27, each with the queue the person chose. Run python evaluate.py messages.csv v5 and note the result. Then add "smoke" and "burning" to the emergency list in classifier.py, run it again as v6, and run python compare.py v5 v6. A good result shows a FAIL at 22 of 27 with two emergencies missed before the change, a PASS at 24 of 27 with none missed after it, and a comparison that says M26 and M27 are fixed, nothing is broken, and M15, M19 and M25 are still wrong.

Official documentation

Ready to use this in a job?

Rolebound applies to matching roles for you.

Start now

Written with AI and checked automatically. Nobody has run these steps by hand, and tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.

    Cookies on Rolebound

    We use necessary cookies to sign you in and keep the site working — those cannot be turned off. We would also like to set analytics and advertising cookies so we can see which pages help people get interviews and which adverts brought them here. Nothing optional is set unless you say yes, and you can change your mind at any time from “Cookie settings” in the footer. Read our Privacy Policy.