Lesson 5 of 6
Roll Out in Steps: Shadow Mode, a Guard, a Threshold and an Off Switch
Put the model into live use gradually: run it beside people first, add a rule that catches emergencies without the model, let it act only when it is sure, and give the customer a switch to stop it.
9 minForward Deployed Engineering
After this lesson you will be able to plan a rollout in steps, build the rules that surround a model in live use, and use a week of real messages to choose how much the system may do alone.
You need classifier.py as it stood at the end of lesson 4, the version saved as v3.
Roll out in steps
Passing the evaluation does not mean switching everything on. Each step below is small enough to stop.
- Shadow mode. The system reads every live message and records its decision. Nothing is done with it. The triage desk carries on as before, and you compare the two.
- Suggestions. The triage desk sees the system's choice beside each message and accepts or corrects it.
- Automatic for confident decisions. Messages the system is sure about go straight to a queue. The rest go to the triage desk.
- Wider automatic. The threshold is lowered only when the numbers support it.
At every step somebody at the customer agrees to move on, having seen figures from the step before.
The rules around the model
Create route.py. It decides where a message really goes, and the model is only one part of that decision.
"""Decide where a message goes. The model is one part of the decision, and not the first part.
Usage: python route.py "the text of a message"
Set AUTO_ROUTING=off to send everything except emergencies to a person.
"""
import os
import sys
from classifier import classify
GUARD_WORDS = ["smell", "leak", "carbon monoxide", "sparks", "smoke", "fire", "burning"]
THRESHOLD = 0.8
def route(text, auto=True, threshold=THRESHOLD):
"""Return (queue, decided_by). The queue "triage" means a person reads the message and decides."""
lowered = text.lower()
if any(word in lowered for word in GUARD_WORDS):
return "emergency", "guard"
queue, confidence = classify(text)
if queue == "emergency":
return "emergency", "model"
if not auto or confidence < threshold:
return "triage", "person"
return queue, "model"
if __name__ == "__main__":
auto = os.environ.get("AUTO_ROUTING", "on") != "off"
queue, decided_by = route(sys.argv[1], auto=auto)
print(f"{queue} (decided by: {decided_by})")Read route from the top, because the order is the design.
The guard comes first. If the message contains one of a short list of words, it goes to the emergency queue and the model is not asked. The words were agreed with the leader of the emergency team. This is the protection that lesson 3 said was needed: it works when the model is wrong and when the model is sure and wrong, because it does not use the model.
Next, anything the model itself calls an emergency goes to the emergency queue, whatever the confidence.
Then the threshold. If automatic routing is switched off, or the confidence is below the threshold, the message goes to triage, where a person reads it as today.
Only a confident answer about an ordinary queue is acted on automatically.
Try three messages:
python route.py "My meter reading today is 20177."
python route.py "I need a new key for my prepayment meter."
python route.py "There is smoke coming from the boiler."meter (decided by: model)
triage (decided by: person)
emergency (decided by: guard)A week in shadow mode
During the shadow week the triage desk sorted messages as usual. Create shadow_week.csv with thirty of them, each with the queue a person chose. The system has never been adjusted against any of these.
id,queue,text
S01,billing,My direct debit went up this month and nobody told me.
S02,billing,I think my bill is wrong. Can someone check it?
S03,billing,I paid by card yesterday but my account still shows the old balance.
S04,billing,Please refund the credit on my account.
S05,billing,How do I set up a direct debit for my bill?
S06,billing,Your bill says I used more than my meter shows.
S07,meter,My meter reading today is 20177.
S08,meter,The smart meter has stopped sending readings.
S09,meter,I need a new key for my prepayment meter.
S10,meter,Can I send a photo of my reading?
S11,meter,The engineer did not turn up to change my meter.
S12,moving,I am moving out at the end of the month.
S13,moving,I have just moved here and I do not know who supplies this flat.
S14,moving,We bought a house. How do we take over the supply?
S15,moving,Final meter reading for my old flat is 9912. I moved out on Sunday.
S16,moving,My new address is 12 Kiln Road.
S17,complaint,I want to make a formal complaint.
S18,complaint,Nobody has called me back in two weeks. This is unacceptable.
S19,complaint,The man on the phone was rude to my mother.
S20,complaint,I am fed up with being sent bills for a flat I left in January.
S21,complaint,Your engineer damaged my meter cupboard and nobody will take responsibility.
S22,billing,I have been charged for a meter reading that is not mine.
S23,emergency,I can smell gas in the hallway.
S24,emergency,There is smoke coming from the boiler.
S25,emergency,My meter is hot and there is a burning smell.
S26,emergency,I think there is a gas leak outside number 9.
S27,billing,Why is my payment higher when I use less?
S28,meter,The display on my meter shows an error code.
S29,billing,I have asked three times for a paper bill and I am still waiting.
S30,billing,After the fire in our block we were moved to a hotel. Please pause my direct debit.Create shadow.py. It runs every message through route at three thresholds and compares the result with the person's choice.
"""Compare the routing with what people decided during a shadow week.
Usage: python shadow.py shadow_week.csv
"""
import csv
import sys
from route import route
def main(path):
with open(path, newline="", encoding="utf-8") as handle:
messages = list(csv.DictReader(handle))
print(f"{len(messages)} messages from the shadow week")
print(f"{'threshold':<11}{'automatic':>9}{'same as the person':>21}{'left for a person':>19}")
for threshold in (0.4, 0.6, 0.8):
automatic = same = 0
for message in messages:
queue, decided_by = route(message["text"], threshold=threshold)
if decided_by != "person":
automatic += 1
same += queue == message["queue"]
agreed = f"{same} ({same / automatic:.0%})"
print(f"{threshold:<11}{automatic:>9}{agreed:>21}{len(messages) - automatic:>19}")
emergencies = [message for message in messages if message["queue"] == "emergency"]
caught = sum(1 for message in emergencies if route(message["text"])[0] == "emergency")
print(f"emergencies: {len(emergencies)}, sent to emergency: {caught}")
print("Automatic and different from the person, at 0.8:")
for message in messages:
queue, decided_by = route(message["text"])
if decided_by != "person" and queue != message["queue"]:
print(f" {message['id']} person: {message['queue']}, {decided_by}: {queue}: {message['text']}")
if __name__ == "__main__":
main(sys.argv[1])python shadow.py shadow_week.csv30 messages from the shadow week
threshold automatic same as the person left for a person
0.4 28 22 (79%) 2
0.6 24 20 (83%) 6
0.8 22 20 (91%) 8
emergencies: 4, sent to emergency: 4
Automatic and different from the person, at 0.8:
S21 person: complaint, model: meter: Your engineer damaged my meter cupboard and nobody will take responsibility.
S30 person: billing, guard: emergency: After the fire in our block we were moved to a hotel. Please pause my direct debit.Choose the threshold with the customer
Each row of the table is one possible setting, and the choice between them belongs to the customer.
At 0.4 the system would sort 28 of 30 messages alone, and 22 of those would match the person: about four in five. At 0.8 it would sort 22 alone and match on 20, which is 91%, leaving 8 for the triage desk.
Show the head of customer service this table and say it in their terms. At 0.8 the triage desk reads about a quarter of the messages and not all of them, and about one automatic decision in ten differs from what a person would have chosen. Tessel chose 0.8 to start with.
All four emergencies reached the emergency queue. Two of them, the smoke and the burning smell, are messages the model alone would have sent elsewhere. The guard caught both.
The last lines show the two automatic decisions that differ. S21 is a complaint about an engineer, sent to the meter team with confidence, because it mentions a meter cupboard. S30 is a false alarm: the word "fire" sent a billing question to the emergency team. The emergency team lead accepted that in lesson 1, and it is the price of the guard.
The off switch
One day something will go wrong: a provider's fault, a flood of strange messages, a bad change. The customer needs one step that stops the automatic part, without calling you.
Here it is a setting. With AUTO_ROUTING set to off, every message except an emergency goes to the triage desk, which is exactly how Tessel worked before.
macOS or Linux:
AUTO_ROUTING=off python route.py "My meter reading today is 20177."
Windows PowerShell:
$env:AUTO_ROUTING = "off"
python route.py "My meter reading today is 20177."
Remove-Item Env:AUTO_ROUTINGIt prints triage (decided by: person). Tell the customer where the switch is, who may use it, and that using it is never a mistake.
Test the rules
These rules are promises to the customer, so each has a test. Create test_route.py:
import unittest
from route import route
class RouteTests(unittest.TestCase):
def test_a_guard_word_sends_the_message_to_emergency(self):
self.assertEqual(route("There is smoke coming from the boiler."), ("emergency", "guard"))
def test_the_guard_works_when_the_model_is_confidently_wrong(self):
self.assertEqual(route("My meter is hot and there is a burning smell."), ("emergency", "guard"))
def test_a_confident_answer_is_routed_by_the_model(self):
self.assertEqual(route("My meter reading today is 20177."), ("meter", "model"))
def test_an_unsure_answer_goes_to_a_person(self):
self.assertEqual(route("Your bill says I used more than my meter shows."), ("triage", "person"))
def test_switched_off_everything_goes_to_a_person(self):
self.assertEqual(route("My meter reading today is 20177.", auto=False), ("triage", "person"))
def test_switched_off_emergencies_still_get_through(self):
self.assertEqual(route("I can smell gas in the hallway.", auto=False), ("emergency", "guard"))
if __name__ == "__main__":
unittest.main()python -m unittestThe output ends with a line saying that 6 tests ran, and then OK. The last test is the important one: with the switch off, an emergency still gets through.
Key takeaways
- Run in shadow mode first: the system decides, people still do the work, and you compare the two.
- Protect against the mistake that must never happen with a rule that does not depend on the model.
- Let the system act alone only above a confidence threshold, and send the rest to a person.
- The threshold is the customer's choice between more automation and fewer mistakes; show them the numbers.
- Build an off switch before you need one, and make sure the safety rule still works when it is off.
Try it
S30 was a false alarm caused by the guard word "fire". Remove "fire" from GUARD_WORDS in route.py, then run python shadow.py shadow_week.csv and python -m unittest again. A good result reports 21 automatic decisions at 0.8, of which 20 are the same as the person (95%), with all four emergencies still caught and all six tests passing. Then explain in two sentences why those better numbers are not a good enough reason to remove the word, and who should make that decision. Put the word back afterwards.
Official documentation
Written with AI and checked automatically. Nobody has run these steps by hand, and tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.