Lesson 5 of 6
The System Design Round: Designing for the Customer's Environment
Design a system that runs inside a customer's network: ask about the environment first, draw the flow of data, say how it fails, and plan the handover.
7 minForward Deployed Engineering
After this lesson you will be able to run a system design interview for a system that lives in a customer's environment: ask about the constraints first, draw the flow of data, say how it fails and who finds out, and plan the handover.
How this round differs
A system design round for a product engineer is usually about scale: millions of users, many servers, data spread across regions. An FDE design round is usually about somebody else's environment. The amount of data is often small. The hard parts are that the system runs in a network you do not control, reads from systems you cannot change, and will be looked after by people who did not build it.
So the marks are in different places. You earn them by asking what the environment allows, by choosing the simplest design that fits, by saying what happens when something breaks, and by planning the day you leave. A design with a message queue and three services for two hundred rows a day loses marks, because it shows you design from habit and not from the facts in front of you.
The prompt for this lesson continues the case from lesson 4: "Design the system that checks Calder Freight's shipments against its invoices every night."
Ask about the environment first
Spend the first five to ten minutes on questions. You already know some answers from lesson 4: IT owns both systems, the billing system gives up its data only as an export file, about two hundred shipments go out a day, and customer names and prices must stay on Calder Freight's servers. These questions fill in the rest. The answers shown are the ones the interviewer gives.
- Where can the tool run? On a Linux server that IT provides, inside the network.
- How do the exports arrive? The warehouse system writes one dated file to a shared folder at 01:00, holding the previous day's shipments. The billing system can write the previous day's invoices in the same way at 01:30.
- Who reads the result, and how? The billing team, each morning, by email through the company's own mail server.
- Who looks after it when it breaks? The IT team's on-call engineer, who knows Linux and scheduled jobs and does not know Python.
- What must never happen? The tool must never change anything in either system, and no data may leave the network.
- Does billing work at weekends? No. Friday's shipments are invoiced on Monday.
Two of these answers shape the whole design. The on-call engineer does not know Python, so the system has to be something they can restart and read the logs of without reading code. And invoices are typed the morning after a shipment, so time now matters in a way it did not in the take-home.
Draw the flow of data
Draw while you talk. A flow of data from left to right, or top to bottom, is enough:
warehouse system billing system
| 01:00, one file a night | 01:30, one file a night
v v
/exports/shipments-DATE.csv /exports/invoices-DATE.csv
\ /
+----> reconcile job, 02:00 <---+
reads the last 30 days of both
|
+----------------+----------------+
v v v
findings file email to billing log file
/reports/ with a summary /var/log/reconcile/
job failed, or an export is missing: alert to IT on callThen say what each part is, and why it is no bigger than it is.
The job is the comparison from lesson 2, extended to read a folder of dated files and started by the server's own scheduler. The on-call engineer already knows that scheduler, so there is nothing new for them to learn. There is no database and no web page. At two hundred rows a night, thirty days of files is a few thousand rows, which the job can read from the start on every run.
Reading thirty days on every run solves the problem with time. A shipment sent on Monday appears in the file written early on Tuesday. Billing types its invoice on Tuesday morning, and that invoice appears in the file written early on Wednesday. So the job must not report a shipment as not invoiced until billing has had a working day to type it. It leaves the newest shipments file out of the "not invoiced" check and, because billing does not work at weekends, it counts working days and not calendar days. When a late invoice arrives, the finding disappears from the next report, because the job works everything out again from the files and keeps no memory of its own.
Say how it fails
This is the part candidates skip and interviewers wait for. Go through the failures one at a time and say who finds out.
- An export is missing. The job checks for both of last night's files before anything else. If one is missing it sends billing a message that says "no report today: the shipments file did not arrive", and it alerts IT. It must never send an empty report, because an empty report reads as "no problems".
- A row cannot be read, for example an amount that is not a number. The job lists that row in its own section of the report and carries on with the rest.
- A column is renamed after a system upgrade. The job checks the header of each file first and stops with a message that names the missing column.
- The job itself crashes. The scheduler sees that it ended with an error and alerts IT. The log file says what it was doing.
- The job runs twice. Nothing goes wrong, because it only reads the exports and replaces its own report for that date.
- The mail server's password leaks. It is the only secret the job has. Keep it in a file that only the job's account can read, never in the code.
The first failure is the one to dwell on. Silent failure is the worst kind in a customer's environment, because the customer keeps trusting a report that has stopped telling the truth.
Plan the rollout and the handover
A design is not finished until it says how the system goes live and who owns it afterwards.
For the first week, run the job by hand each morning and sit with the billing team while they read the report. Compare the findings with what they see in the billing system. That is how you find the false alarms, such as a customer with an agreed discount, before the team learns to ignore the report.
Then schedule it and write a one-page runbook for IT: what the job does, where it runs, where the logs are, how to run it again for a given date, what each alert means, and who to call. Agree by name who owns what. IT keeps the job running. Billing acts on the findings. One person in finance looks at the number from lesson 4 each week: the findings still open after one working day.
Say the trade-offs out loud
End by naming the choices you made and when you would choose differently. This shows that the simple design was a decision and not a limit of what you know.
- The server's scheduler and not a workflow tool. With one job, a second tool is one more thing for IT to run. With ten jobs that depend on each other, a workflow tool would be worth it.
- Files and not a database. If billing wanted to mark findings as dealt with, or assign them to people, the findings would need a database and a small web page.
- Email and not a dashboard. Email reaches billing where they already work. If managers wanted trends over months, a dashboard would be the next step.
- Reading thirty days of files on every run and not keeping a record of what was seen. If the volume grew a hundred times, reading everything again each night would be too slow, and the job would keep its own record.
Use the time well
In a forty-five-minute round, spend up to ten minutes on questions, ten on the flow of data, ten on failures, five on rollout and handover, and keep the rest for the interviewer's questions. Keep talking while you draw. When the interviewer changes a fact, such as "now there are fifty warehouses", say which part of your design that breaks first and change only that part.
Key takeaways
- In an FDE design round the hard part is the customer's environment, so ask where the system runs, what it may touch and who looks after it before you draw anything.
- Match the design to the real volume; two hundred rows a night need a scheduled job, not a platform.
- The worst failure is a silent one, so decide what the customer sees when an input is missing.
- Plan the first week of running it, the runbook and the named owners as part of the design.
- State each trade-off and the condition under which you would choose differently.
Try it
Design this on one page: "A hospital group wants a list each morning of the patients most likely to miss today's appointments, taken from its booking system and given to the reception desks. Patient data cannot leave the hospital's network." Write at least six questions you would ask first, with an answer you invent for each. Draw the flow of data as a text diagram. List five ways the system can fail and, for each, who finds out and how. End with a paragraph on the first week of running it and who owns it afterwards. A good result fits on one page, and every part of the design can be traced back to one of your answers.
Official documentation
Written with AI and checked automatically. Nobody has run these steps by hand, and tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.