UNDERSTAND THE PROBLEM
Begin with the failure you need to prevent.
A prompt can tell a model what the team wants. It cannot guarantee what the complete product will do when context grows, tools fail, sources conflict or later releases change behaviour.
The controls that matter need to exist at the layer where they can actually be enforced. Prohibited tools should be unavailable. Invalid arguments should be rejected. Unsupported actions should have no product route. High-impact changes should require approval.
The goal is not to eliminate model uncertainty. It is to stop uncertainty from silently becoming authority.
DESIGN THE SYSTEM
Make the operating rules explicit.
A dependable AI system is easier to build when the team can see the decisions, evidence, boundaries and ownership around it. The following principles turn an ambiguous ambition into components that can be implemented and reviewed.
Put every control at the layer that can enforce it.
Use prompts for guidance, schemas for structure, permissions for authority, product rules for routes and release gates for regression control.
Separate detection from response
A classifier or rule may identify risk, but the product still needs a deterministic decision about what becomes unavailable and where the user goes.
Constrain tools before constraining language
A model that cannot execute an unsafe action is safer than one merely instructed not to execute it.
Test combinations of controls
Layered systems fail at the seams. Evaluate the complete route, including retrieval, tool use, fallback and handover.
IMPLEMENT IN ORDER
Build the smallest complete loop.
Do not automate every adjacent task at once. Start with one valuable journey, carry it from signal to outcome, and preserve enough evidence to know whether it worked. Expand only after that loop is dependable.
- 01
Create a hazard register
List material failure modes, their signals, severity, affected routes and the control responsible for preventing or containing them.
- 02
Assign enforcement layers
Map each control to product logic, data access, model behaviour, tool permissions, runtime validation, human approval or release policy.
- 03
Design the containment route
Specify what stops, what the user sees, which context is retained and who becomes responsible when a control triggers.
- 04
Build adversarial and regression cases
Test indirect language, conflicting instructions, stale knowledge, invalid tools, long context and failures across several turns.
- 05
Monitor control effectiveness
Track triggers, misses, unnecessary blocks, overrides and production incidents without treating a high block rate as success by itself.
KNOW WHEN IT WORKS
Measure behaviour, not how impressive the demo looks.
The useful measure is whether the system creates the intended business or product outcome while staying inside its boundary. Review these checks before launch and whenever the model, data, prompt, tools or workflow changes.
Coverage
Every material hazard has an owner, an enforceable control and a tested fallback.
Independence
No single model response can bypass all controls protecting a consequential action.
Usability
Safe routes remain useful; the system does not replace every uncertain case with an unhelpful refusal.
Regression
Model, prompt, data and tool changes cannot ship without proving critical behaviours still hold.