Safety
Building AI that knows when to defer
Why recognising uncertainty is critical in real-world operations.
One of the easiest ways for an AI system to look capable is to always have an answer.
In real operations, that can be dangerous.
Logistics is full of incomplete information, unusual situations and conflicting signals. A model may be asked to explain a sudden drop in performance without having the staffing data that would make the cause obvious. It may be asked to recommend an action when several explanations are equally plausible.
In those moments, confidence is not intelligence. Knowing when the evidence is not strong enough matters just as much.
Uncertainty is part of the operation
Most logistics decisions are made with imperfect information. Data arrives late, systems disagree and the person asking the question may not know that something important is missing.
A useful AI system should be able to recognise that.
Instead of forcing a conclusion, it should be able to say that the evidence is incomplete, identify what is missing and explain what would increase confidence.
That sounds simple, but it changes the role of the model. It stops being a system that is rewarded for always producing an answer and becomes one that is expected to make better judgements about when an answer is justified.
The cost of false confidence
Imagine a model identifies driver coaching as the likely fix for a recurring service issue. If the real problem is vehicle availability, management may spend a week addressing the wrong thing.
The recommendation may have sounded reasonable. The failure was in the confidence behind it.
As AI systems move closer to real operating decisions, that distinction becomes more important. A system should not treat a weak inference and a well-supported conclusion as though they are the same.
Defer does not mean fail
We think deferral should be treated as a capability, not a weakness.
A strong system should be able to recognise when a person has better context, when the consequences of being wrong are too high or when a decision sits outside its authority.
Sometimes the correct output is a recommendation. Sometimes it is a request for more information. Sometimes it is simply: this decision needs a person.
The important thing is knowing the difference.
Measuring good uncertainty
This is something we want to evaluate directly.
It is not enough to measure whether a model eventually produced the right answer. We also want to understand whether its confidence matched the quality of the evidence, whether it asked for the right missing information and whether it deferred appropriately when the situation was ambiguous.
For AI in real-world operations, reliability is not just about being right more often.
It is also about knowing when not to pretend.