详细内容或原文请订阅后点击阅览
阈值是价格，而不是百分比
如何通过使用成本不对称而不是固定的置信截止来决定人工智能代理何时应自行行动阈值是价格，而不是百分比的帖子首先出现在走向数据科学上。
来源:走向数据科学have a line like ESCALATION_THRESHOLD = 0.90. Above it, the agent acts. Below it, a human gets pinged.
It’s simple. It’s tunable. It lets everyone feel like autonomy is under control.
But it’s not the right solution.
The fix isn’t a better number. The fix is noticing that the escalation threshold was never a percentage in the first place. It’s a price.
Start with the wrong question
Most teams ask a capability question. Can the agent write the SQL? Can it issue the refund? If yes, let it run.
But being able to do something and deciding to do it alone are different issues. The second is really about the decision itself: What does a mistake cost, and what does it cost to involve a human instead?
That’s the real choice. If the agent acts, you risk the cost of a mistake. If you escalate, you pay for a human’s time, whether the agent was right or not.
Put numbers on it and the comparison is almost embarrassingly simple. One assumption first: the human gets escalated tickets right. We’ll relax that later. For now, act alone when:
(1 - p) * cost_of_error < cost_of_escalation
where p is the probability the agent is correct. Rearranged, escalate whenever:
p < 1 - (cost_of_escalation / cost_of_error)
That value on the right is the real threshold. Notice what it depends on. Not the model. Not a policy from a workshop. Just a ratio between two costs. If errors are cheap, the threshold is lower and the agent can act more often. If errors are expensive, the threshold is higher, and even a confident agent should ask for help.
A fixed cutoff assumes that ratio is the same for every decision the agent will ever face. That’s rare.
This approach is based on Chow’s rejection rule from 1970, which is the foundation of today’s learning-to-defer research. Old ideas age well when they’re right.
Two tickets
The threshold comes out at 1 minus 4/15. About 0.73.
That observation usually lands in conversation.
The catch
The rule in code
Where the numbers get messy
Two factors make the inputs more complicated.