AI hallucination of Chinese nuclear components almost led to US military attack
Ars Technica
AI hallucination of Chinese nuclear components almost led to US military attack
But the military's overall use of AI seems to be accelerating.
You know the thing where AI companies describe their alignment direction as “Honest, Harmless, Helpful”?
The only one that’s marketable to the idiots buying AI services is Helpful. So that’s always top priority even when it goes against the other two.
Average decider wouldn’t stand an “assistant” that sometimes answers “I don’t know”. So instead agent providers train their already hallucination-prone model to be 100% confident in whatever it says. Who cares if it’s true if it looks like it is?