Reliability Theory for AI Control
In the authors' words
Reliability theory gives a mature language for layered systems, but its formal tools are not yet standard in frontier AI control. We apply them to Google DeepMind's defenses against rogue deployment. The same control stack can have cubic, quadratic, or linear rare-failure suppression depending on its failure domains. Birnbaum importance identifies which component improvements buy the most nominal reliability, while prevention changes the population on which recovery is demanded. These results give concrete guidance about what to separate, improve, measure, and test.
Main resultThe abstract does not state a limitation.
Appeared: Wednesday, September 23. arXiv. Preprint, not yet peer-reviewed.
Authors' comment: 14 pages