Planted Errors Cannot Measure What a Clinician Misses
En palabras de los autores
A safety certificate for a clinical AI promises that no more than a stated fraction of the answers it releases carry a defect that would change care. That promise is computed from past cases a clinician marked safe, and clinicians miss things. A missed defect is not filed as a disagreement, it is filed as a safe case, so the certificate rests on a record cleaner than the ward ever was and has to be widened to compensate. Widening it correctly takes one number: how often that clinician misses a defect the model produced. We show that number cannot come from the only test a hospital can run on a working reviewer, planting a known defect and counting the catches. A planted defect is as hard to spot as whoever planted it made it, a real defect is as hard to spot as the model happened to make it, and comparing the two needs the very number being sought. No safety factor recovers it, because one wide enough for every miss rate releases nothing. The failure also runs the dangerous way, because corrections that turn noise into one rate need the mistakes to be unrelated to the case, whereas a defect is missed because it read well, so the risk on record sits below the true risk at every threshold. Across four simulated reviewers that condition never holds, and testing it on a separately drawn sample is affordable only where the answer was never in doubt: a reviewer missing 85% of defects costs 185 adjudicated cases against 147,938 for one missing 3%. Adjudicating the reviewer's own cases instead removes that test and most of that cost, 498 cases for the careful reviewer, so modelling the noise costs more than measuring it. Our library therefore refuses the widened certificate unless the condition is confirmed on adjudicated data, and fails closed otherwise.
En palabras simples Resumen con IA
Este certificado se basa en casos que un clínico marcó como seguros, pero los clínicos a veces pasan por alto defectos reales. Para corregir esto, el certificado debe ampliarse usando la tasa a la que los clínicos pasan por alto defectos que la IA comete. Los autores demuestran que esta tasa de omisión no puede encontrarse plantando defectos conocidos y contando las capturas.
Escrito por Kimi K2.5 usando solo el abstract de arriba. Jev verificó que cada oración esté respaldada por las oraciones del abstract que cita y eliminó las que no. Pasá el cursor por una oración para ver de dónde sale. Se eliminaron 2 oraciones por no estar respaldadas por el abstract.
Apareció: sábado, 3 de octubre. medRxiv. Preprint, todavía sin revisión por pares.