Agentic AI versus pediatric emergency physicians for differential diagnosis of acutely ill and injured children
In the authors' words
Background Misdiagnosis and delayed diagnosis of children in pediatric emergency departments lead to complications, and are particularly problematic in settings with limited pediatric expertise. Large language models (LLMs) have shown promise in diagnostic reasoning for adult emergency department patients, but utility for pediatric cases is largely untested. Methods Retrospective study of patients seen in the emergency department of a major urban pediatric academic medical center and subsequently admitted to the operating room or inpatient service. Records from 100 patients were randomly selected, stratified by age. We compared the accuracy of differential diagnoses (up to five, ranked by likelihood) generated by an LLM agent, Dr CaBot (one configuration with literature search, the other without), with those of four attending pediatric emergency medicine physicians (diagnosticians), all masked to outcome. Encounters were presented to diagnosticians stepwise across four touchpoints: triage, the initial physician evaluation (the a priori primary touchpoint), the full emergency department encounter, and the admission. Ground truth diagnosis was determined by two independent, board-certified pediatric emergency physicians (reviewers) based on the entire clinical encounter. The reviewers, masked to source, then assigned each differential a Bond score (0 to 5; 4 or above was high-scoring). Findings The median patient age was 4 years and 41% had a complex chronic condition. Bond score agreement between the two reviewers had a weighted kappa of 0.64. For the primary comparison at the initial physician evaluation touchpoint, the LLM agent and the diagnosticians both achieved high-scoring differentials in 95% of cases (difference 0%, 95%CI: -5.8, 5.8%). No differences were detected at the other touchpoints. Interpretation Dr CaBot and attending pediatric emergency medicine physicians showed similar performance in generating differential diagnoses from clinical documentation, suggesting potential as a diagnostic aid. Whether this performance improves care, particularly outside pediatric specialty centers, requires prospective evaluation.
Appeared: Thursday, November 19. medRxiv. Preprint, not yet peer-reviewed.