pipette
ESEspañol

Deep learning perturbation models can outperform baselines on calibrated metrics

Henry E. Miller, Gabriel Mejia, Francis Leblanc, Brendan Swain, Bo Wang, Lucas Paulo de Lima Camillo

Peer-reviewed journal

In the authors' words

Abstract Recent benchmarks report that deep-learning-based genetic perturbation models fail to outperform uninformative baselines. Here we introduce a positive control baseline and metric calibration framework, showing—across 14 datasets and 18 metrics—that common benchmarking metrics (for example, mean squared error and Pearson Δ ) are frequently miscalibrated, with reduced sensitivity to measure positive model performance. Under well-calibrated metrics, we find that deep-learning-based genetic perturbation models can outperform uninformative baselines.

Main resultThe abstract does not state a limitation.

Appeared: Saturday, October 3. Nature Biotechnology. Peer-reviewed journal.

DOI: 10.1038/s41587-026-03307-w