pipette
ENEnglish

Neutral Is Not Free: Evaluating Downside Risk in Neutral Launches

Pablo Alcain, Jason Kang, Mia Garrard, Marine Veits, Houssam Nassif, and Abbas Zaidi

PreprintAfirmaciones fuertes, leer con cuidadoUso en el mundo real

En palabras de los autores

Evaluating "neutral launches" (e.g., infrastructure upgrades) using traditional confidence interval overlap is flawed: it is dangerously permissive with scarce data and excessively restrictive with abundant data. To resolve this, this paper introduces Expected Bayesian Loss (EBL), a continuous metric that quantifies both the probability and expected severity of metric degradation. Computable directly from standard frequentist estimates, EBL explicitly penalizes empirical noise and high-variance experiments. Validated against expert decisions, EBL provides experimentation platforms with a rigorous, tunable guardrail that aligns statistical safety with institutional risk appetite.

Resultado principalEl resumen no menciona limitaciones.

Apareció: jueves, 8 de octubre. arXiv. Preprint, todavía sin revisión por pares.