pipette
ENEnglish

On the Benefit of Blocking for Online Experiments with Skewed Data

Abbas Zaidi, Matthew Reimherr, Rina Friedberg, Fabian Martinez, Richard Mudd

Preprint

En palabras de los autores

This paper advocates for using blocked (stratified) assignment in online A/B tests to handle highly skewed population data. While blocking yields only modest precision gains (5-10%) due to the limits of discretization, the authors demonstrate it offers two crucial structural benefits over post-hoc statistical adjustments. First, fixed-weight blocking correctly targets the true average treatment effect, avoiding the severe bias introduced by efficiency-weighted alternatives. Second, blocking localizes extreme outliers, enabling targeted within-block winsorization that effectively controls noise without destroying the tail-end treatment effect.

Resultado principalLimitación que admiten los autores

Apareció: jueves, 8 de octubre. arXiv. Preprint, todavía sin revisión por pares.