X2SBench: an open benchmark for evaluating crystal structure determination from powder diffraction
In the authors' words
Progress towards practical powder X-ray diffraction (PXRD) structure determination requires evaluation beyond small crystals and idealized patterns. X2SBench combines 152,587 simulated structure-pattern pairs with 591 curated measurements, extending evaluation to larger cells, lower-symmetry structures and diverse compositions. Fixed splits, defined inputs and common structural metrics support comparisons by crystal system, atom count and element count. Joint stratification locates weaknesses within these groups. On 558 paired targets, an X2SBench-fine-tuned model improves recovery from simulated patterns but loses accuracy on measured inputs, revealing a gap in experimental transfer. Background subtraction and smoothing improve recovery for one tested checkpoint, while augmentation and reference-lattice comparisons identify further opportunities for measurement adaptation and reliable cell estimation. An open platform provides data access and standardized result submission, establishing a shared basis for community evaluation and progress towards practical PXRD analysis.
Appeared: Friday, September 25. arXiv. Preprint, not yet peer-reviewed.
Authors' comment: 36 pages including 11 pages of supplementary information