Domain Bounds as a Silent-Fault Detector for AI-Ready Scientific Data
In the authors' words
We address an incipient type of data-based faults, driven by increasing amounts of data reuse in AI-enhanced computational science workflows. A dataset is created under specific conditions; its categories, measures, and labels depend on that context, but that context is usually unavailable to downstream users and is completely abandoned when datasets are used to train models. We propose the encoding and association of this context, called domain bounds, with the data to which it applies. By specifying the conditions for valid reuse, a domain bound flags apparently valid data from being reused in an incompatible context, even when every value falls within its expected range. We present cases from computational science and biomedicine where missing domain bounds allow silent reuse errors, and show why description and provenance approaches do not detect them. Our detector uses standard data container technologies, catches faults drawn from a taxonomy of published cases, irrespective of dataset size. Against injected faults it caught 17 of 18 domain-bound reuses that lineage and descriptive records accepted.
Appeared: Monday, September 28. arXiv. Preprint, not yet peer-reviewed.