pipette
ENEnglish

Is spectral predictability enough? Testing {\Omega}-matching for synthetic pretraining under distribution shift

Edric Castel C. Hao, Juan Raphael S. Se\~na, Neon S. De Jesus

Preprint

En palabras de los autores

Synthetic pretraining under distribution shift requires deciding what synthetic data to use. A natural criterion is to match the spectral predictability of the source to the target, but is sufficient? We test this hypothesis, which we call predictability-aware pretraining, across three architectures (LSTM, PatchTST, TCN) and three seeds. Under ideal synthetic-to-synthetic conditions, -matched pretraining can outperform even an oracle baseline (+46.8% MSE reduction, LSTM), though the benefit is already architecture-dependent. On the real-world ETTm1 benchmark, however, this relationship does not hold: sine-plus-noise pretraining at matched yields only a marginal mean improvement +2.3% averaged over all three architectures, up to +65.5% on individual channels), whereas matched chaotic ODE trajectories systematically degrade performance -13.2%). The two sources agree on the direction of transfer for only 53% of configurations across all three architectures (Pearson r=-0.02), indicating that carries little information about which source will transfer beneficially. We further observe an architecture-dependent asymmetry in the response to the direction of the gap. We conclude that matching can help in controlled settings but is not sufficient to predict transfer across sources, and we discuss additional signal properties that a source-selection criterion may need to consider.

Resultado principalLimitación que admiten los autores

Apareció: viernes, 23 de octubre. arXiv. Preprint, todavía sin revisión por pares.

Comentario de los autores: Submitted to ICASSP 2027