pipette
ESEspañol

Case-Specific Priors-Guided Multimodal Fusion for Prognosis Prediction in Head and Neck Cancer

Yue Lin, Shuchang Ye, Shenghai Wu, Dashun Zheng, Mingyuan Meng

Preprint with a published versionBold claims, read criticallyReal-world useCode available

In the authors' words

Accurate prognosis prediction for head and neck cancer typically necessitates integrating highly heterogeneous clinical and pathological evidence by sophisticated multimodal fusion. However, existing fusion techniques are usually grounded in case-generic clinical rationales about complementary synergies across modalities, underestimating case-specific variance. In this study, we propose a prognosis prediction framework that explicitly incorporates Case-specific Priors to guide Multimodal fusion (named CPM). It leverages broad medical knowledge of LLMs to characterize each case's (case-specific) intra-modal contexts and inter-modal relations, as well as case-generic multimodal rationale, as priors, enabling three-stage priors-guided multimodal fusion: (i) Intra-modal Context-guided Calibration (IntraCC) that leverages case-specific intra-modal priors to guide feature calibration within each modality; (ii) Inter-modal Relation-guided Interaction (InterRI) that leverages case-specific inter-modal priors to guide pair-wise feature interaction in modality pairs; and (iii) Multimodal Rationale-guided Aggregation (MultiRA) that leverages case-generic multimodal priors to guide final feature aggregation over all modalities. On the well-established HANCOCK challenge dataset involving four distinct medical modalities, our CPM achieved a mean AUC of 83.98% for predicting 5-year overall survival and 2-year recurrence-free survival, improving upon the baseline by 7.58%, pioneeringly demonstrating the effectiveness of LLM-derived medical priors in guiding multimodal fusion for prognosis prediction. The code is available at https://github.com/yuelin421/CPM.

Main resultThe abstract does not state a limitation.

Appeared: Monday, October 19. arXiv. Preprint with a published version.

DOI: 10.1007/978-3-032-38172-9_11

Published version: Medical Image Computing and Computer Assisted Intervention -- MICCAI 2026, Lecture Notes in Computer Science, vol. 16887, pp. 108-118, Springer (2027)

Authors' comment: Accepted at MICCAI 2026