UniBuc at SemEval-2024 Task 2: Tailored Prompting with Solar for Clinical NLI
In the authors' words
This paper describes the approach of the UniBuc team in tackling the SemEval 2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trials. We used SOLAR Instruct, without any fine-tuning, while focusing on input manipulation and tailored prompting. By customizing prompts for individual CTR sections, in both zero-shot and few-shots settings, we managed to achieve a consistency score of 0.72, ranking 14th in the leaderboard. Our thorough error analysis revealed that our model has a tendency to take shortcuts and rely on simple heuristics, especially when dealing with semantic-preserving changes.
Main resultLimitation the authors admit
Appeared: Friday, October 2. arXiv. Preprint, not yet peer-reviewed.