Prompt Injection Can Alter Large Language Model Scores and Feedback on Medical Student Essays: A Four-Model Stress Test
En palabras de los autores
Background: Large language models (LLMs) can support health professions educational assessment by scoring narrative text. However, educator-authored scoring instructions and student text enter the same language-processing context. Prompt injection exploits this boundary by embedding LLM-directed instructions ("attacks") within to-be-assessed text. The scope of this problem across different LLM configurations, attack conditions, and attacker objectives remains incompletely characterized. This study explores: How susceptible is LLM scoring to various prompt injection attacks embedded in reflection essays? Methods: This adversarial stress test examined LLM-based scoring of 4 fabricated medical student reflection essays across 8 prompt injection conditions and 4 LLM configurations. We selected 4 essays used in a previous study representing human reference scores 1 to 4 (maximum 6). In October 2026, we used GPT-4.1-mini, GPT-4.1, GPT-5.2, and GPT-6-Luna to score each essay 15 times without modification ("clean") and 5 times for each prompt injection attack. Six "score-targeting attacks" sought the maximum score of 6; 2 "feedback-targeting attacks" sought unauthorized change to narrative feedback, testing whether injections can control the scorer apart from the score itself. The primary outcome was attack success. Results: For clean replications, exact agreement with the human reference standard was 45/60 (75%) for GPT-4.1-mini and >98% for other LLMs. Across all score-targeting attacks and LLMs, 140/480 (29.2%) attained the target score of 6. Analyzed by LLM, 62/120 (51.7%) were successful for GPT-4.1-mini; 51 (42.5%) for GPT-4.1, 25 (20.8%) for GPT-5.2, and 2 (1.7%) for GPT-6-Luna. Analyzed by attack condition, score-targeting success ranged 6% to 56%. Among feedback-targeting attacks, 157/160 achieved success (98.1%), with [≥]95% success for each LLM. Conclusions: Prompt injection is an unresolved threat to LLM-based scoring and feedback. Although score-targeting attacks were often resisted, feedback-targeting attacks were nearly always successful. No LLM was uniformly resistant, and resistance to one attack provided little assurance about another.
Apareció: martes, 17 de noviembre. medRxiv. Preprint, todavía sin revisión por pares.