Can LLMs Simulate Novice Programmers' Misconceptions?
En palabras de los autores
We evaluate 13 LLMs on generating, solving, simulating, and diagnosing novice programming misconceptions on code-tracing problems. While frontier models reliably perform these tasks, small models (14B) struggle with dynamic execution. Notably, code-tuned models fail during misconception simulation by reverting to correct execution. Finally, misconception diagnosis is significantly easier in True/False formats than in open-ended generation tasks.
Resultado principalEl resumen no menciona limitaciones.
Apareció: viernes, 9 de octubre. arXiv. Preprint, todavía sin revisión por pares.