Simulating Learners with GPT-4 for Parsons Problems

Parsons Problems ask learners to arrange code fragments into a correct program. This project examined whether GPT-4 could be used to simulate learner interactions with these exercises and, in turn, help surface plausible strategies, errors, and patterns of reasoning.

The work used the model not simply as an answer generator, but as a potential research instrument for thinking through how learners might approach a structured programming task. That framing made it possible to explore both the promise of generative AI for educational research and the limits of treating simulated behavior as evidence about real learners.

It strengthened an interest in careful evaluation: understanding what an intelligent system can represent well, where its behavior diverges from people, and how it can be used responsibly in a research context.

This work was carried out under the guidance of Carl C. Haynes-Magyar, then a Presidential Postdoctoral Fellow in Carnegie Mellon’s Human-Computer Interaction Institute.