[Paper Review] Did AI Teach the Student—or Solve the Problem for Them?

Generative AI without guardrails can harm learning: Evidence from high school mathematics

Authors: Hamsa Bastani, Osbert Bastani, Alp Sungu, Haosen Ge, Özge Kabakcı, and Rei Mariman

Publication Information: Proceedings of the National Academy of Sciences, 122(26), e2422633122, 2025.

Solving a problem is not the same as learning how to solve it

Generative AI is rapidly becoming part of everyday education. Students can ask for explanations, receive immediate feedback, and generate complete solutions to difficult problems within seconds. These capabilities make AI appear to be an always-available personal tutor.

However, helping a student complete a task and helping that student acquire the skills needed to complete the task independently are not the same thing. When AI carries out too much of the reasoning process, students may produce correct answers without developing a meaningful understanding of the underlying concepts.

Bastani and colleagues examined this distinction through a large-scale randomized field experiment. Rather than asking only whether generative AI improves student performance, the researchers investigated how different designs of AI assistance influence actual learning.

A field experiment involving nearly 1,000 students

The study was conducted at a large high school in Türkiye and involved nearly 1,000 students from Grades 9 through 11. Approximately 50 classes participated in four 90-minute mathematics sessions. Classes were randomly assigned to one of three experimental conditions.

Students in the first condition used GPT Base, a GPT-4 interface designed to resemble a conventional chatbot. They could freely ask questions, and the system could provide complete solutions and final answers.

Students in the second condition used GPT Tutor, a safeguarded version of the system. It was instructed not to reveal answers directly but to offer step-by-step hints and questions. Its prompts also included teacher-provided solutions, common student mistakes, and recommended feedback.

Students in the third condition formed the control group. They solved the practice problems using ordinary course materials without access to generative AI.

Each session included assisted practice followed by an unaided exam. During practice, students used the resources assigned to their experimental condition. They then completed a conceptually similar test without AI, textbooks, notes, or other resources. This design allowed the researchers to distinguish immediate task performance from learning that remained after the technology was removed.

Performance improved with AI, but learning did not necessarily improve

During the assisted practice period, both AI systems produced substantial performance gains. Compared with the control group, GPT Base increased practice performance by approximately 48%, while GPT Tutor increased it by approximately 127%.

The results changed dramatically when students had to work independently. Students who had used GPT Base scored 17% lower than the control group on the unaided exam. Although the unrestricted chatbot helped them complete the practice problems, it appears to have reduced the learning needed to solve similar problems on their own.

Students who used GPT Tutor performed at approximately the same level as the control group on the unaided exam. The safeguards therefore largely removed the negative learning effect found in the GPT Base condition. However, GPT Tutor did not produce a statistically significant learning advantage over the control group.

This distinction is important. The study does not show that the guided tutor significantly improved independent achievement. Rather, it shows that carefully designed guardrails can prevent much of the harm associated with unrestricted AI assistance.

AI became a crutch rather than a learning partner

The researchers also analyzed the conversations between students and the two AI systems. Students using GPT Base frequently asked for the answer and relied on complete AI-generated solutions. In this condition, AI often functioned as a cognitive crutch rather than as a tool for reflection and learning.

GPT Tutor encouraged a different interaction pattern. Instead of immediately revealing the solution, it prompted students to identify what they knew, consider a possible strategy, and attempt the next step. This design preserved more of the intellectual work that students needed to perform themselves.

The unrestricted system was also far from perfectly reliable. When the researchers repeatedly asked GPT Base for answers to the 57 practice problems, it produced a correct answer only about half of the time on average. Students nevertheless appeared to have difficulty recognizing or adequately checking these errors.

The finding highlights a central paradox of AI-supported learning. Students require sufficient subject knowledge to evaluate AI output, yet excessive dependence on AI may prevent them from developing precisely that knowledge.

Educational value depends on design

The study’s most important contribution is its demonstration that educational outcomes depend not simply on whether AI is available, but on how the technology is designed to interact with learners.

An AI system that immediately supplies final answers can improve short-term task completion while removing essential opportunities for reasoning, error correction, and productive struggle. High scores achieved with such assistance may represent the performance of the tool more than the learning of the student.

A well-designed educational AI system should therefore be evaluated by how effectively it preserves and stimulates student thinking. It should diagnose misconceptions, ask probing questions, provide limited hints, and gradually transfer responsibility back to the learner.

The regression coefficients indicate that practice performance was 0.137 and 0.361 points higher for the GPT Base and GPT Tutor groups, respectively. Relative to the control-group mean of 0.284, these estimates correspond to improvements of approximately 48% and 127%. On the unaided exam, the coefficient for GPT Base was −0.054, indicating performance approximately 17% lower than the control-group mean of 0.321. The coefficient for GPT Tutor, −0.004, was not statistically significant.

GPT Base provides a complete solution in response to the student’s request for the answer. In contrast, GPT Tutor does not provide the answer directly; instead, it prompts the student to first explain which transformations should be applied and where they are currently stuck.

Implications for schools and teachers

The findings suggest that the educational debate should move beyond the simple choice between allowing and banning generative AI. The more important issue is how students are taught to use it.

Students should be encouraged to request explanations, feedback, and hints rather than complete answers. A prompt such as “Solve this problem for me” could be replaced with “Do not give me the answer; identify the first error in my approach and give me one hint.”

Educational AI systems should also include mechanisms that require an initial student attempt, delay the disclosure of final answers, detect possible misconceptions, and use teacher-validated problem information. AI literacy should involve not only prompt-writing skills but also the ability to verify outputs and monitor one’s own reasoning.

Most importantly, AI should support rather than replace professional educational judgment. The authors identify AI systems that assist human teachers or tutors as a promising direction for future educational applications.

Limitations

Although the study provides strong field evidence, its findings should not be generalized without caution. It was conducted in one high school in Türkiye, focused only on mathematics, and measured relatively short-term outcomes.

Different results may emerge in subjects such as writing, history, or discussion-based learning, where answers are less objectively defined. The experiment was also conducted in the fall of 2023, when students had less experience with generative AI and available models were less capable than newer systems.

The researchers therefore call for further studies across countries, subjects, age groups, tutor designs, and longer periods. GPT Tutor also remained largely reactive; unlike an expert human tutor, it did not proactively investigate misconceptions and adapt instruction through sustained dialogue.

Conclusion

This paper demonstrates why the impact of generative AI on education cannot be judged only through satisfaction surveys or scores obtained while AI is available. Better assisted performance does not necessarily indicate deeper or more durable learning.

The educational value of generative AI lies not in how effectively it solves problems for students, but in how effectively it helps students develop the capacity to solve problems independently. Schools therefore need AI systems that preserve productive struggle, ask meaningful questions, and leave students with genuine responsibility for thinking.

The central question for AI education is no longer simply, “Should students use AI?” It is, “What kind of AI should students use, and under what pedagogical principles should it be designed?”