Is AI making us smarter or just making us look smart?

Here’s an uncomfortable truth: Students can ace every assignment in class and learn virtually nothing. Conversely, they can struggle through tasks and learn quite a lot. This paradox, documented by UCLA researchers Robert Bjork and Nicholas Soderstrom, reveals something critical about learning—and it becomes especially important when we talk about AI.

Their insight is simple but profound. Looking good in the moment (performance) is different from actually changing what’s stored in your brain long-term (learning). When a student uses ChatGPT to write a flawless essay, their performance looks stellar. But has real learning happened? Often, the answer is no.

How your brain actually learns

Barbara Oakley, whose research has influenced millions, describes genuine learning as moving from conscious, effortful thinking to automatic expertise. When you first learn something new, you use what scientists call declarative memory: Every step requires full attention. With repeated practice, those brain pathways become automatic, shifting to procedural memory. For example, when you first learn to drive, every movement demands complete focus. Now you navigate traffic while talking.

Oakley and colleagues have applied this neuroscience to understand AI’s impact. When AI does the thinking for us, this crucial brain transformation never happens. The mental pathways that create genuine expertise simply don’t form.

The illusion of understanding

Here’s an example. Imagine you teach a concept on Monday and test students that same day. They score 90% on average. But test those same students a week later, without review, and average scores drop to 60%. The knowledge never made it into long-term memory. Students performed well on Monday, but they didn’t truly learn.

The week-long gap forces the brain to retrieve information, and that mental struggle to remember is what creates learning. This is what cognitive scientists call “productive struggle” or “desirable difficulty.” Research shows that AI can create a similar illusion: Students may improve immediate performance with AI, but these short-term gains often don’t translate into better long-term retention.

When you hand over thinking to AI—not just calculations, but the mental work of figuring things out—your brain doesn’t build the connections that create real understanding. As Robert Pondiscio argues, “Students who rely on AI to write an essay may submit excellent work, but they have not done excellent thinking.”

Same technology, opposite results

The same AI technology can either accelerate learning or destroy it. The difference isn’t the technology, it’s how it’s designed and used.

When AI is poorly designed: A 2024 study in Turkey gave high school students unrestricted access to AI tools without pedagogical guidance. Their performance dropped 17% compared to students who didn’t use AI. Another study found students using ChatGPT produced better-looking work but dramatically reduced the planning and self-evaluation that drives learning.

When AI is well-designed and teachers are prepared: A Harvard study found students using a well-designed AI tutor learned more than twice as much in less time. The system promoted active thinking and provided scaffolded support. Stanford researchers found similar results, AI amplified teacher expertise rather than replacing teacher judgment. Research in Nigeria showed students achieved in six weeks what typically takes one and a half to two years. The critical detail is that these results required trained teachers guiding every session, asking questions and challenging students to think deeply.

The path forward

AI is neither inherently good nor bad for learning. What determines the outcome is how AI tools are designed and how educators guide their use. The key is preserving the mental struggle that drives learning, whether through task design or strategic use of existing platforms, ensuring AI prompts thinking rather than replacing it.

When AI does the thinking for you, you don’t even know what you don’t know. Science of learning expertCarl Hendrick’s research shows learning requires feedback on what you got wrong. But if AI solves the problem for you, there are no errors to learn from. You never struggled, so there’s nothing to correct. In the words of cognitive psychologist Daniel Willingham, memory is “the residue of thought.” If AI eliminates the need to think, it also eliminates the opportunity to remember. This is why well-designed AI and trained teachers are essential. Together, they ensure students do the cognitive work.

But teacher preparation alone isn’t enough. Those making decisions, from school principals to ministry officials, must understand the distinction between performance and learning. This requires new evaluation approaches that assess learning both with and without AI. When investing in AI, leaders need to ask: Can students think independently afterward? Can they apply it to a new context?

The real investment is building capacity across education systems by:

• Training teachers in both technological skills and pedagogical knowledge to maintain cognitive effort in learning;
• Preparing school leaders and education officials to recognize the difference between impressive performance and actual learning;
• Ensuring policymakers understand what effective AI implementation requires—adequate training time, ongoing support, and proper resources; and
• Creating communities of practice where educators build collective expertise.

In developing countries, these challenges are magnified. Awareness of learning science among educators and policymakers remains limited, making students with weaker foundational skills most vulnerable. They lack the prior knowledge to recognize when AI is wrong, yet they’re the ones who most need productive struggle to build expertise. But the evidence from Nigeria, Harvard, and Stanford shows a path forward: AI works when it amplifies human expertise, not when it replaces human judgment. Our critical investment must be in preparing educators who understand that real learning requires cognitive effort. With proper preparation, AI can support the productive struggle that builds lasting expertise, so students don’t just produce smart answers but become smarter thinkers.

This blog was inspired by the engaging presentations at researchED Chile 2025, where Barbara Oakley, Tom Bennett, Nidhi Sachdeva, Greg Ashman, Rodrigo López and other leading educators explored how evidence-based practice can guide our approach to emerging technologies in education.

Learn more

Science of learning essentials

• Learning How to Learn – Barbara Oakley’s Coursera course (4.8M+ learners) explaining the neuroscience of learning, including how memory works, the importance of productive struggle, and practical techniques for deep learning.
• A Mind for Numbers – Barbara Oakley’s book on how to excel at learning difficult subjects, based on cognitive science research.
• Learning Versus Performance – Robert Bjork’s lab at UCLA, pioneering research on the distinction between performance and learning, desirable difficulties, and retrieval practice.
• The Learning Scientists – Evidence-based strategies for effective learning, including retrieval practice, spaced practice, and elaboration.

UNESCO’s Science of Learning portal

Barbara Oakley on AI and learning

• The Knowledge Project Podcast – Barbara Oakley discusses learning, memory, and the science behind effective education.
• The Memory Paradox – Oakley et al. (2025) on why our brains need knowledge in an age of AI, exploring cognitive offloading and memory systems.

Related perspectives on AI and education

• The Illusion of Learning – Robert Pondiscio on the danger of AI to education, arguing that AI forces us to re-articulate what education is for: not producing polished outputs, but forming capable, discerning minds.
• The Research Brief: What’s New in Learning Science – Carl Hendrick’s monthly review of new research on learning, including recent findings on AI feedback, retrieval practice, and reading instruction.
• Science of Learning Implementation Guide – Carl Hendrick’s free guide for school leaders on applying learning science principles in practice.
• What Will Improve a Student’s Memory? – Daniel Whillingham’s work shows that memory improves when students think deeply about meaning, because we remember what we think about.

---

Source : World Bank