Illusions of Learning

“Have you heard of IXL? I love IXL, it’s so easy, it makes me feel so smart.” – Student

IXL is a computer-adaptive website that many teachers use for skills practice. I have nothing in particular against it. I do think that, more broadly, computer-based personalized learning platforms and the way they are used can fall into the trap of chasing what students like, rather than what’s best for their learning.

Here is an excerpt I often come back to on the science of desirable difficulties in learning:

Not long ago, the California Polytechnic State University baseball team, in San Luis Obispo, became involved in an interesting experiment in improving their batting skills.

Part of the Cal Poly team practiced in the standard way. They practiced hitting forty-five pitches, evenly divided into three sets. Each set consisted of one type of pitch thrown fifteen times. For example, the first set would be fifteen fastballs, the second set fifteen curveballs, and the third set fifteen changeups. This was a form of massed practice. For each set of 15 pitches, as the batter saw more of that type, he got gratifyingly better at anticipating the balls, timing his swings, and connecting. Learning seemed easy.

The rest of the team were given a more difficult practice regimen: the three types of pitches were randomly interspersed across the block of forty-five throws. For each pitch, the batter had no idea which type to expect. At the end of the forty-five swings, he was still struggling somewhat to connect with the ball. These players didn’t seem to be developing the proficiency their teammates were showing. The interleaving and spacing of different pitches made learning more arduous and feel slower.

The extra practice sessions continued twice weekly for six weeks. At the end, when the players’ hitting was assessed, the two groups had clearly benefited differently from the extra practice, and not in the way the players expected. Those who had practiced on the randomly interspersed pitches now displayed markedly better hitting relative to those who had practiced on one type of pitch thrown over and over. These results are all the more interesting when you consider that these players were already skilled hitters prior to the extra training. Bringing their performance to an even higher level is good evidence of a training regimen’s effectiveness.

Here again we see the two familiar lessons. First, that some difficulties that require more effort and slow down apparent gains — like spacing, interleaving, and mixing up practice — will feel less productive at the time but will more than compensate for that by making the learning stronger, precise, and enduring. Second, that our judgments of what learning strategies work best for us are often mistaken, colored by illusions of mastery.

When the baseball players at Cal Poly practiced curveball after curveball over fifteen pitches, it became easier for them to remember the perceptions and responses they needed for that type of pitch: the look of the ball’s spin, how the ball changed direction, how fast its direction changed, and how long to wait for it to curve. Performance improved, but the growing ease of recalling these perceptions and responses led to little durable learning. It is one skill to hit a curveball when you know a curveball will be thrown; it is a different skill to hit a curveball when you don’t know it’s coming. Baseball players need to build the latter skill, but they often practice the former, which, being a form of massed practice, builds performance gains on short-term memory. It was more challenging for the Cal Poly batters to retrieve the necessary skills when practice involved random pitches. Meeting that challenge made the performance gains painfully slow but also long lasting.

This paradox is at the heart of the concept of desirable difficulties in learning: the more effort required to retrieve (or, in effect, relearn) something, the better you learn it. In other words, the more you’ve forgotten about a topic, the more effective relearning will be in shaping your permanent knowledge (Make It Stick, excerpted from 79-82).

Part of my role in the classroom is to engage students in thinking about challenging ideas, monitor their learning minute by minute, day by day, and beyond, and connect concepts over time as we revisit them in more and more depth. I try to do all of that through the lens of a scientific understanding of how students learn. In 2017, too much personalized learning colors perceptions with the illusion of mastery and relies on making content feel easy as a substitute for substantive engagement, trading durable, transferable learning for hollow confidence-building and short-term skill retention.

I am interested in computer-based platforms for supplemental practice if they make my life easier, but personalized learning is far from where it needs to be to take on a primary role in the classroom.

2 thoughts on “Illusions of Learning

  1. David Wees (@davidwees)

    One issue with the baseball example is that these baseball players are basically experts already and the new thing that they need to get better at is exactly what is targeted by the interleaved practice => distinguishing a curve ball from a fast ball.

    I recently read a study that compared interleaved teaching versus blocked teaching and interleaved practice versus blocked practice. Basically everyone in the study benefited from the interleaved teaching. The students who entered the class as the lowest performing students benefited more from blocked practice than interleaved practice. The theory is that the interleaved practice overwhelmed the lower attaining learners’ working memory more easily than the higher attaining learners and so they saw less benefit from it. Unfortunately I cannot find this reference right now so it is possible that I am misinterpreting this study or misremembering it.

    Here’s a blog post that connects this issue to the task propensity issue you’ve been discussing recently.

    This strategy is particularly useful if you’re studying something that involves problem solving – like math or physics – interleaving can help you choose the correct strategy to solve a problem (1). Interleaving can also help you to see the links, similarities, and differences between ideas (2).

    Note that the assumption of this post, and probably many people in the field, is that the whole point of math class is to be able to solve mathematical problems. But while problem solving is a critical aspect of math class, as you and I have pointed out, our goal is to surface mathematical ideas and to be able to use these mathematical ideas to solve problems. So we deliberately step back with students and look for ideas that we can generalize. If the practice and/or work students have done is always interleaved, there will be minimal patterns in the problems for students to discuss and generalize from.

    For me this me this means that I want to interleave practice when the goal is for learners to be able to distinguish between problem types and/or practice things they already know and I want to use blocked practice when I want learners to be able to generalize something from the practice. Note that I prefer to deliberately sequence problems in order to make it easier for students to be able to generalize from the experience, so with blocked practice I would sequence problems deliberately and vary an element of the practice deliberately across that sequence so that learners are more likely to see how that elements affects the problem solving process.

    An example of this for the baseball practice would be throwing a bunch of curve balls for a beginner (note: not an expert) in batting practice and then changing, just slightly, the height of the baseball as it passes through the strike zone. Once the learners have some experience with a curve ball, I’d make sure that the it came back, ideally at potentially random intervals (eg. interleaved practice) frequently. With a solid idea of a curve ball in hand, this would make distinguishing a curve ball from a fastball more obvious and then I could use similar variation or deliberate practice with the fastball. Within a few sessions I’d switch completely to interleaved practice since the players would have developed some expertise with each of the kinds of pitches.

    1. dkane47 Post author

      Thanks for this comment David. I think I need some more time to really digest this, and if you find the reference for that study I’d love to read it.

      I wonder if, to use interleaving equitably, I need a paradigm for success that is specific to the goals of an activity or sequence of problems, and focused on the student experience in that moment. If I’m having students practice graphing sine and cosine functions, the practice could be interpreted as interleaved just by varying the period, amplitude and midline, rather than interleaving those problems with unit circle problems — so the “radius of interleaving” increases over time. One thing I often come back to is that when practice feels less successful and students are getting more problems wrong they may actually be learning more — but I need to be really specific about what they are learning and what success looks like. Otherwise, I’ve generated an unfalsifiable theory — more problems wrong and more difficulties doesn’t lead to more learning ad infinitum.

      Definitely need to think more about this. Thanks for your thoughts!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s