Limitations of Research

I occasionally give students short readings to teach principles of cognitive science and help them understand how people learn. I gave one out this week, linked here if you’d like to read. It’s a one page, 3-4 minute read for students, excerpted from Make It Stick, and focusing on desirable difficulties. We have a short discussion focusing on the idea that effective learning should not be easy; when an activity feels hard, it may just be that a lot of learning is happening in that moment. I also try to communicate that difficulties aren’t universally good — there has to be some difficulty on the way to success, not just difficulty for the sake of difficulty. But I want to send a message to students about perseverance and what real learning should look like.

Reading it, I realized that some of the research I had cited from Make It Stick (published three years ago), is already out of date. One study often cited to show what we mean by desirable difficulties is that reading an article that is slightly out of focus or in a font that is challenging to interpret can actually increase retention of the content. This fits with other results on desirable difficulties — when learning feels harder, there can actually be more cognitive work happening, and that learning can be more durable. But the specific result on out-of-focus text has been hard to replicate and thrown into question.

This isn’t the only example of high-profile scientific findings that failed to replicate or were poorly researched to begin with. Psychologists are struggling with a replication crisis affecting findings that have been textbook staples for decades. The challenges of replication have had an enormous impact on that field. Medicine, often held up by educators as the holy grail of evidence-based practice, has seen its own share of challenges. Many common medical practices are less founded on research than we might like to think, and when research suggests a different approach, practicing physicians are slow to change their recommendations.

I write often about the potential for research to inform teaching. I still believe that there is a great deal of potential there. But at the same time, these lessons are a reminder not to move too fast, or to become too invested in any particular bit of research. Maybe teachers should hesitate to say that we wish we had a base of knowledge like the medical profession. It’s probably easier to be on the outside looking in than to deal with the messy realities of bringing research to practice. And it’s important to remember that research is not absolute and should be incorporated slowly and purposefully into practice, rather than rushing the latest findings into the classroom.

All this brings me back to Dan Willingham’s metaphor for research in teaching. He compares teaching to architecture, rather than medicine. Architects use scientific knowledge to inform things like safety and structure, but that knowledge does not tell them what a building should look like; instead, it provides boundary conditions within which architects can work to meet multiple goals at the same time. Research in teaching that prescribes a specific course of action is probably the wrong way to go; instead, research that informs our underlying goals of student learning can help us to understand what is happening in students’ brains. Research is unlikely to provide any easy answers to teachers. Instead, it is a body of knowledge, built up slowly and carefully and constantly questioned, that creates a more informed profession.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s