Limitations of Research

I occasionally give students short readings to teach principles of cognitive science and help them understand how people learn. I gave one out this week, linked here if you’d like to read. It’s a one page, 3-4 minute read for students, excerpted from Make It Stick, and focusing on desirable difficulties. We have a short discussion focusing on the idea that effective learning should not be easy; when an activity feels hard, it may just be that a lot of learning is happening in that moment. I also try to communicate that difficulties aren’t universally good — there has to be some difficulty on the way to success, not just difficulty for the sake of difficulty. But I want to send a message to students about perseverance and what real learning should look like.

Reading it, I realized that some of the research I had cited from Make It Stick (published three years ago), is already out of date. One study often cited to show what we mean by desirable difficulties is that reading an article that is slightly out of focus or in a font that is challenging to interpret can actually increase retention of the content. This fits with other results on desirable difficulties — when learning feels harder, there can actually be more cognitive work happening, and that learning can be more durable. But the specific result on out-of-focus text has been hard to replicate and thrown into question.

This isn’t the only example of high-profile scientific findings that failed to replicate or were poorly researched to begin with. Psychologists are struggling with a replication crisis affecting findings that have been textbook staples for decades. The challenges of replication have had an enormous impact on that field. Medicine, often held up by educators as the holy grail of evidence-based practice, has seen its own share of challenges. Many common medical practices are less founded on research than we might like to think, and when research suggests a different approach, practicing physicians are slow to change their recommendations.

I write often about the potential for research to inform teaching. I still believe that there is a great deal of potential there. But at the same time, these lessons are a reminder not to move too fast, or to become too invested in any particular bit of research. Maybe teachers should hesitate to say that we wish we had a base of knowledge like the medical profession. It’s probably easier to be on the outside looking in than to deal with the messy realities of bringing research to practice. And it’s important to remember that research is not absolute and should be incorporated slowly and purposefully into practice, rather than rushing the latest findings into the classroom.

All this brings me back to Dan Willingham’s metaphor for research in teaching. He compares teaching to architecture, rather than medicine. Architects use scientific knowledge to inform things like safety and structure, but that knowledge does not tell them what a building should look like; instead, it provides boundary conditions within which architects can work to meet multiple goals at the same time. Research in teaching that prescribes a specific course of action is probably the wrong way to go; instead, research that informs our underlying goals of student learning can help us to understand what is happening in students’ brains. Research is unlikely to provide any easy answers to teachers. Instead, it is a body of knowledge, built up slowly and carefully and constantly questioned, that creates a more informed profession.

Pushing Back Against Personalization

Dan Meyer wrote a great post a few days ago about the dangers of personalized learning, critical of an article on personalized learning in Educational Leadership. This paragraph captures the heart of his criticism:

The medium is the message. Personalized learning is only as good as its technology, and in 2017 that technology isn’t good enough. Its gravity pulls towards videos of adults talking about math, followed by multiple choice exercises for practice, all of which is leavened by occasional projects. It doesn’t matter that students can choose the pace or presentation of that learning. Taking your pick of impoverished options still leaves you with an impoverished option.

You should read his piece, it’s great. But since reading it, one thing has bugged me. When I read about personalized learning I see a kind of orthodoxy to the arguments, an assumption that tailoring the pace, presentation, and content of learning to individual students is inherently good. I want to push back against those premises. Dan makes a great point that the technology just doesn’t exist yet to realize the potential of personalized learning that is often talked about. Personalized learning will likely still scare me when the technology catches up, and that is because I believe personalizing pace, presentation, and content can create problems no matter how well it’s done.

Pace

As a teacher, part of my job is to adjust the pace of instruction to student needs, to speed up or slow down as necessary and revisit content when the circumstances demand. At the same time, part of my job is to hold every student accountable to high standards. There’s an inherent tension there. If I am willing to slow down my instruction until I am 100% sure every student has learned, we may never get anywhere. I’m not proud of that, but it’s the reality. And when that pace gets personalized, the students with a stronger background are likely to pull further ahead and increase existing inequities.

One vision of great teaching is that every student is working on a different topic, at that just right level of difficulty, moving through an endless ladder of content. Another vision is that every student works on polynomials for four weeks. Every student moves their understanding forward, grappling with the big ideas of the unit. Inevitably, that looks different for different students. But working on the same content creates opportunities for collaboration and engagement in math class. I’m not arguing that personalizing the pace of instruction is universally bad. Just that it’s worth pausing and questioning the idea that every student working at their own pace is an automatic win.

Presentation 

His students don’t report to class to be presented with information. Instead, they’re empowered to use a variety of learning tools. Some students, like Cal, prefer step-by-step videos; others prefer songs and catchy rhymes to help them learn concepts. [..] He opens a series of videos and online tutorials, as well as tutorials prepared by his teacher (link).

Lots of arguments for personalizing the presentation of information rest on learning styles. Many educators believe that tailoring instruction to individual students’ learning styles is critical for learning. Others cite research suggesting that adapting instruction to match learning styles is a poor use of time.

But separate from learning styles, I would argue there is a more fundamental misconception at play. Given a choice between different presentations, a learner is likely to choose the mode of presentation that feels easiest. Deeper processing that leads to durable learning often occurs when learning presents some difficulties, challenging students to think in depth about the content. Yet learners are likely to mistake ease of learning for effectiveness of learning; “In short, we often seek to eliminate difficulties in learning to our own detriment” (link). Again, I’m sure there are benefits of personalizing presentation of information, but there are potential downfalls as well.

Content

I’d like to start with a philosophical argument. Folks love to say, “in today’s day and age when you can instantly Google facts, facts don’t matter”. I disagree. Facts need to be in long-term memory to do anything useful with them. But, probably more relevant, research suggests that through “information avoidance”, people are likely only to confirm their existing beliefs as long as they have control over what they are interested in learning.

More concretely, some of my largest successes as an educator have been in changing a student’s perception of their ability as a mathematical thinker. These are few and far between, but when students choose what they would like to learn, we might not like what they choose. What if no women want to be engineers? What if no people of color want to be engineers? That’s an obvious problem. Part of the job of teachers is to empower students, both with knowledge and with beliefs that they can expand the horizon of what they can be good at. And that means pushing every student to learn, even if it’s not what they’re interested in.

I don’t mean to argue against any personalized learning, ever. But I do find that personalization is often assumed to be a good thing, without examining some of the assumptions underlying that thinking.

 

 

 

 

 

 

 

A Double Progression: Exponential Functions and Logarithms

I’m teaching Algebra II, and just completed a unit on exponential functions and logarithms. Students should have seen exponential functions before, but many have forgotten the big ideas. In Algebra I they likely only wrote, interpreted, and evaluated exponential functions. In Algebra II, we’ll be solving a wider variety of problems with exponential functions, and one major new idea is using logarithms to solve for a variable in the exponent.

I taught the unit with a double progression, with two ideas working in parallel. The first several days were focused on writing and interpreting exponential functions. We spent some time working on interest and other applications with money, as well as a variety of other contexts requiring students to construct exponential functions and evaluate them at different values.

At the same time, we were doing short activities introducing logarithms either at the start or end of class. I started with this puzzle from Kate Nowak
Screenshot 2017-03-08 at 9.05.32 AM.png
We then formalized that knowledge, spent some time playing log war to practice and get some muscle memory behind logs, and sprinkled log problems generously through homework. The bulk of class time was spent on exponential functions, but this parallel progression both helped to mix things up and proved to be enough to communicate the big idea of what a logarithm actually means — log base 2 of 16 means 2 to what power is equal to 16. We didn’t get into more advanced log rules at this point, because those rules are rarely necessary to solve problems with exponential functions and I saw as an unnecessary distraction from my key mathematical goals.

Then we moved into problems like this one from Illustrative Mathematics that require logarithms to solve.
Screenshot 2017-03-08 at 9.11.01 AM.png
We had done some guess and check earlier in the unit, but with a solid base to work from, it seemed like a natural step to connect between the two strands of this progression and make our calculations more precise and efficient using logarithms.

I really liked this approach. It helped that I teach 90 minute blocks, so it’s manageable to have multiple things going on in a simple class compared with shorter daily blocks. I also think it made the math we were learning feel more purposeful, as well as having relevant ideas fresh in students’ minds when they became necessary. I’m curious now — for what other topics would a double progression like this be effective?

Test Prep: BC Calculus

I’m teaching AP Calculus BC for the second time this year. I have students who are motivated to do well on the AP exam and I want to support them with preparation for the test without focusing the class entirely on test prep. Here are some strategies I’ve used to try to meet those goals:

  1. Teach math well. If my students know the math they are likely to be successful on the test. This is number one, and is necessary whether or not students plan to take the AP test.
  2. Focus on the big ideas. The College Board’s framework for AB and BC Calculus names six essential mathematical practices that are useful tools for students to take away from the course, as well as the foundation of the AP exam. I try to name these big ideas whenever possible and structure my teaching around them. The mathematical practices are:
    1. Reasoning with definitions and theorems
    2. Connecting concepts
    3. Implementing algebraic/computational processes
    4. Connecting multiple representations
    5. Building notational fluency
    6. Communicating
  3. Know the test. I’ve taken several practice tests and continue working through practice tests throughout the year. I think that the AP Calculus exam is pretty high quality, in that every time I take a practice test I find questions asked in ways I would not have thought of, connections between seemingly disparate topics, and relatively few problems that can be solved by memorizing and applying a single formula or mathematical idea. The AP Course Audit site has a number of secure tests that are great for this purpose.
  4. Pull from a range of sources. If I am relying entirely on one textbook, I’m unlikely to expose students to the breadth and depth of mathematics that they need to be successful on this test. I use ideas from previous AP tests, the Exeter Math 4/5 problem sets, the Active Calculus online textbook, the MTBoS search engine, and my school’s Larson, Hostetler and Edwards Calculus textbook (eighth edition).
  5. Practice test items in small chunks. I want students to be exposed to AP-level questions on a regular basis, but I don’t want to set aside time for full practice tests early in the year. I use the secure tests to screenshot selected questions into five-question multiple choice chunks or single free-response questions that I try to give students at least weekly. Students take a few minutes to work through the questions, then we discuss ideas that come up from those problems. Secure tests are for in-classroom use only, so students shouldn’t be taking problems home but can practice their skills in small chunks, over time, and address challenges as we go.
  6. Memorization checks. There are a bunch of things that students should have memorized for the AP test — arc length formulas, Lagrange error, the Pythagorean Identities, derivatives of parametric functions, and more. I start doing regular memorization checks about two months out from the AP test. These are given like quizzes, but are not collected or graded. I don’t want dedicate too much time to, or evaluate students on, their memorization. At the same time, I want to structure practice so students have a chance to memorize what they need, and make sure students know where they stand so they can put in more work if they would like to. And doing it as a class creates a forum for students to share strategies for memorizing key information or work through common derivations so they have some tools to make connections and buttress their understanding.

This is still a work in progress, and I don’t know that I’m particularly good at prepping students for the AP test. At the same time, these tools have helped me to balance exam prep with class time focused on doing math for the sake of doing math without focusing endlessly on the test.

On Changing Minds

I think the recent article, “Why Facts Don’t Change Our Minds” from the New Yorker, is a must read. Elizabeth Kolbert explores how, once they are formed, beliefs are remarkably difficult to change, referencing fascinating research. The article has me reflecting on how teachers come to believe things about education, and how they may or may not change their minds.

In the last few months, I’ve been in many tense discussions of education, politics, religion, and more. Kolbert’s article is a good reminder that, in conversations where beliefs run deep, it’s a rare thing to change another human’s mind. When someone does change their mind, it is rarely because of the volume of research evidence presented, or the passion with which an argument is made. It’s much more often because of open dialogue and through mutual respect. Opinions are a funny thing.

While I believe in best practices that I find important in my teaching, another core value of my pedagogy is to avoid dogma. I’ve had to remind myself recently that, while this means I disagree with plenty of teachers on questions in education, everyone is doing the best they can. It’s often unhelpful to start arguments with people about deeply entrenched beliefs. I may have strong opinions on learning styles, differentiation, standards-based grading, the Common Core, cognitive science in teaching, inquiry vs explicit instruction, homework, encouraging growth mindset, and more. And I may believe that those ideas are based on facts and evidence. But most of my beliefs about teaching have changed in the last few years. Many will change again. I have trouble changing my mind when conflicting evidence comes around — those changes happen slowly, over time, and rarely because of one conversation.

If I’m being honest, any particular idea I try to push into the head of another human probably matters far less than I think it does. Teachers who disagree with me are unlikely to be doing any harm to their students. They are likely to really believe in what they do, bring an enormous amount of passion and knowledge into their classroom, and want what is best for their students. And I’m unlikely to change their minds anyway.

All this is to say that I want to focus my energy on learning, on engaging in dialogue with other teachers interested in learning, and on building relationships based on mutual respect. I want to recognize that many arguments I might choose to engage in are likely to alienate people who think differently than me. And I want to remember that everyone is doing the best they know how, every day.

When to Interject?

Here are two situations.

Visual Pattern
A visual patterns warmup routine, focused on writing expressions for the number of squares in the nth step of this pattern:
7958030_orig

Groups come up with these two expressions and explain how they relate to the pattern:
screenshot-2017-02-22-at-5-07-03-pm
Another way to write an expression that no student found looks like this:
Screenshot 2017-02-23 at 11.56.55 AM.png
The final expression is a different way of conceptualizing the pattern and offers a potentially useful perspective for future problems. Do I present it to students?

Number Talk
A number talk warmup routine, trying to find a strategy to mentally multiply or approximate 0.48*650. Students offer several different ways to break it down, either by breaking up the 0.48 or by breaking up the 650. Another approach that I’ve found useful for problems like this is to solve it in terms of percents — finding 10%, then 1%, then 50%, 2%, and finally 48%. Do I present it to students?

When to Interject?
There are multiple possible next steps.

  • I could explain the additional perspective and why it’s useful
  • I could present the expression or the outline of the number talk strategy and have students discuss in small groups to figure out how to assign meaning to it
  • I could send students back to groups with an open-ended question of trying to find another method
  • I could move on and keep my ideas to myself

I’m not sure what the best answer is. My fundamental struggle here is that these routines work with a great deal of student ownership. I’m doing plenty of work selecting and sequencing students to share their ideas and building norms for students to be attentive to each other and make connections. But I speak very little, and I typically let students present all of the math. Which is more important: norms of student ownership, or a pedagogically useful piece of more explicit instruction?

A Model of Progression: Geometric Series

I’m still thinking about the idea of a model of progression — breaking down a complex activity into a series of manageable components.

I tried to put this idea into practice with the formula for the sum of a geometric series. Here is a proof of the formula, lifted from Purplemath.

screenshot-2017-02-18-at-1-02-16-pm

I’ve tried to teach this proof before, always pretty unsuccessfully. The long division step seemed like hocus pocus and lost most kids, and they either got lost in the sauce or disengaged completely. Maybe a few kids got something out of it, but it also reinforced student ideas that math doesn’t make sense and is something done to them, rather than with them.

I want students to understand this complex idea, and one essential building block is to understand the following property of polynomial multiplication and division:

screenshot-2017-02-18-at-1-07-58-pm

So the day before I planned to introduce the geometric series formula, we did an instructional routine drawn from Routines for Reasoning called Recognizing Repetition. I gave students these expressions:

screenshot-2017-02-18-at-1-09-09-pm

First, I asked students to distribute and rewrite each expression and notice what was being repeated each time. They worked individually at first and then shared ideas at tables, and each table reached some informal ideas about every term cancelling except the first and the last. Then, I asked students to think about generalizations they could make to formalize their thinking into a broader rule. This led to a challenging discussion — again, full of informal ideas but without many resources to write a formal generalization. I ended up doing some of the work to formalize our collective thinking, and we finished with a short meta-reflection on the process of recognizing repetition and writing generalizations.

My goal with this task was to separate the polynomial operations from the rest of the mathematical thinking, get students comfortable with the essential ideas of polynomial operations, and set them up for better success with the proof the next day.

Working through the proof felt much more successful than I’ve ever been before, but with one additional challenge. There is a parity issue, where the difference between n-1 and  n creates some challenges in figuring out exactly how many terms each expression represents and what the appropriate exponent should be in the final formula. It’s tricky — that if a series includes all terms from 0 to n-1, there are actually n terms in that series. While doing the polynomial operations bit ahead of time made a big difference, next time I need to figure out how to focus on that parity question to help the proof go even more smoothly.

More broadly, I really enjoyed doing this type of thinking, and want to figure out what else I can do to lead into challenging topics in a progression that is designed for student understanding. Next up, exponential functions and logarithms!