Scientists tested 53 different ways to change human behavior.

Have you struggled to get yourself to work out or go to the gym even though you know you should? I’d like to tell you about one of the largest and most effortful studies on behavior change ever conducted. I’ll explain what they found out about what works to get people to increase their exercise, as well as the strange findings from the study that most people will miss even if they read the paper. Remarkably, this study, which was titled Mega Studies Improve the

Impact of Applied Behavioral Science, was conducted on 61,000 people, all of whom had gym memberships. Before the study began, 30 psychologists and behavioral scientists worked in small teams to design methods for increasing these people’s behaviors. Every behavioral intervention that they developed aimed to increase gym attendance over the following four weeks. This resulted in 53 behavioral interventions to test which were administered to participants using text messages. In my opinion, this is a brilliant setup to figure out what truly

Changes behavior. The plan was to test a ton of behavior change techniques on a huge number of participants to figure out what really works to get people to exercise more. The design of the study is important to understand to interpret the results. So, let’s briefly review it. Participants were randomly assigned to different groups. First, there was the control group. These participants received points at the start of the study which could be converted into small sums of money on Amazon. Second,

There was the baseline group. These participants received three things. First, they got planning. That is, participants planned the dates and times when they would go to the gym. Second, small incentives. They earned points usable on Amazon for every workout they completed. And third, reminders. They got a text message 30 minutes before each scheduled workout. So, some participants were in the control group and some were in the baseline group. But each of the remaining study participants was assigned to receive one of the 52

Other behavioral interventions. These 52 interventions incorporated all of the elements that the baseline group got, meaning that they got planning plus incentives plus reminders, but then added another method on top to test different behavior change strategies. After being randomized into one of these groups, participants had their gym attendance tracked for the next four weeks. The researchers then analyzed the extent to which each behavior change intervention really worked for increasing gym attendance. The very first interesting takeaway from the

Study is that being in the baseline group caused people to go to the gym more often than being in the control group. About 0.14 more gym visits per week. That means that the combination of a small monetary incentive for going to the gym plus scheduling when you plan to work out plus text message reminders had a modest positive effect on gym attendance. Not the most mind-blowing result, but it’s good to keep in mind that some combination of small rewards,

Making a schedule and reminders is helpful when you want to change your own behavior. But what about the other 52 interventions? The paper’s abstract says that 45% of these interventions significantly increased weekly gym visits. That would imply that 23 out of the 52 techniques worked to actually increase gym attendance. So that means there must be a ton to learn from these 23 different techniques that scientists developed. Right? This is where things get weird. When the researchers say that

An intervention significantly increased weekly gym visits, what do they mean by this exactly? Later in the paper, we learned that this means a behavioral intervention caused a statistically significant improvement above and beyond the control group. But wait a minute, that’s a misleading comparison. Recall that the 52 interventions each incorporated all the elements of the baseline group, which means that they included incentives, scheduling, and reminders. Then each intervention added one new method on top of all that stuff.

The control group, on the other hand, did not include all of this stuff. Additionally, we’ve already seen that the baseline intervention beats the control group. So, of course, many of the interventions beat the control. They incorporate the stuff in the baseline and the baseline beats the control. To see the problem here, imagine that you’re trying to figure out if lullabibies help people sleep faster. So, what you do is you give people a sleeping pill and singing them a lullaby

And you compare that to people that got nothing. Well, clearly that doesn’t work. In order to see if the lullaby helped, you have to compare people getting the lullaby and sleeping pill to people just getting the sleeping pill. If you leave out the sleeping pill in the second group, it’s not a fair comparison. Going back to the study in question, that means that comparing the 52 behavioral interventions to the control isn’t the most helpful way to

Analyze the data. Doing so makes it seem like many of the novel techniques work, but it could just be the common elements from the baseline made them seem like they worked. So, what is the most informative way to analyze the results from the study? A better approach is to compare the performance of each behavioral intervention to the baseline group, not to the control group. That way, if the behavioral intervention wins, it’s due to whatever unique element that behavioral intervention

Involves, not due to the fact that it includes what’s in the baseline group. Now, to be totally fair, this proper set of comparisons is calculated and discussed in the paper, but it’s not in the abstract. The abstract contains a concise summary of the paper, and it’s often what people read first. So, it’s important that it accurately describes the key findings. The abstract says, “We showed that 45% of these interventions significantly increased weekly gym attendance.” This is a very different

Result from what you get when you do the proper comparisons. Rather than 23 behavioral interventions appearing to work, you find that only four distinct interventions had a statistically significant impact on gym attendance above and beyond the baseline. Okay, so what are these few techniques that actually seem to work? First, there was bonuses after messing up. Remember that participants received points usable on Amazon every time they went to the gym. The idea of this behavioral intervention was to give participants bonus points if

They went to the gym at the scheduled time after they messed up one of their previously scheduled workouts. Essentially, this is a bonus for recovering after a mistake rather than giving up. What else worked in the study to increase gym attendance? It seems that bigger incentives worked, which means simply paying people more points every time they went to the gym. People tend to respond to incentives, so this isn’t really a surprise. Another technique that appeared to work is

Simply telling participants that a majority of Americans exercise and the rate is increasing. The concept here is that people can be socially influenced. If they think this is a common behavior, it seems to get them to do it more. The final technique that appeared to increase gym attendance was the most interesting. It gave participants a choice of either a gain frame or a loss frame. The idea is that participants in this group could choose to either earn

Points each day they visited the gym or they could choose to start with all the points and then lose points each day they did not visit the gym. They were told that their earnings would ultimately be the same regardless of their choice. The only difference is the psychological effect. Do you feel like you’re gaining each time you go to the gym or do you feel like you’re losing each time you don’t? Giving participants this choice seemed to increase gym

Attendance. Those are the four behavioral interventions that appeared to work. But it’s worth noting that the impact of them was modest, less than an additional 0.27 days of exercise per week on average. Furthermore, since these were the most effective techniques selected from a much larger set, we should expect there to be some regression to the mean. In other words, even these modest effects are likely at least a little bit inflated in size. This leads us to our next big takeaway

From the study. Behavior change is very hard. 30 scientists, including psychologists and behavioral scientists, worked to develop 50 interventions, and it seemed that at most four of them meaningfully changed behavior. That’s quite shocking. That’s not all that was included in the study, though. The study also tested whether different groups of people could predict which behavioral interventions would work to increase gym attendance. They had three different groups of people making predictions. These included a group of 300 ordinary

People, a group of 150 professors from schools of public health, and a group of 90 practitioners from companies specializing in applied behavioral science. Remarkably, none of the groups made accurate predictions about which behavior change methods work. In fact, none of them were statistically significantly better than having no predictive power at all. So, not only is behavior change challenging, it’s really hard to predict what will actually work to change it. Overall, I learned a ton from deep diving into this paper. While

I did not agree with how they presented the main findings in the abstract, the researchers who conducted the study put in a herculean effort to develop behavior change techniques and collect this important data. They should be applauded and I’m really grateful for their efforts. To explore how you can change your own behavior for the better, check out our free website, the 10 conditions for change. It provides a comprehensive framework for helping you think through what your current barriers

Are to engaging in a behavior. You can also read the paper that this video is about by clicking on the description box below. If you found this video interesting, I’d really appreciate it if you’d subscribe. And I’d also love to hear from you in the comments. What did you think of this paper? And what’s worked for you changing your own behavior?