“the conclusions of the SPE had been written in advance according to non-academic aims”
Why is this interesting? is well worth subscribing to. This edition is, as always, interesting. Several of the classic phycology experiments weren’t as scientific as you’d have hoped.
This is part of a much broader story of a “replication crisis” in psychology: The challenge of being completely unable to reproduce the results from some of the discipline’s most famous experiments. Here’s how FiveThirtyEight described it a year ago:
The replication crisis arose from a series of events that began around 2011, the year that social scientists Uri Simonsohn, Leif Nelson and Joseph Simmons published a paper, “False-Positive Psychology,” that used then-standard methods to show that simply listening to the Beatles song “When I’m Sixty-Four” could make someone younger. It was an absurd finding, and that was the point. The paper highlighted the dangers of p-hacking — adjusting the parameters of an analysis until you get a statistically significant p-value (a difficult-to-understand number often misused to imply a finding couldn’t have happened by chance) — and other subtle or not-so-subtle ways that researchers could tip the scales to produce a favorable result.
In addition to the SPE (which, in some ways, isn’t even a question of replication as much as whether it was ever a scientific study at all), the list of famous experiments that haven’t held up is full of well-known studies. According to Atlantic writer Ed Yong, “There’s social priming, where subliminal exposures can influence our behavior. And ego depletion, the idea that we have a limited supply of willpower that can be exhausted. And the facial-feedback hypothesis, which simply says that smiling makes us feel happier.” There’s also the “marshmallow test”, which asked children to wait 15 minutes to eat a marshmallow in order to get a second marshmallow. Those who were able to wait longer did better on standardized tests and had fewer behavioral problems. The problem was that the study was done at Stanford and most of the children had parents who were professors and, as a result, pretty well off. When researchers eventually controlled for those factors they found the study’s results didn’t hold up. In other words, kids who had a stable household were more likely to be able to delay gratification, and once you sort things that way there doesn’t seem to be much long-term difference between the kids who waited and those who didn’t.
The challenge, of course, is that for lots of these studies the cultural impact has already been made.