University of Chicago psychology professor Leslie Kay and her research group set out to resolve a 15-year-old scientific dispute about how rats process odors. What they found not only settles that argument, it suggests an explanation for the much written-about “replication crisis” in some fields of science and points to better ways of designing experiments.
Reproducible experimental results are part of the bedrock of scientific method. But a concern is that researchers, particularly in psychology and medicine, are too often unable to replicate the findings of colleagues in other labs.
This has certainly been true of understanding how rats—and by extension, possibly humans—process smell. “There was simply a disagreement in the literature,” Kay said. “Different labs tried to get the same result, and they were unsuccessful.”
The diverging results came from two camps, doing similar but slightly different experiments. What Kay and her group found was that while both were correct, they were asking different questions without realizing it. Their experiments were not, in fact, comparable.
Kay’s group’s study, published this spring in Journal of Neuroscience, shows that the disparate conclusions arise from small but crucial differences in the way the two sets of experiments were set up. By eliminating those differences, and then doing both experiments rather than only one, the group was able to tease out similarities underlying the varying results and discover a general truth about how rats smell.
In both kinds of experiments, rats were trained to recognize pairs of odors by sniffing them and then discriminating between the two after being asked. In the first type of experiment, if a rat smelled odor A (banana, for example), it poked its nose into a hole and got a reward. If it smelled odor B (sweaty socks), it did nothing and received no treat. In the second type of experiment there were two holes; the rat poked its nose into one if it smelled odor A, and into the other if it smelled odor B. Both earned a reward.
The labs that did the one-hole experiments concluded that rats sniff deliberatively, gathering information over time. The two-hole experimenters concluded that rats just do a quick sniff and leave, getting by with whatever information they gather in that short time.
Researchers discussed whether the tasks made the rats respond differently in the two experiments. Did reacting to smell B in the one-hole studies make rats slower overall, or did the different number of rewards affect rats in other ways?
Controlling the variables
Donald Frederick, a graduate student in Kay’s lab, decided to explore those questions by conducting both kinds of experiments in an extremely controlled way, testing the rats on many different pairs of odors. The experiments were designed to be identical to the point that the rats learned to recognize the second odor and discriminate between the two odors by choosing holes.
“For each type of task, we got results that were comparable to what had been found before for that type of task,” Kay said. When faced with the two-hole task, the rats sniffed quickly and acted quickly. When faced with the one-hole task, they took an additional sniff before acting.
Previous researchers had concluded, mistakenly, that their results for a single task held true for the way rats smell in all situations. Because Kay’s group looked at both tasks in experiments that were set up identically, they were able to see that the differences in experimental design between the two types of studies had an enormous effect on the outcomes. The type of rewards used, the precise way the rats were trained and how hungry they were when they did the tasks, among many other factors, all affected the results.
“It’s a little bit overwhelming when you start to realize that everything is going to affect how the animals behave,” Kay said. “But we really have to pay attention to that. Many non-replications may be due to experimental details that people think are unimportant,” she said. “They aren’t necessarily non-replications at all; they’re doing a different experiment.”
Once they eliminated the “noise” created by the differences in the experimental set-ups, Kay and her colleagues were able to discern an underlying similarity in the rats’ approach to the two situations.
“By doing both experiments, we found that the rats are doing the same thing, they’re just doing it in a more compressed fashion for one task than the other,” Kay said. “Because we employed so many different odors and did such a carefully balanced study, we were able to show that, in fact, in both tasks, they’re accumulating information over time. And they extend their sampling times in both tasks when it’s hard to discriminate between the odors.”
The important message, Kay said, is that, “it’s crucial to use multiple tasks in trying to come to general conclusions. We’re all searching for general truths, and we forget that we’ve found a specific truth. When we forget that, we stop looking for what’s really the general truth. Only by using multiple ways of addressing a question within the same lab can we get at the underlying truths about cognition.
—Prof. Jacqueline Stewart
