A new analysis published in Science suggests the field of psychology might not have the “reproducibility problem” discovered in a 2015 study.
The original study re-analyzed and repeated 100 studies published in three psychology journals. The 270 researchers involved in the project were able to replicate the results of just 39 studies. For an additional 24 studies, the results were “moderately similar” to the original research. Reproducibility—the ability of researchers who may have different beliefs or biases to replicate the original findings of a study—ensures the reliability of scientific research. Thus, the failure to replicate these studies was billed as a “crisis” in psychology.
Does Psychology Have a Reproducibility Problem?
The follow-up study, led by Harvard psychologist Daniel Gilbert, looked at the previous study’s attempt to replicate psychological research. The team found a number of problems with the initial study, including replication techniques that did not properly repeat the original research.
For example, to repeat a study that originally looked at American attitudes toward black Americans, researchers substituted native Italians. To replicate a study that required young children to locate items on a large computer screen, researchers instead gave older children the task of locating items on a smaller computer screen.
These apparently small differences in methodology can produce significant differences in research outcomes. In some cases, the new study argues, the original replication study relied on too few subjects for replication to either succeed or fail.
The Debate Continues
As the ongoing debate about replication makes clear, science is a process of testing, re-testing, analyzing, and debating data. The new study does not necessarily overturn or disprove the original reproducibility study, just as the original reproducibility study did not conclusively prove that psychological research is deeply flawed.
The original authors of the 2015 reproducibility study have already published a rejection of the 2016 analysis. They point to a number of concerns with the new research, saying the team erroneously focused on the confidence interval (CI), one of five measures of reproducibility used in the original research.
A related study published in February in PLOS One was able to repeat the results of 75% of the studies in the original replication project. However, the evidence for the conclusions contained in the original study was often weak. This suggests more research is necessary to explore whether psychology has a reproducibility problem.
- Bower, B. (2016, March 3). Psychology’s replication crisis sparks new debate. Retrieved from https://www.sciencenews.org/article/psychologys-replication-crisis-sparks-new-debate
- Estimating the reproducibility of psychological science. (2015). Science,349(6251). doi:10.1126/science.aac4716
- Etz, A., & Vandekerckhove, J. (2016). A Bayesian perspective on the Reproducibility Project: Psychology. PLOS One, 11(2). doi:10.1371/journal.pone.0149794
- Researchers overturn landmark study on the replicability of psychological science. (2016, March 3). Retrieved from https://www.sciencedaily.com/releases/2016/03/160303145733.htm
© Copyright 2016 GoodTherapy.org. All rights reserved.
The preceding article was solely written by the author named above. Any views and opinions expressed are not necessarily shared by GoodTherapy.org. Questions or concerns about the preceding article can be directed to the author or posted as a comment below.