The Science Police?

[gayle_yamazaki]

Many years ago, I worked with a dear colleague to examine the findings of a meta-analysis on research findings. Although Lee Becker is no longer with us, I know that the recent work of the Association for Psychological Science (APS), the Registered Replication Results project, and the Center for Open Science would have been of keen interest. In fact, I think Lee would have jumped into the middle of these organizations with gusto. As a graduate student, I was surprised that in our efforts to replicate the findings of the meta-analysis died on the vine. We were not able to get the basic data or information about the 60 studies that were featured. We were not able to reproduced the findings of just this one publication. At that time, it was not a popular topic and no one had much interest in our failure. Lee, in his quiet and thoughtful way, said "One day it will matter." He was right.

Over the past few years, psychological research has been closely examined and even brought into question. The Association for Psychological Science asked a simple question, "How reproducible is psychological research?" As a result, the Registered Replication Results project began and the Center for Open Science (COS) began its efforts to reproduce 100 studies that had been published in highly regarded, peer-reviewed journals. Just last week the COS published their findings in the August 2015 edition of Science. The lead author, Brian Nosek, revealed disappointing findings. Of the 100 studies reproduced using as close to the original materials and procedures as possible, only 36 of the 100 studies were found to result in statistically significant results. This is far less than Nosek had originally predicted. All of the news was not bad, when original and replicated results were combined and no bias was assumed, then 68% of the studies were found to have statistically significant results.

Psychology is not the only science that has examined itself. Over the years many of the other scientific communities have begun to wonder about their own research and have started to look at their own studies in the much the same way. We will have to wait to see how those examinations turn out. If you're wondering if someone is watching how research is done and enforcing good research practices, there are organizations out there that are kind of like the Science Police. There are online sites like PubPeer.com that allows for open discussion about research and the integrity of research. There is the Office of Research Integrity that activity investigates and reports on research across many disciplines. Academic institutions also launch their own investigations when questions of research integrity are raised. If you're interested to read about these types of research integrity actions, you can subscribe to Retraction Watch blog (RetractionWatch.com) which briefly describes what research has been investigated and the outcome of the investigations across all disciplines. There are some authors who have had in excess of 50 articles retracted from publication.

The work that is featured in this study is important for Psychology and science in general. Science across our disciplines is evolving, we should not be too set in our ways and hold on too tightly to the assumptions we have. Science emphasizes the importance of replication, now maybe the science and practice of replication will finally be recognized and valued.

We owe the 250 researchers who set aside time to participate in the Replication Study a big thank you. Their work is vital and important for psychology and science in general.

Link to Pacific Standard article featuring information from an interview with Brian Nosek: Click Here

Link to structured Abstract about the COS article published in the August 2015 edition of Science: Click Here