Science in crisis: reproducible results trump novel nonsence
Science is based on irrefutable, reproducible results. That’s why we put so much faith in scientific evidence. We trust the results of published research have been rigorously tested and analysed, to ensure the information we are being told (and often basing many important ethical and wellbeing decisions on) is solid. Most methods used in scientific research are assumed to be repeatable, meaning the someone repeating the experiment would obtain the same results.
However, recently many people in the scientific community have been asking the same question: how repeatable are many trusted scientific methods?
This was the topic of debate (and eventually our study topic) between my peers and I on a recent field trip to the Strathbogie Ranges, over the mid semester break. Our initial plan was to repeat methods used in previous years of the subject, to allow for comparison between years and look for trends over time. Somehow, the topic of repeatability was brought up and we decided to complete a repeatability experiment, to test how reliable our method was and how valid our results might be.
This initiated a search for other studies that have solely tested the repeatability of commonly used scientific methods. Surprisingly, the search came up with limited results. Looking into this further, I discovered this has been a common topic in recent news.
Nature completed a study in 2016 to determine how many researchers in the scientific community were concerned with the repeatability of their (or others’) methods. The results are astounding. Over 50% of researchers who took part in the survey admitted they had tried (and failed) to reproduce results from their own experiments, and over 70% had failed to reproduce the results of others. Above all other reasons, researchers that took part in the study indicated that the main cause for unreproducible results is selective result reporting. The pressure to produce unexpected and novel results pushes people to take findings out of context and leave the big picture behind. This often leads to the replicable results being left out completely
The need for sound scientific method still holds true. Major concerns have been raised, particularly in psychology and biomedical fields. As human health and wellbeing is highly important, it is constantly in the spotlight, it is unsurprising that reproducibility of results in these fields are often questioned.
But what about other fields of research? Ecology (my own chosen discipline) is often studied outdoors in highly variable environments. How do you create methods that are highly reproducible when there’s rarely one environment just like another? The large environmental scale and variability of ecology as a discipline seems to so far have saved it from the reproducibility debate. But aren’t human health concerns (and humans themselves) just as variable and large-scale?
So this leads me to the question: have we lost sight of what truly matters in scientific literature? Is publishing true, meaningful results still not the number one priority? Novel and unexpected may not always be the key, and the pressure to produce unique results should be reduced. Or perhaps in future more emphasis needs to be put on methodical design. Although large-scale repeatability experiments may not always be feasible, an aspect of tested reproducibility must be brought in to the scientific method, with researchers being held accountable for their work.