Tip number 11: Mice or humans? Ask about the research method
01 januar 2023
Key points
Some methods provide more solid knowledge than others.
A study of mice cannot be used to conclude whether something affects humans.
Always ask an independent scientist about the weaknesses of a study method.
Animal experiments, field studies, surveys and mathematical models.
Scientists use a host of methods, and while each has advantages and disadvantages, some provide more solid knowledge than others.
To improve your ability to quickly assess whether new research is a big story — and one worth pursuing – familiarise yourself with scientists’ methods.
Following this advice is a challenge, because there are many branches of science, and they use different methods. It’s not possible or necessary to know them all inside out.
The best way forward is: ask the authors of the research and independent scientists to comment on the study’s most significant strengths and weaknesses.
Download the guide
This article is part of the guide 11 tips for journalists: How to avoid blunders when reporting on science. The guide is accessible in three formats:
Online articles regarding each of the 11 tips.
The full guide of 11 tips as a PDF-file.
The 11 tips as a checklist, a one-pager.
Four things to consider
You can also ask yourself four questions about methodology as a litmus test of the study’s news value, whether the topic is science, social science, humanities or health science:
- How strong is the data? The more accurate and representative the data, the more solid the conclusion. Results from a study of mice, for example, cannot be assumed to be applicable to humans. Gaining the best knowledge about a disease in humans requires investigation on humans.
- How thorough and comprehensive is the study? How much data is the study based on? For example, has a water sample been taken in a single lake, or did scientists analyse hundreds of water samples from a wide selection of lakes?
- Do the scientists take into account significant sources of error? Can the scientist rule out that there could be other explanations for the results with the chosen method?
- Does anything sound completely unbelievable? For example, if an archaeologist finds a fossilised tooth and a jawbone and from that concludes that it is evidence of a new human species, ask questions that can clarify what reservations you should include in your article or feature.
Health science: use the pyramid of evidence
When you cover health science, it is easy to overinterpret what a study can demonstrate. Here, you can use the evidence pyramid to get an idea of how much you can extract from the research.
Almost all health research can be placed in the pyramid of evidence. The higher up in the pyramid a study is located, the more solid the scientists’ conclusion typically is – and vice versa.
Therefore, be careful with sharp angles and leads when telling about a study that is at the bottom of the evidence hierarchy.
Read more tips by clicking the blinking icons at the left in the graphic below.
