Science and Trust series - Part 3

Recently one of my uncles told me he's now as well versed in science as any graduate in science because he had been reading news paper reports on science stories regularly. I know which papers he was referring to and the stories he has been reading.

When I told him half of what he was reading could be pseudo-science or misleading stories he was shocked. Then I gave him reasons for my observations.

He asked me an important question then. "How can we identify the authentic science reports?"

This is a bit difficult to do even for people who are well qualified in science subjects. I will provide  some guidelines here to follow while identifying genuine science reports. Follow them strictly and you will become a calibre control machine as time goes by.

1. A single study by just a group of researchers need not be accurate. Personal experiences don’t count for the purposes of scientific research. Self-reports are notoriously unreliable. A single person’s testimony is not useful for scientific knowledge, which requires reproducibility. Only when several studies by people in well established research institutes and universities confirm the reports, then they will become facts of science.

Scientific claims gain credibility by accumulating evidence from multiple experiments, and a single study cannot provide conclusive evidence for or against a claim. Equally, a single replication cannot make a definitive statement about the original finding. However, the new evidence provided by a replication can increase or decrease confidence in the reproducibility of the original finding.

2. Journalists, even those who specialize in science reporting, get things wrong much more frequently than one might think. So if a news source reports a science finding, it can be worth checking the journal article they are basing the report on because there are some untrustworthy science journals too. Moreover, the journalists need not be neutral. They are sometimes associated with vested interests. Therefore, they might try to 'influence' people with pseudo-scientific stories. Some care more about traffic than truth and transparency.

References to original research papers published in peer reviewed high quality journals like Science, Nature, e-life, Cell, PLOS make an article reasonably authentic. There are second- or third-tier journals which are very legitimate but maybe not as well known. The legitimate journals in several countries have the recognition from scientific bodies of the corresponding regions. Try to get information on them. If the writer or journalist quotes several scientists who are specialists on the subject as they are without twisting or spinning their words, you could trust him/her.

3. There are several dubious science journals and publishers which take money and publish rubbish.

Here  you will find lists of some of them.  The predatory journals are hiding among the herd of  legitimate but more obscure journals. Sometimes the editorial board and publishers of old and reliable journals get changed and taken over by people who rely on the name of the journals but don't maintain high standards of the earlier days. Be cautious about these journals.

Some sites on the net are 'science' oriented but run by journalists who don't have much knowledge on science. They select and publish sensational stuff. Just because you see the word 'science' in their domain names, you need not trust them. Check the sources from which they are getting their information. If the sources are high quality and legitimate science journals, then only you can think the stories they are publishing are reliable.

One can also check where the funding for the research is coming from. If it is coming from commercial establishments (1), political parties, religious groups or activists related to various NGOs, take the 'conclusions' of the studies with a pinch of salt.

4. Books and blogs are problematic because you can’t tell for sure whether they are based on authentic data or not unless the authors give citations. People can publish anything they want in them. One has to be cautious when someone cites a book or a blog as their source for an article.

5. If journals and magazines don't take the responsibility of verifying the authenticity of reports in articles and blogs they publish and say 'the views expressed in the blog/article are those of the author and are not necessarily those of our journal', then you can have your own reservations about trusting those views.

6. News paper/magazine science research reports usually quote news agencies which need not be authentic. News agencies sometimes pick up the stories from press releases of research institutes/universities (these could be single-study conclusions, hyped-up results/solutions and not yet published papers). These research results might get disproved in subsequent studies. Be cautious about these reports. Moreover, journalists very often choose sensational stuff over really useful ones.

7. There will be hundreds of research claims each year and we know we cannot either agree with them or dismiss all of them until they are peer-reviewed and the results are replicated. It takes lots of effort and time to follow so many cases and test their validities. So reason first. Discuss with specialists and ask for their advice. Experts see a lot of things like the scientific methods followed by the researchers to decide whether they are genuine or not. Reject the ones that sound silly. Try to test some. Trust the ones that stand the test of time and tough scrutiny - not anything that comes under the headings 'science' or 'research' or 'study'.

8. If the articles are written by scientists or experts in the field themselves giving genuine citations, you can accept them with reasonable trust.

9. If celebrities, journalists and people who don't have adequate first hand knowledge on the subject or people with vested interests give 'advices' on nutrition or health issues without proper citations, you need not take them seriously.

10. Science accepts challenges and look for evidence that might prove it false. If anybody or any claim refuses to accept challenges, then you can consider it as pseudo-science and hit 'ignore' button.

11. Studying, understanding the data, neutrally and critically thinking about what you come across, tallying it with natural laws, then accepting what makes sense is what makes you a real person of science. This is what intellectuals usually do and it is better to follow them.

12. Some science literacy is a must to distinguish between scientific and pseudo-scientific or fraud articles. If you are interested in the subject, it is better to develop  a little scientific knowledge along with learning a few terms of science.

13. Scientific knowledge is progressive, building upon a history of tested and accepted or rejected ideas and experiments. Generally, the most recent information is the most useful because of this process. With pseudo-science, however, the opposite is true: the oldest findings are considered the most valuable, and “lost” knowledge is revered more than research published yesterday. Ancient knowledge was built on the conditions and information processing available then which was very limited both in terms of quantity and quality. It was untested 'scientifically' too. Therefore, it need not be always correct.

14. Just because the word "Shastra" ( or 'sastra' roughly means science - Shastra has a similar meaning to English -logy, e.g. ecology, psychology, meaning scientific and basic knowledge on particular subject but is based on old concepts/ideas/doctrines/theories that are not proven according to present day standards of science) is added after  a word or a field, it need not be science. To be qualified as science in modern terms, the scientific methods and methodology have to be followed strictly in arriving at a conclusion.

15. If an author of a research paper 'speculates' a few things based on preliminary information, instead of showing full evidence and conclusively proves a thing, you can assume that the research work is still at preliminary stages and a lot of work still needs to be done to become a certain fact. Try to distinguish each one of these from others: speculation, hypothesis and accepted theories.

16. If a  write up is based on opinions instead of facts of science, even if it is built on experts' 'beliefs', be very skeptical about it.

17. Very emotional appeals need not be right. Emotions are not equal to facts. Just because a mother cries and tells that her child's autism is caused by vaccines given to him or her, don't trust her without checking the facts first. 

18. Just because some people use all scientific jargon to explain things, what they say need not be facts! People who propagate pseudo-science usually do this to trick man on the street.  What they say might contain some facts, but their claim might not be completely accurate because they falsely connect things that are unrelated. 

19. Good English or good any language is not equal to good science. 

20. Good presentation, either while giving speeches or while writing, is not equal to good science. I  read some pseudo-scientific and anti-science articles so well presented that people were easily misled into thinking that it was genuine science. Don't get cheated by just looks or cosmetic-coated talks or articles.

21. If yous till find it difficult to figure out whether the message you got  is right or not, depending on the nature of the claim, you should check whether it’s also being reported in other (several) media outlets. Chances are that if it really is a “breakthrough” discovery then many other outlets will be reporting the same thing. If it’s a lone WhatsApp message with a certain claim with no evidence, then you should think twice.

22. Observe who is backing up the research: reports of new scientific research should include comments from the study authors as well as an independent comment from someone in a related discipline who was not involved in the work. That’s not always the case though, and just because someone is a scientist doesn’t mean that they’re qualified to comment on the work as they may not have had training or experience in that specific topic area. A physicist can't comment accurately on Biology.

23. Also, find out whether the person reporting  or commenting is affiliated with an organisation that could be swaying them to comment one way or another?

24. To sum up, ask yourself: Who is the person publishing this information? Is this person reliable? What else has this person posted? Does the person publishing this information have something to gain? Are the claims in the post being backed up by reputable sources?

25. Be wary of viral information. Disinformation preys on our biases and our behaviour to gain traction, especially in a crisis. We are much more likely to share something that fits our world view.

26. So check your bias and keep it in control using Critical Thinking and Scientific Thinking.

27. Disinformation needs a crowd—the bigger, the better. Effective campaigns thrive by recruiting unsuspecting members of the public who don't realize they are amplifying and legitimizing posts seeking to inflame tensions or disrupt  life.

28. Correlation is not causation: Just because you can see a relationship between two things doesn’t necessarily mean that one causes the other.

Even if surveys find that people who live longer drink more red wine, it doesn’t mean a daily glass will extend your life span. It could just be that red-wine drinkers are wealthier and have better health care, for instance. Look out for this error in nutrition news.

Now that you know which science stories to trust and which ones to ignore, go ahead and enhance your scientific knowledge in a genuine way.

References:

1. http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pm...

“One of the things that should always be asked about scientific evidence is, how old is it? It’s like wine.” - Naomi Oreskes

(One of the biggest reasons for concern is newness. If it’s brand spankin’ new, few have had a chance to evaluate it, and almost certainly nobody’s had a crack at replicating it).

       Picture source: compoundchem.com

Some Qs people asked me on this issue and my replies to them: