SCI-ART LAB

Science, Art, Litt, Science based Art & Science Communication

Some Qs people asked me on science and my replies to them - part 139 - Scientific Evidence

Q: Dr. Krishna, can you please explain what genuine scientific evidence is?

Krishna: Sure.

Understanding where true scientific evidence comes from, and what it means, is imperative to helping us tackle the most important issues affecting our own lives and the world we live in.
In science, evidence has utmost importance. Unless you provide evidence for your theory, your argument remains inconclusive and unattended. Majority of the people don't accept it. Evidence is what differentiates science from nonsense and is a part of scientific method. 

So, if I ask for evidence in a lab, can you bring someone from outside who can vouch for what you say and say that is evidence? NO! We cannot trust such evidences.

First let us understand what is not evidence

  • An argument is not evidence. Lots of people get this wrong.
  • Anecdotes are not evidence. “My aunt got acupuncture and her gout went away”? Not evidence.
  • Emotional manipulation is not evidence. “Mercury is poison! THEY WANT TO INJECT POISON INTO YOUR CHILDREN!!!!!!!” is emotional manipulation, not evidence.
  • Conspiracy theories are not evidence. (I kind of feel like this really ought not need to be said, yet here we are.)
  • Common sense is not evidence. Common sense is what tells us the world is flat, after all. The universe is under no obligation to make sense to humans.

Scientific evidence is information gathered from systematic scientific research, which takes a lot of time and patience to conduct. Evidence in general means information, facts or data supporting (or contradicting) a claim, assumption or hypothesis .

But evidence has several levels and where your evidence stands dictates the authenticity of your theory.

Evidence can range from non existent, weak, equivocal, consistent with, good, strong or compelling. Compelling evidence corresponds to the confirmation of a thing that is ‘satisfied beyond all reasonable doubt’ . 

Now Scientific evidence differs from evidence presented in courts.  When referring to "evidence from the scientific literature” we mean the empirical studies published in high quality peer-reviewed-journals. And meta analysis or systemic review is the highest form of evidence scientists accept.

Image source: Science Media center - NZ

The lowest form of evidence is anecdotal evidence like the ones accepted in courts (you can buy evidence if you want here), business (oh, so many celebrities in ads support so many products!) and in religion (this God really cured my disease when I prayed to him).  Also like when people tell their stories based on their perceptions and not on reality. For instance, some quack or a God man gives a person a ' herbal medicine'. It might just be a placebo effect, but the person might feel better after taking it. Or his faith (or fear) in his 'guru' might make the person think the 'medicine' worked.  What is the evidence that the 'medicine' actually worked? Unless you test the ingredients of the medicine in a genuine lab, its effects on  a living system by taking  large samples and avoiding all the pitfalls, fallacies, biases ( double blind trials), and confirm the results over and over again (reproduction of the results), you cannot come to a conclusion. Mere perception based on speculation is not evidence! Because it might not be true in reality.

You can watch  in movie and TV serial stories how villains manipulate the evidence. That can happen in anecdotal evidence. People can twist, spin and say things that suit them. You can't trust this anecdotal evidence.  

Again opinions of even experts don't count much. An expert opinion on a particular topic is considered to be at the same level as anecdotal evidence. Of course, if references to other, more rigorous scientific studies are provided as part of the opinion, it can help, but it’s still best to go to the source of the evidence in these cases. 

If a research paper says a substance 'has the potential to effect microbes', it doesn't become 'cure to a disease'. It is a mere speculation and not evidence! Get that right.

And if researchers 'publish' their work in dubious journals that take money to publish rubbish or announce their results in news papers or personal websites or books, they cannot be taken as genuine evidences. 

These are a few examples of such anecdotal or false evidences even highly qualified people give: 

Cowabunga! Can Cow Therapy Cure Cancer?

cowurine.com/cancer.html

https://krishijagran.com/news/co...

https://www.deccanchronicle.com/lifestyle/health.../can-cow-urine-t....

Just ignore them. No genuine scientist accepts them as true evidence.

In the world of scientific research, the highest quality evidence  are meta reviews, which are methods to contrast and combine results from a wide swath of peer-reviewed studies which may be useful in identifying patterns, sources of disagreement and other relationships. Since meta reviews combine the results from a larger number of studies, they can be more statistically significant. However, these reviews should be made by the scientific community only for the conclusions to be accurate.

Decisions about the utility of an intervention or the validity of a hypothesis cannot be based on the results of a single study, because results typically vary from one study to the next. Rather, a mechanism is needed to synthesize data across studies. Narrative reviews had been used for this purpose, but the narrative review is largely subjective (different experts can come to different conclusions) and becomes impossibly difficult when there are more than a few studies involved. Meta-analysis, by contrast, applies objective formulas (much as one would apply statistics to data within a single study), and can be used with any number of studies.

A meta-analysis is a statistical analysis that combines the results of multiple scientific studies. Meta-analysis can be performed when there are multiple scientific studies addressing the same question, with each individual study reporting measurements that are expected to have some degree of error. The aim then is to use approaches from statistics to derive a pooled estimate closest to the unknown common truth based on how this error is perceived. Existing methods for meta-analysis yield a weighted average from the results of the individual studies, and what differs is the manner in which these weights are allocated and also the manner in which the uncertainty is computed around the point estimate thus generated. In addition to providing an estimate of the unknown common truth, meta-analysis has the capacity to contrast results from different studies and identify patterns among study results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies.

A key benefit of this approach is the aggregation of information leading to a higher statistical power and more robust point estimate than is possible from the measure derived from any individual study. However, in performing a meta-analysis, an investigator must make choices which can affect the results, including deciding how to search for studies, selecting studies based on a set of objective criteria, dealing with incomplete data, analyzing the data, and accounting for or choosing not to account for publication bias.

Because large amount of studies are reviewed in meta-analysis, error margins are less,  reproducibility  of the results is covered and biases and fallacies  are surmounted to a large extent. That is why conclusions obtained from meta-analysis - when done in the right manner - is the highest form of evidence and we trust it more than anything else.  

The aim of science is to build more accurate and powerful natural explanations and use the knowledge for the benefit of the world, not to mislead and exploit the world using dubious means. Getting your evidence right is is utmost important in achieving these noble goals.

In the clinical context, these guidelines are followed:

(Source: essentialevidenceplus.com )

Levels of Evidence

Select an evidence rating scale to display detailed information.

Key to interpretation of practice guidelines

Agency for Healthcare Research and Quality:

A: There is good research-based evidence to support the recommendation.
B: There is fair research-based evidence to support the recommendation.
C: The recommendation is based on expert opinion and panel consensus.
X: There is evidence of harm from this intervention.

USPSTF Guide to Clinical Preventive Services:

A: There is good evidence to support the recommendation that the condition be specifically considered in a periodic health examination.
B: There is fair evidence to support the recommendation that the condition be specifically considered in a periodic health examination.
C: There is insufficient evidence to recommend for or against the inclusion of the condition in a periodic health examination, but recommendations may be made on other grounds.
D: There is fair evidence to support the recommendation that the condition be excluded from consideration in a periodic health examination.
E: There is good evidence to support the recommendation that the condition be excluded from consideration in a periodic health examination.

University of Michigan Practice Guideline:

A: Randomized controlled trials.
B: Controlled trials, no randomization.
C: Observational trials.
D: Opinion of the expert panel.

Other guidelines:

A: There is good research-based evidence to support the recommendation.
B: There is fair research-based evidence to support the recommendation.
C: The recommendation is based on expert opinion and panel consensus.
X: There is evidence that the intervention is harmful.

In clinical decision making these grades are usually used: 

(Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3124652/ )

The grading system provides an important component in evidence-based medicine and assists in clinical decision making. For example, a strong recommendation is given when there is level I evidence and consistent evidence from Level II, III and IV studies available. The grading system does not degrade lower level evidence when deciding recommendations if the results are consistent.

Grade Practice Recommendations

Grade Descriptor Qualifying Evidence Implications for Practice
A Strong recommendation Level I evidence or consistent findings from multiple studies of levels II, III, or IV Clinicians should follow a strong recommendation unless a clear and compelling rationale for an alternative approach is present
B Recommendation Levels II, III, or IV evidence and findings are generally consistent Generally, clinicians should follow a recommendation but should remain alert to new information and sensitive to patient preferences
C Option Levels II, III, or IV evidence, but findings are inconsistent Clinicians should be flexible in their decision-making regarding appropriate practice, although they may set bounds on alternatives; patient preference should have a substantial influencing role
D Option Level V evidence: little or no systematic empirical evidence Clinicians should consider all options in their decision making and be alert to new published evidence that clarifies the balance of benefit versus harm; patient preference should have a substantial influencing role

Q: What is triangulation?

Krishna: Triangulation is a method used to increase the credibility and validity of research findings. Credibility refers to trustworthiness and how believable a study is; validity is concerned with the extent to which a study accurately reflects or evaluates the concept or ideas being investigated. Triangulation, by combining theories, methods or observers in a research study, can help ensure that fundamental biases arising from the use of a single method or a single observer are overcome. Triangulation is also an effort to help explore and explain complex human behaviour using a variety of methods to offer a more balanced explanation to readers. It is a procedure that enables validation of data and can be used in both quantitative and qualitative studies.

Triangulation can enrich research as it offers a variety of datasets to explain differing aspects of a phenomenon of interest. It also helps refute where one dataset invalidates a supposition generated by another. It can assist the confirming of a hypothesis where one set of findings confirms another set. Finally, triangulation can help explain the results of a study. Central to triangulation is the notion that methods leading to the same results give more confidence in the research findings.

Four types of triangulation are proposed by Denzin: (1) data triangulation, which includes matters such as periods of time, space and people; (2) investigator triangulation, which includes the use of several researchers in a study; (3) theory triangulation, which encourages several theoretical schemes to enable interpretation of a phenomenon and (4) methodological triangulation, which promotes the use of several data collection methods such as interviews and observations.

Limitation of Triangulation: Triangulation offers richness and clarity to research studies but also has limitations. It adds to the complexity of the research making it more time-consuming. When used as a method for combining research methodologies, triangulation may not be achieved in a uniform or consistent manner. Additionally, researchers may not adequately explain their techniques for blending results. In addition, there may be times when comparison of the findings of two sources is inconsistent or conflicting. Triangulation does not always adequately mitigate problems in a chosen research methodology. The processes of triangulation are complex and require a skilled analyst. Finally, the value of triangulation may be overestimated in some studies.

Views: 886

Replies to This Discussion

RSS

© 2024   Created by Dr. Krishna Kumari Challa.   Powered by

Badges  |  Report an Issue  |  Terms of Service