Eureka: Discover Human Cognition and Psychology

As a psychologist and scientist, I can testify that doing science is hard, but understanding it is even harder. This is especially the case for those who are not scientifically trained. With today’s knowledge being more accessible than ever, we have all the scientific knowledge within the palm of our hand. Unfortunately, this also poses an unprecedented risk of simply misinterpreting and misusing this knowledge, leading not only to the spread of misinformation but also causing potentially severe harm. 

Scientific literature is incredibly hard to understand because it is specialised and highly technical literature. This is not an insult to anyone’s intellectual capabilities. Just like not everyone with a driver’s licence can be an F1 pilot, not everyone can or should understand highly specialised literature. This is why educating the public on approaching scientific information and having experts and science communicators is critical. Hence, the post series.

This first post will discuss how scientific evidence is structured. In the follow-up post, we will dissect scientific articles together. And finally,  I will give you some practical tools to help you choose your sources well. Shall we being? 

The Hierarchical Structure of Scientific Evidence

One of the first lessons I teach my students is that not all evidence is equal. How and at what conclusion we will arrive depends on how strong the evidence is. 

To begin with, to be valid, a piece of scientific evidence must be falsifiable. What does this mean?

 To demarcate science from non-science, a scientific theory (or a hypothesis) must be inherently disprovable. This is also known as the falsification principle, proposed by the philosopher Karl Popper in his book “The Logic of Scientific Discovery”. Here is an example: “All swans are white” is falsifiable because it can be disproven by the empirical evidence for the existence of a non-white swan (e.g., the observation of black swans). Hence, this statement is scientific because there are actual empirical data to test this statement and prove it wrong. On the contrary (I know some will not like this example), statements such as “All even numbers can be divided by 2” are not falsifiable because they can not be tested against any empirical data – even numbers do not exist objectively. In this case, the statement is nonetheless logical and mathematically correct but not scientific. 

Once a piece of scientific evidence is valid, it can be hierarchically organised, as shown in the picture below. In science, we differentiate between primary scientific evidence sources and secondary sources. The primary sources refer to the original data and information as their name suggests. The secondary sources usually compile primary evidence in some meaningful way. 

Let’s see what each of these pieces of evidence are: 

Primary Sources: 

Expert Opinions are scientific views or comments by a group of designated experts based on a review of scientific evidence. We will see later in the series how we can find good experts. 

Observational Studies are studies where the experimenter passively observes the participants’ behaviour without any intervention or manipulation of the observed behaviours. Specially trained individuals record activities, events, or processes as precisely and completely as possible without personal interpretation.

Descriptive Studies are studies where the experimenter briefly appraises the current situation in a particular timeframe. This type of research allows for the development of further research questions but does not examine relationships between different factors.

Experimental Studies use systematic scientific investigation. This is usually based on a design to be carried out under controlled conditions. These studies are intended to test a hypothesis and establish a (causal) relationship between independent (the factors that the experimenter manipulates) and dependent variables (i.e., the measurement).

Secondary Sources: 

Review Studies usually aim to summarize the existing research evidence in a corpus of established knowledge. Review studies are invaluable to science. They often point to future research topics and establish potential biases, contradictions, etc.

Systematic Reviews and Meta-analyses often go hand-in-hand. Systematic reviews summarise the evidence concerning a precisely formulated research question. Meta-analysis is a quantitative technique of extracting all previous data regarding a specific research question and processing and analysing these data to arrive at a single measurement. Based on these results, a conclusion can be made regarding the research question (as a whole).

So, the next time you see someone referring to scientific evidence, evaluate how strong the evidence is. Do they refer to a single article or a meta-analysis? For obvious reasons, I did not include any information such as personal opinions, social media statements, Wiki pages etc. Unless they can be classified as expert opinions, they fall into the category of unsound and unscientific statements. 

Now that we know what scientific evidence is, we will see how to read one next time. If you want to miss this post, subscribe to my newsletter and follow me on my other social media platforms. 

Happy learning!

.

If you enjoy my content, I would appreciate it if you share it. For more posts like this, subscribe to my blog and follow me on my other social media platforms.