Many disciplines, such as science and economics, are based on evidence.
In the past 2 decades, the term evidence-based practice has been used to refer to practice or decision making that is based on a body of evidence (e.g. scientific studies) that has been systematically collected and critically assessed. This terminology was pioneered in the fields of health and medicine, but has also been applied to the environment, agriculture, social sciences, education and economics.
Although evidence-based practice has added much rigour to the analysis of data in these fields, it has led to an overreliance on the word evidence in technical and general publications, including practice guidelines (such as clinical practice or environmental management), government reporting and the media. What constitutes evidence? What constitutes evidence that is reliable enough to guide practice? There is no clear-cut, simple definition, so use of this term can muddy the waters.
Types of evidence
Evidence can be quantitative or qualitative:
- Quantitative evidence is based on data that can be put into numbers. This includes quantitative measurements and statistics that can be analysed with respect to past outcomes, and used to build models to predict future outcomes. The data are often generated by carefully designed comparisons of outcomes in 2 or more controlled systems. This is the preferred evidence in disciplines such as science, medicine and economics. Claims made in these fields usually require meaningful quantitative supporting evidence to be credible.
- Qualitative evidence is based on data that can be put into words, such as narratives or comparisons based on observations and descriptions. This includes identifying themes across a set of events or observations, and seeing where there may be connections or commonalities. In some fields of research, such as social science, comparative quantitative studies are difficult to do, and qualitative studies such as surveys or case studies provide the most robust form of evidence.
- Qualitative examples are supporting pieces of information, such as quotes and extracts from relevant texts, experts and other research. Claims made in a humanities, arts or similar context require supporting information of this sort to be persuasive.
In some fields, mixed methods research combines elements from qualitative and quantitative methods to provide evidence. For example, a researcher might conduct a multiple-choice survey (quantitative) and then ask for further comments in free-form boxes or interview people in person about their responses (qualitative).
The table shows the types of evidence that can be considered for quantitative and qualitative research, and the supporting information that should be provided.
Examples of types of evidence and key information to include when writing about such evidence
| Methodology | Examples of evidence and supporting information for various fields |
|---|---|
| Quantitative | Primary evidence:
Secondary evidence:
Supporting information:
|
| Qualitative | Primary evidence:
Secondary evidence:
Supporting information:
|
Quality of evidence
Within each field of study, some study designs or types of information are considered to be more reliable sources of evidence than others. For example, in clinical medicine, randomised controlled trials are the gold standard because their study design inherently eliminates bias that would otherwise make the results less reliable (e.g. because of the well-known placebo effect). Such trials can be well or less well conducted. To write accurately about evidence in a specific discipline, it is important to be familiar with the hierarchy of evidence in your discipline and what defines a well-conducted study or investigation.
Be specific and accurate
Language about evidence is often vague. In the following examples, it is impossible to tell from the terms used (in italics) just what the evidence is or what makes it ‘high level’, ‘limited’ and so on, especially when the sentences are also laden with other jargon:
Despite a lack of high-level evidence in its favour, laparoscopic adrenalectomy has practically replaced open adrenalectomy in the management of benign adrenal lesions.
Although there is modest high-quality evidence of traditional ‘clinical outcomes’ from self-management programs, these programs are strongly endorsed by consumers.
In contrast to commonly held opinion, there is limited strong evidence that lifestyle factors are a dominant factor in the pathophysiology of …
To date, there is no clear evidence of an increased cancer risk in medical radiation workers exposed to current levels of radiation doses.
Some evidence suggests that changes in snowmelt may also increase the risk of forest fires.
Mounting evidence from diverse archives suggests the occurrence of an abrupt ‘cold snap’ just before the onset of the prolonged cooler late Holocene.
Scientific evidence for warming of the climate system is unequivocal.
This is less of a problem when statements about evidence occur alongside data that explain what the evidence is, or set some parameters about the intended meaning. But when such statements are used in summary information (such as an abstract, literature review, plain-English summary or media release), they can give an inaccurate or misleading message. For example, does modest high-quality mean a few (how many?) high-quality studies that showed a modest effect, or some less-than-high-quality studies that showed a large effect?
To improve the clarity of evidence statements, 4 strategies are helpful:
- Replace the word evidence with a description of the studies behind it.
- Replace imprecise quality adjectives (e.g. good, poor) with a description of the study design.
- Replace misleading quantity adjectives (e.g. no, little, some) with a description of the study results.
- Define or grade evidence against a predefined standard.
The following examples are from clinical medicine, but the same principles apply in other disciplines.
Replace the word evidence with a description of the studies behind it
To improve reader comprehension, replace the word evidence with a description of the research or studies that provide the evidence. Using adjectives such as good-quality or well-designed to describe the studies themselves has more meaning than vague terms such as moderate evidence.
Replace imprecise quality adjectives with a description of study design
Avoid vague language by cutting out as many imprecise adjectives as possible (e.g. high-level, good, consistent, reasonable, compelling, poor, insufficient). Instead, describe what you mean (e.g. X is more effective than Y). Shift the quality descriptors onto the studies or the size of the effect (e.g. well-designed randomised controlled trial, large effect):
Replace misleading quantity adjectives with a description of study results
Avoid adjectives that describe the quantity of evidence. The term no evidence is particularly misleading because it could mean many high-quality studies that show no statistically significant effect, or no studies at all, or anything in between:
- If you mean that there have not been any studies, make this clear and admit that you do not know the answer
There is no evidence that X is better/more effective/superior/more beneficial than Y. [which implies that studies have been done and they have shown no beneficial effect]
Say:
We do not know how X compares with Y because there have been no studies.
- If you mean that a lot of evidence is available, but it does not show a statistically significant effect, clearly state this
Do not say:
There is no evidence that X is effective for Z.
Say:
A large well-designed clinical trial showed no statistically significant effect of X for Z.
- If you mean that there are some studies, but they are inconsistent and do not lead to firm conclusions, clearly state this
Do not say:
There is no conclusive evidence that X is effective for Z.
Try something like:
Four small randomised controlled trials showed inconsistent results, so it is impossible to draw any conclusions about whether X is more or less effective than Y. Further well-designed trials are needed to answer this question.
Although no evidence is the most striking example of this problem, the same reasoning applies to any descriptors that refer to the amount of evidence (e.g. little evidence, some evidence, much evidence). For example, little evidence could mean 1 study that showed a large effect or several studies that showed marginal effects, and so on.
Examples of how to apply these strategies are shown in the following table.
Strategies for precise wording for evidence statements
| Type of studya | Results | Example evidence statement |
|---|---|---|
Several well-designed studies
| Positive effect (effective/beneficial) | Three well-designed clinical trials showed that X decreases [outcome] compared with Y. Three good-quality studies showed that X is effective [beneficial] for … |
| Several well-designed studies | Negative effect (harmful) | Three well-designed clinical trials showed that X increases [outcome] compared with Y. Three good-quality studies showed that X is less effective [harmful] than Y for … |
| Several well-designed studies | No significant effectb | Three well-designed clinical trials showed no significant difference between X and Y. Three good-quality studies showed no significant difference between X and Y. Do not say ‘there is no effect’ – ‘no effect’ is impossible to prove in science, and studies that do not show a statistically significant effect may have wide confidence intervals that include biologically or clinically important effects.c Do not say ‘there is no evidence’. There is evidence – it just shows no significant effect! |
| 1 well-designed study | Positive or negative effect | One well-designed trial has shown that X is more effective (or less effective) than Y … but this finding needs to be confirmed by more trials. |
| 2 or more well-designed studies with conflicting results | At least 1 study shows a positive effect; at least 1 study shows no significant effect | Two good-quality studies show conflicting [inconsistent] results for the relative effectiveness of X and Y (or ‘about whether X reduces [outcome] …’). |
| 1 or more poorly designed studies | Positive or negative effect | One small [poorly designed] clinical trial showed that X was better [worse] than Y. Larger [better-designed] trials are needed to check this result. Do not say ‘larger trials are needed to confirm this result’ because it is not known whether the finding will be confirmed or refuted until the larger trials are done. |
| 1 or more poorly designed studies | No significant effect | Several small studies have shown that X may not be effective for …, but larger [better-designed] trials are needed to further investigate this effect. There have not been enough well-designed clinical trials to either support or refute … |
| No studies | Not applicable | There have been no studies of the effect of … The effect of X … is not known. Do not say ‘there is no evidence for …’. This can be confused with a negative or not significant result. |
a The general term study refers to any research studies (e.g. clinical trials, ecological studies, other scientific research). Well-designed means
a protocol that eliminates bias (e.g. randomised allocation to groups, blinded measurement of outcomes) and with enough subjects or
measurements to allow accurate statistical analysis. Descriptors in the evidence statements will vary according to the type of research.
b The term significant is used here as shorthand for statistically significant.
c An effect can be statistically significant but not be of real biological or clinical importance (e.g. pain might increase but remain within a level
that does not affect daily life).
Define or grade evidence against a standard
To take the guesswork out of the meaning of adjectival statements about scientific evidence, some organisations have defined the adjectives they use. Use these, if appropriate in your field:
- The International Agency for Research on Cancer has defined several categories of evidence(Opens in a new tab/window) that support the carcinogenicity of agents in humans and animals.
- The National Health and Medical Research Council (in Australia) and most other major health agencies worldwide have adopted approaches to grading the evidence (Opens in a new tab/window)that health advice draws on.
- The United Nations Intergovernmental Panel on Climate Change (IPCC) has published definitions of adjectival terms relating to likelihood of an outcome.
IPCC likelihood scale
| Term | Likelihood of outcome (% probability) |
|---|---|
| Virtually certain | 99–100 |
| Very likely | 90–100 |
| Likely | 66–100 |
| About as likely as not | 33–66 |
| Unlikely | 0–33 |
| Very unlikely | 0–10 |
| Exceptionally unlikely | 0–1 |
Avoid biased language
Language about evidence is often emotive. Consciously or subconsciously, writers may use language that reveals their own beliefs or hopes. This can creep into writing in many subtle ways:
The use of the positive word benefits suggests that eating Vegemite sandwiches is beneficial. The writer thinks that, when the research is in, it will confirm their belief in the benefit. But, if there have not been any studies, we do not know if it is beneficial, neutral or harmful. This is the most common sort of bias in scientific literature. Contrast it with an unbiased version:
Another pitfall is revealed here:
The use of the positive word favour influences the meaning of the sentence – because there is ‘not enough evidence’, favour red roses could just as easily be favour white roses. It is better to say:
Another example:
Available implies that there is ‘unavailable’ evidence that might say something different. Non-existing (or potential, possible, future, further …) evidence is just that, so do not imply that it exists and might be favourable for the issue under consideration.