At the beginning of this year, SEI researcher Biljana Macura was appointed Editor in Chief of Environmental Evidence, which is the top journal in the field of evidence synthesis for environmental management. In this Q&A, she discusses the journal, the importance of systematic evidence syntheses, and how they provide reliable evidence for decision-making in policy and practice.
Tell us about the journal, what’s its focus?
Environmental Evidence is an open-access journal from Springer-Nature that facilitates the publication of evidence syntheses. Everything we publish – systematic maps and reviews, rapid reviews, and methodology papers and related submissions – helps to advance the field of evidence synthesis. Moreover, the Evidence in Action articles in the journal discuss the science-policy interface.
Systematic evidence syntheses are very important for evidence-informed decision-making. These reviews take time to develop, but you also get rigorous outputs.
What are systematic evidence syntheses?
These are gold standard methodologies for collating and synthesizing scientific evidence. They aim at comprehensiveness, transparency and repeatability at each step of the review process.
Systematic maps provide a broad overview of the evidence on a specific topic, while systematic reviews answer questions on what works, how and under which conditions.
The reviewing process starts with a review protocol – a detailed methodological plan of the review – followed by comprehensive literature searches, meticulous screening of literature based on predetermined criteria, critical appraisal of study validity, detailed data and meta-data extraction and qualitative, quantitative or mixed-method synthesis.
How can systematic reviews minimize bias?
The whole process of systematic reviewing of literature is geared towards removing sources of bias and assuring high confidence in review findings. For example, systematic reviews use predefined search criteria and conduct comprehensive searches to identify all relevant studies, minimizing the risk of selective inclusion of studies. Moreover, reviews have clear eligibility criteria to assure only relevant studies are included, further reducing a chance of biased study selection. By including unpublished data and grey literature, systematic reviews aim to minimize publication bias.
Multiple reviewers independently screen studies and code data assuring consistency in selection and data extraction. Using standardized critical appraisal tools and frameworks, systematic reviews assess the methodological rigor and validity of each included study.
Improving review reliability and minimizing sources of bias is particularly important if reviews aim to address policy- and practice-relevant questions and if we use their findings to support our policy or practice decisions. In an article in Nature Ecology & Evolution in 2020, we identified major methodological problems with literature reviews and suggested specific fixes to improve rigor and reliability.
What is evidence-informed decision-making?
It’s a process for making decisions based on best available scientific evidence. Evidence-informed decision making involves reliance on systematic evidence synthesis and systematically gathering, critically appraising, and synthesizing relevant research – and other sources of evidence.
Now, this comes with limitations. There might be limited availability of reliable scientific evidence. Real-world decisions are complex, and collected evidence might be inconclusive. Decision-making is often influenced by power dynamics and implementation has challenges. Nevertheless, systematic evidence synthesis aims to be a transparent and comprehensive source of the best available evidence.
How does SEI work with systematic reviews?
SEI embarked on its journey in evidence synthesis back in 2016 when it assumed responsibility for the Mistra EviEM program. Since the finalization of that program, SEI has broadened its scope of evidence syntheses, delving into a diverse array of topics pertinent to both the environment and development.
As part of SEI’s evidence synthesis work (gathered here), in addition to conducting reviews and mapping, SEI researchers are continuously contributing to methodological developments. Together with a group of researchers, in 2017, I co-designed widely recognized standards called ROSES Reporting, for “RepOrting standards for Systematic Evidence Syntheses”. It provides guidance for reporting of systematic reviews and maps in environmental research, contributing to repeatability and transparency of review protocols and reports.
We followed up on the importance of reporting standards for literature review in an article in Nature Climate Change. We are also currently experimenting with developing an AI-powered tool for data extraction from academic and grey literature.
SEI is also a centre of the Collaboration for Environmental Evidence, can you tell us a little more about that?
SEI is dedicated to the process of systematic reviews and hosts one of the centres behind the Collaboration for Environmental Evidence (CEE). The CEE is a network organization that supports and promotes evidence synthesis on issues of greatest concern to environmental policy and practice as a public service. The CEE establishes and develops Guidelines and Standards for Evidence Synthesis in Environmental Management. All Environmental Evidence journal submissions comply with these guidelines.
The SEI centre was established in January 2019, and I currently coordinate it. Through this collaboration, together with other CEE centres, SEI promotes evidence synthesis methodology and supports evidence-based practice and evidence-informed environmental decision-making.
Senior Research Fellow and Team Lead, SEI Headquarters
Design and development by Soapbox.