Five things we learned about scoping reviews from the recent Covidence webinar

If you’ve ever felt overwhelmed trying to make sense of the ins and outs of scoping reviews, you’re not alone. In a recent webinar, the Covidence team was joined by Dr Danielle Pollock, who helped us to demystify scoping reviews, from planning to presenting results. Here are the five key takeaways from that session and some practical tips for your next scoping review.

1. Understanding grey literature

The first major point that came up, and one that confuses many researchers, is grey literature. Danielle explained that grey literature is information not controlled by commercial publishing. Think conference abstracts, theses, government reports, clinical practice guidelines, blogs, and even discussion posts. It’s the stuff you might never find in PubMed or your usual academic databases but sometimes it’s exactly where your answers lie.

Grey literature can be especially useful for scoping reviews because these reviews aim to map all relevant evidence, not just what’s peer-reviewed. For example, Danielle mentioned a scoping review on hospital accreditation and midwifery care where the only relevant sources were policy documents on government websites. Similarly, another review explored guidelines for storing human breast milk at home. 

However, Grey literature can be messy, hard to track, and time-consuming. Danielle suggested asking yourself a few questions before diving in:

• Do I actually need grey literature for this review?
• What specific data am I hoping to find, and will it answer my questions?
• Do I have the skills, time, and tools to search for it properly?

Pro tips from the webinar:

• Plan your grey literature searches during the protocol stage – decide which websites and resources you’ll check. 
• Set boundaries – for Google searches, for example, you could limit yourself to the first 10 pages. Use incognito mode to prevent past searches from biasing results. 
• Document everything. Transparency is key, so future researchers can see exactly how you did your search.
  

In short, grey literature is powerful if used wisely.

“Don’t let the fear of missing something derail your review.”

2. Data extraction tips: collect what matters

Next up, we tackled data extraction, also called data charting. This is arguably one of the trickiest parts of a scoping review because, unlike systematic reviews, scoping reviews can include a huge variety of evidence types – everything from randomized controlled trials to blog posts.

The golden rule is to extract only what’s relevant to your review questions. Danielle says ask yourself, “What will I actually do with this?” And if the answer isn’t clear, skip it.

Practical tips from the webinar:

• Use multiple reviewers – humans make mistakes, and having a second pair of eyes reduces bias. 
• Create customized extraction forms – standardized tools often don’t fit the variety of evidence in a scoping review. Covidence is a favorite tool, letting you design forms from scratch. 
• Pilot your forms – test them on a few articles first to make sure they capture what you need. This is also known as calibration in scoping reviews 
• It may look like organized chaos – when you export your data. That’s okay! Excel can help you sort it out before analysis. 

When it comes to qualitative or textual evidence, extraction can also be the first step in your analysis. For example, if you’re mapping barriers and facilitators in healthcare interventions, you might start by coding each barrier or facilitator as it appears. Later, you can group similar items into categories or frameworks. This approach allows you to stay systematic without over-interpreting the data.

One last note: p-values, standard deviations, and other statistical analyses or inferences are generally not relevant for scoping reviews. If your goal is meta-analysis or hypothesis testing, that’s a different type of review altogether.

3. Presenting data creatively: beyond tables

Scoping reviews can generate a lot of data, and presenting it effectively is a challenge. Danielle encouraged moving beyond “death by table” and embracing creative ways to display results. While tables are fine, adding visual elements makes your review far more digestible for readers.

Presentation tips and tools:

• Evidence and gap maps – great for showing where research exists (or doesn’t) across different contexts or populations. 
• Word clouds – useful for capturing large volumes of textual data – Danielle mentioned one where over a thousand terms describing evidence synthesis were visualized in a single image. 
• Geographic maps – show where evidence comes from, highlighting under-researched regions. 
• Bar charts, SWOT diagrams, and frequency counts – summarize key findings like barriers, facilitators, or methodological trends. 

Tools for creating visuals:

• DataWrapper – user-friendly and quick for charts and maps. 
• Power BI and Tableau – more advanced, but offer highly customizable visuals. 
• Excel – still versatile for basic visualization. 

The overarching advice: be creative, but always provide a narrative summary.

“The visuals should complement your story, not replace it.”

4. Protocols and planning: avoid common mistakes

One thing that came up again and again in the webinar was the importance of planning ahead. Danielle emphasized that scoping reviews are only as good as the thought you put into your protocol. This includes your research question, search strategy, data extraction plan, and even how you’ll present your results.

Key takeaways on protocols:

• Registration – unlike systematic reviews, there isn’t a formal registry like PROSPERO for scoping reviews. Instead, post your protocol on Open Science Framework (OSF) or FigShare for transparency. 
• Deviations to the protocol – sometimes changes are necessary (e.g., realizing you need to extract additional data). That’s fine, but document them clearly in your methods. 
• Frameworks – using the PCC framework (Population, Concept, Context) helps structure your research question and guides search and extraction strategies. You don’t always need all three elements – it depends on your question. 

Common mistakes:

• Confusing reporting guidelines (PRISMA-ScR) with conduct guidelines (JBI) – PRISMA tells you how to report; JBI tells you how to do the review. 
• Doing too much high-level  analysis that’s beyond the scope of a scoping review. 
• Not being transparent about methods, which can undermine credibility. 

The overarching advice? Take your time upfront. A well-planned review saves time, reduces frustration, and makes your results more reliable.

5. Flexibility, AI, and keeping reviews up to date

Finally, the webinar addressed some of the broader questions about scoping reviews, including volume of evidence, AI, and updates. Danielle’s answers highlight that scoping reviews are flexible by design.

Volume of evidence:

• There’s no minimum or maximum number of papers. You could end up with an “empty” review, which can still be valuable to highlight gaps in evidence. 
• Large volumes may require adjusting your methods – maybe one person extracts while another checks, or you bring in extra team members. 

Levels of evidence:

• In scoping reviews, all study designs are considered equally. Unlike systematic reviews, you don’t prioritize RCTs over observational studies unless your question specifically requires it. 

AI in scoping reviews:

• Danielle is a fan of AI for efficiency but cautions that it’s mostly trained for systematic reviews of interventions, not broad scoping review questions. 
• Tools like Covidence have features that can prioritize screening decisions, making your workflow more efficient. Full AI integration for broad scoping reviews is still emerging. 

Updating scoping reviews:

• There isn’t a set schedule. Update when new evidence is likely to change your results – especially important in fast-moving fields. 
• The concept of “living scoping reviews” is gaining traction, similar to living systematic reviews and living guidelines. 

Interest holder consultation:

• Optional, but highly valuable. Engaging patients, policymakers, or other interest holders can strengthen your review and provide real-world perspectives. 
• The level of engagement can range from simple consultation to co-researcher involvement. Plan carefully and ensure resources and support are in place. 

Wrapping up 

Scoping reviews can feel like an enormous undertaking, but with clear planning, thoughtful extraction, and a creative approach to presentation, they are incredibly powerful tools for mapping evidence. Key messages from this webinar include:

1. Use grey literature wisely – only include it if it’s relevant and plan your search carefully. 
2. Extract what matters – forget unnecessary numbers; focus on answering your research questions. 
3. Be creative in presenting data – tables are fine, but visualizations and narratives make your findings come alive. 
4. Plan and document everything – protocols, PCC frameworks, and transparency are your best friends. 
5. Stay flexible – handle large volumes, include all study types, explore AI tools cautiously, and update when necessary. 

Scoping reviews are still rigorous and transparent types of evidence synthesis, however, they explore, map, and reveal patterns in evidence. Approach them thoughtfully, document your process, and embrace the organized chaos that comes with it. The planning and clarity you bring early on will repay you later on in the process.

To learn more about Covidence, visit www.covidence.org, where you can start your review journey for free and invite collaborators from around the world to work together on your review project. Covidence enables teams to collaborate seamlessly on multiple tasks in one place, saving time and helping produce high-quality evidence faster. 

Bio:
Dr Julie Brown is a Principal Consultant Systematic Reviewer at Covidence with a background in psycho-neuroimmunology, applied psychology, and nursing. She has completed over 35 systematic reviews.