Published for Evidence-Based Healthcare Day, 20 October 2026. This year’s theme: “Evidence and AI: People at the Centre”.
Picture this. A researcher is staring at thousands of citations, each requiring a deceptively difficult decision: is this study relevant to my review? It’s painstaking work. And in many cases, a second reviewer will need to check those decisions too. This is exactly the kind of workload where AI and automation can help. It can sort, prioritise, extract and, in some circumstances, make decisions automatically. But that raises a more important question: how much of the decision-making should we actually hand over to AI?
For Covidence, the answer depends on two things: what happens if the AI gets it wrong, and how much human oversight remains. The greater the potential impact of an error, and the more a task depends on interpretation, context and judgement, the higher the bar should be for automation. That means we don’t apply the same level of automation everywhere in Covidence. Sometimes automation can make a decision. Sometimes it should only change what a researcher sees first. And sometimes its role should simply be to make a suggestion. Every automation feature we release must meet performance requirements appropriate to the impact of error and the level of human oversight built into the workflow.
When AI makes the decision: the RCT classifier
The RCT classifier is the most automated of these examples.
Developed by Cochrane and integrated into Covidence, it identifies records that are very unlikely to be randomised controlled trials. When a review team chooses to use it, those records can be excluded automatically, with an exclusion reason recorded. That’s a significant decision to hand to automation, so the performance bar is high. The classifier has been evaluated to achieve more than 99% recall when identifying randomised controlled trials. In simple terms, it is designed to be highly sensitive, minimising the risk that a randomised controlled trial is incorrectly excluded. Human control is also built into the workflow. The review team decides whether to use the classifier. Records where there is insufficient certainty, including those with limited information, continue to manual screening. And every automated exclusion remains visible and reversible, so researchers can inspect the decision, restore a record to screening and report the classifier’s use in their PRISMA flow diagram.
Here, AI can make a decision, but researchers retain visibility and control.
When AI prioritises: most relevant sorting
Most relevant sorting takes a different approach. It uses active learning to reorder references based on the screening decisions the review team has already made. As researchers screen, the system learns from those decisions and brings potentially relevant records closer to the top of the list. But importantly, sorting changes the order, not the outcome. A study appearing at the bottom of the list isn’t automatically excluded or ignored. Every remaining record still receives a decision from the review team. Here, AI helps researchers focus their attention. It doesn’t decide what belongs in the review.
When AI suggests: extraction and intervention suggestions
Further along the review, AI can play an even more supportive role. Extraction and intervention suggestions look through the full text of uploaded papers and suggest information for data tables, with a supporting quote linked back to the relevant place in the PDF. The aim is to reduce repetitive work and give researchers a useful starting point. But a suggestion isn’t an answer. Researchers actively accept or reject each suggestion. They check the source information, decide whether the suggestion is correct and make the final decision about what belongs in their review.
So, how much should you hand over?
Not the whole review. But that doesn’t mean researchers need to choose between doing everything manually and handing everything to AI. Well-validated automation can take on parts of the work it can perform reliably, particularly when researchers can see what it has done and retain appropriate control. Other tasks are better suited to prioritisation or decision support, where AI reduces effort while leaving the judgement with the researcher. The more a decision depends on experience, context and judgement, the more firmly it belongs with people. That’s what “people at the centre” means to us. The goal of AI in evidence synthesis isn’t to remove researchers from the process. It’s to reduce unnecessary work and give them more time and better tools for the work that requires human judgement. So perhaps the most useful question isn’t whether AI can run a systematic review, it’s where can AI genuinely help, and where do people need to remain in control?
