In the first two papers in this series on AI in systematic literature reviews (SLRs), we presented our methodology and results for testing the performance of AI models in title/abstract (TiAB) and full-text screening (FTS), with which we achieved high sensitivity (up to 96% for TiAB screening, and ≥99% for FTS). In this paper, we report our methodology and findings on the subsequent SLR step, initial data extraction, using the OpenAI o1-mini model. Overall, the model demonstrated consistently high accuracy and sensitivity (≥90% for most variables) across our validation datasets. These results also indicate the model is likely to perform very well in full-scale extraction (the subject of our next white paper due to be released within the next few weeks).
Link to White Paper