|
Summary of progress /findings
Project aim:
Develop and validate an AI algorithm to detect scaphoid fractures on wrist and scaphoid radiographs.
Objectives:
- Develop a strategy and complete data labelling;
- Train a model/models to detect scaphoid fractures on radiographs;
- Test the model(s);
- Explore the factors influencing the uptake healthcare AI.
Progress:
Image access
I have acquired access to 40 000 XR scaphoid images and >100 000 free text radiology requests/reports associated with imaging. I am working with the Thames Valley Secure Data Environment (TVSDE) to secure access to a further 100 000 images (XR, CT and MRI).
De-identification of patient records
In collaboration with Oxford University Hospitals NHS Trust (OUHNFT), the TVSDE and Microsoft AI Research, I am working on evaluating pipelines for robust methods of anonymising patient data. We have labelled 2000 scaphoid radiology reports/requests, 1000 oncological histopathology reports and 1000 patient EHR entries with patient identifiers, and I am evaluating and fine-tuning large-language models (LLMs) to redact direct identifiers, such as names, GMC numbers and hospital units.
I aim to complete and submit this work for publication by April 2024. This work will directly inform the OUHNFT protocol for large scale data extraction for research and audit.
Data labelling
I have been part of a successful nine-site UKRI Innovate UK Grant (£1.1m), titled ‘Responsible AI’, and I am the lead applicant for Oxford University (£300k). Through this grant, I have worked with developers to adapt a software to aid easy data annotation and labelling. From January 2024 – April 2024, I have recruited expert MSK radiologists to manually label 600 scaphoid XR and 3000 radiology reports with a ‘ground truth’. I have developed a bespoke natural language processing (NLP) tool to classify reports according to the presence/absence of pathologies including scaphoid fracture. I have adapted LLAMA, a locally available LLM to perform the same task. By reviewing the accuracy of expert labelling of images and test, compared to NLP and LLMs, I will establish a pipeline for processing radiology free text to aid efficient and affordable labelling across any clinical domain.
Model development
I am establishing a collaboration with UK Microsoft AI Research (MSR) to develop a model to generate radiology reports for wrist and scaphoid x-rays. We are currently drafting a research contract between Oxford University, OUHNFT and MSR to allow us to collaborate. I have performed pilot work using a dataset of 1000 paired image/reports to fine-tune a generative model provided by MSR.
Factors influencing uptake of healthcare AI
I published a meta-analysis of AI for fracture detection, in the journal Radiology (IF 11.1) in July 2022. Since then, I have completed a qualitative evidence synthesis (QES) co-produced with patient and public involvement (PPI) partners, proposing an AI-specific extension to an implementation science framework. This has been sent out for peer-review at eClinicalMedicine (IF 15.1). Together with two PPI partners I plan a further qualitative study exploring public views on health data sharing for AI.
|