For the past number of years artificial intelligence (AI) and machine learning (ML) have been subject to almost unbridled optimism, where despite their current flaws and limitations, their role in the future of healthcare seemed guaranteed.
To date, these technologies have been used for disease forecasting, pattern recognition of disease presentations, diagnosis, cancer screening, emergency triaging, and many more applications – often outperforming clinicians. And yet, amidst the greatest public healthcare crisis in living memory, we have yet to see AI-assisted tools effectively contribute to the fight against COVID-19. What has gone wrong? Or has potential always been overstated?
Healthcare workers from John Hopkins University, in a pre-print publication, examine where, despite the opportunities to improve COVID-19 patient outcomes, AI-based clinical decision support (CDS) systems have yet to demonstrate substantial progress. The applications they consider include improved diagnosis, triaging and prognostication, personalised treatment decision support and automated monitoring tools.
The opportunities of AI
Given how rapidly evolving our knowledge of COVID-19 is, it’s crucial that any AI-based CDS system can address variability and uncertainty in clinical decision making. We have seen already in regions of low testing capacity (for instance most of the U.S.), rapid identification and isolation of positive patients proved a massive challenge.
Delays in identifying and isolating positive infections are accompanied by a higher risk of nosocomial infections, a higher environmental risk to healthcare workers, and the over-use of limited personal protective equipment. Hence the opportunity to move away from largely reactive testing and screening, towards a more proactive model where AI-driven systems help to identify aberrant clinical presentations or patterns in patient data is a priority target.
However, since we have only been acquainted with SARS-CoV-2 a mere few months, we understandably don’t have sufficiently diverse and representative data upon which to train and validate AI/ML algorithms. As stakeholders continue to accrue larger datasets over time that help them to better understand the clinical use case, there should also be a focus on the environment in which the disease is encountered. By embedding this knowledge in the workflows, the systems will be better set to rapidly identify priority clinical needs.
How do we capitalise on the data available?
Not all data is created equal. Only carefully curated datasets, with well-defined inclusion and exclusion criteria, will provide any confidence when seeking unbiased estimates of the CDS system. This manner of careful data collection is evidently not easy or particularly practical during a pandemic, and as such many recent publications have relied on ‘convenience sampling’ of data, especially those using public repositories. These manuscripts have a high risk of bias or being overly optimistic.
Aside from the challenges of sampling strategy, the highly private nature of patient medical records poses further technical problems, as fully automated anonymisation is not a straightforward process. If not fully anonymised the data must go through the Institutional Review Board (IRB) for consideration, delaying data collection. The authors explain that whilst they support such data trust reviews, the current processes may potentially impede the rapid development of AI. They continue: “To promote AI readiness, we should consider the implementation of IRB sub-committees dedicated to AI algorithm development with specialised study protocol templates that allow for rapid turnaround review of data science-related projects.”
How can we be better prepared?
In their concluding remarks, the authors assess wherein research AI has significantly contributed to COVID-19 efforts, noting the use of natural language processing for mining literature data and the application of AI in computational biology to better understand the virus protein structure.
In looking to the future Unberath et al. believe that everyone involved should seek to identify the “organizational, institutional, or regulatory hurdles that this healthcare crisis has highlighted, as well as the solution paths that emerged to bring AI-based CDS systems for COVID-19 to the bedside.”
