LUMS Faculty Lead Study on AI-Assisted Diagnosis Published in Nature Health
A study led by Professor Ihsan Ayyub Qazi, Professor of Computer Science at the Syed Babar Ali School of Science and Engineering (SBASSE), LUMS, has been published in Nature Health, one of the world's leading medical and health sciences research journals. The research was co-authored by Dr. Ayesha Ali, Associate Professor (tenure-track) of Economics, MGSHSS, LUMS, and Dr. Muhammad Hamad Alizai, Associate Professor of Computer Science, SBASSE, LUMS.
The research examined whether AI tools could help physicians in Pakistan make more accurate diagnoses. Pakistan's healthcare system faces a shortage of medical specialists and high patient loads, conditions that contribute to diagnostic errors.
In a randomised controlled trial involving 58 licensed physicians, Professor Qazi's team found that physicians who used GPT-4o as a diagnostic aid achieved a mean diagnostic reasoning score of 71%, compared to 43% for those using conventional online resources. All participating physicians completed 20 hours of structured training in how to use AI tools effectively, including how to recognise when the AI gets things wrong.
A secondary analysis found that AI used alone outscored physicians using it as an aid. However, in 31% of cases, physicians outperformed the AI. "It turned out that these cases involved red flags, contextual factors, which the AI seems to have missed," said Professor Qazi.
Professor Qazi notes that reliance on AI could also lead physicians to accept flawed outputs without questioning them. The study suggests that access to AI tools alone is not sufficient. Training in how to critically evaluate AI outputs is essential for safe and effective adoption.
Professor Qazi expects the findings to be applicable to other countries facing similar healthcare constraints, though replication with other AI models is still needed. "This work opens up new avenues that can eventually lead to more safe and effective integration of AI and health care," he said.
