paul yi, um2ii, mura, american journal of roentgenology

Shifting the emphasis: Deep-learning models trained on original radiographs (left) focus on the image labels; models trained on radiographs with the label covered (right) emphasize anatomic features such as bones (red regions indicate greatest emphasis). (Courtesy: ARRS, American Journal of Roentgenology)

Artificial intelligence (AI) has potential to play a pivotal role in many areas of medicine. In particular, the use of deep learning to analyse medical images and improve the accuracy of disease diagnosis is a rapidly growing area of interest. But AI is not perfect. A new study has revealed that radiograph labels can confuse AI networks and limit their clinical utility.

The problem arises due to a phenomenon called hidden stratification, in which convolutional neural networks (CNN) trained to analyse medical images learn to classify the image based on diagnostically-irrelevant features.

For example, a neural network trained to diagnose malignant skin lesions was discovered to actually be looking for the presence of a ruler, included for scale in images of cancerous lesions. Elsewhere, CNNs trained to detect pneumothorax (collapsed lung) on chest radiographs used the presence of a chest tube as a shortcut to identify such events, resulting in missed diagnoses if no tube was present. Other confounding features include arrows on the image or radiograph labels – commonly used to identify the radiographer or distinguish right from left on an X-ray image.

In this latest study, Paul Yi, director of the University of Maryland Medical Intelligent Imaging (UM2ii) Center, and collaborators assessed how radiograph labels impact CNN training, using images from Stanford’s MURA dataset of musculoskeletal radiographs. They hypothesized that covering up such labels could help direct the CNN’s attention towards relevant anatomic features.

The researchers used 40,561 upper extremity radiographs to train three DenseNet-121 CNN classifiers to differentiate normal from abnormal images. They assessed three types of input data: original images containing both anatomy and labels; images with the labels covered by a black box; and the extracted labels alone.

For each CNN, a board-certified musculoskeletal radiologist inspected heatmaps of 500 test images to identify which features the CNNs emphasized. The researchers found that CNNs trained on the original images focused on the radiograph labels in 89% of the 500 heatmaps. When the labels were covered, in 91% of the heatmaps, the CNNs shifted the emphasis back towards anatomic features such as bones.

The team also assessed the area under the curve (AUC), a measure of how well the algorithm performs, for the three training regimes. For CNNs trained on original images, the AUC was 0.844; this increased to 0.857 for images with covered labels. A CNN trained on radiograph labels alone diagnosed abnormalities with an AUC of 0.638, a greater-than-chance accuracy. This indicates the presence of hidden stratification, with some labels more associated with abnormalities than others.

The researchers conclude that CNNs are susceptible to confounding image features and should be screened for this limitation prior to clinical deployment. “Because these labels are ubiquitous in radiographs, radiologists developing CNNs should recognize and pre-emptively address this pitfall,” they write. “Covering the labels represents one possible solution. In our study, this resulted in significantly improved model performance and orientation of attention towards the bones.”

“We are now actively working to better understand hidden stratification, both in identifying potential confounding factors for deep-learning algorithms in radiology and in developing methods to mitigate these confounders,” Yi tells Physics World. “The implications of these issues are huge for ensuring the safe and trustworthy implementation of AI and this initial study has only scratched the surface.”

TECH NEWS RELATED

MR-guided focused ultrasound delivers antibody therapy directly into the brain

World-first clinical trial: Nir Lipsman and colleagues have visually demonstrated that a cancer therapy can cross the blood–brain barrier using MRI-guided focused ultrasound. (Courtesy: Kevin Van Paassen, Sunnybrook Health Sciences Centre) For the first time ever, researchers have delivered an antibody therapy drug directly into the brain to target ...

View more: MR-guided focused ultrasound delivers antibody therapy directly into the brain

Radiation can reverse heart rhythm disorders by reprogramming damaged cardiac cells

Cardiac radiotherapy: Electrocardiograms from control (C) and irradiated (IR) mice, highlighting the PR and QRS intervals (left). PR and QRS intervals, and representative ventricular activation maps, in control and irradiated mice (right). A significantly shortened QRS interval was seen in the irradiated mice compared with controls. (Courtesy: CC BY ...

View more: Radiation can reverse heart rhythm disorders by reprogramming damaged cardiac cells

Iridium Netwerk's medical physics team redefines best practice in radiotherapy QA

The SunCHECK Quality Management Platform is delivering long-run benefits for medical physicists and – more importantly – patients at Iridium Netwerk’s clinics in Belgium Streamlined QA: the SunCHECK Quality Management Platform provides the QA “engine-room” for Iridium Netwerk’s radiotherapy physics service, with 17 medical physicists and six physics assistants working ...

View more: Iridium Netwerk's medical physics team redefines best practice in radiotherapy QA

User experience of MRIdian Linac commissioning and QA with the THALES 3D MR SCANNER

Join the audience for a live webinar at 10 a.m. GMT/11 a.m. CET on 30 November 2021 exploring the THALES 3D MR SCANNER during the commissioning of the MRIdian Linac of Viewray The market introduction of the MR-Linac technology improves the patient care via the real-time imaging of the targeted ...

View more: User experience of MRIdian Linac commissioning and QA with the THALES 3D MR SCANNER

Clinical experience on the independent dose distribution verification with RadCalc

Join the audience for a live webinar at 12 p.m. GMT/1 p.m. CET on 9 December 2021 exploring 3D dose distribution verification based on the Monte Carlo Module in RadCalc Independent 3D dose check systems play an important role in the patient-specific quality assurance. In this webinar, Dr Ruoxi Wang ...

View more: Clinical experience on the independent dose distribution verification with RadCalc

Monochromatic X-ray method promises dramatic cut in radiation dose

© AuntMinnieEurope.com A technology called monochromatic X-ray imaging could reduce radiation dose per mammogram by a factor of five to 10 times, according to a research review published in the European Journal of Radiology. Michael Fishman from Boston University and Madan Rehani from Massachusetts General Hospital proposed that contrast-enhanced ...

View more: Monochromatic X-ray method promises dramatic cut in radiation dose

ASTRO convenes in-person conference for the cancer care community

Clinicians, researchers and healthcare professionals will be able to reconnect at the annual meeting of American Society for Radiation Oncology, which will be held as an in-person event in Chicago, Illinois Back on site: The ASTRO Annual Meeting will include a three-day exhibition, where over 160 companies will present their ...

View more: ASTRO convenes in-person conference for the cancer care community

Patient alignment with lasers from LAP: why lasers are essential for the workflow in your radiation therapy

Join the audience for a live webinar at 3 p.m. GMT/10 a.m. EDT on 11 November 2021 exploring why lasers are essential for the workflow in RT There have been enormous improvements in radiation therapy over the last 20 years. MR-only workflows and the invention of MR-guided Linacs brought new ...

View more: Patient alignment with lasers from LAP: why lasers are essential for the workflow in your radiation therapy

First positronium image recorded during a PET scan

PRISMAP consortium to fast-track nuclear medicine research

Technetium-101: from impurity to multipurpose medical tool

Quantum sensors could revolutionize retinal diagnostic procedures

Proton arc therapy: do we need it; can we deliver it?

Cold atmospheric plasma eradicates residual cancer cells

Heterogeneous anthropomorphic phantoms: reimagining SBRT QA for small lung tumours

Convergent thinking shapes Clatterbridge's single-source strategy on radiotherapy QA

Introducing RadCalc 7.2 and its EPID Module

A review of AAPM task groups 218 and 219

Personalized brain stimulation could treat untreatable depression

High-resolution independent VMAT/IMRT patient QA: Clinical implementation and results

OTHER TECH NEWS

;