The Clinical Evaluation initiative is directed to  evaluation of commercially available AI solutions within AIDA’s scope of imaging diagnostics. AIDA will support care providers that wish to assess whether an existing solution is likely to perform well and provide substantial benefits if adopted. This evaluation can be a preparatory step before going into procurement or be a part of an innovation procurement process. Below you can find summaries of current projects.

Evaluation of the dosimetric accuracy of synthetic CT images of prostate patients in MR-only radiotherapy planning

Ana Rodriguez Sanchez
Region Östergötland

The project aims to validate the feasibility of using synthetic computed tomography images (sCT) of prostate patients for treatment planning in radiotherapy. These sCT images are generated from magnetic resonance (MR) images using the AI based software MRI Planner from Spectronic Medical AB. The goal in our clinic is to transition to an MR-only workflow from the current dual-modality workflow, which requires both an MR, used for target delineation due to better soft tissue contrast, and a CT scan, used for dose calculations based on the electron density information. This change would eliminate uncertainties due to the required co-registration between MR and CT images, reduce costs, save time, lower patients radiation dose, and enhance overall patient convenience.

To validate the MR-only workflow, the project focuses on assessing the dosimetric accuracy (primarily Hounsfield units estimation accuracy) of the AI based generated sCT scans by comparing dose distributions calculated from sCT and CT images, which are considered the benchmark for dose calculations. A critical aspect involves developing a methodology to effectively separate geometric differences between MR and CT scans from other variables, crucial for precisely evaluating sCT dosimetric accuracy independent of variations in patient geometry.

AI decision support for skin cancer diagnostics in primary care – A multicentre randomised controlled trial

Magnus FalkRegion Östergötland

Skin cancer may be difficult to detect, even for experienced physicians. To support the diagnostic process of skin cancers, mainly performed in primary care, an app-operated medical decision aid, called Dermalyser, has been developed, aiming to increase cancer detection accuracy and reduce number of unnecessary excisions. In a previous clinical study on 253 melanoma suspected skin lesions, the sensitivity of Dermalyser to detect invasive melanoma lesions was 100%, and 95.2% when also including pre-cancerous melanoma in situ-lesions. The corresponding specificity levels were 92.6 and 84.5%, respectively.

In an updated version, Dermalyser has been trained to also recognize other skin cancers than melanoma (Dermalyser II). The aim with the present prospective, multicentre, randomised controlled clinical trial is to evaluate the diagnostic accuracy, clinical safety, performance and benefit of Dermalyser II, when used by primary care physicians to detect all three major types of skin cancer (melanoma, squamous cell carcinoma and basal cell carcinoma). In the study, a minimum of 2000 patients assessed in primary care for skin lesions of concern will be included and randomized to either intervention (examining physician may use Dermalyser in their assessment) or control (regular assessment). At follow up, cancer detection rates are compared between groups.

AI-detection of ICH – evaluation of three commercially available algorithms

Johan Henriksson
Södersjukhuset

Intracranial hemorrhages (ICH) make up a significant portion of neurological emergencies, often requiring prompt diagnosis and intervention to improve patient outcomes. The most used tool for diagnosing acute ICH is computed tomography (CT) of the head. The timing of diagnosis depends on how quickly a head CT is completed and subsequently interpreted by a clinician. At Södersjukhuset (SÖS), this is one of the most common examinations, with approximately 20,000 conducted annually, often under stressful conditions by radiology residents.

SÖS is currently exploring artificial intelligence (AI) as a potential diagnostic aid. Recently, various commercially available AI algorithms have emerged, and in a series of studies, we intend to compare, analyze, and evaluate the effects of different vendors’ algorithms for the detection of intracranial hemorrhage. The aim is to improve working conditions, reduce lead times, and enhance diagnostic quality. The overall objective is to investigate whether the radiology department, with the help of AI support, performs better compared to without AI support.

AI detection of vertebral fractures in osteoporosis

Anna Spångéus
Region Östergötland

Skeletal failure (osteoporosis) is heavily underdiagnosed and undertreated, where only a minority of patients get fracture preventive treatments. This despite of osteoporotic fractures being common, costly, and associated with high morbidity and mortality.  The gap between disease prevalence and the inability of healthcare providers to identify and treat these patients are a top priority in several national guidelines. One challenge, in Sweden and worldwide, is a failure to diagnose and react on osteoporotic vertebral fractures in CT scans (thorax/abdomen) made for non-skeletal reasons (opportunistic screening). Automatic detection of vertebral fractures and bone quality markers, such as volume bone mineral density on routine CT scans could have a crucial role in improving the care for this patient group. In this clinical project we aim to develop a validation platform for evaluating emerging AI tools for detection of vertebral fractures and bone quality markers.

Evaluation of a portfolio of radiology AI applications in the clinical workflow

Fredrik Wimmercranz
Capio Sankt Göran Hospital

Our project will evaluate the use of AI tools in our clinical practice at the Department of  Radiology at Capio Sankt Göran Hospital in Stockholm, Sweden. We think that AI tools and technology will come to play an important role in the progress of radiology in the near future and with this project we hope to improve our diagnostic workflow and thereby also improve patient care.  The tools that will be evaluated are part of a portfolio from a single vendor, including applications for intracranial hemorrhage, incidental pulmonary embolism, pulmonary embolism, C-spine fractures, and large vessel occlusion. We will evaluate the usefulness of the AI tools in regards to changes in reading time, reporting time and findings, and we will also qualitatively investigate user satisfaction.

Evaluation of AI-based clinical decision support for melanoma diagnosis

Magnus Falk
Region Östergötland

The project will evaluate Dermalyser, a mobile application empowered with AI that can provide diagnostic decision support for medical professionals when diagnosing skin cancer in clinical settings. Dermalyser enables fast and direct diagnostic decision support with high accuracy when diagnosing skin cancers such as malignant melanoma. The Medical professional takes an image of the patient’s skin lesion and, within a few seconds, receives an analysis from the artificial intelligence. A positive evaluation would mean that fewer cancers are missed, more patients’ lives can be saved, and at the same time, fewer patients may need to undergo unnecessary excisions of benign skin lesions. The application is used with a dermatoscope mounted in front of the smartphone camera. The artificial intelligence is developed based on patient data combined with quality-controlled dermatoscopic medical images of skin lesions.

Evaluation of deep learning based MR image reconstruction and its benefits for MRI-only prostate radiotherapy

Christian Jamtheim Gustafsson
Region Skåne

Magnetic resonance imaging (MRI) has a superior soft tissue image contrast compared to CT and MRI can alone enable planning and delivery of improved radiation therapy to prostate cancer, a novel workflow used in the clinical workflow at Skåne University Hospital, Lund. This is possible through the AI based software MriPlanner from Spectronic Medical which can convert MRI images to synthetic CT images. The drawback of MRI is the substantially increased acquisition time compared to CT to achieve sufficient image signal to noise (SNR). Novel commercial solutions enable deep learning-based MR image reconstruction and demonstrate a decrease in image noise, increased SNR and reduction of subsampling image artifacts. This enables reduction of the MRI scan time with up to 50% (guideline value) for equivalent image quality. As a result, patient examinations can be performed faster with less patient motion artifacts and higher clinical throughput. This project aims to evaluate the clinical benefits for the patient and hospital when deep learning-based MR image reconstruction (GE AIR Recon DL) is utilized and to investigate this AI product compatibility with MriPlanner. The technical limits of deep learning-based MRI image reconstruction for the purpose of improved radiotherapy imaging will also be investigated.

Evaluation of lung nodules and lung parenchymal pattern om CT of the thorax

Henriettæ Ståhlbrandt
Region Jönköping county

Yearly in Region Jönköping county, around 16,000 CT of the thorax are performed (excluding CT of the pulmonary artieries). Identifying and quantifying lung nodules and lung parenchymal pattern is a vital part of interpreting the studies. AI-backed solution CFS SEARCH Lung CT detect, localize and quantifities nodule detection pattern quantification and in lungs, that can be used to process large numbers of lung CTs fully automatically returning quantitative values on disease patterns and nodules. The main purpose is to evaluate contextflow’s LungSearch AI solution in a clinical workflow. Does the AI solution return plausible data? How much is it used in clinical practice by our radiologists? Does it shorten reading times? Does it improve quality, both in the report as well as for our referrers? The aim is to show if the proposed AI solution improves quality, or shortens reading time. The project is initiated and coordinated by the AI process group at the Radiology department at Region Jönköping county, by Dr Henriettæ Ståhlbrandt, together with the company ContextFlow.

Evaluation of AI-based vendor solutions for automatic organ-at-risk and target segmentation for head-and-neck, brain and prostate radiation therapy treatment planning

Christian Jamtheim Gustafsson
Lund University

In radiation therapy, a linear accelerator is used to irradiate target volumes, while sparing surrounding healthy tissue, called organs-at-risk (OAR). It is desirable to maximize the radiation dose to the target, and minimize the OAR dose. One crucial step in planning such treatments is CT/MR imaging and segmentation of target and OAR volumes.

With an overall increase in cancer and an ageing population, a heavier staff workload is observed as the segmentation of target volumes or OAR can be very time-consuming. Furthermore, the anatomical segmentation is mostly performed manually, leading to variation in the segmentations which in turn affects the treatment quality.

Deep learning based segmentation has shown great promise on medical images and has matured into commercially available products. Such products can reach the accuracy of manual segmentations. It is therefore of interest to evaluate such products to approach the above mentioned issues and improve patient treatments.

The project aims to quantitatively evaluate two commercial products for AI based automatic organ-at-risk and/or target segmentation for head-and-neck, brain and prostate cancer radiation therapy treatment planning. Further, the impact on clinical workflow efficiency and productivity will be investigated.

The use of artificial intelligence (AI) to safely reduce the workload of breast cancer screening in a clinical workflow

Håkan Gustafsson
Region Östergötland

The annual breast cancer screening volume in Östergötland County invites roughly 44,000 women. For each woman, each exam is double read independently by at least two breast radiologists. In the case of disagreement, two experienced radiologists discuss the exam during a consensus conference.

Transpara is an AI-driven vendor-neutral system designed for automated breast cancer detection in digital mammography (DM) and breast tomosynthesis (DBT). Transpara has been demonstrated to achieve a breast cancer detection performance comparable to an average breast radiologist  in several retrospective studies elsewhere. Although promising, further assessment  of the performance and fidelity needs to be done.

This study will clinically evaluate the performance of Transpara in the annual breast cancer screening in Östergötland County. The standard reading procedure will continue, but Transpara will act as a 3rd reader and will recall additional cases to the consensus conference. The number of additional cases recalled by Transpara to consensus conference will be measured as well as the positive predictive value (PPV) and changes in recall rate.

The overall aim of the study is to evaluate if Transpara can increase the quality of breast cancer screening in Östergötland County and at the same time reduce the workload of the breast radiologists.

Automated blood volume measurement on head CT

Johan Wasselius
Lund University

Our project addresses an unmet clinical need of exact Intracerebral Hemorrhage (ICH) Volume  measurement by a novel technology developed for modern imaging and fully automated transfer of validated volumetric data all the way to the national quality registry.

ICH accounts for 13% of first-ever strokes, with significantly worse outcome compared to ischemic stroke. ICH volume is the most important prognostic factor for clinical outcome but is rarely measured or reported. The main reason is lack of an efficient tool to make exact volumetric measurements.

We will validate the ICH-volume measurement tool in a large population from Sweden’s quality register for stroke care, ”Riksstroke”. With excellent coverage (>90%) and registration of both clinical variables and outcome Riksstroke is ideal for large scale validation of ICH volume measurement. We plan for a validation of automated ICH volume measurements to both semi-automated, and manual volume measurements done by two independent readers.

We will also establish a new information transfer-route for volumetric data via validation by the radiological examination to the electronic patient record and quality registry to facilitate large scale use.

The project is initiated and coordinated by the Stroke Imaging Research Group at Lund University by Dr Teresa Ullberg and Dr Johan Wasselius, together with the companies Qure.ai and Sectra.