Automatically detect potential nodules in radiographic images (PET and CT), segment them and classify them as benign/malignant. For the latter, also classify them according to the T parameter (TNM classification).

Starting point

The identification of potential nodules within pulmonary images was carried out directly by an expert radiologist who was responsible for manually outlining the identified lesions. Expert users also classified them into benign/malignant and eventually according to the T parameter, which is essential to planning the patient’s treatment and evaluating the effectiveness of the cure.

The workload that radiologists were subjected to each day made it necessary to adopt decision-making systems based on AI. These systems help professionals in their everyday clinical practice, independently managing standard cases and notifying the doctor in case of dubious or complex cases.

Solution implemented

The system developed is made up of different parts that constitute an integrated solution for the management of radiological images and mathematical model outputs, with important operational repercussions in both the dataset creation/artificial neural network training phase, and during everyday clinical activities. Two plugins were developed for Slicer (one of the most popular open-source programs for the scientific display and analysis of medical images), allowing operators to:

  • anonymize DICOM images acquired in the clinic (complying with recent privacy laws) and send them directly to a specific picture archiving and communication system (PACS);
  • annotate DICOM images with the necessary information to train neural networks;
  • display neural network outputs.

Through, it was possible to automate the entire process, from image anonymization to annotation and model inference, creating an efficient, monitorable data flow, all while taking the time restraints of daily clinical practice and the operational needs of the professionals involved into account.
Two mathematical models were created which work in a sequence:

  • the first neural network (U-net) identifies and segments the potential nodules identified in the radiographic images and generates a benign/malignant ranking, assigning a probability to its assessment. The model was trained on sample data from 805 DICOM series coming from the publicly available LUNA database and 1191 series provided by the client. During the network training phase, its outputs were examined by a professional radiologist that either confirmed or rejected the assessments, thereby constructing a solid database.
  • The second neural network (Prototypical Neural Network) takes the malignant nodules and classifies them according to the T parameter, generating probability evaluation. The model was trained on sample data from 586 PET-CT images from patients with diagnosed pulmonary lesions provided by the client.

The system is being tested at the client’s facilities.

System reliability: 87.5% for the task of identification and classification as benign/malignant, 75% for classification according to the T parameter.

Current developments

The system will become part of daily clinical diagnoses as a decision-making support system for radiologists, generating daily reports about the cases analyzed so that the operator can select cases to review.

Through the deployed system, it will also be possible to plan the re-training of the models based on the new data available and new annotations provided about uncertain or more complex cases. This will trigger a virtuous cycle that will lead to the continuous improvement of the models’ performance.