As more never-smokers develop lung cancer, researchers test an AI model to predict patient risk

Massachusetts General Hospital is launching a prospective trial of an artificial intelligence tool designed to predict patients’ risk of lung cancer, a crucial area of inquiry amid a rising incidence of the disease in never-smokers.

The trial, to begin later this year, will test the accuracy and usefulness of an AI system the hospital developed with researchers at the Massachusetts Institute of Technology’s Jameel Clinic. The tool stands out from a crowd of similar models in that it predicts a patient’s cancer risk over six years by analyzing a single low-dose CT scan — without the use of other demographic or medical information, or a radiologist’s annotation.

The goal is to help identify which patients would benefit from additional testing and closer management, filling a crucial gap in screening that allows too many tumors to grow undetected. 

“Right now we’re having a hard time getting people to come back for screening,” said Robert Smith, senior vice president for cancer screening at the American Cancer Society. 

“This tool is giving you a heads-up that this little thing you see is actually going to be a tumor,” he added. “And maybe the thing to do next is see this patient more frequently because of the high probability that you’ll be able to intervene earlier.”  

Smith, who was not involved in the research, said it will take significant work to ensure the AI tool can be helpful in live clinical settings. But he noted that early data on the tool, published this month in the Journal of Clinical Oncology, are encouraging. Researchers reported that the AI model accurately discriminated between patients with high and low cancer risk when tested on historical images from Mass. General, Chang Gung Memorial Hospital in Taiwan, and a national lung cancer screening trial. 

The tool, named Sybil after an ancient Greek prophetess, works by filtering the imaging data through a series of processing layers to help analyze a patient’s lung tissue and probability of future cancer. The researchers reported that when Sybil marked scans as high risk, it correctly zeroed in on the site of future cancers, indicating that its conclusions were not random or drawn from disparate features of the image. The tool could also predict the extent of a patient’s smoking duration, suggesting that it was able to draw biological inferences from the imaging data. 

The authors emphasized that it needs additional testing to ensure that it will maintain its accuracy, or generalize, to diverse populations. Of the patients in the datasets used to train the model from the National Lung Screening Trial, 92% were white. 

“We really want to make sure that it’s generalizing everywhere we go,” said Peter Mikhael, a co-author of the paper from MIT. He said the researchers are working with other hospitals, such as Boston Medical Center and Johns Hopkins, to help further validate the model and assess its clinical utility. “We need people to experiment with the model and give us feedback to know, ‘OK, this is how we found it to be useful or not useful.’ Maybe it makes specific mistakes we would want to look at.”

In the early testing, Sybil’s accuracy held up when used on the Taiwan dataset, which contains a higher percentage of non-smokers. Its ability to assess risk in non- and never-smokers is especially important because of rising rates of cancer in that population and the need for better screening approaches to detect those cases. 

In the United States, screening is primarily focused on smokers, but even that population does not get the testing and follow-up required to catch cancers early enough to deliver effective treatment. The extent of that problem caused Lecia Sequist, a medical oncologist at Mass. General, to switch her research focus from developing new drugs to developing better screening tools to treat patients earlier. 

“So many of the people that I met already had stage 4 disease, which is incurable metastatic disease,” said Sequist, who helped build Sybil and was a co-author on the paper. “A very common question people would ask is, ‘Why wasn’t this identified sooner?’”

She said Sybil’s performance on historical data is impressive, but not the same as accurately predicting future risk for patients in the clinic whose outcomes are unknown.  Its effectiveness in that realm will be influenced by how much stock clinicians put in the calculations of a machine that processes information much differently than they do.

“If our model was able to say, ‘OK, this person should come back six months from now and we happen to catch a cancer six months early, you can change that patient’s life,” said Jeremy Wohlwend, a researcher from MIT who helped develop the model. “It can have a dramatic effect.”   

The prospective trial will take many months, if not years, to execute. Sequist said getting consent from participants will begin later this year. “We’ll be seeing how Sybil can augment the radiologist’s recommendations,” she said, adding that researchers will be especially interested in the outcome of borderline cases where the radiologist doesn’t see anything worrisome but Sybil concludes the patient is high risk and needs careful management.

The researchers’ hope is that Sybil will become a trusted voice in the radiologist’s ear, helping to focus more attention on the patients who need it.

“There’s this whole other level of how people perceive this,” said Sequist.  “How do clinicians perceive this? How do patients perceive this? Do they trust this computer? There’s a lot of interesting things that will affect its real-world usefulness that we’re going to study next.”

This story is part of a series examining the use of artificial intelligence in health care and practices for exchanging and analyzing patient data. It is supported with funding from the Gordon and Betty Moore Foundation.