‘It still prevented cancers’: Experts discuss furor over colonoscopy screening study and dissect the nuances

On Sunday, the New England Journal of Medicine published the interim results of a landmark trial examining the effect of inviting people to colonoscopy screening. The paper, which STAT covered, exploded across medical media and ignited debate over the trial’s results, how to interpret those results, and the popular coverage of the study.

The frenzy resulted over the trial’s main finding — which is that offering colonoscopies to people did not reduce cancer deaths within a 10-year period. This result jarred with the longstanding belief that this screening could almost eliminate colorectal cancers if everyone attended to it.

Even with the debate, there was some clear consensus among experts about the trial, colonoscopy screening, and colorectal cancer screening in general. The main point: colonoscopy screening can prevent colorectal cancer and cancer-related death, even if the study suggested that invitations to colonoscopy were less than convincing. There is a lot of evidence supporting colonoscopy as a procedure, and this study does not suggest otherwise. Not only did experts agree that colonoscopy screening is useful, but also that the study provided further evidence that colonoscopy can prevent cancer.

“It still prevented cancers,” said Samir Gupta, a gastroenterologist at the University of California, San Diego and the Veterans Health Administration who didn’t work on the study, “There aren’t a lot of tests that can do that.”

The other main point is that the trial did not test the efficacy of colonoscopy as a procedure, but rather it investigated how colonoscopy programs perform in the real, messy world. That’s because the trial specifically tested the difference between inviting people to do a colonoscopy versus no colonoscopy; it was not a randomized trial of people who did colonoscopy versus people who didn’t. That makes this trial more of a population or public health study.

“It doesn’t answer the question of: if you’re in front of a patient and they do colonoscopy, will that reduce their risk of dying from colon cancer?” Gupta said.

Nevertheless, a debate is raging over the trial’s implications for patients, cancer screening writ large, randomized clinical trials, and — of course — how the media has covered all these things. STAT spoke separately with several experts who did not work on the study to explore the nuances and compiled a roundtable of different opinions here. These interviews are edited for length and clarity.

What are your views on what we can take away about colonoscopy overall from the NordICC trial published on Sunday in the New England Journal?

Samir Gupta, gastroenterologist and cancer researcher at UCSD: In the best case scenario, this study suggested that colonoscopy can reduce cancer risk by 30% and mortality by 50%. That’s excellent, and people have to set their expectations. The reality is we hoped that we could prevent all the colon cancers with colonoscopy, which is not what these data show. But could colonoscopy do better than that? Yes. It is an extremely operator-dependent test, and we don’t know how mortality and incidence reduction differs by the skill of the colonoscopist.

Ruth Etzioni, biostatistician and epidemiologist at the Fred Hutch Cancer Center: A 20% reduction in cancer is still a successful test. A 50% reduction is excellent. That’s what this study showed. Colonoscopy works as a test, but in the end we probably just need an easier test that people will actually do. That’s the take-home.

Folasade May, gastroenterologist and cancer researcher at UCLA: I’m a researcher that has been looking at non-colonoscopic methods for the last decade. A big takeaway is this paper shows the research I’ve been doing is right: we need to embrace screening with multiple modalities. We can’t recommend universal screening and say, there’s only one way to do that. I don’t agree with the medical societies that say that. Colorectal screening works, and it doesn’t matter which of the seven options you get, as long as you get one. Then, 100%, the follow should always be colonoscopy. If we find blood or cancer DNA from another test, the only way we find where it’s coming from is if we put a camera up there.

Vinay Prasad, oncologist and biostatistician at UCSF: My personal view is that I support colorectal cancer screening, but I think it’s crazy we’re not doing the thing that has the best evidence. That’s flexible sigmoidoscopy, which examines a smaller portion of the colon rather than the whole colon and has randomized trials supporting it, and colonoscopy now has evidence that’s not as good.

Jason Dominitz, director of the national colorectal screening program at the VA: This study doesn’t say that colonoscopy isn’t effective. It says it is effective at reducing colon cancer incidence. In the long run, I’d expect to see a significant reduction in mortality. One thing this tells me is that colonoscopy can only work if it gets done. It doesn’t help the people who don’t do it.

Why isn’t this a trial of colonoscopy, but rather a trial of a colonoscopy screening program?

Etzioni: Technically speaking, it was a trial of colonoscopy, as an intention to treat, and we can talk more about that. But it was an intention to treat colonoscopy that effectively became an analysis of offering colonoscopy. It’s a subtle distinction.

May: The caveat here is that when you have an intention to treat, but less than 50% of people got the treatment, you have to take a step back and ask, “Did we really treat?” So, this trial looked only at one modality of colorectal screening, and I think invitations for that modality — which is just colonoscopy. It’s really a population health approach. That’s where we have big translation problems between media and science with this paper. The paper is fine; it’s not bogus. I don’t agree with the people who say that.

What’s the reception of this trial — and the media coverage around it — been like for you? What’s the atmosphere like from your perspective?

May: It’s been an insane week. Yeah, I knew it was going to be a whole firestorm. Colonoscopy has been very polarizing. It’s kind of funny because I heard on Wednesday, last Wednesday, and then I got your email on Friday, and I was like, “uh oh.” It’s still crazy. You were ahead, CNN was ahead, but people are catching up now. It’s impressive to see how different people are interpreting the data, physicians, non-physicians, researchers.

Prasad: First, it’s been a thrill because you’re getting so much debate on something that doesn’t have Covid in the title. It’s great to talk about important medical issues. Cancer screening has been one of the most divisive and contentious topics, and there’s a big range of views.

Etzioni: Stories about cancer screening, particularly negative stories, generate a lot of emotions. I was involved with the American Cancer Society to move the breast cancer screening age from 40 to 45, and that also ignited a firestorm. This was one finding that just blew up.

Criticisms of the way this trial has been perceived or interpreted?

May: My biggest issue with the NEJM article is the title. The study wasn’t an effectiveness of colonoscopy study but effectiveness of invitation to colonoscopy, and people will read the abstract at most and run with it. That’s where we got into trouble. There’s nothing wrong with the science of the paper or NEJM accepting it. The next biggest gripe, and some media did this, they conflated colonoscopy with colorectal screening overall and said this one study, despite decades of research, goes as far as to say colorectal screening does not work. That is blasphemy. That is deadly. That is false.

Etzioni: Really one word was missing: offering. This was ultimately a trial of offering colonoscopy in a particular population. It was an intent to treat analysis of colonoscopy that effectively ended up being an analysis of offering colonoscopy.

There are some people arguing that the primary analysis or what’s called the “intent to treat” analysis is not as important as what we called the secondary analysis – or the “per protocol analysis” – because only 42% of people in that invited arm actually got the colonoscopy. Thoughts on this?

Prasad: The intention to treat is the analysis that has true randomization. So, we have to ask ourselves, why are we doing a randomized study? Because fundamentally, we believe the people who agree to drink the bucket of laxatives, are compliant, show up on time, and get the colonoscopy are different than the people who say, ‘screw it.’

So, that’s why we randomize. Hopefully this will wash away any differences in the types of people who participate. You balance what you can measure, the ages, the races, the socioeconomics, but also the things you can’t like how often they eat quinoa and how many times they go for a run a week. That’s not in the dataset, but I can rest assured that people in both arms of these studies, there’s the same fraction of people who run three times a week. Then we intended to treat half of this random group with a colonoscopy program, and the other half usual care. That asks the question, are you better off with an invitation to colonoscopy or not, and that’s the intent to treat.

The per protocol just zooms in on the 42% of people who followed through with the colonoscopy and compares them to the control arm. Here’s the thorny part. They had to pick a chunk of them, the ones who are covariate matched or in other words, the same age, ethnicity, the same specific things they can measure. But they can’t match them on the things they can’t measure. So in my mind, a per protocol analysis is a lot like an observational study. You think you’ve matched them, but you don’t know, and you’ve undone some of the gains of randomization.

The study authors noted that in Poland [where 51% of the people who did a colonoscopy in the trial lived] the people who actually did colonoscopy looked like they were higher risk for colon cancer than those who did not. How might that affect the analysis?

Etzioni: If people at higher risk are the ones who show up for colonoscopy, then the intent to treat analysis will still be diluted because all the people who didn’t show up cannot benefit. And the per protocol analysis might actually underestimate the true benefit. But reality is never like that. This is why you can’t just do a naïve per-protocol. It has to adjust for the thing that makes the groups different.

Prasad: When you start to compare the people who followed through with the people who no-showed, you get into this problem. What if the people who followed through were actually the ones most worried because their uncle, their grandfather, their mom had colon cancer, and they were the ones at higher risk. One might argue that the per-protocol there underestimated the benefit. Just as easily, it could overestimate. Put it this way, maybe the people who did colonoscopy are a subset of the high risk group. Yes, they’re higher risk, but maybe they’re also more likely to curtail that risk and do the colonoscopy and eat healthy. Now you fall down the cataract of speculation. That’s also the problem with per protocol. At the end of the day, we just don’t know, and this is why we randomize in the first place.

Gupta: It can go either way. The only way out of this is to address people’s low rate of participation in a true randomized trial. If there can be a study where 80% to 90% of people did the screening test, but I don’t know if we’ll have that. We may or may not see that in the VA’s CONFIRM trial, comparing colonoscopy to fecal screens.

What does this trial, as an example, reveal about randomized clinical trials on cancer screening in general?

Etzioni: Clinical trials are designed to be simple. They are in theory, but nothing is simple when it comes to screening. They happen over a long period of time in a healthy population that don’t always necessarily need the intervention. We need to understand what happened in the implementation — the compliance, the non-adherence, the generalizability. In eight trials of breast cancer screening, for example, they all give different results.

A randomized trial is only ideal in principle. Every randomized trial of screening has had to be post-hoc interpreted and understood. Last, trials often become outdated by the time they are evaluated. Many have different treatments that might improve or reduce the benefit of screening. Thinking clearly about how all these parts come together, then we leave the simple space.

Prasad: Randomized trials are lights in a sea of darkness. You know, it’s a very useful tool, but just being a randomized trial doesn’t mean you’re a good one. They can have fundamental limitations. It’s difficult to do trials, but I’d say it’s even more difficult to implement screening programs in the real, messy world. Whatever gains you see in a trial are, in my opinion, the upper bound of what you gain from screening in the real world. It gets eroded as people don’t come for screening or don’t do the follow up, where in a trial people are more likely to be adherent. I’ve had people with metastatic lung cancer getting a colonoscopy. That’s not helping anybody.

There was a lot of strong reaction especially from gastroenterologists. Prasad, you mentioned that colonoscopy is very well reimbursed. Do you think there’s some financial incentives at play here that may have biased the response?

Prasad: Gastroenterology is one of the most competitive and lucrative specialties. There’s a difference in reimbursement rates for different colorectal screens. Flexible sigmoidoscopy, which only looks at part of the colon, is a few hundred buckaroos a pop. Colonoscopy can reimburse thousands of dollars. If you take away screening colonoscopy, you will deprive them of substantive revenue. So, if you really, in your heart of hearts, believe you’re helping people and every time you do something you get a little bit of money, that combo is the methamphetamine of being a doctor. [Commercial costs for colonoscopy screening can rise into the thousands, for more expensive care, though physicians only receive a fraction of that.]

In Europe, they’re not as incentivized to do it per procedure. So they do this study, and then of course there’s everything wrong with the study. I commend the investigators. It’s very provocative. We’ll get three more studies from Spain, Sweden, and the U.S. What happens if all three are negative? Heads will explode.

May: I’m a gastroenterologist, and I do public health research. I get it. There is a big misunderstanding of how much colonoscopy is getting reimbursed. In some cases, it’s just a couple hundred dollars. It’s lower than people think. But, yes, it’s gastroenterologists’ bread and butter. Absolutely, they are nervous if you take away colonoscopies, you take away their bread and butter, just as anyone else would be.

Closing thoughts?

Gupta: I can see how this is very frustrating for a lot of people. It’s hard. It’s science. It’s not perfect. You don’t always get the answer to the exact question, exactly the way you want it, and you don’t always get the answer you expect. The main thing is we don’t overreact. The study does not mean we throw out colonoscopy!

Etzioni: I have a history of colorectal cancer in my family. My uncle died, and my great aunt on my mother’s side, and I’ve had colonoscopies for years, but never a polyp. Others in my family, every time they have polyps. Their lives have probably been saved by it, but with my negative history, I almost probably don’t need it as first-line. I now use the stool test, because I probably didn’t inherit that gene.

That’s why precision oncology is such a big thing. We are trying to figure out who is at risk, but you know, it’s an uphill battle and you never have it perfect. In the end, we probably just need an easier test than colonoscopy.

May: I have patients texting and calling me and saying, “should I come in for that colonoscopy on Thursday?” That’s a devastating result of this publication. The reality is that every day, there are people with polyps and cancer growing because colorectal cancer is so asymptomatic and it’ll grow for years. You will never know. Hopefully we get the message through that colorectal screening saves lives. People, try to do it. I hope we get that message through. I’m a glass half-full kind of girl.

Get your daily dose of health and medicine every weekday with STAT’s free newsletter Morning Rounds. Sign up here.