‘We’re getting much more aggressive’: Microsoft’s Nuance adds GPT-4 AI to its medical note-taking tool

An AI medical scribe platform is the latest entrant into the AI arms race. On Monday, Microsoft-owned Nuance Communications announced it is integrating GPT-4 into its Dragon Ambient Intelligence platform, which is used by hospitals around the country to ease doctor workloads by using AI to listen to patient-provider conversations and write medical visit notes.

Starting this summer, all providers currently using DAX or Dragon Medical One will be eligible to apply for an early adopter program for DAX Express, which bypasses the human reviewer used as a quality control in the current DAX product, and returns fully AI-generated notes within minutes of a patient visit. This move marks a decisive acceleration of Nuance’s timeline for transferring all of the responsibility of drafting notes to AI. 

“We’re getting much more aggressive,” Peter Durlach, Nuance’s chief strategy officer, told STAT. Right now, there are less than 100 physicians participating in pilot tests of Nuance’s fully AI-powered express mode. Last month, Nuance told STAT that the company had hoped to move into a beta phase by this summer, with a goal of enrolling 400 physicians. 

It’s not clear what role the GPT-4 technology — which has drawn significant buzz for upgrading the abilities of technologies like ChatGPT — will play in the note-taking model. “All I know is that the team has been very firm in saying there’s some magic by combining the two pieces together,” Durlach said. 

DAX promises to reduce physician burnout by relegating the hours of work doctors spend writing appointment notes to an artificial intelligence platform. The AI listens to the doctor-patient visit, transcribes the conversation, and composes a draft note. In most cases, that draft note is checked by quality assurance staffers before being sent on to the doctor for review. That review process results in a lag of a few hours. 

However, the new GPT-4-powered DAX Express is fully automated, eliminating humans from the process until a note gets to a physician for review. The tool provides “an immediate and highly accessible entry point for healthcare organizations to adopt at scale a new generation of AI-powered applications,” according to the Nuance press release.  The company had been working on transitioning to the fully AI-powered express mode version of DAX for years, starting with its alpha pilot and its planned beta program. Previously, Nuance’s stated vision for that “full-service” tool included the option for physicians to bounce an AI-drafted note back to Nuance’s reviewers for another set of human eyes. 

This option will still exist for users of the express mode of DAX that was being developed through the alpha and beta testing programs. But those who apply to join the newly announced DAX Express with GPT-4 pilot won’t have that option for human review.

Paddy Padmanabhan, chief executive of Damo Consulting, a health technology advisory firm in Chicago, said Nuance is seizing an opportunity to jump ahead in the race to build AI voice capabilities for health care providers. Amazon shut down a program late last year to harness its Alexa-branded AI to build voice applications, leaving Nuance to battle with a group of scrappy start-ups that have powerful technologies, but lack its reach and customer base in health care.

“The field is open for Nuance to really accelerate their product development and deployment — and their market share,” Padmanabhan said. He added, however, that moving too aggressively could also backfire. “Anything to do with health care has a heightened sensitivity to risk,” he said. “The whole point of the technology is to make it as accurate as possible, and minimize corrections that need to be made after the encounter, and that’s a work in progress.”

Durlach said that Microsoft, and especially Nuance, are being careful of the trust relationship with their clients and are clear-eyed about the powerful new technology. “On one hand, it really is amazing and the GPT-4 makes ChatGPT look like a toy; on the other hand, it makes mistakes and it hallucinates and it omits things,” he said. “[So] we’re picking applications that we think make sense and where the technology comes to bear in a way that makes sense that if it makes a mistake or two, not going to hurt anybody.”

Previously, clinicians had to log hundreds of visits with DAX and progress across DAX’s “AI readiness score” to get greenlit for express mode. But Durlach said that any user is eligible to apply to use the new DAX Express, whether they’ve used DAX before or not. “Part of the reason we accelerated the launch is we believe the vast majority of the DAX users can go right to automated,” said Durlach. 

However, the accelerated release poses questions about the level of independent scrutiny applied to the product, especially because it is being combined with a powerful but still experimental technology such as GPT-4. 

“I would be way more comfortable if [Nuance] worked with their key customers or the early adopters to make their evaluations public,” said Nigam Shah, professor of biomedical informatics at Stanford University. Physicians and patients are being asked to trust that Nuance, a multibillion-dollar company racing to out-flank competitors, is doing enough testing to ensure that its product is reliable and beneficial.

Shah said the most useful step would be to test Nuance’s product on a publicly available dataset of patient recordings supplied by health systems, so that the assessment of its performance could be truly independent. Since a company is unlikely to pay for such data, Shah said, other parties who want to benefit from the use of the technology would need to supply the data and establish performance standards. 

“If we don’t do that, we’re going to hold ourselves back,” he said. “And lots of snake oil will get sold.”

This story has been updated to clarify the role of physician review in GPT-4-powered DAX Express.

This story is part of a series examining the use of artificial intelligence in health care and practices for exchanging and analyzing patient data. It is supported with funding from the Gordon and Betty Moore Foundation.