One study doesn’t reveal very much — except a limited methodology
A recent New England Journal of Medicine article,
“Does AI-Powered Clinical Documentation Enhance Clinician Efficiency? A Longitudinal Study”, found no statistically significant difference between physician users and non-users of an ambient AI-powered voice recognition clinical documentation system.
Huh. Let’s check out that finding.
More specifically, the authors compared a group of 112 primary care physicians who had available to them an AI-enabled ambient clinical documentation technology and a control group of 103 primary care physicians who did not. The researchers analyzed 3 months (June-August 2023) and found no significant differences in efficiency between the two groups.
At first blush, this suggests that the promise of ambient voice recognition technology as a “cure” for clinicians overburdened by the EHR has been a lot of hype.
I think not.
Among the limitations of this study, the authors relied on the performance of a single vendor’s offering, Nuance’s DAX, and studied its application in only one health system (Atrium). Yes, DAX is well known but is it the best solution available? One cannot know from this study, especially given the rapid evolution of many entrants in the fast-evolving ambient scribe space. Ditto whether Atrium, although a well-regarded organization, had sufficiently supporter a deep embrace of DAX by its clinicians.
As anyone who has worked in hospitals or closely followed the history of EHR’s in healthcare can attest, the utility and useability of any EHR — and, by extension, an AI -enabled ambient scribe — is highly dependent upon extrinsic factors such as:
The quality of the vendor’s installation team;
The size and strength of the client’s IT department;
The judicious customization of features, functions, and flow
The amount, length, and effectiveness of in-service education provided to the end-users;
Whether at-the-elbow support is offered to those who are having trouble mastering the EHR’s intricacies; and last but not least,
Whether the client organization adopts a philosophy and practice of continuous optimization/remediation of its EHR.
These variables lie at the heart of why two health systems can have vastly different results even when acquiring and deploying the same EHR vendor. The same or similar variables are likely to inform the difference between a highly effective AI-enabled virtual scribe installation and a disappointing one.
Nuance DAX’s many competitors would do well to show how their solutions get different results, either because of superior technology and/or because of better customization, installation, training, and support.
Healthcare leaders would do well to look for such data — and to ignore or at least substantially discount the NEJM study described here.
Solutions for Hospitals, Health Systems, and Medical Group Practices: Leadership-Driven Changes
The locus for improving the clinical work environment — ie, curing clinician burnout and staffing challenges — runs through the C-suite. Here are examples of leadership-driven changes to the workplace that my colleague Paul DeChant, MD, MBA and I often recommend:
> Regard clinicians as knowledge workers who are given significant latitude to make clinical decisions without unnecessary administrative encumbrances or delays
> Consistently apply one or more of the improvement sciences (Lean, Six Sigma, operations research, agile, design thinking) in consultation with front-line staff to improve workflow and reduce delays, waste, inefficiency, and job skill mismatches
> Expect all leaders, but especially senior executives, to do periodic job shadowing of front-line staff (where observing and deep listening are emphasized) in lieu of "rounding" (a largely ineffectual, if widely practiced activity)
> Create and require leader standard work (LSW).
> Develop and deploy a sophisticated, deeply ingrained, and rigorous daily management system (DMS) supported by visual display boards or monitors
> Judiciously invest in AI/ML solutions -- selected, tested, and endorsed by front-line staff -- that eliminate or at least markedly reduce data entry, administrative requirements, and/or repetitive tasks that are non-value add
> Support near-continuous optimization and remediation of the EHR (there's no such thing as "it's all set")
> Get rid of superfluous or outdated policies, procedures, redundant approvals, and other stupid stuff (GROSS)
Ready to transform your hospital or workplace?
Are you frustrated by mistrust or even adversarial relationships between front-line clinicians and senior leadership? Organizational Wellbeing Solutions was formed to enable senior leaders to identify the specific drivers of clinician burnout in their organization; and to support leaders in designing and executing a comprehensive plan to stop clinician burnout, increase retention, and improve operating results. A hallmark of our consultancy is correcting the all-too-frequent distrust and alienation Clinicians feel toward the C-Suite generally and the CEO in particular. Let us help you help your organization and its clinicians develop a more trusting, aligned, and productive working relationship.
Comments