Operation AI

Dev Dash, MD, aims to reshape emergency medicine operations through the use of large language models for admissions assessment, critical care cohorting, and more.

AI, thoughtfully applied, has the potential to save millions of physician hours while reducing healthcare costs. The use of AI in care delivery is frequently explored, but Dash, assistant professor of emergency medicine, is testing the use of complex AI models for operational use on multiple fronts:

Admissions Assessment — Dash is exploring the use of a large language model (LLM) to“ingest” complex admissions pathways to assist with patient disposition. Saving just five minutes of physician time per patient could free 1,000+ minutes a day for physicians to treat other patients.

Critical Care Cohorting — Critical care is documented using specific phrases and templates. If an emergency medicine provider uses slightly different language or is missing certain elements, critical care billing codes may not be applied. Dash is training a LLM that would identify patient records as critical care that would have otherwise been overlooked and help identify patients who are sicker than they appear on initial triage.

Evaluating LLMs — Dash states that new LLMs drop every few days. However, most administrators are left scratching their heads when it comes to assessing what LLM will work best for their hospital. Dash is working on a method to score LLMs on multiple dimensions depending on specific clinical use cases and hospital needs.

According to Dash, healthcare leaders should always start with the pain point and how to assist a clinical workflow versus focusing on easily available data sources or trying to use the latest machine learning model.

A pain-point-centric approach should be led by physicians with deep clinical expertise and an understanding of clinical workflows, according to Dash. However he adds, “Physicians who have expertise on both sides — being clinical savvy and able to interface seamlessly with data scientists — are very rare. However, it is easier to train emergency medicine physicians on machine learning methods than it is to train computer scientists on the clinical workflows.”

Dash is also mindful of the ethical pitfalls in current AI and LLM development. “Most technology comes from the English-speaking, western world, where the majority of patient data we can access is from urban, healthy, wealthy patients because they have greater access to healthcare,” he notes. “But models built purely on this type of data will almost certainly fail when applied to other populations. You can destroy a model’s accuracy simply by applying it to patients in a different zip code.”

Updated Spring 2024