Researchers from Yale School of Medicine (YSM) are working to streamline the clinical decision-making process using large language models (LLMs).
First, in 2023, the team launched MedAgents. This tool allows multiple LLMs to act as the various specialists that would be involved in making a clinical decision in different scenarios. Other scientists have created platforms that enable a single LLM to complete this process.
The same team that designed MedAgents has now developed a new tool, MedicalAgentsBench, that helps scientists differentiate between these LLMs. The goal of this action was to find the best way to accelerate the clinical decision-making process.
The platforms that require only one LLM to evaluate a decision are defined as “internalized reasoning models.” The platforms that call for multiple LLMs to discuss and make decisions are defined as “externalized agents.” The researchers wanted to compare the two types in their new study.
While other tools memorize answers to medical questions, MedicalAgentsBench is equipped with more than 800 complex medical questions that allow it to “use multi-step thinking rather than rely on memorization.”
When using MedicalAgentsBench to compare internalized reasoning models vs. externalized agents for clinical decision-making, the team did not conclude that one was better than the other. Instead, they found that “they can complement one another.” The researchers hope their tool can help healthcare organizations with the development of their own clinical decision-making aid models.
