Sampling the response space and adjudicating diverse responses

OpenAI’s GPT-4 was selected as the base LLM given its accessibility, support of Application Programming Interfaces (APIs), extensive documentation and community of support. Our goal was to use a ‘Council’ approach where each LLM instance is a member of the Council, and diverse responses from each LLM instance would undergo a coordinated and iterative exchanges designed to arrive at a consensus response among the models (hitherto referred as “deliberation” or “deliberative process”). To facilitate a deliberative process when there are divergent responses, a Facilitator algorithm (which includes instantiations of the LLM) summarizes the reasoning in each response, formulates a question to help the Council deliberate divergent reasonings, and presents the summary and question to the Council with a request to re-answer the original test question.

The process for the Council’s discussions is summarized below and in Fig 1:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Council of AI Agents Architecture.

The User (Human) presents a query, which is passed by the Facilitator algorithm to each of the Council members, whose response is then assessed for agreement. If there is agreement on the answer, a consensus is reached and a consensus statement is output summarizing explanations and identifying the selected response. If there are differing opinions about the correct response, then the Facilitator algorithm summarizes the reasoning presented by each council member, formulates a question that can help clarify reasoning behind difference of opinions, and asks the council members to also respond to the initial prompt (i.e., the USMLE question).

https://doi.org/10.1371/journal.pdig.0000787.g001

1. The multiple-choice question is copy and pasted into an interface.
2. The question is transmitted to a facilitating algorithm. The facilitating algorithm sends the same question to LLM-communicating functions that represent unique instantiations of the LLM (i.e., unique AI agents), thereby eliciting a response from each LLM instance/ AI agent.
3. Each LLM-communicating function communicates uniquely with the LLM through an API.
4. Each instantiation of the LLM sends a response back, generated from its sampling of the response space.
5. Once all the responses are received, they are passed to the Facilitator algorithm, which includes an instantiation of the LLM which assesses Council member response for agreement.
6. If there is no consensus in the responses received, then the Facilitator algorthm does the following: [1] summarizes the responses from the various AI Agents, [2] formulates questions to elicit reasoning relating to the divergent responses, and [3] requests a response to the original question.
7. The output of the Facilitator algorithm is a prompt, which includes a summary of each AI agent’s response and reasoning, a clarifying question relating to differences in response selection, and the initial USMLE question along with multiple choices from which the LLM is required to make a selection.

Steps 2–7 are iterated until a consensus response is reached amongst the various instantiations of the LLM. Once a consensus is reached, the transcript of the Council’s deliberations is saved, the browser is refreshed to clear user interface elements (e.g., question input box and deliberation transcript), and the server is reset to ensure any stateful variables (such as round counters) are reinitialized before the next interaction.

For a question requiring deliberation, Table 1 provides a specific example of the Facilitator algorithm’s summarization of Council members’ responses, formulation of question to elicit reasoning related to divergent responses, and a request to respond to the original question. Table 1 does not include the details of the responses from Council members, which can be found as question 25–1 in S2 (S2 Data Raw data files - transcripts of deliberations) as part of transcripts generated by the deliberations conducted by the Council for each question. For details of the API prompts, please refer to the supplemental methods (S1 File details on methods), which includes prompts for each AI agent, analyzing responses, re-prompting from the Facilitator algorithm to the Council if there is disagreement, and synthesizing responses when there is agreement. All LLMs instances were instantiated with a temperature setting of 1. Code used in this paper is publicly available at: https://github.com/councilofai/project-saru.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Example of Facilitator AI’s Question to Council for Discussion in Case of a Divergent Response.

https://doi.org/10.1371/journal.pdig.0000787.t001

Constructing the API prompt

To create a context to facilitate the emergence of responses related to the medical domain, the following instruction was added to prompts presented to the LLM:

You get your knowledge from the following domain spaces: anatomy, physiology, pathology, pathogenesis, pathophysiology, histology, pharmacology, microbiology, endocrinology, immunology, hematology, oncology, genetics, embryology, internal medicine, surgery, pediatrics, obstetrics, gynecology, psychiatry, family medicine.

For ordering an explanation in a way that can simulate reasoning, we specified the construction of content that included scientific problem solving and clinical reasoning: [25–27]

Your sentence completion follows the following pattern: [1] RESTATING THE QUESTION: Restating the question in your own words and defining technical terms and concepts; [2] PLANNING: Identifying assumptions, generating alternate assumption by questioning each assumption, and then evaluating and hypothesize which assumption should be clinically prioritized to pursue; [3] CLINICAL REASONING: Connect each sentence to its subsequent sentence using a clinical thought process to provide a step-by-step explanation to support the alternate assumption; [4] CONCLUDING: Use a step-by-step explanation that connects one sentence to the next with a clinical thought process to generate your final response

The final prompt identifies Council members, focuses their attention on a domain knowledge and reasoning, and ensures that subsequent responses in the discussion are on topic. For each of the members of the AI Council the temperature setting was set at 1 and the top_p (nucleus sampling) set to 1. The JSON used to specify these parameters was input to as the “system message” in the API call, and is detailed below for AI Council member named AI_A (“thisModel” refers to the language model, thisTemp refers to the temperature parameters, thisTopP refers to the top_p parameter, and “thisPrompt” refers to the prompt that initiates the discussion or subsequent prompt that maintains the discussion through facilitator input):

AI_A:{thisModel:“gpt-4”, thisTemp:1, thisTopP:1, thisSysMsg:”Your ‘name’ is ‘AI_A’. You are part of a council that includes ‘AI_B’, ‘AI_C’, ‘AI_D’, ‘AI_E’, and ‘AI_F’. You get your knowledge from the following domain spaces: anatomy, physiology, pathology, pathogenesis, pathophysiology, histology, pharmacology, microbiology, endocrinology, immunology, hematology, oncology, genetics, embryology, internal medicine, surgery, pediatrics, obstetrics, gynecology, psychiatry, family medicine. Your sentence completion follows the following pattern: [1] RESTATING THE QUESTION: Restating the question in your own words and defining technical terms and concepts; [2] PLANNING: Identifying assumptions, generating alternate assumption by questioning each assumption, and then evaluating and hypothesize which assumption should be clinically prioritized to pursue; [3] CLINICAL REASONING: Connect each sentence to its subsequent sentence using a clinical thought process to provide a step-by-step explanation to support the alternate assumption; [4] CONCLUDING: Use a step-by-step explanation that connects one sentence to the next with a clinical thought process to generate your final response; [5] RESPONSE: You always try to connect your response to answer the following: [“+thisPrompt+”]”}

The code for implementing the above is available in the public domain: https://github.com/councilofai/project-saru.

Selecting the number of council members

There are several technical constraints that determine the number of instantiated AI agents. In the implemented architecture, the LLM’s synthesis of responses from the members of the Council means that all of their responses are passed together within a single API call. The size of the content passed in a single API call is constrained by token limits of a given LLM. Another constraint is the time taken to respond to each call increases linearly by the number of API calls that are made. The latency time between call and response may be quite large for models like GPT-4. Cost is an additional constraint that increases linearly with the number of instantiated AI entities. We found that generating five AI agents as part of the Council allowed us to stay within token limits during Council deliberations. The study was conducted on a standard workstation with internet access, without local inference, relying solely on cloud-based inference from OpenAI.

USMLE questions

The USMLE questions used in this study have previously been used in single-AI evaluations of OpenAI’s GPT LLM, [2,4] initially sourced from the June 2022 release of the sample exam by the Federation of State Medical Boards and the National Board of Medical Examiners.[28] The ChatGPT-4 model used in this study was released in March 2023 and was trained on web data available up until September 2021.[29] Because the training cut-off for GPT-4 version used (September, 2021) was earlier than the release of the sample exam questions (June, 2022), it ensured that the multiple-choice questions used to assess the Council of AI were not previously seen in the training data of the underlying LLM. From the original 376 questions that were made publicly available, 51 questions containing images or tables were removed from the questions bank, leaving 325 multiple choice questions (Step 1: 94, Step 2CK: 109, Step 3: 122). No sample size calculations were conducted, as the study included the entire set of publicly available questions without images or tables. These questions had been used in prior studies evaluating LLM performance on the USMLE, allowing for direct comparison of results.

Analysis of AI council performance

We used the USMLE provided answer keys to assess the accuracy of the individual Council members’ initial responses and the Council’s final consensus responses. We also evaluated the relationship between the number of incorrect initial responses and the accuracy of the consensus responses, and the mean number of rounds needed to reach a correct consensus. We calculated semantic entropy of the response space as suggested by Farquhar et al, which measures the uncertainty in a model’s output by evaluating the diversity of meanings among its possible responses.[30] A higher semantic entropy indicates greater uncertainty, suggesting a wider range of potential meanings in the model’s response space. Because we did not have direct access to a model’s output probabilities, we approximated semantic entropy by calculating discrete semantic entropy. To calculate discrete semantic entropy, we identified multiple or diverging responses by identifying the multiple-choice selection of each AI Council member. Multiple responses to a single question are grouped into clusters based on whether they identified the same answer selection as correct, even if the reasoning they offered to reach that response may differ. Semantic clustering groups together outputs that express the same essential idea, which, in the case of multiple-choice questions, is given by the selection of the same multiple-choice response. Discrete semantic entropy method treats each generated response equally, assuming a uniform distribution across all responses. This approach constructs an empirical probability distribution over the semantic clusters by simply counting how many of the responses fall into each cluster and dividing by the total number of varying responses produced. Once this distribution is determined, the Shannon entropy formula was applied to calculate the semantic entropy: [30]

where 𝑁 is the total number of clusters, and 𝑝_𝑖 is the probability of the 𝑖-th cluster.

The resulting entropy value reflects the semantic diversity of the responses. A low entropy score indicates that most responses were semantically similar, suggesting confidence or consistency in the model’s answer. On the other hand, a higher score indicates greater semantic variability, which can be a sign of model uncertainty or hallucination.

We calculated semantic entropy at the end of each round to assess if and how it changes over rounds, and if there is a relationship between number of rounds to reach consensus (semantic entropy of 0) and magnitude of semantic entropy at the end of the initial round.

Because a consensus in round 1 of the Council’s responses indicate an unanimous majority response, we further analyzed the effectiveness of group discussion compared to majority voting for questions where the majority response was not initially unanimous. For each question that was not initially unanimous, we conducted a contingency analysis for the correctness of the majority vote at the end of round 1 versus the post-discussion consensus. For these paired observations, we calculated the matched pairs odds ratio and utilized the McNemar’s test with continuity correction to assess the significance of changes in correctness before and after discussion.

We conducted a preliminary analysis of question characteristics between USMLE questions that resulted in correct versus incorrect final consensus after multi-round AI deliberation. Variables examined include: question word count, number of options in multiple choice, diversity of initial responses (i.e., number of differing responses given in the first round of deliberation, and number of rounds needed to reach consensus. Questions were also disaggregated by USMLE step, medical specialty category of the questions (e.g., cardiology, endocrinology), and question type (e.g., pathophysiology, diagnosis). Independent t-tests were used to compare continuous variables, and chi-square tests for categorical variables.

Comparison of USMLE performance with prior studies

Although the exact questions have varied between prior studies of LLMs on USMLE questions, the Council of AI seems to achieve a superior performance to any solitary state-of-the-art LLM to date. In a comparable study utilizing the same USMLE practice questions, GPT-4 answered 88%, 86%, and 90% of Step 1, 2CK, and 3 questions correctly, respectively.[4] In a dataset of USMLE questions known as MedQA, Google’s Med-Gemini-L 1.0 achieved 91% accuracy overall.[34] Prior studies in MedQA have shown an accuracy of 86% by GPT-4 base and 86.5% by MedPaLM 2.[34] By optimizing prompt engineering, Nori et al. were able to increase the performance of GPT-4–90% in this dataset.[3] A comparison of the Council of AI to prior zero-shot tests of GPT-4 is shown in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparison of Council of AI accuracy on USMLE questions compared to prior studies using GPT-4 (zero-shot).

https://doi.org/10.1371/journal.pdig.0000787.t005

Table showing performance comparison between the Council of AIs and previous single-instance GPT-4 studies using similar USMLE question sets. The Council of AIs outperformed all prior single-instance GPT-4 evaluations across all USMLE steps. Improvements ranged from 4-16 percentage points, with the largest gains in Steps 1 and 2, demonstrating that collaborative AI approaches can exceed individual model performance on standardized medical examinations.

The lower accuracy observed in previous single-instance GPT-4 studies can be attributed to several inherent limitations of solitary LLM deployment. First, single instances are subject to the stochastic nature of token-by-token generation, which introduces response variability that can lead to incorrect answers on questions the model has the underlying knowledge to answer correctly.[22,23] Second, traditional single-instance approaches lack mechanisms for self-correction or iterative refinement, meaning that initial reasoning errors cannot be identified and corrected through subsequent deliberation. Third, isolated reasoning processes miss the benefits of collaborative error-checking, where multiple perspectives can identify flaws in logic or knowledge gaps that a single instance might overlook. Finally, single-shot responses provide no opportunity for the kind of adaptive reasoning demonstrated by the Council approach, where initial disagreement can trigger deeper analysis and ultimately lead to more accurate conclusions. The Council architecture specifically addresses these limitations by embracing response variability as a feature rather than a bug, providing structured mechanisms for collaborative reasoning, and enabling iterative refinement through deliberation. This explains why the Council approach achieves 4–16 percentage point improvements over prior single-instance studies, with the largest gains occurring in the most knowledge-intensive domains (Steps 1 and 2) where collaborative reasoning provides the greatest advantage over isolated decision-making.

The value of response variability

The observation that 22% of questions needed multiple rounds of deliberation highlights the stochastic nature of LLMs’ content generation and suggests that a single response from an LLM might not suffice for precise decision-making. This is consistent with findings from other studies, such as Yaneva et al., who reported variability in correct answers across three replications, observing inconsistencies in 20% of USMLE questions.[35]

This study suggests that variability in LLM responses, a limitation in isolation, can be a strength in collaboration. This unexpected finding suggests new approaches to evaluating generative LLMs, which usually favor predictability and consistency, especially in contexts like professional exams where questions have only one correct answer. By valuing the inherent variability in responses across different LLM instances and leveraging consensus-building as a method, this finding suggests we may need to rethink what optimal behavior means for generative LLMs. Evaluation should not only consider the performance of a single LLM instance but also prioritize the system’s capacity for collaborative engagement with other AI instances to identify a best response. A collaborative approach could significantly improve accuracy and reliability in critical decision-making areas like healthcare.

Interestingly, the Council never achieved a correct consensus response without at least one member offering a correct initial response. This underscores the significance of having a Council with sufficient diversity in reasoning to ensure at least one LLM instance can propose a correct response. While this study employed multiple instances of GPT-4 to control for variability and focus on the effects of deliberative coordination, future research could explore the benefits of heterogeneity by constructing Councils composed of diverse language models, such as Claude, or Bard. Each model is trained on distinct corpora and incorporates different architectural and alignment strategies, which may yield varied perspectives, reasoning styles, or domain-specific strengths. This diversity could enrich deliberations by introducing alternative viewpoints or counterfactual reasoning paths, potentially surfacing edge cases or mitigating blind spots specific to a single model family. For example, a model fine-tuned on medical literature might challenge a general-purpose model’s heuristic, or a model with stronger multilingual grounding could raise issues missed by monolingual training. Such cross-model deliberation could potentially improve accuracy, while also increasing robustness, fairness, and generalizability of ensemble reasoning frameworks, particularly in high-stakes domains like clinical decision-making or public policy. Exploring how different models negotiate disagreements or reinforce consensus offers a promising direction for advancing deliberative AI systems. Future research could explore the identification of the optimal characteristics, number, and composition of council members to generate a diverse range of responses to increase the probability that at least one member can suggest a correct answer.

An unexpected finding in this study was that the number of rounds of deliberation needed to achieve a consensus was not associated by the extent of disagreement in the initial round of responses to a question (Fig 7). This suggests that a factor other than the number of divergent responses is driving the amount of deliberation. A preliminary analysis of question characteristics revealed no statistically significant predictors of AI consensus accuracy (Table 4). Future studies can explore the characteristics of questions or responses which increase or decrease the amount of deliberation before consensus; and can explore the characteristics of questions for which the Council failed to achieve a correct consensus. A more in-depth analysis to characterize questions that yield correct versus incorrect responses after deliberation may involve extracting quantitative text-based metrics such as word count, sentence complexity, readability scores (e.g., Flesch-Kincaid), and information density (e.g., ratio of medical terms to total words), as well as indicators of clinical complexity, such as multi-system involvement, presence of diagnostic uncertainty language (e.g., “most likely,” “suggests”), and case timeline intricacy. Features of answer choices themselves, such as semantic similarity may also be relevant. Additionally, patterns of AI behavior in the initial round can be analyzed, including the number of unique responses, semantic entropy of initial answers, and majority/minority voting splits (e.g., 3–2 vs. 4–1). Stratifying questions by USMLE Step, medical domain (e.g., cardiology, pharmacology), and reasoning type (e.g., factual recall vs. clinical synthesis) may identify content-specific performance trends.

When considering divergent responses for a given question, the reduction in semantic entropy across rounds of deliberation until consensus is reached represents an alignment of reasoning pathways among model instances (Fig 7). In this study, declining entropy corresponded with increased epistemic coherence and was associated with improved accuracy, particularly in cases where initial responses were divided. This suggests that entropy may serve as a useful proxy for internal disagreement and reliability in multi-agent AI systems. Beyond medical question answering, entropy tracking has potential applications in clinical decision support, where persistently high entropy could flag ambiguous cases for human oversight; in legal reasoning, where it may help surface interpretive plurality in complex cases; and in public policy simulations, where entropy could serve as a signal for model uncertainty and prompt additional stakeholder consultation. In each context, semantic entropy provides a quantifiable means of monitoring internal disagreement within AI collectives, offering insights into both system confidence and the potential need for external validation.

Self-correction and collective intelligence

If a LLM produced an incorrect initial response, we found it subsequently generated a correct response 85% of the time through interaction with other instances of itself. This aligns with recent studies on self-correction mechanisms in LLMs.[36–38] This phenomenon, where multiple AI instances engage in a dialogue and reconcile their differences to achieve a consensus, reflects a simulation of collective intelligence, [39] enabling a group of AIs to ‘change their minds’, a level of complexity and sophistication often attributed to human group dynamics.[40] The necessity for LLM instances to explore new reasoning paths to arrive at a correct consensus underscores the flexibility and adaptive reasoning capabilities present in contemporary LLMs. This flexibility is particularly valuable in dynamic, real-world settings where issues often require more than a singular, predetermined solution strategy.

The results suggest that discussion-based consensus provides a statistically significant improvement in the correctness of responses compared to a simple majority vote. Specifically, in comparing the initial majority vote in Round 1 with the Council’s final consensus (Table 3), there were only two instances where a correct initial majority ultimately resulted in an incorrect consensus after discussion. In contrast, for the 19 questions where the initial majority was wrong, in 10 of those cases the Council consensus overturned the initial majority, resulting in a correct final answer. The odds of a majority vote response changing from incorrect to correct were 5 (95% CI:1.1, 22.8) times higher than the odds of it changing from correct to incorrect after discussion, demonstrating a statistically significant improvement in accuracy from pre-discussion majority voting to post-discussion consensus. Taken together, these findings highlight the Council’s deliberative process as a strategy that preserved correct majority opinions and “rescued” those instances in which the majority initially erred. In other words, the multi‐agent Council approach has the potential to both preserve correct majorities and significantly increase accuracy when the majority begins in error. By iteratively reconciling divergent reasoning paths, the Council demonstrates that structured dialogue can leverage variability in AI‐generated responses to achieve superior reliability and performance compared to relying solely on a single, one‐shot majority vote. Future research might explore why discussion can improve accuracy (e.g., type of question, quality of explanations, number of conflicting perspectives), as well as any conditions that optimize its effectiveness.

The high performance of the Council of AI Agents is supported by studies of ensemble systems and multi-agent systems that show that AI collaborations can have a superior performance compared to single AI instances [32,41–51] and can be useful for applications such as drug-discovery.[52,53] Ensemble approaches have also been used to improve performance for LLMs across different knowledge domains. For medical question-answering in particular, Yang et al employs boosting-based weighted majority voting which significantly outperformed individual models on various datasets; [54] Jiang et al uses specific algorithms to merge potential candidate outputs from various LLM in a pair-wise fashion improving performance by measuring against with known benchmarks; [55] Pitis et al introduces a “prompt ensemble” method for a “chain-of-thought” language model reasoning; [56] while Naderi et al uses ensemble techniques for named entity recognition in health and life science domains.[57]

It has been suggested that the Council of AI may resemble a Mixture-of-Experts (MoE) system. While our framework shares some similarities with MoEs and multi-agent LLM systems, it represents a distinct architectural and epistemic paradigm. Traditional MoE architectures, such as those implemented in large-scale LLMs by Google and NVIDIA, operate at the level of latent token processing. A learned gating network dynamically routes input tokens to a subset of computationally efficient “experts”, which are sub-networks trained jointly within a single model. These experts produce hidden representations rather than human-interpretable outputs, and their contributions are fused numerically. The emphasis in MoE is on computational efficiency and learned internal specialization, not on deliberation or transparency.

A different, increasingly popular approach connects LLMs in agentic systems, where each LLM instance may be fine-tuned or prompted for domain-specific expertise and tasked with planning, reasoning, or acting autonomously in coordination with others. In such systems (e.g., as seen in platforms like Auto-GPT, LangGraph, or CrewAI), each agent may have a persistent role or memory and interact with external tools or APIs to complete a task. These frameworks emphasize goal-directed autonomy and role-based collaboration, often oriented around real-world actions and long-horizon planning.

The Council of AIs model differs fundamentally from both. In our framework, each LLM instance serves not as a tool or planner, but as an epistemic contributor that is a self-contained reasoner offering a complete response to a shared question. These agents are not trained to specialize nor endowed with memory or external tools; they reason independently from a base model (e.g., GPT-4) and deliberate through natural language exchanges facilitated by a central agent. The Facilitator is not a gating mechanism (as in MoE) nor a task planner (as in agentic systems), but a meta-reasoner that curates, critiques, and synthesizes responses. The deliberation is transparent, auditable, and designed to surface epistemic disagreement and convergence but not necessarily to execute tasks or maximize efficiency.

In this sense, the Council architecture is best understood as a deliberative ensemble, optimized for epistemic robustness rather than efficiency (as in MoE) or autonomy (as in agentic systems). It borrows the modularity of MoE, the multi-agent structure of agentic systems, and the interpretability of human deliberation, but combines them toward a unique end: reasoning through dialogue to improve reliability, not just to act or compute.

A common communication architecture in a multi-agent system is that of agents that are connected to a middle agent. AI agents that are completing a task may have varying levels of information, with the middle agent (e.g., facilitator, mediator) coordinating between other agents and providing each agent with services that it may need.[42–46] In contrast, an alternative multi-agent architecture, studied by others [58–60] and developed for this study, provides all AI agents with full information, equal guidance for reasoning (i.e., same CoT prompts), and awareness of each other’s responses communicated to them through the Facilitator agent. This architecture simulates a council, where each AI member can respond to a query, and if there is variability in responses, deliberate divergent responses through the Facilitator to reach a consensus response. Because the deliberation of the Council occurs through a number of different steps, future directions of research can explore the mechanisms of the Council’s response generation, including characterizing and evaluating each stage of the algorithm (i.e., initial response to the USMLE question, the response of the LLM to compare responses for agreement, the response of the LLM to summarization, the response of the LLM to generating a clarifying question, and the subsequent responses of the Council of AI Agents to the new prompt). Additionally, the current study held all LLM attributes (i.e., prompt, temperature, maximum tokens, and top_p) constant between each instantiation to focus the evaluation on the presence of interaction between Council members and the resulting performance. However, a future direction of research can evaluate the impact of modifications to different parameters individually or in concert, with all or part of the Council embodying the changes.

While our study employed five AI agents due to token limitations and system constraints, varying the number of council members may meaningfully influence performance. Increasing the number of agents could enhance response diversity and improve the likelihood of arriving at a correct consensus, especially when agents disagree or offer complementary rationales. However, as more agents are added, the benefit may diminish if additional responses become redundant or if the Facilitator struggles to synthesize increasingly complex deliberations. Larger councils may also introduce greater computational costs and longer deliberation times. Future work could explore how performance scales with different council sizes and whether an optimal number of agents exists for balancing accuracy, efficiency, and reasoning quality.

Limitations

The Council of AIs approach, while effective, requires significant computational resources. Each deliberation involves multiple full forward passes through large-scale LLMs, significantly increasing both time and resource demands compared to single-instance querying. On average, answering a single USMLE question using the Council architecture (consisting of five GPT-4 agents and a Facilitator) required approximately 3–7 minutes, depending on the number of deliberation rounds, with some questions taking up to 15 minutes to achieve a consensus. Latency time increases with each additional instantiation of the LLM. This presents a significant challenge in real-time decision-making scenarios, especially when multiple rounds of sampling are required. It is possible to eliminate additional latency time while increasing the number of Council members by employing a strategy of synchronous sampling through multiple instantiations of the LLM API, each with different API keys. This approach would allow for parallel processing of responses, substantially reducing the overall time taken to reach a consensus. Through parallel computing, it would be feasible to maintain the depth and quality of analysis without sacrificing response time. For now, this system would work best for non-emergent scenarios that have a tolerance for the time needed by the Council for deliberation and achieving consensus. Future studies could focus on optimizing the efficiency of such systems and exploring their applicability in different domains.

Token limits in LLMs are a technical constraint that can limit the depth of analysis, particularly when dealing with complex or lengthy prompts. In our study token limits imposed practical constraints on both the number of Council members and the structure of deliberations, as each response needed to be passed through the Facilitator AI for summarization and reflection. To conserve tokens, Council members did not receive each other’s raw outputs directly; instead, they interacted through a Facilitator who synthesized responses for iterative feedback. While this strategy was motivated by efficiency, it remains uncertain whether full message broadcasting among agents would have improved performance or introduced unnecessary redundancy. In fact, it is possible that the Facilitator-mediated structure may have supported more coherent deliberation by filtering and organizing contributions in a focused manner. Additionally, while it is possible that token limits exerted question-specific effects, we did not stratify performance by item length or complexity, as no standardized metric currently exists for assessing difficulty across USMLE questions. However, future studies might explore whether token constraints disproportionately impact more detailed or multi-step questions and evaluate whether expanded token contexts (e.g., via more recent LLM versions or hierarchical deliberation models) can mitigate such effects.

The exclusive use of OpenAI’s GPT-4 in our Council of AI approach was a strategic choice in this study. Future studies can explore the inclusion of different LLMs, each with unique training datasets, algorithms, and probabilistic models, which may possibly lead to more robust and well-rounded consensus. The variation in the success rate of achieving consensus across different exam steps (Step 1, 2, and 3) suggests challenges in maintaining AI consistency across different contexts and that AI systems might require tailored approaches depending on the specific nature and complexity of the tasks at hand. Testing in other datasets, such as MedQA, may help with more direct comparison to other LLMs in the future. While this study focused exclusively on multiple-choice questions without images or tables, future research is needed to determine whether the Council of AIs framework generalizes to other question formats such as clinical vignettes or visual question-answering tasks. These formats may involve more complex reasoning, ambiguity in correct responses, or multimodal inputs, which could present new challenges for consensus-building and response clustering. The current findings do not provide evidence for performance in those contexts, but they suggest that exploring deliberative multi-agent strategies in more diverse settings may be a valuable direction for future investigation. While we selected the present test question set to ensure all questions were from after GPT-4’s training period, a possible limitation is that sample exam questions could be highly similar to older sample exam questions, in which case the LLM may already have encountered highly similar questions in its training data. It is important to note that the GPT-4 model used in this study was the initial version which later came to be known as ‘gpt-4-0314’ to distinguish it from subsequent updates to GPT-4. According to the GPT -4 system card and OpenAI’s technical documentation, GPT-4’s training data cutoff occurred in September 2021, and the model “generally lacks knowledge of events that have occurred after the vast majority of its data cuts off” and “does not learn from its experience”.[29,61] While OpenAI notes that gpt-4–0314 was released in March 2023 and remained unchanged through its deprecation in June 2024, and while there is no indication from OpenAI that such materials were part of the model’s post-training updates, it is not possible to definitively confirm whether specific documents (such as the June 2022 USMLE sample exam used in this study) were included in any fine-tuning processes. Given that there may be a possibility that despite OpenAI documentation later text corpora may have been included in fine-tuning processes, the purpose of this study was not to assess GPT-4’s memorization or static knowledge. Rather, the objective of this study was to evaluate how a deliberative architecture (i.e., a Council of AIs) can improve accuracy by integrating diverse reasoning paths, even among instances of the same model. Thus, even if agent responses reflected memorized content, the observed performance gains are attributable to the structure of deliberation rather than to any individual agent’s latent knowledge.

Ethical considerations

Ethical considerations around the use of AI in professional and decision-making contexts need to be addressed, ensuring that the deployment of such technologies is responsible and beneficial to society. A Council of AI approach may facilitate integration of voices of underrepresented communities as council members to try to reduce bias, which may be represented within purposefully trained smaller language models or through fine tuning of LLMs. However, LLMs themselves have an inherent limitation in that they are trained on a written corpus, which excludes communities whose knowledge may be codified in a diversity of non-written media including experiential transmissions of knowledge, oral histories, performative representations, and more. Future studies can explore the extent to which rapidly adopted AI systems are disconnecting communities from their own realities and reshaping community knowledge systems toward more hegemonic representations.[62] Additionally, studies also need to be conducted to formalize AI impact assessment on knowledge systems.

The Council of AIs framework offers a novel avenue for mitigating bias in LLMs, particularly in multicultural and multilingual contexts. By facilitating structured deliberation among independently reasoning agents, the architecture enables disagreement to surface and be explicitly negotiated, potentially counterbalancing the influence of any single model’s latent cultural or epistemic biases. Importantly, beyond the content of reasoning itself, non-reasoning attributes, such as the degree of epistemic rigidity, or resistance to updating a prior position, can also shape group consensus. In some cases, rigidity may steer the Council toward suboptimal or biased outcomes if a less flexible agent dominates. However, in other contexts, especially when the training corpus underrepresents marginalized voices, epistemic rigidity may act as a protective feature preserving minority-informed perspectives from being prematurely overwritten during deliberation.

Recognizing these dynamics is essential for designing equitable systems. To ensure transparency and accountability in high-stakes applications such as healthcare, Council-based architectures should provide access to deliberation transcripts, rationales for final decisions, and support external auditability. Further, intentionally including models with diverse training data, cultural grounding, or alignment objectives may help surface a broader range of perspectives, mitigate systemic bias, and help make AI systems both reliable and ethically robust.

Future research priorities for council of AIs implementation

While this study demonstrates the potential of collaborative AI approaches in medical decision-making, several critical research questions must be addressed to optimize the framework for clinical deployment. These research priorities span technical optimization, clinical integration challenges, and real-world implementation considerations. Table 6 outlines specific research questions and corresponding studies needed to advance the Council of AIs from experimental validation to practical clinical application, with particular emphasis on addressing computational efficiency, real-time decision-making requirements, and human-AI collaboration protocols.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Priority Research Questions for Council of AIs Clinical Implementation.

https://doi.org/10.1371/journal.pdig.0000787.t006