Distributed Consensus for Uncertainty in Medical Diagnosis

Medical diagnosis using large language models (LLMs) like GPT-4 has shown significant promise with advancements such as MedPrompt, which leverages chain-of-thought (CoT) reasoning and choice-shuffling ensembles. However, the exponential growth in computational complexity with an increasing number of ensembles limits the scalability of such systems. This study re-implements MedPrompt from scratch and introduces a distributed framework using Ray to address these limitations. By parallelizing CoT reasoning, our approach achieves substantial reductions in processing time, enabling the use of more than 20 ensembles while maintaining state-of-the-art diagnostic accuracy of 90.6% on the MedQA dataset. Preliminary results indicate that our distributed system reduces processing time from over 4,800 seconds to less than 400 seconds for 20 ensembles.