Publication Radar · Source metadata
Benchmarking decision models against generative large language models on medical multiple-choice questions
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-10-30 · Indexed by Pub-AI: 2026-10-30 UTC
Benchmarking decision models against generative large language models on medical multiple-choice questions
First announced by arXiv: 2026-10-30 · Original record ↗
- arxiv
- 2610.38448
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-10-30 · First detected: 2026-10-30 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.38448v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Medical workflows often require selecting among predefined options. We compared 13 decision models with 15 generative large language model configurations, including different reasoning settings, using 1,273 four-option MedQA questions under text-only conditions. Accuracy ranged from 67.71% to 89.08% for decision models and from 78.32% to 96.86% for generative configurations. Among the tested configurations, Mercury Decide had the lowest mean final-attempt cost (USD 0.0069 per 1,000 questions), while Clef Omni had the shortest median request duration (0.291 seconds). Decision-model probabilities distinguished correct from incorrect answers, but error rates among answers assigned probabilities of at least 0.90 ranged from 1.71% to 12.77%. Answer agreement across three runs ranged from 92.30% to 100%, while probability repeatability varied. Decision models offered resource advantages alongside lower peak accuracy; selecting models for medical workflows requires considering probability quality and repeatability together with accuracy and the resources needed to detect and correct errors.