CausalOdyssey: Multi-Agent Local LLM Reasoning with Per-Patient Causal Structure for Rare Disease Retrieval

Abstract

Patients with rare diseases often wait years for a diagnosis. Existing HPO-based retrieval tools narrow the differential to a shortlist, but a clinician reading that shortlist has no signal for which candidate to trust or what to test next. We present CausalOdyssey, a local four-agent pipeline on Gemma 4 26B built over a classical IC-weighted gene-rescue backbone: a Diagnostician reranks the slate via chain-of-thought over per-patient FCI causal edges and Reactome mechanism context; a Verifier computes deterministic evidence against the top-1 (missing pathognomonic phenotypes, pathway chain closure) and has Gemma issue a confirm/flag/refute verdict over it; an Interrogator selects one follow-up HPO by split-variance scoring; a Narrator emits citation-validated clinical prose. Only the Diagnostician changes the ranking; the other three agents annotate it. On RareBench n=1,122, reranking a top-30 slate lifts R@1 by +12.57 pp over our structural baseline ([+10.07, +14.97]; p=8.3e-24). A no-LLM variant of that baseline, without its inverse disease-count weight, reaches a similar pooled R@1 (36.3% vs 35.3%): the LLM rerank is far better on metabolic cases (RAMEDIS, +16.0 pp) and far worse on case-report corpora (LIRICAL, MME), so its value is corpus-dependent. We also propose a masked-phenotype protocol for evaluating follow-up suggestions without an annotation-based oracle; under it, the Interrogator asks for a patient's hidden phenotypes no more often than simple baselines (0.6% vs 1.2%), although a perfect follow-up would add 11 pp at top-1. The Verifier's verdict separates correct from incorrect top-1s (51.4% vs 12.1% top-1 accuracy under confirm vs refute). For 30% of patients the gold diagnosis is outside our top-30, though LIRICAL retrieves over half of these. All inference is local; no patient data leaves the machine. Demo code and a live demo are public.