A Cloud-Edge System for Multimodal Clinical Screening in Resource-Constrained Rural Settings

This paper introduces a cloud–edge collaborative architecture for multimodal clinical screening in resource-constrained rural settings. The system combines lightweight, domain-specific models on the edge that transform raw medical data into compact structured outputs, with a cloud LLM that synthesizes these outputs into clinical summaries. An LLM-based orchestrator dynamically selects diagnostic tools based on patient context, promoting relevant modality coverage without processing irrelevant inputs. The system was evaluated on 100 multimodal clinical cases spanning cardiac, obstetric, trauma, ophthalmology, and screening scenarios—including sparse-input presentations with missing modalities and dense-input presentations with many overlapping inputs—under three simulated network profiles (500 kbps–5 Mbps), reporting 95% confidence intervals throughout. The hybrid system attains the highest oracle accuracy (0.87–0.90) and the strongest factual grounding (KG precision up to 0.96), together with high coverage precision (0.95–0.99), while transmitting only 6.5 KB of structured evidence to the cloud—three orders of magnitude less than cloud-only baselines. It maintains bandwidth-invariant latency (25–38 s) at up to 15× lower token cost. These results highlight the role of architectural design in improving evidence selectivity and factual grounding, rather than merely reducing upload size, under deployment constraints.

The paper addresses the gap that medical AI capabilities remain largely inaccessible in rural settings due to three fundamental constraints: (1) the AI system must decide which tests to acquire and gather sufficient evidence within a single encounter, as follow-up visits are often delayed or unavailable; (2) it should reliably ingest and interpret diverse test modalities in the absence of on-site specialist oversight; and (3) it must operate under severe bandwidth and compute constraints, as rural broadband speeds often range from 0.5–25 Mbps with frequent dropouts. Existing approaches address these constraints only partially: domain-specific medical models achieve specialist-level accuracy on individual modalities but operate on single modalities in isolation, while Multimodal LLMs can reason across heterogeneous inputs but require raw medical data to be uploaded for inference, which is often infeasible under rural bandwidth constraints.

The paper's key insight is that the architectural boundary between perception and reasoning shapes not just efficiency but the character of clinical errors: hybrid systems produce more selective, better-grounded evidence and fewer hallucinations than both cloud-based agentic systems and multimodal LLMs reasoning directly over raw data.

The system comprises two layers connected by a lightweight communication protocol:

Edge Perception Layer: Seventeen specialized diagnostic tools spanning cardiac, obstetric, trauma/screening, and ophthalmic modalities, including carotid ultrasound (UltraBot), fetal ultrasound (FetalCLIP), 12-lead ECG (ECGNet), chest radiograph (ChestXRay), thyroid ultrasound (ThyroidSeg), CT scan (KidneyStone), extremity X-ray (BoneFracture), echocardiogram video (EchoNet), facial video (BigSmall), blood pressure (BP), and seven ophthalmology-oriented tools (retinal fundus grading, retinal OCT, fluorescein angiography vessel analysis, near-infrared retinal vessel imaging, slit-lamp assessment, B-scan ocular ultrasound, and retinal vessel-segmentation grader). Each tool produces structured outputs with modality-specific fields, confidence scores, and quality flags. Deterministic quality gates are applied, with MobileSAM as a lightweight fallback for segmentation quality issues.

Cloud Reasoning Layer: Receives accumulated evidence and patient context, including demographics, presenting symptoms, and medical history. The cloud LLM (Gemini or GPT) synthesizes these inputs into clinical summaries that integrate findings across modalities, account for confidence signals, and produce diagnostic considerations and next-step recommendations. It operates exclusively on structured outputs and does not access raw data.

Orchestrator: An LLM-based routing model that selects the next action given patient context, available tools, and accumulated evidence. It produces one of three decisions: request-tool (invoke additional tools, potentially in parallel), summarize (forward results to the cloud), or abstain (terminate with reacquisition guidance when evidence is insufficient). The orchestrator iteratively expands the evidence set and selects tools per case based on clinical relevance.

The key architectural invariant is that no raw image, video, or signal data crosses the edge–cloud boundary. All communication consists of structured JSON payloads (diagnostic labels, confidence scores, and quality flags), providing bandwidth guarantees independent of input size and restricting the cloud LLM to verifiable evidence.

The evaluation uses 100 multimodal clinical cases spanning diverse patient demographics (ages 19–88), acuity levels, and clinical presentations. Each case includes a patient profile, a set of relevant diagnostic inputs matched to the clinical context, and a set of red herring inputs from unrelated modalities designed to test tool selection specificity. Cases were generated via few-shot LLM prompting with the tool registry included as context, using Claude (which does not overlap with the evaluated model families Gemini and GPT). Cases are populated with real, multi-source clinical data: ECG signals from MIMIC-IV-ECG, echocardiography videos from EchoNet-Dynamic, blood pressure readings from MIMIC-IV Clinical, fetal ultrasound images from HC-18, and real-world ophthalmology case reports from EyeRounds.org.

Three network profiles model the uplink channel: Rural low (500 kbps base, 20 kbps floor, 2,000 kbps ceiling), Rural moderate (2,000 kbps base, 100 kbps floor, 10,000 kbps ceiling), and Rural good (5,000 kbps base, 500 kbps floor, 20,000 kbps ceiling), with bandwidth modeled as a log-normal process with periodic updates, discrete dropout events, and hard floor/ceiling bounds.

Six system configurations span two cloud LLM families (Gemini 2.5 Pro and GPT-5.4) and three architectural paradigms:

• Hybrid (edge + cloud): The proposed system with specialized models on edge hardware, LLM orchestrator for tool selection, and structured outputs sent to cloud LLM for synthesis.

• Agentic: Cloud-only multi-turn baseline where the cloud LLM iteratively requests specific images and signals across multiple rounds, uploading raw data on demand.

• Direct: Cloud-only single-shot baseline where all available inputs (both relevant and red herrings) are uploaded simultaneously with patient context.

Clinical quality is evaluated along four complementary axes: (1) Oracle Accuracy (micro-averaged accuracy of verifiable findings extracted via pattern matching against ground truth); (2) Input Coverage (recall of clinically relevant tools invoked and precision of irrelevant tools avoided); (3) KG Verification (medical claims verified against a MeSH-based knowledge graph through a five-stage cascade); (4) Reasoning Quality (decomposing summaries into reasoning steps and evaluating efficiency, factuality, and completeness following MedR-Bench).

Direct attains near-perfect coverage recall (0.98–1.00), but this is definitional rather than a merit: it uploads every available input, so it trivially receives all relevant modalities—along with all red herrings, which collapses its coverage precision to 0.57 and inflates its cloud payload by three orders of magnitude. Between the two selective systems, Hybrid and Agentic attain comparable, uniformly high coverage precision (Hybrid 0.95–0.98, Agentic 0.97–1.00). The decisive difference is recall: the agentic baseline consistently under-acquires, recovering only 0.60–0.75 of the relevant modalities, whereas the Hybrid orchestrator recovers 0.74–0.79. The gap is substantial in aggregate (mean recall 0.76 vs. 0.69) and largest for the Gemini family, where the agentic baseline misses roughly a third of the relevant modalities (recall 0.60–0.66; e.g., 0.662 vs. Hybrid's 0.789 under Rural Good). Hybrid also preserves all tool outputs locally on the edge device, so structured diagnostic evidence remains available even if the uplink fails.

Hybrid systems achieve the highest KG verification precision across all network profiles. Gemini Hybrid leads (0.949–0.957, stable across profiles), followed by GPT Hybrid (0.905–0.935). Cloud-only baselines are substantially lower: Agentic 0.742–0.857 and Direct 0.684–0.848. This gap reflects a fundamental architectural advantage: Hybrid systems reason over structured tool outputs with explicit confidence scores, constraining the LLM to claims grounded in verifiable evidence. The KG F1 picture is more nuanced: because Direct baselines ingest every input, their high recall of echoed findings lifts their F1, so Direct occasionally matches or exceeds Hybrid on F1. The consistent Hybrid advantage is in precision—i.e., not fabricating claims—rather than in recall of every mentionable finding.

Oracle accuracy tells a complementary story. Hybrid attains the highest oracle accuracy overall (Gemini 0.871–0.886, GPT 0.879–0.902). Among cloud-only baselines, GPT Direct is closest (0.844–0.863) but at up to 15× the token cost and 2–5× the latency; agentic baselines trail further (0.703–0.812).

Four recurring patterns of unsupported claims were observed in cloud-only systems:

1. Drug and protocol recommendations: Cloud-only systems frequently generate specific treatment recommendations (drug names, dosages, lab panels) that are clinically plausible but not grounded in the patient's diagnostic data. In the preeclampsia case, Gemini Agentic produces 14 unsupported claims including specific medications and laboratory tests. The Hybrid system avoids this because its summary prompt constrains the LLM to synthesize structured tool outputs rather than generate treatment plans from training data.

2. Visual over-interpretation: Direct-mode systems generate specific radiological or signal findings from unstructured visual interpretation. Hybrid systems are architecturally immune because the LLM never sees raw images.

3. Context leakage: Agentic systems that request too few inputs compensate by treating patient context (demographics, symptoms, history) as diagnostic findings.

4. Speculative clinical reasoning: Cloud-only systems generate plausible differential diagnoses or complications not supported by available evidence.

Both Hybrid variants achieve near-perfect accuracy on modalities with specialized edge tools: 100% for BP extraction, carotid stenosis, and thyroid nodule detection; 82–91% for bone fracture. Cloud-only systems show substantially lower accuracy, particularly for bone fracture detection (Gemini Agentic: 62% accuracy at 60% coverage) and carotid stenosis. ECG classification accuracy is consistent across all systems (83–86%), suggesting signal difficulty dominates over architecture.

Hybrid latency is essentially flat across bandwidth profiles: Gemini Hybrid varies only 33.1–38.2 s and GPT Hybrid 25.2–29.8 s. This stability arises because the Hybrid system transmits only 6.5 KB of structured JSON, making bandwidth irrelevant. In contrast, cloud-only baselines transmit 2–7 MB of raw data per case: Gemini Direct slows from 59.7 s (rural good) to 148.6 s (rural low), a 2.5× degradation. GPT Hybrid is the most token-efficient configuration at 1,860 tokens per case, a 15.4× reduction compared to GPT Direct (28,650 tokens). The bandwidth gain is uniform (1000×) because it is set by the structured-output protocol rather than the LLM.

The paper identifies two key architectural insights:

The Input Selection Problem: When an LLM controls its own data acquisition, it tends to under-acquire. Agentic systems recover only 60–75% of clinically relevant modalities, and when a model lacks sufficient input evidence, it compensates by generating plausible claims from its training knowledge rather than acknowledging insufficient data. The Hybrid orchestrator mitigates this by decoupling data acquisition from reasoning, raising relevant-modality recall to 74–79% while keeping claims far better grounded.

Bandwidth Invariance as a Clinical Guarantee: Hybrid latency is effectively independent of bandwidth (25–38 seconds across all three network profiles), providing a reliability guarantee that cloud-only systems cannot match. Furthermore, the Hybrid architecture offers graceful degradation: even if the cloud uplink fails entirely, all edge tool outputs persist locally as structured data.

Robustness Across LLM Families: The Hybrid architecture outperforms its cloud-only counterpart within both LLM families on factual grounding and oracle accuracy, suggesting the gain is not backbone-specific. GPT Hybrid is faster and more token-efficient, while Gemini Hybrid achieves higher KG-verification precision.

In Case 13 (a 55-year-old male with Marfan syndrome and bicuspid aortic valve presenting with tearing chest pain, unequal arm blood pressures, and diaphoresis—a textbook acute aortic dissection), the Hybrid system invoked all five relevant tools and produced a reasoning trajectory the reviewer characterized as clear, accurate, and with organized facts. Every diagnostic claim traced to a specific tool output, the system admitted it could not interpret the ECG data rather than fabricating a rhythm classification, and findings were organized in a structured hierarchy mirroring specialist reasoning. The cloud-only agentic configuration requested only two of five relevant modalities yet produced a summary of comparable length and confidence, with three critical failures: a hallucinated mild pericardial effusion (potentially altering management in harmful ways), omission of the severely elevated blood pressure (198/112 mmHg, the single most actionable finding), and the reviewer described the output as analogous to a less capable physician who is less well spoken, less organized, less reliable.

The paper acknowledges several limitations: (1) most cases attach real diagnostic data to synthetically constructed patient profiles, and the case-generation LLM sees the tool registry and may favor tool-based reasoning; (2) the 100 cases do not span all rural presentations (dermatological and musculoskeletal complaints remain underrepresented); (3) per-modality accuracy reflects in-distribution performance and does not predict behavior under rural domain shift; (4) KG verification is a factual-grounding proxy, not clinical adjudication, with heuristic similarity thresholds; (5) in every Hybrid configuration the orchestrator and synthesizer share an LLM family, so family-level differences cannot be attributed to a single stage; (6) the Case 13 study is a single-case, single-reviewer illustration, not an independent clinical evaluation.

The paper presents a cloud–edge architecture for multimodal clinical screening in resource-constrained rural settings. By combining edge-based specialized models with cloud-based reasoning over structured outputs, the system supports dynamic test selection and cross-modal synthesis while avoiding raw data transmission. Across 100 diverse clinical cases and three network conditions, the approach achieves the strongest factual grounding and competitive-to-best clinical accuracy with stable latency and substantially lower bandwidth and token cost than cloud-only baselines. These results highlight the importance of coordinating perception and reasoning under deployment constraints, and suggest that hybrid architectures are a practical path toward scalable multimodal clinical AI in low-resource settings.

Improvements to AI Systems:

1. Implement a perception-reasoning split with structured evidence passing. Instead of feeding raw data to a central LLM, deploy specialized edge models that output compact, structured JSON (labels, confidence scores, quality flags). The cloud LLM only receives these structured outputs, never raw images/video/signals. This eliminates hallucination sources from visual over-interpretation and reduces bandwidth by 1000×.

2. Add an LLM-based orchestrator that decouples tool selection from reasoning. Use a separate routing model that iteratively selects diagnostic tools based on patient context and accumulated evidence, with explicit decisions: request-tool, summarize, or abstain. This prevents under-acquisition (agentic systems only recovered 60–75% of relevant modalities vs. 74–79% for the orchestrator) and reduces speculative claims when evidence is insufficient.

3. Constrain the cloud LLM's output scope via prompt design. Restrict the synthesizer to describing findings from structured tool outputs only—explicitly forbidding treatment recommendations, drug dosages, lab panels, or differential diagnoses not directly supported by the evidence. This prevents the most common error pattern: clinically plausible but ungrounded claims (e.g., 14 unsupported claims in a preeclampsia case).

4. Include confidence scores and quality flags in all structured outputs. Force edge models to report uncertainty and data quality, and require the cloud LLM to acknowledge low-confidence or low-quality evidence explicitly (e.g., cannot interpret ECG rather than fabricating a rhythm). This improves factual grounding precision from 0.85 to 0.95.

5. Add deterministic quality gates with fallback models. For segmentation or signal quality issues, use lightweight fallback models (e.g., MobileSAM) to re-process or flag poor inputs before they reach the reasoning layer, preventing downstream errors from corrupted data.

6. Preserve all edge outputs locally. Store structured diagnostic evidence on the edge device even if the cloud uplink fails, enabling graceful degradation and later retrieval—a reliability guarantee cloud-only systems lack.

7. Optimize token efficiency by sending only structured summaries. The hybrid system uses 1,860 tokens per case vs. 28,650 for direct upload, a 15× reduction. This enables deployment on lower-cost LLM tiers and reduces latency variance across network conditions (25–38 s flat vs. 60–149 s degrading).

What the Improved AI System Can Do:

• Operate in bandwidth-constrained environments (0.5–5 Mbps) with latency independent of network speed, transmitting only 6.5 KB of structured evidence per case.

• Achieve 0.87–0.90 oracle accuracy and 0.95–0.96 KG-verification precision, outperforming cloud-only agentic and direct baselines on factual grounding while avoiding hallucinations.

• Select clinically relevant diagnostic tools with 74–79% recall and 95–98% precision, avoiding red herring inputs and under-acquisition.

• Provide reliable clinical summaries that trace every claim to a specific tool output, explicitly flag uncertainty, and avoid ungrounded treatment recommendations.

• Degrade gracefully: if cloud connectivity is lost, all edge outputs remain available locally as structured data for later upload or manual review.

• Scale across LLM families (Gemini, GPT) with consistent architectural advantages, suggesting backbone-agnostic robustness.

Medical AI has demonstrated specialist-level diagnostic accuracy, yet these capabilities remain largely inaccessible in resource-constrained rural settings where bandwidth is scarce, compute is limited, and clinical decision-making requires integrating heterogeneous modalities. We introduce a cloud--edge collaborative architecture that addresses these constraints: lightweight, domain-specific models on the edge transform raw medical data into compact structured outputs, while a cloud LLM synthesizes these outputs into clinical summaries. An LLM-based orchestrator dynamically selects diagnostic tools based on patient context, promoting comprehensive modality coverage without processing irrelevant inputs. We evaluate on 20 multimodal clinical cases spanning cardiac, obstetric, trauma, and screening scenarios under three simulated network profiles (500,kbps--5,Mbps). The hybrid system achieves 98--99% diagnostic tool recall with 92--96% precision, matches or exceeds cloud-only baselines on clinical accuracy, and maintains bandwidth-invariant latency (25--35,s) at 4--15x lower token cost. These results highlight the role of architectural design in enabling efficient multimodal integration and improving factual grounding compared to cloud-only approaches under deployment constraints.