Human-supervised, large language model-based clinical decision support aligned to national newborn protocols in Kenya: a pragmatic, early-stage evaluation
Résumé
Abstract Introduction Timely, protocol-adherent clinical decisions are crucial for reducing neonatal mortality in low-resource settings. Translating extensive national guidelines into bedside practice remains challenging. Objective We developed and evaluated AIFYA, a human-supervised, large language model (LLM)-based clinical decision support system (CDSS) aligned with Kenya’s national newborn care protocols. Methods This prospective, mixed-methods, early-stage evaluation, guided by the DECIDE-AI framework, embedded AIFYA into routine workflows at two public health facilities (Level 5 and Level 4) in Bungoma County, Kenya, from September 2024 to June 2025. Primary outcomes were: (1) adoption, measured by cumulative neonatal cases managed; (2) training reach, assessed by credentialed healthcare workers (HCWs); and (3) guideline and citation concordance, evaluated through blinded review of 118 AI-generated recommendations by two neonatologists, with adjudication by a third. Secondary outcomes included protocol adherence and triage-to-decision time Results A total of 50 HCWs were trained, and 550 neonatal cases were managed over 10 months. Among surveyed HCWs (n = 33), 76% were female (mean age 32.1 years). Expert review found 75% of recommendations were correct and 15% partially correct, with strong interrater reliability (weighted Cohen’s kappa 0.85; 95% CI 0.79–0.91). Citation accuracy was 96%. In 40 complex dosing scenarios, 75% of outputs were rated correct. The median triage-to-decision time was 23 minutes (IQR 18–31). Implementation was supported by an offline-first architecture and a facility-based coaching model, sustaining engagement despite staff turnover. Conclusion A human-supervised AI CDSS, directly and transparently anchored to national clinical guidelines, can be successfully implemented in routine, low-resource neonatal care settings. The system demonstrated high user adoption and strong expert-rated concordance. The high citation accuracy is a critical feature that builds clinical trust, ensuring safety and enabling auditable AI. These findings provide a robust foundation for progression to controlled, multi-site trials to evaluate clinical effectiveness.
Citer ce document
Exporter : BibTeX · RIS (Zotero, Mendeley, EndNote)
Accès au document
Texte intégral archivé sur SPHAERO, en accès ouvert : lisible gratuitement en ligne après connexion. Le document reste aussi disponible à sa source. Se connecter
Voir l'article sur le site de la revueLicence et provenance
Licence : CC BY
Notice moissonnée depuis OpenAlex le 30/09/2026. Le document reste hébergé par sa source.
Voir le document à la source →