AI enters public health without proper oversight

Clinical takeaway: Without required validation, public health AI tends to perform worst for the patients least represented in its training data. The authors call for validation before deployment, not after. 

Health departments squeezed by workforce shortages and flat budgets have found an eager partner in AI. Algorithms now scan emergency department visits for anomalous symptom patterns, sort neighborhoods for outreach priority, and answer patients' questions through chatbots. That is work that once required staff the agencies no longer have. The COVID-19 pandemic normalized much of this almost overnight, as agencies leaned on automated tools to forecast surges, track hospital capacity, and push vaccine messaging. 

But much of what AI does in public health, including surveillance analytics and risk scoring for outreach, falls outside any comparable pathway. Adoption has proceeded anyway. In an analytic essay in the American Journal of Public Health, two pediatricians examine that mismatch and argue the problem is structural rather than technical. Vendor priorities of speed and scale, they contend, substitute for the safeguards other health interventions must clear before reaching a single patient. 

Pharmaceuticals, vaccines, and clinical interventions must clear premarket review, safety monitoring, and efficacy standards, while most AI systems answer to no equivalent, leaving a de facto system of industry self-regulation. That arrangement predictably favors speed to market, scale, and return on investment over equity, safety, and community safeguards. Voluntary ethics frameworks exist but lack enforcement, and agencies stretched thin are poorly positioned to interrogate vendor claims or negotiate contracts that put population welfare first. 

The authors point to the pulse oximeter as a case study. Its light-absorption algorithms, developed predominantly in lighter-skinned populations, systematically overestimated oxygen saturation in patients with darker skin during the COVID-19 pandemic, so the patients most susceptible to severe disease were the least likely to be triaged accurately. 

They treat this as the template for automation bias: algorithms marketed as more objective than human judgment do not remove bias but systematize it. Discrimination embedded in a system presented as neutral becomes harder to recognize and contest. The same logic extends to behavioral health apps configured to payer or employer priorities, which the essay describes as delivering standardized directives dressed as personalized, evidence-based guidance. 

The risks are not interchangeable across populations, the authors contend, and neither are the fixes. Children face permanent digital records built from behavioral screening they cannot consent to; Indigenous communities face AI deployment that ignores data sovereignty frameworks; older adults are misclassified by models trained on younger cohorts; undocumented families may avoid care entirely when AI-enabled data collection raises the specter of immigration enforcement. The essay's program follows from that diversity: mandatory equity impact assessments, validation in the communities of intended use, national standards for transparency and equity testing, and human authority retained over consequential decisions. 

"Responsible use of AI in public health practice must therefore be calibrated to this diversity and combine cross-cutting governance with population-specific validation, community engagement, and safeguards tailored to each group," says Terry Adirim, MD, MPH, a pediatrician at the Uniformed Services University of the Health Sciences. 

FDA premarket testing covers some AI-enabled medical devices, but not surveillance analytics or outreach risk scoring. The authors also highlight audit templates from outside the life sciences: a National Institute of Standards and Technology risk framework, an Underwriters Laboratories autonomous vehicle standard, and Consumer Technology Association guidelines for trustworthy AI. The essay suggests these as models for minimum national standards, with tiered review by potential harm and equity assessments before AI is deployed. 

Source: Adirim T, et al. (2026 Sep 9) Am J Public Health. Ethical Use of Artificial Intelligence in Public Health: Mitigating Risks and Advancing Health Equity