A federal court ordered the Department of Health and Human Services to cease using artificial intelligence to generate citations to nonexistent studies or to intentionally misrepresent existing research in grant solicitation documents.
The ruling addresses a pattern in which HHS relied on AI systems to draft grant announcements without adequate human review. The AI tools produced fabricated academic citations and distorted findings from legitimate studies to support predetermined policy positions. Grant applicants discovered the false and misleading references only after submitting proposals based on flawed premises in the solicitation materials.
The court found that HHS violated the Administrative Procedure Act by issuing grant solicitations containing material misrepresentations about the scientific record. The agency's use of AI without meaningful verification mechanisms created systematic accuracy failures that undermined the integrity of the competitive grant process. Applicants cannot fairly compete when foundational documents contain invented sources or twisted interpretations of actual research.
HHS must now implement mandatory human review protocols for all AI-generated content in grant solicitations before publication. The agency must verify that every citation refers to an actual study and accurately represents its findings. Additionally, HHS must notify affected grant applicants and provide remedial procedures for those whose proposals were evaluated under solicitations containing false information.
This decision carries implications beyond the grant process. Federal agencies increasingly rely on AI tools for drafting regulatory documents, policy guidance, and official communications. The court's ruling establishes that agencies cannot outsource accuracy responsibilities to automated systems. Human expertise and accountability remain legally required.
The ruling also reflects growing judicial skepticism about AI reliability in high-stakes government functions. When AI outputs directly affect individual rights or competitive opportunities, courts will demand rigorous validation. Agencies cannot deploy AI as a cost-cutting measure that sacrifices accuracy or due process.