An Expert Tested America.gov's Health Answers. What That Means for Your Medicare Questions

America.gov now answers health and coverage questions in plain language, citing federal agencies directly. That sounds like a faster way to settle a Medicare question than scrolling CMS pages — and sometimes it is. But a detailed independent test published on September 30, 2026 found that the portal’s answers are only as good as the federal pages they are built from, and it identified one specific failure mode that matters for coverage decisions. This article summarizes what the test found and turns it into a short verification routine you can run before acting on any America.gov answer about your coverage.

What was tested

Dr. Céline Gounder, an infectious-disease physician and medical journalist, sent America.gov 37 health questions in the days after its September 29 launch and published the transcript of what it got right and wrong. The portal promises “simple answers only from official government sources,” and its health answers cite the CDC, FDA, NIH and CMS — the agencies whose pages it was built on. It runs on commercial AI models — press reporting identifies Gemini and xAI’s Grok, and Google has confirmed its involvement, though the chatbot itself declined to name its software — drawing on roughly 29,000 government websites.

Two of her findings bear directly on coverage questions, so they are worth understanding as mechanics, not politics.

The Medicare answer that got a federal rule wrong

Asked about insulin costs, the chatbot said most Part D plans cap insulin at $35 for a month’s supply — “participating plans; check your formulary.” That describes the voluntary pilot program Medicare ran starting in 2021, which about 38% of Part D plans joined, according to KFF. It leaves out the law that replaced it: the Inflation Reduction Act of 2022 extended the $35 monthly cap to every Part D plan, effective 2023.

The error runs in the discouraging direction. A reader could come away thinking the cap depends on their plan’s choice, when in fact no Part D plan can charge more than $35 a month for a month’s supply of a covered insulin product. The chatbot had the pilot’s fine print without the statute that superseded it. A page-level AI that assembles answers from government web pages can reproduce an outdated framing even when the current rule is more favorable — which is exactly the kind of error that costs you money if you take it at face value.

The “newest official page wins” rule — and where it breaks

Gounder also probed how the chatbot decides between conflicting official sources. It said it prefers, first, the agency that runs the program for a final determination, and after that the most recent official source. That rule sounds sensible, and most of the time it is.

It fails when a court has paused the newest page. In December 2025, CDC’s reconstituted vaccine advisory committee voted to end the universal hepatitis B birth-dose recommendation, and a January 2026 overhaul of the childhood schedule followed. On March 16, 2026, a federal judge paused those changes pending the underlying lawsuit — the earlier schedule, including the birth dose, went back into effect, and the government has appealed. Asked about the newborn hepatitis B shot three different ways, the chatbot pointed to the December 2025 CDC and HHS fact sheets every time and never mentioned the court order.

The lesson generalizes beyond vaccines: any federal rule can be superseded by a court order, a new statute or a correction that the agency web page does not foreground. An AI that treats page recency as the tiebreaker will silently miss that layer. (Federal health programs are national, but note that state-administered programs like Medicaid operate under rules that vary by state — a one-size answer about a Medicaid appeal may not match your state’s process.)

What it got right

For balance, the same test found real strengths. Typing chest pain and a numb left arm produced “Call 9-1-1 now” as the first words. A toddler swallowing blood pressure pills got Poison Control’s number first. The chatbot walked through Medicare Part D enrollment windows and Medicaid appeals correctly, its Spanish answers matched its English ones, and it gave accurate answers on raw milk and smoking. As a front door — triage, forms, deadlines, which agency owns your problem — it performed.

A verification routine for coverage answers

If you use America.gov for a Medicare or coverage question, run four checks before acting:

1. Ask what the answer assumes. If a dollar figure appears, ask whether it is a statute, a pilot program or a plan choice. The insulin error happened because the chatbot reported a program’s fine print without the law that replaced it.
2. Check the rule is in force. Search CMS.gov for the rule itself, not just a summary — and if the answer concerns a recent policy change, ask whether any court has paused it.
3. Never take plan-specific answers from a general chatbot. Your formulary, your premium, your network: these live in your plan’s documents and in the Medicare Plan Finder with your actual drugs entered. No national AI answer can substitute for either.
4. Keep medical questions with your clinician. The test’s own bottom line applies here: use America.gov to find forms and deadlines; take health questions to your healthcare provider.

The background to all of this — what America.gov is, what the executive order actually requires, and which services are slated to move onto it — is in our overview, What Is America.gov?, and the enrollment-side detail is in Medicare Enrollment Is Moving Toward America.gov: What Changes and What Doesn’t.

Bottom line

America.gov is a genuinely useful front door, and its guardrails on emergencies worked in testing. But its coverage answers inherit every inconsistency in federal web pages and add one of its own: a source-preference rule that can lag a court order. Treat its Medicare answers the way you would treat any good starting point — check the dollar figure against CMS.gov or your plan’s documents before you act on it.

This article is general information about a public tool and federal program rules, not advice about your specific coverage. For decisions about your own plan, use the Medicare Plan Finder or call 1-800-MEDICARE.

Part of our America.gov series: the portal’s legal basis and scope is covered in What Is America.gov?, and the enrollment-side detail in Medicare Enrollment Is Moving Toward America.gov.