The Legal Risk of AI Misinforming Your Citizens

By Tessar Napitupulu, Founder of PT Arfadia Digital Indonesia and Indonesia's GEO pioneer since 2023.

A tribunal has already ruled that an organisation is legally liable for its own chatbot's wrong answers, and a New York City government chatbot operated for two years while telling small business owners they could legally take a cut of employee tips. Neither case is hypothetical, both are decided or documented fact, and neither one tells you, precisely, what happens when the chatbot misinforming your citizens is ChatGPT or Gemini, not something your agency built. That gap is exactly where government communications teams currently sit.

The Case That Changed the Conversation

In Moffatt v. Air Canada, a Canadian tribunal held the airline liable for negligent misrepresentation after its AI chatbot gave a passenger incorrect information about bereavement fare policy. Air Canada argued the chatbot was "a separate legal entity responsible for its own actions." The tribunal rejected that argument outright, ruling it made no difference whether the information came from a static page or a chatbot. The operator, not the software vendor, owns everything its automated agent tells a user.

That principle is binding in Canada and persuasive elsewhere, but it applies specifically to a chatbot the organisation itself deployed and controlled. It says nothing directly about what happens when a citizen asks ChatGPT, a system the government did not build and cannot configure, and gets a wrong answer about a government procedure.

When the Chatbot Is the Government's Own

New York City's "MyCity" chatbot, built on Microsoft Azure AI, advised small business owners they could legally take a cut of workers' tips, refuse cash payments in violation of a 2020 local law, and deny Section 8 housing vouchers, illegal since 2008. Legal Services NYC called the advice "dangerously inaccurate." The chatbot stayed operational for over two years, at a total project cost cited above USD 600,000, before the incoming mayor shut it down in February 2026. It is now the most-cited municipal AI failure in the United States and has directly accelerated city-level AI governance frameworks.

This case sits squarely inside the Moffatt v. Air Canada logic: NYC built and operated the chatbot, so NYC owns what it said. The legal exposure here is comparatively clear, even without a specific AI liability statute, because the chatbot is functionally indistinguishable from any other government communication channel the city controls.

The Harder Question: What About ChatGPT, Not Your Own Bot?

Here is where the legal picture genuinely thins out. Moffatt v. Air Canada covers an organisation's own AI channel. It does not resolve what happens when a citizen asks a third-party engine, one your agency did not build, does not operate, and cannot configure, and that engine misrepresents your public content. This specific question remains legally unsettled in Canada, the United States, the EU and Indonesia alike. Anyone who tells you it is already resolved is overstating the state of the law.

What does exist is a documented pattern of AI engines getting procedural and civic content wrong at a rate that makes the question urgent regardless of how it eventually resolves. A benchmark testing 22,000-plus citizen queries against 11 leading LLMs found high variance and a long tail of significantly wrong answers, with near-zero abstention even when a model should have declined to answer. A Spanish traveller missed a flight to Puerto Rico after ChatGPT omitted a mandatory US travel authorisation from its answer, a case reported by HuffPost in August 2025. Separately, CNBC reported in March 2026 on documented cases of ChatGPT giving "almost right" tax filing answers, confident enough to look correct, wrong enough to produce a filing error if followed literally. None of these failures were caused by a government-operated chatbot. All of them produced real civic harm.

The trend line is not encouraging. NewsGuard's AI Misinformation Monitor found that ten leading chatbots collectively repeated false claims 40.33 percent of the time in December 2024, up from 18 percent a year earlier. The mechanism behind that degradation matters for government content specifically: models that previously declined to answer roughly a third of sensitive queries now answer almost all of them, which increases the surface area for error precisely where citizens are asking about procedures, eligibility and deadlines rather than opinions. A model that used to say "I'm not certain, check the official source" now more often just answers, correctly or not.

What Indonesian Law Actually Says Today

Indonesia has no AI-specific liability statute equivalent to the EU AI Act or New York's proposed bill. Three existing legal frameworks are the closest available analogues, and none of them was written with this scenario in mind.

Framework	What It Covers	Relevance to AI Misinformation
KUH Perdata Pasal 1365	Perbuatan melawan hukum, unlawful act doctrine	A civil claim is theoretically available if a citizen suffers demonstrable harm from relying on outdated content an agency tolerated AI citing
UU KIP, Law 14/2008	Public bodies must provide accurate, complete, updated public information	Operating a domain that AI engines cite with superseded procedural information may itself be a KIP compliance failure
UU KIP Pasal 55	Criminal liability, up to 1 year imprisonment and/or Rp5,000,000 fine	Deliberate creation or dissemination of false/misleading information causing harm, not passive oversight
Sovereign immunity	Broader immunity for government institutions than commercial entities	Applies to discretionary policy decisions, not to ministerial duties like providing accurate procedural information

The practical reading: an Indonesian government institution almost certainly cannot hide behind "the AI said it, not us" as a complete defence, given UU KIP's accuracy mandate applies to the information itself regardless of which channel repeats it. But nobody should represent this as settled case law, because no Indonesian court has yet ruled on a case matching this fact pattern.

Picture the illustrative scenario a legal team would actually face. A citizen asks ChatGPT about eligibility for a business permit subsidy, gets an answer based on a regulation that was superseded eight months earlier, applies on that basis, and is rejected. The citizen never visited the agency's website. The agency never operated the chatbot that misinformed them. Under KUH Perdata 1365, the citizen's strongest available argument is not against the AI vendor, but against the agency, for allowing the superseded version of the regulation to remain the most citable version anywhere online, in violation of UU KIP's currency obligation. That argument does not require the AI liability question to be resolved at all. It only requires showing the agency's own content was stale and discoverable in that state.

That is the quiet reframe worth sitting with: even in a legal environment where third-party AI liability is completely unsettled, an agency's own failure to keep its content current and structurally citable is independently actionable under law Indonesia has had since 2008.

The Criminal Penalty Almost Nobody Mentions

UU KIP does not stop at civil accuracy obligations. Pasal 55 of the law creates criminal liability, up to one year of imprisonment and a fine of up to Rp5,000,000, for anyone who deliberately creates or disseminates public information that is false or misleading in a way that causes harm to another party. This is a criminal provision, not a civil one, sitting inside the same 2008 law most agencies treat purely as a disclosure obligation.

The word "deliberately" (dengan sengaja) matters and should not be glossed over. Pasal 55 does not automatically apply to an agency that simply failed to catch an AI engine repeating a regulation it never updated, since that is an omission, not a deliberate act of creating or spreading false information. It would apply far more directly to a scenario where staff knowingly published or continued to circulate information they understood to be false or superseded, aware it could mislead the public. That is a narrower, more specific fact pattern than "an AI got something wrong," and conflating the two overstates the exposure.

What Pasal 55 does establish, independent of the AI question entirely, is the ceiling Indonesian law places on deliberate public misinformation from anyone, including a public official. It is a useful data point for a legal or compliance team weighing how seriously to treat information accuracy as a discipline, and a reminder that the UU KIP conversation in government digital work is not purely administrative. It has a criminal backstop, even if that backstop targets intent rather than oversight.

The EU Is Already Regulating This

The EU AI Act's Annex III classifies AI systems used in essential public services as "high risk," which brings data governance, transparency and accuracy obligations that a generic probabilistic chatbot cannot meet by design. Enforcement became active in August 2026. Indonesia has no equivalent yet, but the direction of travel among regulators globally, alongside the NY Senate bill creating a private right of action for chatbot-given professional advice, suggests government bodies elsewhere are moving toward formal accountability faster than most Indonesian institutions have started planning for it.

Regulators are not waiting for legislation to catch up before acting. A Munich regional court issued a temporary injunction against Google after its AI Overviews generated what the court described as "independent, new, and substantive statements" that defamed two publishers, a ruling that treats an AI-generated summary as its own actionable statement rather than a neutral aggregation of existing content. The European Commission opened a formal investigation into AI Overviews under competition law in December 2025. Google itself restricted AI Overviews on certain health searches in January 2026, following a Guardian investigation into inaccurate medical answers. None of these three developments involve a government institution directly, but together they establish that AI Overviews and answer engines are being treated as sources of liability in their own right, not shielded by "it's just an algorithm" framing, in multiple jurisdictions simultaneously.

For a government body, that trend cuts both ways. It strengthens the case that a citizen harmed by a wrong AI answer about a government service has a plausible path to some form of accountability, somewhere in the system, eventually. It also means the agency whose information is being misrepresented has grounds to argue the harm did not originate with them, provided their own published content was accurate and current at the time. The second argument only survives scrutiny if the agency can actually demonstrate its content was correct, current, and reasonably discoverable, which circles back to the same structural fixes that reduce misinformation risk in the first place.

GEO as Liability Mitigation, Not Just Marketing

Every mitigation step above is also, functionally, a Generative Engine Optimization program: structured content, visible currency signals, and monitored citation accuracy. That overlap is not a coincidence. The reason official information gets cited less than it should is largely structural, and the reason misinformation risk persists is largely the same structural gap left unaddressed. Framing GEO purely as a visibility or growth initiative undersells what it actually does for a government client, which is closer to a compliance and risk function that happens to also improve findability.

Tessar Napitupulu's Cited or Silent covers the regulatory and compliance dimension of GEO in more depth, including how measurement frameworks like RoGEO adapt when the stakes are civic rather than commercial, free to read. For agencies weighing whether to formalise a GEO program specifically to manage this exposure, Arfadia's Government GEO service starts with the same baseline audit described above, and pairs naturally with the structural fixes covered in our piece on why third-party sites currently outrank official government pages.

---

Frequently Asked Questions

Has any Indonesian court actually ruled on an AI misinformation case involving a government body?

Not that has been documented publicly. This entire area remains legally untested in Indonesia, which is precisely why the risk should be managed proactively through content accuracy and monitoring rather than assumed away.

Does this apply to BUMN the same way it applies to a ministry?

The UU KIP accuracy obligation applies to public bodies broadly, and BUMN carry a public mandate even though they follow corporate rather than LKPP procurement rules. The reputational and civic-harm exposure is functionally similar even where the specific legal category differs.

If we deploy our own chatbot, does Moffatt v. Air Canada apply directly?

The core principle, that an organisation is responsible for what its own automated channel communicates, is the closest available precedent and would likely be persuasive in an Indonesian dispute, though no Indonesian ruling has tested it directly.

Is llms.txt or better schema a legal safeguard?

No. These are structural tools that make your official information more likely to be the one AI engines cite. They reduce the practical risk of misinformation reaching citizens; they are not a legal shield and should not be described as one.

Who inside a ministry or BUMN should own this risk?

In practice it usually sits between legal/compliance, which understands UU KIP obligations, and the digital communications team, which controls the actual page content. Neither owns it by default today, which is itself part of the problem.

---