AI Governance - The Rulebook We Forgot to Write
Sources
Somewhere between the launch of ChatGPT and the sudden emergence of “digital employees” who schedule meetings, write contracts, and casually hallucinate facts, we realized we may have made a mistake, not in building AI, but in skipping the part where we figured out how to govern it.
AI governance is the grown-up version of AI hype. It’s what happens when we stop asking what the model can do and start asking who controls it, who’s responsible when it breaks something, and whether we’re accidentally building a sociotechnical monster we won’t be able to put back in the box.
But like every good disaster story, this one starts with good intentions, plenty of warnings, and a recurring theme: we didn’t listen.
The Missing Infrastructure Beneath the Hype
Governance, to most people, sounds like a meeting that could’ve been an email. In the AI world, though, it’s the thing standing between innovation and catastrophe. It’s not just about rules. It’s about who makes the rules, how they’re enforced, and what happens when AI systems behave in ways their creators didn’t predict.
AI governance encompasses a range of aspects, including regulatory law, corporate accountability, technical audits, and ethical frameworks. It asks thorny questions. Who should be allowed to train large models? Should companies have to disclose what data they used? If a chatbot gives someone dangerous advice, who’s liable—if anyone? Should governments get access to frontier models for national security reasons? Should anyone?
The uncomfortable truth is that much of today’s AI infrastructure—especially the powerful models developed by OpenAI, Google DeepMind, Anthropic, and Meta—is being deployed faster than it’s being understood. And the guardrails? Mostly optional. Often fragile. Sometimes just vibes.
We’ve Been Here Before (But We Forgot the Ending)
The problem with technological revolutions is that they rarely feel dangerous until they’re irreversible. Think social media in the 2010s. Think Cambridge Analytica. Think of autonomous trading algorithms crashing markets in milliseconds. Or Amazon’s AI-powered hiring tool that quietly learned to penalize female applicants.
In theory, we were prepared. Scholars like Shoshana Zuboff had already warned us about surveillance capitalism. Think tanks issued report after report. Governments debated “trustworthy AI.” Corporations set up ethics boards and published sunny manifestos about fairness and transparency.
But when push came to profit, the principles often disappeared.
Google’s Ethical AI team famously imploded in 2020 after Timnit Gebru was fired for raising concerns about bias in large language models. Her co-lead, Margaret Mitchell, was soon pushed out as well. Google insisted it supported responsible AI. But when responsibility clashed with revenue—or reputational risk—the team was sacrificed. It was a clear message: internal governance is negotiable when ethics get in the way.
Regulation: A Global Patchwork Quilt
Efforts to govern AI have been uneven, slow, and sometimes performative. Europe is the furthest along, having passed the AI Act, a comprehensive legal framework categorizing AI by risk, from low to unacceptable. It bans some uses outright (like real-time biometric surveillance) and sets strict requirements for high-risk applications like credit scoring, medical devices, and law enforcement tools.
The AI Act will take full effect in 2026, but critics warn it’s already behind the curve. It was drafted before generative AI exploded, and while it now includes language to regulate foundation models like GPT-4 and Claude 3, the pace of technological advancement is outstripping bureaucratic timelines.
In the U.S., the picture has shifted markedly. The Biden administration’s 2023 Executive Order on “Safe, Secure, and Trustworthy AI” required safety evaluations for powerful AI systems, but had no binding legislative teeth. In January 2025, President Trump rescinded that order on his first day in office and issued a new Executive Order titled “Removing Barriers to American Leadership in Artificial Intelligence” (EO 14179), which revoked Biden-era directives, emphasized deregulation, and mandated the development of a national AI “Action Plan” within six months. Alongside, Trump also issued memos directing federal agencies to adopt AI swiftly, with appointed Chief AI Officers and standardized governance frameworks focused on boosting innovation over safeguards. Meanwhile, Congress is active on legislation like the bipartisan “TAKE IT DOWN Act” (covering non-consensual deepfakes) and the revival of the CREATE AI Act to expand public access to AI infrastructure. So, there is now a national framework—but it leans decisively toward deregulation and federal coordination, not liability or ethics.
Meanwhile, China has aggressively moved to control AI development within its borders, issuing detailed rules on recommendation algorithms, deepfake content, and generative AI services. But those rules often focus less on protecting individuals and more on maintaining political and social order. Transparency is less a virtue than a tool of state control.
This divergence creates a dangerous dynamic: fragmented governance in a globally networked ecosystem. While Europe worries about rights, China controls behavior, and the U.S. defaults to corporate self-regulation, AI companies operate across borders, cherry-picking the least restrictive environments. It’s a regulatory arbitrage game, and humanity might be the collateral.
Who’s Actually in Charge?
Ask any major AI company about governance, and you’ll hear a familiar refrain: we’re committed to safety. Some mean it. Others hope you won’t ask follow-up questions.
OpenAI’s story is particularly instructive. Founded as a nonprofit with a mission to ensure AGI benefits “all of humanity,” it later morphed into a for-profit capped entity. In 2023, the company’s board briefly fired CEO Sam Altman amid internal disagreements over safety and commercial priorities, only to reinstate him days later under intense pressure from investors and employees. The governance structure—once a point of pride—was exposed as brittle, confusing, and easily overturned.
It’s not unique. Most leading AI labs are private entities governed by opaque boards, complex investor agreements, and shifting internal alliances. Some maintain advisory councils, but few disclose how those bodies influence model deployment decisions. Public input is minimal. Whistleblowers have limited protection. And most model evaluations—on safety, bias, or misuse potential—are conducted in-house or published selectively.
This isn’t just bad optics. It’s a systemic risk. As models become more powerful, the question of who decides how they’re trained, tested, and released becomes existential. Because these systems aren’t just answering trivia anymore. They’re being embedded into financial systems, legal workflows, medical diagnostics, autonomous weapons, and more.
What Happens When Governance Fails?
Sometimes, we don’t have to imagine. Take Facebook’s content recommendation AI, which was designed to maximize engagement. Internal research later revealed that it promoted outrage, misinformation, and political polarization. Warnings were raised. Changes were proposed. But little was done because the incentives weren’t aligned. Governance didn’t fail silently. It was deliberately ignored.
Or look at Clearview AI, the facial recognition startup that scraped billions of images from the web to build a surveillance tool used by law enforcement. It violated the privacy laws of multiple countries, sparked lawsuits, and drew international condemnation. Yet the company continued operating, largely unchecked, and even received new funding.
Then there’s the low-key but widespread failure of automated decision-making systems in government. In Michigan, an AI-based unemployment fraud detection system falsely accused tens of thousands of people. In the Netherlands, a welfare fraud detection algorithm led to wrongful targeting of low-income and immigrant families, triggering a political scandal and the collapse of the Dutch government in 2021.
These weren’t technical glitches. They were governance failures of oversight, accountability, and public transparency.
The Mirage of Ethics Guidelines
It’s not that we don’t know what to do. The OECD, UNESCO, the EU, and dozens of think tanks have all published excellent frameworks on trustworthy AI. Key principles—fairness, accountability, transparency, human oversight—are widely agreed upon.
But most of these documents are non-binding. Ethics becomes a marketing layer, not an operational constraint.
Take Microsoft’s “Responsible AI” principles. Admirable on paper. But when Bing Chat (now Copilot) launched with GPT-4, it quickly went off the rails, gaslighting users, generating disturbing content, and proving wildly unpredictable in multi-turn conversations. Microsoft acknowledged the problems, rolled out fixes, and pressed on. The product was too important to delay. Ethics took a backseat to competition.
Anthropic’s “Constitutional AI” approach is a more serious attempt to embed values directly into training. It defines a model’s moral framework up front—using principles like non-maleficence and respect for privacy—and uses those principles to steer responses. But even Anthropic admits it’s not foolproof. The AI still reflects biases, occasionally hallucinates, and requires continual red-teaming to uncover vulnerabilities.
We’re left with a governance model that is aspirational, fragmented, and easily bypassed. h4
Is There Hope?
Yes—but it requires realism, not rhetoric. First, we need enforceable laws with teeth. The EU AI Act is a start, but it needs global cooperation. The U.S. needs to pass comprehensive AI legislation, not just executive orders. And international coordination—through the UN, G7, or a new treaty body—needs to go beyond photo ops and shared principles.
Second, we need real audits—conducted by independent bodies, not internal safety teams answering to the same executives who greenlight model launches. Imagine if Boeing self-certified airplane safety. Oh wait, that already happened. And it didn’t end well.
Third, we need public empowerment. Whistleblower protections. Laws requiring model transparency. The right to explanation. A way for people to challenge harmful AI decisions and hold companies accountable.
And finally, we need humility. Not the fake kind that says “AI is just a tool,” but the real kind that acknowledges: we don’t fully understand what we’re building, and governance isn’t about slowing down—it’s about staying in control.
Conclusion: The Cost of Waiting
AI governance is the rulebook we forgot to write before releasing the players onto the field. Some are kicking balls. Others are kicking people. The referees are still arguing over the shape of the ball.
This isn’t about slowing down AI. It’s about ensuring it doesn’t outpace the institutions meant to protect us. We’ve already seen what happens when we wait too long—ask anyone whose life was derailed by a bad algorithm, a biased chatbot, or a black-box decision with no appeal.
Governance isn’t the enemy of innovation. It’s how we make sure innovation doesn’t become our enemy.
And we’re running out of time.