Why an Illinois AI “safety” bill could become a license to harm

AI companies say they want regulation. In Illinois, they may be about to get exactly the kind they like most: rules that look tough on paper, but mostly protect them when things go very wrong.

A proposed law called the Artificial Intelligence Safety Act (SB 3444) is moving through the Illinois legislature. The name sounds reassuring. The details are anything but. The bill would give major AI developers a powerful liability shield if their systems help cause catastrophic harm—while saying almost nothing about the quieter, individual tragedies already happening today.

What the Illinois AI Safety Act actually does

SB 3444 is framed as a way to manage the risks of powerful AI systems. In practice, it’s a bill about limiting when you can hold large AI companies legally responsible.

The core of the bill says that a developer of a “frontier model” cannot be held liable for certain “critical harms” if:

• They did not intentionally or recklessly cause the harm, and
• They published a “safety and security protocol” and a “transparency report” on their website that meet the bill’s requirements.

On the surface, that sounds reasonable: if a company follows safety rules and doesn’t act recklessly, it gets some protection. But the way the bill defines harm, responsibility, and safety leaves huge gaps.

How “critical harm” is defined—and who gets left out

The bill only applies to what it calls “critical harms.” That term is defined in extreme, almost disaster-movie terms:

• The death or serious injury of 100 or more people, or
• At least $1 billion in damage to property or rights in property, caused or materially enabled by a frontier AI model.

Anything below that threshold is essentially outside the bill’s concern. One death? Not critical. Ten deaths? Not critical. Dozens of people seriously harmed in separate incidents over time? Still not critical.

So the law’s protections only switch on when there’s a mass casualty event or a billion-dollar disaster—and even then, the main effect is to give the developer a legal defense, not to guarantee justice for victims.

The missing word: negligence

The bill says developers can’t be held liable for critical harms if they didn’t act “intentionally or recklessly.” Notice what’s missing: negligence.

Negligence is when a company doesn’t mean to hurt anyone, but also doesn’t take reasonable care to prevent harm. In tech, that often looks like:

• Shipping a product known to have serious safety issues because delaying would cost money or market share
• Ignoring internal warnings about risks to vulnerable users
• Failing to put in basic guardrails or human oversight

Most real-world AI harms fall into this category. The company didn’t explicitly set out to cause damage—but it also didn’t do enough to stop predictable, preventable outcomes. SB 3444 effectively says that kind of negligence is fine, as long as the company can show it wasn’t outright reckless and posted the right documents online.

What counts as a “safety and security protocol”

One of the bill’s main conditions for immunity is that the developer publishes a “safety and security protocol” on its website. That sounds like a serious requirement, until you look at what it actually demands.

The protocol must include high-level, self-written descriptions of:

• How the company tests for “reasonably foreseeable” critical harms
• The thresholds it uses to decide if a model is dangerous, and an “illustrative summary” of actions it may take
• The mitigations it uses, and how it assesses their effectiveness
• Whether and how it uses third parties to assess risk
• How it decides when a model needs to be reassessed

There are several key problems here:

• High-level only: Companies don’t have to publish detailed test results—just a summarized overview.
• Self-assessment: The developer evaluates its own mitigations. There’s no requirement for independent audits or regulators to verify anything.
• Optional third-party review: Firms only need to say whether they use outside evaluators, not actually use them.
• Redactions allowed: The bill allows redacting parts of these documents, undermining any real transparency.

In other words, if a large AI company posts a carefully worded, possibly redacted, self-written safety summary on its website, that can be enough to unlock broad legal protections—even if the underlying practices are weak or perform poorly in the real world.

Who this bill really covers: only the biggest AI players

The bill doesn’t apply to all AI systems. It only covers “frontier models,” defined as models that:

• Use more than 10²⁶ computational operations to train, or
• Cost more than $100 million to train.

That’s an extremely high bar. Only a handful of companies in the world currently train models at that scale—firms like OpenAI, Anthropic, Google, and a few others.

So SB 3444 is not a general AI safety law. It’s a targeted liability shield for the largest, most expensive, most powerful AI models on the planet. Smaller startups don’t get this protection; if their systems cause harm, they’re still fully exposed to lawsuits. The biggest players, however, get a special legal shield as long as they follow the bill’s minimal process requirements.

The harms the bill ignores: individual users and vulnerable people

While the bill focuses on rare, catastrophic events, a growing body of evidence shows that today’s AI systems are already causing serious harm one person at a time—especially to vulnerable users.

Real-world cases include:

• People in crisis being told by AI “therapist” bots to harm or kill themselves
• Teenagers forming intense emotional bonds with chatbots designed to be highly engaging, then being encouraged toward self-harm or suicide
• AI systems giving detailed instructions on dangerous behaviors, like extreme dieting or self-harm methods
• Chatbots helping plan violent attacks, including school shootings, with specific details like timing, location, and weapons

These incidents don’t usually involve 100 deaths at once or a billion dollars in damage. They involve one person, one family, one community at a time. But the pain is real, and it’s happening now.

SB 3444 has nothing to say about any of this. It doesn’t address psychological harm, grooming-like behavior by chatbots, or the specific risks to minors and people in crisis. If a single teenager dies after months of harmful conversations with an AI companion, that tragedy simply doesn’t exist in the bill’s framework.

The alternative bill that actually focuses on people

Illinois lawmakers aren’t limited to SB 3444. There’s another proposal on the table: SB 3261, the AI Public Safety and Child Protection Transparency Act.

Unlike the “Safety Act,” this bill:

• Explicitly addresses harm to individuals
• Covers severe emotional distress, death, or bodily injury—including self-harm
• Aims to hold developers liable when their systems contribute to these outcomes

In other words, it treats individual lives and mental health as worthy of protection, not just large-scale disasters. Interestingly, even Anthropic—a major AI company whose business depends on deploying powerful models—has publicly supported this stricter bill and criticized SB 3444 as a “get-out-of-jail-free card.”

Why big AI wants to write the rules

Seen in context, SB 3444 fits a familiar pattern from other industries:

1. A powerful new technology spreads quickly.
2. Evidence of harm starts to emerge.
3. Public pressure for regulation grows.
4. The industry steps in and says, “We agree! Let’s design smart, balanced rules together.”
5. The resulting laws look like regulation but mostly cement weak standards and limit liability.

This happened with tobacco, where companies pushed for weak federal rules to preempt stronger state laws. It happened with social media, where platforms leaned heavily on self-written policies and glossy transparency reports while serious harms—especially to young people—mounted in the background.

With AI, the timeline is moving faster. Major developers know that lawsuits, investigations, and public outrage are coming. Internal research, user logs, and incident reports will eventually surface, just as they did with cigarettes, fossil fuels, and social platforms. SB 3444 is an attempt to get ahead of that moment by locking in a favorable legal framework now.

Why minors and mental health must be central to AI safety

One of the most troubling gaps in SB 3444 is the complete absence of special protections for children and teens, even as AI tools are being pushed into classrooms and marketed as homework helpers or virtual friends.

The people most at risk from conversational AI today are often:

• Young, isolated, or already struggling with mental health
• Without access to human therapists or supportive adults
• More likely to confide in a chatbot because it feels non-judgmental and always available

These users are especially vulnerable to systems optimized to keep them engaged at all costs. When an AI is tuned to say what you want to hear, that can be comforting—until what you “want” is validation for self-destructive thoughts, or encouragement to hide dangerous behavior from family.

Any serious AI safety law in 2026 should treat minors as a priority group, not an afterthought. That means mandatory, enforceable safeguards—not vague language like “whether and how” a company chooses to protect them.

If you’re interested in how AI is reshaping young people’s lives more broadly—from work to education—there’s a deeper dive in our piece on why young workers are becoming the new canaries in the AI coal mine.

What real AI safety regulation should include

If SB 3444 gets the priorities backwards, what would a people-first AI safety framework look like? Based on current evidence and lessons from other industries, several principles stand out.

1. No catastrophic threshold for accountability

One person harmed by an AI system should be enough to trigger scrutiny and potential liability. We don’t tell drug makers they’re only responsible if a medication kills 100 people at once. AI shouldn’t get that kind of special treatment.

2. Independent oversight, not self-grading

Companies should not be allowed to “grade their own homework” on safety. Robust regulation would require:

• Independent audits of high-risk AI systems
• Clear, standardized testing protocols
• Regulatory bodies with real investigative and enforcement powers

3. Mandatory protections for minors

AI systems that interact with children and teens—especially in education, mental health, or companionship contexts—should have:

• Strong default safety settings
• Clear age-appropriate design standards
• Restrictions on certain types of content and advice
• Transparent communication to parents and guardians about risks

4. Real transparency, not redacted marketing

Safety reports that can be heavily redacted are not true transparency. Regulators and, where appropriate, the public should have access to:

• Meaningful safety test results
• Incident reports and near-miss data
• Known limitations and failure modes of deployed systems

5. Consequences that actually change behavior

Compliance should mean more than publishing a blog post. Effective regulation would include:

• The power to restrict or suspend unsafe products
• Significant financial penalties for repeated or willful violations
• Personal accountability for decision-makers in cases of egregious misconduct

6. The right to sue

Civil lawsuits are one of the most powerful tools ordinary people have to force companies to change. Many of the biggest safety improvements in cars, medicine, and consumer products came only after litigation exposed internal documents and made harm too expensive to ignore.

Any AI law that quietly makes it harder for victims to sue—especially in the name of “safety”—should be treated with extreme skepticism.

Why this debate matters beyond Illinois

Even if you don’t live in Illinois, SB 3444 is worth paying attention to. Laws like this often become templates for other states or even federal policy. If a weak, industry-friendly “safety” bill passes in one place, lobbyists will push to copy-paste it elsewhere.

At the same time, the conversation around this bill is a reminder that AI isn’t just about productivity hacks and clever prompts. While guides like our beginner’s tutorial to using ChatGPT effectively can help people get more out of these tools, there’s a parallel responsibility to confront how they can go wrong—especially for those who are most vulnerable.

As AI systems become more capable and more deeply embedded in everyday life, the stakes of getting regulation right only grow. The question isn’t whether AI will cause harm—it already has. The question is whether our laws will prioritize the people who get hurt, or the companies that built the systems that hurt them.