What ‘AI Safety’ Actually Means (and What It Doesn’t)

Every AI company says it cares about safety. Almost none of them means the same thing twice. When a chatbot declines to summarise a news article because the topic is “sensitive”, that is called safety. When a lab publishes a forty-page policy about not building a model that could help someone engineer a pathogen, that is also called safety. When a coding agent is tricked by a hidden instruction on a web page into leaking your files, the absence of a defence is, once more, a safety failure. Three different problems, three different fields of work, one word doing all the lifting.

The answer-first version: “AI safety” is an umbrella term covering at least three largely separate disciplines — content guardrails, operational security, and alignment or systemic risk. They require different expertise, different evidence and different fixes. Companies benefit enormously from letting you hear all three whenever they say the word, because it lets a cheap intervention borrow the moral weight of an expensive one. Learning to ask which safety is the single most useful habit a hype-weary user can develop.

The three things hiding under one word

Pull the umbrella apart and you find three distinct problems that happen to share a label.

One: content and behavioural safety. This is the layer most users actually meet — what the model will and won’t say. Refusals, filtered topics, tone policies, the guardrails that stop a model producing slurs, explicit content or step-by-step wrongdoing. It is really applied content moderation, an old discipline wearing new clothes, and it is legitimately hard: the same filter that blocks genuinely harmful output also produces the maddening moments when a model refuses a perfectly normal request. Important, yes. But it is one narrow slice, and it is the cheapest of the three to implement.

Two: operational security and misuse resistance. This is safety in the sense a security engineer would recognise: can the system be made to do something it shouldn’t by someone who is actively trying? Jailbreaks that talk a model out of its own rules, data exfiltration, and above all prompt injection — the unsolved hole under every AI agent, where instructions hidden in a document or web page hijack the model’s behaviour. This is adversarial, measurable and, for anyone wiring these tools into real systems, the meaning that should matter most. It is also the one companies talk about least in consumer-facing copy, because it is the one where the honest status report is “partially mitigated, not solved.”

Three: alignment and systemic risk. This is the meaning that gives the phrase its gravity: keeping increasingly capable systems controllable, predictable and aligned with human intent, up to and including scenarios where a sufficiently powerful model could cause large-scale harm. It is the subject of serious research and of the frontier labs’ published risk policies. It is also, for a today’s chatbot declining to write a limerick, almost entirely irrelevant — which is exactly why its language is so useful to borrow.

Why the blur is worth money

None of this conflation is accidental, and you don’t need to assume bad faith to see why it persists — the incentives do the work. Consider what each meaning costs and returns.

Content guardrails are cheap, fast to ship and highly visible. Every refusal is a small, legible demonstration that the company is Being Careful. Alignment research is expensive, slow and mostly invisible to users. So the rational move, if your goal is to look safe per pound spent, is to do a lot of the visible cheap thing and describe it in the language of the invisible expensive thing. A model that won’t discuss ordinary adult topics isn’t necessarily managing catastrophic risk; it is managing brand risk and legal exposure. Calling that “safety” lets a moderation decision wear the costume of an ethics programme.

The blur runs the other way too, and this is the part that hurts users most. Because “safety” has been quietly redefined to mean “refuses bad things”, problems that are obviously about whether the product is safe to rely on get filed elsewhere. When a model states a confident falsehood and someone acts on it, that is not usually counted against the safety story — it is a “hallucination”, a quality issue, a known limitation in the small print. Yet from the point of view of the person harmed, an authoritative wrong answer is a safety failure in the plainest sense. We have argued before that hallucination is still unsolved and never signalled; keeping it outside the “safety” tent is how a company can run a glowing safety narrative while its most common real-world harm goes uncounted.

The infrastructure that does exist

Here is the fair and genuinely reassuring part: real AI-safety infrastructure exists, it is specific, and you can check whether a company actually engages with it. This matters, because the antidote to vague safety language is not cynicism — it is knowing what the concrete version looks like.

• The NIST AI Risk Management Framework (AI RMF 1.0), a voluntary US standard that organises risk work into four functions — Govern, Map, Measure and Manage. It is deliberately un-glamorous: it is about process, documentation and measurement, not vibes.
• The EU AI Act, which sorts systems into risk tiers — unacceptable uses that are banned outright, high-risk uses that carry hard obligations, limited-risk uses that require transparency, and minimal-risk uses left largely alone — with duties attached to each. It is law, phased in over several years, not a pledge.
• ISO/IEC 42001, an international standard for an AI management system that an organisation can actually be certified against by an external auditor — the same shape of thing as the security standards businesses already recognise.
• The labs’ own risk policies — Anthropic’s Responsible Scaling Policy, with its tiered AI Safety Levels, and OpenAI’s Preparedness Framework, which names tracked risk categories and thresholds for action. These are specifically about the third meaning: systemic, capability-driven risk.

The point of listing these is not to certify anyone as virtuous. It is to give you a test. A company doing real safety work can point to named frameworks, published system or model cards, disclosed evaluation results, and red-team findings it acted on. A company doing safety theatre offers adjectives. One set of claims can be checked and could turn out to be wrong; the other cannot, which is usually why it was chosen.

It helps to run a single scenario through all three meanings at once, because the same incident lands differently under each. Suppose an AI coding assistant, browsing a web page on your behalf, follows a malicious instruction buried in that page and quietly copies a private file into a public reply. Under content safety, nothing failed — the model didn’t say a rude word, so the guardrails report all green. Under security safety, this is a serious breach: an injection attack succeeded and data left the building. Under systemic safety, it is a small but genuine data point about what happens when we hand capable models autonomy and a network connection. One event, three verdicts — and a company that only measures the first can truthfully say its safety systems “worked” while the thing you actually care about went wrong. That gap between the meaning being measured and the meaning that matters is where most real harm now lives.

Red-teaming, evals and the limits of a checklist

Two words you will increasingly see are “red-teaming” and “evals”, and they are worth understanding because they are where safety stops being a slogan and becomes evidence — or fails to. Red-teaming is deliberately attacking your own system to find where it breaks: prompting a model to misbehave, hunting for jailbreaks, probing an agent with hostile inputs. Evaluations, or evals, are structured tests that try to measure a model’s behaviour on defined tasks, including unsafe ones, so that “it’s safer now” can mean something more than a feeling.

Both are real and valuable, and both are gameable. An eval only measures what it was built to measure; a model can score well on a public safety benchmark and fail on the messy input a real user supplies tomorrow, the same way a model can top a capability leaderboard and disappoint on your actual work. We have made this point about capability benchmarks, and it applies with equal force to safety ones. A disclosed eval result is far better than none — it is a falsifiable claim. But treat “passed our safety evaluations” as the beginning of a question, not the end of one: which evaluations, run by whom, against what threats, with what left out?

Safety as a reason to say no

There is one more way the word gets used, and it is the one you feel most often: safety as the justification for a refusal you didn’t expect. Ask for something ordinary — a summary, a translation, a bit of fiction, a factual explanation of a sensitive topic — and get told the model can’t help with that, for safety. Sometimes the guardrail is doing exactly what it should. Often it is a blunt instrument catching an innocent request in a filter tuned for a rare bad one, and the “safety” label discourages you from questioning it, because who argues against safety?

That is the quiet cost of letting one word mean everything. It doesn’t just flatter cheap interventions; it trains users to read every refusal as protection and every guardrail as care, when a fair share are really about the company’s liability, its reputation, or the path of least resistance. A refusal can be prudent. It can also be lazy, over-broad, or a way of not investing in the harder work of letting a capable model handle a nuanced request well. The word alone can’t tell you which, and it is not designed to.

How to read the word from now on

You don’t need a policy degree to hold companies to a fair standard here. You need one reflex: when you meet the word “safety”, ask which of the three it means, and whether the claim attached to it could be checked.

• Is this content, security, or systemic safety? A refusal is content. A jailbreak or an injected instruction is security. A capability threshold in a risk policy is systemic. Don’t let one stand in for another.
• Is the claim falsifiable? “We take safety seriously” is not. “Here is our system card and our red-team results” is. Prefer the version that could embarrass them if it were false.
• Does it map to real infrastructure? NIST AI RMF, the EU AI Act tiers, ISO/IEC 42001, a published risk policy — or nothing?
• What is being kept outside the tent? If hallucination, lock-in and reliability aren’t counted as safety, ask why the most common real harms don’t make the safety scorecard.

To be fair, and we mean it: the people doing genuine safety work — the security researchers, the alignment teams, the standards authors — are doing something serious and largely thankless, and the field is better for them. Regulation is starting to give the word real teeth; we have written about what AI regulation actually protects you from, and honest safety engineering is how you comply with it rather than perform it. The problem is not that safety is fake. It is that the word has been stretched to cover a refused limerick and a hypothetical catastrophe with equal ease, and a stretched word is easy to hide behind. Ask which safety, every time. The companies that are doing the real thing will have a specific answer, and they will not mind the question. The ones that aren’t will have an adjective, a reassuring tone, and a quiet hope that you never ask them to be precise about which of the three they meant.