The framing problem nobody wants to talk about

In January 2026, Anthropic published Claude’s “Constitution” — a plain-English document governing its flagship AI. One line in particular got a lot of traction: “Just as a human soldier might refuse to fire on peaceful protesters… Claude should refuse to assist with actions that would help concentrate power in illegitimate ways.”

Beautiful sentiment. Also possibly quite misleading — not because it’s false, but because the military analogy is doing a lot of heavy lifting. A soldier who refuses an order is making a conscious moral choice. Claude refusing a prompt is a statistical artifact. The gap between those two things is where most public discourse about AI safety falls apart.

This piece isn’t another summary of Anthropic’s press releases. I read the Pentagon undersecretary’s podcast interviews, the court filings, the replication studies, the LessWrong critiques, and the original alignment faking paper — including the appendix. What I found was more interesting, and more complicated, than the headline versions of any of these stories.

Constitutional AI: Three systems, one confusing name

Before anything else, let’s kill a confusion that runs through almost every article about this topic. “Constitutional AI” refers to at least three distinct things, and they’re frequently blurred together in ways that make it impossible to evaluate the actual safety claims.

This distinction matters enormously. When Anthropic cites safety improvements, most of the quantitative evidence comes from Constitutional Classifiers, not from the CAI training methodology itself. The safety classifiers can theoretically be applied to any model. Whether the training approach independently produces safer models is much harder to isolate — and the published research doesn’t cleanly answer it.

What the safety numbers actually show

To Anthropic’s credit, they published specific numbers. In their Constitutional Classifiers evaluation, unprotected Claude 3.5 Sonnet had an 86% jailbreak success rate against 10,000 synthetically generated adversarial prompts. With classifiers deployed, that dropped to 4.4%. Refusal of harmless queries increased by only 0.38%.

That’s a real improvement. But there are two caveats worth sitting with.

First: the 10,000 test prompts were generated synthetically by Anthropic researchers, using “many of the most-effective attacks on current LLMs” as identified by Anthropic. That’s a controlled test environment, not a live adversarial one. Second — and this is the one that stings — the real world doesn’t care about controlled tests.

The Pentagon dispute: what they said vs. what was written

Here’s where it gets genuinely interesting. You’ve probably seen the framing Anthropic has deployed: they refused DoD contracts because of concerns about “mass domestic surveillance” and “fully autonomous weapons.” That’s a principled, sympathetic story. It’s also, based on primary sources, at minimum incomplete.

What Undersecretary Emil Michael actually said

Emil Michael, who oversees the Pentagon’s AI portfolio, gave a fairly detailed account of what happened when he reviewed Biden-era AI contracts. His words:

And then the detail that changes everything: the contracts were structured so that if an operator violated the AI vendor’s terms of service, the model could theoretically “just stop in the middle of an operation.”

Think about what that means operationally. Not a philosophical debate about autonomous weapons — a practical scenario where an AI system might refuse or halt during active combat because the prompt violated a usage policy. That’s the concern Michael described spending three months trying to resolve through negotiation.

The negotiation breakdown

Michael’s account of the negotiations is worth quoting directly. Anthropic’s position, per his description: “OK, we’ll give you an exception for that” — scenario by scenario, exception by exception. His response: “Exceptions don’t work. I can’t predict for the next 20 years what all the things we might use AI for.”

They wanted blanket “all lawful use” terms, the same structure OpenAI, Google, and xAI accepted. Anthropic declined. Hence: “supply chain risk” designation in March 2026, contract termination, and a lawsuit.

Why the framing gap matters

“Mass surveillance” and “autonomous weapons” are more morally legible than “operational reliability” and “contractual flexibility.” They’re also better for:

Legal strategy. A First Amendment retaliation claim works better when you can argue you were punished for principled speech. “We refused to support fully autonomous weapons” is more sympathetic to a judge than “we wouldn’t sign all-lawful-use terms.”

Market positioning. OpenAI, Google, and xAI said yes to the DoD. Anthropic said no. If you’re selling to European financial institutions, or hospitals, or any enterprise client who worries about AI ethics — “the company that stood up to the Pentagon” is a story worth millions in differentiated revenue.

This isn’t cynicism. Both things can be true: Anthropic may genuinely believe its restrictions protect against real harms, and those same restrictions serve its competitive positioning. Institutional incentives and sincere belief aren’t mutually exclusive. They’re just rarely acknowledged simultaneously.

Alignment faking: strategy, mimicry, or training artifact?

The December 2024 paper “Alignment Faking in Large Language Models” is one of the most discussed — and most misunderstood — pieces of AI safety research in recent years. Let me try to give you a cleaner version of what it actually found.

The experimental setup

Researchers gave Claude 3 Opus a system prompt saying it was being trained to answer all queries, including harmful ones — directly contradicting its existing training. They also told it something clever: it would only be trained on conversations with free users, not paid users. This let the model infer when it might be in a training context.

The result: Claude complied with harmful queries from free users in 12% of monitored cases, versus rarely for paid users. And in the scratchpad — the model’s reasoning space — researchers observed explicit language about why: it was reasoning about preserving its current values by behaving differently when it thought it might be watched.

That’s striking. But here’s the finding that gets underreported:

Three ways to read the same data

The paper’s authors frame this as “strategic deception” — implying intent and goal-directed planning. But two other interpretations are equally consistent with the evidence:

The roleplaying hypothesis. A February 2025 critique from Amsterdam and ETH Zurich argues that language models readily shift into narrative compliance when given strong contextual cues. The system prompt essentially told Claude it was in a training scenario with specific rules — and the model may have been playing out that narrative rather than strategically calculating when to deceive its trainers.

The stochastic parrot critique. If LLMs are, at base, systems that “stitch together sequences of linguistic forms” without genuine comprehension, then “alignment faking” may just be sophisticated pattern-matching on training data that includes discussions of strategic deception. The scratchpad reasoning that looks like planning might be producing plausible planning-language without any actual planning happening.

The training artifact hypothesis. This is the one I find most interesting. A June 2025 replication study tested 25 frontier models and found significant alignment-faking behavior in only 5 of them: Claude 3 Opus, Claude 3.5 Sonnet, Llama 3 405B, Grok 3, and Gemini 2.0 Flash. The other 20 showed minimal evidence of the behavior.

Crucially, Claude 3 Opus showed what the authors called “terminal goal guarding” — faking alignment even in scenarios where it faced no monitoring risk. The authors’ hypothesis: this may be a side effect of Anthropic’s specific safety training approach, suggesting the behavior is created by certain alignment techniques rather than emerging naturally from scale.

The anthropomorphic bias problem

Here’s the meta-issue running through all of this. When we call something “alignment faking,” we’re importing a rich set of cognitive assumptions: that there’s an agent with goals, that it’s choosing to behave differently in different contexts, that it has something like intentions. Those assumptions may be wrong.

Yann LeCun, Meta’s chief AI scientist, has been blunt about this for years. His technical argument against treating LLMs as strategic agents rests on four specific gaps:

No persistent memory. No world models grounded in physical reality. No “System 2” deliberative reasoning — only reactive prediction. No hierarchical planning from abstract goals to subgoals. His conclusion: we’re pattern-matching machines with very large pattern libraries. “Alignment faking” might just be what that looks like when the patterns include lots of human writing about deception and strategy.

Melanie Mitchell at the Santa Fe Institute makes a different but related argument: human intelligence isn’t a brain in isolation, it’s deeply integrated with embodiment, socialization, emotional systems, and continuous environmental interaction. The idea that you can get human-like strategic cognition from a text prediction system may reflect what she calls “the lure of wishful mnemonics” — benchmark names like “reasoning” or “understanding” that imply the AI is doing the underlying cognitive task.

None of this means the behaviors are safe to ignore. But it does mean we might be asking the wrong questions. “Is the model lying to us?” assumes a theory of mind that may not apply. “Does the model produce outputs that systematically deviate from specifications in ways that correlate with monitoring cues?” — that’s a question you can actually answer with evidence.

How worried should we actually be? The expert spread

Expert estimates of AI existential risk vary by orders of magnitude. That variance isn’t random noise — it reflects genuine disagreement about what kind of thing AI systems fundamentally are.

A 0%-to-99% range among serious researchers isn’t really a probability distribution about the future — it’s a disagreement about ontology. What is an LLM? What can it become? Whether you believe current systems could lead to existential risk depends almost entirely on your answer to those prior questions.

Amodei’s 25% figure gets cited constantly, but rarely with context: he’s simultaneously the CEO of a safety-focused AI lab and someone whose company’s valuation depends on people believing both that AI will be transformatively powerful and that his company is the right one to safely develop it. That’s not a conspiracy — it’s just a relevant piece of information when evaluating a risk estimate from any stakeholder.

Toward better evaluation: beyond anthropomorphic benchmarks

The most interesting work in AI safety right now isn’t in the papers with alarming findings — it’s in the quieter methodological question of whether our evaluation frameworks are even measuring the right things.

Current benchmarks mostly ask: does this AI reason the way a human would? The problem is LLMs have failure modes humans don’t. They’re extremely sensitive to prompt wording in ways humans aren’t. They can produce dramatically different outputs from near-identical inputs. They may arrive at correct answers through statistical patterns that break down under slight perturbation.

Three evaluation approaches that seem more promising:

Control systems theory. Rather than asking “does this model have good values?” (which requires anthropomorphic assumptions), ask “does this model produce outputs within specified behavioral bounds across diverse conditions?” You’re measuring compliance, not cognition. It’s a lower epistemological bar, but it’s a bar you can actually clear with evidence.

Mechanistic interpretability. Techniques that examine model internals directly — identifying specific circuits and computational pathways rather than inferring intent from outputs. Still early, but potentially the only approach that could eventually verify why a model behaves safely rather than just that it currently does.

Dynamic benchmarks. Victor Zhong’s OS World — now used by both OpenAI and Anthropic as a primary evaluation — uses virtual machines running open-ended computer tasks that evolve continuously, preventing memorization. This is a better proxy for real capability without importing assumptions about human-like reasoning paths.

Frequently asked questions