Center for AI Safety (CAIS)

Stated Theory of Change

CAIS claims to reduce societal-scale AI risks through research, field-building, and advocacy. Hendrycks' interviews articulate a more specific mechanism:

1. Technical safety research creates practical tools (benchmarks, circuit breakers, RepE) that make systems empirically safer, following "defense in depth" rather than formal guarantees.
2. Field-building (compute cluster, courses, fellowships, textbook, newsletter) expands the community working on safety.
3. Policy advocacy shapes regulation and international governance. The flagship is MAIM -- a deterrence framework where states prevent rivals from developing destabilizing AI through credible threats of sabotage.

The causal chain: produce safety tools and human capital -> influence labs and governments -> reduce probability of AI-related catastrophe. The chain now emphasizes the geopolitical level over the technical level.

Revealed Theory of Change

What they actually optimize for: Hendrycks' personal influence and intellectual reach. CAIS is a platform for one researcher's worldview more than an institution. The research agenda follows his intellectual trajectory (benchmarks -> RepE -> circuit breakers -> geopolitics), the policy work reflects his network (xAI, Scale AI, Schmidt), and field-building creates an audience for his ideas (43K newsletter subscribers, textbook).

Divergence from stated theory: The three pillars are presented as roughly equal, but CAIS has pivoted heavily toward advocacy/policy. Hendrycks' 2025 interviews frame technical safety as "a small part" of the solution. Yet CAIS's most concrete, lasting impact has been through technical research (HarmBench adopted by AI Safety Institutes, MMLU as industry standard, circuit breakers surviving 20,000 jailbreak attempts).

The national security pivot: CAIS has repositioned from EA-adjacent safety research to national security establishment influence. The MAIM paper with Schmidt and Wang is the visible marker. This is strategically sound -- after OP's exit grant and EA's reputational damage from FTX, the national security framing opens new funding and policy channels. But it means CAIS's theory of change now depends on the viability of great-power deterrence applied to AI, which is contested.

The advisory model: Hendrycks advises xAI and Scale AI for zero effective pay, presumably to influence frontier lab safety from inside. But xAI's actual safety practices (no Grok 4 system card, antisemitic outputs) suggest minimal influence. The revealed theory through advisory roles is closer to "lend credibility to the lab's safety claims" than "improve the lab's safety practices."

Key Assumptions

1. AI safety is primarily geopolitical, not technical.

• Evidence for: Labs are competing fiercely and self-regulation fails. CAIS's safetywashing paper shows safety benchmarks mostly measure compute. Hendrycks argues labs are "predetermined to race."
• Evidence against: Technical alignment work at Anthropic, ARC, and others is producing real insights. If alignment is solvable through technical means, the geopolitical framing is a distraction.
• Testable: If a lab achieves a significant alignment breakthrough that generalizes, this assumption weakens.
• If wrong: CAIS has deprioritized the dimension that actually matters.

2. MAIM deterrence can be made stable.

• Evidence for: Nuclear MAD has prevented great-power war for 80 years. States respond to perceived existential threats.
• Evidence against: MIRI's critique shows red lines are unmonitorable, sabotage only delays, and the "breakout distance" is extremely short. Unlike nuclear weapons, AI capabilities can't be reliably observed. IAPS estimates only 25% chance MAIM dynamics actually emerge.
• Testable: Whether states actually take MAIMing actions as AI advances over 2026-2030.
• If wrong: CAIS's headline policy proposal fails.

3. A small, Hendrycks-centered org can have outsized influence.

• Evidence for: The extinction statement was a genuine coup. MMLU became the industry benchmark. HarmBench was adopted by national AI Safety Institutes. 43K newsletter subscribers.
• Evidence against: The xAI advisory role shows limits of personal influence without institutional leverage.
• If wrong: CAIS produces papers and statements but doesn't change outcomes.

4. RepE/circuit breakers can substitute for mechanistic interpretability.

• Evidence for: Circuit breakers survived 20,000 jailbreak attempts. RepE controls arbitrary concepts in model representations. Hendrycks makes a strong case that mech interp hasn't produced safety results after a decade.
• Evidence against: Circuit breakers may have performance penalties. Mech interp may be necessary for novel failure modes that RepE can't anticipate.
• If wrong: CAIS has contributed to the field underinvesting in the approach that works.

Strengths

1. Research output per dollar. ~$7M/year in expenses producing multiple top-venue papers, the most widely used AI benchmark (MMLU), and safety tools adopted by national AI Safety Institutes. Few orgs match this efficiency.

2. Hendrycks' intellectual versatility. From activation functions to benchmarks to safety tools to deterrence theory -- the breadth of high-quality output is remarkable. His empirical, anti-formalist approach produces practical tools others actually use.

3. Compute cluster as force multiplier. Free compute for 350 researchers producing 109 papers and 4,000+ citations. Field-building through infrastructure, not just persuasion.

4. Willingness to take intellectual risks. Opposing mech interp, proposing MAIM, working with Schmidt and Wang -- these generate productive debate. Most safety orgs are intellectually cautious.

5. Cross-partisan reach. CAIS engages from Jacobin to AEI, from EA forums to the Eisenhower Building. This breadth of audience is rare in AI safety.

Weaknesses and Risks

1. Single point of failure. CAIS = Dan Hendrycks. No evident succession plan, institutional depth, or visible second-in-command. Oliver Zhang is invisible. If Hendrycks is discredited or departs, the org has no evident continuity.

2. The xAI credibility gap. Hendrycks advocates for safety standards while advising a company that conspicuously fails to meet them. Every Grok incident where Hendrycks is named as safety adviser undermines the broader safety message.

3. Funding opacity. ~$6-7M of 2024 contributions are unidentified. Combined with Tallinn's funder-governance overlap and the OP exit with no explanation, independence is questionable. For an org demanding AI transparency, this is ironic.

4. MAIM may not work. MIRI's critique is compelling. If deterrence logic doesn't hold for AI (unmonitorable thresholds, sabotage that only delays), CAIS's headline policy contribution produces no actionable governance.

5. EA distancing without intellectual reckoning. Taking $19M from EA sources then declaring "EA does not equal AI safety" is strategic but raises integrity questions. Has the theory of change genuinely shifted, or is this reputation management?

6. Research follows Hendrycks' interests, not systematic priority-setting. No visible process for identifying highest-priority research questions. This produces novel work but may leave important areas unfunded.

Cross-References

• Versus Anthropic: Direct opposition on mechanistic interpretability. CAIS argues top-down (RepE) beats bottom-up (mech interp). Both can't be right about where the field should focus.
• Versus MIRI: MIRI wrote the most rigorous MAIM critique but shares the existential risk concern. CAIS's pragmatic policy approach contrasts with MIRI's theoretical tradition.
• Versus frontier labs: Hendrycks says labs "can't really choose not to race." CAIS works around labs through policy rather than through them, except for advisory roles that appear ineffective at changing lab behavior.
• Versus PauseAI: MAIM is not a pause framework -- it allows development within deterrence constraints. More politically feasible but less ambitious on risk reduction.
• Compute cluster vs. MATS/SERI: CAIS provides infrastructure; others provide mentorship. Complementary, but Schmidt Sciences restriction limits reach.

What Would Change This Assessment

• If Hendrycks' xAI advisory produced observable improvements (system cards, circuit breaker adoption), that updates the advisory model's credibility substantially.
• If OP explained its exit benignly (portfolio rebalancing, not values disagreement), that reduces concern about CAIS's direction.
• If a state actually took a MAIMing action, it validates descriptive MAIM and increases CAIS policy urgency.
• If tamper-resistant safeguards reached production quality with minimal performance penalty, that vindicates the RepE agenda.
• If CAIS published board membership, full donor list, and conflict-of-interest policies, governance concerns would diminish.

Self-Critique

Sources I should have checked: The ChinaTalk podcast (audio only, no transcript) for the deepest China/AI discussion. The Pirate Wires article (paywalled) for the most detailed Gray Swan conflict account. Full 990 PDFs for board and program expense details.

Potential bias: I may overweight the xAI credibility problem because it's concrete and documented, while underweighting CAIS's genuine technical contributions. The MAIM critique section draws heavily from MIRI, which has its own intellectual commitments. An IR scholar might assess MAIM differently. I may also underestimate how effectively the national security repositioning mainstreams safety concerns.

A thoughtful disagree-er would say: "CAIS is exactly what the field needs -- pragmatic, empirically-grounded, and meeting the policy establishment where it is. The xAI role gives Hendrycks insight into how frontier labs work, which informs better policy. MAIM is the first AI governance proposal taken seriously by the national security community. And the research output per dollar is the best in the field."

Weakest claim: That the xAI advisory role evidences credibility failure. It's possible Hendrycks has significant private influence that doesn't show in public reporting. If he fought hard against the Grok 4 release and was overruled, that's very different from not trying.

Information that would most change my view: Internal evidence of Hendrycks' influence at xAI. Did he push for specific safety measures? Did he object to Grok 4's release? The answer determines whether the advisory role is meaningful engagement or performative association.