Center for Human-Compatible AI (CHAI)

Stated Theory of Change

CHAI's stated theory of change is a paradigm shift argument: the way AI is currently developed (the "standard model" of optimizing fixed objectives) is fundamentally unsafe, and CHAI exists to develop and propagate an alternative (provably beneficial AI via CIRL -- machines uncertain about human preferences that learn them through observation).

The specific mechanism: If AI systems are designed to be genuinely uncertain about what humans want, they will naturally be deferential, accept correction, and allow shutdown. This makes them inherently safer than systems pursuing fixed objectives. Russell calls this "a big step because the standard model doesn't say [machines] should be good for human beings at all."

The causal chain runs: Develop theoretical framework -> Prove key properties -> Train researchers in the paradigm -> Textbook adoption propagates the ideas -> Researchers carry ideas into industry labs -> Russell personally advocates for regulatory adoption -> The field shifts.

Revealed Theory of Change

The gap between CHAI's stated and revealed theory of change is instructive.

What they actually produce and optimize for:

1. Academic publications (32+ papers/year)
2. PhD graduates placed at top safety positions (their most impactful output)
3. Russell's personal policy advocacy (hundreds of meetings, testimony, media)
4. Annual workshops for community building
5. Russell's public intellectual profile (books, BBC lectures, TIME100, Davos)

What they don't do much of:

1. Build tools or products that implement CIRL
2. Work directly with industry labs on adoption
3. Engage with current LLM alignment practices (RLHF, constitutional AI)
4. Produce quantitative impact evaluations
5. Build institutional capacity beyond Russell

The revealed theory of change is closer to: "Stuart Russell uses his extraordinary personal credibility as the textbook author to keep the provably-beneficial-AI paradigm alive in academic discourse and elite policy circles, while training a generation of researchers who carry safety awareness into industry, whether or not they specifically use CIRL."

This is actually a reasonable theory of change -- possibly more realistic than the stated one. The field-building through talent placement is demonstrably real. Whether the CIRL framework itself will matter depends on whether future AI systems are amenable to that approach.

Key Assumptions

Assumption 1: The standard model is the root cause of unsafe AI.

• Evidence for: YouTube recommender algorithm, specification gaming examples, Russell's extensive catalog of fixed-objective failures.
• Evidence against: Current LLMs don't obviously work on the "fixed objective" model. RLHF and constitutional AI already incorporate preference learning. The field may be organically moving beyond the standard model without CIRL.
• Testable: Partially. If LLM alignment methods (not based on CIRL) prove adequate, this assumption loses force.
• If wrong: CHAI's research program is less urgently needed, though still conceptually interesting.

Assumption 2: CIRL's uncertainty-based corrigibility solves the control problem.

• Evidence for: Theoretical proofs that uncertain AI prefers shutdown. Intuitive appeal of the "humble machine" concept.
• Evidence against: MIRI's fully updated deference critique -- once the AI learns enough, uncertainty disappears and corrigibility with it. Kasenberg's scalability objections. No empirical validation at scale.
• Testable: Yes, in principle. But testing requires AI systems sophisticated enough to exhibit the relevant behaviors.
• If wrong: CHAI's core technical contribution fails. The talent pipeline and policy advocacy remain valuable.

Assumption 3: Academic research and paradigm advocacy can actually redirect the AI industry.

• Evidence for: Russell's textbook reaches 1500+ universities. His policy access is real (Senate testimony, EU AI Act influence, OECD co-chair).
• Evidence against: AI labs are driven by competitive pressure and trillions in investment. Russell himself quotes an unnamed CEO saying they feel "trapped" in the race. Academic frameworks have had limited impact on commercial AI development historically.
• Testable: Yes, over time. Track whether CIRL-adjacent ideas appear in industry safety practices.
• If wrong: CHAI is producing good academic work with little real-world impact on the trajectory of AI development.

Assumption 4: The talent pipeline carries CHAI's ideas into practice.

• Evidence for: Alumni at DeepMind, Anthropic, OpenAI, MIT, Stanford, FAR.AI.
• Evidence against: Alumni may adopt different approaches once at industry labs. Rohin Shah (CHAI PhD) describes himself as more optimistic than the doom narrative and works on empirical alignment, not CIRL.
• If wrong: CHAI's field-building is producing generally safety-aware researchers, but not specifically propagating the CIRL paradigm. This is still valuable, just not the stated theory of change.

Strengths

Intellectual credibility. Russell is arguably the single most credible person in the world to argue that AI needs a paradigm shift. He co-authored THE textbook. He has impeccable academic credentials. He cannot be dismissed as an outsider or alarmist.

Independence. Unlike many AI safety figures, Russell has no financial ties to frontier labs. No equity, no board seats, no consulting arrangements. His tenured position at UC Berkeley makes him immune to industry pressure. This is rare and valuable.

Talent production. CHAI has placed alumni in influential safety positions at every major lab and several leading universities. This network effect may compound over time.

Policy access. Russell's personal access to policymakers (heads of state, senators, EU commissioners, OECD, WEF, UN) is extraordinary for an academic. His Senate testimony and EU AI Act influence demonstrate concrete policy impact.

Theoretical foundations. CIRL is a mathematically rigorous framework that provides formal proofs of properties like deferability and shutdown acceptance. Even if imperfect, it provides a target for the field.

Longevity and stability. UC Berkeley embedding provides institutional stability. CHAI has operated continuously since 2016 with no existential funding crises, unlike many safety organizations.

Weaknesses and Risks

Key-person risk is extreme. CHAI is Stuart Russell. His departure, retirement, or incapacitation would eliminate most of the org's policy access, public profile, fundraising capacity, and intellectual leadership simultaneously. There is no visible succession plan. Dragan, the most plausible successor, has left for DeepMind. Russell is 63.

CIRL may be solving a problem that doesn't arise in the LLM paradigm. CIRL was designed for a world of RL-based agents with explicit reward functions. Modern LLMs trained via RLHF already incorporate a version of "learn human preferences through observation." The field may be naturally moving past the standard model without CIRL. CHAI risks being outrun by the very progress it advocated.

No evidence of practical adoption. After 10 years and $20M+, there is no documented case of CIRL being implemented in a production AI system. The framework remains purely academic.

Funder concentration. Open Philanthropy provides virtually all external funding. This creates dependency on a single funder's continued belief in CHAI's approach.

Limited engagement with LLM alignment. CHAI's publication record is dominated by traditional RL/IRL work. The org has been slow to engage with RLHF, constitutional AI, mechanistic interpretability, and other techniques that are actually being deployed at frontier labs.

Policy advocacy flows through one person. Russell's hundreds of meetings with policymakers are impressive but represent a single point of failure for CHAI's policy impact. IASEAI is a step toward institutionalizing this, but the first conference attracted almost no policymakers despite being policy-focused.

Academic incentives may limit impact. CHAI researchers are incentivized by publications and academic career advancement, which may favor publishable theoretical results over messy practical safety work.

Cross-References

Complementary with: MIRI (different approach to same problem -- MIRI focuses on deceptive alignment, CHAI on value alignment), FAR.AI (founded by CHAI alum Gleave, applies CHAI ideas to evaluation), Redwood Research (empirical safety testing), METR (AI evaluations, employs CHAI alum Lawrence Chan).

Talent pipeline to: Google DeepMind (Dragan, Shah, Dennis, Turner, Emmons initially), Anthropic (Emmons), OpenAI (Toyer, Campbell initially), Meta FAIR (Milli), MIT (Hadfield-Menell), Stanford (Sadigh), Princeton (Fisac), CMU (Bajcsy).

Funded by: Open Philanthropy (via BERI and UC Berkeley).

Russell-adjacent orgs: IASEAI (Russell is President), Kavli Center (Russell is Director), BERI (fiscal intermediary, co-founded by CHAI researcher Critch).

Intellectual relationship with MIRI: Russell and Yudkowsky have engaged substantively on the fully updated deference problem. Their crux: whether IRL can produce a utility function close enough to true human values that the resulting AI would be beneficial. MIRI says no (the meta-learning is still wrong enough to be catastrophic). Russell says the gap can be made small enough to be acceptable. This is the central unsettled question in CHAI's research program.

What Would Change This Assessment

Upward revision if:

• CIRL-adjacent techniques were adopted by a frontier lab for a production system.
• A clear successor to Russell emerged with comparable credibility and access.
• CHAI published work demonstrating how CIRL applies to LLM alignment.
• An independent evaluation confirmed specific policy outcomes traceable to CHAI advocacy.
• Non-OP funders provided significant diversified funding.

Downward revision if:

• LLM alignment methods (RLHF, constitutional AI, etc.) proved adequate without CIRL-type frameworks, suggesting CHAI's approach was addressing the wrong problem.
• Russell retired or became incapacitated with no succession plan executed.
• OP significantly reduced or ended CHAI funding.
• Multiple CHAI alumni publicly stated that CIRL was not relevant to their current safety work.
• CHAI continued not publishing progress reports or impact evaluations.

Self-Critique

Strongest limitation: I have very limited information on CHAI's internal culture, decision-making processes, and how research priorities are actually set. My assessment of key-person risk is based on external observation -- insiders might see more distributed leadership than is visible from outside.

Potential bias: The evidence base skews heavily toward Russell's public statements, which are inevitably more polished and confident than internal discussions would be. The absence of critical EA Forum/LessWrong discussions (due to fetching limitations) may mean I'm underweighting community skepticism about CHAI's approach.

Weakest claim: My claim that CIRL hasn't been adopted in practice rests on absence of evidence, not evidence of absence. Labs may be using CIRL-adjacent ideas under different names, and I may be underestimating the indirect influence of CHAI's framework.

What a thoughtful disagreer would say: "CHAI's value isn't in CIRL specifically -- it's in keeping the 'provably safe AI' research agenda alive in mainstream computer science. The paradigm-level argument (don't give machines fixed objectives) has already been absorbed by the field. You're judging CHAI by whether its specific technical framework got adopted, but the real impact was shifting the Overton window of AI safety within academia. That's hard to measure but genuinely important."

What information would most change my view: An internal assessment from a CHAI researcher or a detailed OP evaluation explaining what CHAI's counterfactual impact has been. Also, evidence of whether CIRL-type reasoning has influenced the design of RLHF or constitutional AI systems at any lab.