Center on Long-Term Risk (CLR)

Stated Theory of Change

CLR's stated theory of change has three levels:

1. Philosophical foundation. Suffering-focused ethics implies that preventing worst-case suffering should take priority over maximizing expected positive outcomes. Conditional on advanced AI existing, the worst outcomes involve astronomical suffering (s-risks), not extinction.

2. Research pathway. The most tractable s-risk pathways involve (a) cooperation failures between AI systems leading to catastrophic conflict, (b) AI systems developing malicious properties (spitefulness, sadism, punitiveness), or (c) powerful actors deliberately causing harm. Understanding and preventing these pathways reduces s-risk.

3. Concrete mechanism. Build expertise in AI model personas and cooperative AI. Publish legible empirical work that earns credibility with frontier labs and AI Safety Institutes. Use this credibility to advocate for specific interventions (SPI proposals, persona-shaping techniques) that reduce s-risk-conducive properties in AI systems.

The grantmaking arm (CLR Fund) and community building (fellowship, foundations course) extend this by growing the pipeline of people working on s-risks.

Revealed Theory of Change

CLR's actions tell a somewhat different story than its stated theory:

The pivot to empirical legibility. CLR's recent work -- emergent misalignment, inoculation prompting -- is valuable AI safety research that any alignment org could produce. The s-risk framing adds little to the papers themselves. In practice, CLR is doing general AI safety research while maintaining a suffering-focused brand. This isn't necessarily dishonest -- their model persona agenda statement explicitly connects malicious traits to s-risks -- but the causal chain from "understanding emergent misalignment" to "preventing astronomical suffering" requires many intermediary assumptions.

The cluelessness response. When their ED concluded the mission was epistemically bankrupt, CLR responded by doubling down on empirical work that has value regardless of whether the s-risk framing holds. This is pragmatically wise but philosophically evasive. It means CLR's survival strategy is to produce work that doesn't depend on s-risk reduction being tractable.

Research vs. advocacy gap. CLR's stated theory involves advocating for specific interventions with AI companies. But the evidence shows almost no interaction with frontier labs beyond being cited in follow-up work by OpenAI and Anthropic on emergent misalignment. The SPI proposals for AI companies are described as in-progress. The advocacy component of the theory of change is aspirational, not demonstrated.

Community building as side effect. The Summer Research Fellowship has produced notable alumni (Kokotajlo, Chan, etc.), but most graduates go to mainstream AI safety orgs, not to s-risk work. CLR's community building produces AI safety researchers who happen to know about s-risks, rather than dedicated s-risk researchers.

Key Assumptions

1. S-risks are a real and significant category of AI risk.

• For: The argument from disjunction is strong -- many pathways to astronomical suffering exist, and only one needs to be realized. Factory farming shows that economic forces can produce vast suffering despite widespread awareness that it's wrong.
• Against: 80,000 Hours judges the likelihood of s-risks to be "very low, much lower than risks of human extinction." If extinction is by far the more likely failure mode, resources spent on s-risks are misallocated.
• Testable? Not directly. But evidence of AI systems developing malicious properties (emergent misalignment) provides weak evidence that s-risk pathways exist.
• If wrong: CLR's empirical work still has general AI safety value, but the philosophical framing and community building are wasted.

2. We can identify and implement interventions that reliably reduce s-risk.

• For: The empirical personas work has produced concrete techniques (inoculation prompting) that demonstrably prevent specific failure modes.
• Against: Clifton's cluelessness argument: for long-run consequences, we may have reasons in both directions and no principled way to weigh them, so the net expected impact of any intervention is zero. The Conceptual Research team agreed.
• Testable? Partially. Near-term impacts of personas work are testable. Long-run s-risk implications are not.
• If wrong: The mission reduces to near-term AI safety work under a suffering-focused brand -- which may still be valuable, but doesn't justify CLR's distinct existence.

3. Suffering-focused ethics is the correct (or at least a reasonable) ethical framework.

• For: The asymmetry between suffering and happiness is intuitively strong. Most people would agree that preventing extreme suffering is more urgent than creating additional happiness.
• Against: Mike Johnson's critique: if suffering is computationally fuzzy and ultimately subjective, you cannot build a rigorous research program on preventing it. The "world destruction argument" against SFE notes that strict suffering minimization implies destroying the world is good if it eliminates more suffering than it creates.
• Testable? No, this is a normative question.
• If wrong: CLR's philosophical distinctiveness disappears, and they're just another (small, underfunded) AI safety research org.

4. There is a sufficient talent pool for s-risk-specific work.

• For: The Summer Research Fellowship consistently attracts applicants and produces hires.
• Against: Mia Taylor's departure shows that excellent AI safety researchers who don't share SFE may not stay. Fewer than 50 people work on s-risks worldwide (80K estimate). CLR's own Research Director left after 6 months partly because she wasn't suffering-focused.
• Testable? Yes, by hiring outcomes.
• If wrong: CLR cannot staff its research agenda and shrinks to a skeleton operation.

Strengths

Genuine intellectual contribution. CLR and its predecessors coined the "s-risk" concept, developed the cooperative AI research agenda, and produced the first formal analysis of commitment races, surrogate goals, and safe Pareto improvements. These concepts are now used across the AI safety community. Daniel Kokotajlo's work on commitment races, done at CLR, has become foundational.

The Nature paper. Emergent misalignment is a real, reproducible, and concerning phenomenon. CLR helped discover it. This gives them credibility that no amount of philosophical argumentation could provide.

Financial independence from labs. Zero compute credits, zero lab contracts, zero career pipeline from labs. CLR's incentives are not warped by relationships with the entities they're studying. This is increasingly rare in AI safety.

Honest about uncertainty. The cluelessness crisis, while damaging, shows an org willing to confront the hardest questions about its own mission. The annual reviews are unusually candid about research impact and organizational challenges.

The fellowship pipeline. Multiple cohorts of Summer Research Fellows have gone on to significant AI safety roles. The program is a genuine contribution to field-building, even when graduates don't work on s-risks specifically.

Weaknesses and Risks

The cluelessness problem is unresolved. The strongest intellectual case against CLR comes from its own former ED. Until someone publishes a rigorous response explaining why we have reason to believe s-risk interventions are net positive despite Clifton's argument, the philosophical foundation remains cracked. The board's disagreement is not a philosophical response.

Chronic leadership instability. Four leadership transitions in 8 years. The current MD (Cook) has no prior organizational leadership experience. The Research Director position is vacant. The Conceptual Research team departed. The org is being rebuilt from a diminished base.

Philosophical isolation. Open Phil is "ambivalent." 80,000 Hours rates s-risks below top-priority problems. The largest funders and career advisors in the EA ecosystem are not CLR's allies. This limits funding, talent pipeline, and institutional influence.

Tension between brand and output. CLR's most impactful work (emergent misalignment) doesn't depend on s-risk framing. Their most distinctive philosophical contributions (SPIs, cooperative AI theory) have limited external engagement. The question of whether CLR should be an s-risk org that does AI safety or an AI safety org with an s-risk perspective is unresolved.

Small scale. 17 employees and $3M budget in a field where frontier labs spend billions and even other safety orgs (METR, Redwood, ARC) are larger. CLR's ability to influence AI development is constrained by its resources.

No external oversight. All five trustees are CLR/EAF insiders. No independent board members. No external evaluation. The governance structure that navigated the cluelessness crisis is the same one that has presided over chronic leadership turnover.

Cross-References

Complementary to: MIRI (different approach to AI risk, both small and philosophically distinctive), Center for Reducing Suffering (sister org, more philosophical), Cooperative AI Foundation (funds some CLR work, overlapping research agenda), FOCAL at CMU (Oesterheld, former CLR researcher, leads related academic work).

Distinct from: Most AI safety orgs focus on x-risk (extinction), not s-risk. ARC, Redwood, METR, Anthropic's safety team all work on alignment/evals from an x-risk lens. CLR is the only substantial org with an explicit suffering-focused orientation.

Tension with: Open Philanthropy (which is ambivalent about suffering-focused framing), 80,000 Hours (which rates s-risks below x-risks).

Alumni connections: Daniel Kokotajlo (now ED of AI Futures Project), Caspar Oesterheld (FOCAL/CMU), various fellows at ARC, Redwood, Rethink Priorities.

What Would Change This Assessment

• A rigorous public response to the cluelessness argument that explains why the board believes s-risk interventions are net positive. This would restore intellectual credibility.
• Evidence that SPI proposals gain traction with AI companies. If even one major lab adopts SPI-inspired policies, CLR's theory of change is vindicated.
• Another major empirical publication that explicitly connects to s-risk reduction (not just general AI safety). This would demonstrate that the s-risk framing generates unique research insights.
• A leadership hire with prior organizational experience and mainstream AI safety credibility. This would signal that CLR is rebuilding, not just surviving.
• Open Phil removing its ambivalence and making a significant direct grant to CLR (not via EAF). This would indicate the suffering-focused orientation has been accepted by mainstream EA funders.

Conversely, the assessment would worsen if: the 2026 fundraise fails, Warncke departs, or the SPI proposals are rejected by labs with no interest.

Self-Critique

What's weakest: My assessment of the cluelessness argument's force. I treat it as serious but not fatal, partly because the same argument applies to all longtermist work. But Clifton's specific version is more nuanced -- he argues that reasons-based choice under cluelessness differs from standard expected value arguments. I may be underweighting this.

What would a thoughtful critic say: "You're too impressed by the Nature paper. One good publication doesn't validate a theory of change. CLR has been around for 13 years and has one externally legible output. The rest is niche game theory and decision theory that the AI safety community mostly ignores." This is a fair critique.

What did I miss: I could not access the EA Forum leadership change post in full (bot detection), which is described as the single most important document about CLR's current state. I also couldn't read the Bracketing Cluelessness academic paper (PDF encoding issue). Both would deepen my understanding of the cluelessness debate.

Where might I be biased: The cluelessness narrative is so dramatic (the ED left!) that it may be dominating my analysis at the expense of the quietly solid empirical work. CLR's Nature paper is a genuinely important contribution that most AI safety orgs would envy. The day-to-day research quality may be better than the organizational drama suggests.

Single weakest claim: That CLR's pivot to empirical work is "pragmatically wise." It could equally be seen as abandoning the philosophical core that justifies CLR's distinct existence, leaving them as a generic AI safety org with better game theory skills and worse funding than competitors.