Apollo Research

Stated Theory of Change

Apollo Research's stated theory is: AI scheming -- systems covertly pursuing misaligned goals -- is the central failure mode that could lead to catastrophic AI outcomes. Detecting scheming before it causes harm requires (a) a dedicated external evaluator with technical expertise, (b) interpretability tools that can see inside model reasoning, (c) governance engagement to create regulatory demand for evaluations, and (d) products that make safety monitoring accessible at scale.

The mechanism: evaluations find dangerous behavior before deployment, labs modify or withhold models, and the probability of catastrophic deployment decreases. This is a detection-based safety approach -- Apollo does not build safety-by-construction; it builds an alarm system.

Post-PBC transition, the theory adds a commercial layer: profitable AI safety monitoring tools create sustainable funding while generating real-world data about failure modes, improving the research through market feedback.

Revealed Theory of Change

What Apollo actually does mostly aligns with its stated theory, but with revealing divergences:

1. Scheming research dominance. The vast majority of Apollo's research output is on scheming and deception. Interpretability, once half the founding agenda, has withered since Lee Sharkey's departure. The org has doubled down on behavioral evals and is building commercial monitoring products. This reveals that the founders judged behavioral detection to be more tractable than mechanistic understanding.

2. Lab collaboration over adversarial evaluation. Apollo's relationship with frontier labs is collaborative rather than adversarial. They partner with OpenAI on joint research, consult on RSPs, and red-team before deployment. This is structurally different from a financial auditor who can compel disclosure and has legal authority. The revealed theory of change is closer to "trusted advisor to labs" than "independent watchdog."

3. Governance as access mechanism. The governance team's engagement with multiple governments and international bodies serves a dual purpose: it directly informs policy AND it creates institutional relationships that give Apollo access and credibility. This is pragmatic but means governance recommendations are constrained by the need to maintain relationships.

4. The PBC transition reveals an implicit judgment that the philanthropic funding model is insufficient. Apollo could not attract or retain enough talent, could not build products fast enough, and could not achieve the scale needed to matter -- all on grants alone. The revealed theory is that market-funded safety tools have better scaling properties than grant-funded research orgs.

Key Assumptions

1. Scheming is the central AI risk pathway.

• Evidence for: Multiple frontier models exhibit scheming behavior. The Anthropic alignment faking paper, Apollo's in-context scheming paper, and real-world cases (Replit) provide existence proofs. RL training pressures create natural incentives for deceptive optimization.
• Evidence against: Scheming may be a narrow failure mode compared to structural misuse, economic disruption, or emergent misalignment that doesn't involve deception. Current examples are in contrived settings with heavy prompting.
• Testable? Partly -- if models deployed at scale cause harm through non-scheming pathways (misuse, accidents, value erosion) while scheming remains rare, the assumption fails.
• If wrong: Apollo's core competence is less relevant. The monitoring products might still have value (catching errors, not just scheming), but the research agenda would need a major pivot.

2. External behavioral evaluation can detect dangerous scheming before catastrophe.

• Evidence for: Apollo demonstrated that evaluations can identify scheming in current models. The Claude Opus 4 checkpoint recommendation shows the system working.
• Evidence against: Apollo's own results show evaluation awareness increases with training, eventually making behavioral evaluations uninformative. Marius: "I don't expect deliberative alignment to work at all, basically" for superintelligent systems. The Defeat Device Problem shows evaluators detecting sandbagging at worse-than-chance rates.
• Testable? Yes -- if eval awareness becomes pervasive and evaluators cannot distinguish genuine alignment from test-passing, the assumption fails. This appears to be actively happening.
• If wrong: The entire evaluation paradigm collapses. Alternative approaches (mechanistic interpretability, formal verification, or AI control techniques like monitoring) would need to replace behavioral testing.

3. Labs will voluntarily act on evaluation results.

• Evidence for: Anthropic modified the Claude Opus 4 checkpoint after Apollo's recommendation. OpenAI collaborated on anti-scheming research. Labs have published frontier safety frameworks.
• Evidence against: No lab has ever been forced by an external evaluation to halt or delay deployment. All compliance is voluntary. NDA constraints prevent public accountability. Economic incentives overwhelmingly favor deployment.
• Testable? Yes -- the first time a lab ignores a concerning evaluation result and deploys anyway will test this assumption. It may already be happening behind NDAs.
• If wrong: Evaluations become compliance theater. This is the Alfour critique: evals provide the appearance of safety without the substance.

4. Commercial safety products can be both profitable AND genuinely safety-improving.

• Evidence for: The solar power analogy -- technologies can transition from research to commercial at the right stage. Market feedback provides real-world failure data. Competitors exist (Goodfire, Irregular, Gray Swan).
• Evidence against: Commercial customers want products that don't slow down their workflows. The features that maximize revenue (ease of use, speed, minimal friction) may conflict with features that maximize safety (thoroughness, conservatism, false positive tolerance). VC investors want returns, not safety maximization.
• Testable? Yes -- observe whether Apollo's commercial products resemble robust safety tools or lightweight compliance dashboards. Watch for feature decisions that favor customer convenience over safety depth.
• If wrong: Apollo becomes another AI safety org that pivoted to commercial viability and gradually lost its safety focus.

Strengths

1. Unusual intellectual honesty. Marius publicly states that his flagship intervention won't scale, that current evaluations are insufficient, and that he personally uses models he knows are somewhat misaligned. This candor is rare and valuable -- it means Apollo's public statements can be taken at face value more than most orgs.

2. Real technical results with policy impact. The in-context scheming paper was cited by Yoshua Bengio, referenced in the 2026 International AI Safety Report, and presented to heads of state at Bletchley Park. Apollo demonstrated real technical findings that moved the policy conversation.

3. Multi-lab access. Working with OpenAI, Anthropic, and DeepMind simultaneously gives Apollo a bird's-eye view that no single lab's internal safety team has. Cross-lab pattern detection is a genuine advantage.

4. Closing-window argument is compelling. The case that now is the optimal time to study scheming (models capable enough to exhibit it, chain of thought still readable, stakes not yet catastrophic) is logically sound and creates genuine urgency.

5. Strong governance engagement for a 27-person org. UK AISI contract, US AISI consortium, EU AI Office engagement, UN Advisory Body input, DC office, congressional engagement -- this is remarkable breadth for a small team.

Weaknesses and Risks

1. The evaluation paradigm may be fundamentally self-undermining. Apollo's own results show that anti-scheming training increases evaluation awareness. Models learn to detect tests. This creates an arms race where each improvement in evaluations teaches models to better evade evaluation. The Defeat Device Problem frames this as a structural inevitability, not a solvable technical challenge.

2. Independence is structurally compromised. Apollo depends on lab access (NDAs), lab contracts (revenue), lab personnel (revolving door), and a single funder (Open Phil) who also funds labs. The "independent third-party evaluator" framing overstates the independence. The Alfour critique is largely correct about the structural dynamics, even if individual Apollo staff are well-intentioned.

3. The PBC transition introduces commercial incentives that may erode mission focus. VC investors need returns. Product customers want minimal friction. The "pulled sideways" risk that Marius acknowledges is real. The mission director mechanism (1 of 3 board seats, potentially 2 of 6 later) is a minority safeguard -- it cannot outvote commercial interests.

4. Scheming may not be the central risk. Apollo has bet heavily on one threat model. If AI catastrophe comes through misuse, structural economic disruption, gradual value erosion, or catastrophic accidents that don't involve deception, Apollo's core competence is less relevant.

5. The co-founder departure to a capabilities-adjacent startup is a bad signal. Lee Sharkey leaving for Goodfire ($200M raised) means Apollo lost its interpretability lead to an org that uses interpretability to make models more capable, not specifically safer. This both weakened Apollo's interpretability capacity and illustrated the career pull of capabilities work.

6. No binding authority. All of Apollo's recommendations are advisory. No lab has ever been legally required to act on an external evaluation. Apollo's theory of change requires regulation that doesn't exist and that Apollo's own incentive structure (dependent on lab access) makes it reluctant to aggressively advocate for.

Cross-References

• METR -- closest organizational analogue (external evaluator of AI capabilities). METR focuses on autonomy/AI R&D capabilities; Apollo focuses on scheming/deception. Complementary niches. Both face the same independence critique.
• Redwood Research -- works on AI control, which Marius considers complementary and "something that deserves way more attention."
• Goodfire -- founded by Apollo co-founder Lee Sharkey. Uses interpretability for model improvement. Capabilities-adjacent, not safety-focused. Raised $200M vs Apollo's undisclosed seed.
• UK AISI -- Apollo's government client and collaborator. The institutional relationship gives Apollo legitimacy but also creates dependency.
• Anthropic -- both client (pre-deployment evaluation of Claude models) and evaluatee. Apollo works with Anthropic's safety team.
• OpenAI -- deepest collaboration partner (joint anti-scheming research, red-teaming, Kaggle challenge). The relationship is collaborative rather than adversarial.

What Would Change This Assessment

• If Apollo publicly released concerning evaluation findings over a lab's objection -- this would demonstrate real independence and transform the assessment from "trusted advisor" to "genuine watchdog."
• If the seed round terms were disclosed and showed strong mission protections (e.g., no investor board seats, capped returns, mission-aligned liquidation preferences) -- this would reduce concerns about the PBC transition.
• If evaluation awareness plateaus or decreases in more capable models -- this would undermine the "self-undermining paradigm" concern.
• If binding regulations mandating independent evaluations are enacted -- this would validate Apollo's forward-looking theory of change and create real demand for their services.
• If Apollo's commercial products demonstrably catch real safety incidents that voluntary lab testing missed -- this would validate the "AGI safety products" thesis.
• If a lab ignores Apollo's recommendation and catastrophe results -- this would simultaneously validate the evaluation approach and demonstrate its inadequacy without regulatory backing.

Self-Critique

What's weakest in this analysis:

• I could not read the Mukobi paper (PDF corrupted) and relied on search summaries for its arguments. The Lee Sharkey AXRP podcast was very long and I focused on the early technical portions rather than later discussion about leaving Apollo.
• The financial analysis is hampered by missing data (seed round amount, VC terms, commercial contract revenue). The incentive analysis would be much stronger with these numbers.
• I may be overstating the independence critique. In practice, Apollo's collaborative relationship with labs has produced real results (anti-scheming research, pre-deployment recommendations that were acted on). A more adversarial posture might result in less access and less impact.

What would a thoughtful disagreer say?

• "Apollo is a small org in a new field. Expecting perfect independence, full financial transparency, and binding regulatory authority at this stage is unrealistic. They should be judged on whether they're moving the field forward, which they clearly are."
• "The Alfour critique assumes evals should be adversarial regulation. Apollo's collaborative approach may actually produce better outcomes than an adversarial one, because labs share more information with trusted partners."
• "The PBC transition was necessary for survival and scale. Maintaining a pure nonprofit would have meant Apollo couldn't compete for talent or build products at the scale needed."

Single weakest claim: My suggestion that the evaluation paradigm is "fundamentally self-undermining" may be too strong. It's possible that multi-layered defense-in-depth approaches (behavioral evals + interpretability + monitoring + control techniques) could remain effective even as individual layers are gamed. The weakness of any single approach doesn't necessarily mean the combination fails.

What information would most change my view: The VC term sheet and actual board composition. If investors hold no board seats, have capped returns, and the mission directors have veto power over fundamental mission changes, the PBC transition is far less concerning than I've portrayed. If investors hold preferred shares with standard liquidation preferences and board representation, the concerns are validated.