Forethought

Stated Theory of Change

Forethought claims that AI will likely trigger an intelligence explosion within the next decade, creating a "century of progress in a decade" scenario with many grand challenges beyond alignment. These challenges -- AI-enabled coups, WMD governance, digital rights, space resources, epistemic disruption, institutional lock-in -- are critically neglected relative to their importance. Early foundational research on these topics can have outsized impact, just as Bostrom's "Superintelligence" in 2013-14 catalyzed the entire field of AI safety. Forethought positions itself as the successor to FHI and Open Phil's Worldview Investigations team: a small group of exceptional thinkers tackling big-picture questions that nobody else is asking.

Additionally, they argue that "Flourishing" (making the surviving future truly great) is at least as important as "Survival" (avoiding extinction), and that work on Flourishing is far more neglected.

The causal chain is: Forethought researches neglected grand challenges --> findings reach AI companies, policymakers, and the EA/rationalist community --> this catalyzes new research agendas, policy proposals, and organizational activity --> society is better prepared for the intelligence explosion.

Revealed Theory of Change

Forethought's actions are broadly consistent with their stated theory, with some interesting emphases:

What they actually produce: High-volume, medium-depth research papers self-published on their website. 20+ papers in year one. The most impactful outputs -- the coups paper and the IE modeling -- are the most technically grounded. The "Better Futures" series is more philosophical and speculative. They also produce a weekly podcast and newsletter.

How they actually influence: Through the EA/rationalist network. The 80K Hours problem ranking change is their clearest documented impact. The SAIF VC fund creation and AI company advising claims are weaker. They "communicate to the nerds" -- think tank researchers, AI company staff, EA-aligned government officials -- rather than broader audiences.

Where the stated and revealed theories diverge slightly: The stated theory emphasizes working "unrestricted by the current Overton window" on genuinely novel ideas. The revealed theory shows substantial overlap with existing EA/rationalist concerns (power concentration, lock-in, digital minds). These are indeed underresearched relative to alignment, but they are not entirely new to the community. The most genuinely novel contribution is the "Better Futures" framework and the specific grand challenges catalog.

What they optimize for: Intellectual prestige within the EA/rationalist ecosystem. Publication speed. Attracting exceptional researchers from the same talent pool (FHI alumni, ex-OP, CLR). This is a reasonable strategy for a young org but has insularity risks.

Key Assumptions

Assumption 1: Short AI timelines (2-8 years to mostly-automated AI R&D)

• Evidence for: Current scaling trends, benchmarks approaching human-level on key tasks, unprecedented investment. MacAskill cites the track record of "inside view" predictions outperforming outside-view skepticism.
• Evidence against: Consistent history of AI timeline overconfidence. Current models still fail on many tasks. Scaling may hit diminishing returns.
• Testable? Yes, and rapidly. This assumption will be clearly confirmed or refuted within 3-5 years.
• If wrong: The urgency evaporates. The grand challenges remain important in the long run but the "prepare NOW" framing loses force. Forethought's fundraising pitch weakens.

Assumption 2: The intelligence explosion creates challenges beyond alignment that are currently neglected and tractable

• Evidence for: The coups paper identified a genuinely novel threat model that gained traction. Historical analogies (nuclear weapons governance took decades to develop) suggest pre-planning matters.
• Evidence against: The AlgorithmWatch critique -- these speculative challenges may crowd out work on immediate, concrete AI harms. Also: if alignment is unsolved, most other challenges are moot (you can't govern space resources if a misaligned AI has already taken over).
• Testable? Partially. If AI-enabled coups become a real concern (even attempted), that validates the research. If the intelligence explosion unfolds differently than modeled, the specific challenges may be wrong.
• If wrong: Forethought's research portfolio is misallocated. But even wrong macrostrategy research has option value -- it trains people to think about these questions.

Assumption 3: Small teams doing foundational research can have outsized impact

• Evidence for: Strong historical precedent. Bostrom's "Superintelligence" with a small FHI team arguably catalyzed the entire AI safety field. Early nuclear strategy thinkers (RAND) shaped decades of policy.
• Evidence against: Survivorship bias. Many small teams doing foundational research had zero impact. The current policy environment may be too competitive for a 9-person Oxford org to influence.
• Testable? Only retrospectively, over years.
• If wrong: Forethought's research is intellectually interesting but practically irrelevant. This is the nightmare scenario they're trying to avoid.

Assumption 4: "Communicate to the nerds" is the right dissemination strategy

• Evidence for: The FHI model worked -- ideas percolated from a small academic group into mainstream policy. The EA/rationalist network includes people in key positions at AI labs and government.
• Evidence against: The EA network is smaller and less trusted post-FTX. Ideas validated only within this community may have blind spots. Mainstream policy researchers, IR scholars, and governance academics might reject or ignore work that doesn't engage with their existing frameworks.
• Testable? Yes, by whether Forethought's ideas gain traction beyond the EA community within 2-3 years.
• If wrong: Forethought becomes an intellectual echo chamber producing smart research that nobody outside the community reads.

Strengths

1. Genuinely novel intellectual contributions. The AI-enabled coups paper identified a real, previously under-analyzed threat. The Better Futures framework is a genuine philosophical innovation. The three-types-of-intelligence-explosion taxonomy is useful.

2. Exceptional talent density. Davidson's IE modeling is the most rigorous in the field. MacAskill's ability to synthesize across philosophy, economics, and AI strategy is unusual. The FHI/CLR/OP alumni bring institutional knowledge from organizations that no longer exist.

3. Speed and volume. 20+ papers in year one is remarkable for a 9-person org. By publishing on their own website rather than going through academic peer review, they sacrifice external validation but gain speed -- which matters if timelines are short.

4. Low incentive misalignment. No product revenue, no lab funding, no compute dependencies. The org's financial incentives are about as clean as you can get in the AI safety space.

5. Demonstrated policy impact. The 80K Hours problem ranking change is real and significant -- it redirects career advice to thousands of people. Whether the coups paper influences actual government policy remains to be seen, but the early signs (advising companies, MATS stream, International AI Safety Report contributions) are positive.

6. Intellectual honesty. MacAskill is transparently willing to admit past errors (timelines, FTX). Lizka's self-critical post was published with organizational approval. The founding announcement uses "galaxy-brained" unironically as a self-description.

Weaknesses and Risks

1. Timeline dependency. If short timelines are wrong, the entire urgency narrative collapses. The org has no published contingency plan for longer timelines.

2. Worldview homogeneity. Lizka's criticism is damning: "The set of worldviews & perspectives represented on the team feels too narrow." All staff share EA/rationalist backgrounds. All board members have direct ties to OP, CEA, FHI, or Redwood. There is no institutional mechanism to introduce genuinely outside perspectives.

3. Owen Cotton-Barratt governance concern. An active contributor with a sexual harassment history whose role is deliberately left ambiguous represents a real reputational and ethical risk. The silence about his status is inconsistent with post-FTX governance commitments.

4. The "galaxy-brain" trap. Macrostrategy research is inherently difficult to evaluate. Space governance and "deals with AIs" and "viatopia" may be genuinely important neglected topics -- or they may be intellectually entertaining exercises that distract from more concrete work. The absence of external peer review means there is no quality check beyond internal disagreement.

5. MacAskill reputational risk. MacAskill is simultaneously Forethought's greatest asset (intellectual leadership, network, name recognition) and greatest liability (FTX association, EA polarization). The org's identity is deeply intertwined with his, which creates single-point-of-failure risk.

6. Financial fragility. Two anonymous HNW donors controlling most early funding, with runway only through December 2026, creates real organizational risk. If the fundraiser underperforms, research plans must be cut. The org has no endowment, no diversified revenue, and falls outside major funders' core areas.

7. Impact measurement problem. Beyond the 80K Hours ranking change, impact claims are hard to verify. "Advising AI companies" is vague. The SAIF VC attribution is tenuous. For an org that espouses scope sensitivity and evidence-based reasoning, the impact evidence is largely anecdotal.

Cross-References

Most similar to: Future of Humanity Institute (intellectual lineage, Oxford base, big-picture thinking, small team), Open Phil's Worldview Investigations team (macrostrategy research), and to a lesser extent GovAI (AI governance research from Oxford).

Complementary to: Technical alignment orgs (MIRI, Redwood, Anthropic's alignment team) -- Forethought works on the problems alignment alone can't solve. Also complementary to policy-focused orgs (CSET, RAND AI) that need foundational thinking to build on.

Tension with: Orgs focused on near-term AI harms (AlgorithmWatch, AI Now Institute) that see longtermist/AGI focus as a distraction. Also tension with high-p(doom) alignment researchers who might see non-alignment grand challenges as premature if alignment itself is unsolved.

Gap-filling: Forethought uniquely occupies the "macrostrategy beyond alignment" niche. Nobody else is doing systematic research on AI-enabled coups, intelligence explosion dynamics, Better Futures, or space governance from an x-risk perspective at this level of depth.

What Would Change This Assessment

Positive updates:

• A Forethought-originated policy proposal is adopted by a government or AI company (moves impact from theoretical to demonstrated)
• An external academic review validates the coups paper or IE modeling as high quality (breaks the echo chamber concern)
• A major non-EA funder invests, suggesting the ideas have traction beyond the community
• The org publicly addresses OCB's role and establishes transparent governance documents

Negative updates:

• AI timelines extend significantly beyond 8 years (weakens urgency narrative)
• A former Forethought researcher departs with public criticism of the research quality or culture
• The fundraiser significantly underperforms, forcing layoffs (existential risk to the org itself)
• A specific policy recommendation turns out to be badly wrong or harmful
• Evidence emerges that the AI company advisory work is minimal or performative

Self-Critique

Weakest claim: My assessment of Forethought's impact beyond the 80K Hours ranking change. The org may be doing significant work advising AI companies that I simply can't verify due to confidentiality. I may be underestimating the importance of this unobservable advisory work.

Potential biases: I may be overly critical of the OCB situation relative to its actual importance. I may also be biased by the post-FTX narrative -- holding MacAskill to a higher standard than I would hold other researchers because of the FTX association. The FTX failures were not MacAskill's direct doing, and his governance lessons may be genuinely internalized.

What a thoughtful person who disagrees would say: "You're holding a 1-year-old, 9-person research org to standards appropriate for a major institution. Of course they don't have external peer review, financial transparency, or governance documents yet. The fact that they've published 20+ papers and changed 80K Hours' problem ranking in year one is remarkable. Give them time."

What I might have missed: The ForeCast podcast may contain important material. Manifund donor comments were inaccessible. I also have limited visibility into how the ideas are being received by government officials and AI company staff behind closed doors.

What would most change my view: Seeing a concrete, verifiable policy change at a frontier AI company or government that was directly influenced by Forethought's research. That would move them from "promising but unproven" to "demonstrably impactful."