Foresight Institute

Stated Theory of Change

Foresight Institute's stated path to reducing AI risk has two prongs:

Prong 1 -- Security-first AI: Bring computer security, cryptography, and object-capability architecture to AI systems. Rather than aligning a superintelligent agent, build systems where multiple AI services check each other through cryptographic guarantees, market mechanisms, and formal verification. This draws directly on Mark Miller's object-capability work and Drexler's CAIS (Comprehensive AI Services) framework, which argues that superintelligent capabilities can be delivered as constrained services rather than autonomous agents.

Prong 2 -- Whole Brain Emulation: Accelerate WBE as a "differential technology" -- an AI architecture inherently more interpretable and value-aligned because it replicates human cognition rather than inventing alien optimization. Backed by a $20M Grand Prize endowment and annual fast grants.

The broader philosophy is d/acc (decentralized, democratic, differential defensive acceleration): build technologies that shift the offense/defense balance toward defense without concentrating power. Foresight operationalizes this through a funder/convener model -- making grants, hosting workshops, running fellowships, and building physical hubs (AI Nodes) that embed researchers in a mission-aligned community.

Revealed Theory of Change

Looking at what Foresight actually does (as opposed to says), several patterns emerge:

Community building is the real product. The majority of Foresight's activities -- Vision Weekends, workshops, seminars, fellowships, the Existential Hope website, the Meme Prize -- are community-building activities. Grants are a smaller fraction of total activity than the emphasis suggests. Allison herself describes Foresight's mechanism as "I often match people individually... that takes a lot of time but makes tailored connections happen." This is a convening theory of change, not primarily a research-funding theory of change.

The AI safety framing is relatively recent. Foresight was a nanotech org from 1986 to roughly 2020, with AI becoming a co-equal focus only after Allison's leadership. The pivot has been remarkably successful financially ($430K to $9.4M in 6 years) but the institutional depth in AI safety is thin compared to organizations that have focused on it for a decade or more.

Financial reality lags claims. Despite claiming $4.5-5.5M in annual AI safety funding, the 2023 990 shows $856K in grants. Foresight may genuinely intend to deploy $3-5M annually starting with the 2026 AI Nodes launch, but the claimed figures significantly outpace demonstrated reality. Meanwhile, $5.8M accumulated as net assets in 2024 alone.

d/acc ecosystem positioning is strategic. Foresight has successfully positioned itself as the institutional home of d/acc philosophy, connecting Vitalik Buterin's vision with concrete programs. This positions it to capture funding from the crypto/web3 community that the EA ecosystem doesn't reach. Protocol Labs and crypto donations are visible revenue sources.

Key Assumptions

Assumption 1: AI systems can be made safe through security/cryptography rather than alignment.

• Evidence for: Object-capability systems (seL4) have demonstrated formally verified security. Mark Miller's work is technically serious. Multiple agent architectures do reduce single-point-of-failure risk.
• Evidence against: Security against human adversaries is not the same as safety from optimization pressure. If AI systems are smart enough to be dangerous, they may be smart enough to find vulnerabilities in any security architecture. Traditional alignment researchers might argue this approach secures the locks while the AI has the keys.
• Testable? Partially -- seL4-style verification can be demonstrated on bounded systems. But scaling to AGI-level systems is unproven.
• If wrong: The security-first approach provides real computer security benefits but does not reduce existential risk from AI.

Assumption 2: WBE would be net-positive for AI safety.

• Evidence for: WBE systems would be more interpretable than black-box neural networks. They would inherit human motives. Development is more predictable (invertebrates before vertebrates before humans).
• Evidence against: 80K Hours identifies three serious risks: (1) WBE neuroscience could boost capabilities without helping safety (neuromorphic AI), (2) WBE adds a second dangerous transition to manage, (3) WBE may not remove risks from conventional AI.
• Testable? Only by doing the research. Eon Systems' fly brain is promising but the gap to human WBE is enormous.
• If wrong: The $20M WBE investment accelerates a capabilities technology with unclear safety benefits.

Assumption 3: There is enough time to build defensive technologies before AGI.

• Evidence for: As of 2026, AGI has not arrived. Defensive technologies (cryptography, formal verification, BCI) are advancing.
• Evidence against: Vitalik himself acknowledges that if timelines are 3-6 years, "we can't just accelerate the good." Leading AI labs expect increasingly capable systems each year.
• If wrong: Foresight's entire approach becomes irrelevant if AGI arrives before the defensive infrastructure exists. The d/acc strategy collapses.

Assumption 4: Convening tech-optimists around safety produces real safety improvements.

• Evidence for: Allison cites concrete matchmaking outcomes ($30M company founding, government funding). Workshop attendance by Anthropic researchers suggests frontier labs value the forum.
• Evidence against: "Bringing people together" is notoriously hard to evaluate. The matchmaking claims are self-reported and unverified. The "button soup" critique suggests some community members see the convening as extractive.
• If wrong: Foresight provides pleasant conferences for tech-optimists without measurably reducing AI risk.

Strengths

1. Genuine intellectual distinctiveness. The security/cryptography approach to AI safety is genuinely under-explored. Mark Miller's object-capability work and Drexler's CAIS framework represent a different intellectual tradition from MIRI/ARC/Redwood. This diversity of approaches has real option value -- if mainstream alignment hits a wall, security-first approaches might succeed.

2. Financial independence from AI labs. Zero compute credits, zero lab contracts, zero lab revenue. This gives Foresight credibility to criticize frontier labs that EA-funded orgs (dependent on Open Phil, which shares funders with Anthropic) may lack.

3. Access to non-EA talent and funding. Foresight reaches the transhumanist, crypto, and progress studies communities that the EA ecosystem largely misses. This is a genuine gap-filling function. The crypto/web3 funding pipeline is potentially enormous and mostly untapped for AI safety.

4. 40 years of scientific credibility in nanotech. The Feynman Prize's two Nobel alumni provide genuine scientific legitimacy that newer AI safety orgs cannot match.

5. Physical hub model (AI Nodes). If executed well, co-locating grantees in SF and Berlin with compute and community could produce research infrastructure that purely virtual grantmakers cannot.

Weaknesses and Risks

1. Grantmaking claims significantly exceed reality. The gap between "$4.5-5.5M claimed" and "$856K actual" (2023) is the most damaging finding. Even with charitable interpretation (future deployment planned), this creates credibility problems. Donors and the public deserve accurate figures.

2. Governance gaps are serious for a funder. No published conflict of interest policy despite board member Sonia Arrison funding Foresight through 100 Plus Capital, and board member Dean Tribble's commercial interest (Agoric) in the object-capability technology Foresight promotes. No published grant committee. No published recusal procedures.

3. Grant outcome tracking is absent. One Nature publication (Eon Systems) from the entire grants portfolio is a thin evidence base. For an org asking the world to trust it with $3-5M annually in AI safety grants, systematic outcome reporting is overdue.

4. The WBE bet may be net-negative. The 80K Hours analysis is compelling: WBE neuroscience could accelerate neuromorphic AI capabilities without producing aligned systems. If Foresight's $20M WBE endowment funds neuroscience that primarily helps capabilities researchers, the investment backfires.

5. Breadth dilutes focus. Seven focus areas across AI safety, longevity biotech, and molecular nanotech means AI safety is not even the sole priority. For AI safety donors, it's unclear what fraction of their dollars goes to AI safety versus nanotech or longevity.

6. Community-building is hard to distinguish from event business. Vision Weekend sponsorships ($5K-$20K), $10K-$30K donor benefit tiers, and $350 workshop tickets blur the line between program delivery and fundraising. The "button soup" critique resonates precisely because the org's outputs are largely events and connections rather than measurable research.

Cross-References

vs. LTFF/SFF/CG (established AI safety funders): Foresight funds different things (security/crypto approaches, WBE, multipolar AI) and reaches different communities (transhumanist, crypto). Grant sizes are smaller ($10K-$100K vs. LTFF's range). Foresight lacks the transparent decision-making and outcome reporting that established funders provide.

vs. MIRI: Foresight's intellectual tradition (CAIS, agoric computing) is philosophically opposed to MIRI's agent-foundations approach. However, Christine Peterson is on MIRI's advisory board, and Allison credits the LW Sequences as formative. These are complementary bets on different AI futures.

vs. Anthropic/frontier lab safety teams: Foresight has no capabilities research, no revenue conflicts, and operates at a fraction of the budget. Anthropic researchers participate in Foresight workshops, suggesting the relationship is collegial rather than competitive.

vs. FLI: Closest comparison as a non-EA-aligned organization working on existential risk. FLI is a direct partner ($290K grant, co-organized hackathon). FLI focuses on policy; Foresight focuses on technology.

What Would Change This Assessment

• Published grant outcome data showing concrete AI safety results. If Foresight demonstrated that grants produced tools, papers, or systems that frontier labs actually adopted for safety purposes, this would substantially increase confidence.
• Resolution of the grants discrepancy. An audited accounting of how claimed $4.5-5.5M in annual funding is calculated and deployed.
• Published conflict of interest policy with specific disclosure of the Arrison/100 Plus Capital and Tribble/Agoric relationships.
• Negative technical evaluation of the security-first approach from a credible alignment researcher. If someone demonstrated why object-capability security fundamentally cannot help with advanced AI alignment, Foresight's core thesis would be undercut.
• WBE research clearly accelerating capabilities. If Eon Systems' work (or similar Foresight-funded WBE research) were adopted by capabilities researchers to build more powerful systems, the WBE strategy would be net-negative.
• Successful deployment of AI Nodes. If the SF and Berlin hubs actually produce a functioning community of safety researchers with measurable output by 2027, the convening theory of change would be validated.

Self-Critique

What sources should I have checked but didn't?

• The LessWrong posts (especially the "grifty" post and the 2023 Progress post) could not be directly accessed. These would contain the most detailed community assessment.
• I did not find or read the "Gaming the Future" book or LW sequence by Allison, Miller, and Peterson. This could contain the most detailed articulation of Foresight's AI safety thinking.
• I could not access the 990 Schedule B (donor list) which would reveal major donor identity.

Where is this analysis potentially biased?

• I may be applying EA-centric evaluation criteria to an organization that explicitly rejects EA epistemics. Foresight's "measurement is almost impossible" stance could be correct -- some interventions genuinely cannot be measured, and demanding measurement could bias toward easily measurable but lower-impact work.
• The "button soup" critique and governance concerns may be receiving disproportionate weight because they fit an accessible narrative, while the genuine intellectual distinctiveness of the security-first approach is harder to evaluate.

What would a thoughtful person who disagrees say? "You're judging a convener by funder standards. Foresight's value is connecting people across communities (crypto, security, neuroscience, AI safety) who would never meet through EA channels. The fact that two Feynman Prize winners became Nobel laureates shows the org has genuine talent-identification ability. The grants are growing rapidly and the AI Nodes are about to launch. You're catching them in the build-up phase and treating the gap between aspiration and current deployment as dishonesty when it's just timing."

What's my single weakest claim? That the grants discrepancy indicates a credibility problem. It's possible that the $4.5-5.5M figure accurately represents multi-year commitments, non-cash support, and the AI Nodes launch pipeline that hasn't hit 990 filings yet. I should have more clearly distinguished between "grants given" on the 990 and "total funding deployed" which may include compute and space.

What information would most change my view? Detailed, audited accounting of where the $4.5-5.5M figure comes from and how it is deployed. If this turned out to include $2M in compute provision, $1.5M in office space, and $1M in cash grants, the picture would be different -- still worth scrutinizing, but not dishonest.