← AI Safety Orgs

Beneficial AI Foundation (BAIF)

Research

CG-funded safety research.

Founded
2022
HQ
Cambridge, MA (registered: Wilmington, DE)
Team
3
Structure
501(c)(3) nonprofit
Model
Donations

Theory of Change

BAIF's stated mission: "No one knows how to make tomorrow's AI safe. Let's change that." The causal chain: develop quantitative safety guarantees for AI systems using formal verification, analogous to how the FDA requires quantitative evidence of drug side effects and the FAA requires bounds on engine failure rates.

This takes its strongest form in Tegmark and Omohundro's September 2023 paper, which argues that "proof-carrying AGI running on proof-carrying hardware appears to be the only hope for a guaranteed solution to the control problem." The broader Guaranteed Safe AI (GSAI) framework paper (Dalrymple, Tegmark, Bengio, Russell et al., May 2024) softens this to a "family of approaches" using three components: a world model, a safety specification, and a verifier that provides "an auditable proof certificate."

Tegmark connects this technical agenda to a policy vision: an "FDA for AI" where companies must demonstrate safety before deploying powerful systems. In his October 2023 Senate testimony: "If a person or organization wants to be sure that their AGI never lies, never escapes and never invents bioweapons, they need to impose those requirements and never run versions that don't provably obey them."

What They Do

BAIF's primary concrete output is the Buterin AI Existential Safety Fellowship, co-managed with the Future of Life Institute. Since 2022, the program has funded 19+ PhD students ($40K/yr + tuition for 5 years + $10K research fund) and postdocs ($80K/yr + $10K research fund) at major universities. Notable fellows include Peter Park (AI deception survey, Nature coverage), Stephen Casper (red-teaming, RLHF limitations), and Caspar Oesterheld (game theory and AI). Applications for 2026 are open.

The fellowship includes an unusual anti-lab-pipeline clause: fellows who join frontier AI companies (Anthropic, Google DeepMind, Meta, OpenAI, xAI) within 2 years must donate half their gross compensation to charity.

In-house research is limited. The only confirmed BAIF researcher, Alok Singh (formerly Lawrence Berkeley National Lab), published "Vericoding" (Sept 2025) -- a benchmark of 12,504 formal specifications for verified program synthesis. BAIF's GitHub has 121 public repositories focused on Lean 4 formal verification tools.

QAISI (Quantitative AI Safety Initiative) lists seven prominent research leads -- Clark Barrett (Stanford), Yoshua Bengio (Mila), Steve Omohundro, Bryan Parno (CMU), Stuart Russell (Berkeley), Dawn Song (Berkeley), Max Tegmark (MIT) -- but has produced zero publications under the QAISI banner. These are advisory affiliations at home institutions, not employment relationships with BAIF.

BAIF organized events including the MIT Mechanistic Interpretability Conference (2023), ICML MI Workshop (2024), and Berkeley Proof Scaling Workshop.

Key People

Max Tegmark -- President and Treasurer. MIT physics professor, FLI co-founder and president, FQXi scientific director, Improve the News Foundation co-founder. Author of Life 3.0. Named Time 100 AI (2023). Receives $0 from BAIF; his $457,840 compensation is from MIT. He simultaneously leads at least 5 organizations, raising questions about time allocation to BAIF specifically.

Meia Chita-Tegmark -- Secretary. Tegmark's wife. FLI co-founder. Psychology PhD (Boston University), teaches AI ethics at Tufts. Human-robot interaction researcher.

Alok Singh -- Only confirmed in-house researcher. Former Lawrence Berkeley National Lab (2019-2024). Works on Lean 4 + AI formal verification.

Team size is very small -- likely 1-2 staff plus the Tegmark couple as uncompensated officers. Both the /team and /fellows pages on the website return 404 errors.

Money and Incentives

Revenue: $0 (2022, founding year), $1,865,000 (2023, all contributions), $125,000 (2024). Revenue collapsed 93% from 2023 to 2024 with no public explanation.

Expenses: $821 (2022), $644,009 (2023). Zero salaries or wages in all years. The $644K in expenses is unexplained -- possible destinations include fellowship disbursements, contractor payments, and events.

Assets: $1,230,297 (2023), declining to $720,808 (2024). At 2023 expense rates, BAIF has roughly 1 year of runway.

Donors: Vitalik Buterin (Ethereum co-founder) and Erik Otto (Ethagi founder). BAIF receives zero funding from Coefficient Giving / Open Philanthropy. It is entirely outside the standard EA funding ecosystem.

Relationship to FLI: BAIF's sister organization, FLI (also founded by Tegmark), holds a ~$500M endowment from Buterin's 2021 SHIB donation. The Buterin Fellowship page lives on FLI's website and is described as "run in partnership." With 19+ fellows requiring potentially $1M+/yr in funding, and BAIF's 2024 revenue at only $125K, FLI is likely the primary funder. The financial flow between the two entities is undisclosed.

Donor risk: Buterin publicly criticized FLI's political pivot in March 2026, calling it potentially "authoritarian" and "fragile." He warned that FLI's regulation-first approach "VERY EASILY backfires." This creates uncertainty about future funding for the Tegmark ecosystem.

Business model: Pure donations from crypto-wealthy individuals. No product revenue, no government contracts, no EA funder oversight. This gives BAIF unusual independence but also unusual fragility.

Governance: BAIF has only two officers -- a married couple. No independent board. No advisory board. No public conflict-of-interest policy. Both officers receive $0 compensation from BAIF.

What Others Say

The core technical critique comes from Andrew Dickson's "Limitations on Formal Verification for AI Safety" (August 2024): Mathematical proofs work on symbol systems, not the physical world. The Mars Perseverance rover's formally verified software only covered ~20% of mission failure risk -- the 80% from physical causes remained unaddressed. Modeling whether a DNA sequence is harmful to humans would require "centuries" of progress in computational biology. "Our default stance as engineers and computer scientists should be skepticism."

Practitioner objection: Zac Hatfield-Dodds (FAR.AI talk, Hypothesis testing library maintainer): "Model weights are just completely intractable to reason about... The GSAI framework hasn't actually solved the problem, you've just moved it around -- now you have to convince me that the world model sufficiently accurately matches the real world." He also argues Tool AI is "unstable" because a trivial for-loop converts any tool into an agent.

The GSAI authors' response (Skalse et al., January 2025) retreats to a more modest position: GSAI "is not proposing to be a solution to ambitious alignment" but to "safety engineering applied to ASL-4 AI." They argue sound overapproximation is achievable without perfect world models, and even short-horizon guarantees are useful. This is more defensible than Tegmark's "only path" framing, but much less distinctive.

Dean Ball (Foundation for American Innovation, former Trump White House advisor) in debate with Tegmark: The FDA analogy breaks down because AI is a general-purpose technology. Preemptive regulation risks regulatory capture by incumbents. His P(doom) is 0.01% versus Tegmark's 90%+ -- a fundamental disagreement about whether this work is necessary at all.

ARIA's Safeguarded AI program (davidad/David Dalrymple, $80M UK government funding) is pursuing the same approach with 40x BAIF's peak budget: using frontier AI to develop formally verified control algorithms for critical infrastructure. ARIA's existence both validates the research direction and raises the question of whether BAIF is redundant.

What's Absent

  • No public explanation for why BAIF was created as a separate entity from FLI. Timing coincides with the Nya Dagbladet controversy (Dec 2022 founding, Jan 2023 controversy).
  • No QAISI-branded publications despite listing seven prominent research leads.
  • No explanation for the 93% revenue collapse from 2023 to 2024.
  • No disclosure of financial flows between FLI and BAIF.
  • No independent board members -- only a married couple as officers.
  • No public accounting of fellowship outcomes (publications, career trajectories, impact).
  • Zero community discussion of BAIF as an organization on LessWrong, EA Forum, or Alignment Forum.
  • No positioning relative to ARIA's $80M Safeguarded AI program pursuing the same approach.

Recommended Reading

  1. Vitalik Buterin on how SHIB became a $1B AI war chest (CoinDesk, March 2026) -- The most candid source on the funding ecosystem and Buterin's public criticism of FLI's political pivot. Essential for understanding the money behind BAIF. https://www.coindesk.com/markets/2026/03/14/vitalik-buterin-recounts-how-shiba-inu-tokens-became-a-usd1-billion-ai-policy-war-chest

  2. Limitations on Formal Verification for AI Safety (Alignment Forum, August 2024) -- The strongest technical critique of BAIF's core approach. Argues formal verification cannot produce strong guarantees about physical-world AI systems. https://www.greaterwrong.com/posts/B2bg677TaS4cmDPzL/limitations-on-formal-verification-for-ai-safety

  3. In Response to Critiques of Guaranteed Safe AI (Alignment Forum, January 2025) -- The GSAI authors' defense, showing how the position has moderated from "only path" to "portfolio approach." https://www.greaterwrong.com/posts/DZuBHHKao6jsDDreH/in-response-to-critiques-of-guaranteed-safe-ai

  4. Max Tegmark vs Dean Ball: Should We Ban Superintelligence? (Doom Debates, November 2025) -- The full 90-minute debate reveals fundamental cruxes: 0.01% vs 90% P(doom), and whether the FDA analogy holds for general-purpose technologies. https://lironshapira.substack.com/p/max-tegmark-vs-dean-ball-debate-ban-superintelligence

Show Claude’s analysis
An opinionated read. Read the brief first to form your own view.

Stated Theory of Change

BAIF claims that the path from current AI to safe AI runs through formal verification: develop mathematical proofs that AI systems satisfy safety specifications, using world models to bridge between formal guarantees and physical reality. The analogy is FDA/FAA-style quantitative safety requirements applied to AI. The causal chain:

  1. Fund researchers (Buterin Fellowships) to work on AI existential safety.
  2. Coordinate elite researchers (QAISI) around quantitative safety guarantees.
  3. Develop formal verification tools (Lean 4, proof systems) and benchmarks.
  4. Create proof-of-concept guaranteed safe AI systems for specific domains.
  5. Advocate for regulatory frameworks that require provable safety (FDA for AI).
  6. AI labs adopt formal verification as standard practice, making powerful systems demonstrably safe.

The bolder version (Tegmark/Omohundro 2023): "Proof-carrying AGI running on proof-carrying hardware appears to be the only hope for a guaranteed solution to the control problem." The more measured version (GSAI 2024): formal verification is one important family of approaches among others.

Revealed Theory of Change

BAIF's actions reveal a different picture than its stated ambitions suggest:

The fellowship is the real product. Roughly 19+ fellows funded since 2022 doing real safety research at top universities. This is concrete, measurable, and producing published work. The anti-lab-pipeline clause is structurally innovative -- it directly addresses a failure mode (safety researchers being absorbed by labs) that most orgs only complain about.

QAISI is a letterhead coalition. Seven prestigious names, zero branded publications, no evidence of active collaboration under the QAISI banner. The research leads do their work at home institutions. QAISI adds prestige to BAIF's website but little to the research landscape.

In-house research is minimal. One confirmed researcher (Alok Singh) plus Tegmark's intermittent attention. The 121 GitHub repos are real technical work, but they are the output of what appears to be 1-2 people.

BAIF is operationally an appendage of FLI. Same founders, same primary donor, shared fellowship program, FLI hosts the fellowship page, BAIF's budget is a rounding error relative to FLI's $500M endowment. BAIF appears to exist primarily to provide a separate institutional identity for the Buterin Fellowships and formal verification work, perhaps insulated from FLI's political activities and controversies.

Tegmark's time goes mostly elsewhere. With MIT professorship ($457K), FLI presidency ($500M endowment to steward), FQXi directorship, Verity News, and media engagements (Senate testimony, TED talks, debates, interviews), BAIF gets a fraction of his attention. The $0 compensation confirms this is a side project.

Key Assumptions

Assumption 1: Formal verification can produce meaningful safety guarantees for AI systems deployed in the real world.

  • Evidence for: Formal verification works for software (seL4 microkernel, Perseverance rover flight software). AI theorem proving is improving rapidly (miniF2F from 26% to 63% in one year). ARIA's $80M program suggests the UK government finds this plausible.
  • Evidence against: Mars rover verification only covered 20% of failure risk. DNA synthesis safety proofs would require "centuries" of biology progress. Model weights are intractable to verify directly. World model accuracy is a fundamental bottleneck.
  • Testable: Yes -- within 3-5 years, we should see whether ARIA's Safeguarded AI program produces working verified controllers for critical infrastructure.
  • If wrong: BAIF's core research direction is a dead end, though the fellowship program retains value independent of formal verification.

Assumption 2: AI labs will adopt formal verification under regulatory pressure.

  • Evidence for: FDA/FAA analogies show industries do adopt safety requirements. 95% of Americans reportedly don't want uncontrolled superintelligence development.
  • Evidence against: Dean Ball's critique that general-purpose technology resists domain-specific regulation. No binding AI safety regulation exists in the US. The Trump administration renamed the AI Safety Institute to the Center for AI Standards and Innovation, signaling a lighter-touch approach.
  • If wrong: Formal verification remains an academic exercise without market adoption.

Assumption 3: BAIF as a separate entity from FLI adds value.

  • Evidence for: Separating technical research from political advocacy may improve each. An independent entity may attract donors who wouldn't give to FLI.
  • Evidence against: Zero evidence of this actually happening. Same leaders, same donors, no independent governance. The 93% revenue collapse suggests BAIF failed to build its own funding base.
  • If wrong: BAIF should merge into FLI, which has the endowment to sustain the work.

Strengths

  1. The fellowship is genuinely valuable. Funding PhD students and postdocs to work on safety, with no strings attached to any AI lab, is one of the cleanest interventions in AI safety. The anti-lab-pipeline clause is structurally innovative.

  2. The QAISI advisory roster brings credibility. Having Bengio, Russell, Barrett, Song, and Parno associated with the formal verification agenda lends it weight in academic and policy contexts.

  3. Open-source tooling is a real contribution. 121 GitHub repos on Lean 4 formal verification tools is concrete technical infrastructure that others can build on.

  4. The research direction is plausible for narrow domains. Even if full "provably safe AGI" is a fantasy, formal verification of AI controllers for specific infrastructure (power grids, transportation, biodefense) is a reasonable research bet. ARIA's $80M investment validates this.

  5. Tegmark's public advocacy is effective. His Senate testimony, debate performance, and media presence put formal verification on the policy agenda. Whether or not BAIF is the right institutional vehicle, Tegmark's advocacy creates attention for the approach.

Weaknesses and Risks

  1. Organizational thinness. BAIF is essentially one researcher (Singh), two uncompensated officers (a married couple), and a letterhead coalition (QAISI). The $125K in 2024 revenue and declining assets suggest the organization may not be viable as an independent entity.

  2. Tegmark's overclaiming. The "only path" language in the Tegmark/Omohundro paper is not supported by the evidence. The GSAI authors themselves retreated to "one important approach." Overclaiming damages credibility and makes the field look naive.

  3. Governance vacuum. No independent board, no advisory board, no conflict-of-interest policy. A 501(c)(3) controlled by a married couple with no external checks is a structural governance failure, regardless of the couple's intentions.

  4. The FLI entanglement. BAIF inherits FLI's controversies (Nya Dagbladet, political pivot criticism, Buterin's public frustration) without having an independent identity strong enough to create distance. If FLI's reputation degrades further, BAIF goes down with it.

  5. The 93% revenue collapse is existential. With no explanation and declining assets, BAIF faces a genuine going-concern risk. If Buterin is the primary donor and he's publicly criticizing the Tegmark ecosystem, future funding is uncertain.

  6. ARIA makes BAIF potentially redundant. The ARIA Safeguarded AI program ($80M, dedicated team, specific use cases) is pursuing the same research direction with vastly more resources and institutional support. BAIF's marginal contribution to the formal verification for AI agenda is unclear.

  7. The core technical critique is unresolved. The formal verification community's own members (Hatfield-Dodds, the "Can We Secure AI" newsletter author rebranding away from "Guaranteed Safe AI") are distancing from the strong claims that are central to BAIF's messaging. The gap between formal verification's demonstrated capabilities and BAIF's stated ambitions remains large.

Cross-References

  • FLI (Future of Life Institute): Sister organization, same founders, shared fellowship program. FLI handles policy/advocacy, BAIF is supposed to handle technical research. FLI has the $500M endowment.
  • ARIA Safeguarded AI (davidad): The most well-funded effort pursuing GSAI with $80M UK government funding. Dalrymple co-authored the GSAI paper with Tegmark. Potential to make BAIF redundant.
  • MIRI: Previously pursued formal approaches to AI alignment but pivoted away. MIRI's trajectory is a cautionary tale for formal-methods-first approaches.
  • Atlas Computing / formal methods community: The broader FM-for-AI community is moving faster than BAIF, with tools like Harmonic ($120M Series C), Logical Intelligence, and open-source Lean efforts. BAIF's contribution is one node in a growing network.
  • FAR.AI: The Hatfield-Dodds talk was hosted here. FAR.AI represents the "formal verification is useful but don't overclaim" position.

What Would Change This Assessment

  • QAISI publishes a significant paper. If the seven research leads produce joint work under QAISI, it would demonstrate the coalition is real.
  • BAIF's revenue stabilizes. New major donations, especially from non-Buterin sources, would indicate the org has a future.
  • ARIA succeeds. Working formally verified AI controllers for critical infrastructure would validate BAIF's research direction.
  • ARIA fails. Would suggest the approach is harder than believed and raise questions about whether even larger investments can make it work.
  • BAIF and FLI formally merge or clearly separate. Either would resolve the governance ambiguity.
  • Tegmark steps back from some commitments. If he devoted significant time to BAIF specifically, the organizational output could change dramatically.
  • Independent board members are added. Would signal governance maturation.

Self-Critique

Weakest claim: My assessment that QAISI is "a letterhead coalition" could be wrong. The research leads may be actively collaborating through informal channels not visible in public outputs. University collaborations often don't produce branded publications.

Potential bias: I may be overly critical of the organizational structure because BAIF is so small. Small nonprofits run by founders with zero compensation are common and can be effective. The fact that Tegmark does this for free could be read as admirable rather than concerning.

What I missed: I was unable to access Tegmark's personal research group website (tegmark.org was down) or find the specific Medium investigation article through proper fetching. There may be more recent research output or organizational developments not captured.

What a thoughtful defender would say: "You're evaluating BAIF as if it should be a standalone research lab. It's not. It's a coordination vehicle for Tegmark's technical AI safety work, distinct from FLI's policy work. The fellowship alone justifies its existence. The QAISI branding helps attract attention and talent to the formal verification agenda. The $0 compensation means maximum donor dollars go to researchers. Stop applying Fortune 500 governance standards to a small mission-driven nonprofit."

Most important missing information: The actual financial flow between FLI and BAIF. If FLI funds $2M+/yr through BAIF for fellowships and research, then BAIF's standalone revenue is irrelevant and the 93% collapse is meaningless. If BAIF truly depends on its own fundraising, it's in serious trouble.

Connected to (8)

ARIA Safeguarded AIcollaborator · David Dalrymple
Mila - Quebec AI Instituteadvisor at · Yoshua Bengio
Stanford Universityadvisor at · Clark Barrett
UC Berkeleyadvisor at · Stuart Russell
Foundational Questions Instituteboard overlap · Max Tegmark
Future of Life Institutecollaborator · Max Tegmark
Future of Life Instituteboard overlap · Max Tegmark
Massachusetts Institute of Technologystaff from · Max Tegmark
Sources (55)
Every URL that was read during research.
  1. 1.Home2 — Beneficial AI Foundationbeneficialaifoundation.org
  2. 2.QAISI — Beneficial AI Foundationbeneficialaifoundation.org
  3. 3.Research — Beneficial AI Foundationbeneficialaifoundation.org
  4. 4.Max — Beneficial AI Foundationbeneficialaifoundation.org
  5. 5.Beneficial AI Foundationbeneficialaifoundation.org
  6. 6.Founders — Beneficial AI Foundationbeneficialaifoundation.org
  7. 7.Fellowship — Beneficial AI Foundationbeneficialaifoundation.org
  8. 8.Events — Beneficial AI Foundationbeneficialaifoundation.org
  9. 9.Jobs — Beneficial AI Foundationbeneficialaifoundation.org
  10. 10.Max Tegmark - Wikipediaen.wikipedia.org
  11. 11.Future of Life Institute - Wikipediaen.wikipedia.org
  12. 12.Shiba Inu's shib created a $1 billion AI war chest — and Vitalik regrets itcoindesk.com
  13. 13.Limitations on Formal Verification for AI Safetygreaterwrong.com
  14. 14.Vitalik Buterin Withdraws Support From Future of Life Institute Following $500M SHIB Donation Controversy - Blockonomiblockonomi.com
  15. 15.Vitalik Buterin Slams Future of Life Institute Over Political Pivotthecurrencyanalytics.com
  16. 16.In response to critiques of Guaranteed Safe AIgreaterwrong.com
  17. 17.Max Tegmark - The Lynchpin Factors to Achieving AGI Governance [AI Safety Connect, Episode 1] - Daniel Faggelladanfaggella.com
  18. 18.The AI safety showdown: Max Tegmark on government, Anthropic, and what’s nexttechcrunch.com
  19. 19.Future of Life Institute (FLI) - InfluenceWatchinfluencewatch.org
  20. 20.Technical PhD Fellowships - Future of Life Institutefutureoflife.org
  21. 21.Statement on a controversial rejected grant proposal - Future of Life Institutefutureoflife.org
  22. 22.Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemsarxiv.org
  23. 23.Future of Life Institute to Launch Vitalik Buterin Fellowships Centered on AI Safety Researchcoindesk.com
  24. 24.AI systems are already skilled at deceiving and manipulating humanssciencedaily.com
  25. 25.QAISI — Beneficial AI Foundationqaisi.ai
  26. 26.Beneficial AI Foundationgithub.com
  27. 27.Beneficial Ai Foundation - Nonprofit Explorer - ProPublicaprojects.propublica.org
  28. 28.AI Safety Research Scientist — Formal Verification (Remote)forum.nunosempere.com
  29. 29.A benchmark for vericoding: formally verified program synthesisarxiv.org
  30. 30.Thoughts on Max Tegmark's AI verificationgreaterwrong.com
  31. 31.Provably Safe AIgreaterwrong.com
  32. 32.In their quest for AGI safety, researchers are turning to mathematical proofthe-decoder.com
  33. 33.Provably safe systems: the only path to controllable AGIarxiv.org
  34. 34.Max Tegmark on how a 'put-up-or-shut-up' resolution led him to work on AI and algorithmic news selection | 80,000 Hours80000hours.org
  35. 35.Beneficial AI Foundation (BAIF). No one knows how to make tomorrow’s AI safe. Let’s change that.blog.biocomm.ai
  36. 36.Davidad Dalrymple: Towards Provably Safe AIthegradientpub.substack.com
  37. 37.Davidad's Provably Safe AI Architecture - ARIA's Programme Thesisgreaterwrong.com
  38. 38.Can AI be used to control safety critical systems? A U.K.-funded research program aims to find out | Fortunefortune.com
  39. 39.Proof Scaling Meeting | Proof Scaling Meetingproof-scaling-meeting.vercel.app
  40. 40.Vitalik Buterin donations madedonations.vipulnaik.com
  41. 41.Vitalik Buterin Thanks Shiba Inu Community as Grant Program Beginscoindesk.com
  42. 42.Ethagiethagi.org
  43. 43.News — Erik A. Ottoerik-a-otto.com
  44. 44.Technical Postdoctoral Fellowships - Future of Life Institutefutureoflife.org
  45. 45.Written Statement of Dr. Max Tegmark to the AI Insight Forum - Future of Life Institutefutureoflife.org
  46. 46.Formal Verification is Overrated | Events at FAR.AIfar.ai
  47. 47.Provably Safe AI: Worldview and Projectsgreaterwrong.com
  48. 48.Steve Omohundrosteveomohundro.com
  49. 49.Steve Omohundro on Provably Safe AGI - Future of Life Institutefutureoflife.org
  50. 50.How to keep AI under controlted.com
  51. 51.Applications Open for 2026 Vitalik Buterin PhD and Postdoctoral Fellowships in AI Existential Safety - ResearchConnectmyresearchconnect.com
  52. 52.Can We Secure AI With Formal Methods? November-December 2025greaterwrong.com
  53. 53.Instructor Spotlight: Meia Chita Tegmarkexcollege.tufts.edu
  54. 54.Max Tegmark on AGI Manhattan Project - Future of Life Institutefutureoflife.org
  55. 55.Max Tegmark vs. Dean Ball: Should We BAN Superintelligence?lironshapira.substack.com