OpenMined

Theory of Change

OpenMined's theory of change has evolved significantly since its 2017 founding but now centers on a claim from founder Andrew Trask's Oxford DPhil thesis: "Many of AI's risks in areas like privacy, value alignment, copyright, concentration of power, and hallucinations can be reduced to the lack of attribution-based control in AI systems." The proposed solution is to build infrastructure -- combining cryptography, distributed systems, and deep learning -- that enables AI models to preserve the link between data sources and predictions, allowing data owners to control their contributions and AI users to verify what sources inform outputs.

In practice, the near-term theory of change is more concrete: build privacy-preserving infrastructure (secure enclaves + PySyft) that enables external evaluation of frontier AI models without revealing proprietary data or model weights. If labs can be evaluated by third parties without IP risk, the argument goes, governments and researchers can hold AI developers accountable.

Trask describes the end-state vision as "broad listening" -- AI as a communication technology connecting people, not a centralized intelligence tool. "The most mature version of this tech is a communication technology. Deep learning becomes a communication technology" (CSM podcast, Sep 2025).

What They Do

Secure Enclaves for AI Evaluation: The flagship project. In December 2024, OpenMined piloted secure enclave-based AI evaluation with UK AISI and Anthropic using NVIDIA H100 TEEs, PySyft, and Azure confidential computing. Two organizations jointly governed a computation where each kept assets confidential. The entire process took 28 minutes. However, the pilot used GPT-2 as a proxy model and a public dataset -- not actual frontier models or sensitive data.

NIST CRADA (March 2026): NIST's Center for AI Standards and Innovation signed a collaborative research agreement with OpenMined for privacy-preserving AI evaluation methods -- the highest-profile government partnership to date.

Christchurch Call algorithmic auditing (2022-2023): Deployed PySyft at DailyMotion and LinkedIn, enabling 4 external researchers to study recommender systems without accessing raw data. Funded by NZ/US governments, Microsoft, and Twitter. Described as the first-ever privacy-preserving platform audit at scale.

Other deployments: Reddit external researcher access program, UN PET Lab cross-border statistical collaboration (US Census, StatCan, ISTAT), NAIRR pilot partner.

PySyft: 9.8K GitHub stars, open source under Apache 2.0. Mature enough for deployment at multiple organizations. SyftBox and BioVault represent newer expansions into developer tools and genomics.

Publications: "Beyond Privacy Trade-offs with Structured Transparency" (Trask, Dafoe et al., AAAI/ACM AIES 2020), "Enabling External Scrutiny of AI Systems with PETs" (CSET/Georgetown, 2025), secure enclaves blog post (30+ co-authors, Dec 2024).

Key People

Andrew Trask -- Founder and Executive Director. Simultaneously Senior Research Scientist at Google DeepMind, PhD student (ABD) at Oxford, CFR Term Member, and former FHI/GovAI affiliate. Author of "Grokking Deep Learning." His first job was doing on-prem AI for data too sensitive for the cloud (Digital Reasoning, 2011). Does not appear to draw salary from OpenMined.

Madhava Jay -- Head of Engineering since 2020. Self-taught engineer from Brisbane, Australia (Certificate IV in IT, Udacity AI Nanodegree). Discovered OpenMined while working at a MedTech startup. Oversees PySyft development. Salary: $143,333.

Team structure: 5 paid staff (total compensation: $623K), plus an open-source community of ~18,000 Slack members. The Padawan Program has mentored 550+ volunteers, with 80+ graduating to regular contributors. This volunteer-heavy model is unusual for an org building security-critical infrastructure.

Money and Incentives

Total known funding: ~$22.5M+

Revenue breakdown:

• Coefficient Giving / Open Philanthropy: $16,971,720 (75% of total) -- three grants under "Navigating Transformative AI"
  • Apr 2022: $28,320 (PETs + AI Safety research)
  • Sep 2023: $6,000,000 (Software for AI Audits)
  • Jun 2025: $10,943,400 (Secure Enclaves for LLM Evaluation)
• Future of Life Institute: $1,661,750 (Jul 2025)
• Historical Open Collective (2017-2024): $3,872,436 from diverse sources:
  • Sloan Foundation: $648K
  • Georgetown/CSET: $549K
  • Microsoft: $500K
  • Twitter: $499K
  • NZ Government: $450K
  • Meta: $600K (combined)
  • BitMEX: $300K
  • UCSF: $167K
  • Omidyar: $100K

Business model: 100% grant-funded. No earned revenue, no subscription product, no paid support for PySyft. Open source under Apache 2.0.

Funder concentration: CG/OP provides ~75% of all known funding. This is extreme dependency on a single funder.

Lab funding conflicts: Microsoft ($500K historically + Azure infrastructure), Meta ($600K), and Twitter ($499K) have all funded OpenMined -- the same companies whose platforms OpenMined audits. The Christchurch Call was directly funded by Microsoft and Twitter while auditing their algorithms.

Key conflict: Andrew Trask is simultaneously Executive Director of OpenMined AND Senior Research Scientist at Google DeepMind, a frontier lab whose models could be evaluated using OpenMined's infrastructure. No conflict of interest disclosure or recusal policy is publicly documented.

Salaries: Five paid staff total $623K. Ronnie Falcon (CPO): $192K, Madhava Jay (Head of Engineering): $143K, Peter Smith (CFO): $133K, Bennett Farkas (CMO): $96K, Lacey Strahm (Head of Policy): $58K. Trask does not appear to be compensated by OpenMined.

Financial transparency: No 990 filings available yet (IRS ruling year 2024). Charity Navigator cannot rate the org. First full financial disclosure should come in late 2026 or 2027.

What Others Say

No direct criticism of OpenMined exists in the public record. Despite extensive searching, no one has publicly argued that OpenMined's approach is wrong or that its funding is misguided. The org is effectively invisible to the LW/EA Forum discourse where AI safety organizations are typically scrutinized.

Indirect challenges to the approach:

Bruce Schneier on the TEE.fail attack (Nov 2025): "Yes, these attacks require physical access. But that's exactly the threat model secure enclaves are supposed to secure against." The attack defeats TEE protections from Intel, AMD, and NVIDIA with 3 minutes of physical access.

Gabriel Mukobi (UC Berkeley) identifies 10 failure modes for AI risk evaluations -- 6 ways evaluations fail to improve understanding and 4 ways understanding fails to improve mitigation. "Evaluations could even be harmful, for example, by triggering the weaponization of dual-use capabilities."

Duality Technologies documents TEE limitations: side-channel attacks, vendor trust requirements, single-server scalability constraints, and development complexity.

Open Philanthropy's own RFP on Capability Evaluations (Nov 2025) identifies three challenges for AI evaluations. OpenMined's infrastructure addresses only one (access constraints). The other two (inadequate benchmarks and underdeveloped evaluation science) are about what to evaluate and how to interpret results -- problems that privacy infrastructure does not solve.

Independent endorsement: The CSET/Georgetown arXiv paper provides the strongest third-party validation, concluding that "trustworthy privacy-preserving technical solutions for external scrutiny of AI systems have succeeded in real-world governance scenarios."

What's Absent

• Zero LessWrong / EA Forum / Alignment Forum posts -- extraordinary for a $17M+ CG grantee. The AI safety discourse community has not vetted, critiqued, or endorsed the theory of change.
• No deployment with actual frontier models. All demonstrations use proxy/simulated data. The gap between proof-of-concept and operational safety infrastructure remains unclosed.
• No public conflict of interest disclosure for Trask's dual DeepMind/OpenMined role.
• No public board list -- only one board member (a podcast host) identified through research.
• No public response to TEE.fail attack despite secure enclaves being the core technology of the $10.9M CG grant.
• No independent security audit of PySyft or the secure enclave implementation publicly documented.
• No 80,000 Hours podcast appearance or engagement with canonical AI safety podcasts.
• No engagement with technical alignment research community -- no citations of alignment papers, no collaborations with alignment researchers.

Recommended Reading

1. Interconnects interview with Andrew Trask (Oct 2024) -- https://www.interconnects.ai/p/interviewing-andrew-trask -- The most candid source. Trask explains the technology with genuine enthusiasm, honestly acknowledges the pilot used simulated data, and articulates a vision for retrieval-based models as safer AI architecture. Best source for understanding how Trask actually thinks.

2. Schneier on TEE.fail (Nov 2025) -- https://www.schneier.com/blog/archives/2025/11/new-attacks-against-secure-enclaves.html -- The strongest counterargument to OpenMined's core approach. Short and devastating.

3. CSET/Georgetown: "Enabling External Scrutiny of AI Systems with PETs" -- https://arxiv.org/html/2502.05219 -- Best independent technical assessment of what OpenMined actually does and what it means for AI governance.

4. Attribution-Based Control DPhil thesis pre-print -- https://attribution-based-control.ai/ -- The theoretical foundation. Understand this to assess whether the core claim -- that attribution solves AI's major problems -- is compelling.

5. Mukobi: "Reasons to Doubt the Impact of AI Risk Evaluations" -- https://arxiv.org/abs/2408.02565 -- The structural case that evaluation infrastructure may not reduce risk, challenging the entire value proposition.