Mila - Quebec AI Institute

Stated Theory of Change

Mila has two distinct stated theories of change that operate in tension:

Institutional (Mila proper): Be the world's largest academic deep learning research institute. Train talent. Transfer knowledge to industry. Maintain Canadian AI competitiveness. The mechanism connecting this to AI risk reduction is indirect: by concentrating top research talent in an open-science academic environment (rather than closed corporate labs), Mila creates a space where safety research can happen alongside capabilities research. The AI Policy Fellowship, CAISI hosting, and "responsible AI" strategic priority represent institutional nods toward safety.

Personal (Bengio): AI capabilities are advancing dangerously fast. We lack the technical means to ensure alignment. Commercial and geopolitical incentives create an arms race. The solution requires: (1) a fundamentally different technical approach (non-agentic "Scientist AI" that understands without acting); (2) international governance (treaties, hardware verification, mandatory liability insurance); and (3) public awareness driving political will. Bengio has now channeled this into LawZero, a separate nonprofit.

The causal chain for Mila-as-safety-org: Mila trains excellent AI researchers in an open academic environment --> some fraction work on safety and governance --> their research and policy influence reduces AI risk. The causal chain for Bengio-as-safety-actor: Bengio uses his credibility as a Turing laureate --> to raise alarm and shape policy --> while building LawZero to develop a technically safer AI paradigm.

Revealed Theory of Change

Actions reveal a different picture than statements suggest.

Mila's revealed theory of change is capabilities maximization with safety as a side project. 700+ papers per year, overwhelmingly capabilities research. $250M compute campus. $100M VC fund. 53 startups. 176 industry partners. The AI Safety Studio works on chatbot deployment safety. The policy fellowship trains 13 policymakers per year. Safety-specific funding ($4.7M from Open Phil over 7 years) is dwarfed by the $137M+ from Quebec for competitiveness. The new Scientific Director (Larochelle) came from Google DeepMind and explicitly prioritizes commercialization and adoption.

The gap between stated and revealed theories of change is large. Mila claims safety as a strategic priority, but its budget, hiring, leadership appointments, and research output are overwhelmingly oriented toward capabilities and commercialization. The institution benefits from Bengio's safety credibility without structurally committing to safety work.

LawZero is the real safety vehicle. Bengio's most consequential safety work is now housed in a separate organization with its own funding. Mila takes partial credit ("leading the launch of LawZero") while keeping safety work organizationally and financially separate. This arrangement lets Mila claim safety credentials while pursuing its actual priorities.

The talent pipeline runs to frontier labs. The three most prominent Mila researchers after Bengio (Precup, Pineau, Larochelle) all have primary or recent industry affiliations with Google DeepMind, Meta FAIR, and Cohere. Mila functions as a training ground for frontier lab employees, not as an independent counterweight to them.

Key Assumptions

Assumption 1: Academic openness compensates for lack of institutional safety focus.

• Evidence for: Open-source research enables external safety research. Mila-trained researchers populate safety teams at labs. Academic freedom model lets safety-interested researchers pursue safety work.
• Evidence against: 700+ papers/year without systematic safety orientation. No institutional incentive to prioritize safety over capabilities. Researchers like Pineau explicitly disagree with x-risk framing.
• Testable: What fraction of Mila's output addresses safety/alignment? (Currently unreported.)
• If wrong: Mila is a net capabilities accelerator, not a safety contributor.

Assumption 2: Bengio's credibility-driven policy work reduces AI risk more than his former capabilities research increased it.

• Evidence for: International AI Safety Report influenced 30 countries. Senate testimony shaped US debate. Munk Debate reached mainstream audience. CAISI launched at Mila.
• Evidence against: US declined to back 2026 report. Trump administration actively dismissive ("Doomer narratives were wrong"). Munk Debate audience shifted toward skepticism. Policy progress is slow relative to capabilities progress.
• Testable: Has any concrete regulation been enacted based on Bengio's advocacy? (Limited evidence of direct impact.)
• If wrong: Bengio's pivot was symbolically important but practically insufficient.

Assumption 3: Scientist AI (LawZero's approach) can provide meaningful safety guardrails before frontier systems become dangerous.

• Evidence for: The concept is theoretically well-motivated (non-agentic systems cannot have misaligned goals). Bengio's team has published the approach (arXiv 2502.15657). $30M in funding from safety-aligned donors.
• Evidence against: The approach is untested at scale. $30M is tiny compared to frontier lab budgets. Even if Scientist AI works, it requires labs to voluntarily adopt it. No mechanism to compel adoption without regulation that doesn't exist.
• Testable: Can Scientist AI systems be demonstrated to provide reliable safety assessments?
• If wrong: LawZero produces interesting research but doesn't change frontier lab behavior.

Assumption 4: International governance is achievable before the technology it governs becomes uncontrollable.

• Evidence for: Historical precedent with nuclear arms control. 30 countries participating in Safety Report. CAISI, EU AI Act, California SB 53 represent real progress.
• Evidence against: US actively pulling back from international cooperation. China engagement uncertain. Tech industry lobbying increasingly effective. David Krueger (Mila professor): "I'd expect we need decades for the international coordination problem."
• If wrong: Safety relies entirely on technical solutions (like Scientist AI) or voluntary industry restraint.

Strengths

1. Bengio's unique credibility: As the most-cited living scientist and a Turing laureate who voluntarily pivoted from capabilities to safety, Bengio has credibility that few safety advocates can match. His willingness to acknowledge his own prior cognitive biases makes his argument more compelling, not less.

2. Research talent density: 1,200+ student researchers and 197 faculty in a single institution is unmatched. If Mila genuinely redirected even a fraction of this talent toward safety, the impact would be substantial.

3. Policy access: Bengio has direct access to heads of state, the UN, the OECD, and national legislatures. The International AI Safety Report is a real mechanism for consolidating scientific consensus on AI risks. CAISI hosted at Mila creates a tangible institutional link between research and policy.

4. LawZero's technical approach: The Scientist AI concept is intellectually rigorous and addresses a genuine gap. Non-agentic AI that provides safety assessments without pursuing its own goals could be a real contribution to the safety toolkit if it scales.

5. Open science model: Unlike frontier labs, Mila publishes everything. This transparency enables external safety evaluation and creates a culture where safety-relevant findings are shared.

Weaknesses and Risks

1. Institutional theory of change is capabilities-first, safety-second: Mila's funding structure, leadership, hiring, and research output are overwhelmingly oriented toward capabilities and commercialization. Safety is a side project, not the core mission. Government funding demands competitiveness, not safety.

2. Bengio's departure hollows out the safety case: With Bengio stepping down as Scientific Director and founding a separate organization (LawZero), Mila's claim to safety relevance weakens. Larochelle's priorities are explicitly commercial. Pineau actively disagrees with x-risk framing.

3. Dual appointment conflicts are structural: Precup at DeepMind, Pineau at Meta/Cohere, Larochelle from Google. These are not incidental conflicts -- they define the talent pipeline. Mila trains people for frontier labs.

4. LeCun-Mitchell critique is substantive: The argument that x-risk framing distracts from real harms, grants unwarranted credibility to AI capabilities, and potentially serves Big Tech interests through regulatory capture is not easily dismissed. The Munk Debate audience response (shifting toward skepticism) suggests this argument resonates.

5. Policy impact is limited: The US declined the 2026 Safety Report. The Trump administration is actively hostile to doomer narratives. International coordination faces the same obstacles as climate cooperation -- which has largely failed on the timelines that matter.

6. Element AI precedent: Bengio's previous commercialization attempt failed spectacularly. Mila's current $100M VC fund and $250M campus raise questions about whether the same gap between AI research prestige and commercial viability will repeat.

Cross-References

• CHAI (UC Berkeley): Similar model of academic AI safety research led by a prominent researcher (Stuart Russell). But CHAI was purpose-built for safety; Mila was purpose-built for capabilities and added safety later.
• FHI (Oxford): Analogous case of a university-affiliated institution with outsized influence through its founder (Bostrom). FHI eventually closed; LawZero's separation from Mila may be an attempt to avoid this fate.
• Anthropic: Bengio criticizes frontier lab incentives, but Mila's talent pipeline feeds directly into Anthropic, OpenAI, and DeepMind. The relationship is complementary from a capabilities perspective, adversarial from a safety perspective.
• AISI (UK): Bengio joined their project. CAISI at Mila is the Canadian equivalent. These government safety institutes represent the policy track of Bengio's theory of change.
• Partnership on AI, GCRI, CSER: These organizations work on AI governance and policy. Bengio's International AI Safety Report partially overlaps with their mandates but has more scientific authority due to his credibility.

What Would Change This Assessment

• If Mila published data showing >10% of its research output is safety-focused: This would substantially upgrade the institutional safety assessment.
• If the Scientist AI approach produced a working prototype that frontier labs adopted: This would validate LawZero's theory of change and Bengio's optimism.
• If a major AI incident occurred: Bengio's political influence and Mila's policy infrastructure would become immediately more relevant.
• If Larochelle redirected Mila's research agenda toward safety: His priorities as of September 2025 suggest the opposite, but this could change.
• If the US re-engaged with international AI governance: This would unlock the treaty pathway that Bengio champions.
• If Mila's VC fund produced safety-focused startups: This would demonstrate that commercialization and safety can be aligned.

Self-Critique

Weakest claim: My assertion that Mila's safety work is "marginal" is based on the absence of evidence (unreported safety research fraction) rather than evidence of absence. It's possible that a significant portion of Mila's 700+ papers contribute to safety-relevant topics (interpretability, robustness, fairness) without being labeled as "AI safety." The boundary between safety and capabilities research is genuinely blurry.

Potential bias: I may be overly influenced by the dramatic narrative of Bengio's personal pivot, which makes the institutional story seem duller by comparison. Mila's decentralized platform model means important safety work may be happening without being centrally tracked or publicized.

What a thoughtful disagreer would say: "Mila's open science model IS the safety contribution. Every paper published openly enables safety researchers worldwide. The alternative -- Mila's talent going to closed corporate labs -- would be worse for safety. And Bengio's policy influence, enabled by his Mila affiliation, has done more to shift the Overton window on AI risk than any amount of technical safety research."

Information I most want: (1) What fraction of Mila's 700+ annual papers address safety/alignment topics? (2) Full financial statements showing budget allocation. (3) EA Forum/LessWrong community assessments of Mila as an institution. (4) Canadian T3010 nonprofit filings for detailed financials.

Sources I should have checked: Canadian T3010 filings, Google Scholar analysis of Mila's publication topics, detailed interviews with Mila researchers beyond Bengio and Pineau.