MIT Algorithmic Alignment Group

Stated Theory of Change

AAG operates on two levels. The explicit theoretical contribution is that assistance games provide an analytical framework for studying value alignment: they allow researchers to "identify areas of research that are valuable" and "shape rough abstract ideas of what a safe interaction with advanced systems would look like." DHM is careful to note these are analytical tools, not deployable solutions.

The implicit theory of change is broader: by training interdisciplinary researchers at MIT, publishing in top venues, engaging with policy, and building student pipelines through MAIA, AAG shapes how the field thinks about alignment -- moving it from purely technical optimization toward economic, legal, and institutional framings. The Incomplete Contracting paper with Gillian Hadfield, the multi-stakeholder alignment work, and the policy engagement all serve this agenda.

Casper's theory of change is somewhat at odds with the lab's core research: he argues that technical alignment solutions are insufficient and potentially counterproductive, and that safety requires building permanent institutions. This tension is productive -- it pushes the group toward policy engagement and sociotechnical work that complements the theoretical research.

Revealed Theory of Change

Actions largely match stated intentions, with an interesting evolution. The Berkeley-era work (CIRL, Off-Switch Game, IRD) was primarily theoretical and aimed at the academic AI community. The MIT-era work splits into:

1. Practical auditing and evaluation (Casper-driven): RLHF survey, black-box audits paper, AI Agent Index, model tampering attacks. These directly serve the institutional safety agenda.
2. Policy engagement (DHM + Casper): evidence-based policy critique, international safety reports, TechPolicy.Press articles, public criticism of lab evaluation standards.
3. Interpretability and alignment methods (newer direction): SAE tools, in-context alignment, diverse preference learning.
4. Field building: MAIA, CAMBRIA bootcamp, mentorship (Pivotal, MATS, ERA, GovAI).

The shift from pure theory to practical tools + policy is genuine and visible. The group appears to have updated on the limitations of foundational theory and pivoted toward having impact through auditing infrastructure and governance.

One notable gap: despite Casper's institutional framing, the group hasn't built or proposed specific institutional structures. The "Pitfalls" paper proposes 15 regulatory goals but doesn't establish or design the implementing institutions. This is idea generation, not institution building.

Key Assumptions

A1: Analytical frameworks transfer to practice. The assumption that theoretical insights about assistance games, inverse reward design, and incomplete contracting will eventually inform how real systems are built. Evidence for: some concepts (uncertainty about objectives, reward function as observation) have influenced how the field thinks. Evidence against: no AAG formalism has been directly implemented in a deployed system at scale.

A2: Academic influence on policy matters. The assumption that publishing in FAccT, writing international safety reports, and critiquing evidence-based policy will translate to better regulation. Evidence for: the AI Agent Index generated significant media coverage; Casper is embedded in international policy processes. Evidence against: regulation has mostly progressed independently of academic alignment research; the ICLR policy blog hasn't obviously influenced any legislation.

A3: Multi-stakeholder alignment is tractable. DHM's argument that aligning AI with a single actor is dangerous and multi-stakeholder alignment is needed. This is a harder problem than single-principal alignment. If multi-stakeholder alignment is intractable at scale, the theoretical agenda hits a dead end. Testable: does the group produce algorithms that actually aggregate diverse preferences better than existing methods?

A4: The pipeline matters more than the papers. If AAG's graduates carry alignment-aware thinking into frontier labs, policy positions, and other research groups, the indirect impact through people may exceed the direct impact of publications. Currently untestable: student placement data is almost entirely missing.

A5: Slow takeoff and distributed AI development. DHM's skepticism about intelligence explosion means he expects AI risk to unfold gradually, making institutional solutions viable. If sharp takeoff occurs, AAG's long-term institution-building approach may be too slow. This is a deep worldview bet that shapes everything.

Strengths

Intellectual honesty. DHM's candor about his own framework's limitations -- "will not meaningfully reduce x-risk on its own" -- is rare in alignment research. Casper's willingness to say labs use alignment framing to justify power concentration is equally unusual. This honesty makes their analysis more credible.

Independence from frontier labs. No documented industry consulting, compute partnerships, or financial ties that would compromise judgment. DHM's public criticism of Anthropic's evaluation standards and Casper's critique of lab alignment narratives demonstrate real independence.

Interdisciplinary breadth. The team spans philosophy, economics, law, policy, and ML. The Incomplete Contracting paper, the multi-stakeholder alignment framing, and the parallels to tobacco industry regulation all reflect intellectual range beyond the typical CS-only alignment lab.

Practical policy products. The AI Agent Index, black-box audits paper, and ICLR policy blog are concrete contributions to AI governance infrastructure. These bridge theory and policy more effectively than most academic outputs.

Institutional positioning. MIT's brand legitimacy gives AAG's outputs credibility with policymakers and mainstream AI researchers who might dismiss work from less prestigious institutions.

Casper as bridge figure. Working simultaneously on technical safety (LAT, adversarial training), policy (international reports), and governance (audit standards) -- an unusual combination that connects otherwise siloed communities.

Weaknesses and Risks

Casper dependency. Much of AAG's visible impact -- the RLHF survey, AI Agent Index, policy engagement, international safety documents -- flows through a single PhD student. Post-Casper, the group's policy influence may sharply contract.

Theory-practice gap. No AAG formalism has been implemented in a deployed system. The assistance games framework is analytically useful but has not produced engineering tools that practitioners use. The pivot toward interpretability and auditing tools may address this, but it's early.

Small funding footprint. The $30K CG/OP grant is strikingly small. Whether this reflects CG/OP's assessment of AAG's impact or simply misalignment between funding channels, it suggests the group is not viewed as a priority investment by the largest AI safety funder.

Missing student data. Without knowing where graduates go, we cannot assess the pipeline's effectiveness. If MIT master's students who rotated through AAG end up at frontier labs doing capabilities work, the net effect could be negative.

No frontier access. Academic compute constraints and no documented lab partnerships mean AAG's technical work may not generalize to frontier-scale systems. LAT and model tampering are demonstrated on smaller models.

Institutional proposals without institutional construction. Casper argues for building institutions but hasn't built one. The group critiques existing evaluation standards without establishing alternative institutions. There is a gap between diagnosis and implementation.

Cross-References

CHAI (Berkeley): AAG is CHAI's intellectual descendant. DHM trained under Russell, Dragan, Abbeel. The core research program (assistance games) originated at CHAI. AAG differs by (a) more emphasis on multi-stakeholder alignment, (b) stronger policy engagement, (c) less emphasis on human-robot interaction experiments. CHAI's more direct lab experiments complement AAG's more theoretical approach.

Frontier labs: AAG occupies a useful adversarial position -- willing to publicly critique labs' evaluation standards and framing. This is valuable precisely because many academic groups self-censor to maintain lab relationships.

Casper's broader network: Through MATS, ERA, GovAI mentorship, international safety reports, and UK AISI experience, Casper connects AAG to a much broader institutional network than a typical academic lab.

CBAI ecosystem: Connected to Max Tegmark (FLI founder), Harvard AISST, and the ARENA bootcamp community through CBAI.

What Would Change This Assessment

• Evidence that AAG graduates systematically enter alignment-focused positions would significantly upgrade the pipeline theory of change.
• A deployed system using AAG-developed methods (assistance game formalism, LAT, auditing tools) would demonstrate practical impact.
• Post-Casper policy engagement maintained at current levels would show the institutional capacity isn't person-dependent.
• DHM taking a formal advisory role with a government AI safety body or frontier lab would bridge the theory-practice gap.
• A focused, well-argued critique of AAG's specific approach (not just general CHAI-style concerns) would help calibrate the assessment.
• Evidence of significant undocumented funding (large DARPA grants, industry consulting) would change the independence assessment.

Self-Critique

Weakest claim: My assessment that AAG is genuinely independent from frontier labs rests partly on absence of evidence (no documented industry ties) rather than evidence of absence. There may be informal consulting arrangements, advisory boards, or compute-sharing agreements that don't appear in public sources.

Potential bias: I may be giving AAG credit for honesty about limitations that actually reflects a lack of ambition. An org that says "our work won't directly reduce x-risk" could be praised for candor or criticized for not aiming higher. My framing leans toward the former.

What I'd want a critic to check: Whether DHM's federal grant portfolio (invisible to me) reveals priorities that diverge from the safety-focused work visible in publications. NSF grants often require broader impact statements that may tell a different story about the lab's priorities.

Missing perspective: I haven't found a critic who specifically targets AAG's approach. The critiques I've identified are either self-generated or directed at the broader CHAI tradition. A thoughtful outsider critique of AAG specifically would be valuable for calibration.

Information I most want: (1) Complete list of AAG graduate career destinations. (2) DHM's full grant portfolio. (3) CBAI's 990 filing. (4) Any documentation of informal lab partnerships or computing arrangements.