AE Studio

Stated Theory of Change

AE Studio claims a two-level theory of change:

Meta-level: The alignment field is stuck in local optima. Most researchers and funding concentrate on mechanistic interpretability and evals, which are valuable but insufficient. AE's role is to systematically find and implement "neglected approaches" -- directions that are individually unlikely to succeed but have high expected value given the vast unexplored space of alignment strategies.

Object-level: Biologically inspired alignment, primarily Self-Other Overlap (SOO) fine-tuning, which reduces AI deception by aligning how models represent self and other. The broader research agenda includes self-modeling, consciousness research, reverse-engineering prosociality, and various neuro-inspired approaches.

The operational mechanism is unusual: a bootstrapped consulting business generates ~$31.6M in revenue, and profits cross-subsidize alignment R&D. No external investors means no pressure to prioritize commercial returns over safety research.

Revealed Theory of Change

AE's actions largely align with their stated theory, but with important caveats:

What they actually publish: SOO (one paper, one workshop oral), self-modeling (one paper on toy models), PromptInject (one workshop paper), ESR (one blog post/paper), alignment survey, several blog essays, and consciousness research. This is roughly 2 substantive publications per year since the alignment pivot. Respectable for a side project of a consulting firm; modest for a dedicated alignment org.

Where the money goes: Unknown. This is the most significant divergence between stated and revealed theory. AE claims profits fund alignment, but the actual allocation is never disclosed. The vast majority of 150-198 employees work on commercial consulting. The alignment research team appears to be fewer than 10 people.

Time allocation of the CEO: Judd Rosenblatt's public activity splits roughly evenly between alignment advocacy (podcasts, LW posts, policy engagement, conferences) and commercial operations. His policy engagement (WSJ, City Journal, Congress, NSC) receives significant attention, suggesting AE's theory of change places weight on political influence alongside technical research.

The EVEnet question: AE's leadership launched a blockchain/crypto project (EVEnet) with the same core team in 2023-2024, planned a token launch, and then apparently let it go dormant without explanation. This suggests either significant distraction from alignment work or a quick update-and-pivot (consistent with their stated agility).

What they don't do: AE doesn't build or deploy frontier AI. They don't compete with labs. They don't do evals or red-teaming as a service. They focus narrowly on neuro-inspired alignment research and policy advocacy. This is consistent with a genuine focus on neglected approaches rather than following the crowd.

Key Assumptions

1. The alignment space is underexplored and neglected approaches have high EV.

• Evidence for: Their survey of 125 alignment researchers supports this. The history of science shows breakthrough discoveries from neglected directions.
• Evidence against: The EA ITN framework can justify any research by labeling it "neglected." The most popular approaches (interp, evals) may be popular because they're most tractable, not because of herding.
• Testable: Yes, by whether neglected approaches produce results. SOO is one data point in favor.
• If wrong: AE's meta-strategy is less valuable, though individual results (SOO) could still matter.

2. SOO generalizes from toy deception to real alignment threats.

• Evidence for: Results across 3 model sizes, 7 generalization scenarios, 2 extended scenarios. Larger models show greater reduction in deception. Eliezer's positive review.
• Evidence against: All scenarios involve simple, legible deception (burglar Bob). Real deceptive alignment involves concealed optimization targets, instrumental convergence, and mesa-objectives that are fundamentally different. The "training against internal state" critique suggests models could hide deception from SOO's loss function.
• Testable: Yes, by applying SOO to frontier models and testing against sophisticated deception benchmarks.
• If wrong: SOO is an interesting result that doesn't solve the core alignment problem, similar to RLHF -- useful for current systems but insufficient for superhuman AI.

3. A for-profit consulting firm can sustain high-quality alignment research.

• Evidence for: AE has produced genuine results. Bootstrapped independence avoids funder pressure. Judd's personal commitment appears sincere.
• Evidence against: No governance mechanism protects alignment commitment. Alignment team is tiny relative to commercial operations. Financial incentives eventually dominate personal values.
• Testable: Yes, over time. Watch for: alignment team size trends, publication rate, whether Judd's attention drifts.
• If wrong: AE becomes a consulting firm that once did some interesting alignment work -- like many companies with abandoned "labs" divisions.

4. Biologically inspired approaches transfer to AI systems.

• Evidence for: SOO results. Self-modeling induces simplification. Neuroscience provides a rich source of hypotheses about prosociality and cooperation.
• Evidence against: Neural networks are fundamentally different from biological brains. The analogy between human empathy (evolved over millions of years in social contexts) and fine-tuning loss functions is loose at best. "Inspired by" is different from "implements the mechanism."
• If wrong: The neuro-inspired framing is marketing rather than substance, though individual techniques could still work for other reasons.

5. Alignment can have a "negative alignment tax."

• Evidence for: RLHF improved capabilities. AE's SOO maintains capabilities while reducing deception. Ten examples cited in their military-grade essay.
• Evidence against: The hardest alignment problems (preventing deceptive alignment in superintelligent systems) almost certainly impose costs. The MIRI critique: alignment techniques work until the system is smart enough to game them. Current "negative tax" examples may reflect a pre-catastrophic regime.
• If wrong: AE's policy message ("alignment is good for business") becomes misleading, and the political support built on this message is fragile.

Strengths

1. Genuine independence. No investors, no major funder, no lab affiliation. AE can research whatever they believe matters without permission from anyone. This is rare and valuable in a field where most orgs depend on Open Phil or AI lab funding.

2. The SOO result is real. A concrete technique that reduces deception with minimal capability cost, works across model scales, and received Eliezer's rare positive review. Whether it solves alignment is debatable, but it's a genuine contribution.

3. Recruiting and supporting neglected researchers. The model of finding individuals with strong hunches (Marc Carauleanu as an undergrad, Michael Graziano at Princeton) and providing engineering support to test their ideas is a genuine organizational innovation in the alignment field.

4. Political positioning. Judd's conservative identity and bipartisan framing access political constituencies that the alignment community largely ignores. If AI safety becomes a partisan issue, having credible conservative voices is extremely valuable.

5. Proven ability to execute in hard domains. The BCI track record (Blackrock, Forest Neurotech, Neural Latents win) establishes that AE can do real work in technically demanding fields, not just write blog posts.

6. Low overhead alignment research. SOO fine-tuning on a single A100 in 65 minutes is extraordinarily capital-efficient compared to safety research at frontier labs. AE demonstrates that useful alignment research doesn't require billion-dollar compute budgets.

Weaknesses and Risks

1. Scale mismatch. AE describes itself with Manhattan Project / Bell Labs ambitions but has perhaps 5-10 alignment researchers working on cross-subsidized profits from a mid-size consulting firm. The gap between aspiration and resources is enormous.

2. Toy environment problem. SOO works on burglar Bob scenarios. The alignment community's core concern is deceptive alignment in superhuman systems. The distance between these is not incremental -- it's potentially qualitative. SOO might not help at all with the hardest alignment problems.

3. No accountability structure. One person (Judd) makes all decisions about alignment direction and spending. No board, no external oversight, no published metrics. If Judd's priorities shift, there is nothing structural to maintain AE's safety commitment.

4. Opacity about alignment investment. AE claims profits fund alignment but never discloses how much. This makes it impossible to distinguish between "a consulting firm with a serious alignment division" and "a consulting firm with an alignment marketing strategy."

5. EVEnet and credibility risk. Launching a crypto token with AI Safety R&D allocations and then apparently abandoning it without explanation is a credibility concern. The alignment community already views crypto-adjacent projects skeptically.

6. Breadth without depth risk. The agenda includes SOO, self-modeling, consciousness research, BCI, policy advocacy, startup incubation, field-building, and alignment angels. For 5-10 researchers, this is too many directions. The risk is spreading thin rather than building deep expertise in any one area.

7. Publication rate is modest. ~2 substantive papers per year is reasonable for a side project but thin for an org claiming to pursue the "most important work on the planet." Dedicated alignment orgs at similar size produce more.

Cross-References

Complementary to: Redwood Research (AE lists them as collaborator on AI control), Goodfire (AE does engineering work for them), Foresight Institute (grant funder, consciousness research connection), MATS (AE cited their framework, donated survey proceeds).

Contrasts with: MIRI (theoretical vs. AE's empirical, "negative alignment tax" vs. MIRI's pessimism about alignment being cheap), Anthropic (frontier lab with safety division vs. consulting firm with alignment side project).

Fills a gap: No other alignment org combines for-profit independence, neuro-inspired approaches, conservative political positioning, and the "find and implement neglected ideas" operational model. AE occupies a unique niche.

Potential redundancy: SOO-style representation engineering overlaps with work at Center for AI Safety (RepE), Anthropic (Constitutional AI), and others working on model internals. AE's specific biological inspiration differentiates them but the general approach has competitors.

What Would Change This Assessment

Strongly upward:

• SOO results replicate on frontier models (GPT-4 class) with similar deception reduction
• Independent replication by another group confirms core results
• Disclosure of alignment budget showing >20% of revenue or >$5M/year dedicated to alignment
• Alignment team grows to 20+ researchers with dedicated leadership

Strongly downward:

• SOO fails to generalize beyond "burglar Bob" style deception when tested on sophisticated scenarios
• Alignment team size stagnates or shrinks while commercial operations grow
• Judd's public attention shifts primarily to policy/politics and away from technical research
• EVEnet token launches and creates conflicts with alignment mission

Moderate updates:

• Self-modeling results scale to LLMs (upward)
• Alignment Angels competition results are published showing funded companies (upward)
• Financial transparency about alignment spending (either direction depending on numbers)
• Key alignment researcher departures with critical public statements (downward)

Self-Critique

Weakest claim: My assessment that the alignment team is "fewer than 10" people is based on inference from the team page (which lists mostly commercial roles) and the absence of disclosure, not on confirmed data. If AE has, say, 25 alignment researchers, the assessment would need significant revision upward.

Potential bias: I may be giving excessive weight to the "toy environment" critique because sophisticated deceptive alignment is the alignment community's central concern. But SOO reducing deception in current models has practical value even if it doesn't solve the existential problem.

What a thoughtful disagreer would say: "You're applying big-org standards to a bootstrapped startup that's 3 years into an alignment pivot. AE has produced more concrete alignment results per dollar than most grant-funded orgs. The EVEnet thing was a minor side project. The lack of financial transparency is normal for private companies. Judge them on their research output, not their governance structure."

What information would most change my view: The alignment team size and budget. If AE discloses that 30 people work on alignment with a $5M+ annual budget, this is a serious alignment org masquerading as a consulting firm. If the number is 3 people spending $300K, it's a consulting firm with an alignment hobby. The answer to this single question dominates everything else.

Sources I should have checked: Full Glassdoor reviews (for internal culture and alignment team dynamics), the Ami Magazine interview with Judd (unavailable due to 403 error), and the WSJ op-ed full text (paywalled). I also could not access the second Cognitive Revolution podcast (Cameron Berg, Nov 2025) in full due to size constraints, though I read the catalog description and key excerpts.