Transluce

Stated Theory of Change

Transluce claims that AI safety requires independent, third-party oversight because companies face structural conflicts of interest. The causal chain:

1. Build automated interpretability tools that use AI to audit AI (scalability pillar)
2. Make these tools open-source so anyone can run audits (openness pillar)
3. Conduct public audits that surface model problems and generate pressure for improvement
4. Build institutional infrastructure (AI Evaluator Forum, AEF-1 standard) for third-party evaluation
5. The resulting ecosystem of independent evaluators provides checks that internal safety teams cannot

Steinhardt frames this through a complex systems lens: neural networks are complex adaptive systems where "straightforward attempts to control their behavior lead to unintended consequences." Safety requires diverse, overlapping approaches -- a "swiss cheese model" where multiple imperfect layers provide aggregate protection. Measurement infrastructure built now enables action later, analogous to how climate measurement enabled climate policy.

The theory is grounded in a specific epistemological position: neither pure empiricism (Engineering approach) nor pure theory (Philosophy approach) is sufficient for AI safety. Both are needed, and well-chosen experiments on current systems can tell us a lot about future ones.

Revealed Theory of Change

Transluce's actions are remarkably consistent with its stated theory. This is unusual.

Research output matches stated priorities. Their publications cluster around: (a) automated model auditing (o3 truthfulness, pathological behaviors, jailbreaking), (b) interpretability infrastructure (neuron descriptions, Monitor, PCDs), and (c) evaluation tools (Docent, investigator agents). No significant drift toward capabilities research or commercial products disconnected from the safety mission.

Open-source commitment is real. 10 GitHub repositories, 12,000+ downloads of neuron description tools, Monitor is publicly available. The open core model preserves both accessibility and sustainability.

Institutional building matches theory. Co-founding AEF with METR, RAND, and others is exactly what you'd do if you believed the third-party evaluation ecosystem was undersupplied. AEF-1 standard codifies independence requirements.

Where stated and revealed diverge slightly:

• The earned revenue (~20%) from "private companies and governments" introduces ambiguity about whether Docent users are paying customers or free adopters. If labs like Anthropic and DeepMind are paying, Transluce has a financial interest in maintaining good relationships with the organizations it audits. This is manageable but worth monitoring.
• Steinhardt's role as Open Philanthropy technical advisor creates a relationship with the dominant funder in AI safety that complicates claims of independence.
• The team composition is heavily technical. The governance hires (Stosz, Reich) are recent additions. For an organization whose theory of change requires policy and institutional change, not just technical tools, governance has been under-resourced relative to research.

Key Assumptions

1. Third-party evaluation can work without mandatory model access.

• Evidence for: Transluce has conducted audits using pre-release access (o3 from OpenAI), open models (Llama, DeepSeek, Qwen), and tools that work on model outputs. AEF-1 standard encourages voluntary access.
• Evidence against: Stanford HAI workshop found limited access is a primary barrier. Auditors face "a lack of protection against retaliation." Labs can simply decline access. Voluntary standards have limited enforcement power.
• Testable: Does Transluce sustain pre-release access from multiple labs? Do AEF-1 signatories actually follow through?
• If wrong: Transluce becomes limited to auditing open-source models -- a real but much narrower mandate than their vision.

2. Automated interpretability tools can scale faster than model complexity.

• Evidence for: Automated neuron descriptions outperform human quality at $0.046/neuron. Investigator agents discover jailbreaks automatically. Specialized smaller models can audit larger ones (AI2050 thesis).
• Evidence against: Models are growing faster than interpretability tools. The gap between what we can audit and what frontier models can do is widening. Nanda: "the ambitious vision just hasn't worked out very well."
• Testable: Can Transluce's tools detect a novel failure mode in a frontier model that simpler behavioral evaluations miss?
• If wrong: Transluce's tools provide a false sense of security -- they catch known problems but miss novel ones. This is the most dangerous failure mode.

3. Public audits create sufficient pressure on labs to change behavior.

• Evidence for: The o3 truthfulness finding got TechCrunch coverage and appears to have influenced OpenAI's own reporting on hallucination rates.
• Evidence against: Labs can ignore public criticism, especially from small nonprofits. The power asymmetry between a ~20-person nonprofit and OpenAI ($157B valuation) is enormous.
• Testable: Track whether labs change behavior in response to Transluce's specific findings.
• If wrong: Transluce produces accurate research that nobody acts on.

4. A nonprofit can attract and retain top ML talent in competition with labs and well-funded startups.

• Evidence for: Current team includes ex-OpenAI (Chowdhury), Berkeley PhD students on leave (Laidlaw), MIT PhDs (Meng, Schwettmann). Steinhardt's academic reputation attracts researchers.
• Evidence against: Goodfire pays $200-400K. Anthropic and DeepMind pay more. Transluce's nonprofit structure limits compensation. $11M budget supports ~20 people; scaling significantly would require proportional fundraising growth.
• Testable: Does attrition increase as team members receive outside offers?
• If wrong: Transluce becomes a training ground that loses its best researchers to better-resourced organizations.

Strengths

Intellectual depth and consistency. Steinhardt has been thinking about AI safety since 2011, before it was fashionable. His intellectual trajectory -- robustness, measurement, complex systems, scalable oversight -- is coherent and deeply considered. The "Concrete Problems in AI Safety" paper (2016) helped define the field. The complex systems perspective is a genuinely distinctive intellectual contribution that avoids both Engineering myopia and Philosophy abstraction. This is not a startup pivoting into safety for commercial reasons.

Theory of change aligns with the field's revised consensus. Neel Nanda, the field's most prominent mech interp researcher, has shifted from "interpretability might solve alignment" to "interpretability is one of multiple imperfect tools." Transluce's approach -- build practical tools, conduct public audits, create institutional infrastructure -- is exactly the pragmatic version that Nanda now advocates. Transluce was already pursuing this before Nanda's public update, which suggests they understood the realistic value proposition early.

Concrete, verifiable impact. Docent used for Claude 4 pre-deployment analysis (25+ organizations). O3 truthfulness finding published with data and covered by major media. AEF co-founded with RAND, METR, and others. These are not just papers -- they're institutional facts.

Nonprofit structure protects mission. No VC pressure for revenue growth. No equity valuation that demands returns. The open core model preserves accessibility while generating revenue. 5% overhead is remarkably lean. The structural incentives point toward mission fulfillment, not profit maximization.

Deep ecosystem connections. Steinhardt knows Christiano, Amodei, Olah, and many other AI safety principals personally, going back to 2012 SPARC and 2016 "Concrete Problems." These relationships provide access and influence that a startup without such connections would take years to build.

Weaknesses and Risks

Financial opacity is the single biggest concern. Zero named donors. No 990 data. EIN unknown. We cannot assess: (a) donor concentration -- if 50%+ comes from one source, that source has veto power; (b) lab dependencies -- if Anthropic or DeepMind funds Transluce, auditing independence is compromised; (c) funding sustainability -- $11M is modest and may need to grow with the team. This opacity is not inherently suspicious for a young nonprofit, but it is material for evaluating independence claims.

Governance is minimal. A three-person board where the CEO is a member. One non-founder board member has venture capital ties (McCormick, GreatPoint Ventures). No external safety advisory board with governance authority. No public conflict of interest policy. For an organization whose entire value proposition is independence, the governance structure should be exemplary, not minimal.

The "watchdog" theory of change has structural limits. Third-party evaluators face structural power asymmetries with the organizations they audit: (a) they depend on voluntary model access; (b) they depend on funding from the same ecosystem; (c) their findings can be ignored without consequence; (d) they cannot prevent deployment even if they find problems. AEF-1 addresses (a) partially through voluntary standards, but none of these are solved by technical tools alone.

Investigator agents are a serious dual-use capability. An 8B model that jailbreaks GPT-5, Claude Opus 4.1, and Gemini 2.5 Pro is both a safety tool and an offensive capability. The research paper acknowledges this but Transluce has no public responsible disclosure policy or dual-use governance framework. If these tools proliferate, they could be used more for attacks than for auditing.

Scale mismatch. ~20 people and ~$11M budget vs. frontier labs with thousands of employees and billions in revenue. Transluce can identify problems but lacks the resources to systematically audit even one frontier model, let alone all of them. The theory of change requires many organizations using Transluce's tools -- but if only Transluce itself does the auditing, coverage will be thin.

Cross-References

Goodfire (commercial interpretability, $207M VC): Same field, opposite structure. Goodfire pursues "intentional design" (training-time intervention) with VC money and commercial customers. Transluce pursues independent auditing with philanthropic money and open tools. Key comparisons:

• Goodfire's RLFR result (58% hallucination reduction) is a training-time intervention; Transluce's o3 finding is a post-deployment audit. Different points in the pipeline.
• Goodfire's "Most Forbidden Technique" concern (training against interpretability degrades interpretability) does not apply to Transluce's approach, which treats interpretability as a test set, not a training signal. This is consistent with Amodei's recommendation.
• Goodfire has 10x the funding and 2.5x the team size but faces commercial pressures that Transluce avoids. Transluce cannot match Goodfire's compute or compensation, but faces fewer mission-drift risks.
• Both organizations rely on the assumption that interpretability tools scale with model complexity. Both face the same fundamental challenge identified by Nanda: the ambitious vision of full understanding has failed.
• The field needs both: Goodfire-type tools for model developers to build better systems, and Transluce-type tools for external evaluators to audit them. They are complementary, not competitive.

METR (model evaluations, ARC Evals spin-off): METR does behavioral evaluations (evals); Transluce does interpretability-based auditing. Both are third-party evaluators. They co-founded AEF together. Complementary approaches.

Anthropic (frontier lab, in-house interpretability): Transluce's closest relationship and most complex dependency. Steinhardt co-authored "Concrete Problems" with Amodei and Olah. Docent used for Claude 4 pre-deployment analysis. Amodei publicly endorses interpretability urgency. But if Anthropic is also a funder (unknown), independence is compromised. If Anthropic builds in-house tools that supersede Docent, Transluce's adoption story weakens.

What Would Change This Assessment

Upward:

• Named donor disclosure showing diversified funding with no single source >25%
• Evidence that a Transluce audit directly caused a lab to change a deployment decision
• Independent replication showing Transluce's tools detect problems that behavioral evals miss
• Growth to 50+ people with diversified revenue (30%+ earned)
• Board expansion to 5-7 members with majority independent
• Schwettmann doing a long-form interview articulating her distinctive perspective

Downward:

• Donor disclosure showing >50% from one AI lab
• Key researchers departing for better-funded organizations
• Labs withdrawing pre-release access after unfavorable audits
• Transluce's tools shown to be no better than simpler behavioral evaluations
• Revenue from audited entities growing to where financial dependence is visible
• AEF standards not adopted or enforced

Self-Critique

Potential bias toward optimism about Transluce. I may be overweighting the consistency of Steinhardt's intellectual trajectory and the structural advantages of the nonprofit model. The absence of criticism could be because no one has bothered to critique a small, young org -- not because there is nothing to criticize.

Missing information that would change my view: Named donors. 990 data. Whether Docent adoption is paid or free. Whether labs have denied Transluce model access. Any of these could significantly shift the assessment.

Weakest claim: That public audits create sufficient pressure on labs to change behavior. The o3 case is one data point. It is entirely possible that labs treat Transluce's findings as PR irritants to be managed rather than substantive inputs to deployment decisions.

What a thoughtful disagreer would say: "You're being too generous to the nonprofit model. Nonprofits in the AI safety space have their own pathologies: dependence on a small number of wealthy donors, revolving door with the organizations they oversee, reluctance to publicly criticize funders or close associates. Transluce's ecosystem connections (Steinhardt advises Open Phil, co-authored with Amodei, etc.) are not just strengths -- they're the same kind of institutional capture risk you'd criticize in a for-profit. And the financial opacity you flagged as a concern could be hiding exactly this kind of dependency."

Sources I should have checked: Whether Transluce has received funding from Anthropic, DeepMind, or other labs (not findable without 990s). Whether there are internal governance documents (conflict of interest policies, etc.) that exist but aren't public. Whether any lab has refused Transluce model access.