US AI Safety Institute / CAISI

Stated Theory of Change

Biden-era AISI (Nov 2023 - June 2025): Advance the science of AI safety through three reinforcing pillars: testing frontier models before deployment, issuing guidance/standards, and conducting fundamental research. The causal chain: AISI evaluates models -> identifies risks -> shares findings with labs (who fix vulnerabilities) and government (which can act on the intelligence) -> AI development becomes safer. Safety enables trust, trust enables adoption, adoption drives innovation. NIST's non-regulatory status was a feature: it allowed cooperation with labs who would resist a regulator.

Trump-era CAISI (June 2025 - present): Develop standards for AI that ensure American competitiveness and national security. Evaluate US models vs adversary models. Serve as "industry's primary point of contact" within the federal government. Guard against "burdensome and unnecessary regulation" of American AI by foreign governments. Ensure US dominance of international AI standards. The causal chain: CAISI produces standards -> industry coalesces around them -> American AI remains globally dominant -> national security is protected.

Revealed Theory of Change

CAISI's actions under the new administration suggest a theory of change that is neither the original safety vision nor the stated competitiveness vision, but something more pragmatic:

What CAISI actually optimizes for:

1. Institutional survival (continuing to exist despite hostile political environment)
2. Producing politically acceptable outputs (Chinese model evaluations, cybersecurity standards, agent security work)
3. Maintaining voluntary lab relationships (MOUs with OpenAI/Anthropic continued)
4. Avoiding confrontation on anything the administration considers "ideological" (no bias research, no x-risk framing, no deployment recommendations)
5. Building bipartisan legislative support for codification (the Cantwell-Young bill)

Where actions diverge from both stated theories:

• The original theory assumed CAISI would test American frontier models and publish findings. Under the new administration, publicly visible evaluations are exclusively of Chinese models.
• The current theory assumes CAISI enables American competitiveness. But $10M/year (1/10th of the UK budget) cannot meaningfully contribute to US AI leadership when private AI investment exceeds $100B/year.
• Both theories assume CAISI produces standards. But the AI Action Plan directs CAISI to remove risk categories (misinformation, DEI, climate change) from the AI Risk Management Framework -- which is political interference in standards-setting, not standards development.
• Paul Christiano -- the person whose entire career is about alignment and x-risk -- remains employed as Head of AI Safety at an organization that has deleted "safety" from its name. Either he is doing meaningful safety work that is invisible to the public, or his position is vestigial.

The revealed theory of change is closest to: CAISI maintains institutional continuity and lab relationships through a politically hostile period, hoping that either (a) the political environment shifts, (b) Congressional codification provides durability, or (c) a crisis creates demand for the expertise that currently has no outlet. This is a bet on surviving to be useful later, not on being useful now.

Key Assumptions

1. Standards without enforcement can meaningfully reduce AI risk.

• Evidence for: NIST's cybersecurity framework (widely adopted voluntarily) demonstrates that well-crafted standards can shape industry behavior without regulation. Kelly cited NIST's 120-year track record.
• Evidence against: Voluntary AI safety commitments have already frayed. The arXiv paper argues that voluntary standards substitute for rather than precede binding rules. CAIS: "AI companies are susceptible to pressures from profit motives." Former OpenAI board members: "AI companies can't be trusted to govern themselves."
• Testable: Track whether CAISI-published standards are adopted by labs, and whether adoption correlates with safer outcomes.
• If wrong: CAISI is producing compliance theater -- standards that labs cite for PR but do not meaningfully constrain behavior.

2. A $10M/year budget with ~71 staff can keep pace with frontier AI development.

• Evidence for: Kelly argued CAISI could "do a lot with our budget because of the talent we're attracting" and the 280-member consortium as a force multiplier.
• Evidence against: UK AISI has 10x the budget and ~250 staff. FAS argues $67-155M/year is the minimum for adequacy. CAISI has no secure compute, no grant-making, and no open-source evaluation tools. Private labs spend billions annually on development; $10M for safety is rounding error.
• Testable: Compare CAISI's publication output, tool releases, and evaluation cadence with UK AISI's.
• If wrong: CAISI is structurally incapable of its mission regardless of political environment. This is the most likely failure mode.

3. Chinese model evaluations serve national security.

• Evidence for: DeepSeek evaluation found genuine security gaps (94% jailbreak compliance). Understanding adversary AI capabilities is a legitimate national security function.
• Evidence against: Evaluating Chinese models is the politically safe version of AI safety. It lets CAISI demonstrate competence without confronting American labs. The real national security AI risk may be from insufficiently safe American models deployed at scale, not from Chinese models Americans choose to use.
• Testable: Does CAISI also evaluate American frontier models for national security risks? If not, the national security framing is selective.
• If wrong: CAISI becomes a competitive intelligence tool masquerading as a safety body.

4. Congressional codification will provide institutional durability.

• Evidence for: The Cantwell-Young bill has bipartisan support (Cantwell-Young-Hickenlooper-Blackburn) and broad industry backing (Amazon, Meta, Microsoft, OpenAI, SIIA, BSA, ITI). Cantwell: "It is unhealthy for each administration to restart."
• Evidence against: The bill has been introduced multiple times without passing. Even if codified, Congressional authorization does not guarantee adequate funding (see: NIST's chronic underfunding).
• Testable: Track legislative progress.
• If wrong: CAISI remains a political football, subject to redirection with each administration change.

5. Paul Christiano's continued presence means meaningful safety work is happening internally.

• Evidence for: Christiano is the most technically credentialed alignment researcher in any government worldwide. His continued employment suggests he is doing something.
• Evidence against: Federal employees can be effectively sidelined without being fired. His public silence is total. The organization he leads within CAISI ("AI Safety") has had "safety" removed from the parent organization's name.
• Testable: Watch for any CAISI publication or evaluation that addresses alignment, x-risk, or advanced safety concerns.
• If wrong: Christiano's presence is residual, and the alignment expertise he represents is unused.

Strengths

NIST's institutional legacy. NIST has 120 years of credibility in measurement science and standards. The AI Risk Management Framework (created by Tabassi) is used globally. This institutional brand cannot be easily replicated and provides a foundation for future expansion.

Lab relationships survived the transition. MOUs with OpenAI and Anthropic were renewed under the Trump administration. Whatever happens publicly, there appears to be an ongoing working relationship between CAISI and the major labs. This continuity is valuable.

Christiano's presence. Having the inventor of RLHF -- someone who understands AI alignment at a fundamental level -- inside the US government is significant even if his work is currently constrained. If a political window opens, the expertise is in place.

Bipartisan codification momentum. The Cantwell-Young bill has genuine cross-party and cross-industry support. If passed, it would be the most important structural improvement possible for CAISI.

Real technical work continues. Agent security standards, red-teaming competitions (with UK AISI and Gray Swan), privacy-preserving evaluations (OpenMined CRADA), and evaluation methodology development are substantive technical contributions regardless of political framing.

National security framing as camouflage. The DeepSeek evaluation, TRAINS taskforce, and cybersecurity partnerships allow CAISI to do safety-adjacent work under a politically acceptable label. National security is the language this administration speaks; using it to maintain technical capacity is pragmatic.

Weaknesses and Risks

Catastrophic underfunding. $10M/year is not a serious budget for evaluating frontier AI systems. The UK spends 10x more. FAS recommends 7-15x more as the minimum. At current funding, CAISI is structurally incapable of its stated mission -- whether that mission is safety evaluation or standards development.

No enforcement power, no deployment authority. Same fundamental weakness as UK AISI. CAISI can identify problems but cannot compel action. Non-binding MOUs with non-public terms and no published evaluation results means there is no way to verify impact.

Political redirection. The rebrand from "safety" to "standards and innovation" is not cosmetic. Directing CAISI to remove risk categories from the AI RMF (misinformation, DEI, climate change) is political interference in what should be scientific standards-setting. This damages NIST's 120-year reputation for neutrality.

Brain drain. Kelly (director), Tabassi (CTO, 25-year NIST veteran), and an unknown number of DOGE-driven layoffs have hollowed out leadership and institutional memory. Replacing them with a lawyer (Mayron) as acting director signals a shift from technical to administrative orientation.

Chinese-models-only evaluation pattern. If CAISI only evaluates Chinese models and never publishes safety evaluations of American frontier models, it becomes a competitive intelligence tool rather than a safety body. This would validate the anti-regulatory critique that AISIs exist to provide the appearance of governance without the substance.

Invisible outputs. AISIC consortium has 280+ members but few published deliverables. MOU evaluations are non-public. Christiano's work is invisible. Without transparency, CAISI's impact is unverifiable.

Substitution effect. The strongest critique: CAISI's existence may prevent binding regulation. Labs cite their cooperation with CAISI as evidence that they are being responsible, reducing pressure for mandatory rules. If this substitution effect is real, CAISI is net negative for AI safety.

Cross-References

UK AI Security Institute (AISI): Founded the same week at the same summit. Both renamed from "safety" in 2025. But the UK has ~10x the budget, ~3x the staff, open-source tools (Inspect), GBP 40M in alignment research grants, and a chief scientist (Irving) who speaks publicly with extraordinary candor. CAISI is the lesser version in every measurable dimension except one: it is located in the country where all the frontier labs are headquartered.

ARC (Alignment Research Center): Christiano founded ARC before joining NIST. ARC continues as an independent alignment research organization (now led by others). The question is whether Christiano's work at CAISI has any relationship to ARC's research agenda.

METR (Model Evaluation and Threat Research): Independent evaluation org that Christiano helped launch (spun out of ARC Evals). Does similar frontier model testing outside government. Can be more adversarial because it doesn't depend on lab goodwill.

OpenAI / Anthropic: The labs CAISI evaluates under MOUs. The power asymmetry is extreme: labs control access, MOUs are voluntary, and CAISI provides evaluation services. Whether CAISI can identify and communicate problems the labs don't want to hear about is the central open question.

Center for AI Policy (CAIP): Was the main US AI safety policy advocacy organization. Called DOGE layoffs evidence of "defy[ing] common sense." Then shut down in May 2025. Its closure reduces the external pressure for CAISI to do more.

FAS / CSIS / Brookings: Think tanks that have produced the most substantive external analysis of CAISI. FAS's CAISI+ proposal, CSIS's network analysis, and Brookings (where Tabassi now works) provide the policy infrastructure for future CAISI expansion.

What Would Change This Assessment

Upward:

• CAISI publishes a safety evaluation of an American frontier model (not just Chinese models). This would demonstrate that the safety evaluation function survived the political transition.
• Congressional codification passes with meaningful funding authorization ($50M+/year).
• Christiano publishes research or speaks publicly about alignment work at CAISI. This would indicate that the safety mission persists internally.
• CAISI releases open-source evaluation tools comparable to UK AISI's Inspect.
• Evidence emerges that CAISI evaluations have privately influenced a deployment decision.

Downward:

• Christiano leaves CAISI. This would confirm that the safety function has been fully abandoned.
• CAISI's budget is cut or the codification bill fails.
• AI Action Plan directions are implemented to gut the AI Risk Management Framework (removing entire risk categories).
• A model evaluated (or not evaluated) by CAISI causes significant real-world harm.
• Other countries withdraw from the International Network due to US politicization.

Self-Critique

Weakest claim: That CAISI is doing nothing meaningful internally. I do not have visibility into CAISI's day-to-day work. Christiano's presence and the continued MOU relationships suggest at least some safety work persists. My assessment may overweight the visible political narrative (rebrand, layoffs, Kratsios) and underweight invisible internal work. If CAISI is quietly evaluating American frontier models and sharing findings with labs, the practical impact is significantly higher than the public record suggests.

Potential bias: I may be excessively influenced by the UK AISI comparison. The UK is a much smaller country with a different political system; its AISI model may not be directly transferable to the US context. The US has NIST's broader institutional infrastructure, inter-agency coordination (TRAINS taskforce), and a more developed legislative pipeline. These structural differences may matter more than the budget gap.

What I should have checked but could not: Paul Christiano's current work (he is completely silent). Internal CAISI documents. Unpublished MOU evaluation results. Congressional briefing transcripts. Detailed FY2026 budget allocation within NIST. Forum posts on EA Forum and LessWrong about the Christiano appointment and CAISI transition (blocked domains).

What a thoughtful disagreer would say: "You are comparing CAISI to an idealized version of what a safety institute should be, rather than evaluating it within the constraints of American politics. The US has never had a binding AI safety regulator, and CAISI -- even in its weakened state -- represents more government technical capacity on AI than existed before November 2023. The codification bill, the continued lab MOUs, and the national security framing are pragmatic adaptations that keep the institutional infrastructure alive for a future administration that may re-empower it. Christiano staying is the tell: if the safety mission were truly dead, he would have left."

This is a reasonable argument. But it requires accepting that the entire purpose of a $10M/year federal body with 71 staff is to exist as a placeholder for a future political moment -- and that in the meantime, the world's most advanced AI systems are being developed and deployed without meaningful government evaluation. Whether that bet on future relevance is justified depends on how fast AI capabilities advance. If transformative AI arrives before the next political window opens, CAISI will have been an expensive observer of a catastrophe it lacked the power to prevent.

Single weakest claim: That the anti-regulatory substitution effect is real. I lean on the arXiv paper's argument that CAISI's existence crowds out binding regulation, but this is a theoretical claim about counterfactual policy dynamics. It is possible that without CAISI, there would be even less AI governance, not more. The claim is important but unverified.

Information that would most change my view: Any evidence that CAISI has conducted a safety evaluation of an American frontier model under the Trump administration and communicated findings to the lab. This would show the core safety function persists despite the political narrative.