US AI Safety Institute / CAISI

Theory of Change

The original AISI (Nov 2023 - June 2025) had a clear three-pillar theory: test frontier models before deployment, issue guidance and standards, conduct fundamental AI safety research. Elizabeth Kelly: "Our mission at the AI Safety Institute is really to advance the science of AI safety." She framed the three pillars as a "virtuous cycle" -- research informing testing, testing informing guidance, guidance informing research -- and explicitly rejected the safety-vs-innovation framing: "Safety promotes trust, which promotes adoption, which drives innovation."

The current CAISI (June 2025 - present) has a different theory. Commerce Secretary Lutnick: "For far too long, censorship and regulations have been used under the guise of national security. Innovators will no longer be limited by these standards." OSTP Director Kratsios: Biden "hijacked" NIST and "turned it into a safety model evaluation agency"; NIST should "go back to basics" on standards and measurement science, and then "the industry can do all the evals that they ever imagined."

The new mission centers on: (1) developing standards "free from ideological bias," (2) evaluating US vs adversary AI capabilities for national security, (3) serving as "industry's primary point of contact" for AI within the federal government.

What They Do

Evaluations. CAISI evaluates AI models, but the focus has shifted from pre-deployment safety testing of American frontier models to competitive intelligence against Chinese models. The publicly visible outputs since the rebrand are the DeepSeek evaluation (3 Chinese models vs 4 US models, finding DeepSeek lags in performance and has 94% jailbreak compliance vs 8% for US models) and the Kimi K2 evaluation. The joint US-UK pre-deployment evaluation of OpenAI o1 (Dec 2024) was the last publicly known evaluation of an American frontier model.

MOUs. Non-binding agreements with OpenAI and Anthropic for pre- and post-deployment model testing, first signed August 2024. These continued under the new administration (confirmed September 2025). However, no safety evaluation results from MOU testing have been published under the Trump administration. The full MOU text has never been made public.

Standards and guidance. NIST guidance documents: AI 800-1 (Managing Misuse Risk), AI 600-1, AI 100-4 (synthetic content), SP 800-218A (secure software development). The AI Risk Management Framework (created by Tabassi before AISI's founding) is used globally. The AI Action Plan directs CAISI to revise the AI RMF to remove references to "misinformation, DEI, and climate change."

Agent security. AI Agent Standards Initiative launched Feb 2026; RFI on securing AI agents issued Jan 2026; red-teaming competition on agent hijacking (jointly with Gray Swan + UK AISI, Mar 2026). This is technically substantive, forward-looking work.

Consortium. 280+ member organizations (renamed "NIST AI Consortium" from AISIC). Five working groups on biosecurity, safeguards, risk management, synthetic content, and red-teaming. Specific deliverables are largely unpublished.

International. Founded the International Network of AISIs with 10 countries (Nov 2024). The network was renamed to drop "safety" under US-UK pressure (resisted by other members). International consensus on automated evaluation best practices published Feb 2026.

Partnerships. NIST-MITRE $20M AI cybersecurity centers. CRADA with OpenMined for privacy-preserving evaluations. MOU with GSA for federal AI procurement evaluations. Signature Science $3M contract for CBRN AI testing. TRAINS taskforce for inter-agency national security AI testing (Commerce, Defense, Energy, DHS, NSA, NIH).

Key People

Elizabeth Kelly (First Director, Feb 2024 - Feb 2025). Co-authored Biden's AI Executive Order at the White House NEC. TIME 100 Most Influential in AI 2024. Built the original AISI from scratch in under a year, secured MOUs with OpenAI and Anthropic, launched the International Network, and attracted top researchers including Paul Christiano. Resigned shortly after Trump took office. In her farewell: "There is no other group with the technical skill or subject matter expertise to match AISI across the entire U.S. government."

Paul Christiano (Head of AI Safety, Apr 2024 - present). Invented RLHF at OpenAI. Founded ARC (Alignment Research Center). PhD UC Berkeley. Publicly estimates 50/50 chance of doom from human-level AI. His appointment caused a revolt among NIST staff (March 2024) who feared his EA ties would compromise NIST's objectivity. He remains at CAISI under the Trump administration but has been completely silent publicly -- no statements about the rebrand, mission change, or his work.

Austin Mayron (Acting Director, 2025 - present). Lawyer, not an AI specialist. Former Senior Legal Advisor at USPTO, former Deputy Associate Counsel at the White House. Replacing a tech policy leader with a lawyer signals administrative orientation rather than technical leadership.

Elham Tabassi (CTO, Feb 2024 - Mar 2025). Created NIST's globally adopted AI Risk Management Framework. 25+ years at NIST. TIME 100 Most Influential in AI 2023. Departed for Brookings in March 2025 -- removing the person who built NIST's most important AI governance tool.

Staff: ~71 currently listed (division 5701). 73 NIST probationary employees were fired in March 2025 DOGE-driven layoffs. CAISI was hiring for an AI Research Scientist as of December 2025.

Money and Incentives

Budget: ~$10M/year. FY2026 appropriations: $10M for CAISI (flat from prior years). Biden requested $47.7M for AI at NIST for FY2025; Congress provided far less. Actual spending was reportedly ~$6M in FY2024. The gap between requested and received funding reveals that Congress -- not just the Trump administration -- has been unwilling to fund AI safety at scale.

Comparison with UK AISI: 10x budget gap. UK AISI operates on ~$65M/year (GBP 50M), with GBP 240M committed for 2026-2030. UK AISI also disbursed GBP 40M in alignment research grants and receives in-kind compute from AWS, OpenAI, Anthropic, and Microsoft. The US -- which hosts all the major frontier labs and spends orders of magnitude more on AI R&D -- invests roughly one-tenth what the UK does on AI safety.

Comparison with expert recommendations. FAS proposes a "CAISI+" needing $67-155M/year operating budget and $155-275M in setup costs. Current CAISI funding is 7-15% of the minimum recommended level.

Adjacent funding. NIST-MITRE partnership: $20M (2x CAISI's entire annual budget, directed at cybersecurity). NIST overall: $1.847B in FY2026 (21% increase). NIST has resources for AI when it is framed as cybersecurity, manufacturing, or competitiveness -- not safety.

No external funding. As a federal agency, CAISI has no philanthropic, lab, or industry funding. This eliminates financial conflicts of interest but creates total dependence on political will. No grant-making authority or budget.

Talent competition. Private sector AI researchers can earn near $1M. Government salaries are a fraction of this. Biden's AI Talent Surge aimed for 500+ experts but hired only 250. Only 205 individuals received AI PhDs in 2022. CAISI cannot compete on compensation.

Incentive structure. CAISI's current incentives are to produce work that (1) is acceptable to the current administration (national security framing, US-vs-China evaluations), (2) does not antagonize frontier AI labs (voluntary cooperation, non-binding standards), and (3) can be cited by Congress as evidence that the US has AI governance capacity (justifying continued appropriations). There is no incentive to produce findings that would slow AI deployment or embarrass American companies.

What Others Say

Academic critique: An arXiv paper argues that AISIs and voluntary commitments function as substitutes for regulation rather than precursors to it. The feedback loop: labs adopt voluntary commitments, governments create AISIs to conduct evaluations, and the existence of this infrastructure is cited as evidence that binding regulation is unnecessary. "AISIs may actively prevent the binding regulation that would be more effective."

Safety org critique (CAIS): "Voluntary commitments are insufficient. Despite even the best intentions, AI companies are susceptible to pressures from profit motives that can erode safety practices." Helen Toner and Tasha McCauley (former OpenAI board members): "AI companies can't be trusted to govern themselves."

From within the administration (Kratsios): Biden "hijacked" NIST for "x-risk evals" and the agency "lost its way." NIST should produce standards; industry should do evaluations.

National security analysts (Lawfare): The DOGE layoffs constitute "a self-imposed, likely long-term brain drain" that "cuts against Trump's AI aims." The talent pool is tiny (205 AI PhDs in 2022). "Supporters of the layoffs may contend that by removing staff with a more risk-averse approach, Trump is increasing odds of rapid AI progress. But that argument assumes NIST sets the nation's AI priorities rather than what NIST actually does, which is technical implementation."

Equity critics (Data & Society): "Years of research aimed at addressing well-documented AI harms are being cast to the wayside as innovation is being framed as the only concept that matters." The shift from multi-stakeholder collaboration to "industry's primary point of contact" excludes civil society and academia.

Pro-acceleration critics (Reason): EA "doomsayers" have been "lobbying to create agencies with utterly alarming authority." CAIP's proposed bill would allow the government to "seize and destroy hardware and software." This critique explains the ideological opposition driving CAISI's direction.

Industry (mixed): Amazon, Meta, Microsoft, OpenAI, and Scale AI all signed letters urging Congress to authorize CAISI -- because it produces voluntary standards they can influence rather than regulations they cannot. This paradoxical support validates the anti-regulatory critique.

What's Absent

No public evaluation of American frontier models under Trump administration. DeepSeek and Kimi evaluations were Chinese models. The o1 joint evaluation was Biden-era. If CAISI is testing GPT-5, Claude Opus 4, or Gemini under MOUs, there is no public evidence.
Paul Christiano's voice. The person with arguably the most relevant technical expertise in any government AI role has said nothing publicly since joining NIST. No statements on the rebrand, the mission change, or alignment research.
No open-source tools. UK AISI released Inspect (widely adopted), ControlArena, and RepliBench. CAISI has released no equivalent evaluation tools.
No alignment research program. UK AISI runs the Alignment Project (GBP 27M, 60 grantees, advisory board including Bengio and Shlegeris). CAISI has no visible alignment research.
No grant-making function. CAISI cannot fund independent safety research.
No secure compute infrastructure. FAS notes CAISI cannot host model weights for deep evaluations without dedicated SL-5 compute.
No Frontier AI Trends Report equivalent. No comprehensive public assessment of AI capability and risk trends.
No published AISIC consortium deliverables. 280+ members, 5 working groups, but publicly visible outputs are minimal.