← AI Safety Orgs

Center for Long-Term Cybersecurity (CLTC)

Research

AI + cybersecurity. Berkeley.

Founded
2015
HQ
Berkeley, CA
Team
18
Structure
university-affiliated
Model
Mixed

Theory of Change

CLTC's AI-safety-relevant theory of change operates through three channels:

Standards as governance infrastructure. CLTC develops risk-management frameworks that translate AI safety concerns into actionable guidance for developers and regulators. Their flagship GPAIS Standards Profile supplements the NIST AI RMF with specific practices for general-purpose AI, covering risks from bias and privacy through CBRN weapons and model autonomy. The logic: if you define what responsible AI development looks like in precise, standards-body-compatible language, you create the infrastructure for regulation, compliance verification, and accountability. As Barrett and Newman wrote: "Standards development is just a first step, and should be followed by the implementation of mandatory standards for foundation model developers and by ensuring that regulators have the authority and resources to enforce compliance."

Independent risk threshold definition. CLTC argues that AI companies should not be "the only ones deciding what is 'safe enough.'" Their intolerable risk thresholds work attempts to define measurable limits across 8 risk categories (CBRN, cyber, model autonomy, persuasion, deception, toxicity, discrimination, socioeconomic disruption). The argument: "Relying solely on voluntary self-governance leaves critical gaps in protections against high-risk AI systems that can destabilize economies, manipulate societies, exacerbate structural inequalities, or result in the loss of lives and property."

Talent pipeline. The AI Policy Hub trains interdisciplinary researchers to become AI governance practitioners. Three cohorts, 18 fellows, with alumni now at OpenAI safety research, FBI AI policy, Meta security evaluations, and academic positions.

What They Do

CLTC (founded 2015, UC Berkeley School of Information) operates across two strategic pillars:

AI Security & Policy:

  • GPAIS Standards Profile (v1.0 Nov 2023, v1.1 Jan 2025): First NIST AI RMF profile for general-purpose AI, endorsed by Google DeepMind, GovAI, FLI, CSET, FAS. Developed with 100+ stakeholders over one year.
  • Agentic AI Risk-Management Profile (Feb 2026): Addresses autonomous agents with guidance on proportional governance, human-machine oversight, and defense-in-depth. Panelists from Anthropic, Google DeepMind, Microsoft, GovAI, WEF.
  • Intolerable Risk Thresholds paper (Nov 2024/Jan 2025): Specific threshold recommendations across 8 risk categories, submitted to Paris AI Action Summit.
  • "How to Say No to the Next AI Release" (Jun 2025): Explicitly names Claude Opus 4 and Gemini 2.5 Pro as approaching risk thresholds, calls for independent oversight modeled on pharma/aviation.
  • AI Policy Hub: 18 fellows across 3 cohorts. Placements include Micah Carroll and Cedric Whitney at OpenAI safety (Whitney co-authored OpenAI's GPT-OSS model card).
  • Member of NIST AI Safety Institute Consortium. Staff participate in OECD, ISO, IEEE standards processes.
  • Responded to US government RFI on AI agent security (March 2026).

Public Interest Cybersecurity:

  • Cybersecurity Clinic: 300+ students trained since 2018 providing pro bono cybersecurity to at-risk nonprofits.
  • Consortium of Cybersecurity Clinics: 60+ clinics globally, 2,200 students and 700 organizations served in 2024-25.
  • Cyber Resilience Corps (with CyberPeace Institute): 45+ volunteer programs, 3,900 volunteers, 500 community orgs/year.
  • AI-Enabled Cybercrime initiative (Dec 2024): Tabletop exercises with law enforcement and industry, with Fortinet.

Key People

Ann Cleaveland, Executive Director (since 2018). Background in philanthropy/nonprofit management (ClimateWorks Foundation), not an AI technologist. Organizational continuity through four leadership transitions. Presented at WEF Davos 2024. Draws explicit parallels between cybersecurity advocacy and climate advocacy.

Jessica Newman (departed ~2025 for Microsoft AI). Former Director, AI Security Initiative. Co-authored the GPAIS Profile, intolerable risk thresholds, and most of CLTC's AI governance publications. Served on OECD and IEEE AI governance bodies. 100+ publications. Her departure leaves a significant gap in AI-specific expertise and external visibility.

Nada Madkour, Interim Director, AI Security Initiative. PhD in Technology (AI Risk Assessment). Crucially, she is a BERI (Berkeley Existential Risk Initiative) senior researcher, not a UC Berkeley employee. Connects CLTC to the existential risk funding ecosystem.

Leadership turnover: Founder Steve Weber retired Dec 2021. Faculty Director Chris Hoofnagle stepped down ~2025. Interim Faculty Director AnnaLee Saxenian is retired. CLTC is hiring a permanent AISI Program Director (posted March 2026). The center is in a transitional period.

Money and Incentives

Known funding totals at least $18.2M+:

  • Hewlett Foundation: $15M seed grant (2015), likely additional grants during 2015-2023 Cyber Initiative
  • Google.org: $2.2M (2023, cybersecurity clinics)
  • SFF via BERI: $522K (2023, AI standards)
  • Open Philanthropy: $245K across 3 grants (2021-2024, AI standards/risk management)
  • FLI: Unknown amount (2022, AI Policy Hub)
  • Craig Newmark Philanthropies: Unknown (2024, Cyber Resilience Corps)
  • Industry partners (Fortinet, Meta, Okta, Repsol): Undisclosed amounts

Funding for AI safety specifically is thin: ~$800K+ from identified AI-safety-adjacent sources (Open Phil, SFF/BERI, FLI) vs. $17M+ for the broader center. The bulk of funding supports cybersecurity programs.

Post-Hewlett sustainability is the central financial question. The Hewlett Cyber Initiative ($160M across grantees) ended December 2023. CLTC was an anchor institution but exact ongoing Hewlett support is unknown. The about page solicits individual donors, foundations, and industry affiliates.

Total annual budget is unknown. As a university center, CLTC files no public 990. Based on team size (~15-20 staff/fellows), program scope, and university overhead, annual budget is likely $3-8M but this is rough inference.

Industry advisory committee creates access dynamics. Seven external advisors from AWS, Google (former), Intel (former), Meta, Repsol, McAfee (former) are explicitly tasked with "fundraising and advocacy." Google.org is the largest known single grant ($2.2M), and Google DeepMind endorsed the GPAIS Profile -- the same company whose models the profile aims to govern.

BERI intermediary structure: The $522K SFF grant and Madkour's employment flow through BERI, connecting CLTC to existential risk funding without requiring CLTC itself to adopt x-risk language. Each Open Philanthropy grant to CLTC notes "an additional grant to BERI will support related work."

What Others Say

No direct criticism of CLTC exists in any public source found. Multiple searches returned nothing. CLTC occupies a position too mainstream for EA/rationalist critique and too academic for industry pushback.

The strongest applicable critiques are structural:

On voluntary standards: Holden Karnofsky (Carnegie Endowment) wrote that AI risk management is "still in its infancy" and achieving maturity "could take decades." He frames CLTC-style standards work as an "early release and iteration" approach -- necessary but inherently preliminary. CLTC itself acknowledges this gap, calling for mandatory standards and regulatory enforcement.

On "safety as boundary object": Wang (2024) argued that "AI safety" means fundamentally different things to the existential risk community and the sociotechnical safety community. CLTC bridges both -- its standards address CBRN/model autonomy (x-risk) alongside bias/discrimination (near-term). This bridging can be read as either unusually comprehensive or as diluting both agendas.

On the voluntary adoption gap: The NIST AI RMF is voluntary, and CLTC's profiles build on it. Industry criticism notes uneven adoption creates "disparities across the industry, where companies committed to governance may face higher costs and slower timelines." CLTC's "How to Say No" article explicitly argues against voluntary self-governance as sufficient.

Endorsements are broad and cross-cutting: GovAI, FLI, Google DeepMind, Federation of American Scientists, and CSET all endorsed the GPAIS Profile -- an unusual coalition spanning x-risk orgs, industry, and policy think tanks.

What's Absent

  • Total budget and resource allocation between AI and cybersecurity programs: Not publicly available.
  • No position on SB 1047: Despite being the most prominent AI safety legislation in CLTC's home state.
  • No adoption metrics for GPAIS Profile or risk thresholds: Endorsements but no usage data.
  • No EA Forum/LessWrong presence: Effectively invisible to the rationalist AI safety community.
  • No candid long-form interview with CLTC leadership on AI safety: No podcast equivalent of an 80K Hours or Inside View episode.
  • No details on Newman's departure or Hoofnagle's resignation as Faculty Director.
  • No technical AI safety research: Entirely governance/standards/policy -- dependent on others for technical risk identification.
  • No Wikipedia article: Unusual for a center of this size and tenure.

Recommended Reading

  1. "How to Say No to the Next AI Release" (TechPolicy.Press, Jun 2025) -- The most direct statement of CLTC's AI risk position. Names specific frontier models and their dangerous capabilities, argues for independent oversight. This is where CLTC drops the institutional voice and speaks plainly about the urgency of AI risk. https://www.techpolicy.press/how-to-say-no-to-the-next-ai-release/

  2. "Developing AI Risk Management With the Same Ambition as AI Products" (Carnegie Endowment, Karnofsky, Dec 2024) -- The best contextual piece and strongest implicit critique. Argues AI risk management must iterate as fast as AI itself, implying CLTC's standards work is valuable but inherently preliminary. https://carnegieendowment.org/research/2024/12/developing-ai-risk-management-with-the-same-ambition-and-urgency-as-ai-products

  3. AI Policy Hub Alumni Celebration (CLTC blog, Mar 2026) -- Concrete evidence of talent pipeline outcomes: who went where, what they're doing. Shows the theory of change working in practice. https://cltc.berkeley.edu/2026/03/02/uc-berkeleys-ai-policy-hub-celebrates-a-new-generation-of-leaders/

  4. "Can We Manage the Risks of General-Purpose AI Systems?" (TechPolicy.Press, Dec 2023) -- Barrett and Newman explain the GPAIS Profile, including the 9 minimum best practices and the case for mandatory standards. The intellectual backbone of CLTC's standards work. https://www.techpolicy.press/can-we-manage-the-risks-of-generalpurpose-ai-systems/

Connected to (11)

Microsoftstaff to · Jessica Newman
Berkeley Risk and Security Labcollaborator
CyberPeace Institutecollaborator
Berkeley Existential Risk Initiativecollaborator · Anthony Barrett
Centre for the Governance of AIcollaborator
National Institute of Standards and Technologycollaborator
OpenAIstaff to · Micah Carroll
OpenAIstaff to · Cedric Whitney
World Economic Forumcollaborator
Center for Human-Compatible AIadvisor at · Stuart Russell
CITRIS Policy Labcollaborator
Sources (58)
Every URL that was read during research.
  1. 1.Home - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  2. 2.AI Security Initiative - CLTCcltc.berkeley.edu
  3. 3.AI Policy Hub - CLTCcltc.berkeley.edu
  4. 4.Taking the Long View: Q&A with Steve Weber and Ann Cleaveland, of UC Berkeley’s Center for Long-Term Cybersecuritycdss.berkeley.edu
  5. 5.CLTC Welcomes Ann Cleaveland as Executive Director - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  6. 6.Hewlett Foundation Announces $45 Million in Grants to MIT, Stanford, UC Berkeley to Establish Major New Academic Centers for Cybersecurity Policy Researchhewlett.org
  7. 7.Cyber Initiative: 2014 - 2023hewlett.org
  8. 8.- CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  9. 9.Intolerable Risk Threshold Recommendations for Artificial Intelligence - CLTCcltc.berkeley.edu
  10. 10.Introducing the Agentic AI Risk Management Profile: Expert Perspectives on Governance and Best Practices - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  11. 11.New CLTC Report Provides Framework for Managing Risks of Agentic AIvcresearch.berkeley.edu
  12. 12.People - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  13. 13.AI Risk-Management Standards Profile for General-Purpose AI Systems (GPAIS) and Foundation Models - CLTCcltc.berkeley.edu
  14. 14.Ten years of securing our future: the Center for Long-Term Cybersecurityinspire.berkeley.edu
  15. 15.Marking the end of the Cyber Initiativehewlett.org
  16. 16.CLTC converted to main collaboration — Berkeley Existential Risk Initiativeexistence.org
  17. 17.Nada Madkour - CLTCcltc.berkeley.edu
  18. 18.Anthony Barrett - CLTClive-cltc.pantheon.berkeley.edu
  19. 19.Five Takeaways from the NIST AI Risk Management Frameworktechpolicy.press
  20. 20.How to Say No to the Next AI Releasetechpolicy.press
  21. 21.Can We Manage the Risks of General-Purpose AI Systems?techpolicy.press
  22. 22.CLTC AI Security Initiative Publishes Working Paper on Intolerable Risk Thresholds for AI - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  23. 23.Researchers Submit Response to U.S. Government Request on Security Considerations for AI Agents - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  24. 24.UC Berkeley’s AI Policy Hub Celebrates a New Generation of Leaders - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  25. 25.UC Berkeley to start graduate student research hub on AI policycdss.berkeley.edu
  26. 26.Building the cyber policy talent pipelinehewlett.org
  27. 27.Center for Long-Term Cybersecurityvcresearch.berkeley.edu
  28. 28.New CLTC Report Provides Framework for Managing Risks of Agentic AI - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  29. 29.UC Berkeley Cybersecurity Clinic - CLTCcltc.berkeley.edu
  30. 30.The Cal Cybersecurity Research Fellowship: Eight Years of Impact - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  31. 31.UC Berkeley Launches AI Policy Hub - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  32. 32.University of California at Berkeley - Future of Life Institutefutureoflife.org
  33. 33.Jessica Newman - CLTCcltc.berkeley.edu
  34. 34.Jessica Newman, Author at TechPolicy.Presstechpolicy.press
  35. 35.Shaping the future of security. - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  36. 36.Jessica Newman, Author at TechCrunchtechcrunch.com
  37. 37.Q&A with Andrew Reddie, Research Director - CLTC UC Berkeley Center for Long-Term Cybersecuritylive-cltc.pantheon.berkeley.edu
  38. 38.Berkeley Risk and Security Labfounderspledge.com
  39. 39.UC Berkeley Launches New Initiative to Combat AI-Driven Cybercrimes - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  40. 40.$2.2M Grant to UC Berkeley Will Help Drive Google.org’s $20M+ Investment in Consortium of Cybersecurity Clinics - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  41. 41.AI-Enabled Cybercrime - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  42. 42.Insights on AI-Enabled Cybercrime through Collaboration with UC Berkeley’s Center for Long-Term Cybersecurity | Fortinet Blogfortinet.com
  43. 43.Developing AI Risk Management With the Same Ambition and Urgency as AI Productscarnegieendowment.org
  44. 44.Frontiers | AI and cybersecurity: a risk society perspectivefrontiersin.org
  45. 45.Steven Weber to Retire from UC Berkeley - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  46. 46.Cybersecurity Futures 2030 - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  47. 47.External Advisory Committee - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  48. 48.Contesting AI Safetytechpolicy.press
  49. 49.A Taxonomy of Trustworthiness for Artificial Intelligence - CLTCcltc.berkeley.edu
  50. 50.Artificial Intelligence: “Talk about an AI divide between the US and the EU is exaggerated” | Heinrich Böll Stiftung | Washington, DC Office - USA, Canada, Global Dialogueus.boell.org
  51. 51.Berkeley AI Policy Symposium Showcases Next-Gen Research on Effective AI Governance - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  52. 52.- CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  53. 53.Ann Cleaveland | CyberPeace Institutecyberpeaceinstitute.org
  54. 54.AI Security Initiative Seeking Fall 2025 Graduate Student Researcher - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  55. 55.Beyond Phishing: Exploring the Rise of AI-enabled Cybercrime - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  56. 56.AI Risk-Management Standards Profile for General-Purpose AI (GPAI) and Foundation Models - CLTCcltc.berkeley.edu
  57. 57.From Concept to Community: The First Year of the Cyber Resilience Corps - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu
  58. 58.Chancellor's Award and Gathering Honor Steven Weber and CLTC - CLTC UC Berkeley Center for Long-Term Cybersecuritycltc.berkeley.edu