Underwriting Superintelligence: Backing Agents you can Sue — Rune Kvist, AIUC

AIUC first got our attention with the NFDG backing, and have just announced a $40M series A today, with the most impressive industry advisor list we may have ever seen for an early startup behind AIUC-1, their agent standard backed by real insurance:

From being Anthropic’s first product hire to building the standards, testing, and insurance infrastructure meant to make frontier AI deployable, Rune Kvist is betting that the biggest constraint on AI adoption won’t be capability it will be trust. In this episode, the AIUC cofounder joins swyx and Vibhu to announce a new $40M round and explain why companies like Cursor, Harvey, Lovable, and ElevenLabs are increasingly confronting a problem that gets harder as AI gets better: who is responsible when autonomous systems fail?

We go deep on AIUC-1, the emerging standard for agent security, safety, and reliability; how AI agents are stress-tested for jailbreaks, hallucinations, and data leaks; and why Rune thinks standards and insurance could become critical infrastructure for AI. We also discuss the growing trust gap between governments and frontier labs, AI-enabled cyber and biological risks, why every model can ultimately be jailbroken, what happens when a $20 coding agent causes $200M of damage, whether AI engineers should be certified, and why even after AGI there may be one job the labs can never do themselves: be their own watchdog.

We discuss:

* Why risk, liability, and trust may become the binding constraint on AI adoption

* Rune’s path from reading the Scaling Laws paper to joining Anthropic in its earliest days

* What Anthropic understood about scaling, compute, and the future years before it became obvious

* Why Waymo illustrates the gap between AI capability and real-world deployment

* AIUC’s $40M round and work with Cursor, Harvey, Lovable, ElevenLabs, and other frontier AI companies

* AIUC-1: a standard for AI agent security, safety, and reliability

* How agents are tested for jailbreaks, hallucinations, and data leakage

* Why most AI companies optimize the happy path without seriously stress-testing adversarial cases

* Why AI standards may need to update every quarter instead of every decade

* The emerging trust gap between frontier AI labs and governments

* Cybersecurity, child safety, biological weapons, and the expanding frontier-model risk surface

* Why standards and insurance may need to evolve together

* How Lloyd’s of London can insure AI systems and bring trust to enterprise deployment

* What happens if a $20 Cursor subscription contributes to a $200M plane crash

* The Air Canada chatbot case and how AI failures are beginning to clarify legal liability

* Why copyright may be one of the hardest AI risks to insure

* Evals, mechanistic interpretability, monitoring, and models becoming aware they’re being tested

* The impossible CISO mandate: adopt AI fast, but don’t let anything go wrong

* Why robotics will make AI liability dramatically more consequential

* Whether AI engineers should have Level 1, 2, and 3 certifications

* AIUC’s roadmap across agents, frontier models, robotics, and universal red teaming

* Why AGI could become a question of national sovereignty

* Why the labs can never fully serve as their own watchdogs

* The Big Short problem: how do you stop competing watchdogs from racing standards to the bottom?

Rune Kvist

* LinkedIn: https://www.linkedin.com/in/runekvist/

* X: https://x.com/RuneKvist

AIUC

* https://aiuc.com

Timestamps

00:00:00 AIUC’s $40M Round and the Risk Bottleneck for AI

00:01:07 From Scaling Laws to Early Anthropic

00:07:58 Why Trust, Not Capability, Could Limit AI Adoption

00:12:19 Founding AIUC and Building AIUC-1

00:18:52 How AI Agents Are Audited and Stress-Tested

00:25:26 Frontier Models, Government, and the AI Trust Gap

00:33:32 Cyber, Child Safety, and AI-Enabled Biological Risk

00:38:14 Why Standards and Insurance Belong Together

00:41:45 What Does an AI Insurance Policy Actually Cover?

00:50:44 The $20 Cursor Subscription and the $200M Plane Crash

00:53:53 AI Liability, Monitoring, and Earning Enterprise Trust

00:56:21 From AI Agents to Models to Robotics

00:58:29 Copyright, Adverse Selection, and AI Insurance

01:03:28 Evals, Mechanistic Interpretability, and Eval Awareness

01:08:36 The Impossible Enterprise AI Mandate

01:11:52 Prediction Markets vs. AI Audits

01:14:43 Should AI Engineers Be Certified?

01:19:10 AIUC’s Roadmap, AGI, and Who Watches the Watchdogs?

Transcript

Introduction: AIUC, the $40M Series A, and Risk as the Adoption Bottleneck

Swyx [00:00:00]: Okay, we’re in the studio with Rune from AIUC, the Artificial Intelligence Underwriting Company, with our trusty co-host, Vibhu. Welcome.

Rune Kvist [00:00:10]: Thank you. Thanks for having me. Thank you.

Swyx [00:00:11]: What are you announcing today?

Rune Kvist [00:00:12]: We have raised $40 million, led by Ribbit Capital and First Harmonic.

Swyx [00:00:17]: You first came to my attention when Nat and Daniel invested in you guys. Is the story, like, pretty much the same? Like, what are you today versus what you thought you were back then?

Rune Kvist [00:00:26]: When we raised our seed round, we had a hypothesis that at some point risk was going to hold down adoption. At that point in time, that felt kind of hypothetical, and I think that is now over. Clearly, the moment is now with Mythos and Fable. It’s pretty obvious that literally the binding constraint on adoption is risk. And so for us, it feels like this is a natural continuation of the same hypothesis, but where previously it was speculation, now it feels like fact.

Swyx [00:00:54]: And let’s get a list of the customers that you’re highlighting as part of your Series A.

Rune Kvist [00:00:58]: Totally. Yeah. So we are now working with folks like Cursor, Harvey, Lovable, ElevenLabs.

Swyx [00:01:05]: Yeah. Amazing. Congrats.

Rune Kvist [00:01:06]: Thank you.

Swyx [00:01:07]: So you were famously one of the first hires involved in GTM and product. I’m just kind of curious: what was your path into AI? Just recap.

Rune’s Path Into AI: Scaling Laws, Capital, and Anthropic

Rune Kvist [00:01:18]: Yeah.

Rune Kvist