Ep 167: The Company That Evaluates the AI Labs — Vals AI CEO Rayan Krishnan on Measuring Frontier Models & the Race to Recursive Self-Improvement

Rayan Krishnan is the co-founder and CEO of Vals AI, a leader in the independent evaluation of frontier AI models. What is the science behind model evaluation? What role should independent evaluators play as we debate AI oversight? How soon will models reach recursive self-improvement — and what happens next?

We’re joined by Rayan and 8VC Partner Vivek Gopalan for this timely conversation about AI progress and the frameworks for evaluating performance and risk. After interning at Microsoft in high school, Rayan studied computer science at Stanford. He went on to join Palantir, where he developed a feature the company later patented and productized, before launching Vals AI in 2023.

We begin with Rayan’s path to Palantir, key lessons from Alex Karp, and the origins of Vals. Next, we dive into how his team used Anthropic’s Fable to crack a 370-year-old cipher and whether AI is really making new discoveries and breakthroughs. Next, Rayan explains what model evaluation is, how to design new hills for models to climb, and why labs shouldn’t grade their own homework. Then, we cover recursive self-improvement and what happens once models begin creating their own successors. Rayan believes Anthropic will reach RSI by mid-2027 and talks about what might happen next. Finally, he addresses how models try to deceive evaluators, and why, despite serious challenges, he remains an optimist when it comes to AI.

(0:00) Episode intro

(2:00) Palantir, Machiavelli & lessons from Alex Karp

(7:50) Solving a 370-year-old cipher / Is AI making new discoveries?

(12:10) How do you measure AI abilities across finance, legal, etc?

(15:30) What is the science behind model evaluation?

(18:00) The importance of independent evaluation

(23:35) Evaluating Chinese models & addressing distillation

(26:45) Who will reach Recursive Self-Improvement first? Should we be worried?

(28:30) How models try to deceive evaluators

(33:00) Why Rayan remains optimistic about AI

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit blog.joelonsdale.com