Agent Engineering with Pydantic + Graphs — with Samuel Colvin

Did you know that adding a simple Code Interpreter took o3 from 9.2% to 32% on FrontierMath? The Latent Space crew is hosting a hack night Feb 11th in San Francisco focused on CodeGen use cases, co-hosted with E2B and Edge AGI; watch E2B’s new workshop and RSVP here!

We’re happy to announce that today’s guest Samuel Colvin will be teaching his very first Pydantic AI workshop at the newly announced AI Engineer NYC Workshops day on Feb 22! 25 tickets left.

If you’re a Python developer, it’s very likely that you’ve heard of Pydantic. Every month, it’s downloaded >300,000,000 times, making it one of the top 25 PyPi packages. OpenAI uses it in its SDK for structured outputs, it’s at the core of FastAPI, and if you’ve followed our AI Engineer Summit conference, Jason Liu of Instructor has given two great talks about it: “Pydantic is all you need” and “Pydantic is STILL all you need”.

Now, Samuel Colvin has raised $17M from Sequoia to turn Pydantic from an open source project to a full stack AI engineer platform with Logfire, their observability platform, and PydanticAI, their new agent framework.

Logfire: bringing OTEL to AI

OpenTelemetry recently merged Semantic Conventions for LLM workloads which provides standard definitions to track performance like gen_ai.server.time_per_output_token. In Sam’s view at least 80% of new apps being built today have some sort of LLM usage in them, and just like web observability platform got replaced by cloud-first ones in the 2010s, Logfire wants to do the same for AI-first apps.

If you’re interested in the technical details, Logfire migrated away from Clickhouse to Datafusion for their backend. We spent some time on the importance of picking open source tools you understand and that you can actually contribute to upstream, rather than the more popular ones; listen in ~43:19 for that part.

Agents are the killer app for graphs

Pydantic AI is their attempt at taking a lot of the learnings that LangChain and the other early LLM frameworks had, and putting Python best practices into it. At an API level, it’s very similar to the other libraries: you can call LLMs, create agents, do function calling, do evals, etc.

They define an “Agent” as a container with a system prompt, tools, structured result, and an LLM. Under the hood, each Agent is now a graph of function calls that can orchestrate multi-step LLM interactions. You can start simple, then move toward fully dynamic graph-based control flow if needed.

“We were compelled enough by graphs once we got them right that our agent implementation [...] is now actually a graph under the hood.”

Why Graphs?

* More natural for complex or multi-step AI workflows.

* Easy to visualize and debug with mermaid diagrams.

* Potential for distributed runs, or “waiting days” between steps in certain flows.

In parallel, you see folks like Emil Eifrem of Neo4j talk about GraphRAG as another place where graphs fit really well in the AI stack, so it might be time for more people to take them seriously.

Full Video Episode

Like and subscribe!

Chapters

* 00:00:00 Introductions

* 00:00:24 Origins of Pydantic

* 00:05:28 Pydantic's AI moment

* 00:08:05 Why build a new agents framework?

* 00:10:17 Overview of Pydantic AI

* 00:12:33 Becoming a believer in graphs

* 00:24:02 God Model vs Compound AI Systems

* 00:28:13 Why not build an LLM gateway?

* 00:31:39 Programmatic testing vs live evals

* 00:35:51 Using OpenTelemetry for AI traces

* 00:43:19 Why they don't use Clickhouse

* 00:48:34 Competing in the observability space

* 00:50:41 Licensing decisions for Pydantic and LogFire

* 00:51:48 Building Pydantic.run

* 00:55:24 Marimo and the future of Jupyter notebooks

* 00:57:44 London's AI scene

Show Notes

* Sam Colvin

* Pydantic

* Pydantic AI

* Logfire

* Pydantic.run

* Zod

* E2B

* Arize

* Langsmith

* Marimo

* Prefect

* GLA (Google Generative Language API)

* OpenTelemetry

* Jason Liu

* Sebastian Ramirez

* Bogomil Balkansky

* Hood Chatham

* Jeremy Howard

* Andrew Lamb

Transcript

Alessio [00:00:03]: Hey, everyone. Welcome to the Latent Space podcast. This is Alessio, partner and CTO at Decibel Partners, and I'm joined by my co-host Swyx, founder of Smol AI.

Swyx [00:00:12]: Good morning. And today we're very excited to have Sam Colvin join us from Pydantic AI. Welcome. Sam, I heard that Pydantic is all we need. Is that true?

Samuel [00:00:24]: I would say you might need Pydantic AI and Logfire as well, but it gets you a long way, that's for sure.

Swyx [00:00:29]: Pydantic almost basically needs no introduction. It's almost 300 million downloads in December. And obviously, in the previous podcasts and discussions we've had with Jason Liu, he's been a big fan and promoter of Pydantic and AI.

Samuel [00:00:45]: Yeah, it's weird becau