In this episode, Aishwarya Reganti and Kiriti Badam introduce a powerful new framework for building AI products: the Continuous Calibration/Continuous Development (CC/CD) framework. If you’ve ever shipped an AI demo that looked magical but struggled to scale, this framework will resonate. The CC/CD framework provides a practical and structured approach to navigating these realities and building AI systems that are stable, intentional, and trustworthy. Listen: YouTube: https://www.youtube.com/@lennysreads Apple: https://podcasts.apple.com/us/podcast/lennys-reads/id1810314693 Spotify: https://open.spotify.com/show/0IIunA06qMtrcQLfypTooj Newsletter: https://www.lennysnewsletter.com/subscribe In this episode, you’ll learn: • The two core differences every AI builder must account for: non-determinism and the agency–control tradeoff • The six phases of the CC/CD loop • How to scope capabilities across different versions of your product to gradually earn trust and agency • The role of reference datasets in taming unpredictability and guiding evals • How to design application-specific evals that act as the equivalent of tests for AI • Why control handoffs are essential for maintaining user trust • How to transform deployment from a finish line into the start of continuous calibration • Why you should never “jump to full agency” before the system earns it • How to apply the CC/CD loop with real-world examples: customer support, marketing assistants, and coding copilots References: • Beyond vibe checks: A PM’s complete guide to evals: https://www.lennysnewsletter.com/p/beyond-vibe-checks-a-pms-complete • “Building agentic AI applications with a problem-first approach” [Maven course]: https://maven.com/aishwarya-kiriti/genai-system-design • Cursor: https://cursor.com • “Don’t build AI products like traditional software” [Free lightning talk]: https://maven.com/p/88a325/don-t-build-ai-products-like-traditional-software • GitHub Copilot: https://github.com/features/copilot • Why premature pptimization is the root of all evil: https://stackify.com/premature-optimization-evil/ • Why your AI product needs a different development lifecycle: https://www.lennysnewsletter.com/p/48e7fb02-1fc1-4bb7-bf85-85f4165e8225 To hear more, visit www.lennysnewsletter.com
