Aman Khan: Arize, Evaluating AI, Designing for Non-Determinism | Learning from Machine Learning #11

On this episode of Learning from Machine Learning, I had the privilege of speaking with Aman Khan, Head of Product at Arize AI. Aman shared how evaluating AI systems isn't just a step in the process—it's a machine learning challenge in of itself. Drawing powerful analogies between mechanical engineering and AI, he explained, "Instead of tolerances in manufacturing, you're designing for non-determinism," reminding us that complexity often breeds opportunity.

Aman's journey from self-driving cars to ML evaluation tools highlights the critical importance of robust systems that can handle failure. He encourages teams to clearly define outcomes, break down complex systems, and build evaluations into every step of the development pipeline.

Most importantly, Aman's insights remind us that machine learning—much like life—is less deterministic and more probabilistic, encouraging us to question how we deal with the uncertainty in our own lives.

Thank you for listening. Be sure to subscribe and share with a friend or colleague . Until next time... keep on learning.

Available on Youtube: https://youtu.be/v0eTTn7ZPEc

Available on Substack: https://mindfulmachines.substack.com/p/aman-khan-arize-evaluating-ai-designing?r=eykwy