977: Attention, World Models and the Future of AI, with Prof. Kyunghyun Cho
What’s going to be the next big step function that blasts us forward in AI capabilities? To find out, Jon Krohn sits down with Professor Kyunghyun Cho, whose 200,000 citations and co-authorship of the first paper on attention place him among the most influential AI researchers in the world. In this episode, Kyunghyun explains why today’s models have already captured most correlations in passive data, making the real challenge about actively choosing which data to collect. He also weighs in on the open debate around world models, whether AI needs high-fidelity, step-by-step imagination or whether a high-level latent representation that lets it skip ahead is sufficient and shares the surprising discovery that 80% of his 200 computer science students had never installed a coding agent. Additional materials: www.superdatascience.com/977 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn: (06:43) The story behind the attention mechanism (28:43) Sample efficiency and active data collection (39:04) World models and latent planning (49:52) Teaching undergrads with coding agents (58:21) Reranking, multi-stage ranking, and the foundations of RAG
You May Also Like

Theo - t3․gg
Theo - t3․gg

Rabbit Hole Recap
ODELL and Marty Bent

Daily Tech News Show
Tom Merritt

Artificial Intelligence Masterclass
AI Masterclass

The Milk Road Show
Milk Road

Bitcoin Magazine Podcast
BTC Inc

Talking Tech w/ Wayne
Wayne Hunter

Ungovernable Misfits
UNGOVERNABLE

The Everything Feed - All Packet Pushers Pods
Packet Pushers Team

The Fat Pipe - Most Popular Packet Pushers Pods
Packet Pushers
Community discussion