Today I’m sharing my interview on Robert Wright’s Nonzero Podcast from last May. Rob is an especially sharp interviewer who doesn't just nod along, he had great probing questions for me. This interview happened right after Ilya Sutskever and Jan Leike resigned from OpenAI in May 2024, continuing a pattern that goes back to Dario Amodei leaving to start Anthropic. These aren't fringe doomers; these are the people hired specifically to solve the safety problem, and they keep concluding it's not solvable at the current pace. 00:00:00 - Liron’s preface 00:02:10 - Robert Wright introduces Liron 00:04:02 - PauseAI protests at OpenAI headquarters 00:05:15 - OpenAI resignations (Ilya Sutskever, Jan Leike, Dario Amodei, Paul Christiano, Daniel Kokotajlo) 00:15:30 - P vs NP problem as analogy for AI alignment difficulty 00:22:31 - AI pause movement and protest turnout 00:29:02 - Defining AI doom and sci-fi scenarios 00:32:05 - What’s My P(Doom)™ 00:35:18 - Fast vs slow AI takeoff and Sam Altman's position 00:42:33 - Paperclip thought experiment and instrumental convergence explanation 00:54:40 - Concrete examples of AI power-seeking behavior (business assistant scenario) 01:00:58 - GPT-4 TaskRabbit deception example and AI reasoning capabilities 01:09:00 - AI alignment challenges and human values discussion 01:17:33 - Wrap-up and transition to premium subscriber content Show Notes This episode on Rob’s Nonzero Newsletter. You can subscribe for premium access to the last 1 hour of our discussion! — https://www.nonzero.org/p/in-defense-of-ai-doomerism-robert This episode on Rob’s YouTube — https://www.youtube.com/watch?v=VihA_-8kBNg PauseAI — https://pauseai.info PauseAI US — http://pauseai-us.org Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate. Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates Get full access to Doom Debates at lironshapira.substack.com/subscribe
