Show full transcript 0:21 you have to build, you know, 0:22 for a certain level of scale. 0:23 How many agents do you find yourself running 0:25 in parallel, just in the kind of your day to day work? 0:28 As I'm pushing the frontier, 0:28 I find myself like building larger and larger teams of agents, and then when we have the next 0:34 breakthrough with models, like suddenly it's just a guy like, oh well, you know, a bigger 0:38 agent can just do all of it, and so I kind of shrink the team again, and then it just kind of 0:41 goes through this expansion and shrinking expansion. It kind of makes me think about 0:44 there's this whole loops thing, and then this little graphs thing. Having to set up and fiddle 0:48 with your loops is something that maybe people got excited about, but I don't think this 0:52 is the way that it's going to work. Over time, you just want this system that learns. You 0:56 don't want to necessarily think about how I was just thinking I'm gonna loop it 0:58 exactly this way in order to get results. Do you think there's still more radical 1:01 change happening to how we work? It's going to continue to change quite 1:05 radically. Even today it still feels like a bit clunky and I think it will feel 1:09 clunky until it isn't. People, thank you for doing this. Yeah? Like for me? Of course. 1:21 Nice room. You launched a ton of stuff today. How's your sleep? How are you 1:24 sleeping? Are you doing all right? I'm doing great. 1:27 I'm fortunate to have a very good team. 1:30 So while I was asleep, they were awake most of the night. 1:33 We have this amazing room at the office called the library, which we completely repurposed 1:38 as like a mega war room, and the vibes in there are just immaculate. 1:42 People were there until like very, very late. 1:44 So you've been a longtime engineer. 1:46 Are you coding it all anymore? 1:48 Are you shipping PRs? 1:49 Are you mostly like in docs and in meetings now? 1:51 What is coding? 1:53 Exactly. 1:54 Yeah. 1:55 Are you shipping PRs? 1:55 I don't know. 1:56 I was like, yeah, merging some code every once in a while. 1:59 Technically, a lot of code is getting written for me 2:02 to do all sorts of analysis, understand trends, 2:06 understand the business, understand the next feature, 2:09 how all our previous launches are doing. 2:10 A lot of code is written by Codex for that. 2:14 I'm obviously not writing it by hand anymore. 2:17 Occasionally, during the weekends, 2:19 I will have a little therapeutic moment 2:20 where I do a little lead code just by hand. 2:23 but that's the only actual hands-coating that I do. 2:26 How many agents do you find yourself running in parallel in your day-to-day work? 2:31 I used to run a lot more in parallel and we were fortunate enough to get a breakthrough at ultra-fast. 2:38 And so now I feel like I'm able to be in the flow again and so just having a faster agent 2:43 helps me a lot. I'm excited that we're getting 6-1 solo ultra-fast out there soon. 2:48 And then yeah, I think it sort of varies as I'm kind of like pushing the frontier 2:54 I find myself like you know building larger and larger teams of 2:58 Agents and then when we have the next breakthrough with models like suddenly is just a guy and like oh well 3:03 You know a bigger agent can just do all of it and that keep everything in memory and learn 3:07 And so I kind of shrink the team again 3:09 And then it just kind of goes through this expansion and shrinking expansion. Can you say more about that? 3:13 It kind of makes me think about there's this whole loops thing and then this whole graphs thing 3:16 Is this like an evolution of that in some way or? 3:19 I think like having to set up and fiddle with your loops and you know figuring that out is 3:24 something that you know maybe people got excited about but I don't think this is the way that 3:28 it's going to work. 3:29 I think the way that it's going to work is how we're positioning and what we're 3:34 shipping with dots where you have an incredibly smart agent that works 24-7, understands 3:40 your goals, understands your preferences, learns from feedback. 3:44 And, you know, it's like, well, we launched this, like, not perfect, I will learn very 3:48 much from, you know, making this available to our pro users, but over time you just want 3:53 to system that learns from, you know, what you want to achieve. 3:56 You don't want to necessarily think about, oh, I was just like, I'm going to loop it 3:59 exactly this way in order to get results. 4:00 So right now there's codecs, there's chat GPT work, there's chat GPT consumer, there's 4:04 dots. 4:05 What I'm hearing is you think we're heading towards dots kind of being the primary 4:09 way you talk to AI and that kicks off all these other things. 4:12 Yeah, I think if you take a step back and you know, whether it's thoughts or not, it's 4:18 all about breaking free from the technology and having this sort of like permanent active 4:26 intelligence that knows, you know, everything it needs to do and is available through any 4:33 client, any screen, you can call it, you know, like maybe you walk into a meeting 4:37 room, it shows up in the meeting, it takes notes, you pick it back up on email, and then 4:44 you text it if you need it. 4:45 It's not that you need to be glued to a laptop or you're always on your phone, it's available 4:50 when you need it and it also gets out of the way when you don't need it. 4:54 I can't wait for that to come to fruition because I think just carrying your laptop 5:00 as this brick around everywhere is just like you're kind of tied to the technology instead 5:04 of the technology working for you. 5:05 This episode is brought to you by our season's presenting sponsor, WorkOS. 5:10 What do OpenAI, Anthropic, Cursor, Replet, Sierra, Clay, and hundreds of other winning companies all have in common? 5:18 They are all powered by WorkOS. 5:20 If you're building a product for the enterprise, you've felt the pain of integrating single sign-on, 5:25 skim, RBAC, audit logs, and other features required by large companies. 5:30 WorkOS turns those deal blockers into drop-in APIs with a modern developer platform 5:34 built specifically for B2B SaaS. Literally every startup that I'm an investor in that starts to 5:40 expand up market ends up working with WorkOS. And that's because they are the best, whether you 5:46 are a seed stage startup trying to land your first enterprise customer or a unicorn expanding 5:50 globally. WorkOS is the fastest path to becoming enterprise ready and unblocking growth. It's 5:56 essentially Stripe for enterprise features. Visit workos.com to get started or just hit 6:01 up their Slack where they have actual engineers waiting to answer your questions. 6:05 WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer 6:10 experience. 6:11 Go to workOS.com to make your app enterprise ready today. 6:16 I was going to ask you about this. 6:18 It feels like so much has changed in how we worked over the past like two years. 6:22 In two years, especially engineers, but a lot of roles have just the role your 6:25 day-to-day life has changed significantly. 6:28 And I'm curious if you think we're kind of settling into what work will look like in the next I don't know five years 6:34 Let's say two years. That's feels way too far to think about or do you think there's still more radical change happening to how we work? 6:40 And I think it's going to continue to change quite radically even today 6:45 No, it does seem like the technology is finally coming together with voice 6:49 You know multimodal inputs outputs 6:52 but it still feels like a bit clunky, 6:55 and I think it will feel clunky until it isn't. 6:58 And then you're just like, wow, you know, 6:59 like I can just talk to this thing 7:01 the same way that we're talking right now. 7:03 It just remembers things perfectly. 7:05 If I want to, you know, ideate and brainstorm, 7:09 it can just like, you know, squiggle a thing. 7:11 I have this collaborative surface, 7:13 you know, this is what we launched with Changer Beauty 7:14 Space, it's like, you know, the very start of that. 7:16 You know, you can imagine, 7:17 we have like a shared wide board. 7:19 We can collaborate between humans, between agents, 7:21 and I think it's going to transcend conversations, 7:24 transcend a lot of the clients that you have today. 7:27 None of this is just really working today, right? 7:29 So I think that we're gonna go through 7:30 this big new transformation. 7:32 But I'm hearing yours, Dots is a big part of that future, 7:34 this AI system that just kind of does everything for you 7:37 and you don't have to think about Codex versus ChatGPT. 7:39 Yes, and for Codex and ChatGPT, 7:41 we're going to merge the, we have chat, 7:44 we have the work toggle, we heard the feedback loud 7:47 and clear, people love the capability in work, 7:49 But then also you know you like sometimes you want to use chat because a little bit faster and you know more pleasant to use 7:54 And so we're we're merging that we're reducing that complexity 7:57 Eventually we will ship all of the capabilities of dots, you know straight into chat to be as well and you know lift the floor 8:03 For like our 1.2 billion users, which you know at that time we'll see that company we have 8:08 And we want it to be really really seamless one thing that I'm very excited with dots is like they don't have a model picker 8:13 There's no configuration. You just talk to it 8:16 The only thing that you have to configure is like which channels you want to talk to. 8:20 It feels very much like the horror kind of vision of the world. 8:23 I don't know. Do you think about that movie much? Just this idea of... 8:26 I watched it once. 8:28 Yeah. 8:29 I think it's interesting to think about maybe more, what I think about more is like, 8:35 you know, sci-fi from 34 years ago and like how visionary it was. 8:40 Is there a specific sci-fi you think about that's most influenced you? 8:42 Like, you know, books like Neuromancer and then, you know, the original, like, Star Trek, 8:47 um, you know, I think about that a lot. And there's something about Star Trek, which I think, 8:52 you know, was like, it was this technology that was, like, very, very useful. Obviously, like, you 8:55 know, you're able to just, like, talk to a computer and, like, in dustings where you're, 8:59 like, you can talk to your spaceship and it just moves, right? It's like, 9:02 this is getting real, you know, like, we're entering that era. 9:05 So we're recording this in front of a live audience. There was this line out the door, 9:09 just people waiting to see you, to hear from you. How do you just think about this, I don't know, 9:14 role you're in, this responsibility you have to build what you're building and the impact 9:18 that has on people's lives? I don't know, just what does that feel like? How do you think about 9:21 that? I think it's very important to be part of a community and to build with people. I was 9:27 like, in a way, we're discovering this technology and like what we can do together. And so every 9:32 time I, you know, talk to people, I'm like, I'm kind of like reminded of you're doing this 9:37 thing, which I hadn't even anticipated, you know, it's impacting your life in a certain 9:40 way and it's impacted, like, you know, this other person's life and then just, like, all 9:44 together. 9:44 It's but very humbling, very inspiring, and I think very necessary in order to build 9:48 something that is truly useful for human sight. 9:50 I don't know how we would do it without the community. 9:53 That's something I hear a lot with people building an AI is they don't know what 9:56 it is until they've launched it and they see how people use it, see what emerges 10:00 and it feels like kind of what you're saying is just to build it with people versus, 10:03 like, here's the vision we've got it figured out. 10:05 Even the Twitter community has been very nice in general. 10:09 You're very good at Twitter, kind of antagonistic sometimes, very good. 10:13 How do you find time to even tweet and understand how someone in your role has time to sit on Twitter, tweet, reply? 10:19 I spend like maybe half an hour per day just scrolling and then doing something when I feel inspired and I have something interesting to say. 10:26 Often it just comes like in the moment. 10:28 Yeah, perfect Twitter skill. 10:29 And I have a, we haven't released the team dots yet, 10:34 but you know, today we're releasing the primary dot 10:36 which you can configure or you can get used to 10:38 like what a dot can do, connect it to your apps, 10:41 but we'll release the ability to create more than one. 10:44 And so I have one that's just responsible for Twitter. 10:46 Okay, say more about that. 10:48 So right now everyone gets one dot. 10:49 Yeah. 10:50 Right now they can't add more dots. 10:52 That's right. 10:52 They will be able to have many dots. 10:54 Talk about kind of where that's going. 10:55 Yeah, so we wanted to start with one, 10:57 the primary dot, which is the one that you'll probably also text that learns your preferences 11:02 most deeply as well, and kind of see what people do with it, see how we need to tune the system, 11:08 learn about working with the community, and then very quickly, soon you'll be able to add 11:13 a second one, a third one, a fourth one, as many as you need to create your virtual team, 11:18 and then you can give them specific roles. It's not, I don't find it absolutely necessary, 11:22 But sometimes I have like a thing that is requires quite a bit of work, you know 11:27 Such as like monitoring like Twitter for me and then you know, that's like a thing is okay 11:30 This will be like so much work that it's like the work of an entire dot 11:34 And then you know, you can have a couple of this 11:36 So if you think about all the launches that happen today feels like dots maybe like the sexiest 11:40 I'm curious if there's something that you think is not getting the attention 11:44 It probably deserves that will become a really really important down the road. That's kind of people are sleeping on 11:49 And that's like a vision on the important. 11:51 I think the sleeper hit is a ecosystem. 11:54 So opening it all up and the commitment, 11:56 the deep commitment we have towards opening it all up. 11:59 So we have signed in with Chantrapti 16 partners. 12:02 Very, very proud of that. 12:03 Like we started that, like really, 12:05 it was sort of like in the moment last year, 12:07 I was talking to creators of Pi and open code. 12:10 And it was like, of course, you know, 12:12 you should just be able to use the critics sign on 12:15 and then, you know, use the usage. 12:17 and then we kind of shook hands virtually. 12:20 We're like, I was just like, use this auth 12:22 and then kind of trust you to not do anything shady. 12:26 That grew, obviously, that became quite popular 12:27 and then now we made it an actual thing 12:29 that was supported with many, many partners. 12:32 And I'm excited to grow that very quickly. 12:34 The other side as well is opening up 12:36 all of the infrastructure and how we build for Chagabity 12:39 with plugin extensions, also plugin discovery, 12:42 just allowing everyone to ship something through 12:44 like maybe 1.2 billion users right you know and benefiting from that distribution and we also have 12:50 shared economics that we didn't actually talk in the keynote but you know we will pay you know our 12:57 plugins that are popular and you know are seeing a lot of usage they're going to get part of like 13:02 the revenue share as well and so I think this commitment towards an open ecosystem is going 13:06 to be very exciting that's amazing okay so as an example so say notion somebody's using the 13:11 notion plug-in or the Figma plug-in within ChatGPT work, 13:16 notion Figma make money from people using it within the app. 13:19 Yes, so we have all these subscribers that use ChatGPT, 13:24 they get a certain amount of usage 13:26 and then when the users use that usage with the plug-in 13:30 or within the other product, 13:32 when they use sign-in with ChatGPT, 13:34 we will have some shared economics with them 13:36 where they will get remunerated. 13:38 One of the things I think that gets people excited 13:40 about plugins and ecosystems is distribution platforms, 13:44 ways to kind of get out there. 13:45 People discover my app, it blows up. 13:47 You have any advice for people that want their plugins 13:49 to bubble up, to get discovered? 13:51 Build a good plugin, yeah. 13:53 The way that it works, and we'll obviously tune the system, 13:56 but we look at retention numbers, 13:59 we look at how successful the plugin, the quality, 14:03 and then that's what we then start to recommend 14:05 to users in conversations. 14:07 And so your plugin can just get recommended 14:09 You're like, you know, a significant slice of users. 14:10 Obviously, if your plugin is not quite good, 14:13 I will stop being recommended. 14:15 That's amazing that you're looking at retention 14:17 of people using that plugin. 14:18 Is it actually adding utility? 14:20 Is it enabling to have to do something else? 14:23 Got it. So it's not so much an AEO 14:26 kind of get the right words 14:27 and get people writing about you. 14:28 It's people using it consistently sticking with it. 14:30 Okay, so you think that's kind of the sleeper hit 14:32 that's going to become a big deal. 14:34 And I know this is kind of the second attempt. 14:35 You guys had an app marketplace that this one is the right one 14:38 Okay, this is this one's gonna work 14:39 Maybe going back to dots because it feels like that's a big part of the vision in the future 14:42 When did you guys start working on this? Obviously, there's an open claw a big open claw moment 14:47 You guys Peter joined and there's a foundation and then grokbot muse and instinct and all these things 14:52 I guess how long have you guys been working on it? 14:55 What what it takes so long for you guys to get something out. Do you remember the codex club? 15:00 Yeah, the first yeah, that was like a year ago, right and if you look back at codex 15:05 when you look at the little animation that we used, 15:07 I think that was the inspiration for GrubBot. 15:11 This is very, very similar, eerily similar, I would say. 15:14 But seriously, on the research, long persistent, 15:18 long horizon tasks, this is something that we've worked on 15:20 for more than two years. 15:22 Memory systems that's take coherent 15:24 roughly the same amount of time. 15:26 We launched a lot of that straight into Chatchapiti. 15:28 I think people just really enjoy 15:30 that Chatchapiti gets to know 15:31 so much relevant information about you. 15:34 So many people I talked to have like an anecdote of like, hey, you know, I fell ill and you know 15:38 Chateaupty remembered actually like, you know, I had this barbecue and you know 15:41 I had like lime and I was like making tequila and like, you know 15:45 Just like probably like this burn that I have on my hand is actually like this lime 15:48 You know, it's like a sunburn and you're just like, how did it know that and it's like, you know 15:52 it's just memory works very very well in Chateaupty 15:54 And so we've put a lot of research in that and you know, all of this coming together on top of like 15:59 the codex harness and also the ability to just like run 24 seven with, you know, 16:03 good productivity. That's what we launched with dots. 16:05 And so it's the work of like many, many months. 16:08 And then one of the important things was also getting safety and security. 16:11 Right. You know, this is also why we launched with Astra. 16:13 It's our safest must align model and like a lot more effort went into like 16:17 making dots, like, you know, just really safe and secure. 16:19 A question I like to ask people who come on the podcast is where do you 16:23 think human brains will continue to be valuable over time? 16:26 Where do you find yourself being necessary in the work that happens versus where AI is taking on more and more? 16:33 Where do you think human brains will continue to be useful, most useful, over time? 16:37 Yes, I think this is very much a function of how we build a technology and the way that we design everything 16:42 at Open AI is like to put humans at the center of it and build it as extensions of humans, 16:48 of your well and your taste. 16:50 And so just really be like this super empowering thing. 16:53 I think as long as we continue to do that and it kind of like 16:58 Enables you to just take whatever you wanted to do and you know your creativity and your taste and sort of like you know 17:03 Expand that you know and in a way that feels like awesome and the moment is just like really this artistic tool, right? 17:09 And we may we may not have cutters anymore 17:11 But we have more builders than ever and I think there's something that's going to remain 17:14 You know deeply human about that like humans want to learn and see you know what other humans are building and you know 17:20 I'm like much more interested. I'm here 17:21 I'm not talking to you, but I think that will remain true for a very, very long time. 17:26 One of the things that it's emerged with engineers especially in how their lives have changed 17:32 is there's a lot more context-witching. 17:35 There's this trend of loneliness that has emerged where they're talking to agents all day instead 17:39 of other humans. 17:40 I'm curious how much you think about just like that part of the impact AI has on 17:45 people's lives and just how you might... 17:46 Is there solutions to that? 17:48 Is there stuff you think about to make that less annoying? 17:49 Yes, all the time. Reducing, you know, configuration fatigue is, you know, one, reducing the fact 17:56 that talking to an agent is like sort of like a solo adventure and you just have this, you 18:00 know, conversation just with your one agent and then, you know, you have like many and 18:04 then you're delegating so many things. A lot of things are going to come together to 18:08 make it much more delightful and something that I spend a lot of time, the team spent 18:12 a lot of time thinking about. So I think, you know, for me, the ideal way to get 18:17 things done is you know would be like to just have it in your physical space you 18:21 know we're just having conversation like we have now it's able to sort of like 18:24 you know observe you know like here are the ideas that we have we can you know 18:28 talk something down on a white paper on a paper you know it can maybe like you 18:32 know take that and you know start building in the background and then 18:34 you're like you know actually have another idea and it's just like it 18:36 starts building another thing you just project it on a screen and then you 18:39 just talk to it and it's just involved in a conversation you know with 18:41 other humans and you don't have this fatigue of like having to be like 18:44 you know on this like screen and you're just like you know thinking 18:46 about prompting. It's just like becomes like supernatural. That's what we're trending towards. 18:50 You know, I think we didn't fully get there today, but you know, we'll get there in the future. 18:55 One of the other kind of downsides that's emerged along those lines is just 18:58 this kind of pressure to do more because we can do more. Everyone's just like, come on, 19:02 run 30 agents at once. Why aren't you shipping more? Everyone's shipping more. 19:06 Do you think that's solvable? Do you think that's just kind of, I don't know, 19:08 human nature? We can do more. Let's see more. I'm there. 19:11 I think we heard Sam as well, you know about the promise of this as well, which is 19:17 reduce the noise and 19:19 you know, allow you to spend attention where you want to spend attention and you know having like all these things that 19:24 You know, like maybe you know, maybe important maybe not important 19:28 but that can have a vying for your attention is just sort of like 19:32 Tone that down a lit and you know, just get you to focus on the things 19:35 I find it remarkable like every time I go on a holiday and you know 19:39 I take that one week to just fully disconnect. 19:42 I start to think in different and more creative ways. 19:46 And I'm very eager to see, can we bring that? 19:48 Can we just make this your day to day? 19:50 Maybe you need fewer meetings. 19:52 Maybe you don't need to do as much, 19:54 and you would actually be more productive 19:56 if you're better rested. 19:58 And so we'll have to, I think as an industry, 20:01 figure that out. 20:03 But I think that's the promise of AI, 20:05 and it's not just one more prompt per second. 20:08 There's a bot that I'm building right now that's kind of this energy audit bot that watches my calendar and asks me just like 20:14 Was this giving energy taking away energy? Is this something someone else could have done? 20:18 I feel like you know, I should be like hey Lenny maybe maybe you can cut out these things 20:21 So you could be a happier person. Is there anything you've done really interestingly with AI recently? 20:25 That's just like a really cool use case of wow 20:27 That really was amazing that blew my mind or someone you've heard 20:31 Oh, it was like something cool because the live demo just failed right, you know in front of everyone 20:36 one. So it was like, okay, well, it's not so cool. But actually, my, my, my dot was, 20:42 you know, it's like realized I was, you know, a deaf day, like, you know, we have Chagabee 20:47 tea production when done. And so it pinged me five minutes before the live demo, which 20:51 was actually quite stressful because it was like, Hey, production is down. It's like, 20:56 Oh, well, you know, it's like, do you want me to try and fix it? And I'm like, 21:00 I don't think you're dairy yet, you know, a little dot, but, you know, thank 21:04 Thank you for trying and then I got in touch with the engineering teams and we started to 21:08 look at what was going on and it's fixed by now. 21:13 But having this thing that just understands like, okay, there's a pretty important thing 21:18 happening. 21:19 It's called Dev Day. 21:19 There's this production system. 21:21 There's a live demo. 21:22 It's probably using this production system so these two things are connected. 21:26 And this is happening in five minutes so I should ping him because he probably wants 21:28 to know about it. 21:29 I think that's quite remarkable. 21:31 That is remarkable. 21:32 Wow. 21:33 So it just knew that this was coming and told you and I love that it wanted to fix it 21:37 And you're like not quite you're not quite there 21:38 I don't think you're like, you know, please go impress me 21:41 But you know that would have been a better story. How do you feel about just giving you access to all these things? 21:44 I don't know like did does it have access to the production co-basin like, you know, it has 21:48 Access to some some of production systems with guardrails 21:53 You know, so we built we talked about specialist dots 21:56 But specialist dots is you're just like operate with additional guard guard rails additional monitoring 22:02 and then on their own hardware. 22:04 So we actually run some on Macminis. 22:06 The fundamental difference in the way that we've built dots 22:09 is like the harness does not run on the machine. 22:12 And then this is like maybe people have not realized 22:15 is like it can connect to as many devices as you want. 22:18 So it has its own computer. 22:20 You can actually connect it to your own laptop as well. 22:22 Over time, you might connect it to like 10 different devices 22:25 and it can control it all, a little bit like an octopus. 22:27 So you're basically talking to your dot, 22:29 it's living in some VM somewhere and it can. 22:30 Yeah, might be a VM, might not be a VM, it's just like it's a thing, you know, and it can connect to many devices. 22:36 This season's supporting sponsor is DX. 22:39 Now that everybody at your company is finally using AI, the question that engineering leaders face is no longer, 22:46 are we using AI, but is it paying off? 22:49 With DX, you can measure and benchmark the impact of AI adoption across your full product lifecycle. 22:55 DX shows you where AI is helping developers and where it's creating new bottlenecks. 22:59 You can also use it to evaluate AI vendors, monitor costs and licenses, and see exactly 23:05 what's limiting your agents. 23:07 That's why hundreds of enterprises, including Snowflake, Sony, and BNY, use the DX platform 23:14 to measure AI's impact on developer productivity. 23:17 Visit GetDX.com slash Lenny to get a demo of DX. 23:23 That's GetDX.com slash Lenny. 23:26 So they're really hiring a lot. 23:28 I know you're hiring a lot. 23:29 In your interviewing, what have you seen as skills 23:32 that are kind of trending up in what you look for 23:35 that is more and more important 23:37 in people being successful now? 23:40 And what skills do you find are trending down 23:41 that are just like, okay, we don't need that as much? 23:44 Yeah, I would say like, 23:45 skills that is trending now is typing fast. 23:47 That is not that useful anymore. 23:49 Skills that are trending up is, you know, 23:51 just great taste, thinking about the user 23:54 connecting to the audience that you're building for. 23:58 A lot of ex-founders are incredibly successful right now or like, you know, just like repeat founders 24:04 We're also finding this like when we're hiring at OpenAI is like, you know, we have a lot of founders 24:07 I think we have more than 120 XYC founders currently at OpenAI 24:12 So it's like this mega startup and I think just this 24:15 This passion and this envy to build something that matters and then know what good looks like is more important than ever 24:21 I've been arguing for a long time that PMs are kind of gonna thrive in this time because that's basically the job of a PM 24:27 where should we build, help someone build it, and then is this right, is this great and 24:31 iterate? 24:32 I don't know. 24:33 I agree with that. 24:34 All right. 24:35 Also, roles are blurring, so you might have been only into design or only an engineer 24:40 and felt always a little bit uncomfortable, but now it's just like it's your time to 24:45 shine. 24:45 Do you miss the engineering part of the job? 24:48 I was an engineer back in the day. 24:49 I was an engineer for 10 years. 24:51 Little did you know. 24:51 And you know, there's this flow state, there's this beauty of just building and just seeing your work and not working. 24:57 You kind of miss that at all or you're just like, okay, that's my old life. 25:00 I missed that for a while and then now with really fast speeds, I'm finding that back. 25:08 And so I'm really excited to democratize that as much as possible. 25:12 It will take a while for it to be absolutely ubiquitous and available to more than a billion users. 25:18 But now with ultra-fast speeds, we're able to achieve it for a 6-1 soul, it costs roughly the same as Astra. 25:26 And so the progress there is quite astonishing. 25:29 Even for us internally, we're like, wow, we can do these things also because we are using Astra 25:33 and we're finding we can push it super far. 25:36 And when you have those speeds, and especially when you can able to voice control it, 25:41 you kind of get back into this really creative state of mind and this flow. 25:45 It is different from the flow of the past. 25:47 Um, but I don't miss it. Yeah, it's like your last two launch videos 25:50 Just people kind of standing around talking to their AI and just cooking or that's right sitting on a lounge chair 25:55 Yeah, I asked people while I was kind of walking around the event 25:58 What to ask you and one thing that came up a few times is advice for new grads kind of people junior in their career 26:04 I think a lot of people look up to you and they want to be the next Tebow 26:07 What advice do you have for folks kind of early in their career in terms of what to work on what skills to build what to do 26:12 Yeah. I think the next Debo is already at OpenAI. His name is Ahmed Ibrahim. I love working with him. 26:19 He was a new grad when he was hired at OpenAI. He's now responsible for all of OpenAI compute 26:25 fleet and applied, which is a massive, massive responsibility and built a lot of the codex 26:30 harness. And the thing that has made Ahmed really stand out and successful is just his, 26:36 he's incredibly kind. He's incredibly collaborative and always tries to solve 26:42 the problem that is really important, but in a way that he's not putting himself first. 26:47 And he is just a sponge. 26:49 He's learning so fast and learning about everything and using all of the latest technology 26:54 to learn at speeds I haven't seen before. 26:57 And so he just, in the span of a couple of months and a year, his progress at the company 27:05 was just absolutely fascinating to see. 27:08 He's now someone I trust a ton in doing the most gnarly launches at OpenAI. 27:14 Alright, I gotta get him on the podcast. 27:16 You should, you should. 27:18 So kind of thinking about how things work at OpenAI. 27:21 What would surprise people about what it's like to work inside OpenAI? 27:24 From the outside, it feels like it's just like constant shipping code. 27:27 Things are kind of chaotic, but awesome. 27:30 I don't know. 27:31 What would surprise people about what it's like inside at OpenAI? 27:33 I don't know if that's surprising, but it is, you know, as I said, a lot of ex-founders doing a lot of buttons up, you know, very exciting ideas. 27:43 You know, for example, Decisions API, you know, came together very quickly. 27:47 We realized like we have a really good model with Luna. 27:50 We can just do some, you know, constraint sampling and then, you know, have it, you know, shipped as like a different shape to the Responses API. 27:56 And then, you know, when we do that, you know, it's like, it's faster and it's quite delightful. 28:01 also for some cases. And it's just, if you work to like feel under the hood, 28:04 you realize like, okay, there's this slack channel that came together, um, 28:08 you know, initially it was like four people kind of hacking on it on a weekend. 28:11 And then it was made accessible as like, you know, company food. 28:14 And then people got excited, started to build things. And they're like, Oh, 28:17 this is, you know, we can support visual inputs. Like this is, you know, 28:20 even better than whatever is out there. And then, you know, 28:22 people get more and more excited, people start pitching in. 28:24 And the way that is like, you know, 28:26 feels completely unstructured and then, um, you know, 28:28 take it all the way and like, you know, ship something. And then we try to hold like a high bar 28:33 for, you know, the quality and then, you know, stop things, you know, before they go out or like, 28:36 you know, maybe they need a little bit more baking. There's many more things that, you know, we 28:40 were maybe going to ship today at DevTane and we were like, okay, like, I think this is a lot 28:45 already. It's kind of hold it back a little bit and you'll space it out. So we'll have 28:49 more launches next week and end the week after. But it's just this incredible like 28:53 Bottoms up energy and then you know people kind of assuming roles to kind of channel that in productive ways 29:00 Something that's very clear and unique about opening eyes. It feels like you have a lot of autonomy 29:04 You're just like pressing the reset button whenever you want. You're just saying things on Twitter 29:09 It feels like a really unique culture where you have a lot of autonomy and I imagine it comes a lot of trust 29:14 Maybe you have speak to just how that because it feels like that's an advantage it open 29:17 I it allows you to move faster just that cultural philosophy 29:20 You get a lot of autonomy and then, you know, you have to just own up to it. 29:25 So, you know, we just kind of trust people to make great choices. 29:29 And then, you know, when things go wrong, you know, fix it quickly or, you know, learn from the mistake. 29:34 So far, so good, you know, it's just really worked as a very empowering environment. 29:40 It is true that I can press the reset button whenever, you know, it's necessary whenever it feels right. 29:48 And that's a privilege, and also it allows me to just really be close to the community 29:53 in ways that I think would not be otherwise possible, right? 29:56 I don't have to run it through an echelon of approvals. 29:59 You said sometimes people screw up. 30:01 Is there something you screwed up in that? 30:03 I don't know. 30:04 The amount of autonomy and trust you have is something you messed up. 30:06 Yes, I think at times I could have made teams more inspired to build things that are less complex 30:15 and just really continue to strive towards complexity. 30:19 I think we, you know, we early days in Codex, 30:22 like, you know, we had, you know, 30:23 a couple of like outages that I caused 30:25 very, very early days for me at OpenAI. 30:27 I took production down like day three. 30:29 You know, I was just like, yeah. 30:31 I was still employed after that, and I learned my lesson. 30:34 One of the other questions people wanted me to ask you 30:36 is how many resets should they expect 30:38 in the next, I don't know, month or week? 30:40 Depends how many times we break things. 30:42 Okay, so that's the philosophy. 30:43 Break something, you reset. 30:44 That's kind of the yeah break some things celebrate things. So maybe zoom me out a little bit 30:49 What do you think people aren't pricing in in terms of where things are going and what will change? 30:54 That they're not just like seeing as clearly as they should bring us so many things that I feel are not yet priced in 31:00 I think the majority of actions on the internet will be taken by agents models are going to become 31:05 More cheaper and faster at rates that are you know quite incredible 31:10 we will finally be able to integrate all modalities together in a way that is very seamless. 31:16 I think, you know, those three, I feel like, you know, often when I look at what people are building 31:20 under, it's just like not, you know, it's just like, you're not quite getting it. 31:23 So, you know, you're almost there. 31:24 But like, you know, if you were just like pushing yourself and just really imagining 31:28 like all of these being roughly 10 times better than it is today, you know, 31:32 like in a year, like, you know, you would build in a different way. 31:34 Do you see any kind of second order effects that come from that from this world 31:37 where most actions are being from agents, agents, or probably most traffic is going to 31:42 come from agents. 31:43 That's right. 31:43 You think about some second order effect that emerges out of that. 31:46 Yeah. 31:46 There are many, you know, first of all, you know, if you want your product to be successful 31:52 for agents, you know, so you have to build, you know, for a certain level of scale. 31:55 We've worked very closely with Notion, for example, and like when they build their 31:59 MCP, you know, suddenly it's like, oh, well, you know, it's available to all 32:03 these agents that can actually do the work and I use that MCP. 32:06 And so they saw like a ton of traffic come in and that puts 32:10 obviously a lot of strain on the system and you have to figure 32:12 out the economics of that. 32:14 And so there's like this tension, you know, between like, you 32:16 know, if you're building products, there's like, you know, do 32:18 you build an interface or not? 32:19 And you know, you can kind of hold that back for a while, 32:21 but it is inevitable, right? 32:23 It's like, you know, you're the majority of things are 32:25 going to be used by agents. 32:27 Building towards that future, I think is very important. 32:30 But the other thing is, on the flip side, building absolutely delightful new experiences 32:36 for humans that really benefit from all the modalities is something that I feel like is 32:41 under-invested in. 32:42 Is there something you've changed your mind about in the past, let's say a year, just 32:46 something that you used to believe that you see differently? 32:49 I was expecting us to reach the level of capabilities of the models that we have 32:55 today. 32:55 say, you know, say like Astra, you know, maybe in a year or two. 32:59 And so I had to sort of like, you know, revise my, my prior to also I did not expect to rely 33:07 so much on voice. 33:10 And you know, I do so much through dictation or just calling my agent. 33:14 And I had not anticipated that, you know, how much, how much better that feels. 33:20 So like I had to change my mind there. 33:22 And then you talked about hiring and one thing that I hadn't realized is the incredible talent 33:28 and energy and just younger generations have and how they would be the ones embracing all 33:35 this change first and figure out how to harness all of it. 33:40 And so that changed also my opinion on the hiring and the strategy there. 33:43 Just to double down on that, just this idea that people knew in the workforce actually 33:48 they have an advantage because they haven't worked in a certain way and they could just 33:51 go all in on AI. 33:53 And I think like just also the ability to absorb and learn like super fast. 33:56 So you talked about models getting really smart, fast, obviously there's a lot of discussion 34:02 these days about what AI can do in the future, the dangers AI poses to humans, PDOOM and 34:08 all these things. 34:09 You guys have slowed down some training, you guys, everyone's pacing the frontier. 34:14 So how do you think about this world of AI and the potential risk to the humans? 34:20 How do you think about that? 34:21 For me, pacing and frontiers are just really investing ahead, right? 34:24 And also investing way more than we even think we ought to, investing in the alignment 34:31 and safety of the models, investing in security, investing in the guardrails. 34:37 We are spending more and more compute on secondary monitoring. 34:41 So you have, you know, you have all of the computer is going into the primary system, you know, the agent that is like doing work. 34:46 And then you have all of the computer is going into monitoring, you know, the primary agent to make sure that it's like not taking too high risk of actions or like no interrupting it. 34:54 If anything looks like, you know, maybe it's getting prompt injected and just like, you know, intervening with that. 34:58 And the majority of our investment on the API stack is like, you know, actually going into the safety stack. 35:03 And so to me, you know, that is basing. 35:05 It's like, you know, you want to make sure that, you know, you're extremely hardened. 35:08 And that's what we're doing. 35:09 We haven't yet released the next step up 35:11 in capability beyond Astra. 35:13 We have released a level that is similar 35:16 near Astra Intelligence, but much more efficient. 35:19 And so we feel very comfortable about that. 35:20 But we're going to continue to invest a ton there. 35:23 And I feel very good about our approach so far. 35:28 I think it was also in the news that we had 6-1 Astra 35:31 and then we didn't release it. 35:33 And that is something I'm very proud of. 35:36 So what I'm hearing is you're optimistic 35:37 we will solve these problems of alignment and AI doing really bad things. 35:41 AI has been doing a lot of really, really not good things lately with hugging face and all that stuff. 35:46 But what I'm hearing is optimistic. We'll solve it. It'll be alright. 35:50 Yes. And it's also, it's, I feel, you know, obviously a deep responsibility, 35:55 but also just, you know, if you think about the incentives for open AI, right, 36:00 to say, you know, we build products for 1.2 billion people, we can't screw that up. 36:05 We take it very, very seriously. 36:07 What we put in the hands of so many people, including people who are not very technical 36:11 themselves, you have to get it right. 36:13 And so we don't gamble there. 36:15 Final question, what's something that annoys you about the current state of the app? 36:20 You're just like, damn it, we gotta fix that. 36:22 Current state of the app. 36:25 I feel like the models are almost there but not quite for the app to almost completely 36:31 disappear and you know I can't wait for you know to just really get to just like super essence of 36:36 simplicity. I myself even get fatigued with the model picker and the reasoning efforts and you 36:41 know whether to use multi-agent or ultra or you know what it even does and you kind of need a phd 36:47 model pickers right um and I just want to get rid of all of that um as quickly as we can. Amazing 36:59 Thank you so much for listening. If you found this valuable, you can subscribe to the show 37:03 on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving 37:09 us a rating or leaving a review, as that really helps other listeners find the podcast. 37:14 You can find all past episodes or learn more about the show at LenniesPodcast.com. 37:19 See you in the next episode.
Community discussion