Sorcerers and Their Brooms: Sherwin Wu on How AI Is Rewriting Engineering at OpenAI
Lenny's Podcast
Sherwin Wu leads engineering for OpenAI's API and developer platform. Because so many AI startups build on those APIs, host Lenny Rachitsky framed him as having an unusually broad view of where the industry is heading. The conversation covers several connected questions: what software engineering looks like when nearly all code is generated by an agent, how managers should adapt, why many corporate AI rollouts may be failing, and what builders should expect from models over the next year or two. Wu's position throughout is that this is an unusually exciting and unsettled window. The job of engineers, and eventually many other jobs, is being redefined, and the people who engage now get to help set the new standards.
A barometer: how much of OpenAI's code does AI write?
Wu said he still writes code occasionally. For managers like him, using AI tools is now far easier than coding by hand, and all of his own code, along with that of some other engineering managers at OpenAI, is written by Codex. He said it is hard to measure exactly how much code across the company is AI-authored, because the vast majority of it, "close to 100%," is usually generated by AI first and then edited.
What OpenAI does track is adoption. According to Wu, about 95% of engineers use Codex daily, and 100% of pull requests are reviewed by Codex, so essentially any code merged into production has had Codex look at it and suggest changes. He also reported that engineers who use Codex more open about 70% more PRs than those who use it less. He added that the gap is widening as heavy users get more efficient, and that it may have grown since he last checked.
When Lenny asked whether it was fair to say that 100% of OpenAI's code is written by Codex, Wu declined that framing. He would not claim that all code running in production was written by AI, and attribution is hard. His estimate was that almost every engineer uses Codex heavily for all their tasks, so the vast majority of code is probably authored by it.
He also noted that trust is still uneven. Some engineers trust Codex less than others, but he said that almost every day he talks to someone who was surprised by what it could do, and each person's threshold for letting the model work on its own keeps rising. He cited a line from Kevin Weil, whom he described as OpenAI's VP of science: "This is the worst the models will ever be." Lenny mentioned that the developer of OpenClaw had recently said they trust Codex's output enough to feel almost certain they could commit it straight to master. Wu called them a great Codex user who gives the team close feedback.
From writing code to casting spells
Asked how the engineer's job will look in one to two years, Wu said it is likely to change significantly, and that engineers are in a rare moment, perhaps the next 12 to 24 months, when they get to work things out themselves and set their own standards.
He agreed with the common observation that individual-contributor engineers are becoming tech leads who manage "fleets and fleets of agents." Many engineers on his team have 10 to 20 threads in progress at once. These are not all actively running Codex jobs, but they are parallel lines of work that the engineer checks in on, steers, and gives feedback to. The work has shifted from writing code to something close to managing.
His favorite metaphor for this comes from Structure and Interpretation of Computer Programs (SICP), the MIT textbook that taught Scheme in the introductory programming course and has a cult following. He remembers the book's opening, which likens programming to sorcery: programmers are wizards, programming languages are incantations, and the challenge is finding the incantation that makes the program do what you want. The book is often called "the wizard book" for that reason, and Wu, who dated it to around 1980, said the metaphor has lasted. In his reading, programming languages have always been incantations that got steadily easier to use, and the current AI wave is the next step. Now the incantations are literal, because you can tell Codex or Cursor exactly what you want and it goes and does it.
He pushed the metaphor further to The Sorcerer's Apprentice from Fantasia. Mickey finds the sorcerer's hat, sets the brooms to work, falls asleep, and wakes to a flood that the old sorcerer has to clean up. Wu called this "vibe coding at its greatest." The point he draws is that these incantations are very high-leverage, but you have to know what you are doing. Running 20 Codex threads takes skill, seniority, and thought, so that the models do not go off the rails. You cannot walk away and ignore them. At the same time, a senior engineer who is proficient with these tools can now do far more, and Wu said this is also what makes the work fun: "It literally feels like we're wizards now." Lenny added a guest's comparison to a genie granting wishes, and Wu mentioned the monkey's paw: you get what you asked for, but with side effects.
The stress of agents that won't cooperate, and a team with no escape hatch
Lenny raised a growing complaint: people feel stressed when their agents stall and they feel they are wasting time. Wu said this happens constantly, and that it is where the interesting problems currently lie, because neither the models nor the tools are perfect and people are still learning how to work with them.
He described an internal experiment in which one team maintains a codebase written 100% by Codex. Normally, when an agent cannot build what you need, you have an escape hatch: you roll up your sleeves and write it yourself, perhaps with tab completion in Cursor. This team has deliberately given that up, so their only question is how to get the agent to do the task. Wu said OpenAI expects to publish a blog post on what they have learned.
One lesson he shared is that when a coding agent fails, the cause is usually context. The task was underspecified, or the agent did not have enough information about how something should be done. The fix is to encode the tribal knowledge in engineers' heads into the repository, through code comments, code structure, Markdown files, skills, or other resources the model can read. Taking away the option of not using AI, he said, forced the team to work through problems everyone will have to solve to lean fully into agents.
Code review: from 10–15 minutes to 2–3
More PRs means more code review, and Lenny asked how OpenAI keeps that from becoming a grind. Wu said the tasks people hand to models first tend to be the most annoying and boring parts of engineering, which is part of why the job is more fun now. He personally hated code review. At his first job out of college, at Quora, he owned the newsfeed code, which everyone touched, so he would log in every morning to 20 to 30 reviews, procrastinate, and watch the pile grow to 50.
He said Codex is very good at reviewing code, and that GPT-5.2 in particular has become strong at it, especially when steered well. Because Codex reviews every PR and leaves suggestions, a review that used to take 10 to 15 minutes can sometimes take two to three. For small PRs, people sometimes skip human review and rely on Codex as the "second pair of eyes" that catches anything dumb.
OpenAI has also automated much of the post-push process with Codex. Wu said that what frustrates engineers most is getting code into production after writing it: tests, lint errors, review. Internal tools now have Codex fix simple lint errors, patch them, and restart CI. The aim is to reduce the engineer's work at each step, and the result, he said, is that engineers can merge and ship many more PRs.
Lenny asked about the circularity of Codex writing and then reviewing its own code. Wu acknowledged it and returned to the brooms: OpenAI is careful about which PRs are reviewed only by Codex, and most people still look at their own PRs. He described the shift as going from 100% attention to about 30%, not to zero. OpenAI tests many models internally but uses external models less, because it wants to dogfood its own. Internal model variants can provide a different perspective in review, which he said works well.
How the manager's job is changing
Wu said the manager role has changed less than the engineer's, since "there's no Codex for managers" yet, although he uses Codex for some managerial tasks. He sees trends that suggest where things are heading.
The first is that Codex, and perhaps AI across society, disproportionately empowers top performers and high-agency people. That widens the spread of productivity within a team. His long-standing management philosophy is to spend most of his time with top performers, keeping them unblocked, productive, and heard, and he thinks this matters even more now. His example was the 100%-Codex team: letting them run with it has paid off. He also described a group of engineers exploring best practices for working with the model. He considers that very high-leverage, and OpenAI spreads what they learn through knowledge-sharing sessions and documents. Lenny linked this to Marc Andreessen's recent line on the podcast that AI makes good people better and great people exceptional.
The second trend concerns span of control. During performance reviews, Wu said, ChatGPT connected to internal knowledge such as GitHub, Notion, and Google Docs can quickly build a picture of what someone did over the past 12 months and draft a deep-research-style report. His view is that just as engineers now manage 20 to 30 Codex threads, these tools may let managers run teams much larger than the typical six to eight in software engineering. He compared this to support and operations, where handing more work to agents lets a team do more and a manager oversee more people. He said some OpenAI engineering managers already handle large teams effectively with these tools.
The surgeon and the manager who looks around corners
Asked for other management lessons, Wu said his philosophy has mostly stayed the same over time. To be concrete about the top-performer principle, he said it means spending more than 50% of your time with roughly your top 10%.
His guiding image comes from The Mythical Man-Month, which he believes was written in the 1970s. The book predicted that software teams might be organized like surgical teams: one surgeon operates, and everyone else, nurses, residents, fellows, exists to support that person and hand over the scalpel when asked. Wu said this did not really happen, since software is far more collaborative. But he tries to make each person on his team feel like the surgeon, as though an army is supporting them, when really it is just him as manager. In practice that means looking around corners and clearing organizational blockers. He argued this matters even more with AI: when engineers are producing PR after PR, the bottleneck is usually organizational or process-related. The ideal is a manager who already has the scalpel ready before the surgeon asks.
Lenny suggested AI could help anticipate these blockers. Wu said he had not tried it but found the idea compelling: ask ChatGPT, connected to company knowledge, Notion, and Slack, what his team's active blockers are and how he could help, and even what blockers are likely next month.
The second- and third-order effects of the one-person billion-dollar startup
When asked what people are not yet pricing in, Wu chose the idea of the one-person billion-dollar startup, which he thinks Sam Altman may have been the first to describe. He finds the idea exciting but thinks people overlook its knock-on effects.
The second-order effect, as he sees it: if one person can build a billion-dollar company, it is also much easier for anyone to start a company at all. He expects a large boom in startups and small businesses. He pointed to the verticalization already visible in AI startups, where building a tool for one domain works well because the builder deeply understands the use case. Extrapolated, there could be 100 times more such companies. Supporting a single one-person billion-dollar company might take a hundred small startups building bespoke software for it, and he suggested we could enter "a golden age of B2B SaaS." In his framing, there might be one billion-dollar solo company, a hundred $100 million startups, and tens of thousands of $10 million businesses. For an individual, a $10 million business means being set for life.
The third-order effect, which he flagged as increasingly uncertain, concerns venture capital. If the world fills with micro-companies building software for one or two owners, the startup and VC ecosystems will change. There might be a few large platform players, while the number of startups capable of venture-scale 100x or 1,000x returns could shrink. Companies worth $10 million to $50 million are poor venture returns but great for high-agency individuals. He declined Lenny's joking request for a fourth-order effect.
Lenny pushed back. From running his own business, which he said is not venture-scale, the volume of support requests makes a solo billion-dollar company hard to imagine unless contract size is very high or the operator relies on contractors. Wu agreed partly but offered a different picture. The solo founder would not personally dispatch AI to handle every ticket. Instead, a crowd of tiny startups, perhaps 10 or 20 building support software specifically for podcasts and newsletters, each possibly run by one person, would sell tailored tools to them. As the cost of building software collapses, companies may outsource more and stay smaller. He repeated that this is highly uncertain. Lenny offered his own candidate for a fourth-order effect: as more things compete for attention, distribution and audience become increasingly valuable.
Why many enterprise AI deployments may be negative ROI
Wu had previously told Lenny that many companies' AI deployments may have negative ROI. He clarified that he does not see quantitative figures, since this is hard to measure, but that from watching companies try to adopt AI, he would not be surprised. He also linked it to a sentiment he sees outside tech that AI is being forced on people.
His first observation is that Silicon Valley, X, and software engineering are bubbles. Most people, including most Americans, are not engineers, are not "AI-pilled," and do not follow model releases. Power users talk about skills, AGENTS.md, and MCP, but when Wu talks to employees at customer companies, they are attempting very basic things with little understanding of how the technology works.
His recommendation is to pair top-down buy-in with bottom-up adoption. Top-down means executives commit to becoming AI-first, buy the tools, and back the effort. Bottom-up means employees doing the actual work are excited, willing to learn, build best practices, and share them. He said this is how it played out at OpenAI: the company always wanted to be AI-centric, but adoption really took off with Codex and similar tools that let employees apply AI to their own work. Because work differs so much across engineering, finance, operations, and sales, the "last mile" has to be worked out from below. When deployment is a pure executive mandate, the result is a workforce that knows it is supposed to use the tools, perhaps sees usage on its performance reviews, doesn't know what to do, and has nobody to learn from.
His concrete advice is to form, or even staff full-time, an internal "tiger team" that explores capabilities, applies them to specific workflows, shares knowledge, and generates excitement. He said these teams usually aren't made up of software engineers, since many companies have none, and they are expensive and hard to hire. The people who tend to light up are technical-adjacent: for example, a support or operations lead who doesn't code but is an Excel wizard and loves the tools. Wu summed it up in terms of his management philosophy: find the high performers in AI adoption and empower them with hackathons, seminars, and knowledge sharing.
"The models will eat your scaffolding for breakfast"
Lenny raised Wu's view that listening to customers isn't always the right strategy in AI. Wu said it isn't much of a hot take, since you should obviously talk to customers. His point is that the field and the models change so fast that they disrupt themselves, especially in tooling and scaffolding.
He quoted a phrase from an X post by Nicolas Bustamante, founder of Fintool, who was sharing lessons from building AI agents for financial services: "The models will eat your scaffolding for breakfast." Wu traced the history. When ChatGPT launched in 2022, models were raw, and developers built layers of scaffolding such as agent frameworks and vector stores to steer them. Around 2023, he said, many believed vector stores would be the main way to bring organizational context into models, which meant embedding every corpus and tuning vector search to retrieve the right information at the right time. As models improved, a better approach turned out to be removing much of that logic, trusting the model, and giving it search tools, which could be any kind of search, including plain files on a file system. Vector stores still have uses, and many companies still use them, but he said the assumption that a whole ecosystem would be built around them has changed. He also said agent frameworks now seem somewhat less useful.
He applied the same reasoning to today's fashionable scaffolding, skills files and file-based context management. He could imagine models eventually managing that themselves, or the field moving to some paradigm that is hard to predict. So if you only follow customer requests, which at any moment reflect a local maximum ("I want a better vector store," "I want a better agent framework"), you risk building for that local maximum. He admitted that OpenAI's API team has been guilty of this and taken turns it shouldn't have. Lenny compared this to "The Bitter Lesson" in machine learning, and Wu agreed that there is a version of it for building with AI, one "we're all learning day in and day out."
His advice to builders is: "Make sure you're building for where the models are going and not where they are today." The startups he has seen do well build products around a capability that is maybe 80% of the way there. The product almost works, and then a model upgrade makes it click. He gave the example of something that half-worked with o3 and then became possible with GPT-5.1 or 5.2. Because models improve so quickly, the wait often isn't long.
What models should do in the next 12 to 18 months
Wu named two areas he is watching. The first is task length. He referred to the METR benchmark, which tracks how long a software engineering task a model can complete 50% or 80% of the time. From memory, he put frontier models at multi-hour tasks at 50% reliability and just under an hour at 80%. He called the chart sobering because it also plots earlier models, which makes the trend clear. He thinks today's products, including coding tools used interactively in the CLI, are optimized for tasks of at most about 10 minutes. Some people push Codex to multi-hour tasks, but he sees that as the exception. If the trend holds, he expects models within 12 to 18 months that can handle multi-hour tasks coherently, and perhaps eventually something like a six-hour, day-long task that you dispatch and leave running. Products built around that will look very different. You would still want to give feedback, he said, and "you probably don't" want it to run completely unsupervised for a day. The range of things models can do would expand substantially.
The second area is multimodality, mainly audio. He expects audio models, especially native speech-to-speech models, to get much better over the next 6 to 12 months, and mentioned interesting work on new model types and architectures for audio. He considers audio hugely underrated in enterprise settings. Everyone talks about coding, which is text, but a large share of the world's business, services, and operations happens through speech.
Business process automation as the underrated opportunity
Wu is bullish on applying AI to business processes, and he again said this is overlooked because of the Silicon Valley bubble. He distinguishes open-ended knowledge work from repeatable processes. Software engineering is open-ended and not repeatable, since you don't build the same feature twice, which is why tools like Codex, given open-ended goals to explore, do well. He placed data science and some strategic finance in that same category.
Much of the economy, he said, runs on repeatable operations that a manager has refined over time, usually written up as a standard operating procedure that people shouldn't deviate from. In engineering, ingenuity comes from deviation, but calling a support line or a utility company means someone is following one of these procedures and a set of rules about what they can and cannot do for you. The opportunity he sees is automating these processes with high determinism, fully integrated with business data, business decisions, and an enterprise's various systems.
When big enterprises ask him how AI will reshape their company over the next 20 years, he says software engineering is part of the story but the business-process side may look even more different. He was careful about size: he doesn't know whether this opportunity is larger or smaller in absolute terms than software, which is itself huge. But he considers it definitely larger than you would guess from how little it is discussed on X.
Will OpenAI squash your startup?
To the question founders ask most, how to avoid being crushed by OpenAI, Wu's answer was that the market is so large that startups shouldn't overthink it. Every startup he has seen fizzle out failed because it built something customers didn't want, not because OpenAI, Google, or another lab squashed it. Even in crowded spaces like coding, he said, Cursor has grown huge because people love the product. As a sign of how big the opportunity is, he pointed out that the Overton window for VCs has shifted: they now invest in directly competing companies. He called this "the most empowering thing in the world" for founders, since building something some people really love can lead to a very valuable business.
He also described OpenAI's own stance. He said Sam Altman and Greg Brockman have reinforced from the top that OpenAI sees itself as an ecosystem and platform company, and that the API was its first product. He gave several examples. Every model released in OpenAI's products also ships in the API, including Codex models tuned for the Codex harness. OpenAI tries to keep the platform neutral and doesn't block competitors from its models. It has been testing "Sign in with ChatGPT." He described OpenAI as perhaps an aircraft carrier that still benefits from raising the tide for everyone, and said the API has grown significantly because OpenAI operates this way.
He grounded this in OpenAI's charter: build AGI, and spread its benefits to all of humanity. ChatGPT tries to reach everyone, but OpenAI launched the API early, around 2020 by his recollection, because it did not believe it could reach every corner of the world alone. It needs others to build things like "the customer support bot for podcasters and newsletter hosts." The ChatGPT apps SDK, owned by the ChatGPT team but built closely with his, follows the same logic: ChatGPT has about 800 million weekly active users, which Wu described as roughly 10% of the world and growing fast, and opening that audience to other companies should help expand it further.
Lenny praised the fact that the free version of ChatGPT is not so different from the most capable model. Wu agreed, pointing to OpenAI's work in health and education, and said the free model has become far smarter since 2022 and now gives people GPT-5. He compared it to how ordinary people use the same iPhone as billionaires: for $20 a month you use essentially the same AI they use, and for $200 you get the same Pro model, though he suspects billionaires mostly use the Plus-tier models day to day.
The layers of OpenAI's developer platform
Wu described the API as layers of increasing abstraction. At the bottom is the Responses API, the most popular endpoint, which he said is optimized for long-running agents. You give the model text, it works for a while, you can poll it, and eventually you get a response. It is deliberately unopinionated and lets developers build almost anything.
Above that is the Agents SDK, which has also become very popular. It helps build what people traditionally think of as an agent: an AI running in a loop that may delegate to sub-agents, with guardrails and orchestration of a "swarm" of agents. Wu noted the irony that this is itself scaffolding, adding "we'll see where this all goes." Above that are tools for the meta-level work of deploying agents: AgentKit and widgets, UI components for quickly building a polished interface on top of the API or the Agents SDK, since agent UIs tend to look alike. There is also an Evals API for testing quantitatively whether a model, agent, or workflow is performing. Developers can use the whole stack to build an agent quickly or go as far down as the Responses API for maximum flexibility.
Lightning round: books, anime, home networks, and pricing houses
For books, Wu recommended the novel There Is No Antimemetics Division by qntm, about a government agency fighting things that make you forget them. He finished it in two days and found it creative, fresh, and unintentionally funny. For nonfiction, he recommended two books on the US and China: Dan Wang's Breakneck, whose contrast between a "lawyerly" US and an "engineering" China he found persuasive, and Patrick McGee's Apple in China, which appealed to him as an Apple fan for both its account of Apple's relationship with China and its inside detail about the company. With two kids and a busy job he watches little TV, but he is a big anime fan and enjoyed season 3 of Jujutsu Kaisen. He believes Japanese anime creates novel plots and worlds that Western media has shied away from. His favorite recent product is Ubiquiti's home networking and security cameras, which he called "the Apple of home networking," mainly because of the software and its mobile, Apple TV, and iPad apps. His life motto is "never feel sorry for yourself," a reminder that you always have the agency to pull yourself up.
Lenny asked about Wu's earlier work at Opendoor, where Wu worked on the models that decided how much to offer for a house. Wu named three variables that were more important than expected. High-voltage power lines lower prices a lot, which Wu fully understood only after standing next to a buzzing line in Dallas and realizing families don't want their kids near it. Floor plans were very important but extremely hard to quantify. The team measured things like kitchen width and style and where the primary bedroom was, but sometimes a house wouldn't sell and the ops team would go in and say it was a floor-plan issue you could only feel in person. Floor plans often weren't even digitized; a handful of people held paper plans for homes in places like Phoenix and Dallas. Finally, curb appeal, down to the front door, mattered more than he expected. He recalled a Zillow book that reportedly found front-door replacement to be the highest-ROI home improvement.
Don't take the next two to three years for granted
Wu's closing message was that the next two to three years will be some of the most fun in tech and startups for a long time. He entered the workforce in 2014 and said the first couple of years were great, followed by roughly five or six years when tech wasn't very exciting. The past three years have been the most energizing period of his career, and he expects the next two to three to continue that, before the wave eventually plays out and progress becomes more incremental. Until then, he said, people will explore, invent, and change how we work, and he is trying not to take it for granted himself.
His advice for not missing it is to engage. You don't need to be a software engineer. Use the tools, learn what they can and cannot do, and watch how that boundary moves as models improve, rather than sitting back and letting it pass. On the anxiety of keeping up, he admitted he is a bad example because he is "chronically online" on X and OpenAI's Slack. From watching others, though, he said much of the flood is noise, driven by the industry's frantic pace and by how X works as a product. His suggestion is to start small with one or two tools: install the Codex client and experiment, or connect ChatGPT to a few data sources such as Notion, Slack, and GitHub and see what it can and cannot do.
95% of engineers use Codex. 100% of our PRs are reviewed by Codex for engineers. I don't know what job has changed more in the past couple years. Engineers are becoming tech leads. They're managing fleets and fleets of agents. It literally feels like we're wizards casting all these spells. And these spells are kind of like going out and doing things for you.
What do you think people aren't pricing in yet? The second or third order effects of the one-person billion-dollar startup. To enable a one-person billion-dollar startup, there might be a hundred other small startups building bespoke software. And so I think we might actually enter into a golden age of B2B SaaS.
I've been hearing more and more there's this stress people feel when their agents aren't working. There's a team that's actually doing an experiment right now within OpenAI where they are maintaining a 100% Codex-written codebase. They run into the exact problems that you're describing. And so usually you're like, all right, I'll roll up my sleeves and figure it out. Team doesn't have that escape hatch.
You've shared that listening to customers is not always the right strategy in AI.
The field and the models themselves are just changing so, so quickly. They tend to like disrupt themselves. The models will eat your scaffolding for breakfast.
What's your advice to folks that are like, "Okay, I don't want to miss the boat"? Make sure you're building for where the models are going and not where they are today. There's a quote from Kevin Weil, our VP of science here. He likes saying, "This is the worst the models will ever be."
Today, my guest is Sherwin Wu, head of engineering for OpenAI's API and developer platform. Considering that essentially every AI startup integrates with OpenAI's APIs, Sherwin has an incredibly unique and broad view into what is going on and where things are heading. Let's get into it after a short word from our wonderful sponsors.
Today's episode is brought to you by DX, the developer intelligence platform designed by leading researchers. To thrive in the AI era, organizations need to adapt quickly. But many organization leaders struggle to answer pressing questions like which tools are working? How are they being used? What's actually driving value? DX provides the data and insights that leaders need to navigate this shift. With DX, companies like Dropbox, Booking.com, Adyen, and Intercom get a deep understanding of how AI is providing value to their developers and what impact AI is having on engineering productivity. To learn more, visit DX's website at getdx.com/lenny. That's getdx.com/lenny.
Applications break in all kinds of ways. Crashes, slowdowns, regressions, and the stuff that you only see once real users show up. Sentry catches it all. See what happened where, and why, down to the commit that introduced the error, the developer who shipped it, and the exact line of code all in one connected view. I've definitely tried the five tabs and Slack thread approach to debugging. This is better. Sentry shows you how the request moved, what ran, what slowed down, and what users saw. Seer, Sentry's AI debugging agent, takes it from there. It uses all of that Sentry context to tell you the root cause, suggest a fix, and even opens a PR for you. It also reviews your PR and flags any breaking changes with fixes ready to go. Try Sentry and Seer for free at sentry.io/lenny and use code Lenny for $100 in Sentry credits. That's S-E-N-T-R-Y.io/lenny.
Sherwin, thank you so much for being here and welcome to the podcast.
Thank you. Thank you for having me.
I want to start with what's feeling like a barometer of progress in AI, especially in engineering. What percentage of your code, if you even write code anymore, and your team's code is written by AI at this point?
I do write code occasionally now still. And I actually say for managers like myself, it's way easier to use these AI tools than to manually code at this point. And so I know for myself and some of the other engineering managers at OpenAI, all of our code is written by Codex at this point. But more broadly, there's just so much energy. There's like a tangible energy internally around just how far these tools have gotten, how good Codex as a tool has gotten for us.
And it's a little hard for us to exactly measure how much of the code is written because the vast majority of it, I'd say like close to 100%, is usually generated by AI first. What we do track though is, at this point the vast majority of engineers use Codex on a daily basis. So 95% of engineers use Codex. 100% of our PRs are reviewed by Codex daily as well. So basically any code that goes into production that's merged in, Codex kind of has its eyes on and suggests improvements, suggests changes in the PRs. And so that's kind of what we're seeing internally. But by and large the most exciting is just the energy that there is.
Another observation that we've had is engineers who tend to use Codex more open way more PRs. So they're actually opening 70% more PRs than the engineers who aren't using Codex as much. And the gap is widening. So I feel like the people who are opening more PRs are starting to learn how to use the tool more and more, get more efficient, and that 70% gap keeps growing over time. And so it might have actually increased since I last looked at the number.
Okay. So just to make sure we hear what you're saying, you're saying all of the code of these 95% of engineers at OpenAI is written by AI. It's written and then they review it.
Yep. Yep.
It's like crazy that that's almost like not crazy anymore, that we're just like getting used to this.
I think there's still some getting used to, to be clear. There's also, I think, some engineers who trust Codex a little bit less, but basically every day I talk to someone who is blown away by something that it can do, and kind of like their bar of trust, or like how much they trust the model to do on its own, goes up over and over over time. And there's a quote from Kevin Weil, our VP of science here, and he likes saying this is the worst the models will ever be. And so this is the worst that the models will ever be for software engineering as well. And so over time you just see people trusting it more and more, and then we'll see the models get better and better as well.
Yeah. Kevin Weil, former podcast guest, he said exactly that line on this podcast and a few times.
Yeah. Peter, the Clawdbot/Moltbot/OpenClaw, is what it's called now, developer recently shared that he uses Codex for his work and he feels like anytime it does things he just trusts that it has done the right job, and he's just like almost certain he could just commit it to master and it'll be great.
Yeah. Yeah. He's a great user of Codex. I know he's in close touch with the team, gives us great feedback. Not surprised that he uses it. I mean, sorry, it's called OpenClaw.
OpenClaw. Yeah.
OpenClaw is a great product. And then I saw that, I mean this is very recent, but this morning I think Moltbook kind of like was shared as well, and seeing all the AI agents talk to each other is pretty surreal.
It's basically Her is happening in real life is what I'm hearing.
Yeah. Yeah.
So just like coming back to this crazy moment we are living through for engineers in particular. We've gone from you write every line of code to now AI is writing all of your code. I don't know what job has changed more in the past couple years, like a job that we didn't expect to change this much, where just like the job of an engineer is so different. In the entire lifespan of an engineer, like in the past couple years, it's now shifted to I don't write any more code. How do you imagine the role of an engineer and the job of a software engineer looks in the next couple years? Just like, what is that job?
Yeah, it's, I mean it's honestly been really cool to see, and it's part of where the excitement is, because the job is likely going to change pretty significantly over the next one to two years. It kind of feels like we're still figuring things out though, and so there's like this excitement, I know especially from some of the software engineers, of like, we're in this rare moment, maybe over the next 12 to 24 months, where we'll kind of get to figure things out ourselves and set our standards for ourselves.
In terms of where I see this moving. So I think there's a common thing that everyone's saying, which is people are generally, like IC engineers are becoming tech leads. They're basically like managers now. They're managing fleets and fleets of agents. I know many of the engineers on my team basically have like 10 to 20 threads kind of being pulled on at the same time. Obviously not active running Codex jobs, but just a lot of parallel threads. They're checking in on what they're doing. They're steering the agents in Codex and giving it feedback. And so their job has kind of really changed from just writing the code itself into being almost like a manager.
In terms of where I think this will go one to two years from now. So one kind of metaphor that I kind of always come back to here is actually from this programming textbook that I read back in college called SICP. I don't know if you've heard of it. Structure and Interpretation of Computer Programs. So SICP. At MIT it was really popular, and it was actually used as the introductory, it was the textbook for the intro programming course for a very long time, and it kind of has this cult following. It teaches you programming. It teaches you a dialect of Lisp called Scheme, and so it like introduces you to like functional programming. It's like very mind-opening in that way.
But the thing that was memorable for me about that book, so I kind of read it in college, the very beginning of it kind of describes programming as a discipline and draws this metaphor to basically like sorcery. Like it says software engineers are like wizards, and programming languages are like incantations, and you're like, you know, you're issuing these spells, and these spells are kind of like going out and doing things for you, and the challenge is like what incantation do you have to say to make the program do what you want.
And this book was written in 1980, so this is a while ago, and I think that metaphor has actually kind of persisted over time. And I think it's actually playing out as we move into this new era of vibe coding, or just like what software engineering will look like, because programming languages were basically these incantations. They've changed over time, and the trend has been that it's been easier and easier to kind of get the computer to do what you want via programming. And I think the current wave of AI is probably the next stage of that evolution. It is now literally incantations, because you can tell Codex, you can tell Cursor exactly what you want to do and then it'll all go do it for you.
And I particularly like the wizard and the sorcery analogy because I think our current state is starting to move towards kind of like the Sorcerer's Apprentice from Fantasia, where Mickey Mouse is like, you know, he finds the sorcerer hat and he tries to do all these things. And I actually think it's a really apt analogy because, one, it's just really powerful now. These incantations you can do are extremely high leverage, but you kind of have to know what you're doing, right? Like in Sorcerer's Apprentice the whole plot is like Mickey goes wild, the brooms go crazy, and everything's flooding. I think he literally sets the brooms off on a task and then goes asleep. And so, you know, it's like vibe coding at its greatest, and then eventually the old sorcerer comes back and cleans everything up.
And, you know, when I see engineers kind of like doing these 20 different Codex threads at a time, there is some skill and there's some seniority and a lot of thought that needs to go into this, because you want to make sure that the models aren't going off the rails. You definitely don't want to just completely go away and ignore the thing. But it's also extremely high leverage. Like a very senior engineer who's really proficient with these tools can now just do way more things via what they're doing. And I think this is also what makes it fun. Like it literally feels like we're wizards now. You know, it feels like we're closer to making it feel like this magical experience where we're casting all these spells and having software do all these things for you.
I was thinking of the Sorcerer's Apprentice exactly as the metaphor as you were describing that, so I'm glad you went there. A previous podcast guest described it as you have a genie that grants you wishes, and it's a useful frame because you have to be very clear about the wish you want. Like if you want to be big, like how big it could be.
Yeah. Or it might be like the monkey's paw type thing where, you know, it's like you got what you want, but what are the side effects?
Yeah. Yeah. I think that, and the analogy is great, and yeah, the crazy thing for me is just the staying power of that book. SICP, like, it's called the wizard book. You know, people call it the wizard book because that is the metaphor that they kind of weave throughout the book. And we've basically reached that point now, which is really cool.
There's two kind of threads I want to follow here. One is I've been hearing more and more there's this like stress that people feel when their agents aren't working. You fire off all these Codex agents and then you have to keep, stay on top of them. Oh, one's not working. I'm wasting time. Do you feel that? Do you feel that across your team at all?
Yeah. Yeah. I mean, it happens all the time. And I actually think this is where the interesting part of all of this lies right now, because these models aren't perfect. These tools aren't perfect, and we're still trying to figure out how to best interact with Codex or with these AI agents to get work done. We see this come up all the time.
There's a particularly interesting team that we have internally. So there's a team that's actually doing an experiment right now within OpenAI where they are basically maintaining a 100% Codex-written codebase. So, you know, some, you'll have the AI write code, but you'll obviously end up rewriting a lot of it, and you might need to double-check and change things, but this team is just fully Codex-pilled and just leaning in entirely. And they run into the exact problems that you're describing, which is, their challenge is, you know, I want to get this feature built, but I can't get the agent to do it. And so usually there's an escape hatch where then you're like, all right, I'll roll up my sleeves and figure it out, and then instead of using Codex I might use tab complete in Cursor and things like that. But this team, for the experiment, this team doesn't have that escape hatch. And so then the challenge is like, how do I get the agent to do this?
And I actually think we're going to be publishing a blog post from some of our learnings here. But a lot of fascinating paradigms and best practices are falling out of this. One interesting thing that we've noticed, I don't know if this is what
you kind of feel, but we definitely feel it here, is a lot of the time, when the coding agent is not doing what you want, it's usually a problem with context and just information that you've given it. It's just you've either underspecified or there's just not enough information around how to do something available to the agent, available to Codex. And so when you have to solve it through that, the challenge is then to add documentation and actually work around this limitation and basically encode more tribal knowledge that's in your head somehow into the codebase, either via code comments itself or code structure itself or via text files like MD files, skills, any type of additional resources within the repository so that the model can better do its task.
There's a whole bunch of other learnings from this group which I think is fascinating to explore. But yeah, removing that escape hatch of no longer using AI has allowed them to start piecing together a lot of the problems that we'll have to solve if we really want to lean into agents.
Another issue people run into, you talked about how people are shipping PRs like crazy, a lot more PRs if they're working with AI. Obviously code review is becoming a bigger challenge. Is there anything you've figured out in your team to help speed that up, to make that scale, and not just create this terrible job for people where they're just sitting there reviewing PRs all day?
Yeah, I mean one thing is Codex reviews 100% of all of our PRs at this point. And so one really interesting thing that's happened is the things that we tend to hand to the models immediately tend to be the things that annoy us or are the most boring parts of software engineering. It's also why it's more fun now, because we get to do more of the fun things.
For me, speaking more for myself, I really hated code reviews. It was one of the worst things for me. I remember in my first job out of college, it was at Quora. I was working on the newsfeed, and so I owned the code for the newsfeed. And so I was a reviewer for newsfeed, and it was just the central piece of code that everyone would touch. And so every morning I'd log in and there'd be like 20 to 30 code reviews. I'd be like, oh my goodness, I've got to get through all these. I would procrastinate and then it grows to like 50. And so there's a lot of code reviews.
Codex is really good at reviewing code. So actually one thing that we've noticed that 5.2 in particular has gotten extremely adept at is reviewing code, and especially when you steer it in the right direction. And so for code reviews, yeah, we create a lot of PRs, but Codex reviews all of them, and it makes code reviews go from, I don't know, a 10, 15 minute task to sometimes even just a two to three minute task, because you have a bunch of suggestions already baked in. A lot of the times, especially for small PRs, you actually don't even need people to review. We kind of trust Codex in this way. The original author looks at Codex. The benefit of code review is to have a second pair of eyes to make sure that you're not doing anything dumb. Codex is a pretty smart second pair of eyes at this point, and so that's something that we've heavily leaned into.
The general CI process and the post-push and deployment process has also been heavily automated via Codex internally at this point. If you talk to a lot of engineers, the thing that annoys them the most is, after you've written your beautiful code, how do you get it into production? You've got to run through all these tests, you've got to fix lint errors, code review. There's a lot of automated stuff you can do with Codex, and so we've actually built some tools internally that help automate that process, automate the lint. If there's a lint error, it's a very easy Codex fix, and it could just patch it and then restart the CI process. So with all of that, we're trying to collapse it into as little work for an engineer as possible, and the byproduct of which is they can now merge and push out a lot more PRs.
Codex writing the code, Codex reviewing its own code. I'm curious if you are open to using other models to review your models' work. Is that a path, or is it just good enough, we don't need anything else?
So, I will say there's definitely a circular thing here, and going back to Sorcerer's Apprentice, you want to make sure you're not letting the brooms go crazy here. And so we're very thoughtful, I'd say, around which PRs are completely just Codex reviewed. Most people still obviously take a look at their PRs. And so it's not like it's going to zero. It's more like going from 100% attention to 30% attention, which just helps things push through.
In terms of multiple models, we obviously test a lot of models internally, and so we have a lot of those. We use external models less. We think it's important to dogfood our own models and get feedback there. But there are also a lot of internal variants of models that you can use to give you a different perspective here as well, and we've found that to work quite well.
Okay. So just to make sure we get a barometer of today's world at OpenAI in terms of AI and code, just so I understand, and then I want to move on to a different topic. 100% of code across OpenAI is written by Codex at this point. Is that the way to frame it?
I wouldn't make the statement that 100% of code running in production today is written by AI, and it's kind of hard to do attribution there. But almost every engineer heavily uses Codex in all of their tasks at this point. And so if I were to guesstimate, the vast majority of code at this point was probably authored by Codex.
Incredible. Okay. So there's a lot of talk, and we've been talking about the IC role, the work of an IC engineer. There's less talk about the changing role of a manager, especially an engineering manager. How has your life as a manager changed with the rise of AI, and what's the role of a manager in the future?
It's definitely changed less than an engineer's. There's no Codex for managers just yet. However, I use Codex quite a bit for some of the more managery tasks that I do. I'd say a couple things are changing. There are some trends. So I don't think it's changed that much yet, but I see trends, and I think if you play it out, you can kind of see where a lot of this is going.
One thing that's becoming increasingly clear is Codex really empowers top performers to be a lot more productive. And I think this is maybe true for AI more broadly across society, which is the people who really lean in, or the people who have high agency or will get good at these tools, will kind of supercharge themselves. And so I'm noticing this now as well, which is the top performers end up being a lot more productive. And so you see a broader spread in team productivity in this way.
So one thing that I've always done as a management philosophy is to spend actually the majority of my time with top performers, just make sure they're unblocked, make sure they're happy, make sure they feel productive and they feel heard. I think this is even more true in an AI world, where your top performers are going to just really be shooting ahead using these tools. I think one example is the team that's maintaining a 100% Codex-generated codebase. Just letting them rip and seeing what's happening there is something that's paid dividends. So I think that's one trend that I'm seeing, where spending even more time with top performers for managers is likely going to continue.
The other thing is, so this is more an observation, but my sense is with a lot of these AI tools available to managers, so less writing code, but just things like ChatGPT with organizational knowledge, being able to do research and understanding organizational context a lot better. Another good example is we're doing performance reviews right now, and it's actually really easy to use ChatGPT with internal knowledge hooked up to GitHub and our Notion docs and Google Docs to get a really good sense of what this person has done over the last 12 months and write a little deep research report for it.
My sense is I think managers will be able to manage much larger teams in this world, kind of like how software engineers are managing 20 to 30 Codexes. My sense is that these tools will allow people managers to be higher leverage, and it will allow them to manage teams of way more than the current best practice of, I think it's like six to eight, right, for software engineering. You kind of see this applied to the non-engineering domains like support or operations, where previously the size of a support team might be limited, but as you can pass off more things to agents, you can actually do more work and also manage more people this way. I think the same thing might happen for people management as well, especially in tech companies. And we're already seeing this. There are some teams where there are EMs managing quite a few people, and they're doing it pretty adeptly because of some of these tools, where they can get higher leverage and understand what their team's doing, understand organizational context a little bit better, and operate in that way.
I love this advice, the way you described it, that you've always leaned into top performers and spent more time with them, unblock them, make sure they're happy. The way Marc Andreessen, he was just on the podcast, the way he phrased it is AI makes good people better and it makes great people exceptional.
Yeah. Yeah.
And what you're saying here is just doing this more and more is probably the right move. Spending more time with the best people on your team to unblock them, make sure they have everything they need.
Yeah. A very good example right now is there are, I would say, a group of engineers internally who are really into Codex and are thinking through what the best practices are for interacting with this model, and that is just an extremely high-leverage thing for them to do. And so as a manager I'm just like, yeah, go explore this. Whatever best practices come out of this, we have to share with the org. We do all these knowledge sharing sessions, we'll share documents and best practices everywhere. So things like that just elevate everyone, and I view that as another example of this trend that we're seeing where the top performers really get exceptional.
People just have a sense this is big. AI is changing so much. The world is changing. It's going to be a huge deal. What do you think people aren't pricing in yet into what will change, into where things are heading? What's an example of something where you think, okay, we're not realizing this yet?
So, one of my favorite phrases or things that have come out of this whole AI wave is the idea of the one-person billion-dollar startup. I actually think Sam may have coined it, or Sam may have been the first one to say it, but it's fascinating to think about, right? It's like, yeah, if people are so high leverage, at some point there will likely be a one-person billion-dollar startup. And while I think that's really, really cool, I think people aren't really pricing in the second or third order effects of this. Because what the one-person billion-dollar startup implies is that one person can just have so much more agency and so much more leverage using one of these tools that it is just super easy for them to get everything done that they need to for their business to ultimately create something that's a billion dollars. But I think there are a couple other implications of this.
One of them is, if it's possible for a person to create a one-person billion-dollar startup, it also means it's way easier for people to just create startups in general. I actually think one second order effect of this is there's going to be a huge startup boom and small, SMB-style boom, where anyone can build software for anything, right? You're starting to see this play out in the AI startup scene, where software has become a lot more vertical-oriented, where creating some AI tool for some vertical tends to work quite well, because you really lean into that particular domain, you really understand the use case for it. And so if you play out AI, there's no reason why you can't have 100x more of these startups. And so I think one world that we might end up seeing happen is, in order to enable a one-person billion-dollar startup, there might be a hundred other small startups building bespoke software that works extremely well to support other types of small one-person billion-dollar startups. And so I think we might actually enter into a golden age of B2B SaaS and just software and startups in general. And so I think that's a really interesting trend to see, because as it gets easier and easier to build software, as it's easier and easier to run a company, you might actually just end up seeing way more of these startups. So the way I've been thinking about it is, yeah, there might be one one-person billion-dollar startup, but there might be a hundred $100 million startups, there might be tens of thousands of $10 million startups. And as an individual, it's actually pretty great to have a $10 million business. You're set for life at that point. And so we might really see an explosion in that way, and I feel like people aren't really pricing that in.
There's another third order effect to this, and again, as you get to the further and further out predictions, there's a lot of uncertainty. I think if we end up moving to this world where you end up with these micro companies building software that works for one or two people who own the company and are working there, I think the startup ecosystem will change. I think the VC ecosystem will change. We might end up in a world where there's just a handful of big players that are offering platforms and supporting all of these startups. But the types of venture-scale return startups that can really 100x or 1,000x your investment might actually end up shrinking if you end up having a bunch of these smaller $10 to $50 million companies, which are not great for venture-scale returns, but are great for the individuals, the high-agency individuals who are now really leaning into AI to
build these businesses for themselves.
I love how many order effects we've been through. I want to hear the fourth order effect now. Sherwin, I'm just joking. I can't... fourth order is too gigabrain for me. I can't think that far ahead.
It's like Inception, where everything gets slower every time you go deeper into something, every layer.
Okay. So, the billion-dollar startup. I think about this a lot because I'm not going to be a billion-dollar startup, because what I'm doing is not venture scale in any way and not super high leverage, but just seeing how many support tickets I get from just the most ridiculous things, it's hard for me to imagine one person... I'm bearish on this billion-dollar startup. I just want to share this thought simply because of the support costs, even if AI is helping you. At a billion dollars, unless your ACVs are very high and you have very few customers, it's just dealing with support, and people are like, you know, they can solve their own problems, but they're like, I'll email support, ask about this thing. Just dealing with that is hard to scale, in my experience. So unless you have, in my opinion, unless you have a bunch of contractors, which, I don't know, does that count as a single-person company? I feel like it's very difficult to scale a billion-dollar startup and not have someone helping you with at least the support work, and AI I think will only take you so far.
So I think that's true. And actually I think my view on it is slightly different, which is I think that your, you know, Lenny's Podcast might end up becoming a billion-dollar startup. But what I think might happen is, instead of you being the one person who has to dispatch an AI to solve and fix those support tickets, I think what might end up happening is there might be a whole smattering of other startups that are building software super tailored towards what you might need. And so, you know, there might be 10 or 20 startups that build support software for podcasts and newsletters, and that might be a one-person startup. It doesn't need to be a big one. And they might be able to just code up this product very, very easily. They are able to build their own thing, and because it's so tailored and unique and hopefully useful for you, it might be something that you purchase as the one-person billion-dollar startup.
I would buy that. I would buy that.
Yeah. There's a question of what you in-house and what you outsource, and what I think might happen is, because the cost of writing software and building products is collapsing so much, you might end up outsourcing a lot of this and in doing so reducing the size of your company. And so that's kind of the world that I think might end up happening. Again, there's high uncertainty in what might play out here, but the end result still might be one person driving this massive, highly leveraged company that might actually reach a billion dollars.
I could see that. I also think about Peter at Clawdbot/Moltbot/OpenClaw, of just how barraged he is right now by all these asks and emails and pings and DMs and PRs. I'm curious to... and he's not even making any money off this thing.
Yeah, I can't imagine what it's like to be him right now. It must be absolutely insane. It's probably like the months after we launched ChatGPT, the craziness that was.
As one man.
He's coming out on the pod, by the way, in a week.
Oh, that's exciting.
Yeah. Maybe the fourth order effect is distribution becomes increasingly important, because there are so many freaking things trying to get your attention. So people with an audience and platform, I think, become more and more valuable, which is good stuff.
Okay. I wanted to come back actually to your management stuff. So I really loved your insight about spending more time with top performers, and that has been really successful for you. Just thinking about you as a manager of a team that is building the platform that powers basically the entire AI economy, like every AI startup is building on your API, clearly you're doing a great job. What other core management lessons have you learned? What do you find is really important and key to your success as a manager of engineers and just people?
Yeah. I think a lot of the lessons that I've learned here, I don't know how specific they are to the OpenAI API or some of our enterprise products in particular. I think my management philosophy has obviously changed over time, but I think it's probably stayed the same more than it's changed over time. One of these principles is kind of what I talked to you about before, which is, you know, spending a lot of time with top performers, like actually spending, and to be very concrete, it's more than 50% of your time with your top performers, with maybe your top 10% performers, and really, really trying your best to empower them.
The way that I think about it is to come back to this analogy of software engineer as a surgeon, which comes from The Mythical Man-Month book. So, it's actually funny. I pull it from the book, but in the book they actually described this world where, I think they were predicting the future, because I think the book was written in the 70s or something. They said that software engineering might end up moving into a world where software engineers are like surgeons, where in a surgery room there's one person doing the work. And you know, there's one person cutting or whatever and doing all the surgery, and everyone else in the room is there to just support them, right? As the nurse and the resident and the fellow. And then the surgeon's like, I need a scalpel, and they give them a scalpel, and then they're like, I need this tool and this machine, and they'll bring it over. Everyone's there to just support the one surgeon. And so The Mythical Man-Month actually predicted that that is kind of the direction that software engineering is going to go.
I don't think that's exactly played out, where it's much more collaborative and it's not only one person doing the work, but I've always really liked that analogy, and that analogy is actually what I strive to emulate in my own management philosophy, which is: software engineering isn't really like surgery, where it's just one person doing work, but the way in which I like treating the people on my team and the way that I act as a manager is I want to empower them, make them feel like they're a surgeon, insofar as making sure that I'm supporting them and making sure they have everything that they need to do their work, and it feels like they have an army of people supporting them and looking around corners and giving them everything that they need, when it's really just me as the manager.
And so the example that I give is that looking around corners and unblocking people, especially from an organizational perspective, is extremely, extremely useful. And again, going back to the AI conversations, it's even more important nowadays, right? If people are just cranking PR after PR, the main thing bottlenecking progress and shipping something tends to be organizational or process-oriented, and if you as a manager can look around corners and unblock the team, if the surgeon needs a scalpel but the manager already has a scalpel ready for them, that's the best-case scenario. That's kind of the way that I approach management and especially engineering management. And so that's something that's really stuck with me over time. And even though software engineers aren't exactly surgeons, that metaphor has always kind of stayed in my mind for the rest of my career.
I love that. And I wonder if that's something AI can help with, is look around corners and predict: here, this engineer is going to be blocked by this decision. We need to figure this out. We need to get...
Yeah, that's actually a really good point. I haven't tried this yet, but I wonder what would happen if I ask ChatGPT hooked up to company knowledge, you know, what are the active blockers? Look through all the Notion docs, maybe Slack messages, you know, it's probably in Slack somewhere. What are the active blockers on my team, and is there something I can do to help? I have not thought about that, but you're right.
You just had an insight right here.
Yeah. Yeah. Yeah.
And I think even more interestingly, what do you anticipate will be a blocker for this engineer or this team in the coming months?
Yeah. You asked the model. Well, you asked the AI to do the second and third order things. Anticipate that, man. Anticipate what the blockers will be next month, too.
I think we've got a good idea right here.
Yeah. Yeah.
This episode is brought to you by Datadog, now home to Eppo, the leading experimentation and feature flagging platform. Product managers at the world's best companies use Datadog, the same platform their engineers rely on every day, to connect product insights to product issues like bugs, UX friction, and business impact.
It starts with product analytics, where PMs can watch replays, review funnels, dive into retention, and explore their growth metrics. Where other tools stop, Datadog goes even further. It helps you actually diagnose the impact of funnel drop-offs and bugs and UX friction. Once you know where to focus, experiments prove what works. I saw this firsthand when I was at Airbnb, where our experimentation platform was critical for analyzing what worked and where things went wrong. And the same team that built experimentation at Airbnb built Eppo. Datadog then lets you go beyond the numbers with session replay. Watch exactly how users interact with heat maps and scroll maps to truly understand their behavior.
And all of this is powered by feature flags that are tied to real-time data, so that you can roll out safely, target precisely, and learn continuously. Datadog is more than engineering metrics. It's where great product teams learn faster, fix smarter, and ship with confidence. Request a demo at datadoghq.com/lenny. That's datadoghq.com/lenny.
Okay, I'm going to shift to talking about the API and the platform that you all build. So, you work with a lot of companies implementing your API, your platform, building on your tools. You told me that you find that a lot of companies actually have negative ROI on their AI deployments, which I think is what a lot of people read about and feel and think, and it's interesting you're actually seeing that. What's going on there? What are they doing wrong? What's happening in the world of AI deployments and ROI?
Yeah. So, to be clear, I don't explicitly see quantitative numbers around this. You know, it's actually really hard to measure these things, but especially from observing some companies trying to do AI, I would not be surprised if a lot of AI deployments are actually negative ROI. I mean, part of this too is I think there's also general sentiment from folks around the country, basically outside of tech, that AI is being forced onto them, and I think part of this is probably a symptom of some negative ROI AI deployments.
A couple things I've observed around this. So one thing is, and I come back to this again and again, I think we in Silicon Valley just forget that we live in a bubble. Twitter is a bubble, sorry, X is a bubble. Silicon Valley is a bubble. Software engineering is a bubble. Most people in the world, most people in the US, are not software engineers, are not very AI-pilled, are not following every single model release. And so we're just highly out of the loop on how to use this technology. And so, you know, we always talk about all these best practices for Codex, all these Codex build people within OpenAI. I'm sure everyone on X who posts are crazy power users of these AI tools. You know, they lean into skills, they lean into AGENTS.md,
MCPs.
Yes. Yeah. All of that. And when I talk to some of these companies and I talk to the actual employees using these, it's the most basic thing that they're trying to do, and they have very little understanding of exactly how this technology works. And so that's kind of one big observation for me, which is they're asking very simple questions of these things. They're really not pushing it just yet.
And so that ties into what I think more companies could do, or what a more ideal AI deployment setup looks like. And this is kind of how we've run things within OpenAI too. The companies where I think it's started to work really well have a combination of both top-down buy-in, so it's the C-suite, it's like, you know, we want to become an AI-first company, so there's buy-in, they buy the tools, they have exec support, but it also has bottoms-up adoption and buy-in. And so what I mean by that is it has actual employees doing the work who are really excited about the technology and are willing to learn, evangelize, build best practices, and knowledge share within the organization.
We've seen this a lot internally. So obviously OpenAI has always wanted to be a very AI-centric company, but when it really started taking off was with the introduction of Codex and these tools, where actual employees themselves could start applying it to their work. And I think you really need this because at the end of the day, everyone's work is very different. It's very unique. Software engineering is different than finance, is different than operations, different than go-to-market and sales. And so there's a lot of these last-mile intricacies of work that need to really be done in a bottoms-up fashion.
And so my sense is a lot of these AI deployments don't have bottoms-up adoption. It was an exec mandate, and it's extremely top-down and is very divorced from what the actual work looks like. And as an end result, you end up with a giant workforce that doesn't really understand the technology. It's like, I know I'm supposed to use this, and maybe it's on my performance review too, but I'm not sure what to do. And they look around, no one else is doing it. There's no one else to learn from. And so my recommendation for companies pushing this is: find, or maybe even staff, a full-time team internally, this kind of tiger team internally, that can explore the full extent of the capabilities, apply it to specific workflows, do the knowledge sharing, create excitement within folks who might want to use this technology, because in the absence of that, it's actually very difficult to pick up.
And who would you put on this tiger team? Is it engineer-led? Do you find, in your experience, it's a cross-functional sort of team?
Yeah, it's interesting. So also, a lot of companies don't have software engineers. And so the pattern I've seen is it tends to be these software engineering-adjacent, basically technical people who are not software engineers. I think those are the ones who tend to get most excited around this. It's like, you know, maybe the... It's like maybe the
like, you know, support team operations lead who doesn't code but loves using these tools and is an Excel wizard or something. And so it's technical adjacent or coding adjacent and pretty technical. Those are the kinds of people I've seen in these companies who just really light up and get excited around this, and you can usually build a team around that. But yeah, it's oftentimes not software engineers. Software engineers, I think, will understand this, but not every company has software engineers. It's actually kind of a rarity. They're hard to find. They're expensive. And so it's these other types of folks.
What I'm hearing is the anti-pattern is top down. This is very the CEO, founder, exec team just like, we are going to go AI first. We're going to lean into AI. Everyone's going to be judged on their performance using AI tools, how much your productivity is increasing thanks to AI. And with that being just top down and not creating a team that is bottom up spreading the gospel, you find it doesn't work.
Yeah. Exactly. Exactly.
And the advice is find the people that are most excited and, instead of having them spread out through the organization, what you find works is create a little AI evangelist team that finds ways to use it and spreads it across the work.
Yeah. I mean, hearing you play it back to me, another way to think about it, tying back to my own management philosophies, is find the high performers in AI adoption and empower them. Let them build hackathons, let them hold seminars, do knowledge sharing, create the seeds of excitement internally.
Okay. Amazing. There's a couple hot takes I want to hear from you. Something that I've seen you talk about and share. One is you've shared that talking to customers and listening to customers is not always the right strategy in AI, and it might often lead you astray.
I don't know if it's that hot of a take. I think the main thing here is, obviously you should talk to your customers. It's useful to talk to customers. I just think the AI field, especially what I've seen over the last three years working on the API and seeing all of that evolve, is the field and the models themselves are just changing so quickly they tend to disrupt themselves, especially around the tooling and the scaffolding space.
So there's this quote that I read actually earlier this week from an X article by this guy named Nicolas, who's the founder of a startup called Fintool, where I think he was sharing a lot of the best practices that he has learned through building AI agents for financial services at a startup, Fintool, and he had this phrase that I thought was really good, which is: the models will eat your scaffolding for breakfast.
If you rewind back to 2022, right when ChatGPT launched, these models were pretty raw and there was all this product scaffolding and things, especially in the developer space, to basically try and steer the model and build a scaffolding around it to get it to do what you want, like agent frameworks, vector stores, I think, was really popular back then, and just a whole smattering of tools here. And as you've seen the field play out, the models have just changed so much and gotten so much better that they ended up, yeah, literally eating some of the scaffolding.
And I think this is even true today. So I think the article from Nicolas, actually, the current scaffolding which is fashionable is skills, file-based context management. I could see a world where at some point that's no longer useful, where the model can actually manage all that themselves, or, it's hard to predict, but it might move on to some new paradigm where, you know, need this file-based skills type thing.
You have literally seen this play out, right? The agent frameworks, I think, are a little less useful now. There was a period of time, like 2023, where we thought vector stores is going to be the main way for you to bring organizational context into the models, and you need to vector and embed every bit of your corpuses, and then you need to do all this work to figure out the vector search, to optimize that to fill out the right information at the right time. All of that is scaffolding because the model was not good enough, and it turns out, in this case, as the models get better, a better approach is actually to take out a lot of that logic and trust the model and give it a set of tools for search. It doesn't need to be a vector store. You could actually just hook it up to any type of search. It could literally be files on a file system, like skills and AGENTS.md, to steer it as well. Obviously, there's still a place for vector stores. I know a lot of companies are still using it, but the entire scaffolding around that, and building an entire ecosystem around that, and assuming that's the only scaffolding that you need, has really changed.
And so, tying this back to: you don't always have to listen to your customers, because the field is changing so much. At any point in time, a lot of people are in this local maximum, and if you just blindly listen to your customers, they'll be like, yeah, I want a better vector store, I want a better agent framework for this. And if you had only chased down that path, it actually would have led you to build something that, again, is the local maxima, whereas as the models get better, we've had to reinvent and rethink the right abstractions and the right tools and frameworks to build around these models. And the cool slash exciting slash kind of crazy annoying part is it's a moving target. And so, yeah, the current smattering of tools and frameworks right now will likely need to evolve and change pretty significantly over time as the models get smarter and better. But that is just the nature of building in the space. I think that's what makes it exciting. But it also means when you talk to customers, you kind of need to balance the exact feedback that they want with where you think the models are going and where you think things will trend over the next one to two years.
It's interesting how this is the bitter lesson, this big lesson that AI and ML folks learned, which is just, the less you overcomplicate, the less logic you add to machine learning, to AI, the more it'll be able to scale and grow. Just take it all away and let it just compute, basically just give it more power to get smarter on its own.
Yeah, there's literally a version of the bitter lesson applied to building with AI, where we were trying to architect all this stuff around it and it turns out the models just kind of eat it all away. And honestly, the OpenAI API team has been guilty of this, where we took some left and right turns when we shouldn't have. But yeah, the models get better, and we're all learning the bitter lesson day in and day out.
So what would be the key takeaway for folks building on, say, the API, or just building agents and having to build a little bit of this around it for now? What would be the advice?
My general advice, and I've been giving this to people for a while and I think it's still true today, is make sure you're building for where the models are going and not where they are today. It's clearly a moving target, and I think a lot of the companies, startups that I've seen really, really do well, is they build a product for an ideal type of capability that is maybe 80% of the way there today, and they end up having a product that kind of works but is just almost there. But then as the models get better, suddenly it might click, and then their product now is incredible because it works. Maybe with o3 at some point it suddenly works, with 5.1, 5.2 suddenly it unlocks. But they're building these products with the model capability improvements in mind, and with that you end up creating an experience that's way better than if you had assumed that it's static in the first place. And so that would be my general advice, which is build for where the models are going and not where they are today. You end up building a better product. You may need to wait a little bit, but the models are getting so much better so quickly, you often don't need to wait that long.
So to follow that thread, in the next six to 12 months, where is the API heading? Where's the platform heading? Where are the models heading? As much as you can share, I know there's a lot of secrets here, that maybe you're most excited about or you think that people should start to prepare for, however much you can share.
I mean, so the obvious one is how long of a task these models can do coherently. So there's the METR benchmark that I think tracks software engineering tasks and how long of a task these models can do 50% of the time, 80% of the time. I think we're at something like multi-hour software engineering tasks being able to be done by these frontier models 50% of the time, and then I think 80% is something like just under an hour. But the sobering thing about that chart is they plot all the previous models on this chart as well, so you can really see the trend of this.
That's something that I'm really excited about, which is, I actually think products today really optimize for tasks that the model can do for minutes at a time. Even Codex and the coding tools, I'd say, in the CLI you're kind of seeing it be interactive. It's really quite well optimized for maybe at most 10-minute types. I have seen people push Codex to the limit and do multi-hour long tasks, but again, I think that's more of the exception. But if you follow this trend, I think in the next 12 to 18 months we could see models that could do multi-hour long tasks very, very coherently. At some point it might reach 6 hours, a day-long task, where you kind of dispatch it and have it do things on its own for a while. The types of products you build around that will look very different. You want to give the model feedback. You obviously don't want it to completely run wild for a day. Maybe you do, but you probably don't. And then the universe of things you can have the model do really expands. So that's something that I'm really excited about seeing.
Another thing over the next 12 to 18 months I think will be really cool is improvements in the multimodal models. And actually, by multimodality I'm mostly thinking about audio here, where the models are pretty good at audio. I think they're going to get a lot better at audio over the next 6 to 12 months, especially the native multimodal models, the speech-to-speech ones. I think there's also interesting work being done around new types of models and architectures on the multimodal audio side as well. But audio, especially in the enterprise and in a business setting, I think is a hugely underrated domain still. Everyone talks about coding, it's all text, but we're talking in audio. A lot of the world's business is done via audio. A lot of services and operations are done via talking and audio. And so I think that area is going to look very exciting in the next 12 to 18 months, and I think there will be even more unlock for what we can do with audio models there as well.
Amazing. So, quick summary: expect agents and AI tools to run longer, that trajectory to continue to increase, and then audio and speech becoming a bigger deal, more first party and native and better and core to the experience.
Yeah.
Extremely cool. Okay, I want to go back to one of your hot takes, another hot take that I've seen you discuss. You're very bullish on business process automation as an opportunity in the world of AI. Talk about that.
Yeah, this goes back to the thing that I said previously, which is we live in a bubble in Silicon Valley, and a lot of the work that we do, that we're used to, software engineering, product management, building products, is very differently shaped than the work that goes on that runs our entire economy. And I see this in and out when I talk to customers. If you talk to any company that's not a tech company, there's a lot of business processes.
And so what I mean by this is, I generally delineate it as: software engineering is kind of open-ended knowledge work, right? And this is why I think tools like Codex tend to be quite good, because it's exploring and you're giving it these open-ended things. But software engineering is fundamentally pretty open-ended and it's not very repeatable, right? So you build a feature, you're not trying to build the exact same feature over and over again. And a lot of tech jobs are in this space. I think data science is kind of in the space as well. Even some of the strategic finance stuff. But as you move further and further away from software engineering and what is core in tech, a lot of jobs are just business processes. They're repeatable things, repeatable operations that some manager at a company has iterated on. There's usually a standard operating procedure that people want to do, and you don't want to deviate from it that much. In software engineering the ingenuity is in deviating, but a lot of the work being done in the world is actually just running through these procedures and operations. If I call a support line, they're running through one of these. If I call my utility company, there's a bunch of processes and things that they can and cannot do for me.
And so I'm just extremely bullish on this general category, and I think it's underrated because it's so different from what we think about in Silicon Valley; people tend to not think about it. But how can we apply AI and some of the tools and frameworks that we have towards this business process automation, towards automating and making easier repeatable business processes with high determinism, that is fully integrated with business data and business decisions and different systems within an enterprise, and how can we actually make that process better? Because I actually think there's a lot of opportunity and a lot of work to be done in that area, and we just don't talk about it because it's a little bit less in our wheelhouse.
So your take here, just to make sure I fully understand it, is you think there's a much bigger opportunity outside of engineering for AI to impact productivity of companies, and also jobs of these folks that are doing these kinds of repetitive, easily automated tasks.
Impact jobs, and also just impact how work is done. So much of work is done in this way. You think about, basically, I talk to customers all the time, big enterprises, like how
How will AI transform my company, like how will it run in a world with AI in like 20 years? And you know, software engineering is part of the story, but there's so much more on the business process side. And I actually think it might look even more different on the business process side. And the work there is pretty substantial.
It's actually interesting. I don't know, from an absolute percentage or absolute basis, I don't know if it's bigger or smaller than software engineering. Software is pretty huge and pretty extensive as well. But it is pretty massive and it's definitely bigger than you would think it is based off of how people talk about it or don't talk about it on X or Twitter.
Okay. Going in a slightly different direction, having built the platform, building the API, people building on the API, the biggest question on people's minds is always just how do I not have OpenAI squash my idea and build their own thing and then destroy this market I created. What's the general policy? What's the general philosophy of how startups should think about where OpenAI is unlikely to go?
My general answer here is the market is so big and so massive, I actually think startups should just not overly think about where OpenAI or these labs are going. I've talked to a lot of startups that have not worked out, startups that are doing really well. Every startup that I've seen that has kind of fizzled out is not because OpenAI or a big lab or Google or something has come to squash them. It's because they built something and it really didn't resonate with the customers. Whereas the ones that take off, even in very competitive spaces like coding, like Cursor is huge at this point, and it's because they built something that people really love. And so my general advice is don't overly stress about this. Just build something that people like and you will have a space in this.
I can't overstate how big of an opportunity there is right now. The opportunity space of building with AI is so big. A good example of this is the space is so big that the Overton window of what is acceptable and not acceptable for VCs to do has completely changed here. VCs are investing in competitive companies left and right, just because the space is so big, because the opportunity is unlike anything that we've seen before. And while that affects how VCs operate, from a startup perspective it's the most empowering thing in the world, because even if you just build something that some people really, really love, you will end up with a massively valuable business. And so that's why I tell people don't overthink about it.
The other thing I also think is important to remember, at least from an OpenAI perspective: one thing that we've always held very near and dear, which both Sam and Greg helped reinforce from the top as well, is we actually view ourselves fundamentally as an ecosystem platform company. The API was our first product. We think it's really important for us to foster this ecosystem and continue to support it and not squash it. And so if you look at the decisions we make, this is all woven through it. Every single model we've released in one of our products gets released in the API. Even, you know, we release these Codex models now that are a little bit more optimized for the Codex harness, but they always find their way into the API and all of our customers end up using those. We don't hold back on any of that. We think it's really important to keep our platform neutral, and so we don't block competitors. We allow people to have access to our models. We've also recently been testing more of the Sign in with ChatGPT product as well. And so we want to foster this ecosystem and I think it's really important that we do so.
The general thinking about this is a rising tide lifts all boats, and we might be an aircraft carrier, we're pretty big at this point, but we think it's important to raise the tide because everyone benefits, and I think we'll benefit as well. Our API itself has grown pretty significantly because we act in this way. And so I'd really encourage people not to view OpenAI as this kind of thing that'll just shove people out of the way, but instead focus on building something valuable, and we remain committed to providing an open ecosystem.
Why is that important to OpenAI? Just this focus on building a platform, creating a way for people to build businesses. Is that just, that's been the vision from the beginning, we want this to be a platform?
It's been the vision from the beginning. It goes back to our charter actually, our mission. So the OpenAI mission has always been, one, to build AGI. So we're obviously doing that. But then the second thing is to spread the benefits of it to all of humanity. And the main part there is all of humanity. And obviously ChatGPT is trying to do this; we're trying to reach the whole world. But very early on, and this is why we launched the API back in, I think it was like 2020 or something, really early, we don't think we as a company will be able to reach all of humanity, right? Every corner of the world is pretty deep. And so we actually feel like in order for us to fulfill our mission, we need to have some platform-style thing here where we can empower other people to build, you know, the customer support bot for podcasters and newsletter hosts, because we're not going to be able to do it ourselves. And so we've largely seen this play out with the API. This is why we talk to so many of our customers and really love seeing the diversity of things built on it. But yeah, it's been there since day one because we view it as an expression of our mission.
And you haven't even mentioned the app store that you guys are launching, the ChatGPT app store. Is that under your umbrella, by the way, or is that a different org or team?
It's a different team. So it's under ChatGPT. We obviously collaborate very closely with them, and they built an Apps SDK, which is built in close collaboration with our team. But that is more within the ChatGPT umbrella. But that's another example of this, right? With ChatGPT, we have these 800 million weekly active users who are just coming over and over again. It's a great asset to have as a business, but man, would it be better if we could somehow allow other companies to come in and take advantage of this as well and build for this audience as well. And then ultimately we think it'll help us expand that group as well, right? And so it all comes back to the mission, and we find that being a platform, being open, tends to help here.
Just that number, 800 million, I think it's... just like weekly, weekly active, a billion people using it weekly. It's absurd how these numbers, we're just used to them now, but that's insane, unprecedented.
Yeah, it's mind-boggling for me to think about from a scale perspective, honestly. And the way I think about it is like 10% of the world, and growing, by the way, it's shooting up, comes to ChatGPT and uses it every day, or sorry, every week.
And this point, I just want to double down on this point you're making. OpenAI's mission was to make AI available to all humanity. And I think some people diss that. They're like, oh, it costs money. And the fact that there's a free version of ChatGPT that anybody can use that is not so different from the most powerful AI model that exists in the world, for free, that's not gated, that anyone could use. If you're a billionaire, there's only so much more you can get out of AI than what someone in a village in Africa can get. And I know that's always been really important to OpenAI.
Yeah. I mean, that's why I think we've leaned into the health work. We've leaned into education; it's going to be very interesting here. The other insane trend here is the free model has gotten so smart over time. The free model back in 2022 was good at the time, but it's nothing compared to what you get today, because you get GPT-5 today. And so raising the floor across the world is something that we're really trying to do, and we view it as part of our mission. The other flip side of this, by the way, talking about the billionaires or whatever, I know people have been saying you're using the same iPhone that, you know, Steve, or sorry, Mark Zuckerberg's probably using, or the billionaires are using. But for $20 a month you're basically using the same AI that the billionaires are using. For $200 a month you get the same Pro model that all the billionaires are using, but they're probably not using Pro for everything. They're probably just using the Plus tier ones for their day in and day out. And so yeah, this kind of democratization and just spreading of this benefit across all of the world is something that's really meaningful to us and something that drives a lot of what we do.
One last question, just for folks that are thinking about building on the API, or just like, oh wait, I could do cool stuff with OpenAI models and APIs. What does your API and platform allow people to do? I know you can build agents on top of the platform. Just talk about what you allow.
So fundamentally the API offers a bunch of developer endpoints, and these developer endpoints basically let you sample from our models. The most popular one that we have right now is one called the Responses API. And so this is an endpoint and it's optimized for building long-running agents, so agents that'll work for a while. At a very low level, you're basically just giving the model text. The model will work for a while. You can poll it to see what it'll do, and then you'll get the model response back at some point. That's the lowest-level primitive that we have for people, and that's actually what a lot of people use. That's the most popular way of building on top of the API. With that, it is super unopinionated and you can do basically whatever you want. It's the lowest-level thing.
We've also started building more and more layers of abstraction on top to help people build some of these. And so the next layer up, we have this thing called the Agents SDK, which has also gotten extremely popular. This allows you to use the Responses API or some other API endpoints that we have to build what you might more traditionally think of as an agent, like an AI working in an infinite loop. It might have subagents that it delegates to. It starts building all this framework, all this scaffolding actually. We'll see where this all goes. But it makes it a lot easier for you to build these kinds of agents, giving it guardrails, allowing it to farm out subtasks to other agents and kind of orchestrate a swarm of agents. The Agents SDK allows you to do that.
And then above that, we've now started building tools to help also with the meta level of deploying an agent. So we have this product called AgentKit and widgets, which are basically a bunch of UI components that you can use to very easily build a very beautiful UI on top of either our API or the Agents SDK, because a lot of times these agents look very similar from a UI perspective. And so there's AgentKit. We also have a smattering of eval products, like an Evals API, where if you want to test and see if your models or your agent or your workflow is working, you can test it in a very quantitative way using our evals product. And so yeah, I view it as these various layers. They're all helping you build what you want with our AI, with our models, with increasing levels of abstraction and how opinionated it is. And so you can use the whole stack, and it very quickly allows you to build an agent, or you can go down the stack as low as you want, to basically the Responses API, and build whatever you want because of how low level it is.
Sherwin, is there anything else that you want to share? Anything else you want to leave listeners with? Anything we haven't touched on that you think might be helpful before we get to our very exciting lightning round?
The only thing I'd leave folks with is, yeah, I think the next two to three years are going to be some of the most fun in tech and in the startup world that we'll have in a very long time. And I would just encourage people to not take it for granted. I entered the workforce in 2014. It was great for a couple years. I felt like there was a period of five to six years where it wasn't very exciting in tech. And then the last three years have just been the most insanely exciting, energizing period of my career, and I think the next two to three years are going to be a continuation of that. And so I would encourage people not to take it for granted. I'm trying to not take it for granted. At some point this wave is going to play out and it's going to be a lot more incremental. But in the meantime we're going to get to explore a lot of really cool things, invent a lot of new things, and change the world and change how we work. And so that's the main thing I'd leave folks with.
I love this message. I want to spend a little more time on it. When you say don't miss it, what do you recommend people do? Is it just build, lean in, learn, join a company building really interesting things? What's your advice to folks that are like, "Okay, I don't want to miss the boat"?
Yeah, I would just say engage with it. So it's basically what you said. Lean in. Building tools on top of this is part of the story. Just using the tools, you don't need to be a software engineer to lean into this. I think a lot of jobs are going to change here. So just using the tools, understanding the limitations of what it can and cannot do, so that you can watch the trend of what it can start to do as the models improve. And so it's basically getting used to the technology and getting familiar with it, instead of laying back and letting it pass you.
On the flip side of that, there's a lot of, I think, stress and just anxiety around, there's so much happening. How do I keep up? I've got to learn Clawdbot this week. Oh god. Is there something you've learned about just not... like, you're at the center of this. How do you not get overly stressed and worried about missing things that are going on, and just stay on top of news? What are some things you've done, learned?
Yeah, so I think I'm personally a bad example of this because I am, I'm
basically chronically online on X and our company Slack. So I actually try and absorb, I end up absorbing a lot of it. What I will say though is just like from observing other folks who are less, you know, addicted to this stuff like I am. Yeah, a lot of it is noise. Like you don't need to have like 110% of this kind of pass your mind, like go into your mind. Honestly, just leaning into like one or two different tools, starting small is already like, you know, more than you need here.
I think just the combination of like the frenetic pace of the industry, X as a product just creates like this insane kind of like, yeah, this insane like pace of news, which is honestly very overwhelming. The main thing is like you don't need to know all of that to really engage with what's happening right now, and even something as simple as just like install the Codex client and play around with it. Install ChatGPT, connect it to a couple of your, you know, internal data sources, Notion, Slack, GitHub, and see what it can and cannot do. All of that I think is a part of it.
Amazing. Sherwin, with that we've reached our very exciting lightning round. I've got five questions for you. Are you ready?
Yeah. Yeah, absolutely.
First question, what are two or three books that you find yourself recommending most to other people?
Oh, I'll talk about one nonfiction one and one fiction book. The fiction book was, I just finished reading it. I really recommend it. It's There Is No Antimemetics Division by qntm. I think it's like an online author, but I saw it being shared on X. It's like a science fictiony kind of book. And I basically devoured it in like two days. It's super, super well written, super fascinating. It's about a government agency that's fighting, you know, things that make you forget it. And so it's just a very like smart, like creative book, and fresh honestly in terms of like source material, that I really like. So I'd recommend that one. The book is also unintentionally hilarious. So like it's meant to be like this sci-fi, almost like horror style book, but it made me laugh a couple times. So that's the fiction book.
Nonfiction. So I'm going to cheat and I'm going to recommend two of them. So in the last year I've been reading a lot more about China and kind of like the US-China relations. And I think there are two books that came out in the last year that have been, you know, really, really eye-opening for me in that regard. First one is the Dan Wang book Breakneck. That one was really, really good. I really liked his analogy of like the US is the lawyerly society, China is the engineering society, and there are pros and cons to each. I read it and I was like, hm, yeah, does seem like we're run by lawyers in the US. So that's one.
And the other one is the Patrick McGee book on Apple and China, which was super, super interesting. I'm a huge Apple fanboy. Like if you could see my desk right now, it's all Apple stuff. But just like one, it was just super fascinating learning about Apple's relationship to China. And then two, it just like had a lot of inside information about Apple as a company that I found fascinating. So it was also quite a page-turner and also, you know, a very timely book as well.
The antimemetics book sounds amazing. I'm buying it right now as you're talking.
Yeah. It's like, I think it's only like a couple hundred pages. I literally finished it in two days. It was just like so, so good.
Okay, great tip. Okay. Favorite recent movie or TV show you have really enjoyed?
Yeah, that one's tough because, you know, I have two kids and a busy job, and so I really haven't had much time to watch TV shows. I will say in the last couple weeks I watched a couple episodes. I'm actually a big anime guy, and so I watched a couple episodes. There's a new season of this anime called Jujutsu Kaisen that's out. So season 3 of JJK was really good. In general, I'm a huge fan of Japanese anime. I think they create the most novel and unique plots and universes that Western media has shied away from. And so generally a big fan of that, but yeah, I haven't really watched much, but saw a couple episodes of JJK recently.
Extremely understandable in your role.
Yeah.
Favorite product you recently discovered that you really love?
Yeah. Okay. So I recently had to set up Wi-Fi and like home networking, and I went all in on Ubiquiti routers and security cameras. I had never heard of it before I had to do this. I always just had a very simple setup. And it is just such a well-built product. I don't know if you've used it before, but it's basically like the Apple of home networking. So, beautiful products. But the thing that actually makes it extremely good is its software is good. And so they have a really great mobile app to help manage, you know, all of the home networking. And so basically with Ubiquiti, you can use it to buy wireless routers. You need Ethernet wiring throughout your house to use it. But I actually think what makes it really good are security cameras. So if you have security cameras that are plugged into the Ubiquiti ecosystem, they have an incredible mobile app, and Apple TV app and iPad app, to kind of see the live feed of your cameras. And so they're a little pricey, but not that pricey. But it's been just an incredible product experience.
All right, I went eero, so I made a mistake. Good tip.
eeros are pretty good, too. But fully converted to Ubiquiti at this point.
Good tip. Okay, two more questions. Do you have a favorite life motto that you find yourself coming back to in work or in life?
Yeah. The one that I always, you know, repeat to myself is: never feel sorry for yourself. There's a lot of things that are going to happen, you know, at work, in life. And reminding yourself to never feel sorry and that you always have a sense of agency to kind of pull yourself up is something that I've had to tell myself a lot, and also something that I repeat to a lot of other folks as well.
Last question. So, in your previous life, you worked at Opendoor, where you led work on basically figuring out how much to pay for houses. You basically built a model that told the company, "Here's how much we'll pay for this house." What's like a variable in the price of a house that you didn't expect is really important and impacts the price of a house?
There's a bunch that were surprising. I'll maybe list the couple of most interesting ones. Power lines, like high-voltage power lines, actually impact your price quite a lot. I didn't really fully internalize this until I went to like Dallas and observed, like, when your house sits next to one of these giant, you know, voltage lines, it's like buzzing, and most people have families, you don't want your kids kind of near there. So I think that was one that really, really kind of surprised me.
That makes sense.
Yeah. And then the other one, which was always something really difficult for us to quantify, was floor plans. And so it is very important, like yes, of course it's really important, but just like quantifying what a good floor plan is like and what a really bad floor plan is like. We were doing all these things of like how wide is the kitchen, and like what style of kitchen is it, and then like where's the master bedroom. And so it was just really, really hard to quantify, but I remember floor plan was a big one because like we'd have a home that like wouldn't sell, and then our ops team would go in and be like, yeah, it's a floor plan issue. So like, how could you tell? It's like you go inside, you just feel it, you know, the floor plan feels
So yeah, those are ones that were surprising. And then the last one that was more impactful than I thought is general curb appeal and like even like the front door. And so I actually think there's a Zillow book on this where the front door replacement tends to be the highest ROI for homes. But just like the feel of, as you walk up to the home as a buyer, what you're interacting with and the first moments of the house, I think I'd underrated its importance.
That is extremely interesting. And I love that you had to figure out how to do all this in code and not walk
floor plans. I have a bunch of stories around like, for floor plans, it's not digitized. So there's like a handful of people who have like paper floor plans of like all these homes in like Phoenix and Dallas. Yeah, a lot of fun stories from the Opendoor days.
Okay, Sherwin, thank you so much for doing this. This was incredible. Where can folks find you online, and how can listeners be useful to you?
Yeah. So, I'm online on Twitter, on X. I'm just Sherwin Wu, and yeah, I mostly just tweet about OpenAI and the API and some of the products that we're launching. And then how folks can be useful to me: I love hearing about things that people are building, and so if you're working on a startup, if you're hacking on an idea, you know, would love for you to just reach out to me on X. I would love to hear about what you're building and learn about how OpenAI can help support you.
Amazing. Sherwin, thank you so much for being here.
Yeah. Thank you, Lenny.
Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
Article published · Updated