Netflix's Elizabeth Stone on Systems Thinking, Craft, and Excellence in the AI Era
Lenny's Podcast
Elizabeth Stone, Chief Product and Technology Officer at Netflix, returned to Lenny's Podcast two and a half years after a first appearance that came before generative AI had reshaped how product teams work. Stone now oversees engineering, product, and design; earlier roles include VP of Science at Lyft, COO at Nuna, economist at Analysis Group, and trader at Merrill Lynch. Host Lenny Rachitsky's main question was what happens to roles, skills, and company culture when "everyone can be everything." Stone's answer holds two positions together. AI is making the boundaries between functions more fluid, and she welcomes that. But craft expertise still matters, accountability still sits with humans, and the skill Netflix most needs now is systems thinking rather than narrow specialization.
Role confusion: the "storming" phase
Rachitsky opened with a frustration he hears widely. PMs can ship code, designers can write PRDs, and engineers can do product work, which leaves people asking what their job is. Stone said she hears this inside Netflix too. Her explanation is that any transformative technology brings "a storming phase before you go through the forming phase," and generative AI is in the middle of that storming.
She does not think the answer is to "put AI back into the box." Instead, organizations should be deliberate about getting the benefits while limiting the costs. She considers it healthy that people are experimenting with developing, prototyping, and testing ideas faster. She is skeptical that this means anyone should ship code to production, or that everyone should do everything. Because product and technology sit together at Netflix, she thinks some fluidity is fine when the business problem is clear. Product and design no longer have to wait for engineering to prototype something. They should still work with their engineering partners on how to productize and scale an idea and what guardrails it needs.
Stone named several organizational conditions that make this fluidity productive rather than frustrating:
- Clarity on source-of-truth data.
- Guardrails on shipping code to production, and testing before large changes.
- Deciding where AI output can be trusted and where a review process is needed to check quality.
- A repeated message that humans remain responsible for outcomes.
On the last point she was explicit. An agent may have written the code, or AI may have helped someone run an analysis outside their background, but that "doesn't make people not have the responsibility that comes with what they've created." Investing in core infrastructure and practices, and restating accountability, is how she balances what is possible against what the team should actually do.
What has changed in each function
Asked which roles have changed most since the last conversation, Stone said PMs, designers, and data scientists can now get further through the product development life cycle before engineering has to lead. She added a caution: thousands of prototypes are not valuable unless they target an important business problem, engineering partners know that problem is being worked on, and the work isn't "throwing a bunch of spaghetti at the wall." When the problem is right and there is alignment, she has seen product, design, and data science move faster toward something testable, including by prototyping and writing code.
The second change she highlighted is access to institutional knowledge. Netflix has decades of experiments, consumer insights, and stakeholder input spread around the company. Using that history to speed up current decisions used to be hard. AI is strong at distilling information across many sources and producing a starting set of insights. Stone said she wouldn't rely on this exclusively, but it gives a head start. In her own work, instead of emailing someone to ask what research was done in what year and what was tested, she can find it almost instantly and form her own view of what's actionable. This capability is no longer limited to people who have been at Netflix for 20 years. It spreads across product and tech functions, and she called it a big unlock that business stakeholders in finance, content, and advertising can do it too, then bring a hypothesis to data scientists and engineers for deeper work.
Stone still sees comparative strengths in each function. Data scientists remain the experts on whether data can be trusted, whether it is being interpreted correctly, and how to weigh data against judgment. Product managers remain the best at framing the "what," meaning the problem being solved. Engineers own a craft around the "how": scaling, what high quality looks like, and what problems a given build and deployment approach will create. What has changed is that people can move more fluidly through steps that used to be blocked.
Why functions will survive
Rachitsky raised the "member of technical staff" trend, where titles disappear because anyone can do anything, and asked whether Stone expects separate disciplines to persist. She said yes. She sees "a craft excellence that's really important in the disciplines" that isn't going away soon, even as work blurs across functional lines. Engineers, data scientists, and designers "speak more languages now" thanks to AI tools. But she finds something irreplaceable in how they think about what good looks like, and that holds at every level. She also said great engineering, great data science, and great creativity all remain scarce. Some things have become easier, but in her view that scarcity hasn't dissolved.
Hiring more systems thinkers
Asked which functions Netflix is hiring more or less of, Stone said the answer doesn't map neatly onto functions. What Netflix needs more of is systems thinkers, and she gave examples from different functions.
In central engineering, much of Netflix's past success came from local teams moving fast on specific business problems. They often built whatever stack they needed rather than following a central "paved path." In a world where agents operate across multiple systems and need source-of-truth data, Stone said, shared infrastructure matters more: preferred paved paths, common guardrails, and solving problems once with a core set of capabilities. Netflix is therefore hiring more people who can look across business domains and abstract them into "here's the building blocks we're going to need." She framed this partly as "what got Netflix here doesn't get Netflix there." Engineering profiles are shifting toward distributed systems and infrastructure. She called this additive, since Netflix still has deep experts in personalization, advertising, and content delivery.
In design, the experience design team needs to build templates and systems that define great user design at Netflix. That lets many people, including those without design training, build products that fit coherently into the end-to-end member experience. Stone said she gets "really nervous" about different design languages or interaction patterns leading to shipping "Frankensteins." More of the design team now works on design systems, brand expression, and what makes Netflix distinctive, rather than designing individual features.
She described this stepping back to see the big picture as happening in every function. It requires reorienting existing skills and also hiring for them. Across all of it she stressed mindset: Netflix is not hiring people who aren't excited to explore, comfortable with ambiguity, and comfortable with blurred ways of working. She said that curiosity and innovation mindset has never been more important in her career.
Why systems thinking matters more now
Rachitsky asked whether this is mainly about speed. Stone said velocity is probably part of it. Platforms offer leverage by getting most teams "80% of the way there" so they don't rebuild the same building blocks. Netflix is placing more bets and building more things, and she said treating platforms as a critical enabler is relatively recent for Netflix.
The second reason is scaffolding. More people are doing new kinds of work, which raises risks around access, identity, security, and the quality of code, design, and user experience. She doesn't think it scales for every builder to separately ask what good looks like and what the guardrails are. Those answers need to be encoded in paved paths and ways of working. For data work, that means encoding the source of truth, how to interpret and access it, and which data needs special care. An organization with thousands of people, she said, can no longer rely on tribal knowledge or on finding "the one person who knows this." The problem existed before AI but is more urgent now, and she likes using a new technology as motivation to do work the team already knew it needed.
Rachitsky suggested agents are another reason: giving them context, scaffolding, and design language speeds up their work. Stone agreed and described a Netflix vision of so many agents contributing that the organization must be able to reason about all of it. Humans decide which problems to solve and judge whether the output is impactful and high quality. The work itself will be done by both humans and agents, which brings velocity and also risk. She said engineering in particular has to let people move fast without creating undue downside for the company.
Is the design process dead?
Rachitsky brought up his earlier conversation with Jenny Wen, head of design for Claude Code and Cowork, who argued that the traditional design process is dead. In that view, designers mostly steer, adjust, and think about the big picture when there's time. He asked whether Stone's platform-first approach points the same way.
Stone said she has mixed feelings. She does want infrastructure and systems thinking to let more people do work with strong design built in. But for Netflix's most important priorities, she called design "critical to solve things in the right way," and Netflix still makes time for important design work. That work moves faster now, because designers have more tools and can show more options and iterate and test more quickly. She thinks it would be a mistake to let deep design expertise get squeezed out just because code and analysis are faster. For a large consumer product like Netflix, she said, that would lose something that makes Netflix great: product, technology, and design together make complexity invisible and create a seamless experience. The work may look different, but she doesn't think the mindset goes away.
What's trending down: narrow, deep specialization
Stone said the days of "very narrow, deep specialization feel more limited." There are still exceptions where only a few people in the world understand a technology. She cited Netflix's encoding and playback systems as areas where specialists remain necessary. As a general rule, though, she believes Netflix has fewer specialists than it did 5 or 10 years ago and more generalists who can adapt in several directions. That can mean moving across functions or across engineering areas, such as working on both back-end and front-end systems or connecting to infrastructure with real expertise.
Her reasoning is that specialists can now pick up a broader set of tools more easily, so fewer pure specialists are needed because talent can grow into breadth. She also said a strong attachment to a narrow specialty raises a concern for her about whether someone wants to grow and explore. She acknowledged she doesn't want to be too narrow in that judgment herself.
Asked what counts as a specialist, she gave both layer-of-the-stack examples (front end vs. back end) and domain examples: a payments expert, an ads marketplace design expert, an expert in specific studio production tooling. Domain expertise remains an advantage if the person is willing to ask whether the current tool or approach is still right. She sees specialization declining mainly across layers of the engineering stack, and she wants people working on tools with a lot of inertia to be the kind who can imagine the future version.
How to develop systems thinking
Rachitsky mentioned the book Thinking in Systems and asked how people build the skill. Stone offered what she called a small trick: for each problem, "step out one click" and ask what you are assuming about the broader space. If you're asked to build a new feature for the member experience, pause and ask what larger consumer problem it serves, what content types it needs to support, whether your planned build will scale across content types, whether it could become a platform capability for other areas, and whether the consumer problem is among the most important ones Netflix faces as entertainment becomes broader, more personalized, and more immersive.
She stressed that this doesn't mean solving Netflix's whole strategy. You take what you own and zoom out once. She also warned against staying in the questioning phase too long, because then you stop making progress.
Rachitsky suggested thinking about how your manager would see the problem. Stone said she has received similar advice over the years: find ways to do your job that help your manager do theirs. Looking at your responsibilities from your manager's view, which spans finance, content, and other parts of the business, naturally makes you zoom out and think about how the parts fit together. For engineers, she added the question of how to leave systems better than you found them. That connects to a Netflix engineering principle of doing what's right for the broader organization rather than only for your local team. So in her view systems thinking isn't just about seniority. It's about breadth: whether the way you build something helps colleagues and leaves a stronger base for future work.
AI fluency as a universal overlay
Rachitsky noted that Netflix recently introduced career ladders and levels and asked how AI shows up in them. Stone said Netflix has deliberately not tried to specify how AI changes expectations at each level. Instead, it applies an "overlay" across all talent, both current staff and hiring, that sets an aspiration for AI fluency. What that looks like varies by function, career stage, level, and type of role.
She admitted AI fluency is hard to define. It might mean an experimentation mindset, knowing where AI is and isn't useful, or having actually built things with AI. Because the technology moves so fast, she said the way this appears in ladders changes "almost by the quarter, if not month or day." So the more useful approach is a shared expectation for everyone. That doesn't mean using AI for its own sake. It means good judgment about where it helps, plus openness to exploring. She called this non-negotiable for every role, including the most senior leaders at Netflix, who are also expected to have deep fluency even if they don't write code.
The change also shows up in hiring. Interviews now probe how candidates think about AI and technology, what they use day to day, and how comfortable they are with change. Coding interviews allow candidates to use AI tools, because the job now involves them. Stone said Netflix is "right in the middle of" this shift, not done with it.
AI use cases beyond coding
Asked about impactful uses beyond coding and prototyping, Stone named two. The first is data analysis, information distillation, and modeling: finding past experiments, the right metrics for a problem, and prior consumer research. She said this is faster and higher quality, provided you check the results and confirm with your local data scientist that you're using source-of-truth data. She said it is the use she personally relies on most.
The second is content production. Stone noted that ML and AI were already deeply used in production tools, including creating promotional assets at scale and localizing through subtitles and dubs. She sees generative AI as a big step up in creative ideation, for example in pre-visualization, which brings a creator's vision to life before a production goes to set. There are many post-production uses as well. She pointed to Netflix's recent acquisition of InterPositive, a company started by Ben Affleck that built models and capabilities for relighting, reframing, reshooting, and changing dialogue after filming. She emphasized that these remain led by the filmmaker deciding to try something different. She said many productions are using different tools, some built in-house and some from vendors Netflix enables. Downstream, AI helps produce trailers, images, and artwork at scale so titles find their audiences globally. The same tools apply to advertising, marketing, and off-service campaigns, which Netflix is exploring.
Netflix's long history with AI and ML
Rachitsky recalled the Netflix Prize, which in his recollection was a roughly million-dollar contest to improve Netflix's ranking algorithm, won with an improvement of a few percentage points. Stone used it to make the point that AI is not new to Netflix, especially in personalization. As the catalog grows beyond film and TV into games, live events, and podcasts, matching the right title to the right person at the right moment gets harder and more important. Netflix's history gives it "a running head start" in understanding that problem, even as the technology becomes more powerful.
The same applies on the creative side, where AI and ML have long been used in visual effects and language localization. In both areas, she said, AI builds on Netflix's strength of combining entertainment and technology. Joking about terminology, she noted that what used to be called machine learning is now all called AI, and that much of Netflix's use is not generative. For people who want to work on tech applied to creative work, consumer products, or infrastructure, she said Netflix has all of those problems with AI at the center, "even if Netflix isn't branded as an AI company."
Excellence as an operating system
Rachitsky observed that Netflix's early culture of high agency, autonomy, talent density, bottom-up thinking, fast experimentation, and top-of-market pay closely matches how top AI labs describe themselves now. Stone called Netflix "a little prescient" and described those cultural elements as "excellence as an operating system." Agency and minimal process were never goals in themselves. They came from a strong belief that excellence comes from giving people agency and accountability, pushing decisions as deep into the organization as possible, and hiring people whose judgment can be trusted. When people know they carry that trust, she said, they want to do their best work, and outcomes and motivation both improve. She added that culture isn't static and must evolve as the company grows and its problems change, but the core commitment to excellence and trusting exceptional talent hasn't changed.
Asked for the pillars, Stone said talent density is the non-negotiable starting point. Without it, you can't trust decisions at every level or let people take risks and innovate quickly. Comfort with risk is another pillar: "We don't try to avoid failures, we try to recover quickly when we have them." She cited Netflix's move into live events as an example of taking a lot of risk while knowing it would be imperfect, and said she has never been prouder of the team than watching them work through it. She also named selflessness: decisions should serve Netflix and its members, not personal success or preference.
She then described practices she called "really unnatural for humans." One is not stepping in when she would have decided differently. If a decision isn't material and won't "burn the place down," she lets people make it and asks afterward how it went, acknowledging that she may have been wrong. She has felt this from her own boss and peers: "This is your decision... It's yours in the end." She said it is hard to do when stakes are high and she feels responsible for the organization.
Another is not assuming process will fix problems. Stone said she has never heard anyone claim to have found the perfect way to plan, or to handle feedback, leveling, and compensation. Whenever Netflix responded to difficulty by adding process, it spent more time without getting better outcomes. The instinct to simplify hard problems with constraints works against finding more creative ways to plan and prioritize, so she described resisting what bigger companies usually do and staying comfortable with that discomfort. Rachitsky described the familiar reflex of adding a process after every mistake. Stone said the best people want a blameless retro and feel responsible enough to ask how to prevent a repeat, by sharing learnings and changing how they work rather than through new gates. Leaders, in her view, are trying to grow a resilient, durable team, "not trying to control everything."
The keeper test, mostly as a positive conversation
On the keeper test, Stone said it is often cited only as the moment someone is let go, but it is just as often used to tell people how well they're doing. The entry point is a question, from her to a direct report or from them to her: "How am I doing on your keeper test?" Most of the time, she said, her answer is "I would fight so hard to keep you," followed by specifics on strengths, impact, and how to get even better. The harder version is telling someone they aren't passing right now. Sometimes that comes with feedback and milestones; sometimes it means saying the path to success isn't there. She values it as "good hygiene on feedback" that forces hard conversations and also makes sure strong people feel recognized.
Rachitsky summarized it as asking whether you would hire someone today knowing what you know. Stone added the corollary: if that person said they were leaving, would you fight to keep them? If your reaction is relief, you should have had the conversation sooner. She listed it alongside other practices that people and large organizations don't do naturally, such as maintaining talent density, "context not control," and being "highly aligned but loosely coupled" with only the minimum process needed for clarity on priorities. Maintaining these takes constant diligence, and she said the work and culture are what attract and retain people.
Competing for talent with frontier labs
Asked how Netflix competes with AI labs offering enormous pay, Stone said Netflix has always faced competition for talent and she doesn't feel other companies are "vacuuming up all the good people," because many of them are at Netflix. What has changed is that Netflix has to be more explicit about who thrives there. Those people are passionate about applying technology and building products: they love entertainment, consumer products at scale, and global reach, and they want to use AI and other technologies in that setting. People drawn to the foundational work of frontier model companies are, in her view, a different persona interested in a different problem space. She doesn't think there is a shortage of people excited about applying technology to things they use every day.
Junior talent and mastering the craft
Rachitsky asked how junior people become senior when AI does so much of the work. Stone said Netflix still hires junior talent and considers it important. It has an intern program and a new-grad program, which started a few years ago; before that, Netflix hired only experienced people. In her experience younger people are more open-minded, more native to new ways of working, and fluent in how entertainment and consumer behavior are changing, which Netflix values beyond their functional role.
She insisted craft mastery still matters. Team members remain responsible for the code they submit, the products they build, and the quality of the experience. Netflix needs to invest as much as ever in mentorship on what good looks like, how to use the tools, and taking accountability for output. She called the concern about learning without hands-on work valid, but said juniors still have to review code, test it, diagnose problems, and recognize a good product, which she called a very scarce skill. Training will probably have to change because the tools are different. She also expects early-career people to teach older colleagues "like me" a lot.
Where engineering goes in 5 to 10 years
Stone distinguished writing lines of code in a particular language from understanding how code, systems, and products work, and said she doesn't think the second is going away. If agents know every language and write all the code, people still need to know whether a product is good and why something isn't working as expected. Netflix's fail-fast, recover-fast approach depends on understanding its systems. She might use an agent to detect or triage an anomaly faster, but she still needs to understand what's being built in order to judge it and fix it.
She admitted discomfort. Code written by some models and agents is "very hard to follow": she can see better performance without knowing why, and wouldn't know how to fix it if it broke. She called this unsettling at first glance, possibly because she is still on the learning curve, and said open questions remain about what tests and understanding are needed to be comfortable with it. She expects engineering to develop that fluency over time. Responding to Rachitsky's surprise at how much engineering changed in two years, she said the last 10 or 20 years also brought big shifts, as the last century did for filmmaking. The cycle is just speeding up.
The future of entertainment and AI in Hollywood
Stone said entertainment "is not going to be one thing," and already isn't, which is why Netflix is expanding beyond film and TV into mobile and TV games, cloud games, live events, podcasts, and a broader set of creators. The challenge is making that coherent for members. She described a journey from listening to The Bill Simmons Podcast, to watching Quarterback, to playing a recent FIFA cloud game, across TV and mobile at different times of day. She said Netflix is already building this. The future will need to be more personalized, immersive, and interactive, and Netflix must make discovery and engagement much easier than it feels today, when content can seem fragmented across services.
On Hollywood's mixed feelings about AI, Stone described Netflix's role as enabling creators to use whatever tools they want. Some filmmakers want no AI, and Netflix works with them. A growing number, she said, want to explore whether generative AI allows new kinds of stories or higher quality, and Netflix supports them and everyone in between. She expects both traditional-feeling film and TV and entirely new formats, which requires flexible tools and partnerships and "a creator enablement view rather than a prescriptive" one.
Rachitsky mentioned Spencer Pratt's AI videos and asked whether fully AI-made shows could become beloved. Stone said she has a hard time picturing entertainment without humans at its heart. She sees storytelling as scarce, valuable, and "one and the same with humanity," and finds characters without that humanity less compelling. AI will definitely play a material role in how productions look and feel, but she doesn't see a version without "the human as the backbone." She acknowledged there will be a broad range of AI content, some of it very entertaining, and said Netflix intends to be at the center of shaping it.
Closing thoughts
In her final remarks, Stone said this is an unusually high-velocity, exciting period for building products in entertainment, and that is what keeps her at Netflix. She also warned that the industry can "lose the forest for the trees" by focusing on technology and capabilities, when the goal is great consumer products and entertainment that people love.
In the lightning round she recommended Into Thin Air by Jon Krakauer and Liar's Poker by Michael Lewis. She also mentioned enjoying Remarkably Bright Creatures on her mother's recommendation and using Eight Sleep. She shared two mottos from her parents: "something good happens every day, watch for it," and "the last 5% of effort usually makes all the difference." She described training to ride part of the route alongside the last, mountain-heavy week of the Tour de France. For people who want to follow Netflix's work, she pointed to the Netflix tech blog. She asked listeners to try Netflix's new zero-to-one efforts, including live events, games, and the mobile vertical video feed called Clips, and to send feedback so the team can reach "great and excellent as quickly as possible."
Everyone can be everything now. PMs can ship code, designers can write PRDs, engineers can product, and there's this confusion and frustration of what is my job anymore.
Anytime a new technology comes along, you go through a storming phase before you go through the forming phase of things. We are in the middle of that right now. I don't think that means we should put AI back into the box and say let's not use it.
If we all become builders, will we still need separate functions?
I still see a craft excellence that's really important that I don't think is going away anytime soon. I still find great engineering to be scarce, great data science to be scarce, great creativity to be scarce.
If you look at the early culture deck of Netflix, high agency, autonomy, paying top of market, this is what I hear constantly now from how the top AI labs operate.
Netflix's culture has always been excellence as an operating system. It's a resistance to do the thing that a lot of bigger companies would do and to feel comfortable in that discomfort very often.
What are the ingredients to make this happen?
Talent density is the non-negotiable, being very comfortable with risk-taking in cases where things are not going well, not assume that process is going to fix it.
What have you added to the career ladders within this AI world?
More systems thinkers, people who can look across all the business domains and abstract that to here's the building blocks we're going to need.
How do people learn this?
Small trick, each problem you're trying to solve, step out one click to the what am I assuming is true about the broader space.
Today my guest is Elizabeth Stone, product and technology officer at Netflix. This is Elizabeth's second visit to the podcast. Her first visit, when she was just a CTO, was for the longest time one of the most popular episodes of this podcast. You'll soon see why this is such a killer conversation because when we chatted two and a half years ago, AI was only starting to emerge. And as a long time head of engineering and product and data science, Elizabeth has such a unique perspective on where things are heading and what's worth paying attention to. Prior to Netflix, Elizabeth was VP of Science at Lyft, Chief Operating Officer at Nuna, an economist at The Analysis Group, and a trader at Merrill Lynch.
Before we get into it, don't forget to check out lennysproductpass.com for an entire year free of the hottest and best crafted AI products in the world available exclusively to Lenny's newsletter subscribers. With that, I bring you Elizabeth Stone.
Elizabeth, thank you so much for being here and welcome back to the podcast.
Thank you. I'm honored to be here. Once and now twice.
That's right. That's a rare treat for me. I don't know if you know this, but your first visit to the podcast, your episode ended up being my second most popular episode. You're right behind Brian Chesky for the longest time.
Well, I'm pleasantly surprised and also mildly competitive of how do I get to the first spot? But I'll set that aside for now.
This is our shot.
Brian's amazing, so I'll let that one go.
Yeah, he is, and then there's just like all these fancy AI people that are just coming, you know, coming in hot.
So, it's been two and a half years at this point. A lot's changed. Obviously AI, something AI is allowing people to do is everyone can kind of be everything now. This idea of PMs can ship code, designers can write PRDs, and engineers can product, and everyone's everything. There's a bunch of elements to this conversation. One is that I've heard from people that there's also this kind of confusion and frustration of like what is my job anymore? Like what am I responsible for as a PM, as a designer? Is that something you've experienced?
I hear it within Netflix, for sure. I think anytime a new technology comes along, especially one that's as transformative as GenAI, you go through a storming phase before you go through the forming phase of things. And I think we are in the middle of that right now. I don't think that means we should put AI back into the box and say let's not use it cuz this is complicating all of our preconceived notions about our roles, but I do think it means we have to be much more thoughtful about how do we get the benefits while reducing the costs.
I think it's a great thing that people are experimenting with how can I develop an idea faster, prototype an idea, put together an initial set of code that would allow us to test it. Do I believe that means anyone should be shipping code to production? That everyone should actually be doing everything? Probably not. But I think that it's good for people to be exploring what's possible.
And then, like I mentioned earlier, the benefit of having product and tech teams together is that if the business problem is clear, I think it's okay and it's healthy for there to be some fluidity in the roles that people play because instead of having to wait for the engineering team to be ready to be able to prototype something, product and design can move faster on it. But they should still work with their engineering partner to think through how should we productize this? How do we scale it? What are the guardrails for it? So, I don't think it makes the functional expertise obsolete. I think it means that teams have to be more comfortable with maybe this helps us move faster in a certain direction.
From an organizational perspective, things I think about to make this more coherent or less frustrating are some of the things that have to be in place for us to get the benefits rather than the costs. So, that includes clarity on source of truth data, guardrails on shipping code to production or testing before we make large changes, thinking about opportunities where we can trust the output of AI versus we should have a process or review that helps us check that we're getting high quality outcomes.
And the importance of reiterating that humans are still responsible for what happens. So, it can be that an agent wrote the code or AI helped to do an analysis when that's not really my background, but it doesn't make people not have the responsibility that comes with what they've created. So, I think investing in some of those core infrastructure and practices and reiterating the accountability and responsibility for the outcomes helps to balance some of like what's possible with what we should actually be doing.
This episode is brought to you by our season's presenting sponsor WorkOS. What do OpenAI and Vercel, Replit, Sierra, Clay, and hundreds of other winning companies all have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you've felt the pain of integrating single sign-on, SCIM, RBAC, audit logs, and other features required by large companies. WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built specifically for B2B SaaS.
Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS. And that's because they are the best. Whether you are seed-stage startup trying to land your first enterprise customer or unicorn expanding globally, WorkOS is the fastest path to becoming enterprise ready and unlocking growth. It's essentially Stripe for enterprise features. Visit workos.com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions. WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.com to make your app enterprise ready today.
What's really awesome about having you back on the podcast is we chatted like before AI was a massive transformation in the world. So, it's a really cool arc that we can explore here. This the shift that we've all gone through.
Coming back to the roles of the product and tech team, I'm curious how much these roles have changed in the last two and a half years. If you think about product engineering, design, data science, user research, which roles have changed most? Which roles have changed least? Like, what's most different since two and a half years ago?
So, you've mentioned some of the things, so I'll reiterate them and then maybe build. So, I have found that PMs, designers, data scientists are able to get farther in the product development life cycle before engineering really needs to be front of the line in unlocking things than was true a couple years ago.
I say that with some caution because, like we were talking about, I don't think it's great to all of a sudden have thousands of prototypes if they're not aimed at this is an important problem to solve for the business and the engineering partners are aware that we're solving that problem and that designers and product managers are going to take the lead in starting to shape the idea, but it's not working in a vacuum and it's not throwing a bunch of spaghetti at the wall to see what sticks.
But when it's the right problem, approached in a thoughtful way with some alignment on that, I've seen product design data science move faster in the direction of let's get to something that's testable on this hypothesis.
So, that's prototyping, that's writing code. The other thing I've seen as being very valuable is we have a lot of information running around in the virtual walls of Netflix. We have experiments we've run over decades. We have insights from consumers. We have input from stakeholders across the business. And that was a problem that really presented a challenge of like, how do we get the most out of that long history of knowledge and learnings to say, let's apply that to the problem we've got now to move faster in this is a promising path or this is something that we've learned something about and we could leverage here.
And AI is very powerful at distilling information, looking across a broad set of things, doing an analysis around it, getting to the core of here's some insights to start with. I would hesitate to rely on that exclusively, but I think it's a head start. And I find even in my own work day-to-day, instead of sending an email that disrupts someone of like, remind me what research did we do in what year and what was the question and what was the test we ran? I can find that almost instantly. Then I can form my own, here's what I find interesting about this and I've now skipped a couple steps towards is there something actionable here? So that's data analysis, it's modeling, it's distillation of information and I'm seeing more people do that to your original question.
So instead of that needing to be only the experts who were here for 20 years and saw every experiment or know where to find it, we're now able to do that faster within product and tech across all functions and a big unlock for us is our business stakeholders sitting in finance and content and advertising can do that as well and then bring back an initial hypothesis where they want to work more deeply with the data scientists and engineer and so on.
So there's something there about the hypothesis generation, prototyping, thinking deeply about problems that feels like it's accelerating and that functions are able to do that in a more fluid way.
But I still see comparative strengths. So data scientists are still going to be experts at can we trust this data? Are we interpreting it the right way? What's the data versus judgment that we should be applying here? A product manager is still going to be exceptional at saying, have we really framed the what of this? Like the problem we're solving in the right way? An engineer still has a craft around the how. How does this scale? What does high quality look like? What problems is this going to create for us based on how we build and deploy something? So I still see the nuggets of that comparative advantage. It's just that we're able to move more fluidly in a lot of steps that normally we would have blockers on.
There's so much interesting stuff here. One is this last point you made is something I've been thinking about. If we all become builders, will we still need separate functions? There's this like member of technical staff trend that is happening in the past where it's like, all right, we don't have a title, you could be anything. You don't have to be in a bucket. What you're saying here is you believe we will continue to have specialties, product person, engineer, data science, designer. While they do more of other functions, there's still a lot of value in... Tell me if I'm hearing you correct in having the specific discipline and skill and background.
I still see a craft excellence that's really important in the disciplines that I don't think is going away anytime soon. Even if there's fluidity or blurring of the work across the functional lines. It goes back to what I mentioned earlier of you still have humans who have to make sure that what we're doing makes sense. We're solving the right problems in a way that is best for Netflix members or business stakeholders. And that if I talk to an engineer, a data scientist, a designer, yes, they speak more languages now than they used to because they have the benefit of these AI tools. But there's still something that is not replaceable when I think about the craft and how they think about what good looks like. And that feels true across all levels and you know, I still find great engineering to be scarce. Great data science to be scarce. Great creativity to be scarce. So yes, some things are easier, but that hasn't dissolved in my mind.
Are there functions that you are finding you are hiring more of? Like the pie chart pie expanding say for engineering or PM or design or something and then functions you need less of with AI tools and LLMs rising.
Not sure that it matches exactly to functions, but I can tell you what we're seeing more of, we need more of. We need more systems thinkers in a world with AI. That looks a little bit different across functions, but I could play out a couple examples.
So, in our core infrastructure team at Netflix in central engineering, a lot of what made Netflix successful over time was that local teams with specific business problems could move fast to deliver. They very often were not feeling like they needed to be on a central paved path. They built the stack that they needed to solve the problem and have the impact.
In a world of AI with agents operating across multiple systems, wanting source of truth data, the importance of having preferred paved paths that get the most of the benefits and produce some guardrails so we can make sure we're doing good work, common infrastructure, common paved paths, solving problems once with a core set of capabilities becomes more important.
So, we are hiring more people who can look across all the business domains and abstract that to here's the building blocks we're going to need in a world with AI. So, that's one of the lenses, but also just with a lens of what got Netflix here doesn't get Netflix there. And we're going to have to have a stronger set of infrastructure to move quickly in this future. So, that means that engineering profiles are more distributed systems, more infrastructure, more of that systems thinking mindset than a local business expertise. Though, of course, we still have people who are deep in personalization and advertising and content delivery. So, it's more something additive for us to have that core infrastructure and systems thinking.
If I take another example, like design, it's extremely important that our experience design team is developing templates and again systems thinking for what does great user design look like at Netflix so that they can enable lots of people, including those who are not designers by training, to develop products that are coherent, that fit into the end-to-end member experience. I get really nervous about having different design languages or different types of user interactions and shipping Frankensteins, basically. So, designers need to then be the people we're hiring
again for design systems thinking. How do we think about templates and expression of the brand and what a good user experience looks like and what is Netflix and the Netflix differentiated special sauce. So, there's more people on our design team that have to think that way now than could I help to design a specific feature for a specific product.
So, there's this stepping back to look at the big picture that I think is happening in every single function and that requires some reorientation of skills among the existing team and also hiring people who've got that type of expertise.
And across all of it, it's a mindset shift. So, we are not hiring people who are not excited to explore, try new things, understand lots is changing and feel comfortable with that ambiguity, be comfortable that there's a blurring of how we work and how we partner. That's true for people who are already at Netflix and people who we are adding to the team, that curiosity, innovation mindset has not... it's not been more important, at least in the time that I've been working in this field.
On the systems thinking piece, is the reason this is becoming more important that people are moving so fast that you need to invest in platforms and frameworks and design language and basically teach people to fish so they can not be blocked, or are there other reasons?
I think it's probably velocity. So platforms do have a benefit of leverage. So in general, that's an opportunity with or without AI for a platform to get most teams 80% of the way there. And then they don't have to reinvent those building blocks. We have more bets that we're making across the business, more things we're trying to build. So platform mindsets are good and it's something that is relatively more recent for Netflix to think about that being a real critical enabler.
There is also the sense of a scaffolding in a world of AI. So not just the higher velocity, but you have more people doing more types of work that are different or new like we were talking about. And there's risk that comes with how do you think about access and identity in that situation? How do you think about security in that situation? How do you think about shipping high quality code and design and user experiences? And so I don't think it scales well to have each person who's building something have to go figure out, could you remind me what good looks like here and what are the bumpers or guardrails I should keep in mind?
I think we need to encode that in our paved paths and our ways of working. And for a data science or analytical field to encode here's the source of truth data, here's how to interpret it, here's how to access it, here's what to do with it or not to do with it and to be careful with certain types of data. An organization that has thousands of people can no longer rely on tribal knowledge or I'm going to find the one person who knows this. So this was a challenge that was there before AI. It's probably a more urgent challenge with AI and I like the idea of using AI or any new tech to motivate, like we knew this is work we needed to do. No time like the present to invest in that more heavily across the team.
I wonder if another reason for this becoming more valuable is because agents are now doing a lot of work and giving them the context, giving them the scaffolding, giving them the design language just speeds all that up.
Yeah, and one of the visions we have at Netflix is we will have so many agents that are contributing to doing work that you need to be able to reason and rationalize throughout that. You know, the humans are the ones guiding what's the problem we need to solve. Do I feel like what we're producing is impactful and high-quality output? But the work will be done by both humans and agents. And that creates velocity and benefits and it creates risks. And I think that's important, especially from an engineering perspective, that we figure out how to manage that in a way that lets people move quickly but doesn't create undue downside or risks for the company.
This connects so directly with Jenny Wen, who was on the podcast. She was head of design for Claude Code and Cowork and had this whole design process is dead kind of thesis, and the pitch there is just there's no time for design, the design process. And instead as a designer, you're just kind of steering people and pointing them in the direction and adjusting and also thinking big picture when you have the time. And it feels like that's kind of what you're describing here, is create the platform for people to move fast and then there's no time for design process of a specific new feature.
I have mixed feelings about that because we do want to enable with infrastructure and systems thinking more people to do great work with strong design as part of it. Why not take that opportunity that the new tech provides.
But for our most important priorities, design is critical to solve things in the right way. So, we do still make time for important design work. It can move faster. The designers themselves have more tools in their toolkit, so they can do incredible work at a faster velocity, show more options, learn, iterate, test more quickly. But I think it would be a mistake to say design and deep design expertise and thinking gets squeezed out just because we can write code faster, we can do data analysis faster. At least for a large-scale consumer product like Netflix, I feel like we would lose one of the things that makes Netflix great, which is the product, technology, and design makes a lot of complexity invisible, and makes for a seamless customer experience. That's a design mindset that has to be core to it. So, the work itself might look different, but I don't think we lose the mindset.
That's an awesome counterpoint. So, what I'm hearing is kind of trending up skills, attributes you look for: systems thinking, and this kind of mindset of being comfortable and excited about change and what's coming and not being stuck in your own ways. What are you finding is trending down? What are you less looking for that you used to value more highly?
The days of very narrow, deep specialization feel more limited to me. I can come up with examples where we still need it because there's an industry or technology expertise where there's only a few people in the world who really know how things work. We have examples of that on the team for encoding or how our playback systems work and things that have been incredibly innovative and novel for Netflix. I still believe we need specialized practitioners in those spaces.
But as a general rule, compared to 5 or 10 years ago, I would believe we have fewer specialists and more people who are generalists or adaptable in multiple directions. And that could be adaptable across functional expertise. It could be adaptable across flavors of engineering. So, can I navigate both back end and front end systems? Can I hook into infrastructure with a lot of expertise? I think the mindset now needs to be I can learn that quickly, and that goes back to the systems thinking. So, I think specialists can learn to have a broader array of tools more easily than was true in the past. So, we need fewer of them perhaps because talent's able to grow in that direction. And there's something about sticking to a narrow specialty that maybe triggers for me a concern about the mindset of growing in different directions and exploring, and I don't want to be too narrow even in my own assessment of that, but it's important that people who are specialists still have that sense of I want to try a new way of solving these problems versus the way we have in the past.
And when you say specialist, are you thinking like front end, I'm a front end engineer versus a back end, or are there other...
Yeah, or it could be a domain set of knowledge of, yeah, I'm a deep...
Expert.
I'm a payments expert. I'm an ads marketplace design expert. I'm an expert in this very specific tooling that studio productions use. So, specialist in subject matter expertise is an advantage provided that person is willing to grow and extend into is this really still the right tool or the right way to think about the problem? So, I think it's the layers of the stack from an engineering perspective that there's less specialty. And then tools that are unlikely to be static or likely to have a lot of inertia around them. I would think we would want people who are able to innovate and imagine what's the future version of this. And so we want more talent like that.
Awesome. So coming back to the systems thinking piece, people hearing this are like, okay, I got to work on my systems thinking skill set. How do people develop the skill? Is it just do it for a long time? Work at a lot of complex projects? I think of this book that everyone always references with the slinky on the front, Thinking in Systems. Yeah, how do people learn this?
Small trick. Each problem you're trying to solve, step out one click. Do the, what am I assuming is true about the broader space in solving this problem?
So I was given a task to build some new feature for the Netflix member experience. Let me take one beat and think about what is the bigger consumer problem we're trying to solve here? What's the type of content that this feature is going to be able to support? Do I think that the way I was planning to build this is going to make sense in a way that scales across multiple content types? Or it could be something that's a capability that then is contributed to a platform set of offerings for multiple areas. Is the consumer problem that I'm solving with this feature going to be one of the most important consumer problems that Netflix is going to need to solve as we have an expanding world of entertainment and we want to make it more personalized and immersive?
Those are all questions that, you don't have to boil the whole ocean. You don't have to solve for Netflix's overall strategy and who are we relative to competition. But you take the thing you're responsible for and you just do one zoom out of the problem you're solving and question that. I wouldn't spend too long in the questioning state because then you're stuck. Then you're not making forward progress, but I think that helps people to think in terms of systems and question: are we solving the right problem in the right way that matters for the end consumer?
As you describe it, another way I'm thinking about it is think if you were your manager, what's their broader perspective across not just your one team and problem and KPI, but the larger picture?
I've got advice over years that is similar to that, which is: are there ways that I can do my job that helps my manager do their job? And so if I thought about all the things I'm directly responsible for, but I thought about it from the perspective of my manager, so not just product and tech, but finance and content and other parts of the business, I would naturally zoom out and think about how all these component pieces need to come together and how the whole could be greater than the sum of the parts. I think that's useful thinking. And for engineers to think about how do I leave a better version of these systems? How do I think about the thing that's going to be high quality and scale for others? There's both a how do I help my manager and there's how do I help my colleagues, which is a core part of some of our engineering principles of do the thing that is right for the broader organization instead of just what's right for you locally. That's systems thinking as well. So it's not just seniority, but it's breadth: the way I solve this problem and I build this, is it going to be useful to my colleagues, and am I going to leave a stronger version of things for the future set of innovations that we want to make?
That is awesome tactical advice. Making your manager's life easier is always a good tactic.
Career-wise, several reasons. Yeah.
Following the thread a little bit, I know you all added career ladders and levels recently. It was a new thing; you guys used to not have these things. So kind of all on that thread, what have you added to the career ladders within this AI world, if anything, that you find you want people to lean into more, you're looking for more? Or not, like, did you not change your career ladders and performance criteria?
So, the way we've approached this so far is instead of trying to articulate at each level exactly how AI changes those expectations, to instead put an overlay across all of the talent at Netflix, people on the team, and those who are hiring, to talk about an aspiration for AI fluency. And what that looks like is going to vary by function. It's going to vary based on where you are in your career. That could be what level you're in or what type of role or persona work you're doing.
But the aspiration for AI fluency, which is a tough thing to define. So, does it mean that I have an experimentation mindset? Does it mean that I know where AI is useful and not useful? Does it mean that I've actually built things using AI? I feel like the way that has shown up in career ladders and how we talk about it evolves almost by the quarter, if not month or day, because the tech itself is advancing so much. So, the most useful thing is not to make it level specific or role specific, but to encourage everyone towards the expectation on AI fluency, which doesn't mean use it as a tech for the sake of tech. It's tech where it's useful, to have good judgment about that, and to have the mindset to be open-minded to explore and try new things. That's the non-negotiable for all roles, and that's true at the most senior levels of Netflix, where we talk about we too need to have deep fluency in AI, even if we're not writing code as part of our day jobs. So, that's changed, and then that's showing up in our hiring practices as well. Getting comfortable within interviews exploring how are people thinking about AI or technology? What are they using in their day-to-day or their current job? How comfortable are they with change and exploration? And even for things like coding interviews, allowing candidates, of course, to use AI tools because that's going to be part of what the work requires now. So, those have been shifts that we've made, but I doubt it's a shift that's done versus we're right in the middle of it.
And she's going to keep following this thread. Obviously, AI is transformative for coding. It's a big unlock for prototyping. Are there other use cases of AI at Netflix that have been really impactful that people may not think about or not realize?
So, there's two that come to mind. So, the first is data analysis, distillation of information, modeling, which is, you know, using the tools to get our arms around all the insights we have, similar to what I mentioned before. What experiments have we run? What are the metrics that I should be looking at for a certain problem? What's the consumer research that we've done? And that is much higher velocity and much higher quality, contingent on you check that the results are valid, you work with your local data scientist, and am I using the source of truth data on this? But that's been a great one and that's one personally that I would say I most use some of these tools for. So, that goes beyond prototyping and coding to general analytical thinking and translating data to action and insight.
The other one is on the content production, creation part of the business, which has lots of applications. This was true before GenAI. So, ML and AI were deeply used in a lot of the production tools. We've used them to think about how to create promotional assets at scale, how to localize in subtitles and dubs. So, GenAI is a big step function in where the impact can be in creative ideation. We call those things like pre-visualization, or basically bringing a creator's vision to life before you even get into, you bring people to a set and start to actually go through the production itself. There's lots of use cases in post-production.
So we recently acquired a company, InterPositive, that was started by Ben Affleck, that built a set of models and capabilities that allow you, after you've shot something, to relight, reframe, reshoot, change dialogue in ways that are very impactful to get higher quality content, are still led by the filmmaker creator saying, you know what? I would like to try something else to bring this vision to life. But that impact is extremely promising and we're seeing lots of productions leverage different tools, some of them built in-house, some of them that we enable through other vendors for those content creation use cases. And then as we think about how content comes to the product, I mentioned localization, subtitles and dubs, but also how we create high-quality trailers, images, artwork at scale that then we can use to help make sure that titles find their audiences around the world. Those all are huge levers when we think about the AI impact. So that again goes well beyond prototyping or coding to some of the creative use cases, and you can imagine that just like they work for studio productions for film and TV, they work for advertising, they work for marketing, off-service campaigns, and so those are all areas that we're exploring.
This episode is brought to you by Mercury, radically different banking loved by over 300,000 entrepreneurs. And now with Command. I've been a customer of Mercury's for over 6 years. I have never once thought about leaving. Mercury is basically what happens when banking is built by product people, not by bankers. They make it so easy, dare I say fun, to send invoices, move money around, set up virtual cards for folks on my team. Does your bank have an API, a terminal native CLI, or an AI-ready MCP server? I don't think so. And just recently they launched Command, a conversational interface built directly into Mercury, which acts as your financial operator. I've been using Command to transfer money around, to figure out what categories I've been spending the most money in, analyze my cash flows. And just today I used it to find out how much I've made from a specific sponsor over the past year. I just asked, "How much have I made from X over the past year?" 10 seconds later I have an answer. It is so freaking cool. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC insured bank. Banking services provided through Choice Financial Group and Column NA, members FDIC.
You mentioned how Netflix has been very early to AI and ML for a long time. Younger people may not remember this, but y'all had this contest to optimize things. Yeah, the Netflix Prize. Like, it just showed an example of how early you were to AI and ML. There was, I think it was a million-dollar prize to optimize the Netflix ranking algorithm a little bit. Like whoever could optimize it the most. And I think the winner optimized it by a few percentage points, something like that. And it was a huge deal. All these super smart people around the world. And it happened a few times, right?
I mean, you said it on my behalf. Often when there's questions about how is Netflix thinking about AI, it's great to remind people of exactly that point, that this is not new to us, that especially for personalization, it's been central to delivering a great experience to members. It's impossible to take the breadth of content that we have. There's ever more content. That's one of the challenges we face. And make discovery easier and easier and easier, which is one of the challenges that Netflix has.
And using AI and ML has been a way to do that. You want to personalize the right title for the right person at the right moment, that problem gets harder. The more exciting our catalog gets, the greater breadth of content we have, not just film and TV, but games and live and podcasts, personalization becomes even more important in what that experience is. So, we can take a lot of that history and say, "Okay, well, now how do we solve this problem?" Because the tech is even more powerful, but it gives us a running head start in being clear about the problem to solve, how important it is that Netflix solve that for our members. And then the same is true, as I was mentioning, on the creative side of the house. AI and ML have been in things like visual effects or in localizing language for a long time. Now we say, "What's the next era of that when the tech is more powerful?"
And in both cases, it ends up taking a strength that Netflix has, which is marrying entertainment and technology, and making sure we stay ahead of the game to deliver things that are even better. So, I love that it's part of our history. It still continues to be a strength, and it's going to have to be a strength, given the size of the challenges we're facing around the breadth of entertainment while keeping a great experience.
Yeah. And I love that back then it was called machine learning, and AI was like, "No, no, it's not AI. AI is never going to happen. It's just machine learning."
Well, then all of a sudden we call everything AI, and some of it's machine learning.
That's right.
So, I tried to, you know, it depends, like the thing that is of the moment to describe. So, I think we bucket all of it as AI now.
Yeah, AI has become
of AI use cases that are not generative use cases. So, we could go down a deep dark hole of all the specific things. But in general, I don't think it would surprise anyone that Netflix is using a broad array. And with so much excitement about what's possible, the fun thing at Netflix for the people who work here is that if you're really passionate about the applications of tech for creative outlets, for consumer products, for infrastructure, we have all of those problems and AI is at the center of them, and it's good not to forget that that's true even if Netflix isn't branded as an AI company. AI is a tool that we're very comfortable using to get these great entertainment and technology outcomes.
The other really interesting thing, just to kind of keep complimenting Netflix here. If you look at the early culture deck of Netflix and also our conversation last time, things that emerge from that are things like high agency. This is something core to Netflix in the beginning. High agency, autonomy, high talent density, very bottom-up thinking, super quick experiments and launching, paying top of market. This is all stuff that every AI, like, this is what I hear constantly now from how the top AI labs operate. So we're all ending here, and this is where Netflix has been forever.
Yeah, it's a little prescient in understanding what makes talent incredible. I've thought about all those aspects of the culture at Netflix as, this is going to sound a little bit nerdy, but excellence as an operating system. So the goal of all those cultural elements wasn't the end goal in themselves. It wasn't let's just make sure people have as much responsibility as possible, or let's, you know, we don't like process, so let's make sure that we don't have any of that.
It was instead a very strongly held opinion that you get to excellence by giving people a lot of agency and accountability. By pushing decisions as deep in the organization as possible, hiring great people who can be trusted to have good judgment and make good decisions.
And that ends up driving incredible outcomes plus a lot more motivation and sense of responsibility. It means every person on the team can feel like I'm being given a lot of keys and a lot of accountability for what happens here, and I myself feel like when you know you're carrying that level of trust and accountability, you want to do your best work.
And so there's something that feels very intuitive about Netflix's culture, it has always been aiming at excellence. And when you have great talent and you give them the ability to do their best work without micromanaging it or drowning it in process, you actually get much better outcomes.
And so I do think that the newer era companies are picking up on something that is feeling very familiar to us. And it's not something that comes easily. So having culture is not a static thing. Culture needs to grow and evolve as a company gets bigger, the types of problems you're solving change. But the notion that we're going for excellence and trusting that exceptional talent needs to be able to do their best work, that's unchanged and something that I think continues to be a special sauce for us.
I love this concept, excellence as an operating system. It's very systems thinking, he might say, for how to set up a company.
Exactly, Lenny.
So for people that, like, everyone listening to this will want excellence as an operating system. Like who would not want this? It'd be helpful for people to hear what are kind of the ingredients to make this happen. One is obviously high talent density, just hiring only the best. Two is accountability. There's like the input and the output essentially. Input amazing people, the top people, make them accountable, give them autonomy. What would you say are kind of like the pillars of creating this excellence as an operating system, if founders are listening to this, like, I want to do that?
Well, the talent density is the non-negotiable. Like you have to start with that. If you don't have that, you can't get to a place where you have confidence in decision-making at all levels of the organization, allowing people to take risks and innovate quickly. That's a big part of excellence in the Netflix culture, which is being very comfortable with risk-taking. We don't try to avoid failures, we try to recover quickly when we have them.
I think there's been great examples of that. Our foray into live was a wonderful example of being comfortable taking a ton of risk, knowing it would be imperfect, knowing we would learn fast, and we would be better for it. I've never been prouder of the team seeing how we worked through that. So, you have to be talent density, comfortable that people are going to take the context that you give them, strong judgment and risk taking, and fight for the things that are the best outcomes for the business. You have to be very clear that what you're doing is driving outcomes for consumers and Netflix. So, it's Netflix matters, Netflix members matter. It's not about my own personal success or what I prefer. So, there's a selflessness that is part of this excellence operating system.
And then the other thing I would say is some of the things that are really unnatural for humans to do. So, I could give a couple examples of things to get comfortable with, which is there are certainly days where I see decisions happening, and I think, "Hmm, I would make a different decision." Like, is that really going to be the best thing?
But, my job, especially in the Netflix culture, is not to step in in every one of those cases and overrule or veto or question someone, especially if it's not material, it's not going to burn the place down. Let people make that decision and learn from it. And ask for those reflections afterwards of like, "How did it go? Maybe I was wrong. Maybe the decision was a great one." But it's related to the risk taking and the, like, help people learn how to feel comfortable making their own decisions, especially when they're not all going to be the right decisions, and they're going to learn something tough from it. I felt that myself from my boss and my peers saying, "This is your decision. You know, I can provide input. I can help you brainstorm. It's yours in the end." And it just doesn't come naturally when the stakes are high, when I feel responsible for what the org's doing, to let people lean into risk can be uncomfortable.
And I think that also means in cases where things are not going well, as another example, to not assume that process is going to fix it. So, something I've learned over the past few years, that when planning is difficult, I've never heard someone say like, "Oh, we figured out the perfect way to plan." Or the perfect way to go through feedback and leveling and compensation. But every time we saw that and we added more process, we spent more time without getting better outcomes.
And so, it's another unnatural thing that I think everyone's inclination when things are hard and complicated is you think you're simplifying the problem by putting a lot of constraints around it, but it actually goes against the, like, is there a more creative way to plan or to make people decisions or to make prioritization decisions that actually get us to better outcomes. And so, it's a resistance to do the thing that a lot of bigger companies would do and to feel comfortable in that discomfort very often. So, that's something I feel in my role and I would believe a lot of people at Netflix feel it, because you try not to do the thing that is standard.
It's easy to say that and hear that, but I so know what you mean, where somebody screws up and you're like, "Okay, what was the thing that went wrong? Let's put a process in place to avoid this from happening." And what you're saying is, like, you need to resist that. Because that slows things down and the best people don't want to be working in a place with all these checklists and process and gates and things like that.
No, I think the best people want to know there's going to be a blameless retro, and they're going to feel so individually responsible that they're going to say, "How do I make sure this doesn't happen again?" Not with process, but with, like, how could I share these learnings? How could I do work differently to make sure that I get to a better outcome next time? When you are trusting people to take those reflections and learn and grow, I think you get much better outcomes over time. You get a much stronger team, which I think is part of our role as leaders, of like, you're trying to grow a team that is resilient and durable and knows how to have great impact. You're not trying to control everything.
Which is a key to building a team with high talent density. There's two sides to this that I want to chat about briefly. One is the hiring and the other is keeping the people. So you're famous for the keeper's test. We talked about this last time. Another unnatural thing for people. People that want to understand what this is, they can listen to the first conversation, but how has that evolved over the last couple years? Is that still a core part of the culture, this idea of the keeper's test?
It's often cited in a way where you think of the keeper's test as that moment where you decide to let someone go, that they're not the right fit for the role, and the conversation about that. But it's equally commonly used to have a conversation about how extraordinary someone is. How well they're doing in a role. Because the entry point is for me to say to one of my direct reports, or for them to say to me, "How am I doing on your keeper test?"
And the lion's share of the time my response is, "I would fight so hard to keep you." Let me go through a set of things that I think you're doing such a great job at, what your strengths are, where you're having a lot of impact. Here's how you could be even better. So it's an entry into a conversation that is very positive and uplifting for people, but the framing is, "Do I pass the keeper test?" And then, of course, there's the harder situations where I'm evaluating does someone pass the keeper test, or they're asking me, and this is the toughest thing to say, to be honest, you're not passing that right now. I think you could get there in some cases, and that comes with feedback and what are those milestones? Or in some cases you're saying, we've really tried and I don't see the path to success. So, it's just an anchor and an entry point for a conversation that can go lots of different directions. And the thing I like about it is it's good hygiene on feedback and checking in on how things are going, and forcing a tough conversation sometimes instead of shying away from it. Or to keep great talent, you do need to say you're doing great. Like that's an important part of making peo-
people feel recognized and valued. So, I don't want it to come across that we just have this very negative view of it. I think there's this positive side of the coin as well.
Awesome. I guess just to explain to people what this is so they don't have to go listen to other podcasts, I'll try to briefly explain it. The idea here, a part of the Netflix culture, is that when you have people reporting to you, you should always be thinking, if I were to, would I hire this person today? Knowing what I know about them, and if not, then I should probably let them go. And the idea there is to keep the high bar, to not ever just like settle, okay, this person they're here, I guess we'll keep them around. Is that roughly the way to understand it?
Yeah, and the way it can, it's sort of a corollary to that: if that person came to me today to say they were leaving, would I fight to keep them or not? Or would I say, if my sense is relief of, oh yeah, it probably would be better to have someone else in this role, I should have taken action in having that conversation sooner.
I love, as you said, it's so many uncomfortable things you have to do to maintain
Yeah, well, the keeper test is one, maintaining talent density, context not control among leaders. We talk about being highly aligned but loosely coupled, which is where light process, you know, the minimum to make sure we're clear on the priorities and we can execute them, is what we're solving for. All of these things are not things that human beings or organizations at scale tend to do. So, it's constant diligence to try to maintain the thing that's made Netflix a special place. Cuz in the end, it's the work and the culture that attracts people and retains people, and we need that to be a successful business.
So, that's exactly where I was going to go. So, to make this work, you need to attract the best people. It's always been very hard to attract the best people. Feels insanely hard these days with the amount of dollars flying around, the fancy AI labs, so much competition. There's like everyone's just, you know, it's crazy. What have you found to be effective in convincing the top people to still come to Netflix and join versus all the other fancy places they can go?
Yeah, we've always had a lot of competition for talent. It might feel more pronounced right now, but we have great talent on the team. Maybe that goes without saying, but I feel like I should say it out loud cuz I believe it. We have incredible talent at Netflix, recent hires, long-tenured people. I'm always impressed by the work that the team is doing. So, I don't feel like we've suffered or like other companies are vacuuming up all the good people because so many of them I do think sit at Netflix.
It does feel like we have to be more explicit about the types of people and talent that tend to thrive at Netflix versus other companies like some of the frontier labs.
So, people at Netflix have to be passionate about the application of technology. And the application or building products to solve a certain set of problems. You have to love entertainment. You have to love consumer products at scale. You have to love the global nature of that. There are a lot of incredibly talented people who love that sweet spot. I am one of them, between tech and product and entertainment, and how do you make those things come together in a way that's remarkable? And you use AI to do it. You use other technologies and products to do it. But that has to be something that drives you to be really excited about a lot of the roles at Netflix.
If instead you're by some of the foundational work that the frontier model companies are doing, which is exciting in its own way, it's a different persona. It's a different like, here's the problem space that I want to work in. But I don't think there's a shortage of people who get really excited about the applications of the technology and see the connection to that to things that they love and use every day like Netflix. And so that, you know, that gets me up in the morning and I think it gets a lot of the team members up and we have this conversation about like that's something special that only talent at Netflix can do, or fill in the blank for another industry that's deep in the application of it. I think that's inspiring.
I want to kind of touch on a couple things that I've been thinking about in this world of AI that we're approaching. One is junior people. It feels like everyone's like, there's a good example. You're hiring a lot of awesome senior people that have proven they're awesome and, you know, high talent density, high bars. Also just AI makes it so easy to do stuff that people may not be learning how to do anything. They're like junior engineers I'm thinking, or junior PMs, junior designers. Like there's just like, how do new people become these awesome senior people? Is there anything you think about? Are you hiring junior people? How do you think about this, what happens with junior people not necessarily learning or having a path to learn to become the senior person?
We are still hiring junior people and they're really important to our talent strategy. So, we still have an intern program, we still have a new grad program, which was new for us as of a few years ago. So, prior to a few years ago, we were only hiring more experienced talent across all the functions. Now, we do hire people straight from undergrad and graduate programs and we'll continue to do that. So, even in a world of AI where some things are easier, we were talking earlier about mindset, AI fluency. From my experience, younger folks are more open-minded. They tend to be more native in some of these new ways of working. For a company like Netflix, they're also very fluent in how entertainment is changing, how consumer behaviors are changing, how product and tech is influencing that in the products that they're using. That's really important to have on our team. So, there's the part of the persona, which is who are you as a new grad who's an engineer, but there's also who are you as someone who's in their early 20s and has a perspective on the world that is highly valuable and a comfort with the way the world is changing. So, that's why I say it's a critical part of our talent strategy.
To the, okay, so you step into the role and you have AI tools that didn't exist 5 or 10 years ago, I would say mastery of the craft is still very important. So, going back to, as the team member, I'm responsible for the quality of code that I am submitting for production, I'm responsible for the quality of products that I'm building, how they are designed, what that user consumer experience is. None of that is going away. So, if I think about more junior or earlier career talent on the teams, we need to be investing just as much in the mentorship of: this is what good looks like, this is how you use these tools, but you still take accountability for what the outcomes are, what the quality of the output is.
And I think I mentioned this earlier, I find that mastery and that craft excellence scarce still. So, we want to make sure we're teaching that. I think it's a valid concern of like, how do I get that if I'm not as hands-on as I would have had to be, but you still carry responsibility for reviewing code, testing code, being able to diagnose problems, knowing what a good product looks like. Like, I think that's a very scarce skill to say, "This is excellence in a product that solves a problem that matters and in how it's designed." So, I don't think that craft mastery, the importance of it, is going away. Probably the way we train and grow talent has to change cuz they're going to use different tools, and I can guarantee you that earlier career talent is going to be teaching older folks like me many new things, too. So, I think it goes in both directions.
Where do you think engineering goes in the, I don't know, 5, 10 years? Do you think people need to still understand code? Or do you think there's this abstraction layer that sits on top where you don't even have to learn C++, Java, Python, whatever?
I think there's a difference between being able to write lines of code in a particular language like Python or C++ and understanding how code, computer systems, products work. And I don't think the latter is going away.
Because if we trusted agents to know all the languages and write all the code, we're not going to know why is something, is it a good product? Is it a bad product? Is it working as we expected when it doesn't? Like I mentioned earlier, we take a lot of risk. We fail fast, we recover fast. That requires an understanding of how are these systems working. I might use an agent to help me understand those things, help me detect an anomaly or something that's broken faster and triage it, but I still need to have a fluency of like, what is this thing that we're building and how does it work? So I know if it's good and I know how to fix it. I don't know, I hope that doesn't go away cuz, you know, that's like, how do we make the world a better place through the stuff that we're building? I think requires some understanding of what we've built.
What I'm hearing, which makes sense, is you may not have to write the code but you have to understand it and what's happening. But it's so much harder, just as a person not writing it, to actually, you know, have that instilled in you.
I think that's one of the things that the learning curve is very steep on right now. So looking at some of the code that some of these models or agents are writing, they're very hard to follow. It's like, I know I'm getting better performance from this but I have no idea why, and if this thing breaks I'm going to have no idea how to fix it. That makes me uncomfortable. You know, maybe that's because I'm still on that learning curve of like, how do we operate in that world? Like what's the set of tests or rationalization and understanding that we need to have to get comfortable with it? But at first glance it looks very unfamiliar and very unsettling. So I think engineering over time will evolve to be comfortable with that and have fluency in it and know how to guide new tech and agents and new capabilities to make sure that we feel really good about what the output is.
I wonder what the metaphor is for this, where like, I continue to be astounded by how much engineering has transformed in like 2 years. It's like a completely different drop down. You used to just sit there and write code, and now you're just talking to agents and reviewing code and shipping a bunch of PRs a day.
It feels like it's an acceleration of how much engineering has changed. But if you looked over the last 10 years or 20 years you would say the same thing. So there's just something that's moving faster and it's hard to wrap our heads around how quickly it's moved in the past couple of years, but it's not totally unfamiliar that engineering or data science or product would have these big shifts, just like how filmmaking works. If you go over the last 100 years, it's unbelievably different because of technology and new tools that we brought to it. Just feels like the cycle is speeding up.
Okay, I want to talk about entertainment for a brief moment. I'm curious just like how entertainment will change over time in, say, I don't know, 5, 10 years. Just, you know, today we open up Netflix, check out some shows, watch some videos. It hasn't changed in a while, just that idea of like, cool, I'm going to watch The Pitt and watch it all. I'm going to watch a movie. I got TikTok, I got Instagram feeds of stuff. Like how much different do you think this will be in, I don't know, 5 years? The way we entertain ourselves.
It's already changing at Netflix because entertainment is not going to be one thing in the future and it's already not one thing now. So, part of the reason that we are going beyond film and TV in our offering is because there's an expectation that consumers have of much greater variety across formats, devices, moments of the day that Netflix needs to be able to serve well in order to meet consumer expectations and hopefully exceed them over time. So, when we think about the addition of mobile and TV or cloud games, live content, podcasts, working with a broader set of creators who are now on the Netflix service, all of those things create a greater breadth of what entertainment is and Netflix is able to define and expand that.
And it puts a higher bar expectation on how do we make sense of that for a Netflix member? So, how do we show you this very seamless journey from I listen to the Bill Simmons podcast, to I watch Quarterback because I love that as one of the Netflix offerings in the more, you could say, traditional film or TV space, to I play the most recent FIFA cloud game. And I want to be able to do that in both TV and on my mobile phone because now I'm on the move and I want to be able to discover and engage with the content at different moments of the day. That's already a journey that we're building into Netflix, which I think will become stronger and stronger over time.
So, the future of entertainment isn't going to be one thing and it's going to have to be more personalized, more immersive, more interactive, with this sense of this is a world that I can explore in lots of different directions depending on what I'm looking for in the moment. And the challenge Netflix has is we've got to make discovery and engagement much easier than it feels today. We have tons of content and it can feel very fragmented, especially when you consider all the services or offerings out there. And I think Netflix is very well positioned to understand how to solve that problem across entertainment, product, and tech.
The other element of this is AI, obviously. As an outside observer, it's so interesting to see how in tech, it's like, AI, I love it. It's the future. It's the best. In Hollywood, it's like, "No. Shut it down."
There's a mix. There's a very wide array. So, Netflix's role in this is to enable creators with whatever tools they want to use to bring their vision to life. There are going to be some creators or filmmakers who are on the end of the spectrum that says, "Absolutely not. No AI. That is not how I do production. It's not consistent with my vision." That's fine. We work with those creators.
There's other creators, a growing number of them, I would say, who are very interested in exploring, "Wait, can these gen AI tools make something possible that wasn't possible before? Can I tell a story in a new way? Can I make that story higher quality and more resonant for audiences? Can I do things that are extra creative in how I think about bringing a story to life?" And we support them as well, and we support all the folks who are in the in-between. And that's a really important position for us to be in, again, because entertainment is not going to be one thing. There's not going to be one format. I think there's going to be types of film and TV that feel traditional, and then there's going to be entirely new formats that unbelievable creators help to bring to life, and Netflix wants to participate in that. Which means we need to have a flexibility in the tools that we provide and the types of partnerships we have, and to really have a creator enablement view rather than a prescriptive one that we only do this one way.
I think people are going to be surprised by just how good AI content is. Like Spencer Pratt's videos, everyone's like, "Wow, this is entertaining." Obviously AI, but it's so interesting. Do you think we'll get to a place where it's just like whole TV shows are AI and people love it?
I have a hard time picturing entertainment that doesn't have humans at the heart of it. So that's humans in the creation of the storytelling, which I think is a scarce and valuable skill. Yeah, storytelling is one and the same with humanity, and like knowing what connects with people. So I think humans will always be a core part or a critical part of the story. And I think watching characters on screen who don't have that humanity
feels less compelling to me. And what the power of storytelling really is, to see another human and to watch how they perform a role or bring an emotion to life. That's such a human element. Will AI help to bring that to life? Will play a material part in some of those productions or how we get them to look and feel a certain way? Yeah, definitely. But I don't see the version of it that doesn't have the human as the backbone.
There's a quote that I think is misattributed to Salman Rushdie, which is when a child is born, they first ask for food and water and protection, and then they ask for tell me a story.
It's a thing going back since the beginning of time that storytelling has been a key part of community and social networks and human feeling and connection. So, I love the idea that technology can amplify that and can bring that to life in very new, novel, exciting ways. But, to say storytelling wouldn't have that humanity at the center feels like something would be missing.
We're going to see some wild over the years coming out of this.
Oh, I'm sure. There's no question about that. And a lot of it could be very entertaining. I don't debate that, either. But, I think there's going to be a broad range, and I think Netflix needs to be at the center of shaping that and bringing that to life, which is our plan.
Amazing. Well, we covered a lot of ground, Elizabeth. Before we get to our very exciting lightning round, is there anything else that you wanted to share, leave listeners with, maybe double down on from things we've talked about?
It probably came across throughout, but I would underscore that this is a really exciting time to be building products in entertainment. Everything we talked about of what's changing in the tech and consumers and what is entertainment, we're at this unbelievable high-velocity innovation period. So, it's what keeps me at Netflix. I think it's a fun place to be. I would be missing something if I didn't reinforce that I think that's true. I also think that as an industry we spend a lot of time sometimes talking about the pure tech or the capability and we sort of lose the forest for the trees. We're trying to build great consumer products that people love. We're trying to make great entertainment that people love. And it's their favorite thing that I don't want that to be lost in. Of course, there's amazing tech and product stuff that sits underneath, but in the end, the thing that's most inspirational is what do we bring to people around the world?
And on those lines, there's been such a, the opposite of glut, a drought of new consumer products, consumer experiences. There's very few successes, almost no consumer startup works. And AI feels like an opportunity for something else to work, and I feel like Netflix is one of the rare companies and brands that continues to deliver an awesome consumer product and business. There's just not that many of them.
Yeah. We're going to keep that up.
Well, with that, we reached our very exciting lightning round. We've got five questions for you. Are you ready?
Okay, I'm ready.
All right. What are two or three books that you find yourself recommending most to other people?
I mean, I have to come up with different books than I said last time.
I don't know. But I think that sounds great.
I still like a good throwback. So, two that are coming to my mind: Into Thin Air, Jon Krakauer, and Liar's Poker, Michael Lewis. So, I worked on Wall Street and I like reminding people what it was like in the way back time.
Favorite recent movie or TV show you really enjoyed, which is maybe too hard for someone working at Netflix, but I'm going to see what comes out.
The list is very long. The most recent I watched, Remarkably Bright Creatures, after a recommendation from my mom. It's a tearjerker. Talk about the human part of storytelling.
Favorite product you've recently discovered that you really love?
Critical for my health and well-being, Eight Sleep.
Do you have a favorite life motto that you often come back to in work or in life?
I often go back to the things that my parents instilled in me in very early times. So, at the risk of repeating, maybe. First, something good happens every day. Watch for it. Even in the most stressful times. And second, that the last 5% of effort usually makes all the difference.
These are awesome. They hit me. Final question. I don't know anything about this, but you mentioned you're doing some kind of cycling event.
Oh, yeah.
Tell us what's going on. What are you doing here?
So, my husband and I are doing a trip where we ride alongside the Tour de France for the last week of the race. So, the tour is 3 weeks. The last week has a lot of mountain stages. So, we get to ride part of the route each morning and then watch the race in the afternoon. Not for the faint of heart. So, I'm trying to train up so I can enjoy those rides. It's supposed to be vacation after all.
My god. I love this vacation. We're just going to a race.
I love cycling. I love professional sports. It's fun to be able to participate in it.
So, is this like racing or you just kind of try to go as nonchalantly through the course?
You go nonchalantly. But, still, I think I mentioned, yeah, it's physically and mentally challenging. And, you know, it's not a race, but I don't want to be at the back of the pack. So, I got to be comfortable enough to hold my own.
Wow. I love how different this is from your job. And it feels like something else to do.
It's a good balance and it gets me outdoors and gives me some nice perspective. So, I'm looking forward to it.
Elizabeth, you are awesome. Two final questions. Where can folks find you online if they want to follow you, reach out for maybe anything that came up? And how can listeners be useful to you?
The best place to find me and some of the work we're doing or reach out is the Netflix tech blog, actually, where we're putting a lot of things that I've been talking about up there. We're trying to do a better job communicating about the fun stuff we're working on. So, that's a good first stop, usually. And then how listeners can be useful: try all the new stuff that we're putting out there. Watch the live events, play the games, have fun with the new vertical video feed that we have on mobile called Clips, send us feedback. So, we want to make it better, and a lot of these things are new zero-to-one efforts for us. So, we're trying to get to great and excellent as quickly as possible.
I love that the homework is go watch Netflix and
You can also watch other things, tell us how we can be better, but I'm definitely interested in how can we be better at Netflix.
I love it. I'm going to go do that. Elizabeth, thank you so much for being here and being here again.
Thank you for having me. Always fun.
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
Article published · Updated