Inside Netflix Engineering: CTO Elizabeth Stone on Autonomy, Live Streaming, and Unusual Responsibility
The Pragmatic Engineer
What is it like to work as a software engineer at Netflix, and how does a company of its size let engineers make major decisions without layers of approval? In this conversation, recorded at Netflix's Los Gatos offices, the host of The Pragmatic Engineer asks Netflix CTO Elizabeth Stone about the company's engineering culture. Stone's central argument is that Netflix's autonomy, lack of process, and high talent bar are not goals in themselves but means to excellent work. The discussion covers how the company built live streaming in about 18 months, what went wrong along the way, how it handles performance without formal reviews, and where it is finding value in AI tools.
The scale behind the streaming app
Stone begins with the scale of the operation, which she says is probably larger than people realize. People in her personal life often ask how many engineers it can really take to build the Netflix product. Her answer is that making the member experience seamless, and ideally invisible, already takes a lot of engineers, but the tech organization also builds much more. It builds tools and products for studio productions, runs Netflix's advertising tech stack, provides developer platform and launch capabilities for games, and supports anything related to commerce: plans, pricing, payments, and partnerships.
Across all of this, Stone says Netflix captures more than a trillion events every day. These come from consumer interactions and from activity across the products and services that support decision-making. She describes the company as "quite a global enterprise at this point."
Technology built for the studio
The host notes that payments and ads are common at large tech companies, but custom software for a production studio is not. Stone calls bringing technology to entertainment "very much part of our superpower." Because Netflix is one of the biggest studios in the world, she says it can look for problems it is uniquely positioned to solve for productions.
Her main example is Netflix's media production suite. It replaced what Stone describes as a fairly antiquated, slow, and expensive way of moving media files between creative teams around the world. In her example, a production shoots somewhere in Europe while a reviewer in Los Angeles watches the daily footage. The files travel to Los Angeles, the reviewer adds notes, and the notes travel back in time for the next day of shooting. Netflix also builds other tools for tracking how productions are progressing.
Stone also mentions Scanline and Eyline, a visual effects studio Netflix acquired a few years ago. According to Stone, it does cutting-edge research and technology work on data capture and visual effects, and on techniques that bring productions to life in ways standard camera technology cannot easily achieve.
The host asks about the engineering challenges involved. Stone points first to scale: hundreds or thousands of productions may be in progress at once, and their media files are especially large, complex, and hard to move. That makes the cost of storage, compute, and data transfer a real concern. Latency requirements depend on the use case. Footage reviewed the next day can tolerate delay, but media for live productions must move essentially instantaneously. The other challenge she stresses is quality. Producing very high-quality images and video, both for the content itself and for how it is promoted on the service, creates many engineering problems of its own.
Open Connect: Netflix's own delivery network
Stone then turns to Open Connect, Netflix's content delivery network, which she calls extremely unique. She describes building it as a big bet Netflix made more than ten years ago. By her account, Open Connect now spans 6,000 locations in more than 175 countries. It places local copies of films, TV shows, and games close to members so they get low latency and high quality wherever they press play. Netflix integrates with internet service providers so the content can travel the "last mile" to a phone, TV, or laptop.
The host observes that engineers at most other companies would treat a CDN as a black box bought from a vendor, while Netflix engineers build it themselves. Stone agrees and calls Open Connect an incredible head start whenever Netflix moves into new content types. It became a strategic advantage when the company expanded into live and games, especially cloud-streamed games, and Netflix is extending it to deliver these new types of content.
"Pitch to play": engineering across the whole content lifecycle
Stone says Open Connect is really the end of a long, integrated lifecycle that Netflix has engineered from start to finish. Internally this is sometimes called "pitch to play." The lifecycle begins when someone pitches a title and the content team greenlights it. Data science and engineering teams support those programming decisions. Tech teams then support the production of the content through tools like the media production suite, including transferring files for quality review and checking alignment with the creative vision.
Once a title is ready, it moves through other pipelines. These check whether the promotional assets are ready, whether Netflix can recommend the title to the right audiences, and how the files should be encoded for delivery through Open Connect. Stone argues this is unusual because most companies have not built such a pipeline themselves. The host compares it to a CI/CD pipeline, and Stone suggests imagining that pipeline "times thousands," since it follows the entire cycle of bringing content to members.
Engineers, not top-down architecture, drive the system
The host asks whether such a long pipeline makes Netflix rigid. Stone says this is where Netflix's culture matters. According to Stone, much of Netflix's engineering systems, products, and tools were designed by individual contributors. Innovation came from within teams, not from the top down. Teams have substantial autonomy and rely on local judgment. Stone credits this with producing the end-to-end system, and with letting Netflix rearrange its "puzzle pieces" when new needs appear.
Live is her example. Many components already existed for film and TV on demand. When Netflix moved into live, engineers had to rethink content delivery for live's requirements. They started from what already existed and made their own decisions about how to evolve it. Stone contrasts this with an approach where someone says "let me draw the architecture for you" and everyone builds toward it.
She acknowledges that this has trade-offs. As the company grows, scale becomes more of a challenge, and Netflix has had to evolve how it builds things so the systems can support that growth. In her view, this flexibility is what lets Netflix engineer for its current needs rather than the needs it had ten years ago.
Building Netflix Live: 18 months to the largest streamed event
The host asks how Live came together and whether it was carefully planned or "yolo." Stone laughs and says it was "not quite yolo." Netflix's first live title was a Chris Rock special, which she believes aired in March 2023. The Jake Paul vs. Mike Tyson fight took place in November 2024, so Stone counts about 18 months from Netflix's first live event to what became the largest streamed event ever.
She says the work happened through urgency, scrappiness, and engineers taking ownership. The fight was originally scheduled for July 2024 and was moved to November because of Tyson's health, which gave the teams a few extra months. Teams from Open Connect, encoding, content production and promotion, and discovery decided who needed to be involved. Stone emphasizes that they self-organized: they wrote their own roadmaps, assigned their own owners, and identified which systems needed to become resilient enough for live. She describes the timeline as incredibly tight.
The event reached 65 million concurrent streams. Stone recalls it as one of Netflix's biggest days ever for signups. By the first couple of undercard fights, viewership had already passed Netflix's expectations for the main event. She describes the launch room in Los Gatos as full of excitement and nervousness, with engineers solving problems in real time "because no one had ever seen scale like that." She says she has never been prouder of the team for figuring out which levers to pull to keep the stream as stable as possible.
The event also had problems. Five weeks later, Netflix had to stream two NFL games on Christmas Day, where Stone says the bar for fans is very high. The team used what it learned from the fight to improve resilience. It worked out how to direct traffic if some markets became bandwidth-constrained and how to use quality levers to optimize the experience. By Stone's account, the NFL games "ended up being flawless."
She summarizes the progression as Chris Rock, then a Love Is Blind failure, then Paul–Tyson with many lessons at scale, then the NFL, and now weekly WWE plus other large events. She credits all of it to teams on the ground. She adds that learning fast does not mean avoiding failure; it means learning quickly from failures, iterating, and improving. She says she has seen the same pattern when Netflix built its own ads tech stack, launched games, and shipped its new TV UI.
Inside the control room
The host asks what the control room looked like, guessing it was full of dashboards. Stone says even the dashboards were brand new. The data science and engineering teams built them together for the event. They tracked core quality-of-experience metrics such as time to render, app start time, and rebuffer rates. She says rebuffers were the metric that started to climb during the Paul–Tyson fight.
About a hundred people were on site. Stone sat in a room with roughly 30 or 40 engineers and data scientists, working on laptops and makeshift screens. Everyone had a wired internet connection to avoid Wi-Fi risk, and VPN backups were ready. A launch commander wore a headset connected to the production truck. Stone stresses that it was not streamlined, not perfect, and not like the polished launch rooms she imagines most live productions have. When a metric turned red, the team created makeshift Google Meet rooms so small groups could triage. For each area, such as Open Connect, playback, or discovery (whether people could find the title at all), specific people were named as informed captains or decision-makers in the launch plan. She says the plan grew to 40 or 50 pages of if-then statements.
Stone jokes that she feels she "lost 10 years of my life in that one night." It was stressful, and she had nothing to do on a keyboard; her role was to support the team and trust it to make decisions. She says the NFL games, the Canelo–Crawford fight, and WWE are now far more sophisticated in resilience, metrics, dashboards, and visibility. The early events were very human-driven, and she says that is where many of the lessons came from. She tells her team it is rare to work at a mature company and still build something truly from scratch: to anticipate what could go wrong, prepare for it, and stay calm under pressure when something happens.
After the fight: learning without a mandated process
The host notes that the team had prepared extensively and still had problems with both Love Is Blind and the Paul–Tyson match, and asks how formal the follow-up was. Stone says Netflix does hold blameless post-mortems and retros, and that the learnings are more interesting than who did what wrong. But the process is not rigid. It happens organically and is led by the people closest to the work, who feel strong accountability for reflecting on what went well and what could be better.
She describes the days after the fight as a complicated mix of emotions. The team was celebrating the biggest live stream ever, a company willing to take such a big swing, and systems that did not collapse at 20, 30, and 40 million concurrent streams. If someone had told her beforehand that there would be 65 million, she says, she would have predicted it would not go well. Still, there were hiccups, and Netflix always wants to deliver a great member experience.
Stone says she was awake all night thinking about next steps, with only five weeks until the NFL. In the morning, she found memos the team had already written: what they observed, what could be improved, and what to prioritize immediately. One focus was traffic direction under congestion. The team compared what the algorithms had actually done with what they should do when congested, and looked at how the system could fall back or degrade gracefully under that kind of stress. Stone says none of this could have been designed before seeing how the systems behaved live.
She links this to a phrase in Netflix's culture memo, being "unusually responsible." In her view, it comes from high talent density and from treating people like adults: they get a lot of autonomy and in return they own the outcomes. She sees her own role as offering input and asking questions so she can understand and accurately represent what happened. A leader almost never has to tell the team what to do next.
Guardrails since Live: tiers, testing, and quiet periods
The host asks how many processes are mandated globally, such as required code reviews, feature flags, sign-offs, or CI checks that cannot be overridden, and how many are left to teams. Stone says a lot is left to teams and individual engineers, including early-career engineers. She points back to when Netflix introduced Chaos Monkey. The idea that each engineer is responsible for understanding how and when their system will break, and for detecting and recovering quickly, became a core part of the culture that Netflix has kept.
She describes Live as the dividing line. Before Live, the company had many years of experience with on-demand video. Teams could take smart risks and decide for themselves how much testing and resilience work they needed, backed by strong on-call and support teams. Live raised the stakes, because Netflix cannot be temporarily down during an event while it fixes a problem. Stone admits this was scary at first.
Netflix responded by introducing guardrails. Applications in the critical path for live, especially tier 0 and tier 1 services, face a higher bar for testing so they are ready for the stress of a live event. A central engineering team shares guidelines. Stone argues this actually gives teams less process: if a team meets the guidelines and completes the testing, it does not have to enter a quiet period during a live event. If it has done end-to-end testing, it understands its dependencies well enough to plan for failure. None of this, she says, is a structured gate like a mandatory code review or a checklist that blocks deployment. Netflix also has quiet periods over the year-end holidays, which she calls pretty common, and some "rules of the road" around live events.
The host summarizes the approach as focusing on the impact of each system rather than on process. Stone agrees and says these tools were new with Live. Live required more structure because it was new, because it was riskier, and because it touches so many teams. A live signal travels from the camera to the production truck, then to the origin or cloud, and then to the CDN, with many systems talking to each other in real time. Until Netflix was confident it understood those connection points, it wanted more guidelines, which is when it introduced the tiering system and the rules about who joins quiet periods. Stone says many of these constraints have since been relaxed. Netflix prefers fewer constraints because they slow down teams whose work has nothing to do with Live, and it does not want to slow the rest of the business for one priority.
Talent density and the arrival of levels
The host points out that Netflix could work this way partly because of its high hiring bar, and notes that for its first 25 years or so, Netflix's only engineering level was senior software engineer. Stone says she is still amazed by Netflix's talent density. Before joining a little over five years ago, she had not quite believed it.
In her view, the elements of the culture, including talent density, "no rules," little process, and "context, not control," are all means to excellent work, not ends in themselves. Operating for so long without levels, rules, or process sent people a message: we expect a lot of you. She calls it a very human reaction to rise to that expectation. The best people thrive with high autonomy and high accountability and are not distracted by what might otherwise surround them.
Keeping this up while growing from a hundred to a thousand to several thousand people requires what she calls "scaffolding." Netflix no longer has a single level. Not every role needs 10, 15, or 20 years of experience. Some are a good fit for recent graduates or people with a couple of years of experience, and those people should have different expectations and compensation. Stone says Netflix previously lacked even the vocabulary to talk about building a team with a broader range of experience. Levels also brought IC and management pathways and defined expectations at each level.
Stone emphasizes that these pathways cover cultural behavior as well as skills. Do you lift up the people around you? Do you deliver excellence and accountability? Do you show selflessness, good judgment, and a focus on what is best for Netflix? She cites several engineering principles. One is building for the future teams who will thank you for your work, which means not taking shortcuts and building high-quality, durable products. Another is "think globally, act locally": considering the wider effects on the tech organization and on Netflix when making local decisions. Her personal favorite is "yearn to learn," which she describes as a memeified way of saying be curious and ask whether you are solving the right problem in the right way. She also warns about incentives that push people to do what benefits themselves rather than their team or the company. Netflix tries to discourage this and to celebrate selfless, less visible work, which she believes keeps attracting and retaining the best people.
No formal performance reviews, and the keeper test
The host recalls performance review season as a month-long distraction every six months when they were a manager. Stone says Netflix has no formal performance reviews, which is probably the first unusual thing. There are no rating calibrations of the "meets / exceeds" kind she has seen elsewhere. Netflix still thinks carefully about feedback, performance, and expectations.
The foundation is meant to be continuous, timely, candid feedback, which Stone admits is "easier said than done." It requires trust and deep relationships, and it includes positive feedback as well as criticism. If people live the culture well, giving and receiving feedback should feel normal every day, with no need to wait for a cycle.
Around that sit several formal touch points. The first is an annual 360 process, which Stone calls a safety net. People request feedback from colleagues, the feedback goes directly to the individual, and each person reviews the themes with their manager. It is framed as feedback for improvement, not as an evaluation. The second is an annual compensation review based on Netflix's "personal top of market" philosophy. Managers consider each person's impact, skills, contributions, and value to Netflix and in the market. Stone says this has "a performance flavor" but is not a performance review. The third is promotion evaluation, which happens a couple of times a year and collects feedback for decisions such as moving from level five to level six. Stone argues these touch points together feel more constructive and actionable than typical review structures.
The approach demands a lot of manager attention and judgment, so Netflix adds checks and balances. As head of the tech organization, Stone reviews who is being promoted and how many, the themes in 360 feedback, and where compensation is landing across teams. Netflix also gives managers substantial support when they make keeper test decisions.
She explains the keeper test as a manager asking whether a person truly meets the expectations of the role and what the business needs. She stresses that it works in both directions. Employees ask whether they want to stay, whether the work excites them, and whether their manager is helping them grow. An employee can also use it to ask their manager directly how they are doing and whether there is feedback they have not yet heard. Ideally, she says, this all happens as part of normal business.
Why engineers stay or leave
The host cites data from SignalFire comparing tech companies' talent bar with retention, in which Netflix appears in the top corner: high-talent engineers were the least likely to leave. Stone says she is glad to hear it but that Netflix has to keep earning it.
In her view, people leave when their work does not give them enough challenge and fulfillment, or when they do not feel adequately recognized. No company can guarantee that, but she believes Netflix gives people many chances to solve hard problems with real agency, without heavy rules or top-down command and control, and with responsibility for both successes and failures. Retention is not 100 percent, and she thinks it is good for people to take great opportunities elsewhere; Netflix does not expect people to stay forever, only to feel they are doing the best work of their lives while they are there.
She also names two other factors. Leaders need to set a clear vision and strategy and make tough, timely decisions, and she believes people stay or leave partly based on whether the company's direction inspires them. She points to Netflix's newer bets and things being built from scratch for studios, advertisers, and members. Finally, people stay when they are impressed by the talent around them, which she sees as talent density reinforcing itself.
AI tools: pragmatic experimentation
On AI tools for engineers, Stone calls the topic a huge focus, approached "with a lot of intention and pragmatism." The goal is to find where the tools produce higher quality and more business impact. Uses that lower quality or are only about cutting costs are "really not interesting to us."
For coding assistants, Netflix offers teams many tools and lets them explore which ones fit which use cases. Stone acknowledges the learning curve: changing how you write code, document, and make decisions can be jarring, especially for very accomplished people. With tight timelines and big ambitions, there is little free time for this, so Netflix sets aside some weeks where people can focus on trying a new project or experimenting. It collects extensive feedback, much of it self-reported, on which tools help and which should be "graduated" to paved paths. "GenAI champions" across the business help teams troubleshoot, explain what is available, and report back to central teams.
Stone says Netflix does not treat generative AI as a silver bullet and prefers to be "surgical" about where it creates impact. This mirrors the company's approach for member-facing and creator-facing uses, and it shapes infrastructure strategy: giving access to many options, watching where the market solves problems well, and deciding what to build in-house. For internal tech productivity, she thinks the market is likely solving the problems well and Netflix gains little from building its own tools, but the company still wants to be selective about which ones it uses.
Where AI is showing the most promise
Asked whether certain areas are benefiting most, Stone lists several. Prototyping is much faster. She hopes cross-functional teams of engineers, data scientists, product managers, and designers can quickly turn an idea into a visualization or rough code to discuss it. That code is not necessarily production-ready or meant to become a product, and she says that is fine because it helps teams move ideas forward quickly.
She also points to tedious work that is not the coding itself, such as finding out how systems work, documenting code, and automating much of the big migrations Netflix has planned. The third area is detecting and responding to issues: anomaly detection, response, and deep investigations of problems, where she sees a lot of promise for resilience and engineering health. If AI can take over prototyping, documentation, migrations, and detection and response, she argues, engineers have more time for innovative work on architectures, systems, and products that drive business impact.
She repeats that it is not a silver bullet in any of these areas. She adds that the tools have improved greatly since Netflix first tried them a couple of years ago, when, as she puts it, "they didn't meet the quality bar that we really need."
New grads and senior talent
The host notes that Netflix began hiring early-career engineers a few years ago, while many companies now say they will hire only senior engineers until AI's impact is clearer. Stone says the experience with new grads, early-career engineers, and interns has been great. She notes Netflix started from a very different position. At some large tech companies, 30 to 50 percent of engineers were at level three or four, so she understands why a technology shift might lead them to rebalance. Netflix started at 0 percent in most cases, with mostly level five and above.
Early-career hires brought new skills, new perspectives, and energy, and in Stone's view they also bring native familiarity with AI, since recent graduates are used to using it to build products, write code, and solve data problems. She says Netflix will "absolutely" keep investing in early-career talent. At the same time, she is pushing to add more staff, principal, and distinguished engineers and scientists, because many problems need very senior people. Netflix is investing at both ends of the distribution.
She also frames this as developing talent from within. She hopes early-career hires grow into senior technical leaders, and she expects the most senior engineers to be good role models. Netflix now does more internal talent development than it would have five or ten years ago, which she says has been a huge boost.
Open source and encoding innovation
The host says one surprise from researching Netflix was its open source investment. Beyond the well-known Chaos Monkey, a recent report estimated that about one in five Netflix engineers work on open source projects, the highest share among the companies it compared. Stone says perhaps Netflix should talk about this more. She links it to talent density: strong engineers often want to contribute to the wider technical community. Some innovations are Netflix-specific IP kept as a competitive advantage, but many advance the industry in ways that also benefit Netflix.
Her main example is video encoding, where Netflix is heavily involved both internally and externally. She believes Netflix has won nine technical and engineering Emmys for this work and jokes that she used to associate Emmys only with TV and red carpets. The work improves the quality and efficiency of encoding and delivery, which benefits Netflix directly. Netflix is also a founding member of what she calls the Open Media Alliance, an industry group pushing for open advances in encoding technology. When the whole industry improves, Netflix benefits too, including in future integrations. She cites a figure: compared with when Netflix started producing originals, and despite a much larger catalog, she believes Netflix now needs about 60 percent less bandwidth for the same or better quality, which she credits to its encoding innovation. In her view, Netflix "doesn't lose anything, only gains something" by contributing. She also supports publishing more about its work, such as tech blog posts on what it took to build Live.
Advice for new engineers at Netflix
Asked how a new engineer can succeed at Netflix, Stone answers: "Curiosity. Curiosity. Curiosity." It is the value that resonates most with her. She means asking questions and challenging whether the team is solving the right problem in the right way. Being new or early in your career does not mean you cannot be a source of innovation, since great ideas come from everywhere. She encourages new engineers to be open-minded, experiment, take smart risks, and quiet the fearful inner voice that resists trying something new.
Her second piece of advice is to lean on other people. She says Netflix's engineers are happy to help others succeed, so newcomers should find mentors and ask why something works the way it does, what its history is, and which business problem it solves and why. She calls this another form of curiosity, but one that also draws on the wider community at Netflix.
In closing, the host says two points stood out most: how much open source Netflix contributes, with about one in five engineers involved, and how lightweight and continuous its performance management tries to be. Both, the host notes, feel quite different from how most other big tech companies operate.
What is the scale of the company from an engineer?
When you add that up, we have more than a trillion events that we're capturing every day between consumer interactions, things that are happening across products and services that support decision making.
Live was a big launch last year.
We had a lot of learnings from Paul Tyson because it was such a large event. I've often mentioned it was the world's largest, right?
65 million concurrent streams. Watching that tick up, I think one of our biggest ever days of signups. There were probably a hundred people on site. I was sitting in a room with maybe 30 or 40 both engineers and data scientists. We had our laptops and makeshift screens sitting there. When I think about where we were for Paul Tyson, I joke with people, I feel like I lost 10 years of my life in that one night. We don't have formal performance reviews, which is probably the first unusual thing. So, the way we approach it at Netflix is
Netflix needs no introduction, but its scale can still surprise many people. But what is it like to work at a streaming company as a software engineer? I sat down with Netflix CTO Elizabeth Stone to get more details.
In today's conversation, we cover the unique engineering challenges at Netflix, including the learnings from 3 years of Netflix Live. Netflix's engineering principles and why Elizabeth's favorite is yearn to learn. How Netflix has no performance reviews and what they do instead. How the company uses AI tools and why anime detection and analysis is a great use case that they found for them. And many more details. If you're interested in understanding more about how Netflix works as a software engineer and what it takes to do well in the kind of environment they operate in, then this episode is for you. This podcast episode is presented by Statsig, the unified platform for flags, analytics, experiments, and more. Check out the show notes below to learn more about them and our other season sponsor.
So, Elizabeth, welcome to the podcast.
Thank you. Thank you for having me and welcome to Netflix.
It is so nice to be here at Netflix. I'm sitting in a director chair. It has the Netflix on it. So, it's inside the Netflix offices, which truly feels special. And it just reminds me that Netflix is an entertainment company at its core.
Yeah. A lot of magic happens here. Even in the tech teams, we take things like video very seriously.
A lot of people know Netflix, of course, from the video offerings, the films, the movies. Behind the scenes, what is the scale of the company from an engineer? How can I make sense of how large this operation is?
It's probably larger than people realize. Very often I'll get questions from people in my personal life saying, well, how many engineers can it really take to build the Netflix product? So first of all, it takes quite a few when you think about how do you make the tech work so well that it's basically seamless, in some ways ideally invisible, because members just get to enjoy their experience. But then we also as a tech team build tools and products for studio productions, our advertising tech stack. We build a lot of the developer platform capabilities and launch capabilities for games. Think about anything related to commerce: plans, pricing, payments, partnerships. Those are all things that are supported by the tech team. So when you add that up, we have more than a trillion events that we're capturing every day between consumer interactions, things that are happening across products and services that support decision making. So it's quite a global enterprise at this point.
Some of the things that you mention I feel are kind of somewhat typical at other large companies. For example, building payments, building ads, building some of the things. You mentioned things that I haven't heard in any other company, which is, for example, building custom software for your production studio.
Yeah.
What are some of the kind of parts of the software stack or the software that you're building that it sounds like might be pretty unique to Netflix, that you might not have found elsewhere as an engineer?
Yeah, it's actually very much part of our superpower that we've been able to bring technology to entertainment. We're one of the biggest studios in the world and we have an advantage in thinking about what are some of the problems we could uniquely solve for those productions. So, good examples would be things like our media production suite, which took something that was a fairly antiquated, slow, and expensive way for media files to travel across creative teams around the world and really modernized what that looks like, so that if you have a production that's shooting somewhere in Europe and you have someone sitting in Los Angeles who's going to review the daily clips and footage, they're able to have those media files travel around the world, provide notes or input on that, have those notes travel back and be ready for another day of production just the next day. So things like that media production suite or other tools that allow us to monitor how are we progressing on some of the productions is something that's very novel from Netflix. We also have a big presence through Scanline and Eyeline, which is a visual effects studio that was an acquisition a few years ago that does really cutting edge research and technology for things that affect how we do data capture, visual effects, different ways to think about strategies that bring life to productions that wouldn't be easy just based on standard camera technology.
In terms of the engineering work behind that, what are some of the challenges here? Because what I'm hearing is you're saying, you know, like movie files, etc. To me what rings a bell is it'll be like large amounts of data, probably. I assume latency might be interesting challenges.
It's unbelievable scale when you think about hundreds or thousands of productions that are in progress at any given moment of time.
Oh yeah.
So across all those productions you have media files, which are also especially large, complex and difficult to move. So think about scale, think about cost of both storage, compute and travel of that data. Latency in some cases, you know, it depends on the use case. For some cases, it's capturing media that is going to be reviewed in a way that can be the next day, but for other things, we've got media traveling for things like live productions that has to be essentially instantaneous. The other thing I would say about some of the unique challenges in that space is the level of quality that we're trying to bring. So when you think about some of the challenges to think about how do you create very high quality images, videos, whether that's the content itself or the promotion of that content on the service, there's a lot of engineering challenges that come to meeting that type of bar. The other thing that's familiar externally, which I'm sure you've heard about, is Open Connect, our content delivery network.
Yeah.
Which is extremely unique that Netflix has that. It was a big bet that we made more than 10 years ago to build our own content delivery network, and the scale of that often surprises people. So 6,000 locations around the world, more than 175 countries, and that actually allows us to place local files for film, TV, games that you're going to play so that there can be very low latency and very high quality for members no matter where they click play.
Basically these are server locations at 6,000 different locations inside cities or whatnot where you have... it's like your edge network, right?
That's right. And we integrate with internet service providers to actually get the content to when someone clicks play on their phone, on their TV, on their laptop, that actually gets the content through that last mile to the member or the consumer.
And then when you're inside Netflix, you get to be exposed to some level of this detail, which I guess most engineers at other places wouldn't be. You would use a CDN provider and it will be a black box. But you're building this thing, right?
We're building this thing, and it's been an incredible head start as we think about new content types. So, when we started to go into live, into games, especially cloud or streaming games, as we think about just the breadth of our film and TV offering, Open Connect has been a huge strategic advantage and we're extending that to be able to deliver different types of content. The other thing that's unique is Open Connect as a content delivery or edge network is sort of the end of a very long integrated life cycle that content moves through. So I mentioned that media production suite that's on a studio production; files are being transferred for review, quality, making sure that we're aligning with the creative vision. Once a title is ready to launch, that flows through other pipelines that would think about: do we have the promotional assets? Are we ready to give great recommendations to the right audiences? How do we encode those files so that they're actually ready to be transmitted through Open Connect as the content delivery network? When you think about that, sometimes we lightly call it pitch to play, there's an element of engineering all the way along that life cycle, which is unusual because at many other companies, they haven't built that end-to-end pipeline themselves as Netflix has over time.
And what is pitch to play?
So think about from the moment a title is pitched, that someone in the content team greenlights, yes, we're going to develop and produce this title. There's data science teams, there's engineering teams that help to support those decisions on programming. Then there's tech teams that help support the creation of that content, the promotion of that content, the recommendations and ultimately the delivery of that. So tech basically underlies that whole life cycle.
Wow. So this sounds like a lot more workflows. Usually when I hear of a pipeline, you know, in engineering, we would think of a CI/CD pipeline, and I think we're very familiar with that. You know, you have your code reviews, test runs, and it kind of goes on. But what I understand is it's just a lot bigger.
Imagine that CI/CD pipeline times thousands, because it's actually going to follow an entire bring-content-to-life-for-members cycle.
This is a great time to mention Linear, our season sponsor. After all, Linear was born partially thanks to the pain points that happened during companies scaling up. The idea for Linear came about when their founders were going through hypergrowth phases at Airbnb, Coinbase, and Uber. As you'd expect with real scale, these companies started to slow down. What used to take days started taking weeks and sometimes even months. Not because people work less hard, but because there were a lot more moving parts that needed to be coordinated. Whenever you're adapting to scale, you often pick up new workflows, processes, just things you need to do over time. This creates real workflow debt. Software engineers often get hit the hardest here. Having to check five boxes and three labels when creating an issue just so someone's dashboard populates correctly. We've all been there. It's this accumulation of steps that slows orgs down and it frustrates engineering teams. For companies that have made the switch to Linear, like OpenAI, Coinbase, Scale, the move has been like a hard reset on all this process debt. The results are striking. Scale, for example, cut their bug resolution time in half by switching to Linear. If you're curious about making the switch, it's more straightforward than you think. Linear has native importers for Jira, GitHub Issues, Asana. You can even run them side by side during transition. The team is also happy to work with you to run a 4 to 6 week pilot alongside your existing tool just to prove the impact. Check out linear.app/switch. They have a migration guide that walks you through the entire process. And now back to the episode.
And now my first association seeing this longer pipeline is it sounds like it would be rigid, but of course Netflix is moving really fast. What does it look like from a software engineer's perspective, from an engineering team's perspective, getting a project done? How are these typically done? Is it based on some sort of following, you know, the schedule, or is it a lot more elastic?
Oh, this is probably where the uniqueness of Netflix's culture comes into play. So a lot of the way that our engineering systems, products and tools were built was highly driven by individual contributors thinking about how to build those systems. So innovation is really driven from within the teams rather than top down. There's a lot of autonomy and local judgment in how we build things that has allowed us to build this end-to-end view of how to deliver content in a way that we think delivers the best quality most efficiently and allows us to play with the puzzle pieces of that as we have new needs that come up. So like I mentioned, we had many of those things in place for video on demand film and TV. When we went into live, engineers needed to rethink how are we going to have to change how we think about how we're delivering content, given the requirements of live. They were able to start with what we've already built, but also have a lot of their own decision-making on the how, to evolve all of our systems and products to be able to deliver new content types. So over time the way Netflix has been built has been very driven by engineers within the teams rather than some, you know, top down, overarching 'let me draw the architecture for you and now let's build it in that direction,' which has both advantages but also things that we've had to evolve towards over time, because as the company becomes much bigger, scale becomes more of a challenge. We want to make sure that we're building things in a way that supports that. So it's not like it's static, and that elasticity, to use your word, is something that has allowed us to actually engineer for what Netflix requires today versus what it required 10 years ago.
Can we talk about specifically Live, because Live was a big launch last year. I remember one of the big ones, the boxing match between Jake Paul and Mike Tyson, was a huge event. Can you give us a little bit of insight of how that project started? How engineering teams got involved? Like, was it a small team? Was it multiple teams? I'm assuming there must have been multiple teams working together. And what was your process of getting this to launch? Like, was it overly planned out? Was it just YOLO? Something in between?
Yeah, YOLO. Not quite YOLO. Our first live title was a Chris Rock special, I believe in March 2023. That was our first time bringing live to Netflix members and started what was a very intense period of, if I take it through to that Paul Tyson match, that was November 2024. So you think about that as basically 18 months from our first ever outing on live to the largest streamed event ever, which is what Paul Tyson ended up being.
The way that came to life was with urgency, a lot of scrappiness, and like I mentioned, engineers making it happen. So, typically we would set a goal saying, "Okay, so we've got Paul Tyson scheduled." Originally it was scheduled for July 2024. It was rescheduled because of Tyson's health to November 2024, which gave us a few more months. But picture teams from Open Connect, encoding, our content production and promotion teams, our discovery teams thinking who are the right people to lean in here and help bring this to life. But they self-organize. They develop their own road maps. They think about who needs to be on point for what things. What are some of the systems that we need to make sure are actually resilient enough for live. It was an incredibly tight timeline end to end. Not to mention that we had a lot of learnings from Paul Tyson because it was such a large event.
Yeah.
I've often mentioned it was the world's largest, right?
65 million concurrent streams. Watching that tick up. I think one of our biggest
ever days of signups. So, we were watching the signups go through the roof that day. Then I think by the time we were in some of the first couple of undercard fights, we'd already exceeded our expectations for how big the fight would be. The energy in the launch room here in Los Gatos was palpable. You could feel excitement, nervousness, very real-time problem solving by engineers, because no one had ever seen scale like that. So you get your real-time figuring out how to deliver something in real time. I've never been prouder of the team for figuring out what are the levers we need to pull to keep this as stable as possible.
With those learnings in November 2024, we had about five weeks to be ready for two American NFL football games on Christmas Day, where the bar is very high to deliver well for members and for fans. And so the team immediately took the learnings from Paul-Tyson to say, how do we build greater resilience? How do we think about how we're going to direct content if we end up bandwidth constrained in some markets? How can we really optimize by using some of our quality levers for what that experience is going to be? And those NFL games ended up being flawless.
That from Chris Rock to a Love Is Blind failure to Paul-Tyson with lots of learnings at that scale to NFL and now weekly WWE, not to mention lots of other big events. That was all driven by teams on the ground being relentless in saying, "How do we do this well? What are the problems we need to solve?" And the accountability for learn fast doesn't mean we're not going to have failures, but when we have failures, learn fast, iterate, and deliver ever better. That's really where the beauty of the Netflix culture comes into play. And I've watched the same thing happen standing up our own ads tech stack, being able to deliver games, launching our new TV UI. Each of those was a group of engineers coming up with what's the best way to bring this to life.
You mentioned the control room, but I think most people have not had this experience. It's a live event, of course, you can tweak things. Can you explain to me what was the control room like? I assume it must have been a bunch of dashboards on all sorts of metrics, right? Was it a little bit like that?
Yeah. So, even our dashboards were brand new. So, they were not
You built it for that event. The engineering team put it together. The data science and engineering team collectively put together a set of dashboards that would give us visibility into some core quality of experience metrics. So things like time to render, app start timing, rebuffer rates. It was the rebuffers that we started to see amp up during Paul-Tyson.
There were probably a hundred people on site. I was sitting in a room with maybe 30 or 40 both engineers and data scientists. We had our laptops and makeshift screens sitting there. Everyone was hardlined into internet so that we weren't risking anything with the Wi-Fi. We had VPN backups if anything was to go wrong. We had a launch commander with a headset dialed in talking to people in the production truck, but it was new. So it wasn't streamlined. It wasn't perfect. It wasn't like the fancy launch rooms I imagine a lot of live productions typically have.
And so we're watching metrics, some things start flashing in red, causing you to draw attention to it. We would create makeshift Google Meet rooms, so small groups of people could triage. We had people who were on the hook for being the informed captains or decision makers. So if we have an issue with Open Connect, if we have an issue with playback, if we have an issue with discovery, are people actually able to find a title? There were people that we had in basically the launch plan. Imagine, I think the document ended up being 40 or 50 pages of if-then statements. If this thing happens, then what? It was new for us.
When I think about where we were for Paul-Tyson, I joke with people. I feel like I lost 10 years of my life in that one night because it was so stressful and there's so little, there's no hands-on-keyboard thing I can do to help. I'm there to support the team, to trust the team in making decisions. When I look now at how we're doing NFL games or the Canelo-Crawford fight or WWE, it's much more sophisticated in the sense of the resiliency we've built, the metrics and dashboards, the visibility we have into what's happening. That was very human-driven when we were first coming out of the gate, which was a lot of where the learnings came from. But I say to the team, how often do you get to work at a mature company, but build something truly from scratch like this and think about all the things that might go wrong and how to be prepared for them and then stay very calm and cool under pressure if something happens.
So what strikes me as very interesting about this story is you mentioned that the team had done a bunch of preparation. You mentioned 40 or 50 pages of if-then-else, which sounds way more detailed than I've seen most launches be. You usually have a launch plan, but again, it was a complex launch, so you prepared. Even so, there were hiccups, both with Love Is Blind, both with the Jake Paul match. Can you tell me how the team handled the aftermath? Again, it's pretty common to have blameless postmortems, but I'm more curious on how formal this process is, how less formal, driven by people stepping up. Or again, do you have more rigid process around this, or is it just getting together and people do go and fix things?
Like you said, it's not uncommon to have a blameless post-mortem or retro. So we have that for sure. It's much more interesting to talk about the learnings from something than to overly focus on who did what thing wrong. There's not much that we actually gain from that. It's not a rigid process. It happens very organically and it's led by the people who are close to the work itself and feel tons of accountability for doing reflections on both what went well and what can we do better.
If I take Paul-Tyson as a good example, it was a pretty complicated set of emotions a couple days afterwards. We're celebrating the biggest ever live stream event.
Yeah.
Way bigger than we ever could have hoped for. We're celebrating that we work at a company that takes such a big swing. We're celebrating that we didn't collapse when we got to 20 million, 30 million, 40 million concurrent streams. If someone said to me, you're going to have 65 million concurrent streams, I would have said, "This is not going to go well." And yet, yes, we have hiccups. We always want to deliver a great member experience.
I would like to say I woke up the next morning. I was awake the whole night thinking about what do we do next, and we only have five weeks till NFL. I woke up to a set of memos where the team had already written down: here's what we observed, here's what we think we can improve, here's some of the things we should immediately prioritize. So some of that was how do we direct traffic when we get congested? What were our algorithms doing to think about directing traffic in that moment versus what do we want them to do if we end up congested, and thinking about what are ways that we can gracefully fall back or degrade when we're under that type of duress. There was nothing in that event that we could have created before seeing what happens to the systems live.
And so just knowing, I woke up thinking all my, we're going to have to do this, we're going to have to do that. It was already there in a document. The team feels such accountability to how do we do this better that we don't really require a process to say now we should do a postmortem, now we should develop reflections on this. The team owns that very directly. We have language in our culture memo about being unusually responsible. That's really the talent on the team. It comes with high talent density. It comes with treating people like adults, where they get a lot of autonomy in making decisions and then they own the outcomes, and they have a mindset which is: there was a lot that was exciting about that, there's a lot we can do better, let's go do better now. So I'm there to help provide input, to ask some questions so that I understand and I can represent it accurately. But it almost never requires a leader to say now we must do the following thing, because it's so driven by the teams.
And I have to ask this question, but I'm sensing the answer to it already. When it comes to engineering culture at a lot of companies, you go into a company as a new engineer and you ask around saying, "Hey, what are the processes I need to follow?" Because at a lot of companies, there's a mandatory code review. If you launch a feature, you might need to use a feature flag. Let's say on the mobile app, on the code review you might need to have certain signoffs from people. So there's a bunch. CI/CD always needs to pass. You cannot override it. In the Netflix engineering team, how much of these things are kind of put down at a global level, everyone needs to follow it, teams can decide and do decide, versus just based on the judgment of the engineering team or the engineer themselves?
Yeah, a lot is left to the engineering team and engineer themselves. So even for new or early career talent, one of the things we've evolved towards, so even if I think going back however many years when Netflix introduced Chaos Monkey as a concept, the idea of an individual engineer having responsibility to understand how and when their system will break and how they're going to be resilient, detect that and recover quickly was just a core part of the culture and something that we continued to maintain.
A lot of the where we were a few years ago. Let me talk about kind of pre- and post-live, is probably the useful threshold. Pre-live, there were a lot of ways to take smart risks with video on demand because we had many years under our belt of understanding when something breaks, how are we going to fix that? And it was left to teams to think about the extent of testing and resilience that they needed to have, and great on-call teams and support teams for when something does go wrong.
When we introduced live, there's a different threshold because you have to watch it live. There's no such thing as Netflix is going to be down temporarily while we address something where we were taking a smart risk. That was scary at first, but as we started to introduce what are the guardrails we need to have to do this safely, it was things like introducing, especially for tier zero or tier one applications that were in the critical path for live, a higher threshold for what's the testing that you're doing, to make sure that your system is ready for the duress it may be under in a live event.
We will share those guidelines. It comes from our central engineering team and it gives people an opportunity to have less process, because they're able to say if I pass these guidelines, if I've done this testing, I don't need to be in a quiet period, for example, during a live event, or we've done end-to-end testing so we know those system dependencies very deeply and we're able to prepare for the what-ifs something goes wrong. But that's not a very structured process like a code review or you must check these boxes or some type of gating function for code being actually deployed. We do have quiet periods during the end of year holidays. I think that's pretty common. We'll have some rules of the road around live events to make sure we don't take unnecessary risk, but we are constantly finding ways to, how do we reduce the quiet periods? How do we leave a lot of judgment to the teams, and then they're accountable for anything that goes wrong with their service?
And do I sense it correctly that instead of the process, what you focus on is, let's say, the impact of the systems? So you have the tiering system, tier zero, I guess the most important one, tier one, and I guess goes down, and then the tools, for example, quiet periods or other ways that the teams can then kind of use to manage risk.
That's right. And a lot of those were new introductions once we entered live. So the teams were doing that very much on their own with their own judgment and accountability. Live, we ended up in a situation where we had to be more structured about it because it was new, because it was higher risk, because you must watch it live, and because live is something that touches so many teams. So going back to our conversation around our content pipeline, if you think about live from the camera to the production truck to the origin or cloud to then being able to get to our content delivery network, there's a lot of systems that need to talk to one another in real time for a live event. So there was a new set of things that could go wrong.
Until we were very confident that we knew those connection points, we wanted to introduce more guidelines for how to think about that. That's when we introduced the tiering system. It's when we introduced some things for which services or systems should think about being part of the quiet period or not. But we've already dialed a lot of those things down from when we first started with live, because our preference is to not have to have so many constraints, because it slows down too many teams to be able to make their own improvements and innovations that have nothing to do with live along the way. And we don't want to actually slow down other parts of the business in favor of just one priority area.
Elizabeth was just talking about how above a certain scale, Netflix had to put some basic guardrails in place for high-risk systems, but they kept being worried about introducing too many constraints because that would slow teams down. This problem of wanting both reliability and speed is not unique to Netflix. But the best companies figure out what the right tools are to enable the right culture. Whether it's continuous development or experimentation-first approaches, these cultural values need the right tooling and infrastructure. And this is where Statsig, our presenting sponsor, comes in.
Statsig built a unified platform that enables bold cultures, both continuous shipping and experimentation. Feature flags let you ship continuously with confidence, roll out to 10% of users, catch issues early, roll back instantly if needed. Built-in experimentation means every rollout automatically becomes a learning opportunity, with proper statistical analysis showing you exactly how features impact your metrics. And because it's all in one platform with the same product data, analytics, session replays, everything, teams across your organization can collaborate and make data-driven decisions.
Companies like Notion went from single-digit experiments per quarter to over 300 experiments with Statsig. They shipped over 600 features behind feature flags, moving fast while protecting against metric regression. Microsoft, Atlassian, and Brex use Statsig for the same reason. It's the infrastructure that enables both speed and reliability at scale. Speaking of scale, Statsig processes trillions of events per day. So whether you're a startup or building at OpenAI scale, the platform grows with you. You can integrate it into your existing product data stack easily. If you're interested in building a culture of continuous development and experimentation, go to statsig.com/pragmatic. They have a generous free tier, a $50,000 starter program, and affordable enterprise plans. Just tell them The Pragmatic Engineer sent you.
With this, let's get back to the conversation about Netflix's engineering culture. So, when engineers will be listening or watching this recording, a lot of them will just be nodding like, "Yeah, wow. I'd love to work at a place where we can make these decisions and decide how much risk we take and the process we set." One reason that you can do this, I know for a fact because I know engineers at Netflix, is you have a very high bar for
talent, and you've always had that from the very beginning. In fact, for the first 25 years of Netflix, if I'm correct, the only software engineering level used to be senior software engineer. Can you talk about how you go about hiring? What this bar is? And in your experience, how did Netflix manage to — it's the only company that for, again, decades only had this one level. There were no other things, and this was just this high bar. How did it work?
I continue to be amazed by the talent density at Netflix. I almost didn't believe it before I joined a little over five years ago, of, yeah, I'll believe it when I see it. I think the way to think about talent density at Netflix is a lot of the aspects of our culture, including talent density, are a means to excellence in our work. So none of them are the endgame. Things like no rules, no process, or high talent density, or context, not control — any of the things that we're talking about are key elements of getting to a group of people who strive to do the best possible work they can.
Being able to get through so much of Netflix's history without the complexity of levels or rules or process helped to signify to people, we're expecting a lot of you. And I find it's a very human thing when someone says, "I'm expecting a lot of you," that people step up and do the best work they can. So in some ways it builds upon itself to say, "We have high talent density. We expect excellence. You have a lot of autonomy but also a lot of accountability," and the best people will thrive in that situation. They're not distracted by a lot of the things that you could surround that with. They know that the bar is high and they want to meet that bar. All the people I work with feel that way. You don't need to tell them what to do. They lean in to try to do the best possible work.
Maintaining that over time, especially as we've scaled as a company, is a challenge. So anytime a team grows from 100 to a thousand to a few thousand, you have to think about what's the scaffolding you put around that to make sure we can maintain the spirit of what that culture was.
So, things that have changed over time: we don't still just have a single level. As we think about growing as an engineering or as a tech organization more broadly, not every role requires somebody who has 10, 15, 20 years of experience. Some roles are a great match for someone who's newly out of college or has a couple years of work experience, but you would want to think about the expectations for that person, the compensation for that person, being different. So we did start to build some scaffolding around levels, not to say now we want to have so much structure that it's sort of suffocating. We want to maintain all the great things about a lot of independence and accountability, but we didn't even have vocabulary to talk about how might I construct a team to have a broader array of talent. So that's one of the things we've changed over the last couple of years.
We've also had to think about — one of the things you get as you introduce levels are things like IC or people management pathways. What are the expectations that you have at each level? Some of that is about skills. Every company is going to have that reflected. But a big part of what we have in our pathways and our ways of talking are the cultural things. So do you uplift other people around you? Do you deliver a lot of excellence and accountability in your work? And we hold a high bar for people meeting those things. That's selflessness. That's good judgment. That's thinking about what's best for Netflix.
Some of our engineering principles are things like building for the future teams who are going to thank you for the work that you did today, which means don't take shortcuts, build high-quality, durable products. Things like think globally, act locally, meaning think about the broader ramifications across the tech organization or Netflix even as you make your local decision. And maybe my personal favorite, which is yearn to learn. It's a nice memeified phrase of be curious, think about am I thinking about the right problem in the right way?
That's how we've maintained high talent density, to say those are our ways of working and we expect that from everyone no matter what your level is. And you have to watch out for the incentives that things create. It's very easy to say I'm going to do the thing that I think puts me in a better position rather than my team or the company, and we really try to discourage that and really celebrate when people do the thing that's selfless or do the less glamorous or less visible work, as long as it's better for everyone else. I find that when people behave that way, it continues to attract and retain the best talent.
You mentioned that you're trying to not have things that distract people. Now, when I was a manager, one thing that did distract me, every six months on the dot — guess what? Performance review season. As we just called it, Perf. It was one month of my life, or especially at the end of the year, even longer. And whenever I talk with fellow engineering managers — I just caught up with a friend and he was saying, "Oh yeah, Perf is coming up, so I won't be able to help you in this period." How do you go about this necessary evil, or just necessary process, of performance management? Because I understand it's very different. And related to this, it's very publicly known, the keeper test, which Netflix shares on the website as well. How does this play into it, if it does at all?
Yeah, we don't have formal performance reviews, which is probably the first unusual thing. So when you think about other companies spending that time to talk through each person or assign a rating for whether they meet or exceed — I've seen that at other companies, too — we don't do it that way, but we do carefully think about feedback, performance, expectations, all the things that would feed into the keeper test, which I'm happy to talk through.
So the way we approach it at Netflix is first trying to get to something that looks like continuous, timely, candid feedback. Easier said than done. It requires trust. It requires deep relationships to be able to give someone in the moment very candid feedback. It could be, here's the thing you did great. It's not always a negative or a constructive thing. And to be able to receive that type of feedback. If we're living the Netflix culture well, that's something that would be familiar and comfortable every day of the year. So you're not having to wait for a certain performance review or feedback cycle in order to hear how you're doing or be able to provide that type of input to others.
We do, as kind of a safety net, have an annual 360 process where I would request feedback from a bunch of people I work with. I get requests from a bunch of people, but that's something where you're having a direct conversation with the individual about feedback. It's something I would review with my manager to say, "Here are some of the themes that I heard, some of the things I'm going to work on." So there's an opportunity to think about what is my performance? How are people perceiving our working relationship and my contributions? But it's not structured as an evaluation. It's structured in the context of feedback that helps people improve.
And then separately, once a year we go through compensation review, which is a reflection in ways of what's my level of impact, what skills have I gained, what are my contributions to the company. So you naturally talk about performance as part of thinking about someone's personal top of market, which is our compensation philosophy. So it comes up as a conversation there, where managers really think about, for each person on their team, how do I think about the compensation that reflects this person's value to Netflix and value in the market. So that has a performance flavor to it but is not a performance review.
And then a couple of times a year we evaluate promotions. So in that case, for a group of people who might be up for promotion from, you know, level five to level six, we would collect feedback that helps us make that decision.
So if I look across the feedback — continuous, in the 360 cycle, compensation review, and promotion evaluations — we get quite a few touch points where people are hearing how they're doing, but it feels more constructive and actionable than the performance review structure that other companies have.
What this requires us to do well is still a lot of manager attention and judgment. And it's not a manager in isolation being able to say I think you're meeting the keeper test or I think this should be your compensation. We do have structure around that so that managers are accountable for the decisions that they're making on their team. So if I think about my role in leading the tech organization, I review collectively who's getting promoted, how many people are getting promoted, what are the themes coming up in 360 feedback, where's compensation landing across the teams. So it's a little bit of checks and balances, because we do leave so much to a relatively unstructured process, and we try to provide a lot of support to managers when they are making keeper test decisions. So for context, that's asking the question of, is this person really meeting expectations for the role and what the business requires?
There's a keeper test that a manager might ask themselves and have that conversation with members of their team. But honestly, there's a keeper test that goes from members of their team to their managers. Do I want to stay? Am I excited about the work that I'm doing? Is my manager giving me growth and development, helping to guide my work in good ways? I want to make sure it's clear that we're all kind of accountable for that, instead of it just being a manager makes decisions for their teams. But the keeper test, and asking yourself that question, is a good way to make sure that we're all accountable for holding a high talent density bar in our team. It's also a good way for someone on my team to ask me, hey, how am I doing? Is there any feedback that maybe I haven't heard that I should know about to make sure I'm meeting expectations? And ideally, we do that in a way that feels like normal course of business.
So the keeper test is unusual. The fact that you don't do performance reviews is unusual, or at least not a structured cadence. Someone listening might think, well, that sounds really stressful. However, I looked at data from SignalFire. They had this chart with the retention of tech companies and the talent bar. So higher talent bar here, and retention, and Netflix comes in the top corner above all companies, which means that, based on the data they have, high engineering talent is the least likely to leave at Netflix across companies that are comparable. My question to you: why do you think people leave companies, and why are they staying?
Yeah, I'm glad to hear we were on the upper right, but we have to earn that over time. I personally find that people leave when they're not getting the challenges and the fulfillment that they would like to get from their work, or they don't feel like they're adequately recognized for the contributions that they're making. I don't know that any company can guarantee that everyone loves their job and feels perfectly recognized every day, but we get a lot of kind of at-bats on that for people to feel like I'm solving really hard, interesting problems. I have a lot of agency and autonomy on how I solve those problems. I don't feel constrained by a lot of rules or process. We don't have a top-down command and control culture that really narrows people's contributions, and we expect a lot of that responsibility for both the successes and the failures.
A lot of people love that environment. We fight really hard to maintain that type of environment so that people are excited to stay. It doesn't mean that our retention is 100%. People get great opportunities other places, and I actually think it's good for them to take it, because we're not saying we expect you to be at Netflix forever. We do want people to be excited here, to think they're doing the best work of their life. Sometimes they get opportunities elsewhere, but hopefully that's a positive experience they've had in terms of the work and the culture that they experienced at Netflix.
I also think managers and leaders have a big role to play in why people stay: in setting a vision, in setting a clear strategy, in making tough, timely decisions so that people can do their best work. And in my experience, sometimes people will decide to stay or leave based on that overall sense of, I feel really inspired by the direction we're taking. I think Netflix has had a lot to offer, especially with some of our newer bets and things that we're building from scratch and new experiences that we're bringing to studios or advertisers or members. So hopefully that also builds some of the enthusiasm.
And to finish my thought, I think people stay when they're impressed by the talent around them. This is where talent density builds on itself. If you really hold people to a high bar, great talent's much more likely to want to stay. So this is another reason to make sure we do a really good job with that.
So one of the very exciting things these days, of course, is AI and AI tools, both building with them but also using them as engineers. In your experience inside Netflix, how are the engineering teams using these AI tools for their own work? How are they experimenting with them? What is working? What is maybe not a great fit?
Yep. It's a huge area of focus for us right now, but with a lot of intention and pragmatism about where these tools are actually helpful versus where they're not. And again, the hope is that we identify those places where we actually get higher quality, more impact for the business, versus things that are lower quality or just about cost reduction. That's really not interesting to us. So across any technical application, but including GenAI, we're looking for the thing that is meaningfully advancing our impact for the company.
So for engineers, we are experimenting with a set of coding assistants. The way we approach it is to provide a lot of different tools to the teams so they are able to explore, experiment, decide which tools meet their needs, and start to learn what works better for some use cases or some applications than others. We're trying to create space so that people actually have the time to do that. As we all know, there's quite a learning curve when you're thinking about, I'm going to change how I write code, how I document, how I think about making decisions, especially for the people who are very accomplished in their roles. Changing your way of working can be kind of jarring. And we have tight timelines and big ambitions here, so there's not a lot of free time running around of, let me figure out how to use this new technology.
So we're doing things like enabling the tools. We have some weeks where we let people just be focused on, let me try a new project, let me experiment with something new, which gives people a little bit of space. And then we're collecting tons of feedback from across the team around which tools are actually useful, which do we want to graduate to paved paths, where are the areas where we actually see the most impact. A lot of that is self-reported. It's teams that are experimenting. We have what we call GenAI champions throughout the business, so they're able to help teams troubleshoot and understand what's available, but also feed back to central teams what's working and what's not, so we can continue to advance how we're approaching this.
And I do think we're doing a good job being pragmatic and not feeling like GenAI is a silver bullet for everything that engineering or technical teams are
doing. I think we want to be more surgical about where the impact comes from. And in some ways that reflects the same strategy we're taking for member-facing use cases or creator-facing use cases, where we're trying to figure out, like, where do we actually get a better experience and higher quality, and experiment with a ton of different capabilities, which also has implications for what's our infrastructure strategy? What's our overall strategy of, like, how do we give access to a lot of different options so we can experiment, and then think about where's the market solving this well, where should we build something in-house? It's very likely that for a lot of our tech productivity, the market is solving those problems very well. There's not really a big advantage to us building tools in-house, but we still want to be kind of choosy in which market tools we actually leverage.
You said you're getting a lot of feedback already. You know, people, teams, organizations are sharing what's working, what's not. Are you seeing some areas where these tools, specifically AI coding assistants, agentic tools, are maybe a little bit more helpful? Maybe that be greenfield things, migrations, prototyping, or some other areas?
You named a couple of them very well. So maybe starting at the end there, prototyping is a lot faster, and actually that's a place where, when you think about the cross-functional teams across engineers, data scientists, product managers, designers, we're hoping that we can actually bootstrap things very quickly. I have an idea. Let's visualize that idea. Let's quickly throw together a set of code that would help to bring this to life. That's not necessarily something we would productize or consider production-ready code, but that's okay because it helps teams advance and innovate and kind of workshop ideas very quickly. And then, you know, as you probably know and your listeners know, there's a lot of what can be tedious work. And it's not always the actual coding work where it feels like it's the biggest time commitment. It can be accessing knowledge about how systems work. It can be documenting code. It can be thinking about big migrations that we've had on our plate that we can actually automate much of that work.
And there's also things around detecting issues. So anomaly detection, response, being able to do deep dives of issues. We're finding that there's a lot of promise for GenAI tools in that space, which helps us with some of our resilience and just general best practices and health as an engineering organization.
If we're able to use GenAI tools in those spaces, prototyping, documentation, migrations, detection and response, it leaves a lot of time for the more innovative work. So, how do we think about architectures and systems and products that we're building to deliver business impact? So then hopefully we can actually get more impact for engineers because they're able to actually leverage some of the tools or agentic experiences to minimize the time spent on some of the less impactful activities. But it's really a portfolio of work, and I would say again it's not a silver bullet in any of those spaces. I would say it's come a long way since some of the tools we first started experimenting with a couple years ago, which, let's just say, they didn't meet the quality bar that we really need.
Yeah. No, it's been a big change on this. One thing that's unique to Netflix is you mentioned how for a long time Netflix only hired senior software engineers, senior and above software engineers. About two years ago or so, a few years ago, you've now started to hire earlier career software engineers. Can you tell me how that has changed the culture at Netflix? What you've learned by hiring, and what is your strategy? And are you planning to keep bringing in new grads or interns or early career folks, or are you planning to do... because again, a lot of companies these days are saying, oh, let's just go with seniors for now, at least, especially until we figure out this whole AI thing.
Yeah, we've had a great experience with new grads and early career talent and also our internship program, which you mentioned. We were starting from a very different place than a lot of other tech companies. So when you look at the distribution of levels or talent at some of the other larger tech companies, they had in some cases 30, 40, 50% what I'll call, you know, level three, level four engineers. So when you think about a new technology shift or the work that those companies need to do now, I understand why they might need a different distribution of talent. We were starting at 0% in most cases.
Crazy.
Yeah. Which is why, you know, so we had mostly a level five and above population. So we had a huge opportunity to complement the team we had with earlier career talent, who brings new skills, new perspectives, great energy to the teams, and with a technology shift right now with GenAI, a lot of native AI familiarity. So when you think about somebody who's graduated from school in the last few years, they're very accustomed to using AI, whether it's developing products, writing code, thinking about solving data problems. So it's actually a useful way to bring new skills and perspectives to the team.
I think we will absolutely maintain that investment in earlier career talent because it's been so additive in different parts of the business. I also think everything in its right proportion. Like, there's plenty of problems where we need extremely senior talent. So at the same time, I would say I'm pushing for us to also think about how do we add more staff, principal, distinguished engineers and scientists to the team, because that's also a place, you know, you think about that tail of the distribution being a really important place to maintain strong talent. So we're investing in both of those tails of the distribution.
I love it, because I usually hear companies talk about one or the other, but not both. And I guess at some point, you know, the people here one day will hopefully be there.
Another way to think about it is building talent from within. So we hope that a lot of the early career talent joining has a great experience, develops skills and impact here, and becomes those more senior technical talent over time. And our most senior technical talent have to be great role models there too. So we're doing more of that internally, and that talent development, than we would have done five or 10 years ago, and I would say it's been a huge boost to the team.
One of the most surprising things I've learned about Netflix just very recently is how much you invest in open source. This might sound a bit silly because we know Chaos Monkey is very famous. In fact, everyone knows Netflix is known for that one. But again, a recent report looked at all the companies and what percentage of engineers end up working on open source. And Netflix again was at the very highest bar. This publication estimated that about one in five engineers work on open source projects. Sure enough, I go to your open source page. It's just so much open source. Can you tell me why and how and since when is Netflix doing so much open source, and why do we not know about this? This was new to me.
Oh yeah, perhaps we should be talking about it more. So this is a good opportunity to start. You know, we were talking earlier about the engineering culture and the sense of, you know, talent density, which often comes from a passion to contribute to the broader technical community. So the people at Netflix care deeply about the quality of their work and advancing innovation more generally. For some things, it's Netflix-specific innovation, and it's important we keep that IP as a competitive advantage, but for many, it's something that helps to actually drive broader industry innovation, which also benefits Netflix over time. So if I can give you one example among the list of places where we've been very involved both internally and externally, it's in the encoding space, and driving a ton of innovation, right, video encoding. We've now won, I believe, nine Emmys for these contributions. I always used to associate Emmys, you know, just with the TV and, you know, the red carpet, but we've won a lot of technical and engineering Emmys at this point, specifically on video encoding work.
So, as one example, that helps to contribute incredibly to quality and efficiency of our ability to encode our titles and deliver them. So Netflix gets an immediate benefit by improving the technology in that space. But we are also a founding member of the Open Media Alliance, which is an industry community that pushes for open advancement of encoding technology. If we're able to inspire that work, Netflix actually also benefits because the whole industry uplevels, and we think about integrations with different technologies that we might do over time with everyone helping to push the bar. A statistic I like to cite is, when you look at the catalog now of Netflix content, think about how much bigger the catalog is than when we were first starting with originals. I believe we now require 60% less bandwidth, 60% fewer bit rates for same or better quality with a much bigger catalog. That comes from our media encoding innovation. And having a whole industry that's pushing that benefits anyone who's in the entertainment space and definitely benefits consumers and our members. So that's a good example where it starts from an open source contribution. Netflix doesn't lose anything, only gains something by contributing to the broader innovation landscape. And I'm a strong proponent of talking more about the innovation we're driving. So things like different blog posts that show up in our tech blog, for example, we're talking more about what it took to bring Live to life, as one example. I just think it's a great contribution to the broader community.
This is one of the reasons I love software engineering, because I feel contributing to the open and sharing things, it lifts the tide for everyone.
Yeah, I believe so. Yeah, we definitely benefit, and we are trying to drive better outcomes overall, especially with a real member focus, and so a lot of the technology we're building is able to do that.
So as closing, Netflix sounds like a very different and special place compared even across all of the larger tech companies or even the innovative companies. What would your advice be for a new-start software engineer starting at Netflix? How can they succeed in this environment, and how can they grow up to the expectations at a place like this?
Curiosity. Curiosity. Curiosity. When people ask me, like, what's the Netflix value that most resonates with me and I most love to see across the team, it's curiosity. Asking questions, questioning whether we're solving the right problems in the right way. Just because you're new to Netflix or earlier in the career doesn't mean you're not going to be the source of innovation. If anything, great ideas come from everywhere, and that starts with just being curious, open-minded. Experiment, explore, take smart risks. Try to reduce that kind of voice in your head that is fearful of exploring something new or taking that risk. And I think when people join Netflix and they approach it with that type of curious mindset, they're already set up for success. I would also say lean on other people. So, we have great talent at Netflix, and they are all more than happy to help other people be successful. So don't shy away from finding a mentor, asking somebody, why does this work this way? Can you give me more of the history of this? Can you help me understand which business problem we're solving and why? It's another flavor of curiosity, but it's also about the broader community and really leveraging that at Netflix.
Well, Elizabeth, thank you. This was very, very interesting and I've learned a lot.
Great. Really happy to be here, and I was happy it worked out to have this conversation.
Thank you.
Thanks.
One of the most interesting learnings for me about Netflix was just how much open source they contribute to, and how about one in five engineers is involved in open source work. The other one was how performance management is really lightweight and tries to be truly continuous. Both of these things feel like they're quite different from how most other big tech companies operate. I previously did a deep dive on how Netflix's engineering levels changed from the single senior level to the new five levels. Check out this Pragmatic Engineer deep dive in the show notes link below, as well as deep dives on the engineering culture of other big tech companies like Meta, Amazon and Google. If you enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating for the show. Thanks and see you in the next
Article published