Josh Woodward on Running Google Like a Lab: "Almost Possible" Ideas, Reading Eyes, and Knowing When to Quit
Lenny's Podcast
Josh Woodward leads Google Labs, the Gemini app, and AI Studio. In this conversation, host Lenny Rachitsky opens with a thesis: every company now needs to think and operate like a labs team. The models underneath products change so quickly that what can be built keeps shifting, so companies have to keep trying things with the latest models and find out what has just become possible before a competitor or startup does. Woodward agrees with the premise and adds that "the devil's in the details." In his view every startup is already a lab. The harder question is how to bring that methodology, energy, and excitement into medium-sized and large teams, and how to build an environment where both the ideas and the people who generate them can thrive.
The rest of the conversation covers where Woodward has seen good ideas come from, how Google Labs tracks what is "almost possible," what early product-market fit looks like, when to kill a project, which skills Woodward thinks are rising in value, how planning has changed, and why most corporate labs fail.
Good ideas don't come from scheduled sessions
Asked what patterns explain where the best ideas come from, Woodward says he wishes there were "an answer in the back of the book." In his experience, ideas come from many places. He draws three lessons.
The first is to stay on alert for anything that catches your eye. Good ideas have very rarely come from design sprints or from time blocked on a calendar to "come up with our next idea." They tend to show up in hallway conversations, on weekends, or over breaks. Woodward describes Labs team members who "just can't help themselves" and come back from time off with something they built, or who share a video and say, "look at this amazing thing."
The second lesson is that ideas don't appear in conventional places. "You can't microwave good ideas," he says.
The third lesson is that the idea often matters less than the people and their side projects. Over the past four or five years in Labs, Woodward says, projects such as NotebookLM, Flow, AI Studio, and Google Beam all began with very small groups of people who were obsessed with and curious about something and "just won't stop." The leadership job is to create a space where that can happen, notice it when it does, and help move it forward. Lenny notes that brainstorm meetings were also missing from Woodward's list of sources.
The "almost possible" list and 82 predictions
For people who want something more actionable, such as a founder or someone inside a large company, Woodward describes how his team gets to the frontier. That means playing with the models and also watching which experiences have just become possible. The team uses the phrase "what's almost possible" and keeps an actual list of such things. When an item crosses over, Woodward compares it to a phase shift, like a substance going from liquid to gas. He says he saw one of these two days before the recording and can't stop thinking about it, but declines to say what it was beyond "maybe soon." Once something crosses that line, the team tries to put people around it "and just go."
Technology alone isn't enough, he adds. The capability has to be matched to an interesting user problem, and ideally combined so well that people will pay for it. That combination is what Labs looks for early.
Asked for examples from the list, Woodward describes the team's approach rather than specific items. Labs runs thought experiments about the future of particular domains: software design and development, creativity, work, knowledge. One area they are watching closely now is entertainment, especially immersive entertainment. Under that heading the team keeps sub-lists such as "if a model can get to this kind of latency and this amount of languages in this amount of modality," which helps them reason about what becomes interesting technically. Alongside the technical side, Labs studies where behaviors and trends are heading. Woodward says he keeps pushing teams to look well outside the Bay Area, to other cities, classrooms, and places where they can "mine for ideas" and observe behavioral change.
Lenny sums this up as preparation rather than waiting for inspiration. Woodward confirms it and adds a detail: the document he mentioned contains 82 predictions, each starting with "we believe the future is" or "we predict." He admits making even one prediction is dangerous. Still, he argues you need a point of view about the future while accepting that nine out of ten, "or maybe 9.9 out of 10," will be wrong. "Chasing the future" is one of the team's values, and he says doing it requires throwing yourself out there and trying to live in that future a little.
Separating a real opportunity from a dazzling demo
How does Woodward tell whether an idea is worth pursuing? Part of his signal is personal. Being at the frontier and noticing what surprises or excites him, like the demo he left 48 hours earlier thinking "this is so exciting," and then feeding off that excitement with the team.
Lenny objects that excitement is a low bar now that so much feels magical, or that people have become numb to it. Woodward agrees that excitement is mostly a technology signal and that "there's dazzling demos every other day or every other hour right now." So he also looks for a user behavior that is either universal or shifting, and for monetization: can you reach a group of people who can't live without the thing and will pay because it is so valuable to them? A good idea, he says, has to start from at least one of these areas, and the work is putting them together.
He gives two examples. The first is hearing the first NotebookLM-style podcast. He remembers where he was standing, right outside the room where the interview was recorded, around 6:30 p.m. on what he thinks was a Tuesday. Two team members came over and told him he had to listen. The source material was the proceedings of British Parliament debates, "not known for its riveting material," yet two AI hosts discussed it in a way that was genuinely interesting. The team's reaction was that they had to do something with it.
The second example is from Google Flow. Before Flow there was a project called Whisk, which let users "whisk together" multiple images and then animate them. Woodward describes the level of creative control, and how accessible it was, as something that had never been possible before.
He adds that there are also cases where the team gets excited, builds something, and the product "is just not good." Part of the work, he says, is a numbers game.
Early product-market fit: watching people's eyes
Woodward's first point about product-market fit is that it is "way more art than science." He wishes it were science, since teams know how to run experiments, make data-driven decisions, and set OKRs. But early on, it can come down to a couple of people with conviction who think the future will look a certain way and can't fully explain why. He acknowledges this isn't very useful advice.
His more practical advice is to get outside the building and show people prototypes. When he does, "you're looking at people's eyes and like that is the metric." It isn't DAU/MAU or D7 retention. It is whether someone's eyes light up and whether they lean in. Lenny mentions someone else describing this as pupils dilating, and Woodward agrees. You can tell, he says, whether someone is enjoying it or "bored out of their mind" while you show them "some dumb product." The question is whether you've hit a nerve, a pain point where the person thinks "I want that."
His third point is to fall in love with the problem rather than the product. He finds it takes three, four, or five pivots before a team either has something or concludes it didn't work. In his experience, people who fall in love with a specific product solution first are headed for "a bad time."
He agrees with the common saying that you know PMF when you have it. Over roughly the past year, he lists moments when Nano Banana went viral in Gemini, when NotebookLM's podcasts spread, and when Flow, Stitch, and other projects had "mini moments." In those moments the team is "literally just swarming it and holding on for dear life," which he calls both exhausting and fun. His overall advice is to take many shots, accepting that the early stage is driven more by art and passion than by science.
When to kill an idea: the team usually knows first
Woodward says teams sometimes expect the leader to make the decision to stop a project. He has found the opposite: "The team usually knows before the leader knows." The signs are that passion runs out, or that people have tried everything they can think of and it still isn't working. He compares it to sensing that a relationship isn't working. As a leader, his job is to build an environment where people can ask uncomfortable questions and say, "I don't think this is working."
He gives an example from the day before the recording. The Gemini team had a feature they were excited about and planned to launch. Early data came back, people went out and "looked in the eyes," and the response wasn't there. The PM, whom Woodward refers to as "she," said she didn't think they should launch because it wasn't good. Woodward says he was proud of her and replied-all to thank her. The broader goal is a culture where the team can call out both good and bad results. He says he trusts his teams heavily because they work on the product every day, and if they say it isn't working, the response is to pivot and try something different.
Why one person runs both Labs and the Gemini app
Lenny asks why one leader oversees both Labs, which does zero-to-one experiments, and the Gemini app, which Woodward says has more than a billion users globally. Woodward acknowledges they sit at opposite ends of the scale. He jokes that Labs teams get excited about 10,000 monthly active users, a number most product dashboards don't even show.
He names three things the two share. First, the people. Despite its size, Gemini still has "a zero to one heartbeat," and both attract similar builders, with lines blurring between what each requires. Second, everyone involved is "AI kind of nuts and maniacs" who are building constantly. Third, a shared ordering of priorities: users first, Google second, and their own product (Labs or Gemini) third. The team considers itself "very much part of team Google," and the test is whether someone on the other end loves the product.
Woodward also confirms that all of Labs is currently AI-focused. When the current Labs began four or five years ago, it explored a few other areas, then decided to go all-in on AI for the foreseeable future. The work isn't only software. He points to Google Beam, which he describes as a "magic mirror" that creates a 3D version of the person you're talking to, so the experience is almost like talking to a hologram. He calls this adjacent to AI, "almost like AI hardware." He offers to get Lenny one, and Lenny accepts.
Where consumer AI goes next, and the personal assistant question
Lenny suggests consumer once seemed "dead," dominated by incumbents, and that AI has reopened it. Woodward says consumer feels to him like one of the most exciting and competitive areas right now. He names three areas.
The first is entertainment. Will people keep looking at different kinds of feeds, or can the technology bring people closer together in the real world? He notes there are competing views on whether that future centers on bots or on people.
The second is messaging and chat. That space once felt "sort of maybe solved," but he sees many experiments happening inside and outside Google, so it no longer feels as settled.
The third returns to big problems and scarce things: time on earth, how much money people have and how they can save it, and memories and in-person experiences people share. He thinks new consumer products could address these, and says the form factor, whether a chatbot, a personal agent, or something else, is largely "yet to be invented."
Lenny then asks what he calls a spicy question. Many AI assistants plug into Gmail, Google Calendar, and Docs and do impressive things with that data, while Google, from Lenny's perspective, hasn't shipped an equivalent assistant. Is the obstacle privacy, bureaucracy, or is it coming?
Woodward says parts of it already exist. In Gemini Spark, users can connect all of that and it is "quite powerful." Where Google is iterating is the interaction model. He says the industry has relied on separate modes users switch into, and the lesson "even from the last couple of weeks" is to "forget the modes and toggles" in favor of one prompt box where you say what you want and it does the work. He describes this as a UX simplification happening across the industry, not only at Google. On data, he says one of the most common requests he hears is some version of "Google, I've got all my stuff with you. Just make the thing work," and that many people are reluctant to connect that data to competitor products. "Watch this space. We hear you loud and clear," he says. He points to Google's "personal intelligence" work, a Gemini that is personal, proactive, and powerful, as the vision for roughly the past year, and says a lot of what the team is working on will be out soon. Lenny comments that Google's advantage is distribution and mentions Meta's Muse and Google's AI Mode as examples of products that grew easily on existing distribution.
Underhyped: principles. Overhyped: benchmarks.
For underhyped, Woodward points to discussion of principles and values. Over the summer he wrote an internal document on Labs principles, covering how and why the team builds what it builds. He believes strongly that, whether at Google or a startup, "you speak through your products," and that products encode their makers' principles. He wishes the industry talked more about what it wants to build and what future it wants. Lenny connects this to an earlier conversation with Tara Seshan of OpenAI: when almost anything can be built, the question becomes what world we want, not just what we can do.
For overhyped, Woodward repeats a position he says he has held for about a year: model benchmarks. He says friends at many AI labs still look at mostly the same benchmarks. But given where he grew up and how he approaches product development, he believes most people in the world will never care about an Elo rating, and "they may not even know how to pronounce" it. He sees industry talk dominated by "speeds and feeds and benchmarks." He accepts these are proxies for capability but thinks the industry over-rotates on them and loses sight of whether something is useful and which product or feature will deliver real value. Lenny adds that it's hard to tell whether a new "best model ever" is meaningfully different in practice. Woodward's message to product people is to build great products, since all that intelligence ultimately has to become something people want. With so many "almost possible" things becoming possible each week, he calls this "what a time to be alive as a product person."
Skills rising in value: unlearning rate, explosive endurance, and trust
Woodward says much has been said about taste, judgment, and vibe coding, so he looks for "mispriced signals." He names three.
The first is unlearning rate. People often talk about someone's learning rate or growth mindset. Woodward is looking for how fast someone can learn something and then walk away from it, for example based on what they see when they test their product.
The second is explosive endurance, a phrase from a book about Roger Federer that he read about a year earlier. He takes it to mean the ability to go very hard (the explosive part) and sustain it over a long period by taking care of yourself and the people around you and finding ways to pace yourself.
The third is collaboration and trust. Even as coding moves toward directing many agents and working with fewer people, Woodward thinks how you work with people matters more now. He looks for people who can be excellent collaborators with an agent or a swarm of agents and also with humans. The Labs and Gemini teams believe there is still "so much joy in working with people." The underlying skill is the ability to build and scale trust, and how quickly someone can do it.
Asked why trust is gaining importance, he says part of it is specific to Google as a large organization. He gives two reasons that aren't. As the cost of executing on products falls, more products will be made, and people will team up with different collaborators more often. Labs project teams used to be about five to seven people. Now, he says, they are about two to three, maybe four. He compares the strongest people to talented musicians who form different groups and play different songs. The most valuable people in Labs are those you can drop onto a project, such as a PM who can also code and design, and who can jam with others. With more products and more of these "free agents," he predicts trust-building will become even more important.
Roles aren't blurring into one universal builder
Lenny asks whether any function has become more successful at leading Labs projects. Woodward says the team has been studying and testing this. PM, perhaps because it is more horizontal, seems to have adapted to other functions faster. This shows up in "builder pilots" run across the company. He suggests PMs are used to simulating other perspectives in a meeting, such as legal, business development, or sales, and that the kind of person who chooses PM may be predisposed to this. Still, he says the best people come from every function, including strategy, operations, business development, and marketing, all of whom have become strong builders in Labs.
He then offers another overhyped item: the idea that "everybody's a builder," that this is one new job function, and that all roles are converging. What he sees is that people keep their specialty, which he compares to a college major or minor. If everyone becomes a generic builder, he worries the field loses respect and appreciation for specialties and the expertise behind them. Even when Labs assembles "more of like a builder crew," it still plans for which specialties each team needs.
His advice to new PMs and APMs is that vibe coding and prototyping are great, but they should serve the goal of becoming an exceptional PM. "Don't lose that major," especially early in a career.
Rhythms for sustained intensity: seasons, naming the mode, and fallow periods
Lenny notes a recurring theme on the podcast: everyone feels like they're running a marathon and a sprint at once. Woodward describes how Labs handles this. At any time there are 20 or 30 experiments running, and for each small team the goal is milestones and a rhythm. Some periods are "flat out full-on." The most useful lesson he has learned as a leader is to name and declare those periods, let people opt in or out, and be equally clear when a team is not in that mode. He gives an example: a big model is coming, five products matter for showcasing it, and that is "explosive time." He says PMs are often effective at creating the shorthand or common language that makes these modes clear.
After intense periods, the mode changes deliberately. After Google I/O, many Labs teams sprint, and then June becomes a time to "just go hack, go rediscover what's new, go just go build," and have fun. The temptation, he says, is to see June as unproductive because nothing ships, but that's where the seeds for the next season come from. He admits the team doesn't always get this right. He prefers framing like explosive endurance over older "work-life balance" framing because the goal is to do this as a team for many years: "you can't just like empty the tank in 9 months," but you also have to recognize that the current moment is fast.
Lenny adds that the opposite is also dangerous, since people with too little to do get bored and start looking for other jobs. Woodward agrees and mentions a team saying, "never waste an event": whether it's I/O, a model release, or for a startup some outside moment, it helps to plan around where you want to intersect with the world.
Planning on six-month windows and 100-day milestones
Woodward says someone recently asked about Gemini's two-year roadmap, and his answer was that it has a six-month roadmap alongside a multi-year vision. The world changes too fast, he says. The thing he saw two days earlier became a project within 48 hours, which can't be planned for in advance.
Labs typically plans on a horizon of about six months. He ties that to the "almost possible" window: he assumes each AI lab produces new pretrained models a couple of times a year, with smaller releases in between, which sets an overall cadence. Within that, the team thinks in roughly 100-day chunks about how far a product can move. He says the best Labs teams go from idea to a meaningful milestone in 50 to 100 days. Making that possible means designing a culture that removes blockers so teams can "just let people cook." He describes the overall approach as "rolling six months."
Frontier teams, and why most corporate labs fail
Lenny brings up a talk Dan Shipper gave at the Lenny & Friends Summit, which argued that companies should have a frontier team whose job is to try new tools and report back, so the rest of the company can stay focused. Woodward agrees that every company needs such a group, however it's structured. These people should be so plugged in that if they haven't heard about the latest viral thing on X in the past 24 hours, you'd wonder what they're doing. "They're the people who have like eight $200 a month subscriptions on everything."
He says that is only the first of three requirements. The second is an environment where those people can actually build. The third is the one he thinks most often decides success. He has read several books on the history of corporate labs, and he says the data shows that when a large company starts a lab, it is usually ineffective and fizzles out after three to four years. If it does last, the company often can't commercialize its output, and someone sees the future there and builds it as a separate company, which he says has famous historical examples. The hard part is building a frontier team that can connect to the larger company so it isn't dismissed as "some weird R&D group always coming up with like gimmicks and toys." That takes people who can say, "that thing looks like a toy right now. That's going to be essential to our future in five years." In his reading of history, teams can get the first two parts right, fail on the third, and then none of it matters.
Lenny notes that even Anthropic, already at the frontier, has a labs team. Woodward then describes an internal risk at Google Labs. As some projects such as NotebookLM, Flow, and AI Studio grew large, they risked crowding out new zero-to-one work. So the team now talks about "zero to one labs," "one to 10 labs," and "10 to 100" labs as distinct lifecycle stages. All should share the same innovative energy, but each faces different challenges and optimizes for different things. At zero to one, you're looking at eyes. At 10 to 100, you're looking at conversion waterfalls and CPAs and at how to get people to the product's "magic moments." He calls the later stage "a scaling game."
Setting up an internal lab: independence, a clear game, and "good trouble"
Woodward offers three tips for companies starting a lab.
First, create a space "where weird things can grow." He says you can't place it under an existing business unit. It needs independence, which in some companies means reporting to the CEO or reporting far enough away from the core business.
Second, set the right mindset and be clear about the goal. At Google this is the "users first, Google second, labs third" ordering: the lab exists to make users and Google successful, not itself, which requires humility and collaboration. Companies should decide whether the lab's purpose is to graduate things into existing products or to invent new product lines and categories. Google Labs started as a graduation lab and has broadened over time as the team realized AI transforms and blurs categories. Knowing which game you're playing defines winning and losing. Lenny summarizes this as being isolated in some ways but still part of the same team.
Third, use the lab to challenge how the company works, not just for products and technology exploration. The team's phrase is that Labs sometimes creates "good trouble" at Google. His example: Labs was one of the first teams to say job roles were blurring and that it needed a builder job ladder, and it used Labs to pilot the ladder before rolling it out across Google.
On hiring, Woodward says the team looks for people who "can't stop themselves from building." He describes a "Labs in a nutshell" document, plus a "Gemini in a nutshell" one, which he emails to new team members in their first week and which many have already seen during hiring. Among the traits it describes: they build constantly, they are intellectually curious and consume a lot of reading and podcasts, they care about solving people's problems rather than their own ego or glory, and they add energy, so you'd "run through a wall with that person." He says there are 17 or 18 such traits and offers to share them for the show notes. In interviews, a candidate who says "let me show you this thing I just built" gains instant credibility. He also asks about "unknown unstructured dark caves," situations where the way forward is unclear, and looks for people energized by them. He notes that zero-to-one people differ somewhat from one-to-ten and ten-to-hundred people.
On organizational mistakes, he cites a live debate: whether to give separate small teams their own bets or group them by theme. For interactive entertainment, which he calls "a pond we're interested in fishing in," should one group own it, or should five people take five separate shots? He thinks people tend to rush to consolidate and make the structure clean, and he has learned "maybe the hard way" that sometimes you have to live with messiness, see what emerges, and still be willing to change. The other common mistake is hiring too many people too fast. Headcount growth can be addictive, but he calls it a vanity metric that is "really bad most of the time." He recalls a project years ago that reached 30 engineers, which made him happy, with "zero product market fit but a very inspiring vision." He lets Lenny infer how it ended.
10,000 products or five?
Lenny passes along a question from Logan Kilpatrick: in five years, will Google have 10,000 products or five? Woodward calls it a long-running debate with Logan and admits his answer keeps changing.
His current leaning is that products may not be thought of the same way. Ten thousand separately branded surfaces seems complex and maybe not desirable, even if a "product factory" could produce them. He is increasingly considering a world of more bespoke experiences, possibly generated by models, perhaps with Labs teams building the agents behind those experiences and delivering them through a common surface. He says he's unsure. Labs is running a couple of experiments that try to make the case for both sides: one imagining consolidation into a super app or set of super apps, the other imagining extreme fragmentation.
For now he leans away from super apps. He acknowledges they have worked extremely well for some uses in some parts of the world, but thinks they can be very complex for many people, which makes it hard to drive behavior that adds value. He also notes that the whole debate assumes humans will be the ones tapping through most of these experiences, which may not hold. Lenny says the uncertainty shows that everyone is still figuring this out, and Woodward agrees.
An open question about feedback, and the rituals that encode values
Asked what he'd like to leave listeners with, Woodward points to an unsolved problem: how to gather feedback on products and get insight from it. He mentions agents running overnight to summarize everything that's happening, which he assumes Lenny and many listeners already use. But he frames the real question around what makes a great PM: insight into a particular group of people and what will delight them or solve their problem. Lenny restates it as how to make AI as good as a top PM at spotting opportunities. Woodward says that's one branch. The other is whether there's a collaboration model between people and AI that produces more of the insight that flows naturally around certain people. He calls it "an interesting prompt maybe" and asks people to send him ideas on X.
Finally, Lenny asks about the awards Woodward gives teammates. The TPU Harvester goes to people who do the "grungy work" of efficiency and reclaiming TPUs, since growing products need ever more of them. Winners get a miniature rake made on Etsy and a cash award. When Lenny asks whether it rewards using the most tokens, Woodward says no, it rewards optimizing. A golden band-aid goes to people who fix "paper cuts," the small annoyances that add up in a product, and recipients keep it on their desks. The LLM Whisperer goes to people who push models in surprising, frontier ways. They receive a giant set of ears, because "they can hear things from the model that others can't," plus a nice mechanical keyboard. Woodward says these "dorky symbols" encode values: TPU efficiency lets the products grow and reach more people, fixing paper cuts makes products delightful, and model whispering leads to breakthroughs and new products.
He also describes something like a secret society. A meeting invite with four or five team leaders and no agenda appears on someone's calendar, which looks alarming. Woodward opens by saying, "this is not a bad meeting, this is a good meeting," and then the leaders take turns telling the person why they appreciate them so much. He says it surprises people and is reserved for some of the team's most valuable members.
The conversation ends with the questions Woodward is still working through: how AI could help capture the kind of insight great PMs have, and whether Google's future looks like thousands of products, a handful, or something no longer recognizable as products at all.
My thesis is that every company needs to start thinking and operating like a labs team.
Every company definitely needs a set of people, a team. Their job is to be at the frontier. They're the people who have eight $200-a-month subscriptions on everything.
Everyone has to get really good at trying a bunch of stuff with the latest models, figuring out what is now possible before your competition does, before a startup comes and eats your lunch.
You've got to try to create a space where weird things can grow. If you look at the history of a large company who starts a lab, the lab is usually ineffective and fizzles out after about three to four years. You can't just nestle this under an existing business unit. It has to have its own kind of independence.
You've seen more ideas thrown at the wall and tried and prototyped than maybe anyone else. What are some patterns of where the best ideas come from?
They very rarely come from design sprints. They don't come in conventional places. You can't microwave good ideas.
What skills are you finding are trending up in value?
I'm actually looking a lot right now for people. What's their unlearning rate? How fast can they learn something and then walk away from it?
What's something that might surprise people about what product-market fit looks like?
When you're showing people early prototypes, you're looking at people's eyes. That is the metric.
What are signs that it's time to kill an idea?
The team usually knows before the leader knows.
Today, my guest is Josh Woodward. Josh is head of Google Labs, the Gemini app, and AI Studio. He's been at Google for over 16 years, and his job is to experiment with and scale new AI products. My premise for this conversation, and why I believe it is so valuable, is that every company is being forced to think and operate like a labs team. Because what is now possible is changing so quickly. You have to get very good at playing with the latest technologies, running tons of experiments, and uncovering new opportunities before your competition or startups do.
Josh runs the longest-lasting and biggest labs team in the world. And he shares a ton of very specific lessons about where the best ideas come from, how to know when to quit an idea, what it looks like when you have product-market fit, what skills are becoming more valuable in today's era, why roles aren't actually blurring into one builder role, and so much more, including some spicy stuff. A huge thank you to Steven Johnson, Sophie Miller, Jeff Hong, and Logan Kilpatrick for suggesting topics and questions for this conversation.
Before we get into it, I've noticed a lot of people don't finish the episodes, which is totally fine, but you end up missing the best stuff. So, to help you, I've created a new Lenny's Most Replayed Moments YouTube channel that is exactly that, the most rewatched and shared segments from our long-form episodes. Check it out and subscribe at lennyspodcast.com/mostreplayedmoments. There's also a link in the description. With that, I bring you Josh Woodward.
Josh, thank you so much for being here. Welcome to the podcast.
Thank you. So great to be here.
It's even better to have you here. I want to first lay out the premise for this conversation and I want to get your take if you agree with this.
Yeah. Sounds good.
Worldview. So my thesis is that every company needs to start thinking and operating like a labs team because the underlying tech that we're all building on is changing so fast. Which means what we can do is changing so fast. Which means that now everyone has to get really good at trying a bunch of stuff with the latest models and basically figuring out what is now possible before your competition does, before a startup comes and eats your lunch.
And so it feels like the skills that will matter more and more now are things that you all have gotten really good at, which is finding great ideas, figuring out how to run a ton of experiments, figuring out what to double down on, building teams that can operate this way, at the same time not distracting the rest of the company from all the shiny stuff and letting them focus. So broadly, does that resonate?
Yeah, I think your premise is spot on. I think the devil's in the details with all those things you just said. So I think what's easy for, in some ways every startup is a lab, and the real question is how do you bring some of that same methodology, ideas, energy, excitement into bigger teams or even medium-sized teams? But I absolutely think it's needed, and then the question is, how do you build that environment and nurture and cultivate the kind of both ideas but also the people that can really thrive there?
Awesome. Okay, that's exactly what we're going to talk about.
Yep.
This episode is brought to you by our season's presenting sponsor, WorkOS. What do OpenAI, Anthropic, Cursor, Replit, Sierra, Clay, and hundreds of other winning companies all have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you've felt the pain of integrating single sign-on, SCIM, RBAC, audit logs, and other features required by large companies. WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built specifically for B2B SaaS.
Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS. And that's because they are the best. Whether you are a seed-stage startup trying to land your first enterprise customer or a unicorn expanding globally, WorkOS is the fastest path to becoming enterprise-ready and unblocking growth. It's essentially Stripe for enterprise features. Visit workos.com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions. WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.com to make your app enterprise-ready today.
So, I'm going to start with coming up with ideas, finding great ideas. If you were to reflect back on the best ideas that emerged out of Labs and even in Google in general, what are some patterns of where the best ideas come from?
Yeah, it's interesting because a lot of them, I wish there was an answer in the back of the book where there was one thing. Maybe one observation, Lenny, is they come from a variety of places. So one lesson I've always felt is you always have to be watching for something that catches your eye. Maybe we come back to that in a second.
They very rarely, at least in my experience, have come from design sprints or times where you set aside on a calendar, okay, this is where we're going to go come up with our next idea. They usually come when people are swimming in the hallway, maybe on a weekend, often over a break. A lot of our labs teams will be hacking and building on stuff and they just can't help themselves, and they'll come back or they'll share a video and be like, look at this amazing thing.
So I think one lesson is always on alert. Second lesson is they don't come in conventional places. You can't microwave good ideas. And maybe that leads to a third thing I've seen is sometimes it's less about the idea and who are the interesting people, and more importantly, what are their interesting little side projects and passion things, and creating the chemistry around that.
Because usually what we found, at least in Labs over the last four or five years, is that all these ideas, whether they're Notebook or Flow or AI Studio or Google Beam or any of this stuff, they get started like very small groups of people who are just obsessed and usually very curious about something, and they just won't stop. And so how do you create a space where that can happen and then be on alert to catch it and try to funnel it forward?
So what I'm hearing here is, interestingly, not design sprints. I didn't hear brainstorm meetings in there as a place good ideas come from.
Yeah.
Is there an example maybe, because someone listening to this, I'm trying to think about what can they take from this to help them find great ideas? Say they're a startup founder or at a big company playing with the latest models. What are some of the conditions that you find most help?
Yes. I mean one thing we try to do a lot, and I think this would apply in any of those situations, is how do you get yourself to the frontier? And that can be both with playing with the models but also just watching what are the types of experiences that have just become possible. We have a phrase on the team, we talk about what's almost possible, and we just have a list actually where we're trying to track those things. And when one crosses over, it's almost like a phase shift when something goes from solid to gas or liquid to gas or something. You're like, "Ooh, that just became possible." I saw one of these two days ago and I can't stop thinking about it.
Are you able to tell us or it's a—
Maybe soon, but it's like if you're at the frontier playing with the models, watching what's happening, and you have this list, then when you see something, whether it's on X or wherever, then you're like, "Okay, that just now has become possible." And then we try to think about how do we get a team around that and just go.
And so that may be, as you're in startup land, if you're thinking or maybe you're starting to pitch ideas to VCs or whatever, that's how I'd be trying to think about it. And then of course tech alone isn't enough. You've got to match that to an interesting user problem. And ideally, if you can put those together in a way that's so good, people will pay for it. That's the combination of things we start looking for pretty early.
So that's an interesting tactic there, is make a list of what is almost possible and then watch for, okay, wow, this is now possible. Let's explore this now.
Yeah, that's right.
Is there anything on that list you can share of things that are not yet possible but almost possible that you're waiting for, watching for?
Well, maybe I can share a few areas, because I think how we think about it in Labs is we have a bit of a thought experiment we go through where we're like, what is the future of a certain thing going to look like? So the future of software design and development, or the future of creativity, the future of work or knowledge.
One area we're watching very close right now is just what is the future of entertainment going to look like? And this whole idea around immersive entertainment is super interesting just at a technical level for many reasons, but also you think about the user behaviors and what does entertainment look like last year, this year versus next year or maybe in three years? And so that's one where we've got a whole list underneath that where we're like, okay, well, if a model can get to this kind of latency and this amount of languages and this amount of modality, you can start to think through what would be interesting from the tech side.
And then we do a lot of work in Labs around where are just behaviors and trends going. And for that, some of it, of course, you can study in the Bay Area, but for a lot of our teams, I'm always thinking about how do we get way outside the Bay Area? And thinking about where are the other cities, the classrooms, the other places to almost mine for ideas and these behavioral changes.
So, an interesting takeaway here is this. You're not just waiting for inspiration to strike. You're doing a lot of pre-work to think about here's what is almost working. Here's where we think the future might be going. Here's areas that are going to be ripe for disruption.
That's right. Yeah. In this doc I talked about, we have 82 predictions that all begin with "We believe the future is" or "We predict." This is dangerous ever to make one prediction, let alone 82. But I think you have to almost have a point of view about the future knowing that nine out of 10 or maybe 9.9 out of 10 you're going to be wrong. But I think that's one of the things we really try to cultivate in the team is how do we chase the future? That's one of the values, and trying to chase it. You almost have to throw yourself out there and try to live in it a little bit.
What are signs to you that an idea is worth pursuing when you're in the shower, you have this idea? What's a sign that, okay, that's worth betting on, at least to start with? And what's often just a false signal, like okay, no, that's actually not going to work?
Yeah. I think one thing for me, especially when you've got these lists going and you're thinking about the future, when you see something that surprises you or maybe that's very interesting to you in a certain way, and if you can describe it to yourself, like this thing I saw about 48 hours ago, I left that demo and I thought this is so exciting. And I think that's maybe a signal I use a little bit, is trying to be at the frontier and then what makes me interested or excited and then feeding off that with the team together.
I feel like that bar is too low these days because everything is so magical, or it's the opposite now. Everything, okay, great, this thing is doing my taxes for me. Of course, that's just the way.
Yeah, yeah.
I think this is one of the things. Obviously that excitement is maybe a tech thing, and there's dazzling demos every other day or every other hour right now. But then I think trying to think through where's the user behavior maybe that is universal, or maybe there is some shift on that side. And then the way we think about monetization too is can you get to a group of people that can't live without it? They literally will pay you because it's so valuable to them, and then how you can put those pieces together. But yeah, sometimes it's got to come from one of those areas, I would say.
Is there a great example or story of a product that emerged in this way?
Yeah, I remember still the first time I heard the first NotebookLM podcast, exactly where I was standing. It was right outside this room. And it was probably at 6:30 p.m. on a Tuesday or something, and there were two people on the team who came over and were like, "You got to listen to this." And it was the proceedings of the British Parliament debates. Not known for its riveting material, but these two AI hosts were talking about it in a way that was literally interesting. And that was one where we're like, we got to just do something with this.
And that was one. I remember the first time when I saw something come out of Google Flow. We had a project before that called Whisk where you could whisk together multiple images and then start to animate them. And it was a level of control and creative channeling that had never been possible, that I kind of made it was like that accessible. So maybe those are two examples. We also have other examples where we get excited, we try to build some product, it's just not good. So that's the other thing, is a little bit of a game of numbers in this as well.
Let's talk about product-market fit and signs that something's actually working. I feel like you've seen more ideas thrown at the wall and tried and prototyped than maybe anyone else. I don't know if that's true, but it feels true. What are signs to you that this is something we need to double down on? And what's something that might surprise people about what product-market fit looks like?
Yeah, I think one of the maybe trickiest parts of it is it is way more art than science. I wish it was science. We know how to run science experiments, make data-driven decisions, set OKRs, all this process that
Helps. Sometimes in the early days, Lenny, it's literally a couple of people on the team who might just have conviction, and they're just like, "I think the future is going to look like this, and I can't explain it." That's what's hard. That advice is just not that useful or helpful. But I would say that's some of it.
I think maybe a little more useful, we talk a lot about how do we get outside the building and just show people prototypes. And when you're showing people early prototypes, what I'm always doing in those is you're looking at people's eyes, and that is the metric. It is not a DAU. It's not a DAU over a MAU. It's not a D7 retention. It's none of that. It's literally, "Hey, Lenny, did you see this? What do you think about this?" And if your eyes kind of light up, if you lean in, these are the kind of signals you're looking for in the early days.
Maybe the third thing I would say, and this one's also maybe on the flip side of your question, is the people who get excited about things, usually we try to kind of think about what's the problem we're falling in love with, not the product. Because what I found is it takes three, four, five pivots before you either got something or you're just like, "That's it, didn't work." So I think when people fall in love with an actual product solution first, it's going to, at least in my experience, lead to a bad time. So that's the other thing I think we try to think a lot about.
And usually people talk about PMF, you sort of know it when you have it. I think that's true. In the last year or so, when Nano Banana went viral on Gemini, when the NotebookLM podcasts have gone, when stuff's happened on Flow or Stitch or other kind of projects that have had these mini moments, you're literally just swarming it and holding on for dear life. And it is both so exhausting and so fun. But you almost have to take a lot of shots, I would say. And in the early days, again, it feels a little more art, passion-driven than science, unfortunately.
Those are actually very concrete, specific things to look for. The eyes light up. A way I've heard someone else describe this is their pupils dilate.
Yes. That's right. And you know too, right, when you're talking to someone, if they're having a good time, if they're into it, or if they're just bored out of their mind and you're showing them some dumb product. So it's really, can you hit a nerve with a pain point in their life where they're just like, "I want that"? And that's what makes this job so fun, is trying to figure out how many of those nerves you can hit.
So having that context of it's clear when things are product-market fitting and their eyes are lighting up, what are signs that it's time to kill an idea?
Yeah. One of the things on this one, and I talk a lot to the team about this too, is sometimes a team is looking for, "Hey, the leader needs to make the decision," and that's kind of where it comes from. What I found is the opposite. The team usually knows before the leader knows, and the signs are, does the passion start to run out? You've tried everything you can think of, and it's just not working. And people know when a project's not working or a relationship's not working. That's something that people see.
And so I think as a leader, what I try to do, say in Labs or even some of our features in Gemini, is how do you create an environment where people can ask those uncomfortable questions and be able to say, "Hey, I don't think this is working"? And yesterday, just as an example, we had a feature we were really excited about in Gemini we were going to launch, and we got some of the early data back. People went out, looked in the eyes, and it just wasn't there. And so I was so proud of the PM on the team because she was like, "I don't think we should launch this. It's not good." And so I replied all, was like, "This is great. Thank you for doing this."
And so it's like, how do you create a culture where the team can call both good things and bad things out? And I guess in that way, as a leader, in my case, I trust the team a lot, because they're the ones every day trying to make this thing go. And I think if they're like, "Hey, it's not going," then it's like, "All right, that's okay. Let's pivot. Let's do something different."
I have to ask you this question. I was going to ask it later, but I'm so curious. It's so interesting that you oversee Labs, which is zero-to-one experiments, try stuff that's totally new, and then Gemini, the Gemini app, which is one of the most popular apps in the world with probably a billion users. How did that come to be? Why is that the structure under your team?
Yeah. Well, I guess in some ways they are very different ends of the scale. You're right, Gemini, over a billion users. It's global. Labs, we joke on Labs, sometimes our early projects, the zero-to-one stuff, we'll get so excited about 10,000 MAUs, and most other products' dashboards don't count that low. So this is kind of the joke on the team.
But I think maybe there's a few common pieces, though, even though the scale is different. I think one, the type of people that are attracted to these things, even Gemini, despite its size, has very much got a zero-to-one heartbeat all over it. And so I think part of it is the type of builder, the type of person you're attracting, these blurring lines between what it takes. Two, we're all AI kind of nuts and maniacs. We love this stuff. We can't get enough of it. We're building all the time.
The third one, though, I would say is a little bit of everyone on the team. We talk a little bit about the order of how we think about our priorities, and the mantra on the team is users first, Google second, and our product third. And so Labs third or Gemini third. And I think that unites us too. We don't try to do things for our own team. We're very much part of Team Google, but also ultimately it's like, is someone loving this on the other end of this product? And so that does kind of tie it all together, but they are on different ends of the spectrum for sure in terms of size, like conventional numbers.
Yeah, it's quite confusing as an outsider, but I get it. Is everything you're doing in Labs now AI-based? Is there anything non-AI-driven in Labs now?
Yeah, it's all AI-driven right now, for sure. When we started four or five years ago, we had a few other areas we were exploring, and we kind of made the call, at least for the foreseeable future, go all in on AI. We don't just do software stuff, though. Google Beam is one of the projects we're working on. It's this incredible kind of magic mirror that can create a 3D version. We should get you one.
Where you can kind of talk to someone almost like they're a hologram on the other side. Yeah. So we're exploring other adjacent things to AI. You could consider that almost like AI hardware in a way.
I will take it. Thank you. I'll send you my address.
So it feels like consumer right now is in a really fun place where there's always been the sense of, "Consumer's dead. There's nothing you can do. Facebook, Google, there's no way to compete against these juggernauts that have taken the consumer world." But AI feels like it's unlocking a lot of opportunity in consumer. I'm curious what your hypothesis is for where a major breakthrough might happen in a consumer app.
Yeah, I feel like consumer is one of the most exciting, competitive, all the things happening right now. So it feels completely opposite to me from that kind of view. A couple of areas I think that are kind of interesting. One, I think there's a big question, we talked a little bit earlier, of just where's entertainment going to go? Are we going to be looking at different types of feeds forever, or are there ways that this technology can bring us maybe closer together in the real world or do things with other people? There's different views on where that could go. Is it more around bots or more around people? And so I think that's a very interesting space.
I think we're watching right before us now, the entire messaging, chat space felt like, "Hey, that was sort of maybe solved." There's lots of different experiments happening on that right now, both inside Google and outside. So that feels like a whole other area that maybe is no longer as fixed as it once felt.
And then maybe the third one I would throw out is I still feel like going back again to the problems, what are some of the big problems people face in the world? And I think, what are things that are super scarce? Time on Earth, how much money people have and how they can save money, these kind of memories or experiences that people can have IRL together and that kind of thing. So I do feel like there's maybe new consumer products to be built that can go after a lot of that stuff. Whether it takes the form factor of a chatbot or a personal agent or whatever that's going to look like, I think is yet to be invented in a lot of ways.
Okay. Let me ask you a spicy question along these lines. I'm so curious to get your take. So it feels like every AI assistant app that's launching these days basically sucks in Gmail and Google Calendar and Docs, all the Google data, all this juicy Google stuff, connects and does all these magical things with it. Within Google, I cannot do any of these things. It's so hard to build anything. First of all, you may be working on an AI assistant, but there's nothing there yet as far as I can see. And then just generally it feels like everyone's just taken all this juicy data and doing magical things with it, and Google hasn't. What's kept you guys from doing something like that? Is it privacy concerns? Is it bureaucracy stuff? Is it coming? What can you say?
Yeah. So there's parts of it out there today. If you're in Gemini, in Gemini Spark, you can actually connect all that stuff, and it's quite powerful. I think where we're iterating on that, though, is we've been in a world where, even if you look at some of the other competitors, there are different modes almost that you go into. And I think the lesson even from the last couple of weeks is forget the modes and toggles. Just one prompt box, tell it what you want, and it just does stuff for you. So that's definitely not only where we're headed, where others are headed. I think that's a broader, almost UX simplification happening across the industry.
On the data bits there, it's interesting because actually one of the main things people will ping me about or others is like, "Oh man, Google, I've got all my stuff with you. Just make the thing work." And there's a lot of reluctance actually to be connecting it to a lot of other competitor products. So watch this space. We hear you loud and clear. There's a lot of work, if you go online and just look under Personal Intelligence, is sort of the way we think about this stuff. You want a Gemini that is personal, proactive, powerful, that sort of works for you. And so that's been the vision for the last year or so that we're headed towards. And there's a lot of stuff we're playing with right now that will be out soon.
All right. I'm excited. It just feels like I have so many little schedules and things running within my email. I'm just like, "Man, why isn't Google just doing this?" It does feel that the advantage you all have is distribution. So once something happens, similar to Muse recently, it's going to be very easy to grow it and convince people to use it, similar to within AI Mode. So anyway, okay, I'm excited. Things are happening.
Things are definitely happening.
So kind of going a different direction, what do you think's underhyped in AI right now, and what do you think is overhyped, if anything is overhyped?
Oh yeah. Underhyped, I was thinking a lot about this over the summer. I ended up writing a doc internally just around Labs principles of how and almost why are we building what we build? And so maybe the underhyped thing is more of a discussion around principles, values. I believe very strongly, whether you're at Google or a startup founder or anywhere in between, you speak through your products, and your products actually represent and almost encode those principles. And so I wish there was a lot more discussion in the industry happening around that, and what kinds of things do we want to build? What kind of future do we want? So anyway, that would be my underhyped one, because I think there could be more of that.
I think the overhyped one, I think I've been saying this maybe for a year now, but I still think it's overhyped: model benchmarks. And a lot of friends and a lot of labs and everyone's still looking at the same set of benchmarks for the most part. But I think where I grew up and where I'm from, and I guess how I think about product development, is most people in the world are never, ever, ever going to care about an Elo rating. They may not even know how to pronounce Elo. It just doesn't matter.
And so that's where I kind of get back to, I feel like a lot of the talk in AI right now is about speeds and feeds and benchmarks. And these are proxies, obviously, for where things are and capabilities. But I still feel like sometimes as an industry it can overrotate to that and lose sight of why is this useful to someone, and what's the product or the feature that's going to deliver real value to help people? So anyway, those would be my over-unders.
Yeah. And it's also along those lines, it's also hard to know what is better. How do you know what's going on? You have this new model. It's the best model ever. In practice, it's not that much different than the previous model that also everyone thought was the best model ever. We have AGI every time there's a model launch. We have AGI now. And so the benchmarks are trying to show you here's where it's better. But if you didn't, you're just sitting there, "Okay, this is okay. This is better. I like what it's doing here."
Yeah, I think this is also, though, where I feel like a lot of folks that follow you and listen to you, it's like, please, please, please think of amazing products. Because ultimately all these model capabilities and benchmarks and all the intelligence gets translated into just, can you build something people want? And so to me, it's like, what a time to be alive as a product person, because you have this huge menu of things that were almost possible that are now possible more and more with each passing week.
The first point you made there was something that came up recently in a conversation with Tara from OpenAI about how now that we could just build everything and AI has so many opinions about where to go, what you said is exactly right, which is what do we want the world to look like? What is the vision of the future we want? Okay, we can do all these things. That's great. Maybe we don't need to do all these things.
Right.
So I think that's a really important point there.
Yeah, definitely. Definitely.
This season's supporting sponsor is DX. Now that everybody at your company is finally using AI, the question that engineering leaders face is no longer, are we using AI, but is it paying off? With DX, you can measure and benchmark
The impact of AI adoption across your full product life cycle. DX shows you where AI is helping developers and where it's creating new bottlenecks. You can also use it to evaluate AI vendors, monitor costs and licenses, and see exactly what's limiting your agents. That's why hundreds of enterprises, including Snowflake, Sony, and BNY, use the DX platform to measure AI's impact on developer productivity. Visit getdx.com/lenny to get a demo of DX. That's getdx.com/lenny.
What skills are you finding are trending up in value in the people that you're working with, and what do you feel are trending down? When you look at people who are doing really well now, what are the attributes and skills of those people versus things that are maybe less important now?
I feel like there's a whole lot out there on taste and judgment and vibe coding and all of that. The things I'm looking for, I'm always trying to find: what are the mispriced signals? One, I think, is how we'll hear sometimes people talk about what's someone's learning rate. Do they have a growth mindset? What's their learning rate? I'm actually looking a lot right now for people: what's their unlearning rate? How fast can they learn something and then walk away from it based on maybe what they're seeing when they test their product or something? Unlearning rate is one.
Another one I'm interested in: I read a book a year or so ago about Roger Federer, the amazing tennis player, and there was this phrase in there about Federer had explosive endurance. It's just this beautiful turn of phrase, with the alliteration and everything. But there's something around how are people taking care of themselves and the people around them where they can go really hard, and that's the explosive part, but they can do it over a while, and they find ways to pace themselves. In Labs, we have seasons of Labs where we talk about this. So I think that explosive endurance, this unlearning rate.
And then maybe another one I would say is, as much as coding and other things are becoming, maybe trending towards you being able to direct lots of agents and not having to work with as many people, I still feel like collaboration and how you work with people matters even more now. One of the things we look a lot for is: can someone blend and bridge these worlds where they're an incredible collaborator with an agent or a swarm of agents, but also with people? What we believe on the Labs team and on the Gemini team is that there's still just so much joy in working with people. But how do you find the right chemistry? Maybe one thing I'd say there is people that can build trust, that can scale trust. That underpins any kind of great collaboration. That's another signal I'm always looking for: how fast can this person build trust with people and create that kind of environment, if that makes sense?
These are so interesting. On that last one, is that a growing, emerging, even more important skill you're finding with the rise of AI? I'm curious why.
I think so, for a couple of reasons. I would say one maybe is uniquely Google. It's a larger organization, so you've got to be able to do that. But the ones that don't feel uniquely Google: one, it feels like as the execution cost of making products goes down, we'll be able to make more products. You may make the same products with the same people, but I think there's a lot more, you probably see this too, people will team up with different people to make stuff.
It's very interesting when we're thinking about almost fielding a new team for a Labs project. We used to say it's about five to seven people. Now it's about two to three people, maybe four. But those people, you can almost imagine super talented musicians who team up with different people, and then they play different songs together. The songs are the products. Those are the MVPs in Labs, where you can be like, hey, drop this sort of PM, but that can also code and design because all the roles are blurring, with other people and see how they jam on stuff together. So I think that's where my prediction is that skill will almost increase in importance because there will be more products and because there'll be more of these free agents jamming on stuff together.
Along those lines, I imagine you have been pulling designers, engineers, and PMs into Labs, and they all work together, and sometimes different functions take leads. Has there been a shift in who you find is most impactful and successful in leading product ideas, Labs teams, in terms of the ratio of PM versus design versus engineering, for example? Are PMs becoming more valuable or becoming more valuable, something like that?
It's interesting. We've been studying this a lot and actually testing a lot of this too. I would say, in general, PM happens to be the function, maybe by nature because it's a bit more horizontal, that feels like it's maybe been able to adapt to some of the other functions faster. We see this when we'll run little builder pilots across the company. That comes back as, oh yeah, that's maybe, I don't want to say it's easier, but maybe PMs are more accustomed to, all right, I'm in this meeting. I've got to simulate something as a lawyer or as a BD person or as a sales, do you know what I mean? I think in some way maybe there's some inherentness in who selects into that. So that definitely is true.
But I would say our best people can come from any function. I would also even widen it, Lenny. We're seeing strategy and ops people and BD people and marketing people be builders in Labs, and they are really good. I think maybe this is another overhyped, underhyped thing. I think right now people are maybe overhyping everybody's a builder, and it's just this new job function, and everything's the same. I actually feel like, at least what we're seeing, is that people still have their specialty. It's like their major or minor in college or something. I feel like there's a risk, if everybody becomes a builder, that you lose a lot of respect and appreciation and obviously the expertise that comes from those specialties. So I think when we set up teams, even though we may be like, this is more of a builder crew, we're still thinking about where are the specialties, and that ends up being super important as well.
I completely agree. I like this metaphor of the major and minor. That's a really fun way to think about it. You can be better at building, but your PM skills are still the primary thing you're amazing at.
Definitely. I think that's the advice we give, especially to a lot of new PMs on the teams, like APMs or people earlier in their career. It's great you're vibe coding and you're making all these prototypes, but do that in service of becoming an exceptional, extraordinary PM. Don't lose that major. That's what you need to hone, especially when you're early on.
I love the way you described it, where PMs are really good at simulating other functions. It's such an AI way of thinking about collaborating.
Yeah, yeah.
And this idea of explosive endurance. Someone tweeted this recently. They're like, what's it called when it's both a marathon and a sprint?
It's that. Talk to Roger Federer. Yeah, yeah.
And I feel like everyone's in that right now. That's just a constant theme and trend on this podcast and just in tech right now, is everyone's in this marathon plus sprint mode.
Yes, yeah.
Is there anything that you do to help people deal with that? Because it feels like it's nonstop.
One of the things we try to do, and I mentioned it a little bit earlier, but I can unpack it some, is we think of seasons in Labs. At any given point, there's 20 or 30 experiments running simultaneously, but for each individual little team, trying to create a little bit of milestones or a rhythm to the work. Sometimes it is just flat out, full on. Maybe the best lesson I've learned as a leader is almost to name that, declare that. People opt in or opt out of that. But also be very clear when a team is not in that mode.
How do you try to architect almost a rhythm to the year where it's like, hey, we've got a big model coming. These five products are super important to showcase this model. We think this model will propel them. You five, it is explosive time. Do you know what I mean? So it's trying to create a common language around that. In a lot of ways, PMs sometimes can be very effective if they can come up with what is the shorthand or the common language. We actually put a lot of thought into that too, of all right, you all are on right now.
Whereas after they go through that, we may be like, say after a Google I/O, a lot of Labs teams will sprint. After I/O, June is like, hey, just go hack, go rediscover what's new, just go build, have fun. It's so interesting, Lenny, because the instinct or the temptation is like, oh man, it's June. It's unproductive. We're not shipping. And it's like, no, actually those are where all the new seeds are coming from for the next season. Do you know what I mean? Trying to think through the rhythm of that is one way we try to do it.
I don't think we always get it right, though, by the way. One of the things we're talking a lot on the team is, in the past people would talk about work-life balance, and there were other ways this conversation got framed. Why I like things like explosive endurance or concepts like that is we want to be doing this as a team for many, many years. You can't just empty the tank in nine months and just kill yourself, right? You have to have a view towards the long term, but also recognize the moment you're in is fast.
I love the tactic of just naming it, having a lot of power and just making it clear this is the mode we're in now. It won't be forever.
Right, right.
And something I found is the opposite also is dangerous. When you have too little to do and too little stress and too little pedal to the metal, people get bored and start looking for other jobs. They're just like, this sucks.
Yeah. I think there's always something there, though. You're right. We talk on the team a little bit about never waste an event or a date. You can think about, even as a startup, where's that moment you want to intersect with something in the world? Whether that for us is a Google I/O or a model release or whatever, I think that's always healthy to be thinking about.
I got a question the other day on Gemini. They're like, what's our two-year roadmap? And I was like, we have a six-month roadmap on this thing. We have a multiyear vision, but I think the world has changed so much too. Some of it is the thing I saw two days ago is now a project that happened in 48 hours. You can't plan for that, in a way. That's the other element you almost have to recognize in the moment.
So along those lines, how has planning changed in the past couple years? You guys are different from the rest of Google because everything is new, but what have you seen in terms of timelines and time frames shift in planning with the rise of AI?
We usually are thinking somewhere on the order of about six months. Usually I say that because that's that almost possible window where you assume every lab is cranking out some new pre-training model a couple times a year. There's individual points in between. So you have an overall cadence of where the models are, and then I think we're almost imagining, in about a hundred days, usually we always think about where can you move your product meaningfully forward. Our best Labs teams, they all do that. They'll go from idea to some meaningful milestone in 50 to 100 days. Part of that is also how do you design a culture and an environment where you unblock a lot of stuff so you can just let people cook? That's the other part of creating that environment. We think about planning as almost rolling six months, maybe, to answer your question.
I had this Lenny & Friends Summit. It was last week, as of today we're recording this, and Dan Shipper gave this really interesting talk about how to build where the tech is just constantly changing underneath you. His advice is to have a frontier team that's dedicated to testing out the new stuff. Basically, it's their job: try all the new stuff, report on what we need to pay attention to, so that the rest of the company can not have to be distracted and focus on the task at hand. How do you feel about that as a model for companies?
I think every company definitely needs a set of people, a team, however you want to manifest it, that their job is to be at the frontier. They're almost like, if they haven't heard about the latest hot thing viral on X in the last 24 hours, it's like, what are you doing at your job? Do you know what I mean? I think there's definitely an element of you want people plugged into the hive mind of how people are building and thinking and all the model releases. They're the people who have eight $200-a-month subscriptions on everything. You have to architect the organization where that can exist.
I think the second thing, though, that's just as important is the environment in which those people can build. That's the other part of that. And then I think the third part, which is honestly where, if you look at the history, and I've read a bunch of books on this, because if you just look at the data, a large company who starts a lab, the lab is usually ineffective and fizzles out after about three to four years. Or they have duration, but the company can never commercialize the things coming out of the lab. So you get things like someone comes by and sees the future, and then they do it as a separate company or whatever, right? There's famous examples throughout history of that.
I think there is this interesting balance of: can you have a team of people who live at the frontier, like Dan says, who are building frontier things, but can somehow bridge or connect across the bigger company in a way that it's not, oh, that's just some weird R&D group always coming up with gimmicks and toys? Do you know what I mean? I think it takes a certain person or a certain set of people who can actually be like, hey, that thing looks like a toy right now. That's going to be essential to our future in five years. That's actually where, at least studying history, sometimes these frontier ideas fall short. You can get the first two things right and then fail on the third one, and then the whole thing really doesn't matter. Do you know what I mean?
Yeah, it's interesting. Even Anthropic has a Labs team, the most cutting-edge company. They're at the frontier already. They're like, okay, but we also need a Labs team to actually watch the frontier frontier.
Yeah, that's right. One thing we joke about too inside, as Labs has grown now and we've had a few projects thankfully have hit and
There are lots of users now and growing. We've actually internally, to this point of common language, what we started to find was a bigger project like Notebook in terms of users, or Flow or AI Studio, or these things that have grown out of Labs, there's a risk they crowd out the zero-to-one stuff.
And so the terms we've started using are there's zero-to-one Labs, there's one-to-ten Labs, and there's ten-to-a-hundred. And these may not mean anything to your audience, but to us, we actually encoded these are different stages of the lab's life cycle. And we want all of these products to have this innovative energy that's common. But inside each of those stages, there's very different challenges and very different things you're optimizing for. And in zero to one, you're looking at eyeballs like we talked about. And maybe ten to a hundred, you're looking at waterfalls and CPAs and thinking about how are we actually converting people to these magic moments in the product? And it's a scaling game in a lot of ways.
Someone listening is convinced they should set up a frontier team, a labs team, and it feels like labs teams have evolved a little bit. It used to be build the next big business unit is the idea. Here it's like, okay, what's a new technology that is unlocking something in our core business? I wonder if that's true, but that's what it feels like. But I guess just with that in mind, what are maybe three tips for someone that wants to set up a labs team internally?
Yeah, I think one, you've got to try to create a space where weird things can grow, and you can't just nestle this under an existing business unit. It has to have its own independence, is one. And in some companies, that means it needs to report to the CEO or it needs to report into enough of a distance, in a way. And we can go into that if you want, but that's super important.
I think the second thing is a set of people. And in the case of us, this was this users first, Google second, Labs third thinking, where the job of the lab is to make users successful and make Google successful. It's not to make itself successful. And so there's a bit of maybe a humility or a collaboration or this mindset of what does success look like? And I think for us, how we've broken that down and how I would encourage companies to think about it is, is your goal of this lab to graduate things to existing products, or is your goal to go invent new product lines and categories? And in our case, we started and it was a graduation lab, and it's expanded over time as we've started to realize AI really transforms and blurs a lot of categories. But being very clear about what game you're playing and what does winning or losing look like would be the second thing. So isolated in some way, but also part of the same team.
And I think the third thing, and we talk about this a little bit on Labs, and I could imagine for maybe CEOs listening or other C-suite people, use your lab not just for product development or technology exploration, but even for challenging how the company works and almost the way things happen. And that's actually the phrase we use on the team, is Labs sometimes creates good trouble at Google. So there's bad trouble you don't want to create. But good trouble, I'll give you an example. We were one of the first teams that were like, we're seeing the job roles blur. We have to have a builder job ladder, and we need to think about it differently, and we should use Labs as the place to pilot it before we roll it out across Google. And so that's another way you can think of the multidimensionality of a lab, in a way.
You just think about the processes and ways of working, not just products necessarily. That's really interesting.
Yeah.
What about the type of people that you would put on this lab team? You mentioned this unlearning rate is a really interesting way to think about it. What else do you look for, and what's a person you want to avoid on the lab team?
Yeah, great. Man, we are obsessed about finding people who just can't stop themselves from building. I mentioned that earlier. We have a whole doc on the team, Labs in a Nutshell. We have one on Gemini, too, Gemini in a Nutshell. The first week someone joins the team, I email it to them, and most of them have seen it before as part of some of our hiring screening, too. They build all the time. They are intellectually curious. They're reading, they're on X, they're listening to podcasts. They are obsessed about solving people's problems, not about their own ego or glory. They are the people, when you're around them, they add energy. You leave a conversation, you're like, I want to be with that person. It's like, I'll run through a wall with that person. Do you know what I mean? We've got seventeen or eighteen of them. Maybe I can send some of them to you after if you want to include them in the show notes.
Yeah, but it's such an important part. And that third thing I talked about as a tip for setting up a lab, it's not just about job ladders and stuff. It's literally about who are the people of the future that you are trying to bring in that are going to think different and challenge things different? And it's a different type. And what we've also found as these products have grown, the zero-to-one people are different a little bit than the one-to-ten and the ten-to-a-hundred, too. So it's like, how do you create some commonality? But I do think when you're talking to someone, you're looking at, in an interview, if I'm talking to someone, they're like, hey, let me show you this thing I just built: instant credibility. Or I'll ask them questions about unknown, unstructured dark caves, these events where they don't know which way is forward. How do they figure it out? People that are energized by that, huge credibility boost. So yeah, there's definitely a certain type that you're looking for.
In terms of the org structure, what's the biggest mistake people make?
Yeah, I think it's actually, I'll give you a live one we're talking about right now. We're trying to figure out how much you isolate different subteams and going after things versus try to create groupings of thematic things. For example, we've talked about interactive entertainment as an area or a pond we're interested in fishing in. And it's like, how does that—do you do that as one set of people doing that? Is it five people just take five shots at it? So a mistake sometimes, I think, is people maybe rush to a group, and maybe the bias is towards consolidate, make it clean. And I think what I've learned, maybe the hard way over the years, is actually sometimes you have to live in the messiness a little bit and almost see what emerges, but still be willing to make those changes. Those are org design, team kind of things.
The other thing I'd say sometimes is a common mistake: you hire too many people too fast. This happens at big companies. It happens at startups, too. And it can be so addicting to sometimes look at, oh, our team doubled or tripled in the last quarter or six months or whatever. And these are just vanity metrics that are really bad most of the time. So I think that's the other thing that's a common mistake. I remember I was on a project years ago, and we got to thirty engineers, and I was so happy, and we had zero product-market fit, but a very inspiring vision. And you probably know the outcome of how that ended, basically.
In our last few minutes, I'm going to ask you a question Logan Kilpatrick suggested to ask you. I think he's head of AI Studio, right? I forget his current... Okay, cool.
Great.
He's a great former podcast guest. I don't know exactly where this question is coming from, but I'm so curious. His question is, in five years, do you think Google is going to have ten thousand products or five?
Oh yes, okay. This is a long-running debate Logan and I have on the team. But it's funny, at least to me, I keep switching my answer. So maybe I'm a bad... Right now, I'm more in the camp of, I don't know if we're even going to think of products the same way.
So I think one framing of it is ten thousand products from Google. That's ten thousand surfaces that have their own brands, and it's all complex and everything. I'm not sure that'll be possible, to your earlier point, what's possible or what should we do? Because you can have a product factory that's just going to turn out these products. So that's one thing you could think about. I'm more and more thinking about, are there almost experiences or things that are going to be more bespoke that the models might make? And that might mean there's labs teams behind it making the agent that does this thing that you experience maybe in a common surface. I'm not sure.
But I also am not—we have a couple of experiments in Labs going on this right now of, can we prove both sides of this debate? And maybe that's why he threw it in. And sometimes that's what we'll do, is we'll take both sides of it and say, okay, if this thing really consolidates down and there's a super app or a set of super apps, what's that world going to look like? And maybe if this thing super fragments, what's that world going to look like?
And so right now I'm probably more in the camp of, I think super apps have worked in some use cases in some parts of the world, obviously extremely well. But I think for a lot of people using them, they can be very complex, and it's very hard to actually drive the behavior that adds value because there's so much complexity. But this is also assuming humans are going to be the ones tapping most of these experiences as well. And so anyway, this is a fun one to debate, and one of Logan and my favorites. We'll chat on this one a lot.
I love that it's just unclear where we're going. It just shows we're all just trying to figure this out.
That's right.
That's such a good example of that. Okay, well, before we get to our very exciting lightning round, Josh, is there anything else you wanted to share? Anything else you want to leave listeners with? Anything you want to double down on?
Yeah, I guess one thing I'd think about, and I'm always trying to think of this, too—if people have ideas, please just let me know on X or whatever—is I still think there's a lot of space right now on how to get feedback about the products and experiences you're building, whether it's in Google Labs or as a startup founder or whatever. And so that feels like another area of how do you listen and aggregate, collect, get insight, inspiration from feedback loops? I feel like there's a lot of interesting stuff happening with AI applied to that right now. And I'm sure all of you probably do, too, Lenny, on your stuff and my stuff, of agents running overnight summarizing everything that's happening. But I feel like there's an interesting, what makes a PM great? A great PM has insight about a certain type of people and the types of things that are going to delight them or solve their problem. And I feel like that's an interesting, how do you crack that feedback loop with AI and these new types of products we're building? So anyway, that's something also in the back of my mind right now.
Yeah, so the question there is just how do you get AI to be as good as an amazing top-percentile PM at identifying opportunities and insights?
Yeah, I think that's one branch of it. And I think the other one is, is there a world where—I don't know, there's probably some people in your life, just like they're in my life, when you're around them, insights seem to flow pretty naturally, and you get to a lot of interesting perspectives. And is there a collaboration model with people and AI that enables more of that? I think it's kind of an interesting prompt, maybe.
Sounds like we're getting into B2B SaaS software over here at Google Labs. I love it. I'd buy that.
Okay, I know you have to run soon, so I'm going to skip to just one lightning round question. I heard you give these hilarious awards to teammates. My favorite is the TPU harvester, but there are others. Share these awards you give and what they mean, what they're...
Oh yeah. So the TPU harvester, we just need more and more TPUs as our products are growing. And a few of us wanted to figure out a way to reward people who do the grungy work of efficiency and reclaiming TPUs, harvesting TPUs so we can build the next features and fuel our growth. So we give out a rake to these people and a cash award.
Like an actual rake?
Like a little miniature rake we made on Etsy. And that one is really fun, and we turned it into this big celebration. So this is people that save TPUs, that optimize, make them...
Optimize, not use the most tokens.
That's right. We talk to them separately. But I think that one's a rake. Another one, people that go in, fix paper cuts, these little things that add up in a product that annoy people. We made a golden Band-Aid, and we give that to people. They put it on their desk as a golden Band-Aid for fixing paper cuts.
We have a thing in Labs. We're really interested in people who can push the model in these frontier ways, in surprising ways. We have this thing called the LLM whisperer. They're able to whisper to the model. They get a giant set of ears because they can hear things from the model that others can't, and a really nice mechanical keyboard.
And there's more of these things, too. But we're trying to find ways to encode the value in these dorky symbols that mean something. Becoming more efficient with our TPUs enables us to grow and reach more people. Paper cuts, these are what make the product delightful and excellent. And then obviously the whisperer with the ears, these lead to breakthroughs and new products. So yeah, we're always coming up with weird stuff like this.
I love this. These kinds of traditions are the best. I heard there's also a big thank-you one where you get to talk to leadership or something like that.
Yeah, this one is a little bit of a secret society, in a way, where we'll basically put an invite on someone's calendar. And it looks kind of scary because there's four or five leaders on the team. There's no agenda. And then they show up and we take turns. We go around the circle and we just tell them why we appreciate them so much for being on the team. And it's pretty fun, and it totally surprises people. And I always start the meeting with, this is not a bad meeting, this is a good meeting. Usually your mind may assume the worst, but we're actually doing the opposite. These are some of the most valuable people.
Wow, that is awesome. I love all of these. Josh, how can listeners be useful to you?
Oh, send us feedback. So many of you do already. I would say we get a lot of joy, and we're very grateful for all the people who use the stuff we're trying to build. And if we can be helpful on the Labs side or the Gemini side, too, please, let's make it two-way. And thanks again, Lenny, for having me. It's been great to spend some time with you.
I think we're going to help a lot of people figure out how to build, stay at the edge. I really appreciate you being here, Josh. Thank you so much.
Yeah, thank you. Have a great day.
Bye, everyone.
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.
You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
Article published · Updated