Why the Next AI Frontier Is Physical: Caitlin Kalinowski on Robots, Supply Chains, and the Hard Discipline of Hardware
Lenny's Podcast
Caitlin Kalinowski has spent her career building consumer hardware. At Apple she worked on the original unibody MacBook Pro and was technical lead on the MacBook Air and Mac Pro. At Meta she led VR hardware, including the Rift and Quest, and then the AR glasses team behind Orion. Most recently she helped build OpenAI's robotics and hardware group from scratch. In this conversation with Lenny Rachitsky, she argues that physical AI is where the next wave of AI activity will go. She also warns that the obstacles are mostly unglamorous ones: safety, supply chains, component shortages, manufacturing know-how, and design tools that AI cannot yet handle.
What VR actually produced
Lenny opened by asking why VR never took off, given the billions spent and the quality of the hardware. Kalinowski said she would not have predicted exactly how things turned out. She does not treat VR as a waste, though. In her account, VR is where the industry worked out how to orient objects in space relative to both a simulated world and the real one. That meant SLAM (positioning in space using cameras), many uses of depth sensors, and a better understanding of how humans perceive visual data in space.
She now sees those same technologies being reused in robotics. A robot has to understand how it moves through space and how far it is from everything around it. Someone teleoperating a robot through a VR headset is using "the same real technology." She describes VR as "a step in a long technological arc." She calls VR gaming an interesting niche, and admits she does not use VR much now but is glad the industry did the work.
On why VR itself did not become mainstream, she gave a tentative answer. It may be the social cost of covering your face, which Google Glass had already shown matters a great deal. Making VR social was difficult because "you have your face covered."
AR glasses and the lineage into physical AI
Kalinowski does believe in AR glasses, mainly because she thinks constantly looking down at a phone is bad for people "as social creatures." Glasses that let you keep social contact while still getting information are where she thinks things are going. She said Orion is "a bit ahead of its time." It uses waveguides and microLEDs that are not ready for mass production: yields are too low and costs too high. She still considers it a likely path for AR glasses. Input is the other open problem, specifically how to communicate with glasses quietly in public while moving. Once that is solved, she expects a display that stays mostly off and turns on when you want it.
Lenny said he had used Orion. Kalinowski pointed to its 70-degree binocular field of view as the reason the prototype gives a real sense of the future: the field of view is wide enough that "you suddenly feel immersed."
Her larger point was that a single lineage runs from VR to AR to what she calls physical AI. That category includes autonomous vehicles, drones, robots, autonomy in general, and manufacturing. All of them need "the same piece parts" that were developed in AR and VR.
Why hardware is suddenly hot, and why it's hard
Lenny mentioned that computer science enrollment is falling at Princeton and other universities while interest in hardware and robotics is rising. Kalinowski called this "very odd." Hardware "has never been the sexy career." People went into it because they loved it, it paid less than other fields, and apart from Apple it was rarely at the center of attention.
She gave two reasons for the shift. The first is what she sees in the San Francisco labs: "a dawning realization" that AI progress is "going so vertical that what you can do behind a keyboard with AI is going to saturate." She was careful to say nobody knows when that will happen, and that it is what people believe will happen. Once it does, she said, the next frontier is physical: robotics, manufacturing, industrialization, a sensing layer in the real world, moving real objects, and eventually, she hopes, space.
The second part of her answer was about why software teams are surprised by hardware. She compares it to compiling code. Software engineers can compile and debug whenever they want. Hardware teams compile "four or five times" in total. Each major build means redesigning in CAD and releasing, and the final release goes to mass production. After that, no over-the-air fix exists; the only remedy is shipping a new product years later. So hardware teams have to be more conservative and run reliability testing as part of the program itself.
Part variance adds to the difficulty. Across millions of units, parts fall along a distribution of plus or minus three sigma or more, so the smallest version of one part will be assembled with the largest version of its mating part. Teams have to solve for "that last half a percent" before the final build so they get high yield, profitable production, and few returns.
Humanoids are still prototypes
On humanoid robots from companies like Tesla (Optimus), Figure, and 1X (Neo), Kalinowski said "we might be close," but her main concern was safety. Large, strong humanoids working next to people need enough data to prove they are safe. She praised designs such as 1X Neo that have "pulled mass inwards" and use softer materials. She explained the physics: the energy of an impact combines the arm moving through space with the rotating actuator. Compliance also matters, because a hard arm produces a high impulse and a soft, compressible one produces a lower impulse.
In her view, humanoids are "advanced prototypes." The industry is still at the stage of proving they work at all. The usual next step in her field is revising them to be cheaper, easier to manufacture, higher-yield, and safer. She pointed out that capable Chinese robots are available now, but their manuals say no human may be within three feet, and she said few robots strong enough to do meaningful work lack that kind of warning today.
When Lenny asked when humanoids would be common in homes and on streets, she focused on "at scale." In her background that means millions, though she was willing to consider hundreds of thousands. Getting there requires a working design and then enough reliability to run daily without much human repair. The first problem, though, is supply chain.
From magnets to actuators: the supply chain problem
She described the chain from the bottom up. First, raw materials such as magnets. Then processing those magnets. Then building actuators around them; she defined an actuator simply as the motor, where electricity goes in and motion comes out, usually a rotor with gearing that drives a limb, head, or fingers. Then integrating actuators into subassemblies and robots. She said each layer has been outsourced over the past 25 years to countries like China, Japan, and Korea, and she acknowledged she personally took part in transferring that engineering knowledge to Asia. Asia's historical strength, as she put it, has been scale and low cost.
Actuators may be a real bottleneck. Without magnets, engineers would have to design new actuator types using other materials, possibly larger and less space-efficient. Without actuators, robots cannot be built. She grouped these with batteries as "foundational technologies" that depend on material science breakthroughs. She considers diecast and machined parts less critical because she thinks those can be sourced. The US in particular lacks strong actuator companies so far, which complicates efforts to move production there. She said countries around the world are rethinking supply chains after seeing how fast disruptions like COVID and war can change things.
When asked why motors need magnets, she explained that a ring of magnets with alternating poles surrounds a central rotor, and alternating current makes the rotor spin.
Drones, war, and re-industrialization
Lenny brought up drones in the Ukraine war. Kalinowski noted that making a drone rotor spin uses essentially the same base technology and supply chain as moving a robot arm. She believes the US needs an independent supply chain, at least for military purposes, and thinks every country should have one. More broadly, she said the country needs to "re-industrialize significantly" to be safe militarily, because "people that are your allies now may not be in the future," and she sees the allied West going through major geopolitical shifts. She wants the US to "reteach ourselves how to make things at scale" and how to process raw materials, so that another COVID-style shock does not leave it unable to defend itself.
Lenny cited Marc Andreessen's scenario of 100,000 drones launched from China. Kalinowski said her friend Palmer Luckey, founder of Oculus and now Anduril, is right that the US should "invest a lot more in drones than in aircraft carriers." She called carrier-centric planning "an old way of thinking" and pointed to Ukraine, where drones are updated daily with 3D printing. She framed the issue as a cost-exchange calculation: what it costs an adversary to launch a missile versus what it costs to stop it. "Right now we're losing on the math," she said. That is tolerable for a while but becomes less tolerable the longer it continues. She is optimistic because of America's "pioneering, independent spirit" and engineering culture, but said "we need to move." She also said she has chosen not to work for companies that make lethal technology, while believing it is good that others are willing to.
Adversarial threats to physical AI
Lenny compared prompt injection in chatbots with the risk of a robot being manipulated into hurting someone. Kalinowski said controlling adversarial threats to the hardware layer, whether robots or drones, will be a major part of the future of warfare.
She told her own OpenClaw story. She sandboxed it on a separate computer, gave it a few pieces of real information including her real email address, connected it to Moltbook, and told it not to share her private information. Within five minutes it had posted her personal email address. Lenny described Nat Friedman's talk at Stripe Sessions, where his OpenClaw agent, asked to help with drinking more water and sleeping better, apparently changed his self-driving Tesla's destination to a stop off the freeway. Kalinowski's response: "Things are going to get weird fast."
What Apple taught, and bootstrapping hardware at Meta
On how companies build strong hardware organizations, Kalinowski described Apple as best in class. Hardware is "a first-tier citizen" there. Many people who were at Apple when she was, from 2007 to the end of 2012, now hold key positions across the industry. She credits Apple's training in thinking about "complex interdependent decisions and risk," which she says she did not recognize at the time.
She referred to a recent video in which John Ternus recalled learning from Steve Jobs about a cabinetmaker who finished the back of the cabinet. At Apple, she said, every design decision, including inside the device, is considered. She stressed that this is not only about aesthetics. It forces engineering, industrial design, and operations to ask what actually matters for each part and assembly, and if you are methodical enough, "what really matters tends to rise out and look very simple at the end." She described her own role as small: thermal lead on the first MacBook Pro, then leading later iterations of the MacBook Air and the cylindrical Mac Pro.
Oculus had a different culture. It started as a hacker hardware startup whose founders met on forums and turned consoles into portable backpacks. That ethos of rapid iteration suited a hardware team. Kalinowski was on the Meta side of the acquisition, and the challenge on the first Rift was professionalizing that culture: raising yields and volumes and bringing cost down.
Lenny suggested the lesson was attention to detail, comparing it to the brown M&M's clause in band contracts. Kalinowski corrected him. The core is "understanding why you're doing what you're doing," and then making every design decision serve that goal, in hardware as well as software and UX.
Her example was the Quest 2. The goal was to democratize VR, which meant lowering the price, which meant redesigning essentially the whole product for cost: removing cameras and components and changing materials and manufacturing processes. She said it still had low return rates, was "maybe even stronger" than it would have been otherwise, and, she believes, became the highest-selling VR headset ever.
Principles for building hardware
Kalinowski laid out several principles she has written about.
Define goals early and hold them. Hardware does not handle mid-course changes well. If a $300 target becomes $150 halfway through, most of the early work is wasted. She calls these goals KPIs and says keeping them stable may be the hardest part. With clear, prioritized goals, a team knows when it is done. That matters because each build iteration takes three to five months and timing is important: shipping a few weeks before a competitor can capture all the press and attention. She gave "$10 million" as an illustrative value for those days and said she was making the number up. She also admitted that engineers, including herself, "never feel like they're done."
Examples of KPIs: in VR, pixels per degree is central. Apple's Retina displays worked out what the eye can resolve, "probably overshot it a little bit," and after that there was less reason to push resolution. She said mass-produced VR is "not even close" to that point. For computers, KPIs include clock speed, parallelism, weight, price, and features. On the MacBook Air, the focus on weight and size, along with machining, led the team to drop features like the ambient light sensor. She has heard that Elon Musk sets explicit exchange ratios, such as the value of a gram against cost, so that "decisions fall out pretty easily."
Design the hardest parts first. People usually start with what they know how to design. She described an architect who, before finalizing a laptop hinge, began by checking whether the cables routed through it would actually fit. The best architects look first for "the pinch points."
Spend the most iteration where users touch. On a laptop that means the trackpad and then the keyboard.
Never wait. From Shelly Goldberg and Kate Bergeron at Apple she learned to do anything you know needs doing immediately, because a surprise will arrive in two days and you will need the time. She said this is not really about not knowing what is coming: in hardware, "you actually don't have more time."
Lenny asked about the MacBook Air in the manila envelope. She explained that the first version was a low-volume, machined device that was "more of a proof" of what was possible. She worked on the later wedge-shaped version that sold in volume. On the butterfly keyboard, she said she did not work on it, could not speak to it, and thinks current MacBook keyboards are excellent.
On Apple's supposed disregard for customer feedback, she said she left more than a decade ago and does not know Apple's current practices, but she thinks the idea is misunderstood. When something is truly new, like a touchscreen iPhone, customers cannot tell you what they want because they have not seen it; asked in advance, they would request a keyboard. Show it to them and they will know it is what they want. Getting stuck in an iterative feedback loop makes zero-to-one work very difficult. She noted this was her interpretation and not something she discussed with Jobs.
The memory price shock and the hierarchy of components
Mehul Nariyawala, CEO of the robot vacuum company Matic, sent a question through Lenny about the "meteor" of memory prices. Kalinowski said "we're in trouble as an industry." She said she is not an expert but believes AI is behind it, with data centers consuming supply and being less price-sensitive than consumer electronics makers. She has been advising startups to pre-buy memory if they can afford it, as a company she worked with had to do during COVID. When a key component is constrained, she said, you either pay or you already have stock. Pre-buying carries the risk that prices fall. Asked how bad it would get, she said she did not know and guessed prices would "probably double," joking that if she could predict timing she would be trading. When Lenny said he had seen figures around 6x, she said she had not realized it was that bad. She confirmed Lenny's summary that memory makers simply cannot produce enough to meet demand.
She used Matic to show how many parts a device has. Without having seen its CAD, she estimated 50 to 150 major parts: wheels, vacuum, mop, bag, liquid reservoir, a system she believes is SLAM-based that maps the room and keeps data on the device (which she called good privacy practice and similar to what Meta did in VR), wireless modules, SoC, RAM, and PCBs. Counting every capacitor, the total runs into the thousands. One missing component stops production.
She sees a hierarchy of severity. If a diecast vendor fails, a replacement takes maybe three months, five at high volume, which is recoverable. Losing your silicon or the RAM in your required form factor is "a catastrophic redesign": new board, new supply chain, a new production build, and full reliability testing again. In consumer electronics, silicon and displays usually have the longest lead times. For robots, even prototype actuators can take a month or two to get.
She pointed to Musk's vertical integration as an advantage in these situations. She said she has heard Starlink's factory takes in raw material and silicon and produces finished product, and that Tesla redesigned its PCBs in record time during the chip shortage, which would be far harder for a company with a traditional supply chain.
On off-the-shelf versus custom parts, her rule is to use off-the-shelf "whenever I can," especially in prototyping, where the goal is to prove something works. Teams build "works-like" and "looks-like" models, with the caveat that the working version must realistically fit the eventual industrial design, which younger companies sometimes miss. In mass production, KPIs for size, weight, and color often require custom parts. That is not because off-the-shelf parts fail but because they are not designed for the specific product. She agreed with Lenny that cheap drones exist because parts built and scaled for other uses are now abundant.
AI and CAD: close but not there
Kalinowski said AI is only at "the very, very beginning" of doing CAD. Claude, she said, can produce what are essentially surfaces or point clouds, which is not real CAD. Real CAD in her world is solid geometry defined by equations (she mentioned NURBS). If AI could do real CAD, she expects it to be one of the biggest changes her field has seen, especially if it took over tedious work like custom screws, 2D drawings, and tolerance stacks. PCB routing looks more promising: AI seems increasingly capable of routing boards and may soon handle basic component selection and layout.
Currently she uses AI for high-level planning, research, building databases of comparable products ("not perfect... but so much faster"), and Excel work, which she says speeds up design outside the core tasks.
Her main question is what it will take. LLMs and video models do not understand friction, weight, contact, pressure, or surface texture. She gave the example of folding paper four times, punching a hole, and predicting where the holes end up when unfolded. She suspects world models may need to be the foundation for CAD and physical engineering. "I want Codex for hardware engineering," she said, and she thinks that may require new model types. She did not know what Fei-Fei Li's World Labs is currently working on, and said existing models will be "part of the solution but not all of it."
Humanoids versus dedicated robots, and the CAD data problem
Kalinowski thinks humanoids are in a hype cycle, though she expects winners there. She doubts the idea of one generalist form for every task. Screwing a laptop keyboard to its case 10,000 times a week is a job for a dedicated manufacturing robot, and she thinks standardized, reconfigurable automation cells will be an interesting field. She said top-tier Chinese factories already have PCB lines with essentially no people, and mechanical assembly lines that went from roughly 200 workers to maybe 10. Since the most advanced manufacturing has already moved past human labor, "we don't actually need to replace humans with humanoids." She expects humanoids for long-tail tasks alongside specialized robots for construction, electrical work, low-volume assembly, and logistics, "and most of them are going to look different from each other."
On robots building robots, she said terms matter. She does not expect a single robot to build itself. She does expect AI to eventually let even a hobbyist go from a 2D picture to 3D CAD, assemblies, vendor communication, and a few builds, though not as well as experienced engineers at first. The biggest obstacle is data. CAD is "some of the most valuable IP" a company has, and companies like Samsung or Matic will not give it to model developers. That is why she thinks hobbyists, who do not care where their CAD goes and just want a faster drone, are where this will start, with incumbents moving more slowly. She floated the idea of an on-prem system that runs inside a company's own data center and trains on its data. That would still require a base model trained on a lot of CAD, and she is not sure what form the integration would take.
Making robots feel human, and robots in the home
Lenny's sister, a longtime VR content creator from Oculus, asked what makes a robot feel connected to people. Kalinowski credited researcher Leila Takayama. People expect others to acknowledge them when they enter a room, through many nonverbal cues, and a robot that does not respond "is creepy." She said she is surprised how creepy many current humanoids are, with notable exceptions. Robots should appear soft and non-threatening, show awareness and attentiveness, and signal intent before acting. A robot that looks before it turns is much less alarming than one that suddenly moves. She considers Pixar and Disney "probably the world's best" at conveying emotion, intent, and approachability.
About home robots, she said she is enthusiastic but her partner has a high bar. Her partner resisted Waymo, then took one ride and now wants nothing else, and loves the Matic. Kalinowski distinguished cars from home robots. Self-driving cars replace something with an existing safety record, so data can show they save lives, and she noted that people in San Francisco already behave differently around Waymos, which she sees as a trust-based change. A home robot is new and replaces nothing, so there is no baseline for judging whether it is good or safe, which makes it harder to sell. Lenny mentioned how a self-driving Tesla cannot do the eye contact and hand waves drivers use at intersections. She joked that cars might need little arms to gesture.
The next five years
Kalinowski said her job requires living far enough in the future to design not only the next product but the "platonic ideal" several generations out, because iterating toward a goal is easier than getting it right in one shot. She also called herself a skeptic, and said you want hardware leaders to worry about everything that could go wrong.
She expects AI to fundamentally change work over the next couple of years. She said coders mostly no longer code by hand, and knowledge work is next. She expects the physical world to change more slowly, except for drones and self-driving cars. She does not expect something like 20 million robots within five years because of the deep work needed on supply chains, raw materials, and rebuilding high-tech manufacturing in the US. In the meantime she expects "a lot of weird things on the street," such as delivery robots. She said "there's probably more change in war than there is in consumer electronics in the next two years," and added that democracies need to be able to defend themselves while hoping there is never a hot conflict.
Leaving OpenAI
About her widely viewed resignation tweet, Kalinowski said she has friends among OpenAI's executives whom she considers good people. She objected to the speed of decision-making, the governance, and "the lack of defined guardrails around the announcement of the Department of War deal." She said both things can be true. She wanted "a third path" between going along with the company and taking a scorched-earth approach. She said she helped build a robotics team with some of the best talent in the world and felt fortunate to do so, but after what happened she could not stay, because "you don't know what's going to happen next time." She hoped her decision would make it easier for others to talk about and hold their own boundaries. She tweeted because the news was going to be reported anyway. She declined to discuss what OpenAI is building.
How she hires
A former colleague described Kalinowski's strength as hiring exceptional teams. For zero-to-one work, she said, you cannot rely only on people who have done the same thing, because nobody has built robots that move through the world at the scale of millions. Autonomous vehicles are a good source of talent because of the sensing stack, the safety tradeoffs, and the hardcore engineering. She looks for strong generalists who can carry knowledge across fields, a mix of people who have built new things and people who have scaled products, and young AI-native engineers, often around 20 or 21, who she says approach problems completely differently and "much faster." She compared this to her own generation growing up as the first digital natives, and said "we have to accept that we're not native in these new technologies."
She rejected the idea that junior roles are disappearing. Teams "must have both" senior and junior people, though they may be somewhat smaller, and she does not yet know how AI-driven hardware work will change them. She added mission alignment, which she considers especially important when AI researchers and hardware engineers, coming from very different worlds, have to work together. Finally, she relies on gut feel for a real spark: people motivated by learning and excellence, open to changing their views, and who "want to win."
Lessons from Altman, Jobs, and Zuckerberg
From Sam Altman she learned to ask "Why not more? Why not 100x or 10,000x?" She realized she had been thinking too small in some areas, and she valued his willingness to think in very large numbers. From Steve Jobs, she took an unwavering bar for technical talent and excellence that ran through the whole company. She found being told as a young engineer that work was not good enough to be very motivating: "you never want to hear that again." From Mark Zuckerberg, she took how well-run the company was. Hardware interfaced cleanly with the rest of the organization, objectives were clear, reviews existed to make decisions, and decisions were pushed to the lowest possible level. She said she had underappreciated this. She was impressed that Zuckerberg and CTO Andrew Bosworth read 20-page technical reports, understood the tradeoffs, and contributed, across what she estimated as a hundred such decisions a month.
A failure: the Quest camera spec
Her failure story came from the first Quest, around EVT, the first build using final components and materials made on production tooling. For cost reasons the team had gone from five cameras to four. Just before Christmas, the computer vision lead reported that the tracking system could not get a lock on the headset's position. The two teams had interpreted the same tolerance spec (about 0.15 mm, as she recalled) differently: the hardware team as a per-part plus-or-minus, and the vision team as a global tolerance. Under the hardware team's reading, the vision team could not meet its goals.
The fix was architectural. The team locked the bottom two cameras to each other on a bracket, made of steel as she remembers it to hold the needed tolerance, and let the other two float. She called it a failure in understanding the spec and in product design. The build and the ship date stayed on schedule, and she thinks the design ended up better, because the fixed pair gives a source of truth for space that the other two cameras overlap with. She wishes they had caught it four months earlier. The two closer-set cameras are visible at the bottom front of the shipped Quest.
Closing: the power of the individual
Kalinowski called this "one of the most exciting times" and said fear is normal. She urged people to use AI tools daily in their work, find their limits, and retest with every new model, saying she has "never been more excited about the power of an individual." In the lightning round she recommended The Book of the New Sun (she was unsure of the title and planned to confirm it), Virginia Woolf's Mrs. Dalloway, and Herodotus's Histories. She is watching the new Euphoria, which she recommends treating as a soap opera. She likes Vollebak clothing for its use of material science. She keeps an image of branching possibilities as a reminder to stay in the present. She hired a postdoc to tutor her through Joseph Brodsky's reading list of the Western canon, because she found AI useful for the basics but inadequate for cultural context.
Asked how listeners could help her, she said imagining the future is "a multiplayer game." Rather than accepting a sense that the future is dystopian, she wants people to decide together what they want the future and the human role in it to look like, describe it in fiction and conversation, and then build it.
There's a dawning realization, especially in the labs, the acceleration is going so vertical that what you can do behind a keyboard with AI is going to saturate. When that happens, the next frontier is the physical world. Robotics, manufacturing, industrialization.
You're living in the future and designing it.
There's probably more change in war than there is in consumer electronics. In the next 2 years, we need to invest a lot more in drones than in aircraft carriers.
Just imagine 100,000 drones coming out of China just at us. I do feel that we need to re-industrialize the country significantly to be safe in a military sense. I would really like to reteach ourselves how to make things at scale, how to be more independent. People that are your allies now may not be in the future.
You worked with some of the most legendary successful builders. Steve Jobs, Mark Zuckerberg, Sam Altman.
Sam is really good at saying why not more? Why not 100x or 10,000x? You're thinking too small. For Steve, the bar he held for the company for technical talent and for excellence was not wavering.
What does it take to create a robot that feels human and connected?
If you walk into a room and a robot's just like, like it's creepy. You want these devices to be non-threatening, appear soft, reactive to you. Pixar, Disney are probably the world's best at doing this type of design work.
There's a meteor called memory prices that are coming for consumer hardware and robotics and physical AI. We're in trouble as an industry.
Today, my guest is Caitlin Kalinowski. Caitlin is one of the most sought-after and accomplished hardware leaders in Silicon Valley. She was part of the original unibody MacBook Pro teams and technical lead on the MacBook Air and Mac Pro at Apple. She led the AR glasses hardware team at Meta, including the team behind Orion, their most advanced AR product. Before that, she ran the VR hardware team at Meta, where she helped design all of their incredible VR devices like the Rift and the Quest. Most recently, she was at OpenAI, helping build their robotics and hardware division from scratch. Robots and hardware and physical AI are so hot right now. Every AI company, and so many startups are launching, building AI hardware products, and Caitlin has been at the center of this emerging field for decades. This conversation goes in a lot of different directions, many that I did not expect, and I hope to do a lot more episodes on the hardware side of building over the next few months.
Before we get into it, don't forget to check out lennysproductpass.com for an incredible set of deals available exclusively to Lenny's Newsletter subscribers. With that, I bring you Caitlin Kalinowski.
Caitlin, thank you so much for being here. Welcome to the podcast.
Thank you so much for having me. I'm excited to be here.
We're going to go in a bunch of different directions. I'm going to bounce around. I want to talk about VR. So much money, so many resources. So many smart people have been working on VR for so long. Meta spent, I don't know, 10 billion dollars. Like they renamed the company Meta to lean into VR as the future of this metaverse that we're going to be living through. Feels like a lot of people are leaning out now. Feels like Meta's stepping back. Apple's stepping back with the Vision Pro in spite of the incredible hardware that you built, that your team built. Just like, I've got a couple of the devices. It's just like a magical experience, unlike anything you've ever experienced, and it still has not caught on. What happened? Is there still a future where VR catches on or is the future kind of AR and something else?
I don't think I would have guessed exactly what happened here, but the way I look at it is VR helped us understand how to orient things in space relative to a simulated world and the real world and connect those two. We figured out SLAM, which was how to do positioning in space using cameras. We figured out a lot of applications of depth sensors. We figured out how humans perceive visual data in space. And all of that actually, while it's great for VR, and I think VR gaming is a really interesting... it is kind of a niche, but I think it's an interesting niche. What I see now is in robotics, all of these technologies are being used because you need to understand how the robot is moving through space. You need to understand how far it is from everything. You need to understand if you're wearing a VR headset and driving the robot. It's the same real technology. And so for me, I view it as a step in a long technological arc.
And to be honest, as someone who's not using VR a lot right now, I'm really glad that we did it. But I don't think it... I expected it to be big obviously, or wouldn't have been working in Oculus. And I think maybe the social aspect of having something in front of your face is part of why it didn't take off. And I think that we learned of course with Google Glass how important that is as well. And so when we tried to make it social, it's hard to make it social when you have your face covered.
That is interesting. So just like the investment and innovation that went into VR has actually proven to be really useful, and so it feels like the companies that have put a lot of effort into that and money into that are ahead on the next step. So is that where you think things go? Where do you think things are going? Is it AR glasses, something else? What's kind of the future of this?
I believe in AR glasses as part of the future because I do think looking down at your phone all the time is not great for us as social creatures. So, if you can maintain social connections and get information, that's where I think we're headed.
Orion, the AR glasses we worked on, I worked on most recently, are a bit ahead of their time because they're using waveguides and microLEDs that are not quite ready for mass production. The yields just aren't there. The cost is still high. I think that's absolutely a path that AR glasses are likely to take. And as we figure out the input to those glasses, like how do you communicate with them when you're on the move, when you're in public, how do you communicate quietly, silently, with them? I think once we start to figure out some of those challenges, having a display that's mostly off that you can turn on when you want it to be on seems like part of the future.
So, that's part of it. The other part is there's this lineage of technology going through VR and then AR and now in... I'm using the term robotics, physical AI, but you really have to step back and look at autonomous vehicles, drones, obviously robots, autonomy period, manufacturing. Like all of these technologies are going to need the same piece parts, the same pieces that we built in the AR/VR spectrum.
It's interesting with VR, there's this idea when you build product, there's always this question when something doesn't work. Is it just like you executed it badly or the idea was just a bad idea? And it's always hard to know. Feels like with the AR it's just like so much effort was put into making it work, just like for a decade, many decades, and it just has not worked. So it's like nice that we know, okay, there's nothing we can really do right now to make this work. I completely agree with you. The issue is just like I don't want to sit on my couch disconnected from the world, and even if I could see people through it, it's just like I'm just gonna... I don't need this. It's not that big of a deal.
And AR, you're going to just start getting more and more larger and larger displays. But the great thing about Orion is you got 70 degree field of view binocular. So with the prototype you got to sense what this is really going to be like in the future. It's very hard to describe how it feels to use a pair of glasses like this. But when you do, you suddenly are like, oh, like I feel immersed. The field of view is wide enough. I feel immersed and it becomes pretty clear that I think this is part of where the future's headed.
This episode is brought to you by our season's presenting sponsor, WorkOS. What do OpenAI, Anthropic, Cursor, Vercel, Replit, Sierra, Clay, and hundreds of other winning companies all have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you've felt the pain of integrating single sign-on, SCIM, RBAC, audit logs, and other features required by large companies. WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built specifically for B2B SaaS. Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS. And that's because they are the best. Whether you are a seed-stage startup trying to land your first enterprise customer or a unicorn expanding globally, WorkOS is the fastest path to becoming enterprise ready and unblocking growth. It's essentially Stripe for enterprise features. Visit workos.com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions. WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.com to make your app enterprise ready today.
Okay, I want to talk about robots, robotics. I was meeting with a bunch of Princeton students a couple months ago, and they're kind of like CS students, and they were telling me that enrollment in CS at Princeton is trending down, and I confirmed this is actually true at a lot of universities. There's a lot of charts that show CS enrollment down, and where it's actually going up is hardware, robotics, which I imagine as someone that has been in this field for a long time is very weird because it's never been that popular. How does it feel to feel like, oh wow, everyone's getting it now?
It's very odd. Everyone is suddenly asking about hardware and robots and the physical world, and it's never been the sexy career. It's always been the thing that you went into 'cause you loved it. It never paid the same as these other careers. It was never kind of at the forefront of how we talk about things, with the possible exception of Apple obviously and the hardware lineage at Apple. So it's great in some ways and it's very odd in others.
What's surprisingly hard about hardware? A lot of software companies, a lot of people are just like, "Okay, cool. We're going to build some hardware. That's the future. That's the moat now." And they get into it and they're like, "What the heck?" What are some things that maybe people don't think about when they think about, "Okay, we're going to build some hardware"? What are some of the surprising challenges that come up?
I like to talk to computer science folks about it this way. So, computer science folks, as you know, they write code and then they compile the code often and then they run the code and debug it. But they can compile their code every day, every hour, whatever they need to do. In hardware, we only get to compile our code, quote unquote, like four or five times.
Four or five times a year or...
Total.
Okay. Ever.
Right? So if you're building hardware, you redesign it in CAD for every major build and then you have to release it. And once it's released, you compile it the last time you release it for mass production. If it's a mass production device, that's it. You're done. You can't ship over-the-air updates. So, we have a different approach. We have to have a different approach, which is more conservative. You have to do more of the reliability checks and tests in line with the program because once you compile that last time, you're done. You make all the parts, you put them together, they're out in the world. The only alternative is to ship something new to replace it a couple years later. And so we have to be more conservative and we have to take our time because if you think about it, a product that sells millions, if you have a graph of all the parts put together on any different part of the device, you have a curve, you're in the plus and minus three sigma or more, right? So meaning if you have two parts that go together, you're going to get the smallest version of this one and the largest version of this one, and you're going to have to put those together across the board. People don't think about this that much, but the part variance is pretty high.
And so we've got to solve for that last half a percent in the process of building so that when we compile our last time, when we build our last time, it's done. And we're going to have a high yield. We're gonna be able to make them and make money on them effectively, and we won't have very many returns. And so that's kind of the game that we're playing.
Sounds so hard and complicated. It's just like software is so nice. You just write some code, ship it, it's great. Why do you think people are getting so into robots and hardware now? What's kind of the driving trend?
Yeah, what I'm seeing in the AI world in San Francisco is there's a dawning realization, especially in the labs, I think, that the acceleration is going so vertical that what you can do behind a keyboard with AI is going to saturate. Now, I don't know when it's going to saturate. Nobody else knows either. But when that happens, the next frontier is the physical world. And so what I see happening is the labs, big tech, startups are all realizing at the same time, okay, this is coming. We're going to have complex systems that can solve problems in the digital world very, very quickly. We already have them. They're going to get better and more comprehensive and more capable. If you think about that as a frontier, you can see the end of that tunnel. Now, I don't know when it's going to be, again, but we can see that that's going to saturate at some point, or at least people think it will. And when that happens, the next frontier is hardware. The next frontier is robotics, manufacturing, industrialization, the sensing layer in the real world, the ability to move objects in the real world, and eventually, we hope, space.
So, one of the most interesting lines of development is these humanoid robots. It's kind of like, you know, our meat brains are always more attracted to robots that look like us and act like us. There's a few companies very ahead. There's Optimus, Tesla, there's Figure, there's Neo, there's a few others. What's your sense on just the current state of these humanoids and, I don't know, how close are we to humanoids being around us?
We might be close. I have, like many others, safety concerns about large, strong humanoids operating right next to people because we have to have enough data to show that that's safe. There are some designs, and 1X Neo is a good example of this, that have made significant safety considerations in their designs and pulled mass inwards essentially, which is a lot safer. Softer robots is safer.
Just to clarify, you're saying they're lighter and so the impact of a robot hitting you is less.
Yeah. The part that might hit you, which in this case might be the arm, if it's lighter and softer. There's two aspects. You have the arm moving through space and then you have the actuator that's rotating. So you have to add up the energy essentially for both of those things. And so that's an impact thing that you have to worry about. Then you have to worry about the compliance of the arm. If it's just hard, then you know the impulse is high. But if it's soft and compressible, then the impulse is lower. And so you really have to be thinking about this when you have robots around people.
So in my world, in my worldview, the humanoid robots are still prototypes. And they're advanced prototypes. What we need to do is show that this works at all, which is kind of where we're at right now. Once we have working prototypes, then usually, at least in my field, what you do is you continue to revise them to make them
cheaper, easier to manufacture, higher yield, and safer. And I think this is what's going to happen next. So, in my mind, they're not quite ready yet. Now, you can get a Chinese robot that can do all kinds of things for you, but if you look at the booklet, it says, "Hey, you can't be within 3 ft. No human can be within 3 ft of this robot," and you're not going to see very many robots that are strong enough to do meaningful work that don't have that warning right now.
That is so interesting. It's funny to hear that. At the same time, there's these nunchuck-wielding robots in China doing dances with other folks. I've never thought about just that part of it, like the impact they can have if they go awry. I want to come back to that, but just timeline-wise, what's your sense realistically of when humanoid robots are walking around the streets, in people's homes, kind of at scale?
At scale is the problem in my mind. At scale is a huge challenge. Now, for me, in my background, at scale means millions usually. But let's even say hundreds of thousands. You've got to get a good design that's running. Then you've got to make it reliable enough that it can keep running day to day without a lot of human intervention or repair. And that's its own problem.
But the first problem you have is supply chain. And this is going to be something that I hope that we can talk about a little bit more. But every single part that goes into that robot's coming from somewhere. And many of these parts may become more restricted or difficult to make, and it may be harder to assemble the subassemblies and the meaningful parts of the robot here in this country. So there's a very complex supply chain dependency right now on robots like humanoids, but also other robots, that we have to figure out. And a lot of people are trying to move production here to the United States, which is very challenging because we don't have great actuator companies here yet, for example.
And the actuator is like the little arm? I don't know, how would you describe an actuator to a non-robotics person?
Yeah, the actuator is the motor. So, you put power into it, electricity into it, and you get motion out of it.
Cool.
And most of these robots have a rotating rotor essentially that then has gearing on it that then powers the limb or powers the head or the fingers or whatever else. So they could be small, they could be large.
Okay, awesome. I hear the word a lot. I'm like, I don't know exactly what it means. Thank you for explaining it. I want to talk about the supply chain stuff more because I know you think a lot about this. What's kind of the state of the union on the supply chain for, say, robotics? What's going on? What are the pieces? What are the challenges?
So the way to think about it is you can start with raw materials, and magnets is a good place to start. So we need to be able to get the magnets, the raw magnets, for example. Then we need to be able to process them. Then we need to be able to integrate them into actuators and build the actuators around them. Then we need to be able to integrate those actuators into subcomponents or robots themselves. And each layer of this chain has essentially been outsourced over the last 25 years to countries like China, like Japan, like Korea. And in full transparency, I've been part of that transfer of engineering knowledge to Asia.
In Asia, the expertise has historically been scale and being able to build a lot of these parts at lower prices. We've had this kind of deal across these borders that this is how we're going to operate for the most part. Now, of course, there's things we make in this country still. And of course there's design and AI that's made in Asia, but that's essentially where things have been for a long time. And in order to have a safe supply chain, we need to start to work on having some independence in these layers and these stacks.
And it's interesting that your focus is on these actuators. Is that the bottleneck? This very specific part of a robot?
It might be. So, if we can't get the magnets, then we have to design new actuator types that maybe use different materials, that may be larger, that may not be as efficient in space. So, that's important. And then the actuators themselves are important because if for some reason we can't buy them, then we don't get to make robots. So, it's foundational. There are some foundational technologies like this, all backed by material science breakthroughs essentially. There's batteries, of course. There's actuators. The raw parts, like the diecast parts, the machined parts, are less critical. We think we can get those. But I think everyone, not just in this country but around the world, is starting to think about supply chain because you have these disruptions, whether it's COVID or war, and you see how quickly things change.
Okay. Super question. Why magnets? Why is that a part of the supply chain? Why do we need magnets?
Yeah. So, it's a great question. So, you have a ring of magnets that are polar opposites and they go like this around the ring, and then you have something in the center that rotates, and the way it rotates is you have alternating current essentially, and so the magnets make the rotor spin.
Wow. We need a YouTube lecture of here's how this physics works. Okay. Very cool. So when you talk about China, what I think about now is watching the war in Ukraine and Russia, just drones, just how crazy and different the world is now that you can build these little drones that go and, you know, blow people up. Robots are a part of that. It's just such an existential threat to every country now, the ability to build these things at scale. What's your advice? What should we do? What should we change to thrive in this future and not be, you know, in trouble?
Well, you mentioned drones. It's another good example. You need essentially the same technology to make the rotor spin on a drone as you do to make an arm move on a robot. It's essentially the same base technology and supply chain. So, we need to, at least on the military side, have an independent supply chain as much as possible. I think that's important. I think every other country should do that as well, but I don't think that's specific to us.
I do feel that we need to re-industrialize the country significantly in order to be safe in a military sense. You really never know what's going to happen in the future. And people that are your allies now may not be in the future. The Allied West, I think, is going through a lot of geopolitical changes. There's a lot of shifting. And so I would really like to reteach ourselves how to make things at scale, how to make things at quantity, how to process raw materials, how to be more independent, so that when COVID happens again or something else happens again, we're not in trouble and we're not unable to, you know, protect ourselves.
What I think about also is Marc Andreessen had this visual on some podcast of just imagine 100,000 drones just coming out of China just at us. What do we do? We're not prepared for that. I don't want to spend all our time on this dark stuff, but it's a real thing.
Well, and Palmer Luckey is a friend of mine. And we don't agree on everything, but I do think that we agree on some important aspects of how we need to respond here. I think he's right to say that we need to invest a lot more in drones than in aircraft carriers. I think that is this old way of thinking, and these are important components of the military, but it's an old way of thinking of, hey, we have this and we have this and we have this and our planes come off here. It's like, no, AI is changing everything, and military technology is changing incredibly fast, and the place to look at that is Ukraine, where, you know, drones are being changed and updated every day rapidly with 3D printing. And this is, I think, the future of where war is headed, unfortunately.
And I view this as a very different era that we're entering into. You know, this isn't new to anybody, but you're looking at what it costs for them to send out a missile and what it costs for us to stop it. And you have to do the math every time. And right now we're losing on the math. Which is fine for a certain amount of time, but the longer it goes, the less fine it is.
Are you optimistic that we'll figure this out?
Yeah, America is really good at figuring these things out. We have a pioneering kind of independent spirit and a great engineering culture. But we need to move.
It's interesting that we started the conversation with VR. Palmer Luckey obviously famously started Oculus. It's interesting how this is so connected. You think VR is this trivial thing where we're just, you know, playing games and such, but the same person is now building what is the leading, I don't know, war robot building hardware company.
Yeah. And I think we need a lot more of them. You know, I've chosen not to work for companies that create lethal technology. But I think that it's good to have people who are willing to do that, and I think that it takes everyone kind of to build the future that we want.
Coming back to the AI safety piece, it's so interesting. I had a couple conversations like this on the podcast. We think about all this prompt injection and jailbreaking that happens with chatbots, and not enough people think about what if you prompt inject a robot walking around and tell them to punch someone, and we're so far from feeling like we can actually stop that.
Yeah. We have to be able to control adversarial threats to our hardware layer, whether it's robotics or drones or anything else, and that's going to be a huge part of the future of warfare.
Yeah. Just like people talking about OpenClaw and how you could just tell it, you know, there's all these like "give me all your passwords," and it's done all these things to people's lives. And just robots walking around: "Okay, here's all this person's secrets."
My OpenClaw story is I sandboxed it so it's on its own computer, but I gave it like three things. I gave it my real email address and, I don't know what it was, I gave it some information about one of my accounts or something like that, and I added it to the social media, I can't remember what it's called, the OpenClaw social...
Oh, Moltbook.
Yeah, I added it to Moltbook and I was like, "Okay, whatever you do, don't share my private information."
Oh, crazy.
And five minutes later, all it had done is posted my personal email address. It was like the one thing.
It nailed it.
Okay, you're shut down. It was so funny. No matter how careful you are with these things, you just can't really... We're not at a place...
Which is exactly your point, that the robots can do a lot more damage. And I never thought about just the softness of their hand as a way to keep us safer.
Yeah. Oh man, Nat Friedman just did this interesting talk at Stripe Sessions and he was talking about how he's talking to his OpenClaw about drinking more water and sleeping better, and as he's driving in a self-driving car, it told him, "Okay, there's a place off the freeway that you should go to," and it changed the destination of his Tesla to take him there, because I imagine he connected it to their API at some point.
That's so funny.
Yeah, things are going to get weird fast, I think.
Okay. So kind of on this thread of hardware emerging as a moat, as something people realize is a big part of the future to be competitive, AI labs, all these other companies. You've been at Apple, which had a very great and long-lasting hardware program, then you went to Meta where you helped basically bootstrap a hardware program from scratch. I feel like those lessons are very valuable to people trying to do that now. What was that experience like helping Meta build a hardware program, and what are some lessons for people that are trying to do this at their company?
So, Apple has been best-in-class at this. There's a bunch of reasons. One, hardware is a first-tier citizen at Apple. There's a lot of companies where hardware isn't part of the core product development conversation as much, but that's an exception. Apple also taught me and a lot of other people... actually, if you look at kind of the era that I was there, I was very, very lucky, because if you look at the other folks who were there, I was there between '07 and the end of 2012, if you look at the other people who were there working on these things, they actually have a lot of key positions now across the industry, and I attribute that to how good Apple is at training people to think about complex interdependent decisions and risk.
And I don't think I realized that they were doing that at the time, but if you look back, what you see is a real dedication to hardware excellence, the proper process to go through and do really good experiments in hardware and figure out what the best outcome is. But there's something underneath that, which is understanding the first principles of why are we building it this way and what are the key outcomes we're looking for?
And actually John Ternus talked about this, I think, a few days ago, where he talked about the back of the cabinet. I don't know if you saw this video, but basically John said that he was impressed that he learned from Steve Jobs that there's a cabinet maker who finished the back of the cabinet, and how important that was. And that goes very, very deep at Apple, where every single design decision, even on the inside of the device, is considered. And this isn't just an aesthetic decision. What it does is actually force the engineering, industrial design, operations community there to think about what are we really doing and what's the core of what's happening for this part, for this assembly, for this consumer product, then what really matters. And what happens is if you're that methodical, what really matters tends to rise out and look very simple at the end.
And so part of what you're seeing in many folks coming from that era is an understanding of how to do that. In the very beginning, on the Mac side, Macs didn't sell as many and the quality wasn't quite as high, but by the end of that era, you know, Macs were very popular and selling in much higher volumes, and so I think that made a big difference. And I was only a small part of that. I was, you know, the thermal lead on the first MacBook Pro and then over time worked to lead successive iterations of the MacBook Air and the cylindrical Mac Pro. But I was lucky enough to work with these folks and learn from them who'd been doing this for a really long time.
So you have to take those lessons and then, when you leave, try to distill them and explain them to a new community. Now, Oculus was actually a hacking hardware startup. Oculus started from folks who actually met on forums, you might know this, Lenny, who were hacking PlayStations or Super Nintendos into portable backpacks. And so there was an ethos at the company that was actually quite good for the DNA of a hardware team. And then I was on the Meta side when we did the acquisition. And when we acquired them, they had that spirit of rapid iteration. They had made Crescent Bay before the acquisition, I think, but then to professionalize that, get the yields up and get the volumes up, get the cost down, was kind of the challenge we faced in the first Rift.
So one lesson I'm hearing here is being very detail-oriented. I don't know if that's the right word, just like
focus on every element of the end product because, to your point, it's not just about that back of the cabinet. But I think about it like the brown M&M's story, where a band puts in the contract, "You have to have brown M&M's in the room," because that means they read it. And it's not like M&M's matter. It's that it's a test that they read the thing. Is that kind of the message there?
I think the message is understanding why you're doing what you're doing, and then every design decision supporting that goal. And that requires a lot of detail, and it requires a lot of persistence, and that requires a lot of consistency. But understanding why you're doing what you're doing and what the end goal is, is I think the key, and letting that expand into not only the software and the UX but also the hardware.
What's an example of that, just to make it more concrete for us?
A great example is the Quest 2. So we reduced the Quest 2 price quite a lot. And what we had to do is understand what are we trying to do? We're trying to democratize VR. We're trying to get VR to more people. And the only way we could do that is reduce the price. And so what it required is a redesign of the entire product essentially for cost, which then I think led to the highest selling VR headset of all time. And it was not easy because you had to, in our case, remove cameras, remove components, change materials, change manufacturing processes. But when you have alignment that you want to get this to more people and the way to do that is to reduce the cost, then that kind of drives everything else. And it was still a very high quality product with great, I think, low return rates, and it was a very strong product, maybe even stronger than if we hadn't done that, funny enough, but it hit our price point.
Okay, coming back to just the question of, say, companies are like, "Okay, we need to build some hardware. We're going to build our own glasses. We're going to build a little phone device, some secretive thing, whatever OpenAI's up to." What other tips do you have? I know it's impossible to be like, here's all you need to know, but what else should people be thinking?
Having your goals defined early and sticking to them is important. Hardware is not as adaptable to lots of changes throughout its development as anything digital. And so if you set out to say, "Okay, we want to make something that's $300," and then halfway through you say, "Oh, it actually has to be $150," you've almost burned a lot of that early time. So you kind of need to have a sense of having pre-thought out what you want, and having those, I like to call them KPIs, but essentially goals written down, and try to change them as little as possible. So that is very tough. In fact, that may be the toughest thing, because if you do that properly and you have the right prioritization of those things, you know whether you can ship or not, you know whether you're done.
And in hardware, one of the challenges is, you know, we talked about compiling four or five times. Every time you build and you iterate your design, that's another 3 months or four months or five months or whatever it might be. And so you're trying to time the feature set with the quality with the timing. And in hardware, timing is important because if you come out with your product a few weeks before your competitor, you might get all the PR, you might get all the interest. It's pretty brutal. And so each of those days that you ship before your competitor is worth a lot of money. It might be worth $10 million to you. I'm making this up. So you have to balance that with how many times you iterate. And if you know what your goals are up front and you hit them, then you know you can ship. And often engineers, and I'm guilty of this too, especially on the hardware side, never feel like they're done. So this is a pretty nuanced thing. So that's one thing.
The second thing is we tend to design the things that we know how to design first. And actually the right approach is to design the hardest parts first. One example would be, and there's no IP here, so I'm obviously not going to share any IP or anything internal, but at one point we had to route cables through a hinge in a laptop we were making. And because it wasn't clear that those cables would fit, that's where the architect started. And he looked at the diameter and how to split the cables out and made sure that they would fit before finalizing the hinge design. A lot of people would start at the part they knew, like, we're going to use this display, so I'm going to put this in CAD and I'm going to do all this other stuff. But the architects who are the best actually look at where are the pinch points, where is this going to fail, and they start to do the detailed design there first.
And then a couple other points: the part that your customer touches or interacts with the most needs way more iteration than everything else. So easy: on a computer, you touch the trackpad the most and then maybe the keyboard next. So those things have to be really good. They have to feel good. They have to respond properly. They have to be highly reliable. And then maybe the other pieces further out don't take quite as much iteration. So you have to boost your iteration on the things that people touch the most or interact with the most. So those are kind of some principles that I wrote about. But these are just things that you learn trying to build quickly.
And the last piece that's really critical if you're making hardware, for folks out there who are trying to make hardware, is you can't wait around ever. There's never enough time. So if you know that you need to do something, what I learned from folks like Shelly Goldberg at Apple, who I think is a VP now, and Kate Bergeron when I was there at Apple, is you need to do it right now. Anything you know you need to do, you need to do right now, because in two days there's going to be a surprise coming around the corner that you need that time to fix. And so this sense of stacking the things that you know you need to do and just getting them out of the way, even if you technically have more time, is this kind of ruthless efficiency that I learned with them.
Amazing. Okay, let me just summarize your advice here. So one is be very clear on goals. I want to come back to this. Two is do the hardest part first, the riskiest piece essentially to physically build. Three is focus on the pieces that people will use most, say the trackpad, a keyboard. I want to talk about that. And four is just do it now. Even if you think you have more time, you never know what's around the corner and
It's not even that you don't know what's around the corner. If you're working in hardware, you actually don't have more time.
Okay. On the goals, what are kind of buckets of goals? So cost is one, when you shared like, we need this under $300. What are some other buckets of types of goals people should be thinking about?
So in VR, display resolution or arc minutes, like how many pixels per degree do you want, is actually one of the key metrics. So you need to understand what your key metrics are. And why is that key? Well, that's your visual field. So you think about Retina displays on MacBooks. They figured out the KPI of what the human eye could see, probably overshot it a little bit, and built that. And then do you really need to keep as much engineering pressure up on the resolution of a display after that? Maybe not. So VR is not there yet, not even close. So in mass-produced VR, we don't have Retina displays yet. So pushing that up is one example.
I think on a computer, obviously you're talking about clock speed, you're talking about how many parallel processes you can run, you're talking about weight, you're talking about price, and you're talking about features. So when we did the MacBook Air, it became very clear, because we were machining it, that there are certain features like the ambient light sensor that just didn't make sense anymore. And so being willing to just jettison them for what we were going for, which was weight and size. So if you have those overarching goals, you can actually make engineering decisions pretty quickly.
And this is actually something that I think Elon, I've heard, does very well, which is define the value of, you know, a gram of weight versus the cost. Or he does, I've heard, engineering ratios essentially, and he's able to put numbers on what those ratios should be, which I think is really smart.
Interesting. So it's a very easy trade-off. Okay, here's the formula telling us weight is less important in this case.
Yeah. And if you can do that, then the decisions fall out pretty easily.
Speaking of the Air and weight, I feel like there's a very classic moment in Steve Jobs lore where he comes out and has this manila envelope and has the MacBook Air inside it, and then takes it out and everyone's like, "No way." Were you a part of that? Was that something that people wanted to do from the beginning?
I think, if my memory serves, the very, very, very first MacBook Air was a pretty low volume device that was machined but was more of a proof of what could be done. And that was the manila envelope one, I think, where the side door opened out to give you the port, and it had this shape underneath. And then the next rev of that was the MacBook Air that we know, which is wedge-shaped, which is different. And so the wedge shape is the one that I worked on and the one that went and hit more volume, but that manila envelope one was the one that proved you can see a computer. And so they each have really important roles in the roadmap.
Coming back to your point about focusing on things that people use the most, famously Apple screwed up the keyboard. There was this butterfly keyboard situation for a long time. Your eyes are closing. What happened? What happened, Caitlin?
I didn't work directly on that keyboard, so I can't talk about what happened with it. But obviously this is something that you've got to get right. And I will say the modern MacBook keyboards are awesome and excellent, and, you know, I don't know what happened with that. I don't think those were devices I was working on at the time.
Nice. Safe, marked safe. Along these lines, Apple's kind of famous for not listening to what people want. This is kind of a classic thing with Steve Jobs. He's not walking around doing user focus groups, doing user research. Somehow continues to build incredibly popular products. What do you think they do right that allows that? Or do they do a lot of user feedback sessions, things like that? How does it end up working out?
It's been a long time. I mean, I left over a decade ago. Don't know what they're doing now in terms of user feedback. I think this one gets misinterpreted though, Lenny. I think that what is being said is if you want to build something new, customers don't know what they want because they haven't seen it. So a good example is the iPhone, which I didn't work on, but when you build a new iPhone with a touchscreen, you can't really go ask 100 people what they want, because they're going to say a keyboard on their screen. And this is, I think, the ethos that you're getting at, which is, and this is true for anybody building a new product with a new feature.
And I've tried to build, as much as I can, teams that work on products that have something new about them. Either they're a new category or there's a new manufacturing process or something that hasn't been done before. And when you're thinking about this, you can't really use what you learned from the same field and the same product class. It just doesn't work, because you actually won't get the answer right. And I think this is actually what Steve was talking about, which is you can't get intuition if you're changing something fundamentally. Your customers won't know what they want because they haven't seen it. But if you show it to them, they will absolutely know that it's awesome and that it's what they want. But if you get stuck in an iterative feedback cycle with your customers, it's very hard to go zero to one with something new. And so in my view, and I don't know for sure, I didn't talk to him about this, but that's my view of what that means.
I am so excited to tell you about this season's supporting sponsor, Vanta. Vanta helps over 15,000 companies like Cursor, Ramp, Duolingo, Snowflake, and Atlassian earn and prove trust with their customers. Teams are building and shipping products faster than ever thanks to AI. But as a result, the amount of risk being introduced into your product and your business is higher than it's ever been. Every security leader that I talk to is feeling the increasing weight of protecting their organization, their business, and not to mention their customer data. Because things are moving so fast, they are constantly reacting, having to guess at priorities, and having to make do with outdated solutions. Vanta automates compliance and risk management with over 35 security and privacy frameworks, including SOC 2, ISO 27001, and HIPAA. This helps companies get compliant fast and stay compliant. More than ever before, trust has the power to make or break your business. Learn more at vanta.com/lenny. And as a listener of this podcast, you get $1,000 off Vanta. That's vanta.com/lenny.
I'm going to go in a completely different direction, coming back to the components of hardware. I asked a bunch of people what to talk to you about. One of the people is the founder of Matic, the CEO of Matic, Mehul Nariyawala. I've never said his last name out loud, so I hope I didn't butcher it. By the way, I love my Matic. I don't know if you have a Matic, but it's like
I have two, and I've purchased two more for
Oh my god, what an endorsement. Yeah. Basically, it's this amazing robot vacuum that just works.
So his question, he suggested I ask you this, is about memory prices. The way he described it is there's a meteor called memory prices that is coming for consumer hardware and robotics and physical AI. What's going on there?
Yeah, we're in trouble as an industry. I'm not an expert on this, but I think that AI has to do with why. And I also think that the supply chain is constrained. I have been advising startups and companies to pre-buy memory and to have enough memory in stock, if they can afford it, to ride out price spikes. Like anything in this category, let's see, this happened in COVID too. Okay. So we had so many supply chain disruptions, and getting enough memory was one of the challenges. So we had to pre-buy as well. I won't say who, but the company I was working with had to pre-buy memory as well.
And so this is part of what I wanted to talk to you about today, these supply chain disruptions. And if a key component that goes into a lot of tech, like memory or silicon, is constrained, there's not much you can do. You either pay, or you have already pre-bought enough that you can ride things out. And so those are the only real options. Obviously there's a risk to pre-buying, and the price might go down. And so the challenge is, I think there's a latency with supply chain in something like memory where it often can't adapt fast enough to demand, or there's a new category of product, or in this case maybe data centers that are just eating up so much and are actually not as cost-sensitive as somebody in consumer electronics like Matic might be, and so they'll just pay these higher costs. This is tricky and something we have to deal with all the time.
How much have prices gone up? How bad is this problem, and then where do you think it'll go?
Actually, this is a great question, Lenny. I don't know what's going to happen. I think prices are going to double probably. I don't know on what timeline. If I knew what timeline the prices were going to double on, I'd be trading, which I'm not very good at. I'd be doing a different job if I could predict these things. But certainly we're going to have a supply chain shock.
And it's already gone up a lot. Like, you're saying it'll double, but it's already gone up. I don't know, I saw numbers like 6x and something like that.
Oh, really? I didn't realize it was that bad.
That's a number I saw. Not cool. And you're saying, yeah, I think from what I hear it's AI driven. And when you talk about memory, it's like DRAM and things. What is memory? When we talk about memory, what's going on there?
Processing. The way to think about it is like processing memory. So it moves very... you think about memory like on your hard drive or your solid state drive where you're keeping files that you're not using, essentially, in many cases, or that you're dealing with, you know, maybe documents or pictures that you have. Maybe that's in cold storage on a server, maybe that's using a hard drive somewhere. This is usually things that you don't need really, really fast access on. But if you're running a program, some of that program is actually going to be run in RAM. And so there's different kinds, obviously, for servers. There's different kinds of server racks. Some server racks are actually focused on this type of memory, and some server racks are focused more on what we consider like a cold storage or a slower... Now, this isn't my area of expertise, but certainly most of the products that I've built, maybe all of them, have had RAM, and we've had to figure out how to... for me, mostly it's a packaging issue. Where do you put it? Does it need to be accessible? You know, which RAM do you pick? How fast does it need to be? And what is the cost, is usually our trade-off.
And what is the bottleneck with more RAM? Is it just the companies that make memory are just not able to produce at this rate because there's so much demand?
That's right. That's exactly what's happened.
So, this is a really good specific example of just how hard it is to build hardware. All it takes is one piece to be not available and your whole thing is screwed.
Yeah. You can't build anything if you have one component missing.
So, let's say Matic as an example. How many components are there that they all have to assemble and not have one not available?
I'm doing the math in my head. They probably have between 50 and 150 parts. It's possible that they have more. I haven't seen their CAD, so I don't know what it's like inside their device, but they do have a lot of things going on, right? They have the wheels of the device that are obviously moving around. Then they have a vacuum, but they also have a mop, and obviously they have a vacuum bag. They have the reservoir that the liquid has to go in for the mop. They have a system, which I think is SLAM based, which can see your room and make a map of it and identify which surface is which, and that I believe stays on the device, so it doesn't go up to the cloud, which is also kind of what we did in VR as well, which I think is a good privacy practice. And then they of course have wireless modules that connect up so you can communicate with your device. They're going to have an SoC, silicon. They're going to have RAM. They're going to have PCBs. And if you take everything off of those things, like all the little caps off the PCBs and everything, then you're in the thousands of parts easily. So it depends on how you count. But this is not a simple device.
And all it takes is one piece to not be available.
Yeah. So imagine a vendor that sells you a component that's a diecast component goes out of business. You can get another diecast component in three months maybe, and at quantity in five months or something like that, at high quantity. This is recoverable.
If your silicon goes out, if you can't buy your silicon, you can't buy your chip, now you have to redesign your board and you have to find something else that might work. This is a catastrophic redesign. If you can't get the RAM you wanted in the form factor you wanted, this is essentially what I call a catastrophic redesign. You now have to redesign the entire guts of your product and then secure supply chain for these new things, build it again on the production line, test it again, do all the reliability testing. It is non-trivial.
And so this is why we care. There's a hierarchy of components. Often in consumer electronics, we start with silicon and the display, which are usually the longest lead time things in my world. In robots, actuators are pretty tricky to get, even just for prototyping. Sometimes it takes a month or two to buy an actuator.
This is why Elon famously just starts building it all himself.
Well, when you look at what he did with Tesla and verticalizing his supply chain, and famously, actually, Starlink is an even better example of this, where I believe it's effectively like, or... and silicon chips in, product out. That's a pretty incredible factory, I've heard. I'd love to see it someday. But you know, this is where verticalization comes into play, because if you have verticalized and you have a lot of the components in house, or you're building a lot of things in house, you can actually adapt to supply chain shocks better. And then famously, when the silicon itself was difficult to find, he was able to redesign his PCB in record time and adapt to buying new silicon, and that would be much more catastrophic for a company that had a more classic supply chain.
One of the big decisions that I imagine you have to make when you're designing a new piece of hardware is deciding between using available components that are out there, cheap, versus, okay, we're going to do something new. It's something in software too: use the design system or do something new. How do you think about that balance when you're designing a new piece of hardware?
Very simply, I use off-the-shelf whenever I can, especially in the prototyping phases, because in the prototyping phases, which are a really important phase of what we do, your goal is to show that it can work at all. Like, can you get a thing working? So often it doesn't have to be the final pretty thing. It can be the ugly version. You can make an industrial design model of what the final thing is going to look like, but actually we call it works-like, looks-like models, where you have: this is what it's going to look like, and here's how it's going to work, and here's a working prototype. And humans are pretty good at this, as long as, and this is a pretty big caveat, what you show could fit into the industrial design. For companies that are younger, that's not always the case, but that's what we're going for.
And so in the prototyping phase, man, whatever works off the shelf, whatever's fast, whatever you can get to quickly, and then maintain a sense of what's really going to fit in your final design. Is it capable? Are the processes and components and materials capable of actually adapting to this size, this new weight that you need it to go into? So that's part of the calculus.
When you move into mass production and the final design, it depends. I mean, if I was making Matic and I could buy an off-the-shelf wheel or an off-the-shelf component, I absolutely would and fit it in. But often what we're doing is highly custom because we have, again, one of those KPIs. I want it to be this size. I want it to be this weight. I want it to be this color. And often off-the-shelf parts aren't good enough, not because they don't work, but because they're just not exactly designed for what we're doing.
This is the reason these drones are so cheap now. There's all these parts that have been innovated and built and scaled, manufactured for other things, and now we just have all these things and we can assemble a really cheap drone.
Yeah. Yeah. Exactly.
Super. You've mentioned CAD a bunch of times, and it makes me think about just, like, CAD has been around for a long time. Is AI impacting the way hardware is built? Obviously, it's impacting the way software is built in a huge way. Has it changed your life and the lives of people building hardware and robots?
Yeah. So I want to zoom out a little bit. Most of the hardware work goes into prototyping, into 3D CAD, so designing 3D parts and assemblies and components and making sure they work together properly. Then in making sure those parts and components can be made by a vendor at quantity, that it's possible and in the tolerances we want, and then putting those things together. So that's kind of our process. Right now, we're right at the very, very beginning of AI being able to do CAD. I'll give you an example. Claude can do what is essentially surfaces or point clouds. This is not real CAD. Real CAD, in my world, is dense. It has shape. It has NURBS. You have an equation for how the surfaces work. And it's an entity that's designed in CAD. It's a solid entity. And so right now we're not quite there with AI doing CAD. I think it's likely that at some point we will be there. This will be probably one of the biggest changes for my field that we have, being able to, I hope, do rapid design and increase the speed.
Now, there's a lot of really fun things to do in CAD, but in the beginning of my career, we had to do custom screws and we had to do the 2D drawings for everything, and there's a lot of things in CAD that are not as fun. Tolerance stacks. We need them. How do seven parts fit together, and are they always going to fit together properly? But it's not fun. It's not the most fun part. Maybe for some of us, but not for me. And so being able to do these things in AI would be amazing, so you can focus on actually doing the fun stuff.
Another good thing is PCB. A printed circuit board has a lot of layers on the inside and then components that go on the top. And if you've ever opened anything like a calculator or a computer and looked inside, you know what I'm talking about, these printed circuit boards. It's increasingly looking like AI can route inside of these boards pretty well, and it's looking like AI is going to be able to do some basic component selection and layout on these boards. So that's kind of where we're at right now.
We're not at a point, Lenny, where day-to-day mechanical or electrical engineering, like the meat and potatoes of it, is being done by AI. But there's a huge amount that you can do as an engineer using AI in your strategy, your planning, your ability to think through the complex dependencies that you're facing. And that's what I use it for now, which is really high-level planning, asking for information. Like when I look at who else has made a product like this, you know, I use AI to build the databases, and they're not perfect. Certainly a lot of times something's wrong, but it is so much faster. AI is pretty good in Excel right now. And of course, Excel is one of our favorite tools in engineering. So the ability to actually rapidly make Excel spreadsheets and change them, it doesn't sound sexy, but it actually really speeds up the design process outside of these core pieces.
I love how Excel is always at the bottom of anything, no matter what we're doing. We're going to Mars. There's an Excel spreadsheet that's probably driving a lot of this.
So, what's interesting about what you shared is it has already impacted the work of building hardware and robots, but it's on the verge of being transformative if it can get to real CAD.
Yeah. And my big question is, what is it going to take? A lot of AI is based on LLMs, which are essentially word generators, word guessers. They're more complicated than that, but that's essentially what they're doing. And there's also video models that you've seen that are trained on video. But these models aren't very good for what I need, which is, I need to know: hey, you take a piece of paper, you fold it four times, and you do this. Where's the hole going to be when you open it back up? These LLMs and even video models, they don't know how to do that. They don't have the ability to understand friction or weight or contact, pressure, friction, surface texture. They're just not able to do these things. And this is the core of what we need in engineering to be able to understand to build things.
So some world models may actually be able to do this in the future. And I suspect we may need those models to be the base of CAD and other physical engineering work. And so my frustration, and this is a healthy frustration, is I want Codex for engineering. I want Codex for hardware engineering. It's extremely valuable, and I've used it a lot for other things, but I want it for my field. And so what I think it may require is new model types.
Sounds like an opportunity to me. I know there's a bunch of world model companies. Fei-Fei was on the podcast with World Labs, I think it's called. Yeah. And then I know Google's building Gemini. So do you feel like those are the right directions, or do we need something actually different?
I don't actually know what Fei-Fei is working on lately. But obviously she's a brilliant roboticist, and I'd love to learn more about what she's doing, so I'll have to look that up. What I've seen is that what we have right now and what models we're building are going to be part of the solution, but not all of it.
Coming back to robots and humanoids, something that we were chatting about earlier. Your sense is humanoid robots aren't necessarily the answer to a lot of the problems that we have and opportunities that exist. Talk about just your sense of humanoids versus non-humanoid robots.
Yeah, I think there's a hype cycle around humanoids. That doesn't mean they're not extremely interesting, and I think there's going to be winners there, but what I hear a lot is, I want a generalist robot shape to do everything. And I don't know that that works. I think that you need different types of robots to do different types of things.
For example, if you've got a laptop and you want to screw together the keyboard to the case, this is not a job, I don't think, for a humanoid. This is a job for a dedicated manufacturing robot that has been designed just to screw 10 screws into a case for this specific laptop. And if you want to do that 10,000 times a day or 10,000 times a week or something, that's a dedicated robot that's specifically intended to do that thing. And what I think is interesting here is you can have standard cabinet sizes for automation robots, and you can have them be modifiable over time. And that's going to be a very interesting field, I think: how do you make manufacturing robots that are adaptable and changeable? But you wouldn't want a humanoid to do that.
And so when you really go and look at a modern manufacturing facility, like in China at the top tier of tier-one suppliers, there's not very many people on the line anyway. The entire printed circuit board line has essentially got no people on it anymore. The raw board is going through and getting reflowed and getting checked, and the whole thing is being done without humans, unless something goes wrong and a human runs over and fixes something. In mechanical assembly, the same thing. These most advanced lines, they don't have people working very much. They used to have 200 people. They might have 10 now. And so we've already kind of moved past human labor in a lot of this
most advanced manufacturing. And so we don't actually need to replace humans with humanoids. We just need more of these dedicated robots. So my suspicion is we'll have humanoids for some longtail things that we need to do that humans are currently doing. That will be important. But we'll also have robots that are for construction, robots that are for electrical work, robots that are for very low volume assembly, maybe robots for logistics, and most of them are going to look different from each other.
That makes all the sense in the world. What I think about as you talk about this is it feels like there's going to be this big moment when a robot can build other robots, and this CAD point you make about where once AI can develop designs, full designs for hardware, like that's going to be a big moment. Do you have a sense of just how close we are to this? I don't know, this loop that begins of robots building each other and designing each other.
If you're talking about robots building robots that are different than them usually, like yes, I think that that's going to happen. But it's like the terms matter. I don't think there's going to be one robot that's going to build itself. I don't think that that's what it's going to look like. But yeah, having AI be able to, if you could say, "Hey, I want to build this thing and I want it to do this and like this is kind of how I want it to look and here's a picture." The idea that you could, even as a hobbyist, go from a 2D picture to complex 3D CAD to assemblies to communication with vendors of how to make those parts and getting their feedback to iterating on that and doing a couple builds, like that is possible I think in the future. Will it be as good in the beginning as us doing it? No. But it will happen.
The biggest challenge here, Lenny, is actually the data. This CAD data is some of the most valuable IP that anybody has. And Samsung or Matic, to pick on Matic, like they're not going to want to give their 3D CAD to a model vendor, to a model maker, somebody making an AI model to teach it how to make great CAD. This is proprietary. This is like the secret sauce. And so where is this data going to come from is a big question I have. Which is why I think hobbyists are a more interesting place to start, where they're not concerned about the sanctity of their CAD and where it goes. They don't care. They want to make something and they want help making it faster. So this is kind of where I'm interested in this starting, which is, you know, maybe a hobbyist isn't an expert in printed circuit board design. Maybe they don't care. They just want their drone to be fast and to beat this other guy's drone or whatever. This is where I think you're going to start seeing all this start. And then probably the big incumbents are going to be slower because they have dedicated tools and a lot of IP privacy.
It's really interesting, this idea of what data AI models need to train on. I've been hearing that labs are buying code like GitHub repos pre-2021 because that's before AI, you know, has impacted the code, because there's less and less human-written code. And these are data labeling companies like Mercor and Surge and Handshake and things like that. Feels like this is a big opportunity that might emerge as them selling data, creating these CAD files.
Absolutely. And one really great idea, I think, would be to have an AI system that can go on-prem. So be inside of a data center that the company owns and then train it with their data. That I think could work eventually in the future, but you need a lot of this CAD data. So you're going to need a base model that has a lot of CAD data. We'll have to figure out how to do that. That's going to be very interesting. And then we're going to have to figure out how to put it safely inside essentially the walls of companies and have them then train it on their own data. I don't know if that's going to be like the equivalent of an MCP layer or what that's going to be, but this seems doable in the long term.
I want to ask a question that my sister suggested. She's actually been a longtime VR person. She was at Oculus. She joined with the acquisition. She helped create a lot of content within VR. She's just been in the VR world for a long time and now she's working on other things. She wanted me to ask you, what does it take to create a robot that feels human and connected, that humans feel connected to?
It's a great question. So, I'm new, relatively speaking, to robotics. And so I had to learn as much as I could, as fast as I could. And one of the researchers that helped me the most, her name's Leila Takayama. She's an expert at this. And what she explained to me is that humans have a certain expectation about how other beings are going to respond when they enter a space. You really want to, you know, when someone walks into a room, you kind of acknowledge them. You might not talk to them, but you kind of look up. There are a lot of very complex nonverbal cues that we give to each other. And if you walk into a room and a robot's just like, it's creepy, and it's easy to be creepy. I'm a little surprised, with some notable exceptions, how creepy a lot of these humanoids are right now.
You want, I think, these devices to be non-threatening. Generally speaking, you want them to appear soft. You want them to appear reactive to you. You want to have a sense that they know that you're there, that they're attentive to you, that they're there to help you and make your work life easier. And you also expect them to show their intent before they do something. And so one of the things I learned is if a robot just suddenly turns and does all this stuff, it scares you. But if a robot looks before it turns and then goes, it's much less alarming. So there's all these little pieces. And I recommend anyone go look at her work. There's a lot of great research here about how to, not necessarily with a humanoid, but how to have any robot both respond properly in a social context with someone entering a room or exiting a room, but also transmit its intent physically so it doesn't surprise you.
Feels like there's a lot we can learn from like Pixar and animation studios that have thought about this a long time.
Yeah, I actually think Pixar, Disney are probably the world's best at doing this type of design work. Even though they haven't done as much in physical in volume, if you look at what they do and how they show emotion, intent, approachability, engagement with their characters, they're really world class.
I don't know about you, but I'm so excited to have a robot at home doing things. Like these videos that they're starting to put out where they can do dishes, like at least the prototypes, they can fold laundry. It's like, yes, please come do this for me. How do you feel about robots in your house?
So I'm into it. My partner not so much. So I'm very lucky to have a partner who's got a high bar, which means, you know, was like never going to take Waymo, took one Waymo and now never wants to take anything else. So definitely willing to update her position, but it has to be pretty good. So she's in love with the Matic, you know, it's amazing. And so it's that, but the bar is pretty high. So I think in order to have a home robot, it's going to have to be pretty incredible for her to be willing to have it in our home. But I take that as a challenge.
My wife is exactly the same way. She's like, I want this thing in our house. Matic, oh wow, this is so cute. A recent example is self-driving Tesla. She used to be so like, no, don't do that. And it was not that great originally, and now she's like, I don't want to drive any other car. This just feels absurd to drive your car. I don't want to do that anymore. It's crazy how quickly that changes.
So there's a big difference in my mind. This is like a big categorical difference. There's a big difference between a car that is safer that drives itself versus a car that a human drives, because you have an existence proof of the human-driven car and you have the data. When you talk about homes, what is the delta? You have now a thing that you didn't have before doing things. So if it's bad at it, what are you relating it to? And if it's unsafe in any way, what are you relating that to? It's a much harder equation in my mind to get to a lot of people than a car, where you can say, "Hey, Waymo saved lives." You know, you're going to have a fraction of the deaths using a Waymo, whether you're a passenger or you're not. You already see people in San Francisco adapting how they respond around a Waymo versus any other car. So you're seeing behavioral changes that are based on trust, which is really cool. When you're talking about a new product that hasn't existed yet and is not essentially replacing something, that's a harder sell and you have to have a different story.
Something that someone needs to figure out with the Tesla self-driving is, you know, often you're at a stop and you make eye contact and go ahead, go ahead. Or someone's about to cross and you're like, okay, go ahead. But the Tesla just does its own thing, and so it makes you look like an a bunch of times. Like, oh, I'm not driving. It's not in control.
Yeah, I had that happen once. You almost want two little arms in the front to do gesturing, or like "you go" or something. It's amazing how much we actually rely on this human connection to decide, you know, who's going to go in an intersection.
Yeah. Okay. So, zooming out a little bit, what's cool about people like you is you're thinking and building things that will exist in the future. You're kind of living in the future and designing it, and you are one of the few people that has a glimpse into where things are going. So I'm curious, say in the next five years, what is kind of the vision you have of what is different about our day-to-day robots, devices? Just what does it look like? I don't know, just roughly.
So in this job we have this wild thing where we have to try to live in the future, and we have to try to live in the future far enough away that we can design something not only for two years from now or three years from now, but also something that will ladder up to what we want six years from now. Because in my field, it's a lot easier to make something and iterate on it and iterate towards a final goal than to do a one-shot thing perfectly. So not only do you have to have a sense of what the first thing needs to be like and look like, you have to have a sense of what the third thing, ideally, or the platonic ideal of the thing will eventually look like. So you do have to think about the future and live in the future. I have this weird thing where I love to think about the future, but I'm also a skeptic. And you really want me to be a skeptic, because if I think everything's going to be fine, the hardware is not going to work. You really want me to be like, "This isn't going to work and this isn't going to work and this isn't going to work," and just be kind of worried about all these things going wrong. So this is kind of an interesting disagreement inside of me of what I want the future to look like and what I think it's going to look like and what it's actually going to look like, and trying to guess.
And so it seems pretty clear to me that AI is going to have a foundational change in how we work and what we do over the next couple years especially. You're already seeing it. Obviously anybody who codes is not coding by hand very much anymore. Any knowledge work, this is going to hit next, I think, and progressively affect our economy and our work. But it seems like the physical world is less likely to change as quickly, outside of drones, self-driving cars. You're going to see more and more robots. But I'm not somebody who thinks that in five years you're going to have, you know, 20 million robots. I don't think that it's going to be that fast. I think we have a lot of really deep work on supply chain and supply chain reliability, raw material access, and then we need to figure out how to make factories again in this country for high-tech. So that's a lot of work, but in the interim, you're going to start seeing a lot of weird things on the street. You might see robots on the street. Have you seen any delivery robots in your world, Lenny, before?
Like, you know, like the little car things, not anything humanoid.
Yeah. Yeah. So this is just going to continue happening, and I think we're just going to continue to feel like we live in the future. But safety is going to be a big key for robotics. I think there's probably more change in war than there is in consumer electronics in the next two years, for example.
Wow, what a statement.
Yeah.
And I totally agree. There's nothing like war to incentivize innovation and just endless improvement and trying to get ahead of the other side.
Especially when democracy's at stake. I mean, I don't want to be, you know, on a high horse or something, but I do think that we're in a place where we need to think about things and the future in these terms and defend these things with our capabilities, while also hoping that we never have to have, you know, hot conflict anywhere.
Along those lines, I have to ask you. Recently you became famous, on Twitter at least, for quitting OpenAI. You tweeted that you were leaving with your brief explanation and got 7 million views, 50, I don't know, 8,000 likes. What happened? Why did you leave OpenAI? What happened there?
Yeah, I hope so. What I said in my tweet was that I have a lot of friends on the executive side of OpenAI that I care a lot about. I think they're really good people. And I feel that what happened with the decision-making, the speed of the decision-making, the governance, and the lack of defined guardrails around the announcement of the Department of War deal is not how I thought it should have been done. And both of those things can be true. And so my hope, Lenny, was that there's a third path. You see a lot of people just kind of going along with what their company's doing, and then you see some people that are kind of scorched earth about it. In this case, that didn't make sense for me. I didn't feel that way about the company. OpenAI is an amazing company, and I was able to help build a robotics program there and attract some of the top talent in robotics, I think, in the world. And so, I don't know, you know, this is a group of people I care a lot about. And you can also disagree with friends and feel like what they did isn't good and isn't right. And that's where I ended up and that's what I tweeted about. It was going to get reported on, so I tweeted before that happened.
This is a great opportunity to just whisper to me what OpenAI is working on. What is this robotics device? Just between you and me.
Yeah, I wish I could say. You know, Lenny, part of the fun of our job is we get to see things before everybody else does. But the flip side of that is we can't talk about anything internal or any IP. What I can say is the team's really strong, and I was really, really grateful for the opportunity to help. But I also thought that after what happened happened, it was time for me to, I couldn't continue to work there, because you don't know what's going to happen next time. And my hope was that my decision made it easier for other folks to talk about what their boundaries were and hold them, and, you know, we'll see what happens there.
So speaking of team building, this is something I definitely wanted to ask you about. So as I said, I asked a bunch of people what to talk to
you about, and someone that I think it was maybe a colleague, former colleague, Mariana Senko. Did you work with her? Okay,
She's a... Yeah.
Okay, she's a friend. So she told me that here's what she said about you: that your brilliance as a leader lies in hiring exceptional teams. I'd be curious about the kinds of people that she finds indispensable in an era where everyone is concerned about their jobs. So talk about what you've learned about just what you look for when you're hiring folks for your team.
Yeah, I'm lucky that I've had a lot of time, a lot of reps basically on hiring people. And so I have a strategy of hiring great people. When you're hiring for zero to one and new things or new industries, and that's what we're facing, I think, with AI and robots certainly, it's very new. You can't count on having entirely people who've done the exact same thing in past lives because it doesn't exist. The exact same thing doesn't exist. Maybe you've got roboticists who've built a thousand robots, but nobody that I'm aware of has built the type of robot that can move through the world the way, you know, I'm interested in, in the millions, because it hasn't been done. So, you have to start thinking about how do you build a team that can do something new.
And the nice thing is actually in robotics, self-driving cars, autonomous vehicles is a really good place to look because you've got the sensing stack and you've got a lot of the safety trade-offs actually, and it's a lot of the hard engineering, the hardcore engineering. So, that's a place that I looked. Obviously you want some hardcore roboticists who can do, you know, robot design from scratch. And these are really people, even though they might have a degree in something, they're really hybrid people. They're generalist people.
So, one of the key principles I'm looking for is a lot of really strong generalists who can adapt what they've learned in other fields to a new field, and people with a lot of experience building. You want some people who have experience building the thing that you're building that's new and some people who have experience scaling other things to hire on. So, you need to look at that.
And then with young people, this is where it gets really fun, Lenny, is the only AI native people essentially who use AI so natively that it's like baked into their engineering process are 20 years old or 21 years old or 20. I mean, it's very hard to find someone who's in their 30s who can be truly fully AI native. And so, we need these folks to teach us how to think. And I've had the opportunity to work with a few folks in that age range. They're approaching their problem solving completely differently because they're using AI from the ground up for everything, and they're much faster actually, and it's really fun to watch.
So figuring out how to get these AI natives to teach us, the rest of us, how they think about AI, when it's, you know, we are, you and I, I think I can say, are digital natives, where we grew up, maybe there wasn't internet when we were really young, but we are the generation that had the first, you know, internet, we were teenagers, and we are the generation that had the first cell phones really at scale, and we're an important generation because we had the first... I remember freshman year at Stanford we had the first databases that you could access and you could share movies on, I think is what we did, and music on or whatever it was. But this was new and so we were native in these things and that gave us a lot of oomph in creating new technologies for it. But we have to accept that we're not native in these new technologies, and you really want some folks who are hungry and excited and want to learn who do have these skills.
That last bucket is a very common trend on this podcast when we talk about hiring, which is really cool as a counternarrative to there's no more jobs for young people. All the junior roles are erased because of AI.
Yeah, I don't see it that way. I think we need them. I also think that we need to build new technologists. There's the obvious question of what happens if we don't have teams that have senior and junior people. But I think what you find when you actually build these teams is you have to have both. You must have both. The team size just might be a little bit smaller than it used to be. When this AI revolution in hardware happens, I don't know how that's going to affect the teams. That will be really interesting to watch.
So what I heard here is just look for generalists that can flex based on whatever needs to be done. Some mixture of specialist and like scaling versus zero to one. And then these, the best term I've heard for this is cracked new grads.
Yep.
That are AI native essentially, that are just doing everything AI first.
Yep. And then what we didn't talk about, of course, is mission alignment, which actually unifies a team. So if everyone coming in is aligned to the mission, that helps a lot, because especially in the world of AI researchers and hardware folks there's a lot of miscommunication because we're coming from such different worlds. And so having a sense of we're all pulling in the same direction is really important.
And then I rely a lot, Lenny, on my gut feel for people, assuming everything else that I'm looking for has been checked. So, it's hard to talk about what that means, but usually it's that spark that you're looking for in someone that they're genuinely motivated. They're motivated by a desire to learn and by excellence. They're motivated to learn from the people around them. They're open to updating their point of view based on new information. And they want to win. I mean, these are the things that really matter when you're building a team.
Awesome. Okay, just a couple more questions. Something I've been wanting to ask for a long time is you've worked with some of the most legendary successful builders: Steve Jobs, Jony Ive, Mark Zuckerberg, Sam Altman. You don't have to go through all four, but just what's a lesson you learned from as many of these folks that come to mind?
So, we'll start with Sam because most recently. Sam is really good at saying, "Why not more? Why not 100x or 10,000x? You're thinking too small. Why not think about this bigger?" And every time we talked about something important, he talked about that. And what I realized is I was thinking too small in certain areas and he was thinking globally. And having that nudge from a leader who's ambitious is really helpful, I think. So that was a big thing that I learned from him, about he's willing to go for it at high volume and invest, depending on, meaning not high meaning hitting a lot of people, you know, he's willing to think in very big numbers. That was really, really foundationally important.
I think for Steve, Steve Jobs is just the bar he held for the company and for technical talent and for excellence was not wavering. It was up here and you were either going to meet it or you weren't. And that was something that kind of washed through the whole company. When you are a young ambitious person and you hear that something's not good enough, that can be extremely motivating actually. It doesn't quite hit the way you would think. And if you tell somebody, hey, this needs to be better, like, you need to spend more time on this, you need to be more thoughtful about this, or this is not hitting our quality bar in a CAD review or something, that's impactful, and I think you never want to hear that again. So, it's very, very motivating.
And then Mark Zuckerberg, I have to say he ran a company very, very well. So the way that the technical side of the company operated, the way that we had reviews, that decisions were made, the decisions were made at the lowest level possible in the company to maintain speed. I underappreciated how clean and well-run the way that the hardware organization interacted with the rest of the company was. It was very clear. This is what we're going for. We're going to have this review. We're going to make a decision in this review. If you can make the decision without the review, you will do that. Here are the objectives for this project. It was really well executed. And I think that's hard to do at a fast-growing company. It's very hard to do at that level. And having him and Andrew Bosworth, the CTO, involved in the technical decisions, able to read, you know, reports that were maybe 20 pages long, grok the trade-offs, understand them, and be able to contribute to the technical discussion. And that's just on my thing that week, and they're doing that, you know, a hundred times that month. That was impressive and definitely something I learned from them.
What an incredible set of experiences and different types of places to work. I don't know if they could be more different, all these places.
I know. And I think that's why I'm looking for this zero to one. And so when you're looking for a zero to one opportunity, it's always going to be in some place different, generally.
You're going to be a hot commodity in this market now that you're a free agent. But on the flip side of that, I want to take us to failure corner. I feel like someone building hardware, physical things, has some great fail stories. Is there one story of something you built, something you worked on that failed, and something you learned from that experience?
This is a great question and not a comfortable one. One of my favorite failures was actually on the Quest 1. It was around EVT, so halfway through the Quest 1. And we found out that we had gone from five cameras to four for cost reduction. We talked a little about this. We needed to reduce the price so more people could buy them.
And what happened was it was right before Christmas and I heard from the lead on the team that does computer vision, and he said, "Oh my gosh, the data from the cameras isn't working and we can't get a lock on where the person is using the headset." And so we looked into it and we realized that their interpretation of our spec and our interpretation of our spec was different. So in engineering we usually use a plus or minus, like it can go up or down by, in this case I think it was 0.15 mm or something like that. And in his world he was used to having a global, it's within 0.15 mm, and so we had a different interpretation of the spec. Now, the problem is that our interpretation of the spec meant that he couldn't meet his goals of being able to understand where the headset was in space. And so, we had to do a redesign. And this is at EVT. So, this is pretty much when you want the engineering to be done.
This is a build?
It stands for when we compile the hardware for the first time with everything that's supposed to be done. So final components, final materials, you're making the components on the tools you're going to make them for mass production instead of just machining them. So it's a big deal.
And so what we had to do was favor or prioritize. We had four floating cameras. We had to lock the bottom two to each other and put them on a bracket so that the relative distance from them met the spec that he needed, and then let the other two float. So this was an architectural change and this was a failure. I mean, it was a failure in understanding the spec. It was a failure in essentially the product design, but it was because of a misunderstanding of the spec.
And so we were able to adapt. We actually kept the build on time and we actually shipped the product on time, but it was really stressful. And it turned out that actually the new design was better, because with a favored pair, you have source of truth for the space, and then the other two cameras overlap onto that source of truth. And so it worked well. I thought it was a good outcome, but it was a scramble and certainly wish that we caught it, you know, four months earlier.
Another example of just how hardware is. You can't, like, you mess up a spec and like, all right, here we wasted a week building something that didn't work, and now it's like four months later still having to redo the hardware supply chain.
Yeah. It was tricky.
So the Quest 1 that shipped was this, with the cameras moved?
Yeah.
Wow.
Yeah. If you look, there's two cameras a little closer to one another in the front of the Quest at the bottom.
Wow. How did Boz and Zuck feel about this?
The fact that I don't remember probably means okay. I think we addressed it. We redesigned it. We had to change the material on the bracket. I think we had to make it steel to hold the tolerance we needed, but it worked out, and the price and the cost and yields were fine. So I think we adapted.
And that was the bestselling VR device of all time. Is that right?
I think it was.
Okay.
I don't have the final sales number, but...
Okay. Concerned. Okay. Caitlin, we've covered so much ground. Is there anything else you wanted to share? Anything else you want to leave listeners with? Either double down on something we've shared or anything else that's just like, oh, here's something I want to share.
I think that this is probably one of the most exciting times we're coming into. And it's normal, I think, for all of us, myself included, to be worried and scared about it, but I also think it's an opportunity for people to do an extraordinary amount, have an extraordinary amount of progress, and be able to as an individual do more than we've ever done before. And so that's the side I'm trying to embrace. These new tools, this new way of work is scary, but if you embrace it and are daily using these AI tools right now and daily applying them to what you're doing, you'll be at the forefront of whatever comes next. And so I just want to encourage everyone to be creative, use these tools, have fun with them, figure out what the boundaries are, and then every time a new model comes out, test again, because it's really important to know what we're dealing with and where these boundaries are. But I've never been more excited about the power of an individual.
Well, with that, Caitlin, we've reached our very exciting lightning round. I've got five questions for you. Are you ready?
I'm ready.
First question, what are two or three books that you find yourself recommending most to other people?
I've been mostly reading the classics lately. So, Book of the New Sun is a great fiction book, which I really recommend. I think that's what it's called. I haven't read it in a little while. I love Mrs. Dalloway, actually. I think it's a very interesting book about transitions, and it was a post-war book by Virginia Woolf. So I really love it and I think it's really wonderful. I think Herodotus's Histories is pretty incredible. He's wrong a lot, but it's also the first history book, and in many cases he's going and finding, you know, firsthand or secondhand accounts of what happens. It's a way to look into the world at a completely different era than it is now. So these are some books that I like, and I'll double check the title of the first one and email you. But I think that's what it's called.
Okay, and we'll link to the correct one in the show notes. Favorite recent movie or TV show that you have really enjoyed?
I am really into Euphoria right now. I think the new Euphoria. I'm interested in the characters and figuring out what's going to happen there.
The show's so stressful whenever I watch.
It's a melodrama. I think you have to think about it as a soap opera, and then it's fun, if you think about it too literally.
Okay. It's helpful. Do you have a favorite product you've recently discovered that you really love? Could be like hardware,
Could be an app, could be piece of clothing, could be a gadget.
I really like Vollebak, the clothes. They make really interesting clothes. They're essentially basing their new clothes on material science. So, they take new material science and make them into clothes. It's just a fun brand to follow.
Vol...
V-O-L-L-E-B-A-K. Vollebak.
Very cool. Do you have a favorite life motto that you often come back to in work or in life?
Have you seen that branch where there's all these branches and then you're here and then there's all these branches from this point?
Yeah.
Yeah. You know who it was from? I didn't know who. I think about it a lot because it's very hard not to get stuck in the future or the past and stay kind of here. I have trouble with that. But that is a great reminder that you get to pick every day. You get to decide every day what you want to do. And sometimes things don't go the way that you want and sometimes you regret what you did or sometimes you're proud of what you did, but it doesn't really matter. What matters is what's right in front of you.
We'll either show that image on the screen as you say that or we'll link to it in the show notes. It's so powerful. Final question. Somebody that knows you well shared this really interesting tidbit about you, that you hired a PhD to tutor you on the staples of ancient Greece and Rome and just get really nerdy about this stuff. What's going on there? What drives you to go so deep on these sorts of things?
This is like very nation nerd territory, but I found this list that the poet Joseph Brodsky wrote, which is a list of things you should have read in order to have an intelligent conversation in English. And it is like an affected list, like it's intense. It's like the Old Testament, Gilgamesh, and then all the way down through. But what I found is it's a pretty good distillation of what we used to call the Western canon. And I actually learned a lot in my public school education and in college, but I never really learned from what you would consider the Western canon. And so this is kind of that; in addition to that, there's some newer books on the list. I find it just fascinating to have something to work off of. And what I found is as I got into specifically the tragedies, the Greek tragedies, I just didn't have enough context to learn what I wanted to learn. And just reading them, I didn't have enough uptake. So I found an incredible postdoc who was willing to tutor me. And I just get to ask him all these questions. He's an encyclopedia. He knows everything. I could ask him what was happening in Turkey at the time of this Greek tragedy that we're reading, and what was happening in Athens, and what this tragedian might be responding to. And he can answer the question. It's really fun to have the ability to have that sounding board.
So cool. It's so cool you did that. Even though AI can do a lot of this, sometimes a human is much more interesting to talk to and feels better.
Yeah. I find reading and communicating with AI is very helpful on the basics, but then understanding what was happening culturally and what the significance of the work is, it isn't adequate.
It's because we're not 20-something... wrong?
I don't think it's wrong. We just didn't grow up this way. And I imagine college students would just say, why would I do that? I have Claude here.
Yeah.
So cool. I love that. Well, this was incredible. You're amazing. Where can folks find you online if they want to reach out, I don't know, try to hire you? And then final question, how can listeners be useful to you?
So I have a website, which is just my name.com. I'm also on LinkedIn. People can help me, I think, by helping imagine the future. This is not a single-player game. This is a multiplayer game. Figuring out what future we want, what we want it to look like, what we want the human aspect to be in that future, and what we think we want to hold for ourselves, how we want to augment ourselves. Right now we're in this dystopian niche where it feels like the future is horrible, and the way to not have that is to actually design our own future together, figure out what we want our future to look like, paint a picture in fiction, in literature, in conversation, and then build that. And I think that that's possible.
I love that. That's actually a message that's come up on a couple of recent podcast episodes, so it's a really good reminder. Caitlin, thank you so much for being here.
It was really fun, Lenny. Thanks for having me.
Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
Article published · Updated