Will Larson on the Revised Rules of Engineering Management
The Pragmatic Engineer
For much of the 2010s, new engineering managers were told to stop making technical decisions. Will Larson says the job has moved away from that rule. Larson is CTO of Imprint, was previously CTO at Calm and Carta, held leadership roles at Stripe and Uber, and wrote An Elegant Puzzle, Staff Engineer, The Engineering Executive's Primer, and Crafting Engineering Strategy. In this conversation on The Pragmatic Engineer, Larson argues that the management habits of the zero-interest-rate era no longer fit how software is built or how companies now judge engineers and managers. The conversation covers Larson's own path from a small Kentucky college to the CTO role, the Digg v4 relaunch as seen from inside, what has happened to career ladders and the CTO track, and why Larson thinks middle managers now face a choice between reskilling and leaving the profession.
From a four-person CS class to Yahoo
Larson's first computer was an early color Macintosh, used mostly for a children's drawing program. In high school came multi-user dungeons (MUDs) and IRC clients, whose scripting and triggers taught regular expressions. Larson did not write real software until college.
The college was a small private school in Kentucky. About 13 students were in the whole computer science program, and four graduated in Larson's year. Larson contrasts this with schools like Waterloo, MIT, or Stanford, where students trade interview answers, do co-ops for years, and are carried along "an escalator" into the industry. A small school had no such path. One talented graduate from the year before had gone to work at a horse-racing betting software company in Lexington with perhaps five engineers, because that was the kind of job people knew how to find.
After graduating in 2007, Larson taught English for a year in Gifu Prefecture through the JET Programme. That year was when the writing started. Back in the US, Larson wrote tutorials on Yahoo BOSS, a build-your-own-search service, including using it from Python and Django. The team behind BOSS noticed someone was writing about their product. That led to a remote contract job starting around early 2009. Yahoo was in a hiring freeze after the financial crisis and going through repeated layoffs, so it could take on contractors but not full-time staff. After 12 months, compliance rules required contractors to go full-time, and full-time meant relocating. Larson moved from the East Coast to California.
Yahoo's homegrown world and the server black market
Larson describes Yahoo as interesting because it predated the open-source ecosystem that made later startups easy. It built almost everything itself. Servers ran yApache, Yahoo's fork of Apache. Code was written in yPHP, the UI used YUI, packages were installed with yinst (a Perl tool for walking dependencies), and replication outside the database ran through a Kafka-like log Larson recalls as yRepl.
The Yahoo story Larson likes most is about incentives. Co-founder David Filo personally approved every server allocation, on the theory that this would force teams to justify their needs. According to Larson, the actual result was a black market. One engineer on Larson's team had kept around 30 servers from a cancelled project and effectively traded them for favors. Because nobody gave servers back, Yahoo ended up with large numbers of outdated, power-inefficient machines, and launching a project meant finding someone friendly with spare hardware. Larson wonders whether Filo knew about the black market and kept the mandate anyway.
Inside the Digg v4 relaunch
Larson next joined Digg, then a top-10 website. Unlike Reddit, whose subcommunities voted content up through an algorithm, Digg centrally controlled what reached its front page. Larson says this made quality more consistent but created a central-moderation bias problem. Groups traded favors in IRC channels to get stories promoted, and some people sold front-page access as a business. When Digg later changed its algorithm, Larson received death threats from people who said their livelihood had been taken.
Larson joined three to six months before the v4 launch. The Friday before Larson started, the CEO was removed and founder Kevin Rose returned. Digg v3.5 was a PHP application on sharded MySQL, running on owned hardware in a colo. Larson says v4 was a rewrite meant to address an existential problem. Roughly 60 to 70% of traffic came from search into "permalink" pages, where a story posted to Digg often outranked the original source. Those pages were also where Digg sold most of its ads. A series of Google algorithm changes, including Panda, was cutting that traffic, and Facebook was drawing users away. Larson recalls that the company had one profitable month before an algorithm change hit and it started shrinking. It had about 110 to 120 employees and was burning through cash in an era when funding rounds were a few million dollars.
Larson says the launch went badly for several reasons and is careful about responsibility: Larson had been there about three months and was only in a second job. The central technical decision was replacing the sharded MySQL system with Cassandra. Larson recalls a joke at the time that Cassandra was a Trojan horse for wrecking competitors, since anyone who launched on it was in trouble afterward. Larson's assessment is that the MySQL setup was complicated but worked, and Cassandra "basically didn't work." The team responsible for load testing never did it. Two days before launch, Larson and another engineer, Rich, were asked to do it themselves and started by installing JMeter. In a conversation with the head of the Cassandra effort, they established that the system could serve reads or writes, but not both at once. The launch went ahead anyway.
Larson calls launch day "the most interesting day I've ever had professionally." There was a launch party with caterers in tuxedos carrying sushi and champagne, people upstairs getting massages, and some groups playing ping-pong. Meanwhile, the engineers sat around a large table in the middle of the room. Digg could not afford enough hardware to run the old and new systems side by side, so engineers were wiping old servers one by one and installing the new software, which made rollback impossible. The new system did not come up, and the site showed a maintenance page for about 24 hours. Larson describes it as "magical realism."
Many people gave up in the following days, and Larson says a group of about six kept the site running. In the first two days, Larson and Rich wrote a caching layer from scratch on Redis, which kept the site up for hours at a time. The system still leaked memory so badly that every process needed a rolling restart every hour or two. Finding the root cause took Larson about four weeks. A new Python API service built for third parties used a mutable list as a default parameter. In Python at the time, that list persisted between calls, so every user ID ever passed in accumulated in memory. The bloat cascaded into memcache, which then fell over and overwhelmed Cassandra. Once found, it was trivial to fix. The new CEO started around the same day Larson fixed it, and over the next year nearly the whole company was laid off.
Larson does not conclude that the rewrite was clearly wrong. If Digg had kept the old site, it also would have died, so Larson isn't sure the company made the wrong choice. At the time, though, Larson was "pretty salty" about it.
Into management by accident, then Uber
Larson became a manager at Digg with about two and a half to three years of experience, "because I was the only one dumb enough not to say no" when the other managers had been fired. Larson was later made a director, so the company could show a director during acquisition talks. Digg was split up: the patents (including an early version of in-stream ad units) went to one buyer, the brand and site to Betaworks, and the team to SocialCode, a Washington Post Company subsidiary doing Facebook advertising.
After a couple of years there, managing around 12 engineers, Larson wanted a larger leadership role. The problem was that big roles require having already held big roles, and small startups never offer the scale. Larson's reasoning at the time, which Larson says has changed since, was to join a company growing fast enough to create that scale. In 2014 that was Uber. A former Digg infrastructure engineer, Jeremy Grosser, had brought much of Digg's stack to Uber, including Clusto, a key-value infrastructure tool that Larson says only Digg, Uber, and Second Life's company may ever have adopted.
Larson was hired to run what was called SRE. It was really a general infrastructure team, and Larson says they read a DevOps book the week before starting to figure out what the job was. The interview had included the traveling salesman problem on a whiteboard. Uber had about 2,000 employees and 200 engineers. Larson's team of about six owned databases, Kafka, server management after racking, and service provisioning. Two years later the team was 70 people, all hired externally, because Uber doubled engineering headcount every six months and every other team was hiring just as hard. Larson says they learned to hire at Uber. Stories included an engineer who got an offer at 4:30 p.m. on a Friday and started Monday at 8, and a candidate who was rejected four times, got an offer on the fifth attempt, accepted, and quit within six hours to take an offer from Twitter a block away. In the early days, engineers manually reallocated software across servers every Friday to survive rush hour.
Larson describes Uber's culture as having almost no management support: "sink or swim." It was also a "no dependencies" culture, close to the slogan "let builders build." If you were blocked, you were expected to work it out yourself. Larson found that empowering. At around 27, Larson ended up co-signing a purchase order worth tens of millions of dollars for Uber China with the head of TechOps, because nobody else would sign it. No one ever followed up. Larson admits that was probably not good corporate governance, but says it let people do remarkable things. The host, who also worked at Uber, adds that people burned out for a reason, and that Uber created its first formal apprentice manager program around 2017 because bad managers had become a problem.
Stripe, Calm, Carta, and Imprint
Larson spent about four years at Stripe, from 2016 to early 2020. Over that time the company's valuation went from about $5.5 billion to about $96 billion, and headcount went from roughly 500 to 1,500–2,000, growing about 70% a year. Larson reported first to an early engineering leader and then to David Singleton, who joined as head of engineering and later became CTO. Larson was therefore always one step away from the top job. Two reasons drove the move to the CTO role at Calm. One was financial diversification, since nearly all of Larson's savings were tied up in a company Larson expected would never go public. The other was wanting to know what it was like to fully lead and whether Larson could make better decisions from that seat. Larson stayed at Calm about three and a half years, then spent two years at Carta, and joined Imprint about a year to a year and a half before the recording.
What a CTO does now: technical decisions are back
Asked what a CTO does, Larson goes back to the 2010s lesson: managers should "never make a technical decision again." That was "literally" what Larson was told, and Larson says it is "definitely not the ethos today." Part of the CTO role's appeal for Larson was that it is where the technical and managerial career paths meet again. Larson deliberately has both managers and individual contributors as direct reports. Of 14 direct reports at the time of recording, four or five were ICs. Larson also says plenty of CTO decisions across the industry made no sense, which made Larson want to try. Larson credits Stripe and Singleton with high decision quality: you could disagree with a decision, but rarely call it nonsensical.
Larson says the most fun part of the current era is that leaders are not just allowed into the details but required to be there. Larson reports having merged more than 500 pull requests to production in a year and a half at Imprint, more than in the rest of Larson's career combined.
At Imprint, Larson also leads product and design, a unified engineering-product-design structure. Larson does not claim this is a lasting trend. The reasoning is that during a period of rapid change, someone has to make fast decisions about how agents are adopted across functions. For example, if designers aren't using tools like Claude Design or the Figma MCP, they become the bottleneck. Imprint has been trying engineer-led design for second-tier features. Larson says this removed a bottleneck but caused friction over who is responsible when engineers diverge from the design system. Having one person who can make those trade-offs has helped. Larson believes the way software gets built is changing more than at any point since Larson started in late 2008, but avoids predicting much further ahead.
How management culture changed, and the staff engineer problem
Larson traces how management changed over time. At Yahoo, Larson had the same manager for two years and two one-on-ones in that time. One was about another employee's performance, and the other came after Larson gave notice. That manager saw the job as finding high-impact work across the company for the team to deliver. Helping individuals have rewarding careers wasn't part of it. Digg and early Uber had essentially no management. Stripe in 2016 had almost no managers, after following a Google-inspired model. One manager had 40 to 50 direct reports, and another led about 30 people with only one manager underneath.
Larson says that during this period the idea of the manager as a facilitator, rather than a decision-maker or doer, took hold. Larson wrote Staff Engineer (2021) partly out of frustration with a related belief: that staff engineers should mainly get interesting, personally rewarding problems. When Larson left Stripe with a team of about 250–260, the organization had serious business and scalability problems. Some of Larson's best engineers asked for more interesting work, and Larson's answer, in effect, was "I only have suffering to offer, but this suffering is what's going to make the business succeed." Many of those engineers saw that as a breach of the management contract. Larson thinks the belief never held at the executive level, since the most senior people at Stripe and elsewhere were doing the valuable work the company needed. Larson says the industry has since clearly moved away from it.
ZIRP explains (most of) the 2010s
The host asks whether zero interest rates drove these habits. Larson mentions Ranjan Roy's "ZIRP explains everything" piece in The Margins, which Ranjan co-writes with Can Duruk, Larson's former Digg colleague and later Uber hire. Larson agrees that ZIRP explains how 2010s management changed. Salaries rose, and the ability to hire, close, and retain became what separated successful companies from the rest. That shifted leverage to engineers. Companies pitched themselves as relaxed and focused on career development, often in self-defeating ways. Larson's example is the idea that engineers shouldn't be on call and that a second-tier SRE group should be paged instead, even though, in Larson's experience, everyone knows engineers being on call for their own work makes the software better.
The host suggests that managers in that era were rewarded for low attrition and filled headcount, which pushed them toward people-pleasing and shielding engineers from the business. The host has heard from engineers who moved from large companies to seven-person startups and felt stuck, having never learned to ask what the business needed. Larson agrees and adds that engagement surveys, while useful, became goals in themselves instead of things to interpret. Larson's broader point is that if managers are forbidden from making technology decisions, they are left with headcount, process, scheduling, and prioritization. That, in Larson's view, is a natural result of splitting the two roles, and it helps explain the bloated organizations of the past. Larson now tells teams that hiring is "the least interesting way to solve every problem."
Do career ladders still matter?
Larson still believes in career ladders, citing research that rubrics lead to fairer decisions and noting that without them, maintenance and "glue" work gets undervalued while flashy work gets overvalued. What belongs in a ladder, though, is now unclear. Larson sees many small companies using the same off-the-shelf ladder from some consultancy and considers those outdated, with no replacement yet.
For staff and principal engineers, Larson focuses on decision-making in ambiguity. The example is Stripe's multi-region project, where the engineers leading it kept asking the product team to decide CAP-theorem trade-offs about cross-region replication. Product could not possibly make those calls. Larson saw it as the engineers finding a way to get stuck instead of making trade-offs, building conviction, and testing assumptions with prototypes. Those trade-offs increasingly require understanding product implications, user goals, and unit economics. Larson doesn't think agents are especially good at this, partly because they often don't see the trade-offs, which sit in implicit context outside the code.
For early-career levels, Larson is wary of "artificial specificity." Many companies effectively separate an early-career engineer from the next level by about three years of experience, and Larson thinks time and role are probably good enough. Overly detailed rubrics invite gaming. At one company, a staff-level mentoring requirement led people to "collect mentees" who didn't know they were being mentored. Larson prefers constraints to comprehensive rubrics. At Carta, which was under cost pressure, Larson set a hard ratio of staff engineers to other engineers. It annoyed many people, but Larson considers it more honest about the fact that the goal was managing costs. It also forced the company to hire more junior engineers to build a pipeline. At Imprint, where the goals are velocity and quality, there's no such ratio. Larson's summary is that leadership means knowing what you're doing, explaining how you're doing it, and letting people accept the consequences.
The changing role of the software engineer
Larson says it's too early to know where the engineering role is going. Some claim future software teams will be five people at any company size. Larson says "we're definitely not there right now," though some small teams are spending "a staggering number of tokens" productively. For someone focused on outcomes, Larson finds this the most fun period of their career. Larson has also met people who loved flow state and have fallen out of love with programming. One is retiring because, despite being more productive, he no longer enjoys writing software.
Larson's working model is that the time to implement software is approaching zero, but software is still very hard. You still have to make decisions, scale the system, meet user expectations, and get the unit economics right. Engineers are now running ahead of PRDs and high-fidelity mocks, which is causing problems for design teams. Larson says Imprint has not solved this.
Larson argues that domain complexity is still a moat. Carta is low-scale but extremely detailed: cap tables have many exceptions, accuracy matters for auditability, law firms have strong views, and founders may be in legal disputes over vesting. Larson isn't sure such businesses have benefited much from AI tools yet. Larson expects that to change but doesn't know how.
On whether specialists are merging, Larson asks why more designers aren't shipping production code now that it's possible. The host says that after asking around roughly 20 companies, including Linear, Ramp, and OpenAI, they found almost no designers pushing production code outside prototyping. Larson pushes back on the view that a good design system makes product designers unnecessary. Fitting components together, finding missing ones, and keeping a unified experience across screens are still hard, and good designers are short on attention. Imprint's goal is for engineering to handle base cases with the standard design system and linters while designers focus on exceptions and extending the system. Larson says it doesn't work that way yet and calls it, for now, a "skill issue" the company is still working on.
Durable teams, executive sponsorship, and accountability
The host quotes Larson's "revised rules" essay on durable, high-ownership teams. Larson calls this "the only way to build scalable organizations" and also a non-consensus view. When a key partner project is late, companies still move 15 more people onto it despite The Mythical Man-Month. Larson thinks this is often about optics, so that a miss doesn't look like anyone's fault. Larson argues it also destroys what does work: small pods that own an area. At Stripe, Larson inherited about 30 people, roughly 24 reporting directly to Larson, grouped them into pods, and stopped thinking about each pod once it showed progress, which freed attention for the hardest problems. Larson did the same at Uber, Calm, and Imprint. The host adds that a better alternative to adding people is to remove distractions from the team on the critical path and ask for explicit, time-limited commitments in return for time off.
Larson says that at a fast-growing company there's a crisis every day, so rotating people to crises creates a crisis for everyone every day. A Stripe colleague compared the company's early years to defenders running along a castle wall toward whichever point was about to fall. The goal was to station people in many places so each group handled its own section. Larson is unsure whether AI pushes the point at which the "running" model breaks much further out, and suspects attention runs out before coding ability does.
The host raises Slack's agent integration as an example of a long-standing team with deep domain context shipping something competitors haven't. Larson mentions a paper about British imperial bureaucracy growing as the territory it managed shrank, noting that companies do accumulate overhead. Larson's impression, though, is that strong products usually come from one or two highly empowered people close to the leadership team who make unpopular trade-offs. Larson's own example is Imprint's move from Jira to Linear, done in under four weeks, when Larson recalls only about three of roughly 230 people thought it was a good idea. It gave agents a central place to find, pick up, and complete tasks, which the company had struggled without. Larson says people are now glad it happened. The other side, Larson argues, is that executives should be judged on whether the risks they push are good ones. If people had hated the change, "they should be rallying to get me fired." Larson thinks empowered sponsors are too often not held accountable for bad calls.
Becoming a CTO: bureaucrat and technologist
Larson describes the old path to CTO as a short engineering stint, then management, then climbing through middle management. This selected for people who were good at operating, not necessarily at technology. Still, Larson says you do need to be a good bureaucrat to be an executive. Much of the job is managing incentives, such as knowing how a given performance review process will fail. Larson learned this from necessity: designing interview processes from scratch at Uber, then working on and improving Stripe's more structured processes. Larson cites Seeing Like a State: if you've never been a bureaucrat, you can't see the way a bureaucrat sees, and if you can only be a bureaucrat, you don't know how software gets made.
The host summarizes it this way: middle managers often become good bureaucrats who lose their technical edge, and engineers promoted straight to CTO lack the bureaucratic skills. Larson agrees and adds that line managers often make better instinctive executives than middle managers because they're used to the details. Their judgment is better on the non-bureaucratic parts of the job and worse on the bureaucratic parts. Now that middle layers are thinning, the bureaucratic skills are harder to learn, while decision-making, product understanding, and knowledge of how software ships matter more. Larson doesn't think the path is easier or harder overall, just harder in different ways. Executives also often run functions they've never done. Larson has run customer operations at Imprint and support engineering and data engineering at Stripe.
Advice for stuck middle managers, and the tech lead manager role
The host relays questions from two heads of engineering who feel they may be in a career trap, with the VP role unlikely at their companies. Larson says it's a hard time for middle managers, especially those trained in the 2010s empowerment and functional-bureaucracy model, because "people have changed the rubric on you." Larson describes their own move from a team of about 250 at Carta to about 25 at Imprint as a deliberate chance to reskill, and says that in management today "either you have to get back into the details," or accept that your current skills may not be valued in three years. That means deciding whether you're retiring or reskilling. Larson considered stopping after Carta but wanted to keep participating.
Larson's concrete advice is to be open to a backward or lateral step. Larson argues line management is now more educational than middle management, because AI tools for line management and writing software are very good, while LLM tools for headcount planning and performance reviews are "basically terrible." Larson's example is Jasmine, a former Carta colleague who previously ran engineering at Mux. She joined Imprint as a VP who directly manages a team and is writing a lot of software, something she never had time for before. Larson's advice is to tell people if you're willing to do this, because they'll otherwise assume you wouldn't, and Larson would hire such people "all day long." Larson acknowledges many people feel the industry betrayed them after working hard under rules that then changed, and says they are right that they worked hard. Larson's response is that the industry has always been volatile, the stable 2010s made people forget that, and "we really do just get paid to do things that are valuable to the companies."
On the tech lead manager role, which both Larson and the host had once called a trap, Larson says it is "still a trap, but it's a highly demanded trap," often because there's no alternative as organizations flatten. Larson sees two patterns: managers with larger spans (12 to 14 reports instead of four to eight), and directors managing tech lead managers. Some of Imprint's best hires came from pure management roles and wanted to write software again. Larson wants people with the ability to do bureaucratic work but not the desire to, who use that ability when needed and default to building. Larson admits current interview processes aren't designed to find them.
Writing, AI, and books
Larson started the blog Irrational Exuberance in 2007 and is nearing 20 years of writing. It started as networking through tutorials, which Larson notes an AI would now write on demand, then shifted to lessons from Uber's hypergrowth and expanded at Stripe, which Larson describes as having a real love of writing. Larson has written four books and says they probably won't write another for at least a decade, maybe ever. Larson believes writing is a unique way for individuals to change the industry, and credits Staff Engineer with establishing "staff-plus" as a concept, something no single job could have done.
Larson won't write with AI. Larson calls their own writing idea-focused and minimal rather than craft-focused, and says publishing AI "slop" would throw away an audience built over tens of thousands of hours, which they could never rebuild. Larson's rule is: "if I don't have something to say, I just don't say something." The host describes writing as how they think, and describes testing half-formed observations on social media while drafting. The host also tried a chatbot trained on their writing and found it repeated opinions they had since changed. A "Will Larson bot" would still say the TLM role is a trap.
Larson points to an essay on becoming influential in tech, which the host calls the best post on the topic. Larson's argument is that people who don't want to write feel they must produce volume, so they turn to AI. If the goal is visibility, Larson thinks one well-tested piece that gets traction beats hundreds of posts, unless you run a subscription business. Larson cites Tanya Reilly, who has written relatively little but produced the influential "Being Glue." Larson says candidates with large volumes of unread AI content give them pause: it looks like following a recipe without tasting the result, and Larson reads it as poor judgment rather than ambition.
For book recommendations, beyond Larson's own books, including Crafting Engineering Strategy from the previous year, Larson names Donella Meadows's Thinking in Systems: A Primer, which Larson says formed their view of management as a mental model rather than a business book. Larson also recommends books that offer a different perspective, such as works on type systems and Hillel Wayne's book on TLA+. Larson rarely uses either directly but argues the ideas address one of the biggest problems right now: if more software is written by agents, making verification cheap becomes critical.
One of the worst parts of being an engineering manager in the 2010s is when you joined the ranks of engineering management, one of the first lessons you'd get is, "Hey, you need to never make a technical decision again."
I remember that's literally what I was told. And that's definitely not the ethos today. One of the ideas that I was really frustrated with at the time was this idea that staff engineers principally got to work on interesting, fun problems that they found personally rewarding. Some of my very best engineers were like, "I want the more interesting problem." The moment is like, "I only have suffering to offer, but this suffering is what's going to make the business succeed."
People are wondering, "Am I in a bit of a career trap?"
It's a hard time in middle management, right? What I found personally very rewarding is getting back into the details of writing software. This is a unique moment to actually reskill. I do think if you're in management, I think right now either you have to get back into the details or...
How has engineering management changed in the last few years? Will Larson is one of the best people to talk about this. He's been an engineering executive for the last 10 years at four different companies, wrote some of the most widely read books on engineering leadership and staff-plus levels. And I really appreciate how deep and thoughtful of a thinker he is.
Today we cover how zero interest rates heavily influenced a decade of management habits in the 2010s, many of these habits no longer helpful. How the role of engineers is changing and why solving problems that the business cares about is becoming much more important to thrive inside of companies. The path to becoming a CTO today and why line managers often make better engineering executives than middle managers do, and many more. If you're an engineer or engineering manager and are wondering what your career path could look like with AI changing the industry and how the path to CTO looks like today, this episode is for you.
This episode is presented by turbopuffer, a ridiculously scalable, fast and cheap hybrid search engine built on top of object storage by an engineering team that I've really grown to like after spending time with them. The turbopuffer engineering team is doing something really cool. They are completely redesigning their storage architecture from first principles to make sure it's faster, cheaper, and more reliable at scale.
The new system called turbopuffer V3 is not yet in prod, but it already passes CI. Still, it's not nearly as performant as the current production version, V2. And so, the turbopuffer team is now perf-maxing query plans until they are as fast or faster than V2. They're doing all of this in public, documenting how they increased performance all the way up to the V3 production launch. And it's so cool to follow along and see the different performance wins and how they did it.
For example, we see how full-text search, the red line, started out as 129 times slower than the current system, but now it's only four times slower. And in the development log, we can also see that the team used things like batch scoring or batch reads to increase performance. A lot of these wins are basic foundational work right now, but the team will be sharing some very cool systems work over the next few weeks.
But hang on, why is the team doing a full rewrite? It's because search is changing, and so is turbopuffer. Turbopuffer started as a vector database, but it's no longer a vector DB. It has evolved into a high-performance database for search. The V3 discards the vector primary index, and all indexes will share the same columnar storage system. Every query will run on vectorized query engines, which is something you don't usually see in search databases, but which will result in both very fast search and just generally a very fast database.
The turbopuffer team is keeping a work log of their journey, and you can follow along at turbopuffer.com/v3. It's a blast to see how it's being built.
Will, welcome to the podcast.
Thank you for having me. Excited to be here. Finally chat in person.
Finally in person, because we know each other for so long. Both started writing. We will talk about that writing as well, and also just hung out before, but this is nice. I want to jump into the beginning. How did you get interested in computers? Were you interested in computers to start with?
From the time I was five. We had a Mac IIc or something like that. This was not black and white, not the very earliest one, but it was pretty basic. And I think it had one thing that I understood how to use as a four-and-a-half- or five-year-old, which is Kid Pix. Did you ever use, see Kid Pix?
I didn't. I know of it, but I never used it.
It wasn't a super sophisticated thing. I don't even know if you could save files, but it was basically a graphical program for children, and you could make squares and circles and little graphics. But that was the introduction to me, and I remember using that computer a lot. This is an era when there really was no internet to speak of, really no way to figure out how to do anything other than just banging on it, which is the fundamental of writing software in some ways, is just banging until you figure out how to make it work.
So I think that was a great introduction, but to be honest, I wasn't doing a whole lot in software other than multi-user dungeons. Did you ever play a MUD growing up?
Multi-user dungeon. No, what's a MUD?
It's a text-based game. And so, basically, the idea is that World of Warcraft is actually a graphical text-based game that we basically derive from these ideas of MUDs, or multi-user dungeons. But when I was a kid in high school, that's when I started doing more software. So a little bit, you start with IRC, Internet Relay Chat, and mIRC, which I don't even know what the M stood for, to be totally honest, as a client. Could do a little bit of scripting with regexes. But then a lot of the MUD clients, as a kid, could do a ton of triggers, regexes, and that's where I got a little bit further into it. Learned my regexes there, but really I didn't start writing software at all until I got to college.
And then at college, what did you start writing? How did you get into a more serious level?
So I came into college not really knowing what I wanted to study, and I thought I might do physics or English or economics. My father was a professor of economics, so there's some family lineage to try to live up to. But ultimately I fell into computer science at my school, which was a small private school in Kentucky. There were, I think, 13 people in the computer science program the year that I graduated. And that was the entire program. So I think there were four graduates my year, and then the juniors and sophomores in total were 13, and so it was quite small. So I think there were four that graduated my year, but really that was when I got into it.
And why? I just enjoyed—we had an intro course that we were required to take. And it's funny, with a lot of American colleges, especially smaller private ones, they have random requirements. So I was required to take an introduction to Christianity course, the history of Christianity, then also required to take an introduction to computer science or some other course. I forget what the other alternative was. So I really enjoyed the computer science one.
Did you decide that you're probably going to be a professional software developer, engineer, programmer, whatever we called it back then?
So I think one of the interesting things when you talk a lot about careers is a lot of folks go to elite schools. So you talk to someone from Waterloo, and they're like, "Oh, how did you get your job?" They're like, "Well, literally everyone at Waterloo is coached on how to go do software." They're literally trading the answers to interviews with each other. So they know the interviews and drill on the interviews before they go in. They also have an amazing program. They've been doing co-ops for years. They're great. They're great candidates. They tend to do really well.
But if you go to Waterloo, you go to MIT, you go to Stanford, there's this path, and you just put your foot on it, and you do well, and it carries you there. And these are amazing students.
Almost.
But it is an escalator. If they're good enough to stay in the program, they tend to get there. Going to a small school with, again, four-ish folks that graduated my year in computer science, there really was no path. I remember one of the folks that graduated the year before that was working in software. He went to go work at a horse racing betting software company in Lexington, Kentucky, which I don't think there were more than five engineers there. And this was a really talented, really capable person. And this is what they knew how to find.
And so for me, it actually was interesting. I wasn't quite sure what I was going to do. I actually ended up teaching English in Japan on the JET Program, the Japan Exchange and Teaching Programme. And so I was in the middle of nowhere in Japan, in Gifu Prefecture, for a year. But that's when I really started writing a lot. So I was like, "Hey, I'm definitely not going to be an English teacher for more than one year. It's a one-year contract." I did the one year. I had no thoughts about doing a second year, and I came back and started trying to figure out how to actually break into Silicon Valley from there.
And you had the goal to break into Silicon Valley already. Why Silicon Valley? Can you take us back to what Silicon Valley was like back then?
First, at the time I didn't know what Silicon Valley was, right? I just wanted a job that paid well. And I think this was—so I graduated in '07, and then the next year I was teaching English, and then I actually started working for Yahoo in, I guess, maybe early '09, something like that. Really interesting time, though. So initially I was working for Yahoo from the East Coast as a remote contractor. It was an interesting moment.
So I started working with Yahoo because I wrote a series of blog posts on something called Yahoo BOSS, Build Your Own Search Service. And this was a product that they had launched on this one team, and I think it wasn't getting a lot of traction in the industry. But when it launched, I was writing a lot. So I did three or four tutorials on building BOSS things in Python, BOSS things with Django. Made a Django—I forget the name. It might have been Django Springsteen, was the name of the search library to use Yahoo BOSS with Django. And so I think they got excited because someone was writing about it, which also got me excited. And that ended up ultimately in this contract job.
And then ultimately they had the policy where you had to move after a year as a contractor and go full-time. And the reason I was hired as a contractor, though, is they had a hiring freeze for basically that entire year.
Because this was right after a financial crisis.
And so they're just going through layoff after layoff. And so they could hire a couple contractors, couldn't hire anyone full-time. And then at 12 months they are like, "Hey, now you have to go full-time or you're not employed," because there's the compliance obligations to avoid. And so they're like, "Hey, and if you go full-time, you got to move." And so basically, I had a job at Yahoo if I moved, or I didn't have a job at all. And so I moved from the East Coast out to California. Pretty easy at the time. Didn't have a lot of obligations other than employment.
Yeah. And back then, Yahoo, I guess, their star was fading with Google, but they were still a pretty good company, right? And good engineering challenge. WhatsApp founders famously came from Yahoo.
I think Yahoo honestly is a really interesting company, right? And so I think one of the challenges for Yahoo is if you came into tech in 2007, 2008, a lot of open source made things easy. Yahoo, though, is a company that predated all these open source libraries. And so how do you run a server at Yahoo in 2007, 2008? You run YApache. So they had their own fork of Apache with a Y in front of it. What programming language do you run on YApache? Well, of course you would run YPHP, which was their fork of PHP. And what UI library would you use? You might know this one, but it's YUI.
And then how do you install things? So if you think about Ubuntu, you'd have apt or whatnot. Well, you'd use Yinst, which was this Perl script that would navigate the dependencies for installation. And so just a really, really fascinating place to come up, but every single thing there they made from scratch. Even they had a precursor to Kafka, which I think was called YRepl, which basically would be a read log to do replication outside of a database. And that was a core tool for managing state. Just a fascinating place.
And I think if I just had one quick Yahoo story that I loved: David Filo famously approved every server that was allocated in the... And so one of the founders would review. And so the theory was this would drive accountability, right? You had to prove you needed more servers. What it actually did is it created the black market of servers. And so on my team there was one engineer that had 30 servers from a prior canceled project he never gave back. And so he would basically sell them to you, effectively, right? And so if you couldn't get approval...
For favors and IOUs and stuff. And so it actually meant they had massive numbers of outdated servers that were using inefficient power wattage because people would never give servers back. And so you had to find someone who liked you with servers to loan for your projects to launch. Fascinating how these incentives drive just wild, wild situations.
Well, I guess that's one thing you remember for when you later became an engineering leader.
Yeah, it was very instructive, right? And I'm curious, did David Filo know that people literally had a black market of servers? I assume he probably did know, but didn't change the top-down mandate.
Then after Yahoo, is that when you ended up at Digg?
Yeah.
At the time, a very famous company for a very amusing story. Can you tell us about how you decided to switch? Because it was a startup, right? Digg was a startup back then.
So depending who you talk to, there are going to be different narratives, but I think the Digg narrative is basically it was, before Reddit, the big community site, but it was centrally curated. And so Reddit had these subcommunities where the communities voted with the algorithm on what they wanted, the Reddit algorithm. Digg was actually centrally curated on what went onto the front page. And so it had a very different experience, but it also meant that the quality was consistent, and you had less of the bias in subreddit problem. Instead, you had a bias in central moderation problem.
And there were communities, there were IRC channels where people would basically trade favors trying to get things onto the Digg front page. And in fact, for Digg, before we changed the algorithm—and I had people saying they'd come murder me for stealing their livelihood, because their entire livelihood was selling access to the front page. The new algorithm didn't support that, and they were literally threatening our lives. And so it was a fascinating place.
But Digg was a really important website. And when I joined Yahoo, this wasn't true, but a little bit before I joined Yahoo, literally the front page of Yahoo was a static file on a net, a...
NetScaler, which was basically a filer, which is a way to store files. And Yahoo Sports was just a static page, right? And so Digg in that era, early on, was a very important static page. But then it became not a static page, a dynamic of sorts community, I think, really was extremely important to folks at the time, though it's faded into slightly less relevancy over time. And it's been restarted a couple of times since, but never really picking up the traction it once did. I think it was a top 10 website at one point.
Yeah, it was top 10, and that's why the story that I remember is they did a big rewrite. I think it might have been a v4 one, right? And from the outside, what I remember is Digg was doing good. It was popular. There were complaints, a bit like with Stack Overflow maybe a few years ago, and then Digg did this massive rewrite and disaster followed. And it was a very juicy story. Everyone was talking about it. What can you tell us? What actually happened, and what did it look like from the inside? Because you were building this thing.
Yeah. So I was there for that launch, and I was there after that launch. And so I joined Digg about three to six months before that launch went out, and it was a fascinating time. So I joined, and then the Friday before I started, the CEO got removed, and then Kevin Rose came back in as—
He was founder, right?
Kevin Rose was the founding CEO. He was there for a while, and then he stepped back, and then he came back in. But he came back in, and we had this very hard deadline to launch Digg v4, which was a rewrite. And so Digg v3.5 was the prior version before Digg 4, basically entirely PHP software running on distributed MySQL. And this was a different era, right? So Digg was running on hardware that we owned in a colo, and so it wasn't just spin up some capacity or whatnot.
But Digg v4 was trying to solve a bunch of problems. But first and foremost, the internet was changing at that time. So this Digg, at one point a top 10 website, was starting to lose traction to Facebook, and it was starting to lose search traffic as well. And so there were a series of large Google changes. And I think we don't hear about Google search algorithm changes that much these days, but back then there was the Google Panda algorithm change, and there were a handful of others. And they were starting to really reshape how traffic flowed through. So for a company like Digg, a substantial amount of the traffic, I want to say 60 to 70% of the traffic, was actually coming in via search.
Oh.
Into something called the permalink. And so the permalink pages would be, say you write a great blog post. That would then get posted to Digg, and then we'd have the reference to the permalink and also a small link to your site, of course, and comments about it. But we would take a ton of traffic through these permanent pages that would be higher PageRank than the original source themselves. And you can argue now, maybe we shouldn't have been. Probably not. I think that would be the perspective today. But from the perspective of that moment, we were actually creating the discoverability and the conversation around it. This was before Twitter. This was before Facebook got substantial distribution. This was a really early moment. And so for us, that was the core of where people came into.
Also, it was really where we sold the most ads. And so losing that traffic was basically tanking the company's financial prospects. And so Digg v4, from the outside, Digg was working pretty well. It seemed like maybe it was growing, but really what we knew is Digg was shrinking. We had one beautiful month of profitability, and then an algorithm change hit, and we started shrinking. And we knew that this is existential for us. We were going to simply not exist not that far down the path.
When I joined, I want to say 110, 120 people, something like this. But this was in 2010, when funding rounds were a million dollars or $3 million or something. It's not like today when people are raising $500 million and you can have a team of however many. It was a very different moment. So even with 110 people, that was pretty substantial, and we were running down the remaining coffers pretty fast.
The launch internally, did it go poorly? Did it go well? Because from the outside, again, there was a perception that it went terribly. I think a lot of it was the community was upset, and then there was a bit of celebration as well, like, well, they messed it up. They kind of deserve that. The mob, the people especially who were very involved, they didn't like change.
First, there's a really fascinating set of cartoons someone made, which is Digg versus Reddit, and Digg is blue, and then Reddit's red, and there's this giant war that someone did, a four- or five-part—it's this very intricate comic someone wrote around this time. And it's kind of interesting to see how much energy people put into it. But yeah, there was a lot of passion, and I think it's one of the challenges of a really passionate community. When you foster a passionate community, then you change what you are, the community doesn't like that. Because there are literally these folks, like MrBabyMan was a really big one, one of the top Digg curators, unsolicited curators of content getting to the front page. And he and many others, they lost their business, right?
But the launch went pretty poorly. And why did it go poorly? It went poorly for a lot of reasons. So first, the launch technically wasn't really ready. And so we made a lot of really aggressive technology decisions. And again, I came in about three months before this went live, so I don't personally take a ton of responsibility. Also, again, think about where I was in my career. Basically, it was my second job. I'd been at Yahoo for two years. So I won't own this one. Later in my career, I'll own anything that happened. But this one wasn't really my fault.
But basically, we made the decision to launch on Cassandra. And there's a joke at the time, which is that Cassandra was a Trojan horse basically had launched to basically ruin competition in the industry, because anyone who launched on Cassandra was basically screwed afterwards, which I think certainly was true for Digg. And so running Cassandra to replace the sharded MySQL system. That sharded MySQL system was pretty complicated, but it definitely worked. And Cassandra, I would argue, basically didn't work.
But I remember there was a team responsible for load testing Cassandra before our launch. And two days before the launch, this other guy, Rich, who was fantastic, he and I were pulled aside, like, "Hey, well, they never did the load testing. We need the two of you to do the load testing in the next two days." And so Rich was just a software engineer like me. And so we're like, well, I guess we'll install JMeter and try to figure out how JMeter works to do some load testing.
And we had this really interesting conversation with Chris, who was kind of the head of the Cassandra initiative. I was like, "Hey, I can't do the reads on Cassandra." He's like, "That's not true." I was like, "Well, I can't do the writes." He's like, "That's also not true." I'm like, "Okay, Chris, if I'm writing to Cassandra, I can't read from Cassandra. If I'm reading from Cassandra, I can't write to Cassandra." He's like, "That is true." It's like, "Well, we're supposed to launch in two days." And then we launched, and we didn't fix it. We just launched.
And so I think the company thought that we were literally dying, and the company literally was dying. It was running out of money. But it was the most interesting day I've ever had professionally. Can't remember when I got to the office. Probably around 8:00 a.m.
When you say interesting, you mean stressful.
You can let me paint the scene a little bit. So it was a launch, but they also decided to have a launch party. So the entire day, there were caterers in tuxedos with sushi trays, champagne.
Oh wow.
And so the entire office was there to have this celebration. And then in the center of the room, there was this table. It might have been a boardroom table or something, and pulled out this giant table, maybe three or four times the size of this, where the engineers were sitting. And we didn't have enough hardware to run the new system and the old system concurrently. And again, this was hardware we owned. So they were literally wiping the old servers and installing the new software. So we couldn't roll back. We didn't have the money to, because again, we were running out of money. Enough budget for the party, but not enough budget for the servers, right? So we were wiping server by server.
Spinning it up.
And ultimately, the damn thing didn't turn on. It was functionally what happened, right? And so we had this down-for-maintenance page up for, I think, roughly 24 hours, and then we were able to scrap—
While the champagne and the tuxedos are coming around. Surreal.
And there were people upstairs getting massages. I remember certain groups, who won't be named, playing ping-pong while we're trying to get this up. And it was just sort of like magical realism.
I see what you mean by interesting. This is not how I imagined a launch like this.
You can never imagine a launch like this. It was so surreal, truly like a magical realism moment, where there's all this normality happening while this site's literally imploding. And it was unlike anything I've seen, but it's also a moment I take a lot of pride in. And so I've been there three months. I think the interesting thing is we did get the site kind of working, but there were a lot of issues. And so people just gave up. Day by day, people just stopped showing up, and a small group was really the only group left, of maybe six people trying to keep it working. And we did.
And so in the first two days, the same guy, Rich, and I wrote a caching system from scratch on top of Redis, which helped the site stay on for hours.
Yeah.
But we still had huge memory leaks, and so we were actually restarting every process every hour or two. And the only way to keep it up was rolling restarts of every process. And so it took us a full month to debug. And I have a blog post I wrote a long time ago about it, but basically the core bug was we created an API service that could be used for third parties, and it was in Python. The people writing it didn't know Python very well. And this was Python then calling another Python gRPC that was hitting Cassandra.
And basically, in Python, when you instantiate a default parameter, if it's mutable—I don't know if this is still true, by the way. I hope it's not still true, but at the time it was true—where if you basically put a list as a default parameter and then you mutated it, appended IDs into it, it would persist in memory. And so each time that function was called, it would have the same dictionary or the same list. And so the memory of the user IDs, basically every time they call this API to look up user IDs to get the user objects back, all the users that were ever passed into it would still be there. And so it just cascaded over and over and over until we restarted. And we installed memcache kind of in between these, and the memcache would start falling over, and then it would destroy Cassandra.
So really finding that bug, it took me, I think, four weeks to actually find it. Once you found it, trivial to fix. But we got it up, and then our new CEO started, I think, basically the same day that I fixed that. And then basically the entire company got laid off over the next year.
Yeah.
Pyrrhic victory in the sense that we did solve it. It was working, but I think you can't really have the central site be hard to use or down for a month. But if you go to the counterfactual, what would the company have done differently? If they had simply kept up the old site, they also would have died, right? And so I don't know that they made the wrong choice, but at the time I certainly was pretty salty about it.
Yeah. How did you eventually get into engineering leadership? I know you and me overlapped a little bit at Uber. Actually, we both worked there, but we never worked together.
So the end of the Digg story was we got acqui-hired. And so they sold the patents. Digg did a lot of innovation. So in-stream ad units, basically like you see on X now, where they'll have an ad that kind of looks like the stories from people that are posting, Digg owned the patent of that. They were the first company to do it. They owned a variety of other patents. And so the patent portfolio got acquired by one company. Digg.com, the actual brand and the website, got acquired by, I think, Betaworks out of New York at the time. I think they've sold it subsequently a few years ago. And then the team got acquired by SocialCode, which is a third company, a subsidiary of the Washington Post Company from the time when that was a holding company that owned the Post, Kaplan, and this company SocialCode, which basically did advertising on Facebook. And so the team went there, and I was there for a couple of years.
But when I left, I was thinking about, okay, I want to be in engineering leadership, but I can't get hired into a really large role because at that time I was managing maybe a team of 12 engineers or something like that. I can't get hired into this big role because I haven't done the big role. How do we get hired into this big role? And so if you're at a small startup, the scale is not there. Even if you go be a head of engineering at this 15-person startup, there's just five engineers there. And so you can never get into the scale they're looking for for these large roles.
And so what I was looking at at the time—and my thoughts on this topic changed a lot since then, but this is just honestly where I was—it's like, well, if I want to manage a large team, I got to join a company that's growing really quickly. And I looked around, and where is growing really quickly? Uber was growing really, really, really quickly.
15, 16?
I think this is 2014.
This is when growth just started to kick off. This was very early.
This is pretty early.
You had to be in San Francisco to know that Uber was doing really well. Outside of SF, I think news started to spread in 2015, and then by 2016 it was global news.
That sounds right. But the way I knew is that one of the engineers, Jeremy Grosser from Digg, who was on infrastructure there, left and joined Uber. And actually, there's an interesting technology adoption curve, because Digg made all these technologies. One of them is called Clusto, which I think Digg, Uber, and then Second Life, that company—I forget their actual name, but whatever it is—may be the only three companies in the world that ever adopted this tool, Clusto. But it managed basically a key-value store to manage infrastructure, a little bit like Consul from HashiCorp is very, very similar.
Just years before Consul was out.
Yeah, it was probably made originally in 2008, 2009, something like that. And so he had actually brought a ton of the Digg stack over into Uber because he was one of the first infrastructure engineers there. And so his team was hiring, and I
Really was a generalist engineer. At Yahoo, I was actually hired to do front-end engineering, which I don't know if I was super qualified for. And then at Digg I was doing frontend, Cassandra work, backend, just everything.
And then they hired me to run this SRE team, but it wasn't really an SRE team. It was just an infrastructure team that they called SRE. I read an SRE book the week before, trying to figure out what the job was.
You probably read the SRE book that existed, which is the Google SRE book, right?
I don't think that book existed then. It might have.
What book did you have access to?
I think there was a DevOps book, might have been what I was reading. And it might have been The Phoenix Project might have been out around then. I'm forgetting the exact timeline, but there was something like The Phoenix Project. Maybe there was one other. And I just showed up, and then, of course, classic interview, I remember doing the traveling salesman problem on a whiteboard in Python, stuff like that, which didn't really evaluate my DevOps acumen very well. But somehow, I think the company was a little bit desperate on hiring at that point, but did get the job and started.
And then you were leading a team already. You were growing a team.
So the team, when I joined, I probably was managing six people, but it was managing an incredibly broad range of responsibilities. So Uber, when I joined, was 2,000 people. It was 200 engineers, and this team of about six was managing our databases, our Kafka, our actual servers—well, the management of the servers. There was a tech ops team that managed literally racking them and installing them. But once they were racked, we managed that. Our service provisioning, we owned basically everything other than there was a two-person developer experience team. There was this tech ops team that actually racked and installed the servers, but we owned all the other infrastructure, and it was quite an experience.
And then in terms of just focusing on your leadership journey, your management journey. So at Uber, you started at this team. How far did it go, and then what were the next steps between Stripe, Carta?
Yeah, absolutely. So when I joined, that team was, I think, roughly five or six folks. When I left two years later, that team was 70, and that was all external hiring. And I think that was the most amazing thing about Uber at that point. Uber doubled the engineering headcount every six months. And so you can't just steal people from other teams because those teams are hiring a ton too and also really underwater. And so I really learned how to hire at Uber. I had no idea how to hire before I joined Uber, but they were just hiring so many folks.
I think one of my favorite stories was there's this guy out of Eventbrite that I managed for a year and a half. We gave him an offer at 4:30 p.m. on a Friday, and he started Monday at eight. But I have so many Uber hiring stories, some of them funnier than others. I remember someone who interviewed with Uber four times, and I saw him interviewing a fifth time. And they got rejected four times, and the team interviewed him a fifth time. I was like, we cannot hire this guy. We've already interviewed him four times. Stop wasting his time. But he got an offer for that team, and he joined, and then he quit within six hours and took a different offer because he got an offer from Twitter, which was right a block away.
A block away, or somebody was also—
He spent more time interviewing for Uber than working there, despite accepting the job. And there are just so many wild stories from the Uber experience there. But that team, six to 70 over two years, and our scope increased a lot in that time, but it was just a really wild time.
And it's hard. Some of the companies today, I think, remind me a lot of that Uber experience. But Uber, when I joined, basically every Friday we were literally manually reallocating software between servers, trying to keep the servers up, because typically Friday rush hour was our peak time of the week. Though a Saturday you might have more volume, but that would be spread over more hours because people aren't all leaving work at the exact same time. So many wild stories of just what a manic place that was to work at.
But I was there for a couple of years. Then I went to Stripe. I was at Stripe for about four years, from 2016 through early 2020. Then I was actually at Calm, and a half, three years. CTO, that was my first CTO job. It was really deliberate at that point. I think I was really worried. So I'd been at Stripe for four years. Stripe had done really well. I think I joined at 5.5 billion, and I think it was at 96 billion four years later.
Company valuation. Yeah.
It really came together. But also, I think the quality of the work we were doing was going really well. And I want to say I was roughly—someone that joined the day I joined at Stripe was employee 500. I don't know if that was me or someone else in the batch, but I think by the time I left it was 1,500 to 2,000 folks, something, at the company. And so Stripe grew about 70% year over year the period that I was there, which is a lot less than Uber, which doubled every six months. And obviously Uber subsequently contracted much more aggressively than Stripe ever did.
Which is the natural consequence of doubling every six months.
Yeah. But at that point I was just worried that my entire savings were all in one company that was never going to go public, which I think has been true. And so it was a bit of a diversification play, but also the desire to—what does it actually feel like to fully lead? And so for my entire time at Stripe, I was basically one step away from the CEO. So reporting into Jeff Winner for my first year, then into David Singleton, who was my manager for the rest of the—
Stripe's CTO later, or maybe at that time as well.
He joined as the CTO. No, that's not true. He joined as the head of engineering, and then he became the CTO maybe a year or two later, something like that. But I was reporting to him from when he joined through the time I departed in early 2020. So I was always one step away. But I was like, what if I could get into that seat? Could I do things some better? And that seemed like the right thing to try for anyway, and has been educational ever since.
And then you have had several CTO gigs then. So it was Calm, three and a half years CTO. Then you were at Carta.
So I was at Calm for, I think, three and a half years. Then I was at Carta for two years, and then I joined Imprint a year, year and a half ago, something like that, which is where I am today.
Okay. So now, three-times CTO, I need to ask you: what does a CTO do? As an engineer, if you put yourself back as being an engineer, of course, I think I myself had aspirations when I was an engineer. It could be nice to one day be a CTO, because in your mind it has some management, but also a lot of CTOs you see are technical and close to the thing. How do you define the CTO? I know you have a book on this as well, on engineering executive careers, and there's a lot of answers, but what is your mental model? How was it, and how is it now, especially because now with AI, I think your role has changed?
I think one of the worst parts of being an engineering manager in the 2010s is, I think, when you joined the ranks of engineering management, one of the first lessons you'd get is, hey, you need to never make a technical decision again.
That's what it used to be, right? I remember.
That is literally what I was told. And I think you'd see people, the most important thing is to let go of technology decisions. And this was the ethos then, and that's definitely not the ethos today. But I think the thing that I aspirationally really loved is the CTO was the only place where this engineering career, this technically oriented one and this managerial one, converged.
And so one of the things I've believed strongly in and continue to do today is having both engineering managers but also individual contributor engineers report directly into me in CTO roles. And so today I have 14 direct reports right now, and I think four or five of them are ICs, and then the rest are managers. But having that balance is really—for many roles in management, you're just not allowed to be a technical leader, or you weren't for a long time. And so to me, actually getting to the level where they converged again and you were allowed to be a leader on the technology and the management side was one of the big appeals.
But also, there's so many bad CTO decisions out there. And I don't mean this to reflect any of the individuals I've talked about. I think David in particular was really excellent. And the decisions at Stripe were generally—decision quality at Stripe was pretty high. And so sometimes you could be like, I disagree, but you couldn't be like, this makes literally no sense. But often, especially outside of the Stripe experience, there were just decisions that just didn't make sense, and I thought maybe I could do it better.
But I think the thing that I've really loved in the most recent era is you're really not just allowed to be in the details again, you're really required to be. And that to me has been the very most fun part of being back in software in this era. I think I've done over 500 pull requests in a year and a half at Imprint, going all the way to production, which is substantially more than I did in my prior career combined. And that's pretty cool, and that's—
And then you did Imprint as CTO. Are you more than CTO at Imprint?
So I joined as the CTO, and right now I'm also leading the product design teams.
Right. So product, design, and engineering all reporting into one person.
Yeah, it's a unified EPD structure.
Is this a new trend, one person? Because CTPO I started to hear about two years ago, chief technology and product officer, two, three years ago. And now I'm starting to hear design also going into one person a little bit.
I don't know if it's a trend or not. I do think that one challenge I've been seeing with agents getting more mainstream is you actually have to be able to make decisions and make them quickly. How do we do things? How do we not do things? How does agent adoption work for product? How does it work for design? Hey, design, if they aren't using the new tools like Claude Design or the Figma MCP or whatnot, then they're really the bottleneck. How do you split the responsibility there on who owns design now?
And so Imprint, for example, has been experimenting with this developer-led design model for second-tier features, and it's created a ton of friction. It's solved this bottleneck. It's also created a lot of who's responsible for these design gaps that show up when engineers do lead design and sometimes don't follow the design system properly. And so having a unified point that can actually make these trade-offs, I think, in this moment of immense change, is pretty powerful.
I do think, just in my core, I believe the industry is changing more in terms of how we create software than it ever has in my career. And I started in late 2008. I've never seen a change like this. A lot of the changes I've seen were more subtle or more cultural or social. Just how I create software is really different than it was even a year ago. And having a centralized mechanism during this moment of immense change I think has been really helpful for us. But I don't personally think it's a long-term trend. But I don't know. I'm trying to avoid predictions about too deep in the future at this point. Everything's changing a lot.
Will just said how he's never seen the way we build software change this much in his whole career. One thing that is changing is the stack that we use for Git hosting, which brings us nicely to our seasonal sponsor, Entire. Git hosting has a problem that starts with AI and ends with agents. Engineers like you or me are creating more code with agents. Those agents are pushing even more code, and many of us are running parallel agents, which results in even more code. GitHub is clearly struggling to keep up and has frequent outages. But there's a solution. Entire was founded by GitHub's last CEO, Thomas Dohmke, to rebuild Git hosting for the agent era from scratch.
Entire was built to be very fast and to have your repos regionally close to you to reduce latency, allowing for fleets of agents to push in parallel. And Entire's performance is next level. Their repo can handle 418 pushes per second, which is up to 89 times faster than every competitor on the market. When GitHub is down, you can still keep working, and you don't even need to migrate away from GitHub. You just sign up to Entire, and the platform mirrors your repo.
Also, have you ever wondered what prompt resulted in this code being generated? I find that the prompt and the conversation with the agent carries more information than the PR itself. Entire captures all the prompt history with your agent right in the repo, easy to check back, and has a pretty innovative UI to show all of this. If you're looking for Git hosting that works even when GitHub is down, head to entire.io/pragmatic, install the CLI, and monitor your repo with a click. I've already done it. Oh, and did I mention it works with any agent, and it's open source?
Oh, and one more thing. Something tells me they're releasing a full Git forge soon, from CI to a replacement for the pull request and code hosting natively on Entire. Get in early on what could turn out to be the next GitHub for the agent era.
I also want to mention our season sponsor, Antithesis. Agents are writing more and more of the code that we ship to production. We know that. So how can we trust that our system works correctly and without bugs? Code review is an increasingly naive and ineffective approach to do so, which is why Antithesis goes well beyond this practice. Antithesis runs your whole system in a hostile simulation. This simulation includes both targeted testing and fuzz testing. By running this simulation, it finds every bug before your users do. And because the simulation is fully deterministic, it doesn't only find bugs, it gives you a perfect reproduction of every issue, which makes it much easier to fix issues.
The first thing I thought when I heard about Antithesis is that automated bug discovery and fully deterministic statistics sounds like science fiction, but actually it's hardcore engineering under the hood. Jane Street, Fly.io, and the etcd community ship agent-printed code with full confidence because they know it's been verified by Antithesis. To see more case studies and details, head to antithesis.com/pragmatic.
And with this, let's get back to Will and what changes he's observed across the industry. But why don't we go back to when you started in the industry? Can we talk about the changes you've seen in software engineering? Let's just focus on your experience. So, to talk about the Silicon Valley companies, we know they're a bubble, but they're a useful bubble to talk about. How did building software change, leading teams change, starting from 2007 all the way to today?
I don't know what your first experience was like as an engineer, but when I was first an engineer, first as a contractor, I was 22 for Yahoo, working remotely. No one else on the
Team was remote. And then when I went full-time, I had the same manager. So I had the same manager for two years. I had two one-on-ones with that manager over two years.
I had no one-on-ones, or once a year for performance—not even performance, for pay conversations. And then they would say, "Yeah, good job. Here's your raise." And then promotions used to be like, "Congratulations, you're now promoted." And it was more about, "Yeah, you're making more money. Cool."
And I didn't have any expectation that I'd have more than two. And the first one was basically he wanted to talk about another employee's performance, trying to get some signal about whether they were doing poorly. So it wasn't feedback from me or something. And the second one was, "Oh, you gave notice, you're leaving. What's going on?" And that was just kind of normal.
And so his job, the way he saw it, and I think he was really good at it, was to go find an opportunity in the company for the team to have impact. So he was going around the company searching for something we could do that was really valuable, and he brought that back. And then our job was to implement it and make it work, and that was just how the work was done. And there was no idea that he was helping me have a rewarding career or something like that. That, I think, was irrelevant to him.
We're talking pre200 pretty much. Yep.
And then when I got to Digg, I think all the managers basically got fired. I actually became a manager at Digg because I was the only one dumb enough not to say no when they asked me. But I had two and a half, three years of experience tops at that point. And I knew nothing about management. And again, I never interacted with my manager at my prior job. I interacted with a tech lead to set technical direction and then got upset if I screwed something up. But that was kind of the entirety of the leadership model there, right?
So I came in and I was a manager, but there was really no management at Digg other than me for the last year, and I didn't know what I was doing whatsoever. So really I learned a lot at Uber, but again, at Uber everyone was so busy there was no management there either. It was really sink or swim, and either you figured it out or your team really struggled, because if you didn't get the decisions right, the scale of the growth around you would just crush your team. And so again, really no support.
It was interesting because by 2017, when I became a manager, Uber put together its first apprentice manager program, which actually, talking with people—so when I became a manager there, I had a few days of getting together, mentors assigned, all that. Most engineering managers never, ever had that. And it was brand new at Uber. They made it more formal. So they actually did some internal DIY training, honestly. But yes, before that, nothing. And they did it because I think they realized that bad managers were becoming a really big problem.
Yeah, which makes sense. I think people were just figuring it out. And I think the Uber I worked in was extremely myopic. So one of the values, I might get it wrong, was basically, "Solve it yourself." That's not literally one of the stated values, but people didn't go get blocked in other teams. They just did it themselves. And maybe "Let builders build" might have been a variant of this.
Yeah, there was the "Let builders build," "Be an owner, not a renter." And when you combined the two, there was something about principled thing, but yeah, it was like, just roam your way and don't ask for forgive, just get it done.
Yep. And there was sort of a no-dependencies culture, which I found really empowering, where it's like, "Oh, I'm blocked." It's like, "You're not blocked. You need to go figure it out." And that was really painful sometimes, but it really meant you could always take responsibility and just figure it out yourself.
Sometimes I remember signing purchase orders for Uber China where we couldn't get anyone to sign it. And I ended up signing a multi-ten-million purchase order, and I was probably 27, or—there was no one to approve it, and it had to get done. And so Amir, the head of tech ops, and I were just kind of staring at each other, and it was like, "I guess we'll just sign it." And the thing that I loved about Uber of that era is no one ever talked to me about it. There was never a follow-up. Probably not good either. From a corporate governance perspective, that's maybe a little bit ad hoc, but we really didn't get in each other's way and were able to do amazing things. Also an extremely challenging place to work in its own right.
Yeah. People left and burned out for a reason, a lot of them.
100%. But really, all the way through then, I finally get to Stripe around 2016, and Stripe sort of famously didn't have a management culture either. So when I joined, there were, I think, three managers. One of them, who I really loved working with, Raylene, I think had somewhere between 40 and 50 direct reports, because Stripe had read the paper from Google when they used to have basically no managers for some period of time.
And how it didn't work.
Initially managed—no, they initially followed the model where there were no managers for a long time. And so there I kind of joined in the wave of managerization as one of the early managers in that wave. But I think there were three or four managers: Raylene, Jorge, or there were one or two others. Jorge was managing a team of, I think, 30, with maybe one manager underneath him for those 30 folks. Raylene, I think, had no managers. It was a very chaotic time as well. But again, coming out of the Uber experience, I was very familiar with chaos at that point. But also the business was growing really, really fast.
And so I think really that period at Uber, though, is when the cultural change around what is the role of manager, I think, started to cement itself a little bit. And this role of manager not as a core doer or a decision maker, but as a facilitator, really started to take hold. And that was pretty fascinating to watch.
And one of the reasons why I wrote Staff Engineer, which I think came out in 2021, was actually collecting feedback for some of the staff engineers I'd worked with. Because one of the ideas that I was really frustrated with at the time was this idea that staff engineers principally got to work on interesting, fun problems that they found personally rewarding, which wasn't what I needed for my staff engineers at all. I needed them to go work on the business problems and the scalability problems that I, as a leader—and when I left Stripe, I want to say my team was 250, 260, something like that—I had all these problems.
And so some of my very best engineers were like, "I want a more interesting problem." And it's like, "I only have suffering to offer, but this suffering is what's going to make the business succeed." And so how do I refocus them? And I think many of them at the time felt like that was a violation of the management contract. My goal and my obligation was to give them interesting work they wanted.
Oh, interesting how, when we look back, just how it changed from 10 years before.
Totally, right. And so I think that was a fascinating time to be in, but I don't think that perspective was ever really right. If you look at the people who are really senior at Stripe, but also everywhere I've worked, those folks were doing the really valuable work that the company needed. You never saw that perspective really take hold in the executive ranks, but a lot of the folks coming in really had that belief. And I'd say, without naming names, certain companies that they came from had very strong versions of that, where the obligation of the manager for senior hires was to give them fun work. And I think the industry's really reverted on that one for sure.
Was this something to do with zero interest rates? We often come back to this, which is so curious. I wrote about this as well, how an interest rate, which feels so abstract and we sometimes hear about it, how it probably impacted funding in tech for a very long time. Probably hiring startups, startups that were funded that maybe didn't have a business model, but they were still funded, and ultimately software engineering archetypes, manager types, management decisions. And then the music stopped sometime around 2022, where efficiency became more important, and now we're kind of seeing this. And now I think a lot of us are asking, and I'm also asking myself the question, everything that we did during that time, was it right at the time?
Did you ever read—I'm going to get his name wrong, but I think it was Ranjan Roy's "ZIRP Explains Everything" blog post from The Margins?
Yes, The Margins. We'll also link that. I love his writing. Yes.
Well, one of my colleagues at Digg, who then I hired at Uber, was Can, who's the other writer on The Margins along with, I think it's Ranjan Roy. So I read that piece, and it really resonated, right? I do think that ZIRP has explained the entire 2010s in terms of how management styles changed, salaries went up, then competition for hiring became—your ability to hire and close and retain became the differentiator between these successful companies, the less successful, because you simply couldn't retain great people. And that pulled leverage back into the engineers' hands and created a very different environment.
And so part of the way companies were pitching themselves was more relaxed, more focused on you, more career development, and I think in ways that were often kind of self-defeating. So I think there's this broad judgment of the Google perk culture. And I don't think any company that did incredibly well, other than maybe Google and Meta, had that level of perk orientation. But I do think, ignoring the perks, there are many other ways that companies try to focus on rewarding work, interesting work.
Companies like, "Oh, engineers maybe shouldn't be on call. That's for the second-tier SRE to be on call." Which was interesting because that was actually a place where every single person I've worked with knows engineers being on call for the work makes the software better. And yet companies were still kind of like, "Oh, maybe for the enjoyment of it, we should let folks not be on call for the work they're doing, and instead page other people that aren't involved in the creation of the software." And that orientation really reveals, I think, the shift in priorities in those companies.
Well, I also wonder if that was a time where managers were rewarded for having low attrition on the team, meaning few people left, being able to hire on the team to fill headcount. Because often there was a time—this might sound crazy—but filling headcount was difficult. Let's say you had a team of 30 as a director or senior engineering manager. You had 10 headcount this year, and if you couldn't fill that, meaning you couldn't hire the right people, and this was, let's say, senior and above, so you couldn't just get new grads because that would be too easy, then it kind of counted a little against you.
Or someone else would get that headcount, which might help their promotion, because it was kind of obvious if you had 40-plus people, you might be senior manager, that's your director title. And so managers who did well in this environment became a bit of people pleasers, and also they started to shield their engineers from the business, for example. And I wonder if the shielding from the business has bitten back a lot of people, because now, as engineers, you're increasingly expected to be savvy about the business, understand the business. That's when you're going to be in demand. That's when, in a startup or other places, you get to hit the ground running.
And I still hear stories from people who worked at this big company, let's say a big observability company, at any of them, and they went to a startup, a seven-person startup, and they say, "It's terrible. They're not letting me do things." And I'm like, "Hang on, you're a seven-person company. How is it?" And you're an experienced person. How is it that you're not asking, "What does the business need to do? Let's do it. Boom." And they've been socialized in a different place for a while.
Yeah. I think that is a huge part of it in terms of the incentives. I do think the orientation around headcount, filling, team management, the employee survey results—these are really valuable tools. I get a lot out of them. But I remember spending a lot of time at these faster-growing companies analyzing the quality of a manager based on the survey results. But survey results, you have to really interpret them. High scores or low scores don't mean good or bad manager. There's context to ask other questions about, like, "Hey, why is your team all upset about this?" That doesn't mean you're a good manager or not, right? And I think these became a little bit a goal of their own rather than an interpretation mechanism, which in a lot of ways is just lazy middle management in terms of how these are deployed.
But I definitely saw that, and in general, I hate headcount conversations. I think they're almost never the most interesting conversation, but I think in a world where managers are not allowed to make technology decisions, the only lever they have to actually execute faster is headcount or prioritization.
Oh, so are you saying that at that time during ZIRP, where the conventional wisdom was, "Do not make technical decisions," this forced savvy and smart managers to put way more focus on the non-technical side, potentially underinvesting in the technical things, maybe just not making decisions that should have been done, and overindexing on the soft side, or maybe shielding their team from the business, or all these things because they put their energy there?
Oh, 100%. I think that's very true. What I tell my teams that I work with now is hiring is the least interesting way to solve every problem. We almost always should never talk about hiring. But the corollary to that is if you have open headcount, you ask for more headcount, we shouldn't talk about headcount either, because what matters then is the hiring throughput, and you actually can't use more headcount anyway.
But yeah, I think we basically told managers they couldn't think about technology solutions, and then what do they have? They have process solutions. They have scheduling, or they have people. And so I do think this was the natural consequence of the bifurcation of these two roles, which isn't to say we shouldn't bifurcate at all. But I do think we need effective leadership to have both, or they overindex on hiring, and that drives a lot of the bloated orgs we saw in the past.
Let's talk about careers, career development. Let's start with individual contributors. How do you think career opportunities have changed for software engineers working at startups or midsize companies that you see? Because we used to have a pretty clear career ladder: one, two, senior, staff. It was clear on how you get promoted. To start with, AI tools are out. They're writing the code, which used to be Eng 1 and Eng 2. They're now getting better at architecture. We're talking about loops. Do we have a career ladder? Should we have a career ladder? If you are a manager at a company, should you be worried about putting together a career
Ladder right now?
It's a good question. And I do think the expectation is certainly from the teams that we should have career ladders. And the belief, which I think is backed up in research, is that having rubrics drives fairer decisions than not having rubrics. And so I think this anchoring on career ladders is something that's here to stay. And I do think if you don't have that, then you do undervalue certain things. Maintenance work, glue work, gel work, whatever you want to call it, I think gets underemphasized, and flashy things that actually work get overemphasized. I think that's definitely true.
So I'm still a believer in career ladders. I think it's still valuable, but what should be there, I think, is complicated, and I think it's much less clear about what should be in it. I don't know which consultancy has been selling some of these career ladders. Small company by small company, you see the exact same career ladder from some packet you can buy somewhere. And so I think all of those are just really outdated, but we don't have the next version of it.
That said, when I think about folks getting to staff or principal these days, what I really care about is—I've been thinking about doing an add-on to Staff Engineer. I'll get to it in the next decade sometime, but it's really about decision-making in complicated situations. And so first, this idea of navigating ambiguity.
At Stripe, I remember working on this project where we wanted to go multi-region. And I kept getting from the engineers trying to lead it, "We need product to decide how cross-region replication works," or, "What's the product behavior that we support if there is only consistency and availability but no partitionability?" They wanted the product team to make these CAP theorem trade-offs, which the product team can't possibly make CAP theorem trade-offs. And what they were doing is they were finding a way to get stuck, when instead what you need someone at staff or principal level to do is to navigate ambiguity by making some trade-offs, building conviction, figuring out how to make a prototype that tests whether the assumptions they made are valuable and workable or not.
And so I think these ambiguous problems are still a really challenging one and, to me, a strong marker even today. Even with agents, you still have these complicated trade-offs that you have to make. And some of the trade-offs, to your point about understanding the business earlier, do come from not just understanding how the technology works, but the product implications. And to understand product implications, you have to understand what your users are actually trying to do. You have to understand the unit economics of the product that you're offering and the cost to service it and what you're charging. And so increasingly, I think you have to be quite broad to actually work these ambiguous problems.
But I do think decision-making has always been where staff and principal engineers have to make the impact. And that hasn't changed at all. I don't actually think agents are exceptionally good at making these trade-offs. I think the agents often don't even recognize the trade-offs because they're not in the context window, subtle implicit things that live outside of the software.
What about Eng 1, 2 careers? I just had a senior engineer ask me, saying, "AI clearly has gotten as good as an Eng 1 engineer writing code, because that's what we expected a lot of times from them." And then this person was saying that, "At some point it'll come to senior, but it's not there right now." But I still remember this one because obviously there's truth to it. How do you think of Eng 1, 2, new grads or people who are under senior? They don't yet have the battle scars or the expertise or the judgment that you can fully trust them, both in terms of hiring but also developing them.
So one of the things I try to stay away from is what I call artificial specificity. What's the difference between an Eng 1 and Eng 2? I think it's actually pretty hard to be super concrete about it. I think a lot of companies successfully say the difference is three years of work experience. And they say, "Is that equitable?" And I don't know if there's any super equitable way to evaluate Eng 1s versus Eng 2s. And again, a really bad Eng 1 you can identify, but the difference between a really good Eng 1 and Eng 2 often is time in role, in terms of just building more context over time. But it's not super, "They can do this or can't do this," especially with the coding harnesses. They can do anything, but actually deciding whether they should be doing it is increasingly tricky. And in the wrong situation, where they're getting a lot of direction, you can't even perceive their judgment in a lot of cases.
So I actually care less and less about some of these things, where I think just general things like time in level, stuff like that, is probably good enough for a lot of these. I do worry that getting incredibly detailed on these criteria is a bit of a fake victory, where it's like, "Oh, it's made it really precise." But often, you've been in calibration meetings where there are people who play to the rubric.
I remember at one company I worked at, to get to staff there was a mentoring requirement. And so we'd have people that literally went out and would talk to someone and then report them as a mentee, and they were collecting mentees. And then you talk to that person: "I didn't know why I was talking to them. They just talked to me once." And so this is where you get what you measure. So I really think for promotions there's always been a lot of judgment. Whenever you try to remove the judgment, I actually don't think you get good outcomes there either. And so having the rubric to help guide people, I think that's useful. But ultimately it's pretty subjective. And for a lot of these things, I think actually having constraints is more interesting than having really comprehensive rubrics that are used to make judgments.
So here's an example, something I did at Carta. At Carta there's a lot of cost pressure.
Cost pressure meaning to reduce costs, let's say, or manage costs.
Exactly. And so I actually had a ratio: can't have more than this number of staff engineers relative to the number. And it just was not possible to exceed that ratio. That was very annoying to a lot of folks, but I think it actually is more honest where the industry was 15 years ago. Fifteen years ago, I think there wasn't a role past senior in most companies. Senior really was the ultimate role for almost every company. And so as people have gotten further along in their careers and we have more people with 20, 30, 40 years of software experience, they have opened up more of these levels. But a lot of these levels are ultimately more time-in-role titles rather than really a definitive, obvious change in the capabilities of the individuals that are doing it.
I try not to worry quite as much about the details. And I do think having a hard block of one to eight or whatever it is is really helpful if what you're trying to do is manage costs, because when you're managing costs, what you have to be honest about is that you're managing costs. And so at Carta, that was the situation we were in. Now at Imprint, we're not trying to manage costs, we're trying to manage velocity and quality level. And so we haven't thought about that at all. But this is where I think part of leadership is knowing what you're doing, explaining how you're doing it, and then letting people accept the consequences of that.
And so because at Carta we were going to have a fixed ratio, that meant we had to start shifting hiring to be less senior, so we could actually have more of a pipeline of folks coming in and filling those levels. From that fix, a lot of things came from it, which worked really well. But if we had just tried to fix it without figuring out the other parts of the hiring earlier in the funnel, it wouldn't have worked. But I don't know. I'm less of a believer in a perfect description of things that are true than I was earlier in my career. Just so many exceptions to everything.
The thing with the rubrics and with expectations, it used to be, when both of us worked at Uber, you later worked at Stripe, it was pretty well understood what was expected of the role. And with the levels, yes, I agree that one, two, senior, these can be murky, but there was a pretty clear expectation at some point. For example, a senior, you could trust them. Typically, I think the industry started to understand at some point, for closer to a decade, a senior, you could trust them with a project. If they had junior engineers, they would mentor them. A staff, you could trust them to set the direction of a unit or multiple projects and so on.
And the Eng 1s and Eng 2s, they could execute well, coding without too many issues, took feedback. They were almost like apprentice levels. I think an Eng 2, it wasn't a serious promotion in terms of—it wasn't a destination. It was just they're growing. Some companies even had the expectation, as you were, that you need to grow to a certain level. But they were almost like apprentices, honestly. They were just getting to have good judgment and all these.
Now that we're all using AI harnesses a lot more, including for reviews, code reviews are slowly going away as well. We're doing less of what we used to do, which is coding and the very visible feedback between each other. How would you define what the role of a software engineer is? And is it different from before? Is it more ambitious? How do you think about it? Because it is changing. I feel it's getting more ambitious somehow, but I'm not sure I'm able to put my finger on exactly how, especially from a CTO perspective.
And is this the general role, or is this the level three, or—
No, this is the general software engineering role, because I feel each role has changed a lot. Maybe the staff engineers never did too much writing code at midsize and above companies because they were busy making sure we're building the right thing. So maybe it's hard to separate because now no one is writing too much code, and even the smallest roles, they will need to decide, "Are we doing the right thing? Am I doing it right?"
First, I think we're so early days in terms of where we're going, and there's certainly people you're going to talk to, they're like, "Hey, the software teams of the future are five people for any size company." And maybe that's true. Maybe we'll get there at some point. We're definitely not there right now.
I don't know if we're going to get there soon.
But there certainly are companies that have a small number of engineers spending just a staggering number of tokens in productive ways, and that's a pretty cool thing to see. Where is the role of software going? I think we're going to find out. And this is actually why it's such a fun time to be in the industry right now. The most fun thing is, I think our profession is changing in a much more extreme way than anything I've seen since I entered the industry. And for me, someone who's always been very oriented around the outcomes of the work I'm doing, it's just been the most fun time I've had.
But I've also spoken to folks who got basically joy from being in the zone, being in that flow state. And they have fallen out of love with software because what they loved was the actual art of doing it. And for them, a few folks, like one person I've spoken with recently, he's retiring. He's like, "I just don't enjoy writing software anymore."
And he's lost this flow state.
He's like, "I'm more productive," but they just don't enjoy it. And so I think the role is shifting, but I think it's shifting in a really interesting way. And to me, I've had this idea of software that the time to actually implement is approaching zero. These systems are still very slow in terms of, okay, let's say you can instantly implement stuff. You still have to make all these decisions. It still doesn't work in all these ways. It's going to scale poorly in all these ways, still going to break the user expectation, still going to have the wrong unit economics to actually be a profitable business. And so I do think even if software were literally free to write, it's still really, really, really hard.
And so today you'll see this: if software's free, then all of a sudden engineers are running ahead of the PRDs from product and running ahead of the high-fidelity mocks from design. And that's creating a ton of issues for design teams trying to work with this engineering model, and something we're still trying to figure out at Imprint, to be totally honest.
That also could create opportunities where, if an engineer also puts on a design hat or a business domain hat and now runs an experiment full on, too. And obviously now there's only so many experiments you can run without confusing your customers, which now goes back to strategy, someone having to say no. And also, who are you building with? If it's for people, they will have a finite amount of attention. Netflix has—I don't know how many new movies they bring out, but I know that when I open Netflix, they don't overload me. I mean, they do, but they don't show me all 8,000 movies. They figure it out. And so I wonder if software is becoming faster, as engineers we will need to discover the constraints of the other parts of the business, the domain, the customers, the reality, like you say, unit economics, how money can be made.
I think something I thought about a lot working at Carta—so Carta is a low-scale business, by which I mean there's not hundreds of thousands of requests per second coming in. There's no massive infrastructure problems there, but it's an incredibly detailed business. How cap tables work is really, really, really specific.
Yeah.
And it's very complicated, and there are tons of exceptions. And then doing—
It's important, right? It's a little thing, but you need to get that right.
Accuracy is really important from an auditability perspective. You're also working with law firms that have a real strong point of view about how everything should be done very particularly as well. And you could be in a legal battle between two founders who are arguing about how much equity was actually vested or something like that. Similarly, the other businesses Carta went into, like fund administration, also incredibly detail-oriented.
And so there are a lot of businesses out there where the domain context has always been the hard thing. And it's not super clear to me if businesses where the domain context is really the hard part have benefited that much yet from all the AI tools. It's still incredibly hard to do cap table management because of the details, the laws, the relationships with the law firms in terms of training the paralegals there. There's all these little pieces that I think—complex domain businesses.
Still have a lot of defensibility in this day and age. And I don't know how that's going to change. I think it will change. Every two to three years there's this kind of "product engineers are the new king, product engineers are the new superhero" meme that comes around. Now it's the AI engineer.
Now it's the AI engineer, who really are product engineers that use AI techniques, right? And so the question is—
Which is pretty much every product engineer these days.
100%. Right. So it's unclear. I've never really quite understood what AI engineer means. I think it means someone who uses harnesses and wants to get paid a lot of compensation, is what it means. But it probably means something else too that I've just missed the nuances on. But can you actually get individuals to hold all this context in their head?
And so even today, people used to talk about the front-end engineer who's actually a good designer or the designer who could actually write production front-end code. That person's always been rare. And now a designer with a little bit of taste can write production code, right? And vice versa, a front-end engineer with good design taste can do design. Why aren't we seeing more of that yet? And so I think that's a social constraint.
Well, it's a really good question because I was asking the same thing. I saw some data from Linear showing that PRs for designers and product managers are up. Someone messaged this to me, a fellow engineering manager, said, "Hang on, I've been asking around. We should be seeing more designers push to production, right? Or product managers." Asked around 10 companies. I asked around another 10, including the likes of Linear, Ramp, OpenAI. I have yet to hear, except for some exotic edge cases or just one example, of designers pushing production code. They will find bugs. They will identify them, but they will still get an engineer involved who confirms this approach works in the codebase, especially when we're talking about outside of prototyping. They do prototype more. But even we're not there.
I wonder if it's an accountability thing. Even though I can change a light bulb, I will still call the electrician when I need to do something a bit more on my house because it's electricity, right? And I can read the books. I wonder if it's this or specialization. I don't know.
One of the things I've been thinking a lot about is there's one worldview right now, which is if you have a great design system and you've actually used that design system, you don't need product designers anymore. That's a viewpoint that a few people have. I haven't found that to be true, though. Even with a good design system, understanding how all these different components fit together, understanding the missing components in your design system, figuring out when you have trade-offs across different screens, how you manage those trade-offs while presenting a unified experience for the user, these are still really hard things to do.
And so I know "Attention Is All You Need" is actually not about human attention at all. But I do think there's some dimension where people are still constrained on attention, and great designers and great product designers and great design system designers as well are all so constrained on the details of what they're doing to do it well that they just don't have a ton of time to also be branching out. Does that inherently have to be true? I don't think it inherently has to be true. And I do think in theory, if you have a great design system and you have a ton of consistency and a ton of linters in terms of "it has to be used this way," all of a sudden your design system folks, your product designers, could be just looking at the exceptions much more aggressively.
That is where we're trying to get to at Imprint, for what it's worth, where the base case is just done by engineering using the stock design system, and product design is more working on the really complicated versions of it, along with extending the design system with the design system folks. It's not working that way yet, though. It's still a little bit tricky. But as the folks deep in AI say, I kind of do assume this is a quote-unquote skill issue and that we will get to the other side of it. But it's still a skill issue for us. We're still figuring that out.
One thing related to this is you wrote "Revised Rules of Engineering Management," a really nice article. We'll link it in the show notes below. In one of the rules that I want to highlight, you wrote, I'm quoting you, "Durable high-ownership teams, so domain context, are even more important. One of my biggest lessons at Uber was that persistent, durable teams work magic by accumulating domain context, building a sense of camaraderie, and feeling an increasingly strong sense of ownership over an area as they continue to work on it." Now this was from Uber, like 2014, '15, so more than 10 years ago, and you're saying that you feel this is very relevant, right?
I think this is the only way to build scalable organizations. And it's interesting, though. I think this seems kind of obvious, but it's actually a bit of a non-consensus take, if you will. And so a lot of times you're at a company, say there's 50 engineers, and you have a really important project for a partner. That project's late. There's still the Mythical Man-Month thing. They're like, "Hey, we're moving 15 more people to work on this key project." And The Mythical Man-Month, I want to say, was written in like '80, '84.
'75. '75, and the second version, '95.
And that's a while ago. And so—
More than 50 years.
We know this doesn't work. And so first, when people do things like this, a lot of it's optics and creating the perception of doing everything possible.
Yes.
So that when you miss, it's not your fault. But actually, not only does it not work, it also destroys the thing that does work, which is creating little pods that are able to own areas.
And so when I joined Stripe, I took over the team Jorge was leading at the time, and he moved on to a different part of the company, I think to lead Connect. And so I had 30 people, and one manager was managing four or five of them. And then I was directly managing, say, 23, 24 people. The first thing I did was, "All right, what are some pods?" And figured out some areas I could put together who could own areas. And then once I saw some progress, I just stopped thinking about them entirely till something went really wrong, so I could focus on the bigger priorities for me, which is the problems you couldn't solve. This works so well.
And so at Uber, did the same thing. When I came into Imprint, did the same thing as well. Same at Calm. Carta was a really well-scaled organization by the time I joined it, so I didn't need to do the same there. I think this just works incredibly well. I think if you do this well, people actually do better work than you will from top-down dictation simply because they understand it really well, and it gives you the ability to actually put your attention on the highest leverage or most painful parts of the company.
And so, yeah, I think this is a strange non-consensus take, but it just works incredibly well. But optically it almost always looks wrong. And that's one of the reasons it's hard to do, where you have a partner launch and you're like, "Hey, I'm not going to move these folks. I'm going to let them stay on what they're doing. It's really important. Instead, the team that's on the critical path, we're going to work hard as hell together." We're going to burn to get through it. We're not going to pull everyone over, because if you don't do that—
Or what you might have done in the past, especially in ZIRP, where you had meetings, recruitment, all that. You would say, "For this team on a critical path, we remove them from all of the BS."
Totally.
The recruitment, the university trip, you don't need to attend all-hands and all these things, so you can be heads-down. Which actually, by the way, works so much better than adding people.
And also sometimes telling people, "Look, this will be a two-week push. Anyone who can commit to it, please do so, including putting extra hours, and in return you can take time off afterwards." Or you create incentives and you just treat them as adults, and you're saying, "Can you?" And oftentimes people will. I know in my life outside of work, sometimes you are okay taking a thing for this one thing, especially when you know that I will repay you in kind somehow.
And if you don't do that, the thing that happens is actually at a real growing company, there's a crisis every—
Yeah.
And that means if you rotate people to crisis, there's a crisis for everyone every day, and it creates this sense of devastation. And so I remember talking with someone at Stripe one or two years in. We were talking about this, and he's like, the way Stripe had run up to that point, his metaphor was it was like people on a wall trying to defend a castle, and they were always running to the place where they're about to get overwhelmed. But they just couldn't be everywhere because they were getting faster and faster running, but they still were not everywhere all at once.
And what we were trying to do was switch to this metaphor: we had many people, many places. And so the people working in one spot had to solve that spot, but it meant they didn't have to be everywhere. Those are the two trade-offs. But to be a successfully scaling company that's growing quickly, you do have to get to a mode where it's not the same people running everywhere at once, because those people just completely burn out over time. And there's a certain point where they just can't manage the coverage anymore, no matter how amazing they are. Although I think the question with AI is, is that point much further than it was before? But I'm not sure if it is. I still think their attention runs out before their ability to write code does.
Also recently, something that I was thinking about with the durable high-ownership teams is just an example with Slack, on how you have Google Chat, you have Mattermost, you have Microsoft Teams and Slack. Let's say these are four big competitors for the chat market, and it's pretty well understood what's there. And you have AI. You're a startup, you could also instruct an AI to copy Slack. And here's the interesting thing: if you do it, the AI would copy Slack's UI and/or behind open-source things. First of all, there's a lot of decisions that have never been documented that are behind the scenes that people at Slack will know, and it takes years, which if you build it, you'll figure it out if you're big enough, from the concurrency, deliverability, platforms, yada yada yada.
But taking that aside, Slack launched Slack agents, which was a super sweet integration with coding agents that do the work inside of Slack. It just works beautifully. It's been well thought out. Clearly this is a team that has the domain context, and they've clearly been working behind the scenes with teams, partners. I'm kind of like, "Hang on, why has none of the other competitors launched it? Why has Google Chat, who have amazing people as well?" These are all people that have the domain.
And I think this is where I'm starting, this one example, just starting to realize there is probably value in Slack having all these people who are working. I'm sure we would be horrified of how many people work there. I know it from Uber, like, "Why does Uber have this many people?" But they're working on this. They're not only able to operate what they have, but they keep reinventing what customers will want as the world changes. The other companies seem to have not responded, even though agents have been here for a while. They just haven't responded. Slack has not only responded, they did something that is objectively good, and now it's copyable. But how did you get there, right?
There's some paper about the size of the bureaucracy of managing the British Empire, and the size of bureaucracy, the number of bureaucrats, kept growing even as the territory getting managed kept shrinking. And there's this inherent thing where companies do just expand and create bureaucracy and create more overhead. So there's something there where sometimes companies have just lost the plot on their ability to actually move.
But my assumption, I think this is true for the Muse work for Meta, my impression is that when really interesting, good products come out, there's one or two highly empowered people that are connected to the founder, the CEO, the executive team, that are pushing it through and that are able to make painful trade-offs that are not well-liked. And those can be legal versus company trade-offs. Those can be product trade-offs. Those can be the sacred elements of how things have to be done, not doing it that way. And so my guess is for Slack there was some sponsor. And look, people hate overemphasizing executives, but I do think executive sponsorship to actually empower folks to do something difficult—
Imprint moved over to Linear three or four months ago from Jira. We did it in less than four weeks, and there were basically zero people that thought it was a good idea when I decided to do it. We still did it in less than four weeks, and now people are really glad we did. It solved a lot of problems we had about having centralized issue management so the agents could actually know where tasks were, pull them down, work them, actually do task unification, etc. We were having a really hard time managing workloads of tasks without that. But there were maybe three people out of 230 that thought it was a good idea before I did it. And that one I just pushed through. And I think now people are like, "Oh, that was a good idea," but they just didn't think it was.
And sometimes you do have to have someone who's willing to take the risk. And then the flip side is you then need to judge execs and other senior leaders on, are the risks they made us take good? And so I think three months into the Linear migration, no one talks about Jira anymore, but certainly no one's upset about the change. People are really glad with it. But conversely, if people had hated it, they should be rallying to get me fired, right? But I think a lot of times people don't get held to account for this. And so that's the flip side of that highly empowered executive sponsor that does make these great products. Sometimes we let them do incredibly boneheaded things and don't hold them to account.
Which is a great segue to talk about executives. You've been a CTO, which is a form of executive. Can you tell me what, in your interpretation, an executive is, and what is the career path today to get to that level from an engineer? Like, "Okay, I like the idea of executive. I like to be able to make decisions, including unpopular ones, helping people, helping companies." What was the old path, and what's the new path, if there's any?
So I think the old path, you're a software engineer for some number of years, could be pretty short, then you get into management, and then you climb through middle management, and then you get to the CTO role.
Management, engineering manager, senior director.
Exactly. And so I think there are a lot of problems with that. First, it means
That often the people that were selected for management initially weren't the most focused on the technical pieces of the work because they're trying to get promoted to management. And then the selection bias is really towards people that are able to operate well, which doesn't necessarily have a whole lot in common with the technology side. Often they happen to be really good at technology, but there wasn't a strong selection bias for it.
But you really do have to be a good bureaucrat to be an executive. And this is one of the interesting challenges where a lot of my work is incentive management. It's knowing, hey, if we roll out this sort of performance review, here's how it's going to fail. And so we just shouldn't do that. We should do this other thing instead. And so if you haven't done the reps of being a bureaucrat, you actually can't run a function because a lot of running a function is, in fact, being a bureaucrat.
So how did you learn to be a good bureaucrat?
I was a bureaucrat, right? Out of necessity. So at Uber, there weren't really rubrics. There wasn't an interview. We didn't have any structure to how we interviewed when I was there. So you had to make it all up yourself for your function. And again, we were doing tons of interviews, and each time we didn't give an offer out, I was so pained because I was doing so many interviews. So I had to figure that out. And then at Stripe, Stripe was a much more structurally cohesive company, and so we did have a lot of structured process. And then I got to work on a lot of that and learned a lot there.
You kind of got pulled into it. You picked up the pace.
100%. And a lot of the structure I thought wasn't working well at the point that I joined, to kind of work to try to iterate and improve little pieces of it because we were growing pretty quickly. And so really working these middle management jobs, I think, did educate me on how to be a good bureaucrat, which is part of being an effective executive, because you can just allocate processes that work and avoid processes that don't work, think about incentives that work and don't work, structure that does work and doesn't work.
Seeing Like a State—you're not allowed to be in Silicon Valley if you haven't read Seeing Like a State. But it's basically a book about how states see citizens and accomplishments and how that shapes how policy works. And it's the same thing: if you've never been a bureaucrat, you don't know how to see things as a bureaucrat. But also, if you can only be a bureaucrat, you don't know how anything actually works, don't know how software actually gets made, and so it's very damaging to you.
Am I getting this right that you're saying to be a good engineering executive, so VP of engineering or CTO, you want to be excellent at technology, knowing how it works, being hands-on if you need to be, or just be hands-on, and also be a good bureaucrat where you can manage, and you want both? And you're saying that if you go the middle management route, oftentimes it might happen that you either never were a good technologist or you stop becoming that by the time you get up there. So you're a great bureaucrat, but not technology. But if you just go from an engineer, great technologist, straight up here—let's say you're a startup tomorrow, congrats, you're a CTO—you will not be a good bureaucrat.
I think that's exactly right. And it's interesting where oftentimes I think line managers are better instinctual executives than middle managers because they're used to working the detail.
So they can make better calls for the team or the company.
Their judgment's just better on all these non-bureaucratic dimensions of the job, but then their judgment's terrible on the bureaucratic dimensions of the job. And so this is why I think being an executive is hard, and why it's a hard career path, is that in the 2010s in particular you had to climb these layers of middle management to learn this bureaucratic component of it. And that was the only way to get scale to actually understand that. Now those layers are kind of gone, but the interesting thing is the other half of the job is now really emphasized in terms of better decision-making, understanding the actual product, understanding the actual business, understanding how software is shipped, in a way where we kind of deemphasized that.
So I think people are like, is it easier? Is it harder? I don't think it's any easier or any harder. It's a lot harder to become a bureaucrat now, which, again, negative connotations, right? But learning how to actually operate a function is really difficult work. And as a C-level, you'll often end up running three or four functions, often one you don't know much about yourself.
So, for example, at Imprint, I was running the customer operations team for much of the last year, which is not something I'm incredibly detailed on. I have some exposure to it and had an amazing leader in place, but I've had to learn how to do that, and I've had to do that a number of times. At Stripe, I was managing support engineering, which is not something I've done personally myself. I was running data engineering, which is not something I've personally done myself. And so you learn how to run that way when you have to.
It's really hard to get the jobs to learn how to operate a function that way where you don't actually know the details of the work, but you get to be a much better, more detail-oriented leader for longer. And that's really where I think the best future executives are line managers, because middle management has historically selected for a totally different set of skills. How do you get both? It's incredibly hard now, and it was always hard in different ways.
And what's your take on middle management slowly dying? Do you agree?
It's definitely a hard time to be a middle manager right now, especially if you came up in the 2010s. You were kind of selected and trained in this empowerment model, more of this functional bureaucracy model. And now people are like, we don't want it and we don't need it. And so you've kind of risen to this pinnacle, but you can't get to the next step because these other skills that are more in-the-details execution were really deemphasized. And people have changed the rubric on you, which is super, super frustrating for folks who get trapped by that. That's definitely the challenge that folks are in now. And I don't know how this wave of execs are going to happen, but it's always been a hard role to get.
Well, and I do want to ask you, two people asked me for advice, and I'll share both of their stories, and you can decide if they're similar or different. One is a head of engineering who has about seven people reporting to them at a company and was thinking, I wonder about my future because I'm seeing about this middle management. The other one was a story from another head of engineering who has three EMs reporting to them and three ICs, reporting to the VP of engineering. They've been shifting to AI transformation and in the eng department also doing similar things. The VP of engineering is satisfied with their work, but they're under the VP of engineering, and this person is also wondering.
Both people are wondering, am I in a bit of a career trap? And they asked me to ask you: you have a lot of experience. What tactics would you be able to suggest to them to figure out what career path could be for them, to figure out what they might want, what is possible? And again, I think the tricky thing in both cases, both people have spent a bunch of time in middle management. It's not that they're not technical, but they had this path, and it was pretty clear where they would go next would be VP of engineering for both of them, but in their current company it will not happen, or unlikely.
It's a hard time in middle management, right? There's a lot more competition for these roles. And so I think what I found personally very rewarding is getting back into the details of writing software. And so when I left Carta, my team was 250, maybe something like that. When I joined Imprint, the team I was leading was, I think, 25. And so for me this is a unique moment to actually reskill.
If you're in management, I think right now either you have to get back into the details or the world is changing so radically that the profession you know will not be valued in three years. And I think you sort of have to decide, am I retiring now or am I reskilling? And I thought about that too. At Carta, I could just be done, right? Is this what I want? But it's not what I want, right? I really want to still be participating. And I really genuinely believe the way that we work has radically changed in the last year. It hasn't changed everywhere. There's a long tail that's going to take a long time to catch up, but in the places I work, it's changed a tremendous amount. And for me personally, it's a lot more fun.
So first I think there has to be some openness to taking a backwards step or a lateral step. Just what people want is very different, and you have to be open to getting a different look at it. And so I think going back into a line management position, for example, is actually much more educational than middle management. The tools for line management and writing software are so good. The tools for headcount planning or doing performance reviews are basically terrible, right? All LLM applications for these things are not good. So I do think you're missing a lot in middle management right now. I would try to find my way out of it.
But thinking a little bit about some folks that I hired at Imprint, one of them that comes to mind is Jasmine, who I worked with at Carta, came over, and she was hired as a VP, but she was a VP that was hiring or managing a team directly.
But she was a line manager.
She was a line manager, a really experienced line manager. Was running engineering at Mux some time ago, and she came in to line manage, and she's doing just a tremendous amount of software, which she would have never done at Carta because she could have never gotten time to do it. And at Mux, she was also way too busy running the function to actually be in the details of writing. Those people that are willing to actually go back and get their hands onto it are so valuable. And if that is who you are, you have to let people know, because otherwise people would think you would never take that opportunity. But I would hire more people like this all day long.
We were talking about in the 2010s, under ZIRP, how engineers got a little bit too used to, oh yeah, my company and manager is here to do what I would like to do. I would like to do platform, no products, even if the business needs it. Just not for me. I'd rather quit. And sometimes they quit, like they did. I wonder if there's a little bit of this with management where, again, I don't mean to imply too many things for these folks, but generally, I have this path. I thought I will be VP of engineering soon, and I want to get there. But is that what the industry wants? Is that what the other companies want? No. Right now they want really good, let's say, line managers.
Yep.
Similar to those engineers, the best engineers I know, they kind of put their ego on the back, and when the business said, "Look, we need product to build," they're like, "Look, I'm great at platform. You know what? Let me be the best product engineer I can be." I felt with you as well that you had 250 people, I assume compensation, equity, status, and you went down to a lesser-known startup with not as well-known a brand yet, or at the time, taking a lot smaller team, smaller responsibility. You left the ego at the door, if you know what I mean, and then you learned so much more.
Yeah. I think in a lot of ways it was just a much better situation for me. And so there are a lot of pieces that weren't, oh, this is a hit. That's not really how I thought about it. It was the perfect situation for me because what I knew I wanted was getting back into the details, relearning how to do software in the current age. That's what I believe is going to be the unlock to the longevity of my career. Also, though, it's just much more fun. And so to me it's just much more energizing. And I've always been someone who kind of got tricked into management, going back to the Digg story, where they just needed a manager, and so I said yes.
You've been tricked ever since.
And then they made me a director so it looked like they had a director in acquisition conversations, so it was more palatable. But I didn't know how to line manage, let alone direct. They kind of needed one on paper. Very kind of them, though. So for me this has been what I wanted, and this has been a unique opportunity. But I think for a lot of folks there is this sense the industry has betrayed them and that they were owed something that they've worked for. And the thing is, they're right. They have worked incredibly hard, but the rules have changed.
And so I think that at certain moments the world looks very stable, and our industry looks very stable. And I think in the 2010s we forgot that our industry is actually very volatile. Our industry's changed just a staggering amount. Every decade things are just radically different than they were in the decade before. And we've sort of lost sight of that. And I think that people that came up in an era of a lot of stability don't remember that sometimes.
And so I do think we have to be willing to take lateral moves. We have to be willing to be a little bit humble. But if doing the actual work that's the thing that matters to the company is humility, then we've sort of lost track of what we're doing here, because we really do just get paid to do things that are valuable to the companies, right? There's no entertainment orientation to why these companies employ us.
Yeah. What about the TLM role, tech lead manager? This was a role that we declared dead. You and me both declared dead as a dead end. This was back in, you know, the tech leads who were given a very small team to manage as well. And we were clear in ZIRP that it's just a career rut. And if your experience is great, I think you also made it clear that it's a trap. I think you have that article. What do you think about this role now?
It's funny. It's still a trap, but it's a highly demanded trap. And I think in many cases there's no alternative. So it's less a trap than simply what's there. And so I think in a lot of companies, management is getting flattened.
So we're starting to see a director, let's say, have a bunch of tech leads and an org of 30 people and a couple of tech leads here and there.
Yep. I think you're seeing two models. You're seeing that there are engineering managers, but they're managing much larger scopes. And so they're actually less hands-on with the individuals than they were before because they're managing, say, 12, 13, 14 people instead of four to eight. And then we're also seeing directors are then managing tech lead managers instead of managing these line managers. And different companies are trying different things. But I think this is where I see the industry kind of
Asking for folks right now. And I do say some of the best folks we've hired at Imprint have been folks that were in more pure management roles that came in but really wanted to be writing software and have just been doing an amazing job of doing a blend of both. It is hard to find those folks where it's almost like, interview them to be a great EM and then, do you just get frustrated you're not writing more software? Really, that person is what you want. But it's hard. Not how we've been interviewing them so far. I think we've just found a few great folks, but that is the role that we really want.
And again, it goes back to this thing where you want people who have the capability to do the bureaucratic parts, but don't want to. The capability but not the desire, because those are really the folks that I see super effective these days. They can use that when they need to, but they default to actually writing software, which is really where most of the value is.
As a closing topic, I wanted to ask you about how you write. You've been such a prolific writer. I really enjoy your blog. Whenever there's a new article, I check it out. There's also—you can subscribe. There's an email that I subscribe to and I actually get it in my inbox. How you approach writing and what you think about AI, and also what is your take on—there's a lot of AI writing going on, not just for social media on LinkedIn, we know that, but also for some authors who before AI, they didn't write with AI, but now they're starting to a little bit.
So, I've been writing for about 20 years online in my blog, and I founded Irrational Exuberance in 2007. And then, next year—it actually might have been May or June or something when I started writing it. So I'm getting close to this 20-year anniversary.
When I first started writing, I was simply trying to figure out how to network my way into a job, basically. But I did a ton of tutorials, just, here's how I did this. And this was a different era. Now the sort of thing that you just ask an AI, how do I do X, and it would write a great tutorial just for you, or maybe just do it for you directly.
Then as I started working at Uber, I started writing a little bit more of the things I've been learning in hyperscale. And then it really took off when I was writing at Stripe because I think Stripe was a place that both had a lot to learn, but a real love of writing, and that to me really energized my work. And then wrote my first book, the culmination of the things I had written at Stripe, An Elegant Puzzle, and then got a lot of momentum. I think a total of four books. Probably not writing another book for at least a decade. Maybe not ever.
But I think it's just been so transformational for me. I think so much of when people know me, which is not that very often really, but when people know me, it's exclusively from the books that I've written or the blog. And I really do believe writing is a unique way that we as individuals can actually change the shape of the industry. And I think when you've done that, there's little things that pop up.
So for you, the trimodal salary piece, right? And that is, first, a memorable name, that helps, but also I think was a really important observation that a lot of people didn't understand. I think you've changed a lot of people in how they approach compensation. And then Patrick McKenzie has his work on negotiation. So impactful, probably created so much wealth for people in the world.
I think he might try to track how much he's created on it, but I'm not sure if that's still true.
But for my books, I really think I've changed the industry. And I think Staff Engineer, notably, I think the Staff+ concept, that book created Staff+ as a concept, right? And there's other things that I think are less quotable but have mattered. And there's nothing I could have done in my work in any company that could have impacted the industry in this way.
This many people, and so positively. I remember both An Elegant Puzzle and Staff Engineer. I read it and I took a lot of guidance from it.
And that's just so powerful. And then I think about writing with AI, and I've spent tens of thousands of hours over 20 years creating this, writing this stuff. I'm not super craft-oriented in my writing, by which I mean I'm really trying to get ideas out more than I'm trying to write the best blog post. I've never been craft-oriented in terms of writing great blog posts. I'm just trying to get the ideas out, written somewhere.
I think what I like about your writing is there are very punchy ideas. It's not about the writing per se. I would say you're a decent writer, or it's enjoyable, but I don't like it because it's enjoyable to read. The idea comes and is there, and it makes me think, and I'm not thinking because of how you presented it. And again, the presentation is fine, but the idea is what brings me home.
Yeah, it's pretty minimal writing.
And some of your articles are short.
Yeah.
It's not even about the length. I'm way more wordy in my writing than you are, which you will know, but it's just my natural style. But in yours, it doesn't have to be. What I like about yours, you don't waste a breath in your writing when there's not much. And that's complete.
And then I think about that, all the stuff I put together, all the things I've tried to change people's minds on. And to me, writing with AI would just be like throwing that away. And so I think I've never done a newsletter trying to collect a huge number of folks. It's more, if you want to read what I'm writing, this is what it is. It's not my business like it is for you. But still, I've gathered some number of folks, and I think they read what I write, and that's really powerful.
And I know if I write something, often there'll be hundreds of companies talking about the idea there in the next week, and that's just such a privileged spot to be in. If I just wrote slop, I would just throw it away, and I could never get it back. I'll never spend another 20 years doing this, right? And so when I think about the advantages I have that I've carefully built over two decades, why would I throw that away?
And so if I don't have something to say, I just don't say something. I'm not trying to manufacture content or something like that. I think it would be such a devastating moment to me to throw that all away for nothing. If I was throwing it away for something, I could maybe get there, but I think it would just be an uncompensated loss if I were to write AI slop articles. I don't think anyone would read them, and then I would throw away this thing that I've spent so long putting together. And I assume that you feel the same way. Is that true?
Yeah. I feel that to me writing is how I think. And when I write the drafts, a lot of my social media activity, actually—and I've never talked about this before—but sometimes people ask me, what is your social media strategy? I have a large number of followers, which is just a byproduct of doing things consistently. And in my case, honestly, it's researching and thinking about—I almost do full-time or close to full-time thinking about stuff that's happening in the industry and talking with engineers. Usually what happens, I start writing a draft about something and I had this idea. I put it in a tweet, just in another window, and I keep writing it. And I later look back, it's gotten traction because I guess it was an interesting idea.
Sometimes it's controversial. Sometimes I actually test some out. I'm not quite sure, I'll just put it out there. I call these a little bit like brain farts, which is not entirely true because I am thinking about things, but I don't really care about them, and I'm actually more interested in the conversation. So sometimes it then distracts me when I go back, but it then makes my ideas form. And a lot of my ideas are about observations. Is this happening? Are you seeing this? Which just generates conversation. And to me that is the writing.
And what I found with AI, there was a time where it was popular to feed all your writing and have a bot that you can talk to, and some people wrote a Pragmatic Engineer bot, which I didn't like the idea to use, but anyway, I still tried it out. And it was almost hilarious because, for example, my opinions change all the time, my views change, and it would spit back something that I said years back. And I know that we talked about TLM. If you had a Will Larson bot, it would say TLM is a trap, and now we're talking about, well, actually there's way more to it.
I tell people, don't put your thinking—whenever I see someone write on LinkedIn, especially a fellow professional, or they message me, they often comment with their real voice, and I'm like, why don't you just write that? Imperfect, that's you. I like the you, and I don't like the sterile thing. And I also don't like how with AI it's got the same voice for millions of people using it, whereas millions of people talking have millions of different voices.
So this connects to something I think is really related to this, which I think is a lot of people believe to be successful they need to be writing content, but they don't really want to be writing content. And so they're like, "I really hate writing content, but to get promoted, to get my next job, I need to be writing content. How do I do that?" And it's like, obviously AI, right? And I think something I've been trying to push back on folks on for a while. I wrote a piece, I forget the exact name, but how to be a tech influencer or something like that.
We're going to link that post. It's such a—not catchy, but almost a clickbaity name. It is the single best post about how to be influential in tech. And I will link it. It's one of my favorite articles. I forward it to so many people. So, by the way, thank you for writing that.
Of course. And the reason I wrote it was these people were like, "Oh, I need to write 200 things, 300 things. I need to write every week. I need to do—" And it's like, well, if your goal is to be visible, what you need to do is write one thing that gets traction. Spend a lot of time on it. Test it. Figure that out. You don't actually need to write a ton of stuff. There's almost no value in writing a ton of stuff, unless you are running a business that is based on a subscription. If you're just trying to get your ideas out, if you're just trying to be a little bit more visible, if you're trying to show proof of your ability to write cogently, you can write two things. You can write three things. You can write one thing every two years. And you don't have to write a lot.
I think about Tanya Reilly, who has not written a lot on her blog, but when she writes something, it says something really interesting, right? And so she's not a volume writer, but she's been able to have really impactful pieces. I think Being Glue is the most obvious one that has really been transformative. And so I think even for people that do write a lot, people don't remember the vast majority of what they write. There's a few pieces that stick out. So I really think that if people reoriented around the real goal, which is maybe to have some visibility, one or two good things matters so much more than volume.
And so I wish people would stop creating volume if they don't actually—I think it doesn't serve their goals. And in fact, I've seen some candidates that I've interviewed over the last year where they have just AI slop content and a huge volume of it. And I don't know. I'm pretty conflicted on those candidates because it's like, hey, people aren't reading it. You keep doing it, and it's sort of like, why are you continuing to do something that doesn't serve any purpose because no one's responding to it? So it just feels like people who are running a recipe but aren't tasting the product. It's cargo culting, right? They don't actually know why they're doing it. And so to me it actually implies poor judgment rather than showing a sign of, oh, this is an ambitious person trying hard.
Yeah. In closing, what's a book that you would recommend, or books that you enjoyed reading?
Certainly, I think I would recommend my books to anyone who enjoys spending money on things.
Your books are all excellent, really.
Thank you. And certainly I think Crafting Engineering Strategy came out last year and I think is the newest one, but I think there's a lot of interesting ideas in there. For sure, I think Thinking in Systems: A Primer by Don Meadows is one that I think about a lot.
Read it based on your recommendation.
I think it's honestly the book that formed my view about management and operating in companies. I think that it's not really about operating companies, but it's a mental model you can use.
That book I think is incredibly, incredibly helpful. But really, any book that forces you to have a different perspective on something. And so I spent a decent amount of time thinking about typing a couple years ago, not typing on a keyboard, but how to have better type systems, which is not that relevant to my work on a day-to-day basis. And then there's another great book whose name I'm losing on TLA+ and using that.
Is that Hillel Wayne?
Yeah, exactly.
Yeah, he was on the podcast. He has the famous TLA+.
Fantastic book. And look, I'm not using a lot of TLA+ on a day-to-day basis, but I think just the ideas there are incredibly powerful in terms of using languages to increase the verification or verifiability of your software, which is one of the biggest challenges of our time, right? If all of our software is getting written by someone else, which is an agent, making verification cheap is really critical. And even if TLA+ is not the tool for that in many cases, thinking about that problem—and this is the same reason why typing is really interesting—really, really valuable. And so I think right now those are some useful ones.
Thank you, Will. It was awesome to have you and finally have this super nice conversation.
Likewise. Thank you for having me.
I've known Will for so many years, and it was great to sit down in person. And we did it at a time when there's a lot of change happening with traditional best practices when it comes both to engineering management as well as engineering career progression.
One really interesting topic was how far managers used to bend backwards to find interesting projects for their engineers. Will recalled how at Stripe some of his best engineers asked him for more interesting problems. Back then he already pushed back, saying that the only thing he has is suffering to offer, meaning not particularly glamorous problems, but that suffering is what's going to make the business succeed. But back in the 2010s and the early 2020s, lots of engineers saw this approach as not particularly great management behavior. And yet today it's kind of a default for career stability and promotions. As an engineer, especially at the senior-plus levels, you will want to help the business solve their most pressing problems. And this includes having empathy for those problems.
I also found it eye-opening how Will thinks about
What it takes to be a great CTO. He thinks of it as you need to be two things. One, a great standout technologist who knows what to build and how to build, as well as a bureaucrat who can run a whole function, including a function you might not be an expert on, like product or design. In the 2010s, lots of managers learned mostly about the bureaucracy part. And by the time they became CTOs, they lost the technology side. Today, with smaller teams and less middle management, many CTOs will be technologists first, which Will thinks might be better overall, and I have to agree with it.
Finally, I appreciated Will's advice to middle managers. Middle managers being engineering managers who have teams that other managers report to them. Will did not beat around the bush. Yes, today it's a hard time to be in middle management. The two paths he sees looking forward are either you go back to being closer to where software is being built, work on a smaller team or in a line manager role, or just accept that your expertise of mostly being a bureaucrat but not a technologist could not be valued that much in a few years' time.
Will himself took a step backwards when he left the 250-person engineering team at Carta to set up a 25-person team at Imprint, but he did it consciously to learn how to build with AI changing the engineering profession. I do find it inspiring that Will not only talks about the importance of learning the new rules of software engineering but also walks the walk.
Do check out the show notes on related The Pragmatic Engineer deep dives on zero interest rates and engineering and EM careers. If you enjoy the podcast, do subscribe in your favorite podcast player, and a special thank you if you also leave a rating on the show.
Thanks. And I'll see you in the next
Article published · Updated