Ramp's Geoff Charles on Finding the Next Bottleneck When Coding Is No Longer the Constraint
Lenny's Podcast
At Lenny and Friends Summit, Ramp CPO Geoff Charles argued that faster AI coding doesn't make product development fast. It moves the bottleneck somewhere else. Charles's claim is that the winning team is the one that finds each new bottleneck fastest, removes it, and moves on. With coding largely automated, they argued, the bottleneck has shifted onto product managers. The talk walked through five stages of Ramp's product development loop, the internal AI tools the company built for each, and what Charles thinks this means for the PM role.
The Racing Analogy: The Driver Is Only Part of the System
Charles opened with a personal story. They once entered the 24 Hours of Lemons, a race where participants buy a $500 car and drive it for 24 hours. They crashed hard. They were fine, but the car was not, and Charles admitted the crash was entirely their fault as the driver.
In professional racing, though, Charles said the driver is rarely the problem. By their account, the driver accounts for only about 15% of the impact on a race. The rest comes from how the driver interacts with the car, and how the car interacts with the team. Even the best drivers can only perform up to the level of the system around them. Winning comes from removing the bottlenecks around the driving, not just from driving faster.
The pit stop was their main example. In the 1950s, changing tires took 67 seconds. Today it takes 1.8. Charles pointed out that this didn't happen by asking mechanics to work 37 times harder. Teams found the bottlenecks and iterated through specialized roles, better technology, and more practice. They also cited Formula 1's pace of change: according to Charles, only about 10% of the roughly 16,000 parts in an F1 car carry over to the next year. They asked the audience to imagine 90% of their code changing every year, calling that "the bar."
In Charles's analogy, the pit-stop clock starts when the car comes in and stops when it leaves. For a product team, the clock starts when a customer feels pain and stops when the customer has a product that solves it. AI removes a bottleneck but also moves it. The team that finds the next one fastest wins.
Why the Bottleneck Has Shifted to PMs
Charles described the familiar lifecycle: identify, define, build (traditionally the longest phase), and improve. Engineers have automated much of their work, so coding is now much easier. Their advice to product managers was to "be more like engineers" and "be as lazy as engineers." With more code being produced, PMs now have more to define, collaborate on, coordinate, ship, test, and release. Charles argued that PMs therefore need to invest in their own "factory." The question becomes how quickly a team can identify the bottleneck and redesign the factory around it.
The rest of the talk covered five steps and how Ramp has iterated on and automated each.
Step 1: Identify, and Sifting Through Customer Pain
Customer pain lives in many places. Charles listed Gong, Zendesk, LogRocket, surveys, and the occasional angry email to an executive. Context is spread across silos and competing opinions, so the first bottleneck is separating signal from noise.
Charles noted that LLMs alone don't solve this. A one-million-token context window, they said, holds less than 0.5% of Ramp's Gong transcripts.
Ramp started small with a "hate channel" that posts customer complaints every day. Charles showed real quotes and joked that it "got out of hand really quickly." The team then rebuilt from scratch and created a customer insight agent. It pulls from all of the company's data sources using traditional ETL and pipelines, uses vector search, and clusters feedback around shared context. It also understands Ramp's product, teams, and features, and the rest of the organization can access it.
Next, the team experimented with ways to put the insights in as many hands as possible: a Slack agent that answers questions, an HTML dashboard, and Charles's favorite, a "hate podcast," in which listeners hear about 100 customers describing how the product is broken. Charles stressed that this doesn't replace talking to customers. Because the data is traceable, it tells you exactly which customers to talk to.
Step 2: Define, with an Agent Called Glass
Charles criticized AI tools that open with "What do you want to build?" Faced with a superintelligent system asking that, users just stare. They questioned whether that is the right starting point, and argued that AI becomes actionable when it is connected to a company's own systems.
Ramp built its own agent, Glass, and gave it the context it needs. It connects to data in Snowflake and to user research, so it can pin down the job customers need done using both qualitative and quantitative evidence. It also understands Ramp's product strategy, how Ramp writes specs, and the codebase. Charles said this means a PM no longer has to ask an engineer "Is this possible? Would this break something? What am I missing?" The PM asks the AI instead, which in their words becomes the PM's "tech lead."
Because Glass knows Ramp's product principles, design system, and codebase, it can produce a prototype that works inside the actual product. Charles argued that engineers don't want any single artifact from a PM. On its own, a prototype is "useless," and so is a long spec. What engineers want is the combination: qualitative and quantitative evidence that the problem is real, requirements they can hand to their coding agents, and a prototype to draw inspiration from. Charles called this bundle "the next contract" between product and engineering.
Step 3: Build, and the Chain of Bottlenecks Inside It
Charles acknowledged that saying building is no longer the bottleneck is an oversimplification. Removing one bottleneck exposes the next, and within the build phase they described three in a row.
Coding. Charles said everyone should have a coding agent by now. Ramp built its own, called Inspect, because it wanted a strong harness and an agent that truly understood its codebase. Inspect runs inside Slack, is fully provisioned, can run in under five seconds, and returns a deployed preview so the PM can interact with what's being built. Charles reported that Inspect has had a million sessions, that 75% of Ramp's PRs are built by it, and that about a thousand PRs in the past month came from non-engineers. Their caveat: coding agents work well only on top of a strong architecture and codebase, and that foundation determines how much of the company can use AI.
Code review. More code means engineers spend their time combing through it. Ramp built Review Buddy, which understands the codebase, quality checks, and security concerns, and finds the right human reviewer to loop in. It also sees the prompts that produced the code, so it can audit how AI was used in the first place, which grounds review in visible context. Charles said 93% of PRs are now handled automatically by Review Buddy. That leaves the strongest engineers to focus on the 7% that raise real questions about correctness or risk.
Testing. Charles described the old PM routine: spin up QA environments, tweak fixtures, click through screens, and check that the engineer didn't skip a favorite feature. Ramp automated this with Testo, a browser-based QA agent. It spins up the product in 100 different combinations based on production data and uses the product like a user would. Given an instruction such as "pay an invoice, but amortize it," it clicks through and reports blocking issues like bugs. Charles said it also gives "extremely thoughtful" design and quality feedback, such as flagging confusing flows or violations of the design system. That feedback goes back into the development loop. Charles reported that Testo caught 425 bugs in the last 30 days, bugs they would rather Ramp find than customers.
Step 4: Coordinate When Human Attention Runs Out
Once a company ships this much, Charles said, the bottleneck becomes human attention. PMs are flooded with notifications, and coordination becomes the job. Too much process slows builders down. Too little leaves everyone confused.
Their principle was that "every question is an API." Ramp built what Charles called "gadgets," which understand the intent behind a question and connect it to the formal record: the roadmap, specs, and customer calls in Notion, plus Slack and Linear tickets. For an agent to be useful, it has to be able to read the organization, which means the organization has to be legible to agents.
Charles gave several examples. Status questions like "Are we on track?" are answered by an agent that sees the full picture, knows who owns next steps, and cites evidence. The agent also updates the roadmap, posts status updates, and pings people who are late on deliverables. Sales questions such as what a feature does, how to sell it, whether it's available in Brazil, what it's for, and what it costs are handled by Gadget, which routes anything it doesn't understand to the right person. For launches, because Gadget understands the product, it can write the help center article and help draft the blog post and customer email. Charles said 85% of questions directed at PMs are now fully answered by AI, and the answers to the rest are fed back into the system.
Step 5: Improve, and Automating the Small Loops
Charles argued that PMs naturally drift toward small, reactive work: tasks that are easy, visible, and high-confidence and give a dopamine hit. They called this "a huge mistake." Citing an earlier talk by Claire on ambition, they said focusing on small things isn't ambitious, and PMs should automate their way out of those loops.
At Ramp, AI runs the entire loop for most small issues. It routes the issue to the right team, matches it against the Linear backlog, dedupes, counts, and ranks. It then drafts a plan and checks with a human in Slack ("Is this ready to go? I'm ready to code"), writes the code, runs it through testing and CI/CD, and updates the knowledge base on launch. Humans are in the loop "only just barely," across thousands of small loops, so they can focus on big ones. Charles reported that thanks to these autonomous loops, 60% of UX issues found by customers, salespeople, or Ramp itself are fixed within 24 hours.
Three Questions: Speed, Budget, and the Future of Product
Charles closed by addressing three questions they expected from the audience.
How do you know you're moving faster, with quality? Charles admitted product velocity is harder to measure than a lap time, and some things don't quantify easily. They told a story about Niki Lauda in 1974: after a lap in a Ferrari, Lauda told Enzo Ferrari the car drove, handled, and braked poorly, then went to the engineer and fixed it. Charles said Ramp's culture is similar. Their answer was to hire a "driver" who knows what speed looks like, put that person in charge, and let them challenge you, even if that is hard for some leaders.
What about limited budgets, whether engineers or tokens? Charles said that in 2006 Audi had a slow car and asked how it could win without being faster. Its answer was fuel efficiency: fewer stops and more time on track, which by Charles's account brought three straight wins. Constraints force a choice of one dimension in which to be world-class. Charles's advice was to "embrace your constraints, but not your bottlenecks": pick the one bottleneck that could 10x your company and start there.
Are PMs automating their own jobs? Charles said it is "amazing" for PMs to be out of the loop on the many things engineers can drive, because it creates leverage, and PMs shouldn't fear that. They see the role evolving along three tracks. The first is a technical track, which Charles believes the industry underinvests in: PMs who build the factory itself, identifying bottlenecks and removing drag, shipping products that help the company or its AI build for customers. The second is the tastemaker, the "driver," arguably the most important and most visible role: the person holding the steering wheel, doing what AI can't, and setting the bar for great product. The third is the GM: Charles expects PMs to expand into marketing, sales, growth, and operations, owning business outcomes and leading whole functions.
The Factory Is Always Last Season's
Charles summarized three takeaways. First, speed comes from finding and removing the bottleneck in your build process, not just coding faster, and every team has one. Second, solving one bottleneck exposes the next, and the job is to move through them as quickly as possible. Third, product leaders should obsess a bit less about the product they deliver and more about the factory that builds it.
They added that everything shown in the talk is already outdated. As in F1, teams copy each other, and Charles invited the audience to copy Ramp and surpass it, assuming some already had. Quoting Enzo Ferrari, "The best Ferrari ever built is the next one," they said the best product a team will build is its next launch, and that starts in the software factory it builds today. Charles closed by calling on the community to share more openly how they build, so everyone can deliver more for customers.
Hello everyone. First off, it's been absolutely amazing to see the speakers before me. I want to give one more round of applause for the incredible amount of content and preparation.
I'm inspired. I've rewritten my speech three times. I learned how to do storytelling. And so here's a story for you. I learned from Ron about velocity. We love speed. I love speed. I love speed so much that I once signed up to race cars at Le Mans. I'm just kidding. Lemons. 24 Hours of Lemons. It's a race where you buy a $500 car and you race around for 24 hours hoping to win.
Unfortunately for me, my car did not make it. I crashed. I crashed hard. I was okay. The car unfortunately was not. Now, this crash was entirely my fault as the driver.
But in real professional racing, the driver is rarely the problem. In fact, the driver is only 15% of the impact on the race. The real impact is the interaction between the driver and the car, between the car and the team. In other words, winning isn't just about driving faster. The best drivers in the world can only perform up to the level of the system. Winning the race is about removing the bottlenecks around the driving.
That's the key point I want to drive today. Take for example the pit stop. In the 1950s, it took 67 seconds to change tires. Today, 1.8 seconds. How did 67 seconds turn into 1.8? Certainly not by asking the mechanic to work 37 times harder. All you leaders, you know what I'm talking about. They found the bottleneck and they iterated to remove them. Specialized functions, better technology, more practice.
And F1 is relentless. 90% of the parts of a car change every year. Only 10% of the 16,000 parts of an F1 vehicle carry over to the next year. Now, I want you to also think about what if 90% of your code changed every year? Because that is the bar.
Why am I telling you all this? We're in a race. Speed is everything. Execution is everything. And we're in a race to build with AI.
For the race car, it starts when the car comes in for new tires and the clock stops when it leaves. For us, the clock starts when there's pain and it leaves when the customer has a product that solves it. AI simply removes the bottleneck, but moves it. And the best team, the winning team, is the team that can find the bottleneck faster, remove it, and move on to the next one.
Now, we all know the product development life cycle. We identify, we define, we spend a lot of time building, and then we improve. Now, the engineers, fantastic engineers, they've been able to automate a lot of their work and coding is now much easier.
So, here's a tip. Be more like engineers. Be as lazy as engineers. Because the reality is that the bottleneck has now shifted to us. We need more things to define. We need more things to collaborate. We need more things to coordinate, more things to ship, more things to test, more things to release. The bottleneck has shifted to us and so we need to invest in our own factory. The speed of this loop determines the outcome of the race. And the question becomes, how quickly can you identify the bottleneck and redesign the factory around it?
What I'm going to show you today are five of these steps and how Ramp has gone far in iterating and automating through it.
The first step is identify. Identify the problem. Identify the pain, right? The issue is that that pain lives somewhere and there's a lot of pain, a lot of data. Gong, Zendesk, LogRocket, a survey, a very angry email to an executive. The problem is that context lives all over the place. Too many silos, too many opinions. And so the first bottleneck to identify is how to sift through the noise.
And LLMs are great, but a 1 million token window, that's less than 0.5% of Gong transcripts at Ramp. So we start small. We created a hate channel and every day it posts all the lovely things that our customers say. And yes, these are real quotes. That got out of hand really quickly. Yes, I know. I am not proud of it. No one's perfect.
And so we had to redesign from scratch. We built an actual customer insight agent. That agent pulls from all the data sources at a company. It uses traditional ETL and pipelines. It uses actual vector search. It has clustering around the same context, understands our product, and understands our teams, and understands the features that we have, and it's accessible to the rest of the organization.
Once we had this working, we asked ourselves, "How do we get this to as many hands as possible?" And so we again experimented. A Slack agent where you can ask questions, a nice HTML dashboard that people can log in and read. My favorite one was the podcast. The hate podcast. Great way to start your day. You listen in, 100 customers yelling at you about all the ways your product is broken. The concept is simple, right? Get people access to this data as fast as possible. That doesn't mean don't talk to customers, it means you actually identify exactly which customers to talk to. Because the data is traceable.
Okay, once you've found what to focus on, the next step is to define your product. How do you go from concept to a fully scoped idea? And so, we turn to AI, right? And the AIs all kind of look the same. They ask you, "What do you want to build?" And you stare. You have this super intelligent being asking you what you want to build. Is that really the best question to ask? Is that really the best starting point? Right? The goal with AI is to connect it to your systems. That is how you're going to make AI extremely, extremely actionable.
And so, we did that. We built our own AI agent called Glass, and we gave it all the context it needs to do the work. It connects to all our systems so that it understands both the data with Snowflake, it understands our user research, and it can nail exactly the job to be done that our customers are asking for. That gives us specificity with both qualitative and quantitative data.
Glass also understands our product strategy. It understands how we define product specs. It understands our code base. And so, instead of a PM bothering an engineer asking them, "Is this possible? Would this break something? What am I missing?" you ask AI. AI can be your best thought partner. So, now AI is your tech lead.
And finally, you can build a prototype. Because it understands our product principles, it understands our design system, it understands our code base, you can actually define and build a prototype that actually works within your product.
Your engineering team don't want any of these individual things. They don't want your prototype. It's useless. They don't want your long spec. That's useless. They want the combination of qualitative and quantitative data to convince them that this is a problem, the actual requirements that they can use for their coding agents, and a prototype that they can actually get inspired by. That is the next contract.
So, now you have a strong foundation for which to go. The next step is to build. I mentioned that building is no longer the bottleneck. Obviously, it's a bit more nuanced than that. When we focus on one bottleneck, the next one appears.
The first bottleneck was coding. Everyone should have a coding agent at this point. We built Inspect. Inspect is our own coding agent, and the reason why we built it ourselves was because we wanted to have a strong harness, and we wanted to make sure that our coding agent really understood our codebase. It works inside of Slack. It's fully provisioned. It can run under 5 seconds, and it can return an actual product in a deployed preview, so that the actual product manager can actually use and interact with the code that's being built. Anyone at Ramp can now code.
Inspect has a million sessions. 75% of our PRs are actually built by Inspect, and a thousand of those PRs over the last month were submitted by a non-engineer. One tip here, though, is that coding agents are really good when you have a strong architecture and a strong codebase. This expands how much of your company can actually use AI.
You now are shipping a lot of code. The next step is code reviews. Becomes the next bottleneck. Engineers are just constantly combing through a ton of code. And so, we focused on that. We built Review Buddy. Review Buddy fully understands our codebase, fully understands our actual quality checks, our security concerns. It finds the right human reviewer to loop in, and it also understands the context, all the prompts that led you to build that code, so it can actually audit how you used AI in the first place. So, the review and the collaboration is now grounded in visible context.
93% of our PRs are now automatically handled by Review Buddy, so that the highest quality engineers are spending time on the 7% of PRs that truly matter. That have a lot of questions around are we building this correctly? Are there any risks?
The next bottleneck is testing. Previously PMs, I'm sure you're familiar with this, we spin up QA environments, we try to tweak different fixtures, we click through the screens, we make sure that the engineer didn't skip out on that one feature we really liked. We automated that. We built Testo. Testo is a browser-based QA agent. It spins up the product in 100 different combinations based on the actual production data that we have access to, and it runs in the product like a user.
We give it instructions. We say, "Hey, pay an invoice, but amortize it." And it clicks around and actually gives us feedback, both blocking feedback, bugs, issues, blockers, and honestly, extremely thoughtful design and quality feedback around things that were confusing, things that break our design system or break our design language. And that goes back into the product development life cycle.
Testo in the last 30 days has caught 425 bugs. I think these are bugs better caught by us than our customers. And they send it back through the loop to fix. So, now you're shipping a lot of product.
The next step is to coordinate. How do you make sure that everyone in your company is in the loop? Things are moving so fast, you're shipping so many different things. The next bottleneck becomes human attention. We've all been there. PMs being inundated with notifications. And so, our job becomes coordination. I see a few people coordinating right now.
Too much process, right? And it slows down the builders. Too little process and everyone is confused. And so, human attention becomes that bottleneck. How do you solve this? Well, every question is an API. Every question is an API. And you architect a system. We built Gadget.
A gadget essentially understands the intent of these questions and connects it to the formal record, the road map in Notion, the specs in Notion, the customer calls in Notion, Slack, Linear, our tickets. The principle here is simple. For you to empower an agent, the agent needs to be able to understand and read the organization. And the organization needs to be legible to your agents.
I'll give you an example. A question is like, "What's the status of this project? What's the status of this launch? Are we on track?" An agent handles that. It understands the full picture. It understands who owns the next steps and answers with evidence. But not only that, the agent updates our road map. The agent gives status updates. The agent pings people who are late on their deliverables. There's a lot of fun ways you can actually scale.
Or another question, sales, asking questions about, "What is this feature? How do I sell it? Is it available in Brazil? What's the use case? What's the price?" All of these things are fully understood in Gadget. If it does not understand, it routes it to the right person.
But we can take a step further. A launch. Gadget understands the product and so it can write the help center article. It can help write the blog post. It can help write the customer email. So on and so forth. Again, start one place and you'll see how far it expands.
85% of questions that are being asked to PMs now are fully answered with AI. And those that are not are answered and fed back into the system. So you're shipping a lot of products now. Everyone's fully aware and coordinated. Everything is perfect.
But that's just the start, right? You are constantly improving on the product and the loop starts again. Now, PMs, we naturally gravitate towards the small reactive things, the things that we have high confidence in, the things that are easy, that are visible, the dopamine hit, oh we've solved something. I think that is a huge mistake. Claire, I think, had a great talk around how to be ambitious. You're not ambitious because you're focusing on small things. You need to automate your way out of these small loops. And so we did just that.
For most small things, an AI fully runs the loop. It routes it to the right team. It matches it with your backlog in Linear. It dedupes. It counts. It ranks. It runs the plan. It has a human in the loop through Slack around, "Hey, is this ready to go? I'm ready to code." It writes the code. It runs through test and CI/CD just like we talked about. And when it launches, it runs through the knowledge base.
These are small loops. And humans are in the loop only just barely. Thousands of little loops with thousands of little things, so you can focus on the big ones. Thanks to these autonomous loops, 60% of UX issues that are identified by either a customer or a salesperson or ourselves are fixed within 24 hours.
So at this page, you're probably asking yourselves three questions. The first question is, okay, how do we know we're moving faster? Right? How do we know we're moving faster with quality? After all, we talk about product velocity, how do you measure it? It's certainly harder than a lap time at a racetrack.
Well, some things are not easily quantifiable. Niki Lauda in 1974 took a lap with a Ferrari car and told Enzo, "This car is a piece of... It drives poorly. It handles poorly. It brakes poorly." Got out of the car, went to the engineer, fixed it. That is not dissimilar to the culture at Ramp. I'll start there.
But the point I'm trying to make here is that you will know if you're moving fast enough if you actually hire a driver that knows what speed looks like. And you put them in charge and you actually allow them to challenge you. That might be hard for some leaders, but that's what we need to do to be challenged by people who know what great speed looks like.
The second question might be, what about our budget? We have limited resources. We have limited resources in terms of engineers or simply with tokens. How do we compete with companies with no constraints?
In 2006, Audi had a bad car, a very slow car. And so they asked themselves, how can we win if we can't go faster than anyone else? Their answer? Fuel efficiency. Fewer stops, more time on the track. And they won three years in a row.
So constraints force you to choose a dimension in which you can be world-class. So embrace your constraints, but not your bottlenecks. Find the one thing that you're really, really good at. Find that one bottleneck that you think can 10x your company and just start there.
And the last question, something that we've talked a lot about today, is the future role of product. Are we automating our jobs? What's going to happen if these loops are running without PMs?
Well, the first thing I'll say is I think it's amazing for product managers to be out of the loop for a lot of things that engineers can drive. This helps us get a ton of leverage. And I don't think we should be scared about that.
Our jobs, though, I think will evolve in three ways. There are three tracks that I see for product managers. The first track is the technical track and something that I don't think we are investing in nearly enough. It's the PM that's building the actual factory, the technical product manager that is identifying bottlenecks and removing the drag in your organization. They're not shipping products for the customer, they're shipping the product that helps you build the product for the customer or helps AI do so.
The second role, arguably one of the most important roles, certainly the one that gets the most limelight, is a tastemaker. It's the driver. Ultimately, you need someone who's holding the steering wheel and doing things that AI cannot do and holding the bar for what great product, great taste
looks like. And finally, the GM. I think PMs will absolutely evolve and expand their role to not just the product organization, but the marketing organization, the sales organization, the growth organizations, the operational organizations. PMs will become GMs and they will own the actual business outcome and lead the entire function.
So, we walked through a lot. I'll recap three main takeaways. The first is that speed is not about just coding faster, it's identifying and removing the bottleneck in your build process. And everyone has one. Scientifically, there is always one.
Two, as you actually solve that first bottleneck, the next one appears. And your job is to move through them as fast as possible.
And lastly, us as product leaders, we need to obsess a little bit less about the product that we're delivering and a little bit more about the factory that helps us build products faster.
Now, I just walked through a lot here. A lot of great things that we built. And I did so because the reality is that everything you've seen here is outdated. Just like F1, teams will copy each other. I want you to copy us. I want you to actually hopefully surpass many of the things you see today. And I'm sure people here have gone much further. The reality is that it's all about the next season.
And so, I'll leave with this. Enzo Ferrari said, "The best Ferrari ever built is the next one." The best product you'll ever build is the next product that you'll launch. That begins in the factory. That begins in the software factory that you build today.
I hope you copy us. I hope we get to copy you. Let's share more together as a community around how we are building these products so we can deliver more for our customers. And I hope you all stay safe out there. Thank you.
[applause]
Article published · Updated