Why Rust Feels Different: Alice Ryhl on Reliability, Ownership, and How the Language Is Governed
The Pragmatic Engineer
Rust has a reputation as one of the most admired programming languages and also one of the hardest to learn. In this episode of The Pragmatic Engineer Podcast, Alice Ryhl talks through what makes it distinctive. Ryhl is a software engineer on Google's Android Rust team, a core maintainer of Tokio (the most widely used async runtime in Rust), and a Rust language team adviser. The central question is why developers stick with a language that has such a steep learning curve. Ryhl's answer is that Rust is built around catching the mistakes programmers keep making, and that it combines this reliability with the ability to run in low-level contexts like the Linux kernel. The conversation also covers how Rust is governed without a single leader, how features move from proposal to stable release, how Rust is progressing inside Linux, and where AI tools fit into Ryhl's work.
From Minecraft mods to Tokio maintainer
Ryhl's path into programming began with Minecraft. They wanted to write their own mod, so they learned Java. The modding didn't get very far, but the programming stuck. That happened just before high school. After high school, Ryhl worked full-time as a software engineer for a year, then did a three-year bachelor's degree and a two-year master's. They joined Google part-time when the master's program started and switched to full-time after finishing.
Ryhl's involvement with Rust started long before Google. During school, they spent a great deal of time on the Rust users forum answering questions, and estimates they have written around 10,000 posts there. Later they became active in chat servers, including the Discord server for Tokio. They kept answering questions there too, and when the same questions came up repeatedly, they fixed the documentation. That documentation work is how they came to maintain Tokio.
Ryhl describes Tokio as an asynchronous runtime, roughly "the standard library for Rust when you're using async." Their comparison is JavaScript in the browser. The browser has an event loop holding tasks that are ready to run, and it executes them one after another. With await, one task can pause while another runs on the same thread. Tokio also keeps a queue of runnable work and runs it. The difference Ryhl highlights is that Tokio can be multi-threaded, with several queues running in parallel.
"When it compiles, it works"
Asked what drew them to Rust, Ryhl points to a feeling many Rust users describe: once the code compiles, it works. They stress that this belongs in quotes. Bugs obviously remain possible, and the saying is not literally true. Still, Ryhl thinks there is a real reason people say it.
The starting point is a type system, which Ryhl considers a prerequisite for any language to feel this way. But Ryhl argues Rust does better than many other typed languages. The classic example is null in Java. Ryhl recalls that Tony Hoare, who invented null, called it his "billion dollar mistake," because any function call could potentially crash the program. In Rust, a value that might be absent has to be marked explicitly as such, and the compiler then forces you to check before using it. You cannot forget the check the way you can in Java.
On Rust's growth, the host notes that the language began around 2006 and seems to be appearing in more and more places, citing Oxide as a company that went all-in on Rust. Ryhl mentions, with some amusement, having recently checked the TIOBE index and found that Rust had overtaken PHP and Go there.
The pitch for TypeScript developers
Ryhl says the case for Rust depends heavily on which language you are coming from. For TypeScript developers, Ryhl sees Rust as a backend language, a good fit for backends and API servers, and would not use it for the frontend. The argument is about sleep: you don't want to be woken at night because your web server has problems, so you want a language with as few bugs as possible. Zero would be ideal but is hard to achieve. "As few bugs as possible" is how Ryhl sums up the idea of Rust.
Beyond null handling, which Rust does through its enum type (Ryhl acknowledges TypeScript can do something similar and is fairly good on that front), Ryhl highlights several features.
Error handling. Rust largely doesn't use exceptions. A function returns the error as a value, using an enum that is either the result or the error. The ? operator, placed after a call, means "if this fails, return the error." Ryhl's point is that this is very easy to use but not invisible the way exceptions are. It takes one character, not zero. If you forget the ?, you get a compilation error. You can also handle the error manually. The goal is to eliminate the situation where an implicit error condition you never considered takes down your server.
Documentation tests. A comment becomes a documentation comment when it starts with three slashes instead of two. Examples written in that documentation are turned into tests. If you change the underlying code and break an example, the test fails, so documentation examples cannot silently drift out of date. The host calls this the first real solution they have heard to a problem developers have complained about for over a decade.
Initialization. Rust will not let you use a variable before a value has been assigned to it.
Parsing input. Ryhl describes the Serde library. You define a struct with typed fields and declare that it should be parseable from, for example, JSON. Serde is a macro that generates code checking that the JSON has the right shape: all required fields present, with the correct types. It generates native code specific to that JSON shape, which Ryhl says makes the checks very efficient.
Exhaustive matching. Rust's equivalent of a switch statement is match. You write a branch for each enum variant, and missing one is a compiler error. A catch-all case is allowed, but most of the time you list every case. When a new variant is added later, the compiler points to every place that needs updating.
That leads to what Ryhl sees as a broader reliability benefit: refactoring. Ryhl describes changing something like a return type and then fixing compiler errors "until the compiler stops shouting," at which point every affected place has been updated. The host remarks that the language designers seem to have thought hard about what typically goes wrong in other languages and tried to fix it in the compiler. Ryhl agrees that the pattern shows up everywhere: if you get something wrong, either the code won't compile or at least there is a lint for it.
The pitch for C++ developers: memory safety
Ryhl thinks the argument is even stronger for C++ developers. In JavaScript, a mistake might take down your server, which is bad enough. In C++, Ryhl says, a mistake is a security vulnerability most of the time. Something as trivial as an off-by-one on an array can become exploitable, and "this just keeps happening."
Ryhl defines memory safety as the property that no matter how stupid the code you write is, it will not have a certain class of bugs. That class is the kind that usually leads to security vulnerabilities, such as reading past the end of an array into random memory, or destroying an object and using it afterward so that you are touching memory now belonging to some other object.
Ryhl gives a classic kernel exploitation example. Suppose an object is freed and its memory reused for a task_struct, which essentially represents a process and has a user ID field. Code writing zeros to memory is common. If a stale reference causes a zero to be written into that user ID field, the process is now root. From there an attacker can take over the machine, and as Ryhl puts it, once you're root, you've lost.
The host summarizes the C++ pitch as memory safety eliminating an entire class of bugs that turn into some of the most serious threats software faces. Ryhl adds that this comes on top of all the reliability features already discussed for TypeScript developers.
Garbage collection and why Rust avoids it
For Ryhl, Rust's uniqueness lies in the combination: it is reliable, yet it has no garbage collector and is usable in low-level contexts like the Linux kernel or firmware.
Ryhl explains the difference this way. In Java or C#, a piece of code periodically checks all your objects, determines which are no longer used, and cleans them up. In Rust or C++, a value is cleaned up when it goes out of scope. For embedded use cases, a periodic collector may be impossible or unacceptable. Ryhl argues it can also matter on the backend: if a request arrives while the collector is scanning all objects, you get a latency spike and the response takes much longer.
As for what Rust code looks like, Ryhl says it resembles many other languages. It uses braces and semicolons rather than Python-style indentation, so someone who knows similar languages can read it and get a rough idea of what it does.
What trips up newcomers: data structures
Asked what developers struggle with most, Ryhl names how to put data structures together. Code logic can mostly be written as in other languages, but data structures are different.
Ryhl's example is a book with an array of pages. In TypeScript, you might give each page a field referencing the book, while the book references its pages, creating a cycle. In Rust, objects usually have to be designed so they are not cyclic. They should form a tree or a DAG. Cycles become possible with reference counting, with some caveats. What happens with learners is that they try to build cyclic structures without such tools, hit a compiler error, change the code, and hit another compiler error. The common trap, according to Ryhl, is that people keep changing the code when the actual fix is to change the struct.
Ownership, reference counting, and the borrow checker
Ryhl explains ownership as the idea that when a variable contains an object, the object is only there. It is exclusive. If you write let a = some string and then let b = a, that is a move. Using a afterward is a compiler error because its contents have been moved away. This matters because there is no garbage collector. When b goes out of scope, it must clean up the string. If a were still valid, the string would be cleaned up twice, which is not allowed. In garbage-collected languages, a and b simply go out of scope and the collector cleans up later. In Rust, cleanup happens at scope exit, and ownership ensures that the moved-from variable has nothing to clean up.
Reference counting is one of several pointer types in Rust. Ryhl describes Arc: you have an object somewhere and an Arc pointing to it. Calling clone on the Arc increments a counter and gives you a second Arc sharing the same memory, however large the object is. When an Arc goes out of scope, the counter decrements, and when it reaches zero the object is cleaned up. This is the way to express that an object genuinely needs to live in multiple places with no single owner.
References are another pointer type. A reference is just a pointer to the object, with no runtime checking. That means the compiler must guarantee that the last use of the reference happens before the object goes out of scope. Creating a reference is called borrowing: the owner owns the value, and the reference is a borrow. The borrow checker enforces these rules. It also handles mutation. Ryhl's example is holding an immutable reference to element five of a vector and then calling clear on the vector. The borrow checker prevents you from using the reference to element five afterward. More generally, it ensures that a mutable borrow does not overlap with other borrows: you can have one writer at a time or any number of readers. It checks this within the scope of, for example, a function.
"Fighting the borrow checker," a phrase common on forums, describes being unable to get out of these compiler errors. Ryhl again traces this mostly to struct design. A common mistake is putting a reference inside a struct. Rust assumes it can verify the reference's scope by looking at a single function, but if the struct is passed between functions, that analysis may not be possible and you get a compiler error. The solution is often a different pointer type, frequently the reference-counted one. Once again, the answer is to change the data structure. The host notes this explains why Ryhl singled out data structures as the main hurdle.
Unsafe: an escape hatch for building new features
Rust has a keyword called unsafe, and the host asks what it is and why it exists at all.
Ryhl calls unsafe the escape hatch. Rust's guarantee is that if you use no unsafe, then no matter how stupid your code is, it will never have the bugs that typically become security vulnerabilities. With unsafe, some guarantees remain but they are weaker. Each unsafe operation has a list of rules, and violating them can lead to those bad bugs. Ryhl emphasizes that unsafe does not disable the borrow checker or anything like that. It only unlocks some additional operations that are not safe in general, so you have to verify their correctness yourself.
The vector example illustrates this. Normal indexing with brackets checks the length. Accessing element five requires a length of at least six, and otherwise the program crashes. In very performance-sensitive code, perhaps in a loop, you may want to skip that check. Vector provides a separate function, get_unchecked, which returns the element without checking the length. Its signature is marked unsafe fn, and such functions can only be called from an unsafe block. All an unsafe block does is allow calling functions marked unsafe.
In practice, Ryhl says, unsafe usually doesn't appear at all in something like a backend server. Its main purpose is adding new features to the language. Imagine Rust had no vector type, but did have functions to allocate memory, free memory, and read from a pointer. You could write a Vec struct containing a raw pointer (unsafe to use), a length, and a capacity, with private fields so nothing outside the module can touch them. If anyone could simply set the length field to 20, that would be bad. Using field privacy and careful API design, you can build a vector that safe code can use without being able to break it. Whatever mistakes a user makes, the result is a crash or a proper length check, not memory corruption. "As long as you design your API right," Ryhl says, "you can add new language features by using unsafe." The other classic use is calling into C libraries: you can wrap a C library in a safe API that enforces calling it only in correct ways.
Crates and Cargo
A crate is simply Rust's word for a package. Rust's tool Cargo does a lot in one place. cargo run compiles and runs your code. The Cargo.toml file specifies dependencies and other crate information, and running the project downloads dependencies into what's called a registry, which is basically a directory of every crate you've downloaded. cargo test runs tests, including documentation tests. cargo doc generates documentation. Cargo also supports benchmarks and examples. You don't need separate tools for fetching dependencies, running code, and generating docs.
The biggest difference from something like pip, in Ryhl's view, is that nothing is installed onto your machine globally. With pip, you either install packages globally or use virtual environments. With Cargo, everything is local to the project. The only shared piece is the registry, which exists to avoid downloading the same thing twice.
Ryhl shares an anecdote about Linus Torvalds. At the Linux Plumbers Conference, Torvalds wasn't seen during the entire event until he appeared at the very end of the social event. When Ryhl went to say hello and mentioned working on Rust, he immediately started explaining that he didn't like Cargo. His reason: Cargo downloads code from the internet and runs it, like any package manager, and he doesn't want anything on his computer doing that other than his distribution's package manager.
The host raises supply-chain attacks, which have been a growing problem in the Node ecosystem. Ryhl acknowledges that in principle Cargo has the same exposure. If someone got hold of Ryhl's keys for publishing Tokio releases, they could upload a malicious version. Ryhl doesn't think Rust has had many problems with this compared to npm, but isn't sure, and calls it a hard problem, particularly when someone can impersonate a library maintainer. Ryhl believes the ecosystem does things like delete malicious crates.
Where the ecosystem is mature and where it isn't
Ryhl considers the frontend the least mature area. There have been attempts to compile Rust to WebAssembly and run it in the browser as a TypeScript replacement, but if Ryhl were building a web application, they would use Rust on the backend and TypeScript on the frontend, not the WebAssembly route. Backends are a great fit, and so are command-line tools. Rust is also expanding into Linux and many embedded projects. Ryhl notes that some C codebases are starting to say that since Rust made it into Linux, maybe they should adopt a memory-safe language too, and mentions that Git and CPython are having that discussion, likely among others.
Governance without a benevolent dictator
Rust originated at Mozilla but, Ryhl notes, is no longer a Mozilla project at all. Today there is the Rust project, made up of teams: a language team for the language itself, a library API team that decides on the standard library's API, a compiler team, and others. The people on these teams run the language.
Unlike Python or Linux, Rust has no benevolent dictator for life. Ryhl's description of how decisions get made is blunt: if the team doesn't sign off, it doesn't happen. Particularly contentious or large issues would probably be discussed in person, for example at Rust Week, which includes the Rust All Hands, an event that brings Rust developers together in one place. The host contrasts this with companies, where team boundaries are often set top-down. Ryhl says that in language design, the teams really are the structure, and in Ryhl's experience they have been good at delegating, sometimes saying that something is a libs-API decision and they will go with whatever that team decides.
The RFC process and final comment periods
Big decisions go through RFCs (requests for comments). Ryhl has written one, for a feature called "derive smart pointer," which makes some standard library types less special. The idea is that you write the RFC before implementing, though people sometimes start implementing early. The host notes that the same thing happened with RFCs at Uber.
Ryhl thinks the RFC template is quite good. After the summary, the first important section is motivation: why the feature should exist. Two sections Ryhl finds especially interesting are the guide-level explanation and the reference-level explanation. In the first, you explain the feature as if writing a tutorial for it, as though it already existed. In the second, you explain it again as if it already existed, but as it would appear in the language reference. The host compares this to Amazon's practice of writing a press release before building a feature, since both force you to think from the user's perspective. Other sections include rationale and alternatives, prior art, and future possibilities. Ryhl sees rationale and alternatives as especially important because it lets you answer questions before they are asked and explain why you chose your design over others. Prior art covers questions like what C++ did.
RFCs are submitted as pull requests with a Markdown document to the RFCs repository, where people discuss them. A document written elsewhere is called a pre-RFC. Once the relevant team is happy, a process used for essentially all team decisions kicks in: the final comment period (FCP). Someone tells a bot to start the approval process, and it posts a GitHub comment with a checkbox for each team member. Once all but at most two members have checked their box, a two-week final comment period begins. Ryhl explains the rationale: if everyone else checks immediately, the remaining people still get a chance to look, while things keep moving at a sensible pace. Team members can also file a concern, which pauses the process until resolved. After the two weeks, the proposal is accepted. The same FCP mechanism can be applied to any pull request or issue, such as a change to the language reference that the team wants everyone to agree on.
From nightly to stable: feature flags and the six-week cycle
After an RFC is accepted, the feature is implemented through pull requests but placed behind a feature flag. Rust calls these nightly features: unavailable normally, but usable with the nightly build of the compiler, where people try them out experimentally.
When a feature seems ready, the pull request that removes the flag includes a stabilization report, which Ryhl describes as a relatively recent invention. It explains how the feature has been used, identifies dangerous edge cases, and points to tests for each one. The team runs the FCP process again to agree on stabilization, and then the change is merged into the main branch with no flag.
From there, the timeline is fixed. Somewhere between zero and six weeks later, a beta release of the compiler is cut that includes the feature, and six weeks after that, the beta becomes the next stable release. There is no concept of beta-only features. The beta period exists to test the upcoming stable release so problems can be fixed before it ships.
Smaller changes have lighter processes. For a new function or type in the standard library, the library team uses an API change proposal (ACP), which is just a GitHub issue describing the interface and why it should exist. If the library team approves, you implement it as unstable, then stabilize later. The compiler team has major change proposals (MCPs). Despite the name, these are for changes big enough to need an issue but not big enough for an RFC. The classic example is a new compiler flag.
Editions versus versions
Besides releases every six weeks, Rust has editions: 2015, 2018, 2021, and 2024. The key difference from versions, Ryhl explains, is that when you use compiler version 1.90, everything uses that version, but different crates can use different editions. Rust has a very strong backward compatibility guarantee. The idea is that code you write keeps working forever, and a crate can stay on its edition indefinitely.
Editions are how Rust makes breaking changes "without breaking people," for example syntax changes. When async/await was added in an edition, code on the older edition couldn't use it, and could still have a variable named async. In the newer edition you can't. Crates on different editions still work together: a library written in the 2021 edition can be used by a project on the 2024 edition.
The host suggests an edition sounds like a major version in other languages. Ryhl says the big idea is to change the language while keeping old code working. The difference from something like Python 2 versus Python 3 is that editions can be mixed and matched freely.
Ryhl recently became a language team adviser, which they describe as "language team light." The team trusts the adviser's opinion, but the adviser doesn't attend every meeting and has no FCP checkbox, though they can still file concerns. Full membership would mean weekly meetings on Wednesday evenings, which for Ryhl is normally dinner time.
Who pays for Rust, and how to contribute
The host recalls Greg Kroah-Hartman telling them that about 80% of frequent Linux contributors are employed by companies that value their work in the kernel. Ryhl considers the Linux kernel truly unique in having thousands of contributors doing it on work time. In Ryhl's view, the Rust project does quite well at having people paid by employers, but nowhere near the same extent. The Rust Foundation also offers grants. A student who wants to work on something, for example, might get funding. Ryhl thinks that's a great thing the foundation does.
For engineers who want to contribute, Ryhl suggests the issue tracker, but thinks the best starting point is usually something you personally want to change. Much of the Rust project's discussion happens on Zulip, a chat server where you can talk to people. For Rust in Linux, the Rust for Linux website (rust-for-linux.com) has a contributing page listing drivers you might work on and linking to the issue tracker. Ryhl also points out that you can contribute to the Linux kernel without touching the kernel: implementing a Rust language feature the kernel needs, or helping move the Rust-in-GCC project forward, both help Rust in Linux.
Rust in the Linux kernel: no longer experimental
Ryhl has attended Linux Plumbers for several years as part of their Rust for Linux work, and says each time it has been obvious that things had totally changed since the year before, four times in a row. One year people might see Rust as a nice little side project, and the next year they have Rust code in their own subsystem. At the most recent Linux Plumbers in December 2025, the big news was that the Linux kernel maintainer summit had agreed Rust is no longer experimental in the kernel. The host asks whether that gives Rust the same status as C. Ryhl says not the same status, but the removal of the experimental label means, in essence, "we've proved this is going to work."
The host mentions a US Department of Defense regulation barring agencies from using non-memory-safe languages. Ryhl doesn't address that specific claim but says multiple governments have made similar moves, essentially telling projects that their use of C or C++ causes an unacceptable number of memory safety vulnerabilities that simply wouldn't happen otherwise.
AI tools: feedback loops, code review, and a cautionary Makefile
Ryhl says they have been trying AI tools and are still learning how to use them. They like the Gemini command-line interface.
On whether Rust suits AI agents, Ryhl connects it to the earlier refactoring story. People often say tests matter when using AI because they let the AI check whether it did the right thing. Similarly, agents can talk to the compiler, which tells them what to fix. Ryhl thinks Rust could be a promising candidate for agents on that basis, and when the host adds that certain bugs are harder or impossible to ship, Ryhl agrees there is an aspect of that.
At the Linux kernel maintainer summit, one topic was AI for code review. Some people had set up bots that feed patches sent to the mailing list into an AI agent that posts a review. Ryhl reports that Linus and others said these reviews were actually very impressive for kernel code, and that this use case seems to excite the kernel community. Ryhl ties it back to memory safety: these bugs are trivial mistakes that can be made in a huge number of places. Even a very good programmer who makes them only 0.1% of the time will still make them, so if AI catches many of them, that's valuable. The host adds that AI might help find missing edge cases in tests, and Ryhl agrees. The host observes that this is one of the first AI conversations they've had focused on raising quality rather than just speed.
Ryhl has written a few patches with AI. It worked okay, but always required their review before going to the list. One positive experience involved a commit that made Binder faster. A reviewer asked for a benchmark, which Ryhl wasn't looking forward to writing. The AI found an existing benchmark, modified it in a clever way, ran it, wrote a Python script to parse the output file, and presented the results in a table. Ryhl's assessment: at least in that instance, it saved time, if nothing else.
Asked whether AI could give Rust learners a false sense of understanding, Ryhl says it is "totally a danger" and gives an example from the kernel build system. Ryhl asked an AI to add support for a feature, and it added the necessary Rust build flags to a Makefile. But the C side passed a few additional flags that weren't strictly required yet existed for a reason, and the AI ignored them. Any human, Ryhl says, would have asked why not add Rust versions of all the flags.
Learning Rust, and what Ryhl is working on now
For experienced engineers learning Rust, Ryhl recommends the official Rust book, freely available on the Rust website (noting, amused, that Rust calls its online tutorials "books"). Beyond that, the way to learn a language is to build a project with it, perhaps a web server. For intermediate developers who have some Rust experience and want to go further, Ryhl recommends a book by Jon Gjengset. Ryhl also likes Rustlings, which gives you unfinished Rust code to complete.
Ryhl's current favorite feature is one they are still building. The Linux kernel needed in-place initialization: the ability to construct a value while knowing where it is being constructed, so it doesn't get moved afterward. Ryhl is excited about the work to add this to the language but stresses that it is very much ongoing.
In closing, the host draws out a theme from the conversation: Rust seems deliberately designed around the question of which mistakes programmers keep making and how to eliminate them, as seen in memory safety, the absence of null, errors as values that can't be ignored, and documentation examples that run as tests. The host's hypothesis for Rust's rise is that it occupies a category that didn't exist before: a language that is both reliable and performant, usable for kernel work while also being a strong alternative for higher-level backend development.
What would your pitch be on why it's worth checking out Rust or working with Rust? You need a language that's reliable, that's going to have as few bugs as possible. That's the idea of Rust. What is the ownership model?
The idea that if you have a variable containing some object, then the object is only there. So, it's kind of exclusive. We talked about memory safety, but there is a keyword in Rust called unsafe. Generally, when unsafe is used, it's to add new features to the language. As long as you design your API right, you can add new language features by using unsafe. What are editions?
Editions are basically the way that Rust makes breaking changes without breaking people, because they might change the syntax of the language. Do you use any of the AI tools for your day-to-day work contributing to Rust projects? So, I have been trying to use them. Honestly,
Rust is quietly spreading as a language of choice to build reliable and performant applications. But what makes it different? Alice Ryhl is a software engineer working on Google's Android Rust team, a core maintainer of Tokio, the de facto async runtime for Rust, and is a Rust language team adviser. In today's conversation, we cover the pitch on why Rust is worth considering whether you're using TypeScript or C++ today, how concepts like ownership, the borrow checker, and the unsafe keyword work, and what are things that trip up newcomers to Rust, how the language is governed without a benevolent dictator, and how RFCs and editions work, and many more. If you want to understand what makes Rust different and why so many engineers say once it compiles, it works, this episode is for you.
This episode is presented by Antithesis. Verify your system's correctness without human review or traditional integration tests and avoid bugs or outages.
This episode is brought to you by Sentry. Sentry is application monitoring software built by developers for developers. The first time I used Sentry was 10 years ago back at Uber, where Sentry helped keep us honest on when and where our services were breaking. I also use Sentry today to help me understand if the services and APIs I built for The Pragmatic Engineer are healthy or not. Sentry shows you the full context on issues, stack traces, user actions, environment details, and even the exact line of code that caused the issue. It supports pretty much every modern tech stack: TypeScript, JavaScript, Python, Go, and others. It works on backend and frontend and mobile, you name it. One new feature Sentry launched is Seer, their AI debugging agent. Let me show you. I open the Seer agent and ask about what are some repeated errors happening on my backend. Seer figures out that a repeated issue is a network call failure. I can then ask for more details and debug more efficiently with this AI agent integrated neatly in Sentry. Seer is a neat tool to fix the hard issues, the ones that are just hard to debug. Check out Sentry at sentry.io/pragmatic and start monitoring today.
Alice, welcome to the podcast.
Thank you for having me.
It's really nice to have you here. How did you get into software engineering?
It actually all started with Minecraft.
No way.
I wanted to write my own mod for Minecraft, so I learned Java. I didn't get very far with the Minecraft modding, but that's where it started.
How did you continue? Did you go to university?
This was just before I started in high school. And then after high school, I had a year where I worked full-time as a software engineer. And then I moved on to starting my bachelor. Did that for 3 years, and then I did a master's for 2 years.
How did you end up at Google? Was that straight out of university?
Actually, I started part-time at the same time as when I started my master's, and then I switched to full-time after I finished.
How did you get involved with the Rust community? Was it at Google? Was it before Google?
Oh, way before.
Way before.
I've been doing Rust for a long time. When I was in school, I spent a lot of time on what's called the Rust users forum. Well, I was answering questions, really. I have maybe 10,000 posts on there or something. At some point, I also started being active in some chat servers, the Discord server for something called Tokio. I kept doing that, answering questions. When I saw common questions, I would fix the documentation. That's how I got into Tokio, which I'm now one of the maintainers of.
And then for those of us not as familiar with Rust and Tokio, what is Tokio inside of Rust and why is it important?
So Tokio is, well, it's an asynchronous runtime for Rust. You can think of it as the standard library for Rust when you're using async. I mean, if you compare with something like JavaScript in the browser, you might compare Tokio with the browser itself. For example, in JavaScript, you have this loop, this event loop, which has all the tasks that are able to run, and then they get executed one after the other. And especially if you use the await stuff, then you can have tasks pause and then another task start running on the same thread. And Tokio does something similar. It has a queue of things that are able to run, and then it will run them. So unlike JavaScript, Tokio can be multi-threaded. So you can have multiple queues running in parallel.
This seems like a pretty core part of Rust as a language, as an ecosystem. How did you gravitate towards this? Because it sounded like, if I understand, you were lurking on the Rust forums. You were helping out here and there. What drew you to this part of the language, or the ecosystem, should I say?
I think part of what I liked about Rust is this feeling that as you write the code, when it compiles, it works. I mean, this has to be in quotes, right? Because obviously it's possible that there are bugs, but this is something a lot of people say about Rust, and there's a reason people say it even though it's not necessarily literally true.
How do some other languages compare?
To begin with, I think to have a language that feels this way, you have to have a type system. That's where it all starts. Yeah, I do think that even compared to other languages with type systems, I think Rust does a better job than many languages, even others with type systems. I mean, the classic example is Java's null. It was Tony Hoare who invented the null, and he called it his billion-dollar mistake, because it's so easy to have... I mean, every time you call a function, you might have a crash in your program.
And in Rust, I think they're really good at making sure that when you call a function, there's no chance that it might be null, right? That problem just doesn't exist, so you can't have that kind of crash.
And so you have to explicitly say this object might be null, and then the compiler will force you to check for null before you use it. So you can't forget like you can in Java.
Can we talk about Rust as a whole? This was a language that was created 20 years ago, in 2006, and it feels like it's become a lot more popular over time. Do you know of the scale that Rust is at in terms of usage or popularity, or numbers that are known among the Rust community?
One thing I found kind of funny: I checked the other day, and we have actually overtaken PHP and Go on the TIOBE index.
Oh, that index tries to estimate the usage of languages, right?
Usage or popularity or something like that.
Wow. Because the feeling I have in general is that Rust is popping up in more and more places. More companies are building on Rust. Oxide is a good example. They decided to go all in on Rust, and there's a sense of Rust becoming a popular place to build, I guess, high-performance applications, even increasingly contributing to the kernel. As an engineer who might know other languages like TypeScript, Java, etc., what would your pitch be on why it's worth checking out Rust or working with Rust?
That depends a lot on which language you're coming from.
Oh, let's come from TypeScript. It's one of the most popular languages right now.
The pitch from TypeScript would be a lot different than the pitch from C++. But okay, so to begin with, I think when it comes to TypeScript, Rust fits in as the backend language. That's where I would put it. I wouldn't use it in the frontend. I think it's a pretty good fit for backends, API servers. One way to put it is you don't want to be woken up at night because there are problems with your web server. You need a language that's reliable, that's going to have as few bugs as possible. I mean, obviously, it would be nice if it had zero bugs, but, you know, it's going to be hard to get there, but as few bugs as possible. That's the idea of Rust. I mean, I already mentioned null, but it does a lot of stuff like that. So there's no null checking, and this is done through, you know, it has this enum type, which, I mean, I think TypeScript can do something similar. TypeScript is actually kind of good on that front. But okay, the other thing I think is quite good is error handling. So on one hand, Rust doesn't really use exceptions. So it actually returns the error as a value. So you return a value that's, using an enum, either the result or the error. And the way this is done is that there's an operator, question mark, which says... so you write my function and then question mark at the end. And this means if this function fails, return the error. So it's really easy to handle errors, but it's not zero characters like it is with exceptions, right? So it's explicit, on the other hand. And if you forget to put the question mark, that's a compilation error.
Wonderful.
So you have to check it.
Yeah.
Right. And of course you can also handle that manually. But the point is it's this idea of, there are these things where you write some code and there's some implicit error condition you didn't think of, and now you just, you know, took down your server or something. Another thing I quite like is how it handles documentation.
How does it?
For one, when you have a comment, you make it into a documentation comment by having three slashes instead of two. Now the thing is, you can of course write examples in your documentation, and Rust makes all examples into tests. This means that if you change the underlying code, now your test fails. And so this means that you can't forget to update your examples in your documentation.
Yeah. And I guess there's also things like, can you forget to initialize a variable in Rust?
I mean, no.
It doesn't allow you, right?
You have to set a value before you use it for the first time.
Yeah. And I guess the thing with checking the format of incoming JSON, it also forces you to do that.
Yeah. So there's a pretty cool library called Serde in Rust, where you have your struct, so your type, you know, with fields, and then you can say, I want to be able to parse this from, for example, JSON. And then Serde will, it's a macro, it will generate code which checks the JSON, that it's in the same format, it has all the fields you need and they have the right types, and it generates native code that's specific to that particular shape of JSON. So the checks are really efficient.
And again, yeah, so it helps you avoid errors. And there's this thing which I learned just very recently, thanks to you, as we were talking ahead of this: the switch statement, right? So in almost every language you have a switch for an enum, and you handle cases, and then you might have a default or, you know, everything else, and sometimes you forget one of them. It's not a big deal, or maybe it's a big deal, but in Rust you cannot do that either, right?
That's right. So in Rust it's not called switch, it's called match, but it's the same idea. You can match on your enum, and then you can have a branch for each possibility, and if you are missing one, that's a compiler error. Of course you can have a catch-all case if you want to, but most of the time you would just list all the cases, and then in the future, when you add a new variant, the compiler will tell you, oh, you need to update your code here, here, and here. And I think this kind of leads to another way that Rust really helps with reliability, which is that if you're refactoring, I think Rust is really good at telling you all the places you need to update. I've done this sometimes where I would refactor something. I change the code, I change the return type or whatever it is, and then I just fix the compiler errors until the compiler stops shouting. And then once I've done that, I've updated every place I need to update.
I get a sense that the language designers have thought really hard about what are ways that typically go wrong in a lot of other programming languages, and they just try to fix it through the compiler. The documentation example is the one where I'm still like, wow. It happens all the time: you have a comment example or not, and then it gets out of sync, and we always complain about this and we don't know how to fix it. I know I've been complaining for a decade plus, and Rust is the first language where I hear an actual solution, even if it's not a perfect one.
And I really think this pattern, just everywhere you look, you have this kind of thing again and again, that, oh, if you messed up, you know, either it won't compile, or at the very least there's a lint for it. They just catch a lot of cases at compile time.
Now, we've had the pitch from TypeScript or similar languages. What about the pitch from C++?
So here I actually think it's even stronger. The thing with C++ is that if you make a mistake, right, in JavaScript maybe you take down your server, which is already bad enough, but in C++, when you make a mistake there, now it's actually a security vulnerability most of the time.
Mhm.
If you do something as trivial as an off-by-one in your array or whatever it might be, that's a security vulnerability, and this just keeps happening.
Small mistakes become security vulnerabilities.
And in Rust, so Rust is memory safe, right? I mean, we talked a bunch about different ways that Rust is more reliable, and we didn't even touch memory safety.
Let's talk about memory safety.
Memory safety is this idea that no matter how stupid the code you write is, it's not going to have a certain class of bugs. And this is, you know, the kind of bug that usually leads to security vulnerabilities. You know, the kind of thing where you read past the array and you just look at random memory, or you destroyed an object and then you used it afterwards. So now you actually touched the memory of some other random object.
Yep. And then an attacker who figures this out could populate something there, eventually get that code somehow executed or configuration read, and then boom.
The classic example in the kernel is, let's say you have some object, and you manage to make it so that the object that's actually there, right, because the original object is gone, so the memory got reused, and now it has a task struct, it's called, and that's basically your process, and it has a field called user ID. And it's pretty common for code to write zeros to memory. But if you write a zero to the user ID,
Now you're root.
That's a root user.
Yeah, that's a really classic way of exploiting this kind of vulnerability.
And then once an attacker manages to do that, they can take over your server or whatever is running. And then of course from there on it can just spiral out of control, right? Once you're root, you're lost.
Yeah.
Do I understand that a very strong pitch of Rust, especially coming from C++, is that memory safety eliminates this whole class of bugs, which can turn into security vulnerabilities, which are one of the most serious threats any software can have?
Plus all of the reliability ones I mentioned in the beginning.
That is a pretty good pitch. We just talked about how Rust offers several safety features like memory safety, error handling, and a type system. This is because reliability is not about one thing but several things at
once. This is where I need to mention our presenting sponsor Antithesis. Like the designers of Rust, Antithesis also believes there's no one silver bullet for reliability. You need many different tools and approaches. This is why they've released Hegel Rust, a free open-source property-based testing library for Rust built by the team behind Hypothesis. The Rust compiler is brilliant at catching whole categories of bugs at compile time, but at the edge cases, the weird input combinations, the assumptions that turn out not to hold, those runtime bugs need a different tool.
Hegel provides powerful ergonomic property-based testing for fast local development. It'll check edge cases you never thought of and catch unknown unknowns before they bring down production. And if you try Hegel and like it, your Hegel tests will run in Antithesis as written, so you can easily add determinism and the world's most thorough runtime verification to your reliability arsenal. Go to hegel.dev to learn more.
I'd also like to mention two conferences this year where I'll be talking and where we can also meet. On the 4th of June, I'll be doing a keynote at Craft Conference in Budapest, Hungary. This conference is one of the very best ones in Europe focused on software craftsmanship and one where I'm a returning speaker. For more details, check out craft-conf.com.
And on the 15th and 16th of September, I'll be doing a keynote at LDX3 in New York. This is the festival for modern engineering leadership. Last year, I was at LDX3 in London. And so, I'm excited to be back this time in New York. For more details about this conference, head to leadub.com. If you'll be at either of them, I'll see you there. And with this, let's get back to Rust and to Alice.
I wanted to ask why Rust is getting so popular, but I think we're starting to answer this question, right?
I think where Rust is really unique is in the combination of things. So on one hand, it doesn't have a garbage collector and it's usable in low-level contexts like the Linux kernel or firmware or whatever.
Why is it a good or a bad thing to have a garbage collector? Java has a garbage collector. C# has a garbage collector.
So, a garbage collector says once you're done using your objects, there's going to be a little piece of code that checks all of your objects and says this is not used anymore and then it cleans it up. Whereas in languages like Rust or C++, the variable is cleaned up at the end of the scope when it goes out of scope. And in the other one, they have to detect afterwards. And this kind of little piece of code that runs every so often to check all your objects, for embedded use cases, this might simply be not possible or unacceptable.
The performance overhead, the fact that you will not be able to control the memory as much.
Even for back end, it can be a problem because if you have a request incoming right when it checks all of your objects, you have some sort of latency spike where it takes much longer to reply. So that's one of the reasons it can be helpful in back end as well.
For someone who has not yet seen Rust code, how would you describe it? How it compares to, for example, TypeScript.
I mean in many cases Rust code is similar to many of the other languages. It has braces. It's not like Python where you know you use indentation. It has braces. It has semicolons and so on. So it's pretty easy to read. If you know some similar languages you can look at it and you can get a rough idea.
Yeah, I think you'll figure it out. It's not that foreign. Rust has a learning curve on the side of the more difficult languages to learn. What do you see devs who are new to Rust typically struggle with, and what are the things that just make it click for them?
I think the most tricky thing people run into is how should you put together your data structures? I mean for your code you can mostly do the same things as you can in other languages. I mean that's not entirely true but the big thing is really the data structures. So if you have, I don't know what would be a good example, okay, let's just say you have an object for a book and it has an array of pages and then you have an object for each page. If I was writing TypeScript I would probably have a book field in the page that references the book and then the book of course also references the page, right? So it's cyclic. So the book goes to the page goes back to the book. And in Rust you usually have to design your objects so that they're not cyclic. It has to be a tree or a DAG if you know what that means.
Yeah. Yeah. Directed acyclic graph.
So if you go out and actually start to use reference counting, it becomes possible to have cyclic objects with some caveats like you mentioned. But what people end up doing when they're learning the language is that they try to make cyclic objects without anything like that. And then it just doesn't work and they're going to run into all sorts of problems in their code while they're constructing. They're maybe creating their thing and then they get a compiler error and then they try to change the code and they get a compiler error again. But the thing that a lot of people struggle with is that they keep trying to change the code when the solution is to change the struct.
I can see how that's frustrating. Rust has another model that might be new, especially for people coming from other language types: Rust's ownership model. What is the ownership model?
The ownership is just the idea that if you have a variable containing some object then the object is only there. It's kind of exclusive. So for example, if you have a variable let A equals your string or whatever and then you do let B equal to A, then this is a move. And so this means that using A afterwards is now a compiler error because its contents have been moved away. And of course this is important when we don't have a garbage collector because then when B goes out of scope, it has to clean up the string. Now if A was also valid then when they both went out of scope it would clean up the string twice which is not legal.
In most other languages the garbage collector takes care of this. A and B just go out of scope, do nothing, and then later it cleans up the string. But here we actually have to do it when we go out of scope. And the ownership model allows you to move from A to B and then A becomes inactive or unusable and does not get cleaned up because it doesn't have a value to clean up anymore.
And how does reference counting relate to all of this?
In Rust, we have a bunch of different types which are essentially different kinds of pointers. And one of the pointers is called Arc, and it's reference counted. And so the idea is you have some object somewhere and then you have an Arc to the object and what you can do is you can call clone on the Arc and this increments a counter and now you have two Arcs to the same underlying memory. So the object might be really big but you have two Arcs that share the same memory and when they go out of scope the counter is just decremented and when the counter reaches zero the object is cleaned up. So this is a way of saying this object actually needs to be in multiple places. There's no one place that owns it. And so this way you can use a counter to know how many owners there are. And then when the last owner goes away, it gets cleaned up.
And on Rust there is also another thing that was new to me, the borrow checker.
So another pointer type we have in Rust is called the reference. And a reference is basically, all it is is that it's a pointer to the object and that's it. So we do no checking at runtime. So of course this means at compile time we have to make sure that the last use of the reference has to be before the object goes out of scope. And creating a reference is called borrowing, right? The owner owns the value. And now we have a borrow of the value, the reference, and the borrow checker checks that the last use of the reference is before the object goes out of scope. So let's say you have an immutable reference you're reading, then if you change the object, let's say it's in a vector. So I have an immutable reference, I'm reading element five, and then I change the vector, I call clear. Then the borrow checker also ensures that you can't use the reference to object five afterwards.
Yes. Because as soon as you change it, it went out of scope. It's now cleared. Right.
Yeah, the way it works is that if there are mutable borrows, it ensures that the mutable borrow starts after the previous borrows have ended, basically. So they don't overlap. So you can only have one writer at a time or you can have any number of readers. It checks that on the scope, like in a function.
This concept is new to me as someone who's not used Rust, for sure. It's very interesting. One thing I've read on forums is people complaining about so-called fighting the borrow checker. What can make it challenging?
When you write some code that uses the object in a way that doesn't follow the rules I mentioned, that's a compiler error. And fighting the borrow checker is when you, you know, can't get out of those compiler errors, I guess. I think a lot of the time this has to do with the struct. One common mistake I see is that if you have a struct with a reference in it, Rust kind of assumes that it can check the scope of that reference by just looking at a single function. But if you have your struct and you're passing it over functions, then it might not be possible to make that analysis and so you just get a compiler error. And so in this case the solution is maybe to use a different pointer type. For example, the reference-counted pointer type often solves this kind of bug, right? So the solution is again to change the data structure.
Yeah, I'm starting to understand why you mentioned that data structures were a place of learning coming from other languages. Oftentimes the solution seems to be just think about your data structures, understand them and do it in the Rust way, right? We talked about memory safety, but there is a keyword in Rust called unsafe. What does this do and in what cases do people typically use it? When does it make sense to use it and why does it even exist? That's just a naive question from me.
So let me begin with the what and let's take the why afterwards. So the what is: unsafe is the escape hatch, essentially. So I explained before how there are certain bugs where if your program has one of those bugs, that's usually a security vulnerability. What Rust ensures is that if you have no use of unsafe, then no matter how stupid your code is, you will never have one of those bugs. Now, if you do use unsafe, then there are still some guarantees, but it's a bit weaker because each unsafe operation that you can perform has a list of rules. And if you violate these rules, then you might end up with one of these bad bugs. But of course, if you don't, then it's okay.
And it's interesting to point out here that unsafe does not disable the borrow checker or anything like that. It just gives you a few more operations you can perform that are not safe in general. And so you have to check yourself: yeah, this is actually okay in this particular case.
I mean let's take the vector example again. Normally when you index, index five, it will say oh let me check the length. So if the length is at least six
Yeah, because it starts at zero.
then it's okay, otherwise you get a crash. But maybe you're writing some super high performance code and you want to avoid this check, or if you're in a loop, every time.
So unsafe will just avoid checks. May that be runtime or compile time checks or both?
What I would say here is that vector has two different ways of getting a value. There's the, you know, normal bracket operator we use from other languages, which will do the check and crash your program if you get it wrong. But there's another function called get_unchecked. And so if you do write vec.get_unchecked(5) then it will give you element five without checking the length. And this function, in the function signature, it says unsafe fn. That's the function signature.
And so you can only call this function from an unsafe block.
Got it.
And so all that the unsafe block lets you do is call functions marked unsafe.
And then in practice, for sensible use cases of unsafe, is it usually to do with high performance code typically? What cases have you seen which are legitimate, where this is a great use case for unsafe?
So usually it will never show up in, say, a backend server. You would have zero uses of unsafe there. Generally when unsafe is used, it's to add a new feature to the language. Let's say the language didn't have a vector. The language still has a function to allocate memory, a function to free memory, a function to read at this pointer address. Then you could write struct Vec, the pointer is here, and this is a raw pointer so it's unsafe to use. The length is this. The capacity is this. And then you can write your little API. Of course the fields are private so you can't access them from outside the module. I mean if I could do vec.length equals 20, just access the field, that would be pretty bad.
And so you can write your own vector API and using field privacy and good API design, you can add a vector to the language if it doesn't already exist. And unsafe, if encapsulated properly in this kind of API that doesn't permit you to mess it up, then you can have a safe vector that you can use from safe code and no matter how stupid the thing you do with that vector is, it's not going to do something bad. It's just going to crash or whatever, check the length properly. As long as you design your API right, you can add new language features by using unsafe.
The other classic example would be to call into a C library. You can write a library that calls into C and you can even have a safe API and then you can enforce that you only call the C API in the right way.
Can we talk about the Rust ecosystem, the broader ecosystem? And when it comes to this, the first thing that people come across, including myself, is the crate ecosystem, Rust's package manager. What is a crate and how does it compare to other package managers in places like npm or pip in Python?
So crate is just the Rust word for a package. So it's just a package, right, of your code.
So the way it works is that Rust has a tool called Cargo and Cargo does a lot of stuff. It's actually a pretty neat tool that's kind of all in one. So it's both what you use to run your code. You do cargo run and it will compile your code. You have this thing called Cargo.toml where you specify your dependencies and other information about your crate. And when you do cargo run, it will download the dependencies to something called a registry, which is basically just a directory with all the crates you've ever downloaded.
Yeah.
And you can also do cargo test. It will run all your tests. Cargo doc, it will generate your docs. Cargo test will also run your doc tests. There's even benchmarks and examples. So it does the whole suite, so to say. So you don't need different tools to fetch dependencies and run your code and generate docs.
I think the biggest difference from something like pip would be that they're not installed to your machine, right? With pip you either install things globally or you have these virtual environments. So with Cargo it's all local. You just run cargo run and it's only local to this. The only thing that's shared is this registry, which is just to avoid downloading the same thing twice.
You told me a funny anecdote about Linus Torvalds and his reaction, or what he told you about Rust and Cargo.
The first time I was at the Linux Plumbers Conference, at the very end. So Linus wasn't anywhere to be seen throughout the entire conference, but at the very end of the social event he
showed up and I went over to say hi. I mentioned I worked on Rust and immediately he started telling me about how he didn't like Cargo, and so the reason for that is that Cargo downloads code from the internet and runs it, like any package manager.
Yep.
And he doesn't want any code on his computer to do that other than his distribution's package manager. He doesn't trust anybody else to do that.
In the Node world, we're seeing more problems with vulnerabilities being injected into packages. Just a bad actor overtaking packages, you know, that they put in whatever code might be from crypto, which is I guess the better part to security vulnerabilities. Does Cargo have this problem as well? Just like any package manager that is on the internet?
I mean, in principle, yeah. I mean, ultimately, if you somehow got a hold of, you know, my keys to upload new Tokio versions, you could upload a malicious one. I don't think we've had much problems with it compared to something like npm, but I don't really know if we... I mean, it's a hard problem, right?
It's a hard problem. Somebody can impersonate the maintainer of a library. Then I mean I think they do do stuff like delete people squatting malicious crates and stuff like that.
Yeah. Where do you see the Rust ecosystem being the most mature and the least mature right now?
My opinion is that the least mature area is front end. There have been some attempts to compile Rust to WebAssembly and then run it on the web as a front end, as a replacement for TypeScript. But if I was writing a web server, I would totally use Rust for the back end and TypeScript for the front end. I would not really go the WebAssembly route.
On the other hand, for back end, I actually think it's a pretty great fit. And there's also stuff like command line tools. I think it's a really, really great fit for that. And then of course we are expanding into Linux and we are also expanding into a lot of embedded projects, and there are even projects that are like C code bases that are starting to say, hey, it went into Linux, maybe we should start using it too. Maybe we should have a memory safe language too. I know that Git is talking about it. I know that CPython is talking about it. There are probably others. QEMU maybe.
Can we talk about how the language is built? Who builds Rust? What's the process for doing it? How does it compare to a project like Linux? I know it's not a language, but it's still a large open source project.
It's really an open source project. I mean, you may know that it originally originates from Mozilla, but it's not a Mozilla project anymore at all. So today there's the Rust project, which has a bunch of different teams. There's the language team, which, you know, deals with the language. There's the library API team, which decides on the API of the standard library. And these teams kind of govern it. So it's the people who are in these teams that run the language.
One interesting difference compared to some other popular languages like Python or projects like Linux is they have a benevolent dictator for life and Rust does not. How is this working, and how are decisions made, especially when they're contentious or when it could help for, you know, just someone to just make a decision?
Honestly, if the team doesn't sign off on it, it doesn't happen. I mean, if something is really contentious or really big, it will probably end up being discussed at a conference, for example. So there's a conference called Rust Week where they have an event called the Rust All Hands, where they're basically taking all of the Rust developers and putting them in one place. So if there was a problem like that, they would probably discuss it there.
And how are these teams structured? So like you said, there's compiler, language, library, and dev tools at the very least. How do they define the boundaries? Is it just a team kind of roughly defining them and then you just kind of agree? Because as I'm thinking at a corporate level, like as a company would run, teams are often founded top down. Leadership mandates this will be this team, that team. There's bottoms up happening, but oftentimes it's top down just because it's easier. But here, as I understand, there is no top down. It's people self-organizing.
Yeah, really, when it comes to the design of the language, the teams are really... that's really it. Teams have been pretty good, as far as I've seen, at delegating. So I've seen them sometimes say, we think this is a libs-api decision and we will go with whatever libs-api decides.
One thing that is interesting to talk about is the RFC process. So there's request for comments. RFC, request for comment, is the way the Rust project makes big decisions, but, you know, big ones is important to say. Basically the way it works is, let's say you have a language feature that's kind of big. For example, I had one called derive smart pointer, which basically makes some types in the standard library less special. So I mean, sometimes people begin implementation already, but the idea is that you write the RFC first.
By the way, that's no different inside of, like, at Uber we had an RFC process, and same thing, usually you follow it, but sometimes you kind of start to build it and then just...
The idea is you write the RFC first.
Yeah, I know. It's just funny how it happens everywhere where there's engineers. I think it's is to build.
You write this doc and it has a template, and I think it's actually a pretty good template. The idea is the first section is, I think, motivation. Well, the first one is summary, but the first important one is motivation. So you explain why this feature. And then I think they have two really interesting sections. They have the guide-level explanation and the reference-level explanation. And what this is, is that in the guide-level explanation, you explain your feature as if you were writing a guide, as if the feature already existed. And in the reference-level explanation, you explain your feature again as if it already existed, but as if it would be in the language reference instead of a tutorial.
This is really interesting. You know, Amazon does, just related to this, when they ask people to build a feature, they have a press release. Like, imagine if you announced this. I just love how both this and what you're saying, it forces you to just think from a very different perspective, right? From how people will use this feature. And then other sections, I think you mentioned rationale, alternatives, prior art, future possibilities.
Yeah, so that, you know, you get to explain. So rationale and alternatives, I think, is a pretty important section because you get to answer all of the questions before they get asked, and you get to explain why did you pick this design and not some other design. And I think that's usually a pretty big part of an RFC. And then of course it's good to look at what did C++ do and what can we do in the future as well.
And once you have the RFC, you send it out. What happens then?
Well, somehow you get people to look at it. There is an RFCs repository and you can open a pull request there with your markdown document with the RFC, and people can discuss it. You might also write up your doc somewhere else; then we call it a pre-RFC if it's not in the RFC directory, but it's kind of the same thing.
Let's say that the language team, like it's a language team RFC, and they're happy with it. So I said the RFC was how the language makes big decisions, but then they actually use a process that comes up for all decisions essentially, called the final comment period. And so the idea is they tell the bot to please start the approval process, and then it will create a comment on GitHub with a checkbox for every team member. And then team members check their own checkbox. And once everybody from the team except for at most two have checked their checkbox, then the final comment period will begin, which is 2 weeks.
Oh. So when it's only two checkboxes left, then two weeks starts. What's the rationale for this? This is very interesting. Is it a historic fact more, like...
I mean, you know, let's say that everybody checks that box immediately and then the two other people didn't get a chance to look yet. So it's to make sure that everybody has a chance to see it.
Yes, I understand.
And also to keep things moving at a sensible pace. So there's another part of the process, which is concerns. Team members can file a concern, which basically pauses the process until it's resolved, and then once the two weeks pass, it's accepted.
So that's essentially how the team makes decisions, and this is not just RFCs. If you have, I don't know, a pull request changing something in the reference, then the language team might say, oh, we need to make sure that everybody is on board with this change. They'll tell the bot to start an FCP on this random PR or some random issue or wherever it might be. The same process applies.
And then once, let's say it's a language feature, okay, the RFC is accepted, we know what we're going to build, then the feature is just built. You start to open your pull requests and get it into the language, pretty much. But you put the feature behind a feature flag.
In Rust we have this thing called nightly features, which basically means that you can't use it normally, but if you use the nightly build of the compiler, you can. Once you have your, you know, your feature... I mean, you might begin, but, you know, once you have your accepted RFC, you start submitting pull requests, you implement your feature, it gets merged, and people start using it experimentally.
On a nightly build.
Yeah. And then at some point you might say, okay, I think this feature is ready. And this is kind of a recent invention, but we have come up with this idea of a stabilization report. In the pull request that removes the feature flag, we write up a little report saying, for example, how it's been used and explaining, like, oh, here are the dangerous edge cases, here are the tests for each of the dangerous edge cases. So that kind of stuff. And then they use the FCP process again to agree to stabilize it, and then it's merged. And so now you have a feature without a feature flag.
Yeah.
In the main branch.
Yes.
Between 0 days and 6 weeks from that, there will be a beta release of the compiler and it will be...
And it will be in it.
Yeah. It will be in it, and then 6 weeks later the beta release becomes the next stable release.
Okay. So Rust has a strict six-week release cycle, unlike with Linux. Well, Linux also has a rough emo, but here you just take the stable branch. You're not doing anything special.
Yeah. And so we don't really have a concept of beta features that are only available on beta.
You have the nightly, pretty much. Nightly, or people can get on to the latest branch, which is not yet cut.
Yeah. The main purpose of beta is to test. Like, you get six weeks to test the stable release. So hopefully if there's any problems, we can fix them before it goes out into a stable release.
And you mentioned that RFCs are only for large decisions that need to be made, and you haven't written that many either, just because they're large. What about smaller decisions that you also want to get consensus on?
So here there are a few different ways to do it. So let's say you want to add a new function in the standard library or a new type. Then the library team actually has this thing called an ACP, an API change proposal, which is basically just an issue on GitHub. So you open your issue, you describe your API, describe what the interface is, and explain why you should have it. And it's much smaller than an RFC would be. And then the library team can say, this is okay with us, and then you can implement it on unstable or nightly, and then later you can stabilize it. And similarly, the compiler team has, it's called, major change proposals.
You need to say how it's called.
MCPs.
That is going to confuse a lot of people outside of... we're not inside of Rust. Yeah, it is what it is.
Which is for the smaller features. Yeah, even though it's called major, right? It's big enough that they need you to put up an issue, but not big enough to need an RFC. And for the compiler team, the classic example would be a new compiler flag.
Now, Rust has new builds every 6 weeks, right? But it also has editions. There's one in 2015, 2018, 2021, 2024. What are editions?
So the thing about editions that is different from versions is that if you're using version 1.90 of the compiler, everything is using that version. But different crates can use different editions. And so I might have a crate using the 2025 edition of the language and I can keep using that forever, because Rust has a really, really high backwards compatibility guarantee. So you write all of your code and the guarantee is that it's going to keep working forever. That's the idea anyway. Editions are basically the way that Rust makes breaking changes without breaking people, because they might change the syntax of the language. For example, async/await was added in an edition, and so code using the old edition can't use async/await there. You could have a variable called async if you want, let async = 5, and then in the new edition you can't. But you can still mix and match code written in different editions, so they work together.
Oh, interesting.
So I might have a library written in the 2021 edition and you can write your library in the 2024 edition, or your binary project, and then you can still use my library.
Why do you think Rust decided on editions, which feels a bit more confusing to me as a developer, as opposed to versions? Like when I look at Java's versions or PHP's versions or even Ruby on Rails versions, you know, there's always a major number. And I might be wrong, but it sounds to me that an edition is almost an equivalent of a major version in other languages. Is that putting it correctly or am I missing some details here?
Really the big idea is to say we want the old code to continue working, but we still want to change the language. And so the difference from other kinds of versions of languages, I mean Python 2 versus Python 3 comes to mind, is that you can totally mix and match in any way you want.
You are a language team advisor, or you recently became one. What is a language team advisor and what do you do as part of this role?
We have of course the language team, which, you know, they have meetings every week and they do a lot of stuff and so on and so forth. The language team advisor role is a way to be part of the language team light, in some sense. So you're someone that they've said, "Okay, we trust this person's opinion," but you don't necessarily go to every single meeting. Like, there's no checkbox for you on an FCP, though you can still file a concern. So yeah, it's really a way to participate in or help the language team without full membership. And full membership might entail a lot of things, right? For example, for me, it would entail going to meetings every Wednesday evening instead of, like, when I would normally have dinner.
From your observation, what is the corporate influence like on Rust? In the sense that when I talked with Greg KH, he was telling me that in practice about 80% of the frequent Linux contributors are typically employed by a company which sees value in adding their features or maintaining their features in Linux, which turns out to be a nice kind of, I guess, symbiosis in some ways. For Rust, the people working on it, are they usually paid by their employers like you are? I'm sure there's people who are just every now and then contributing, but people who are spending more time on this. Do you see a pattern of corporations actually supporting this, or is there a foundation that also supports people to work on this full-time or close to full-time?
I would say here that the Linux kernel is truly unique in this particular aspect, in that they have thousands of contributors doing it on work time. I actually think
that the Rust project is doing pretty well at having people do it and get paid for it by their employer, but it's nowhere near the same extent as the Linux kernel. The Rust project also has a few other interesting things. So the Rust Foundation have some grants that allow, I mean, for example, if you're a student and you want to work on something, you might be able to get a grant, some money from the Rust Foundation. I think that's a super cool
thing that the foundation is doing, especially I think getting people involved, students involved, or the people who would find it harder or more daunting to get involved in a project like this. Speaking of which, for a software engineer who would be interested to contribute to the Rust ecosystem, what would your suggestion be in terms of both resources or ways to get started?
I mean, obviously you can go to the issue tracker and look for issues that interest you. I think that often the best way to contribute something is if you have something you want to change about it. I think that's often a pretty good starting point. So if you wanted to contribute to the Rust language itself, Rust does a lot of its stuff on something called Zulip, which is a chat server where people talk, and you could go there and talk to people.
The Rust for Linux project has a pretty nice contributing page. So, rust-for-linux.com, and there's a contributing page there with, for one, a few different drivers you might be able to contribute to, and they have links to the issue tracker there.
Another thing that's really cool is you can contribute to the Linux kernel without contributing to the Linux kernel, right? Because you might take a Rust language feature we need and implement it, and now you've contributed to the Linux kernel indirectly. Or maybe you want to work on the Rust in GCC project and help move that forward. And so I think there are a lot of different projects other than the Linux kernel that would help Rust in the Linux kernel a lot to contribute there.
Can we talk a little bit about Rust in the Linux kernel, on how that has evolved? How you got involved in writing Rust in the Linux kernel? And have you seen the approach of especially Linux kernel devs change or soften towards Rust? Because I know it was a very heated topic earlier on.
Linux has this conference called Linux Plumbers, which, as part of my work on Rust for the Linux kernel, I've been going to for the past few years. And one thing I think has been really obvious is every time I've gone, you can say things have totally changed since last year. I've experienced this like four times in a row.
That's pretty cool.
Yeah. I think there's really a lot of stuff that's happening. I mean, one year I might go and people think, "Oh, that's some nice little thing you have there, right?" And then the next year now they have Rust code in their subsystem. It kind of keeps going. The most recent Linux Plumbers from December 2025 was there. The big news was that at the Linux kernel maintainer summit, we agreed that Rust is no longer experimental in the kernel. That was really big for us compared to the previous year.
Yeah. So now that means that it's official, like Rust has the same status as C, which is the language that the kernel is written in, right?
I guess. Well, not the same status, but not being experimental. It clearly marks that it's more stable. Experimental sounds like it's not as stable. Basically, we've proved this is going to work.
I mean, based on what you've laid out with just memory safety, and of course being able to use unsafe when you need to, that already shows a lot of, I guess, promise. Plus there's some regulation as well, right? I think the US Department of Defense passed some regulations saying that they will not allow their agencies to use non-memory-safe languages for the concern of security vulnerabilities. And this would practically mean that Rust and other memory-safe languages could be used, but C, C++, or maybe one day systems that are built in these might see larger scrutiny.
Yeah, there have been different stuff like that from multiple different governments, right? These governments are saying, "You guys are using C or C++ in this project and it's causing an unacceptable amount of memory safety vulnerabilities that simply don't happen if you just don't. So what if you didn't do that?"
In this conversation, we managed to go for a very long time without even mentioning AI once, which, given where the world right now is and how these tools are spreading, that's pretty impressive. I was interested, do you use any of the AI tools for your day-to-day work building Tokio, contributing to Rust projects? Have you found a need for them at all?
So, I have been trying to use them. Honestly, I'm still learning how to use these tools, but I have used them. I quite like the Gemini command line interface. I think it's pretty neat.
These tools are pretty proficient at outputting code, or, you know, giving a prompt and it putting out code. But in your day-to-day work, how much code do you actually type as opposed to reviewing code, thinking about what to do, making a plan? As we were talking about the RFC process, for example, my sense was most of that was thinking about what to build, getting a consensus, making sure it's right, and it almost sounded like the actual code itself would be, I don't know, lesser. Of course, there's a lot of work involved, but you see what I mean. The actual time spent will probably be less than everything else around it.
I think it's still an important part of the process. I mean, a lot of people talk about that when you use AI, it's really important to write tests and that kind of thing, because then the AI is actually able to tell if it did it right. I explained this story from before about how I was refactoring something. I just did what the compiler told me. I think the same kind of principle applies with agents, in that they can talk to the compiler. It will tell them what to fix.
So I guess this could be a case. We'll see. But Rust could be a pretty promising candidate to use for agents, because they can get more feedback, and it's just harder to ship certain types of bugs, or maybe impossible to have certain types of bugs.
Yeah, I think there is an aspect of that.
For places like the Linux kernel, where correctness is extremely important, reliability is very important, there's already multiple layers of human review in place. Do you see potential for AI-generated code to prove helpful somewhere, like actually truly helpful, again assuming that these things can generate as per specification or as per what you need? Or could this be one of the places where we might not need that much? Because the kernel, to me, feels like a place where there's a lot of people coming in contributing, and there's a reason that change can propagate slower.
When I was at the Linux kernel maintainer summit, one of the topics that came up was using AI for code review. And there were some people there who had, for example, set up bots where, when you send an email to the mailing list, it would feed it into an AI agent which would leave a review. That kind of stuff. And for example, Linus and others were talking about how these reviews were actually really impressive for kernel code. At least in what's being discussed in the kernel community, that kind of use case seems like something people are excited about.
And it sounds like this would not be a replacement, obviously, but just one more way to get feedback, maybe doing it quicker, and just an additional safety net, if you will, right?
I mean, that's the thing with memory safety, right? You make some sort of trivial mistake. It's not some complex thing. It's just you make some trivial mistake, and there's a bajillion places you could make it. So even if you're a really good programmer and only make it 0.1% of the time or whatever, it's still going to happen, right? And so if the AI can catch a lot of those, then that's pretty good.
Well, and I guess some creative use cases could be, for example, tests are a big thing, but AI might be able to help figure out if there are edge cases missing, again if the language itself does not support it. From what I understand, there could be a lot of potential to add more safety nets that do not exist today.
I think that's right.
I like this approach. So this is one of the first conversations I'm having about AI where we're talking about how to increase quality with AI. Because, again, in startups or wherever people are building, there's a lot of talk about how it can make things faster, faster iteration, and there's usually a quality trade-off that we usually do not talk about. But I love that we're actually talking about the things that can increase quality, or at least keep it at the same level.
I mean, I have actually written a few patches with AI myself. It worked okay sometimes. I mean, it definitely required my review before sending it out to the list. But another place I remember using it was, I had this commit that made Binder faster in some way, and one of the review comments I had received was, "Hey, can you run a benchmark?" I said, "Oh my god, now I have to write a benchmark." But then I got the AI to benchmark it for me. And what it actually did was go and find an existing benchmark and modify it a little bit in a quite clever way. And then it ran it for me, and then it wrote a Python script to analyze the text file with the numbers and presented it in a nice table.
So it sounds like this was toil work that you could have done, but—
Yeah, of course.
—you might not have done as good of a job, because you might not have found that utility, you might not have figured out to do that.
I mean, at least in that instance, it saved me time, if nothing else.
For a senior engineer who would like to learn Rust, or just build with it, what would your suggestion be to get started? Of course, AI might be one of the things which makes it a lot easier just to get started, but if you would like to learn, what do you recommend for people who actually want to become proficient at the language?
So for one, I would say that the Rust book, which is on the official Rust website, is actually really good, right? And it's freely available online. Rust has this idea of calling tutorials books for some reason, even though they're online, which is kind of funny. But anyways, I think the Rust book is really good. Other than that, I think honestly the way you learn a language is you have a project. You implement some sort of project with it. I mean, maybe some sort of web server or something.
And one more thing on AI. So it's easy to get started with languages these days, because you can just prompt the AI to write this or that. In the case of Rust, do you think that could create a false sense of understanding? Especially as we talked about the importance of understanding structures, of understanding compiler issues which are there for a reason. Tools could just generate through this and create code that runs.
Yeah, it's totally a danger. I recently tried using it for something where I guess I was not writing Rust code, I was writing makefiles. I wanted to add some support in the Linux kernel build system for some feature, and it went into the makefile and added the Rust flags necessary to do it. But then I looked at the code, and it had added the necessary build flags, but on the C side it was passing a few more flags, which were not required per se, but they were there for a reason, and it had just ignored them. Any human looking at this would be like, why didn't you add Rust versions of all the flags?
What is your favorite Rust feature, if there is one?
I'm currently working on a new feature which I think is really exciting. It's something that we ran into in the Linux kernel where we needed in-place initialization: the ability to construct values while knowing where they're being constructed, so they don't get moved afterwards. I'm pretty excited about our work to put that into the language, but it's very much ongoing. I think that's pretty cool stuff.
And as a closing, what are books you would recommend that you've enjoyed, and why?
I already mentioned the official Rust book. I also think that Jon Gjengset has another book that I think is really good, and this book is kind of aimed at the intermediate Rust developer: the Rust developer who has gotten some Rust experience but wants to go further. I think that's also a pretty good book for that audience. Some other resources which are not books but which I think are also really good: there's this thing called Rustlings, and the idea is that they give you some unfinished Rust code and your task is to finish it.
Oh, nice.
And I think that's a pretty cool way to learn the language as well.
Awesome, Alice. Thank you so much for this. This was really interesting, and I've learned a bunch.
Thank you for having me on.
I hope you enjoyed getting into the details about Rust as much as I did. One thing I appreciated talking about is how deliberately Rust seems to have been designed around the question: what mistakes do programmers actually keep making, and how do we eliminate those mistakes? A few examples include memory safety, not having a null, and errors being values that you cannot ignore.
The documentation example is also an interesting one. Documentation tests, or doc tests, are automatically run as tests. If your example breaks, then your build breaks. This is the first time I've heard of this clever idea implemented in a way that's easy to use in a language.
Finally, the question of why Rust is becoming so popular might be because it's in a category that did not exist before: a language that's reliable and also performant at the same time. This makes it usable for high-performance computing cases like kernel work, while also being a good alternative for higher-level back-end languages.
Do check out the show notes for related The Pragmatic Engineer deep dives that go even deeper into back-end topics and stories of building languages like Kotlin and Swift. If you've enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show.
Article published