Grady Booch on the Third Golden Age of Software Engineering

Open on YouTube ↗
Overview

AI tools now write surprisingly good code, and many developers fear this means the end of software engineering. Grady Booch, co-creator of UML, a pioneer of object-oriented design, and a longtime IBM figure, argues the opposite. In this conversation on The Pragmatic Engineer, Booch walks through the history of the field as a series of rising levels of abstraction. He describes two earlier "golden ages" and a third that, in his view, began around the turn of the millennium. He places today's AI coding agents inside that third age: developers have faced this kind of existential crisis before, the tools are changing but the problems are not, and Dario Amodei's prediction that software engineering will soon be automated is, in Booch's words, profoundly wrong.

28 min read

What makes software "engineering"

Booch starts with the name itself. He credits Margaret Hamilton as probably the first to use "software engineer." She had just left the Manned Orbiting Laboratory project and was working on Apollo, one of very few software developers among mostly male hardware and structural engineers, and she wanted a term that set her work apart. The NATO conference on software engineering came a few years later. Booch notes that its organizers chose the name as a somewhat controversial one, much as "artificial intelligence" was controversially named for its first conference.

For Booch, what the name captures is that software people, like structural, electrical, or chemical engineers, build reasonably optimal solutions that balance static and dynamic forces. Perfect solutions are not available. Software is an extraordinarily fungible, elastic, fluid medium, but the same kinds of forces still act on it. He lists several:

  • Physics: information cannot travel faster than light, and hardware limits how large systems can grow.
  • Algorithms: sometimes a solution is known in theory long before anyone can implement it. His examples are the Viterbi algorithm, essential to cellular phones, and the Fourier transform, which could not advance progress until it became something computational.
  • People: Can you find enough people and organize them into teams? Booch jokes that the ideal team size is zero, the next best is one, and it grows from there. Some systems are so large, and so economically and socially important, that no individual could build them, and the software has to outlive the people who wrote it.
  • Law: digital rights management is one example.
  • Ethics: this is the force he considers most overarching. We know how to build certain things, but should we?

Balancing these forces, in a medium he calls wonderful, is why Booch says software developers are engineers. This definition matters for his later argument, because it is the basis of his claim that automating code does not automate software engineering.

Before software, and the first golden age

In the earliest days there was no software as such. Programming the ENIAC meant putting plugs into a plugboard, and hardware and software could not be told apart. Only in the late 1940s and early 1950s did the two begin to separate. Booch points out how recent this is: "digital" was coined in the late 1940s and "software" in the 1950s. If software were placed on Carl Sagan's cosmic calendar, he says, it would occupy the last few nanoseconds.

The early software was all bespoke and tied to a particular machine, mostly written in assembly language. But organizations were investing heavily in software and wanted faster machines without throwing that investment away. Booch credits people like Grace Hopper with seeing that software could be treated as a business and an industry in its own right. The turning point he names is IBM's move in the 1960s to a whole architecture of machines sharing a common instruction set. Hardware could now improve without discarding software. Booch describes this as an engineering, business, and economic decision at once, and says it opened the floodgates. That began the first golden age, which he dates roughly from the late 1940s to the late 1970s.

The central problem of the age was complexity. By today's standards the systems were almost laughably simple, the equivalent of "hello world," but they were hard problems at the time. Because software was still closely coupled to machines built mainly for mathematics, the dominant abstraction was algorithmic: think Fortran, "formula translation." The world was decomposed into processes and functions rather than data. Booch names Ed Yourdon, Tom DeMarco, and Larry Constantine as the era's defining figures and mentions the rise of entity-relationship ideas on the data side. Flowcharts were invented as aids to thinking. Labor divided into analysts, programmers, keypunch operators, and computer operators. Booch attributes this division to economics: machines cost far more than people, so work was organized to make the best use of rare machines.

Most software of the era automated existing business processes and numerical work. Companies had entire floors of people doing accounting and payroll, and that was the low-hanging fruit: automation made those processes both faster and more precise. To a programmer at the time, the center of gravity looked like IBM, insurance companies, and banks.

Innovation at the fringe: defense, SAGE, and the "loom of sorrow"

Booch stresses that the real innovation of the first golden age happened at the edges, especially in defense. Software was moving into aircraft, missiles, weather forecasting, and medical devices. Because Russia was, as he puts it, the clear and present threat, there was demand for distributed, real-time systems, while most business systems were not real-time.

His main example is the lineage from the experimental Whirlwind machine to SAGE, the Semi-Automatic Ground Environment, built in the 1950s and 1960s. Booch says the last installation was decommissioned around the 1990s. Before missiles, the fear was a fleet of Soviet bombers crossing the Arctic. The DEW line, a distant early warning system across Canada, fed data into SAGE. By some reports Booch cites, SAGE used 20 to 30 percent of all software developers in the United States. He adds that there were perhaps only a few tens of thousands of developers then, but it was still the largest project around. Graphical CRT interfaces, the ancestors of today's user interfaces, have their roots in Whirlwind and SAGE. He also mentions the "Mother of All Demos" as experimentation with human interfaces that sat outside the mainstream.

On the academic side, researchers such as David Parnas, C.A.R. Hoare, and Dijkstra were studying the formal properties of systems and treating software development as a formal mathematical activity.

Booch sums up the pattern with a phrase from the documentary on computing he is making: two great forces run through the history of computing, commerce and warfare. So "much of modern computing is really woven upon the loom of sorrow," an allusion to Jacquard's loom. The internet and microminiaturization both came out of government funding, and "we owe a lot to the Cold War." The broader lesson he draws is that software tends to grow: once we know how to build something and have patterns for it, we find economically interesting ways to apply it elsewhere.

The software crisis

Booch says cracks in the first golden age became visible in the late 1970s and early 1980s, and he describes the NATO conference as one of the first big public acknowledgments. NATO realized it had a software problem: insatiable demand, and no way to produce quality software at speed. That was the "software crisis." Asked what exactly the crisis was, Booch lists four parts. Software was expensive, slow to produce, and of poor quality, and demand kept growing. He notes this is a different kind of crisis from today's worries about surveillance or system crashes. The nature of the problems changes in every golden age.

Microminiaturization fed into this. Booch traces Silicon Valley's growth to the transistor and notes that Fairchild's first customer was the Air Force, mainly for the Minuteman missile. Most early Silicon Valley transistors went to Cold War programs, which built the economic base that led to integrated circuits and then personal computers.

By the late 1970s the US government saw a "problem of Babel." By its count, at least 14,000 programming languages were in use across military systems. Jovial was a popular one. ALGOL, not a military language, together with the formal work of Hoare, Dijkstra, and Wirth, led to applying mathematical rigor to languages. The government's answer was the project that produced Ada, which aimed to reduce the many languages to "one language that ruled them all." Booch says Ada absorbed the research of the time: abstract data types, Parnas's information hiding, separation of concerns, and Knuth's literate programming, which he relates to what we now call clean code. No other organization had the weight or economic power to push such ideas at that scale.

Objects versus processes, and the road not taken

Alongside this, Bell Labs had produced C and Unix. Booch describes Bjarne Stroustrup as a "crazy researcher" who wanted to bring ideas from Simula, which Booch calls the first object-oriented language, into C to address C's problems. The broader realization in academia and at the fringes was that algorithmic abstraction was not enough and object abstractions were needed too.

Booch finds this split in Plato: a dialogue in which one participant argues for seeing the world through processes and flows, and another through things. He connects the latter view to the Greek origin of the word "atom." Parnas and the designers of Simula, he says, applied this old philosophical dichotomy to software.

He also describes a third path. After Fortran, its inventor John Backus, who became an IBM Fellow, turned to functional programming, which views the world through stateless mathematical functions. Booch interviewed Backus a few months before his death and asked why functional programming never became mainstream. Booch reports the answer: functional programming makes it easy to do hard things but "astonishingly impossible to do easy things." Booch believes that is why functional programming still has a real but niche role today.

The second golden age: objects, PCs, and counterculture

Booch identifies three forces that pushed the field into its second golden age: growing complexity, the difficulty of building software big enough and fast enough, and the value of distributed systems that defense work had shown. The fruits of microminiaturization produced the personal computer. Asked whether this was the first time hobbyists could really get their hands on computing, Booch says yes, at least at scale. There had been earlier hobbyists; he mentions Pascal building a calculating machine to spare his father tedious accounting. But post-war disposable income in the US, plus the availability of military-driven transistors and chips from electronics shops in Silicon Valley, made hobby computing widespread. "Play is an important part in the history of software," he says.

He recommends the book What the Dormouse Said, which argues that the rise of the personal computer was tied to the hippie counterculture: "power to the people," Stewart Brand, the Merry Pranksters, and The WELL, which Booch calls the very first social network, a bulletin-board system. He speaks warmly of Brand, who has just released Maintenance: Part One, a book on the problems of maintaining systems, including software. The host notes that Stripe Press is publishing it.

Booch entered the field at this time, working at Vandenberg Air Force Base on missile and space systems, including an envisioned military space shuttle. He shares a few anecdotes: launches about twice a week, evacuating his building for Titan launches because an explosion on the nearby pad would have destroyed it, and secret spy-satellite launches that were never really secret, because local hotels filled with contractors and the highway filled with spectators.

By the late 1980s, he says, the world was ready for object-oriented programming and design. The key difference from the first age was the level of abstraction. Instead of a raw "lake" of data and separate algorithms to manipulate it, data and processes were brought together in one place. His favorite example is the source code for MacWrite and MacPaint, written in Object Pascal and viewable through the Computer History Museum, which he calls one of the most beautiful pieces of software he has seen. He says many of its design decisions still persist in systems like Photoshop, a story about how long software lives. The 1980s and 1990s were vibrant, with the "three amigos" (Booch, Ivar Jacobson, and Jim Rumbaugh) plus Peter Coad, and Constantine and Yourdon back on the scene. Booch acknowledges mistakes: the field overemphasized inheritance, which "was kind of wrong," though the core idea of classes and objects endured.

Open source, platforms, and moats

Booch describes reuse as a recurring economic pattern. In the first golden age, people kept rebuilding the same things: drum and disk handling, teletype output, screen output, sorting. These were codified and packaged. IBM SHARE, a user group whose members literally shared software, was in Booch's account the earliest open-source software. It was customer-driven, with IBM supporting it. At the time manufacturers essentially gave software away; IBM only began charging separately for software in the late 1960s and 1970s. In the second golden age the same thing happened at a higher level of abstraction, for example libraries for writing to "newfangled CRTs" that gave no competitive advantage to the owner but let everyone build more.

Growing libraries and distributed systems led to service-oriented architectures. Booch recounts how the web grew beyond passing links: Netscape added images to HTML, and people wanted to pass messages over HTTP, so the internet became a higher-level medium for moving information and processes. SOAP and service-oriented protocols followed, which he sees as the predecessors of today's platform era: islands surrounded by APIs. Asked to define a platform, he points to AWS and Salesforce: "economically interesting castles defended by the moat around them," with access across the moat sold for a fee that is "not even a slight fee." Such businesses exist because building the capability yourself is so costly and complex.

Booch recalls getting his first email address in 1987 on the ARPANET, when a roughly hundred-page book listed the email address of everyone in the world. As email and similar things became commodities in the second age, software moved into "the interstitial spaces of civilization." The first age's problems were largely solved and became part of the atmosphere. "The best technology evaporates and disappears and becomes part of the air that we breathe."

Y2K, the dot-com crash, and AI's parallel history

Around 2000 came the dot-com crash, as many internet businesses did not make economic sense, and Y2K. Booch pushes back on the retrospective view that Y2K was a non-event: from inside, he says, there was a lot of heroic work, and without it many problems would have occurred. He calls it a good example of the best technology being invisible: money and effort prevented a problem that never showed up. The host remembers the stress of that New Year and how people came away distrusting such predictions.

Booch then fills in a parallel history of AI with its own ages. The first, in the 1940s and 1950s with Herbert Simon, Allen Newell, and Marvin Minsky, focused on symbolic methods. Neural networks were tried too: the SNARC used about five vacuum tubes per artificial neuron. A UK report concluded that the money was not producing results, and the age ended with the belief that neural networks were a dead end. Booch attributes this to missing computational power and missing algorithmic abstractions. The second AI age, in the 1980s, centered on rules and inference engines, alongside a lively period in hardware such as the Lisp machine and Thinking Machines. It too ended in an AI winter, according to Booch because it did not scale beyond a few hundred if-then statements.

The third golden age began around 2000

Booch's somewhat controversial claim is that the third golden age did not start with today's AI but around the turn of the millennium. Its sign was another rise in abstraction: from individual components to whole libraries, packages, and platform services. Need messaging? Use a library. Need to manage a large body of data? Use something like Hadoop, which he notes was not yet around then but whose seeds were growing. Methodologies and languages followed.

He sees AI coding assistants as a reaction to that growth. So many libraries exist that not enough people know how to use them, and tools like Cursor and ChatGPT speed up their use. In his framing they continue forces that already produced the third age.

The problems of this age differ from earlier ones. First, there is so much software that managing it is itself a problem. Second, safety and security: it is easy to inject something into the software supply chain, and he cites Stuxnet as an example of espionage through software. Human issues that software was once insulated from are now front and center. Third, economics: companies such as Microsoft and Google are too big to fail, and if they sneeze, part of the world catches a cold. Fourth, ethics: it is possible to track where someone is at every moment of the day, but should we?

"This is not the first existential crisis"

The host describes a sense of existential dread among engineers that grew over the recent winter break, when new models went from decent autocomplete to generating code good enough that the host began to trust it. Because coding has been so closely tied to the profession, many developers are asking what comes next.

Booch's answer is that developers have faced the same crisis in the first and second ages, and "this too will pass." His advice to worried people is to focus on fundamentals, because those skills will not go away. He recalls meeting Grace Hopper, whom he describes as a "fireplug" of a person, and recommends her appearance on David Letterman's show. Her recognition that software could be separated from hardware threatened early machine builders, who argued that nothing efficient could be built without being tied closely to the machine and wrote that it would destroy their work. The same fears arose with Fortran, when assembly experts insisted they could write tighter code than any compiler; Booch says moving to higher-level languages proved them wrong. Those people saw that their skills would be replaced by the very thing they had helped create. The difference now, Booch says, is scale: then it was a few thousand people, now millions are legitimately asking what this means for them.

When young developers ask whether they chose the wrong field, he tells them it is an exciting time, because the field is again moving up a level of abstraction: from machine language to assembly, to higher-order languages, to libraries, and now this. He finds it freeing to be relieved of the tedium while the fundamentals remain. The fundamentals matter when building software meant to endure. For throwaway software, "do what you want," and he sees many people using agents to automate things that were never affordable before, especially for a single user. He compares this to the hobbyist side of early personal computing: good ideas and skills will come from it, even if much of it will not last.

The host gives an example: a neighbor who is an accountant had ChatGPT build Apps Script tools for their accounting team, personal throwaway software from someone outside the field. Booch celebrates this. He recalls that artists and gamers were drawn to early PCs and the Amiga as a new medium of expression. Much of the current lamenting, he argues, comes from people narrowly focused on their own part of the industry who miss that the industry is expanding, with more software written by non-professionals.

Booch's response to Dario Amodei

The host brings up Anthropic CEO Dario Amodei. According to the host, Amodei predicted about a year earlier that around 90 percent of code would be AI-generated, which people dismissed but which the host considers to have come true. Amodei now says software engineering will be automatable in 12 months. The host points out that coding is only a subset of software engineering.

Booch first says he uses Claude and calls it his go-to system. He has used it with JavaScript, Swift, PHP ("of all things"), and Python, mostly to learn unfamiliar libraries, since in his words Google search and the documentation are both poor. But he stresses that he brings decades of experience with the fundamentals, and that in every engineering discipline the fundamentals remain while tools change.

He then puts Amodei's statement in context: Amodei leads a company that must make money and must speak to its stakeholders, so outrageous statements get made; Booch believes this one was made at Davos. Booch calls it "utter" nonsense, trailing off before the final word and calling it the "technical term," and says Amodei is profoundly wrong. He accepts that AI will accelerate some things but denies that it will eliminate software engineering, because in his view Amodei misunderstands what software engineering is. Engineers balance the forces Booch described at the start, and code is only one mechanism. None of what Amodei or his colleagues describe addresses those decision problems; their work automates the lowest levels, which Booch likens to what compilers did. "Your tools are changing, but your problems are not."

His second objection concerns scope. Tools like Cursor are trained mostly on problems solved over and over again, such as a UI over CRUD or web-centric systems, just as libraries grew up around the recurring problems of the first age. Paraphrasing Shakespeare, he says there are more things in computing than are dreamt of in Amodei's philosophy: computing is much larger than web-centric systems at scale, and a great deal remains unautomated. Third, he describes these agents as good at generating patterns, which he sees as a new abstraction: societies of objects and algorithms that work together, describable in English.

English as a programming language, and the D3 example

Booch addresses the obvious objection that English is imprecise, ambiguous, and full of nuance. His reply is that engineers already work this way: they tell someone "I want the system to do this, it looks kind of like this," give examples, and someone else turns it into code.

His concrete example involves the D3 JavaScript library, which he had never used. He found a site called Victorian Engineering Connections, built for a museum by a man named Andrew, where you can enter a name such as George Boole and explore that person's social network interactively. Booch wanted something similar. The creator gave him the code, which used D3. Booch asked Cursor to build the simplest possible version, with about five nodes, so he could study the code, then asked it to change the nodes' appearance by type. He was expressing needs in English as he would to a person, and the tool shortened the distance between what he wanted and a working result. He calls this a breakthrough, with the caveat he raised against Amodei: it works where the task has been done hundreds of times before. He acknowledges Feynman's view that doing it yourself is the only way to understand, but says there is too much in the world he is curious about to learn it all.

He closes the point with a definition: a language precise and expressive enough to build executable artifacts is a programming language. English is good enough, much like COBOL was, in well-structured domains, producing good-enough solutions that people who know the fundamentals can nudge and clean up. That, he says, is why fundamentals matter.

What becomes obsolete, and what matters more

The host notes that every jump in abstraction made some skills obsolete, such as hand-optimizing for a particular instruction set, and asks what will happen this time. Booch points first to the software delivery pipeline, which he says is far more complex than it should be. Getting something running is hard without a pipeline, and companies like Google or Stripe have huge custom infrastructure. He treats infrastructure as software and sees it as low-hanging fruit for agents, where automation brings clear economic value and security value. He expects job losses there and a need for people to reskill. He also expects losses among those who build simple applications, such as a straightforward iOS app, since people can now prompt their way there. He calls this fine, because it hands a new generation what professionals used to do, as happened with PCs.

His advice to those affected is to move up a level of abstraction, from programs and apps to systems. People who can manage complexity at scale and balance the many forces, human as well as technical, will not lose their jobs. If anything, he expects greater demand, because those human skills are rare and delicate.

Foundations: systems theory, brains, and multi-agent architecture

Asked which foundations people should study, Booch says his "happy place" when facing a hard problem is systems theory. He recommends Simon and Newell's work, particularly The Sciences of the Artificial, and the Santa Fe Institute's work on complexity and systems.

He illustrates with his work on NASA's Mars missions, thinking about long crewed missions and robots on the Martian surface. He realized NASA essentially wanted to build HAL, and he keeps a HAL replica behind him as his "sword of Damocles," a reminder to stay humble. He jokes that NASA did not want the "kill all the astronauts" use case. He framed the problem as systems engineering, because the intelligence had to be embodied in the spacecraft, whereas most AI today, Cursor, Copilot and the like, is disembodied with no connection to the physical world. His work focused on embodied cognition. Around the same time he studied with neuroscientists to understand the brain's architecture, and began to see structures from systems engineering that apply to very large systems.

For today's agent programming, which he thinks people are only beginning to understand, Booch points to earlier work on multiple agents: Minsky's Society of Mind; global workspace ideas associated with Baars and the blackboard architectures in early AI systems such as Hearsay; and Rodney Brooks's biologically inspired subsumption architecture. He observes that a cockroach has no central brain yet does magnificent things, and that the entire neural network of a common worm has been mapped without producing "evil worms running around the world." Something more is happening, and biological systems have architecture. Looking at architecture through biology, neurology, and real-world systems, as Simon and Newell did, is what guides his thinking about the next generation of systems. "There is nothing new under the sun in many ways."

How to thrive at the start of a new age

The host observes that agents are spreading everywhere, citing reports that they will be part of Windows 11, and asks what people who thrived at the start of earlier golden ages did. Booch returns to imagination. Software begins with imagination, which is almost unlimited, and is constrained by physics, algorithms, ethics, and cost. Now some of that friction and cost is disappearing, so people can focus on imagination and build things that were impossible before, because they could not have raised a team, afforded it, or had the reach. There will be losses for those with a vested economic interest in the old ways, he says, but he considers it a net gain.

He admits it is frightening as well as exciting, and says that is how it should be. On the cusp of something wonderful, you look into the abyss and either decide you will fall in or decide to leap. "This is the time to soar."