How AI Is Eroding Japan's Language Barrier: Google's John Hultquist on the New Shape of Cyberattacks
TBS CROSS DIG with BloombergJapanese companies have recently reported a string of unauthorized access incidents, data leaks, and system disruptions. In this episode of TBS CROSS DIG's "1on1 Tech," business editor Masahiro Nakagawa asks John Hultquist, chief analyst at Google Threat Intelligence Group and a veteran of more than 20 years in cybersecurity, what is driving these attacks and how AI is changing them. Hultquist's position is consistent throughout. AI has moved from a translation and advice tool to an autonomous operator for attackers. Japan has lost some of its traditional protections. Defenders need to use the same technology, and use it now, while the threat is still at an early stage.
What Google Threat Intelligence Group Does
Hultquist describes his group's job as tracking threat actors around the world through many channels. One major source is Mandiant, the incident response company inside Google. When an organization suffers a serious cyber incident, Mandiant goes in to work out what happened. Because this happens constantly and globally, it produces a steady flow of intelligence about attacker behavior. The group also looks for threat actors in Gmail, "increasingly now" in Gemini, and in the criminal underground, using other methods he does not detail.
He explains why this matters with an analogy. Doing cybersecurity without knowing what attackers actually do, he says, is like "building walls for an enemy that's got airplanes." You can't build a solution until you know what you're facing.
Why Japan Is a Target Now
Hultquist says the recent incidents in Japan do not look coordinated. He believes they are probably the work of separate financially motivated actors who see a large opportunity. Japan has many major brands and a lot of valuable data, which makes extortion attractive. He also points to "tech debt."
He makes a broader point about how countries harden their defenses. Some nations go through a series of major breaches and harden their systems as a result. In his view, Japan has not had that experience. He says he told colleagues on a previous visit that Japan had not suffered the same serious breaches the US had, and he thinks this made organizations more reluctant to make major security investments. Attackers, he says, are now exploiting that lack of investment.
The second factor is language. Hultquist argues that the language barrier protected Japan for a very long time, because it was hard for outsiders to master Japanese or pass as Japanese. He thinks that advantage "might be eroding" now that anyone can get translation through a model.
Nakagawa asked how much of the current activity is linked to AI. Hultquist was careful: much of it "would have happened anyway." His best evidence is that criminal threat activity appears to be dropping or plateauing in the United States while rising elsewhere. He thinks attackers are always looking for new fertile ground, and working inside the Japanese language is simply a new opening for them.
He also notes a regional factor. Many phishing kits built specifically to compromise Japanese users are coming out of the Chinese underground, and that activity is increasing. When asked whether these Chinese actors use AI, he said it is "probably a major part of it." He added that there is no threat actor now that isn't using AI somewhere in its operations, because the technology is good at too many of the things they do every day.
Three Stages of AI in Attacks
Asked what AI has changed most, Hultquist describes a progression in three stages.
Social engineering. The earliest use he saw during intrusions was deception, especially through translation. He names North Korean actors as early adopters. They consistently target Japan, including with a scheme to land IT worker jobs in the country. That scheme requires a lot of conversation during job hunting, and his team has seen them use AI to carry it out.
Technical troubleshooting. In the last few years, models became able to answer highly technical questions accurately, and they are especially good at troubleshooting. "Hacking is really just technical troubleshooting," Hultquist says. His team began to see actors using Gemini for help in the middle of intrusions, including what he calls a serious cyber espionage actor. He says they would not ask outright for help stealing credentials from a network they were in. They would phrase it cleverly, for example by posing as red teamers or as someone troubleshooting, to fool the model. If the model answered without its controls triggering, he says, the answer could be very strong. "Models are really good at hacking, unfortunately."
Agentic operation. The third stage, which he says defines the current era, is agents. Models can now do this work without close human involvement. Attackers can send agents out on their behalf and need only minimal involvement themselves.
Keeping Attackers Out of Gemini
Hultquist confirms that threat actors have tried to use Gemini to carry out cyberattacks on multiple occasions. Google runs an operation to keep them out, similar to how it keeps them out of Gmail and other Google properties. The group tracks actors globally and disrupts their operations whenever they appear. He says it is important to Google to keep the technology "in the hands of the good guys."
Nakagawa pointed out that a prompt alone can't reveal whether someone is an attacker or a defender, and asked whether tracking relies on things like IP addresses. Hultquist agreed that this is a good example. The team watches for indicators of compromise (IOCs) and other known attributes of specific actors so it can shut them down when they enter Google's technology.
Every Model, Open Models, and Stolen Compute
Asked which other models attackers use, Hultquist said he has seen them use "just about every model you can imagine." They still use frontier labs' products, including GPT, whenever they can. The drawback is that commercial products have many controls. Running an operation through one also means "somebody else gets to see your entire operation," which spies and criminals want to avoid. So many actors also use open or local models that nobody else can see.
He highlights a growing trend: stealing compute. Attackers opportunistically compromise someone's server, install a local model, and run it on the victim's expensive processing power. This gives them tokens for free. Hultquist calls this a big concern: while legitimate users pay for tokens, attackers who steal access may gain "a bit of a token advantage."
Nakagawa asked whether attacks require frontier-level models. Hultquist said "not anymore." Many open models are now capable enough to handle different parts of an attack. He says it is still rare to see a single intrusion run end to end, "soup to nuts," by one system. Attacks are still carried out in portions, which lets actors mix different models.
Asked whether giant agents are already coordinating entire operations, he said there have been some highly orchestrated cases, but they still likely involved many models. He expects things to move toward fuller orchestration: "we'll get there." He gives two reasons attackers currently split operations up. Running everything on one model creates a single point of failure. And state actors, especially espionage actors, want tight control so they don't trigger detections on the way in, since getting caught ruins the operation.
Agentic Attack Tools and Open-Weight Models
Nakagawa mentioned reports that a Chinese agentic attack tool had been used against Japan and against Korean financial institutions the previous month. Hultquist said tools of this kind have been showing up often in incidents and in chatter. He named one tool that appears in many incidents (its name is unclear in the recording) and another called HexStrike, which his team has seen used in Japan. As these tools coalesce, he expects they will make this work very easy and let almost any threat actor "uplevel" its operations. That means defenders will soon face much faster, larger-scale operators.
Nakagawa then raised open-weight models, citing China's GLM 5.3. He noted that AI labs such as Anthropic had argued that releasing such models causes harm by putting them in attackers' hands, and asked whether Hultquist agreed. He did not take a position on the broader debate. He said attackers will obviously prefer models that let them work without monitoring or controls, but that he can't judge the other externalities. His point was narrower: attackers always want to stay under the radar, and if an option is strong enough, they will use it.
What Japanese Companies Should Do
On priorities for Japanese leaders, Hultquist called this "a once-in-a-lifetime opportunity" to get ahead of the threat. He expects it only to scale up and become more serious. He said the best time to act was yesterday and the second-best time is today. He described his approach as "never waste a crisis." Because organizations can now see how serious the threat is, they can invest against it. He even called the timing "kind of ideal," since this is only the beginning. Acting now gives organizations a chance to get ahead of a much larger threat and to pay down the tech debt he mentioned earlier.
What Individuals Can Do
Hultquist said most of this activity targets enterprises, not individuals, although individuals' information gets swept up in it. He gave two ways individuals can still be drawn in.
First, attackers using models often obtain a random person's credentials and then use that person's account to carry out what can become a major incident. "You don't want to be a part of that," he said. His advice is to lock down accounts so that you are the only one using them.
Second, he said state threat actors in Japan very often come through small office and home office routers. Everyone can help by power cycling home routers and keeping their firmware updated so third parties can't exploit them. These steps may not directly protect the individual, but they make things harder for the adversary.
Who Is Attacking Japan, and the Extortion Business Model
Hultquist said state-backed hackers from China, Russia, and North Korea have long been interested in Japan and are active worldwide. Most of the recent incidents, however, come from financially motivated cybercriminals whose business model is extortion.
He described how that model works. Because attackers can demand payment directly in cryptocurrency, any stress they put on a target can be turned straight into money. They only need enough pressure to get a payout. He traced the history in steps:
- Criminals first targeted individuals by encrypting single computers with ransomware.
- They realized companies cared more and paid more.
- They then found that leaking data could be a more powerful lever than disruption alone.
Today, he said, they increasingly leak data rather than only disrupting systems. They will use anything that adds pressure, including threats against CEOs, which his team has seen more often, and other disruptions such as DoS attacks.
How Google Defends Itself
Asked how Google handles attacks at its scale, Hultquist said Google, like every major organization, is targeted, including routinely by opportunistic actors. A core practice is using intelligence to anticipate what to expect, then reviewing Google's own infrastructure to address those weaknesses. Inside Google, he said, you can't really discuss next steps in security without first looking at what attackers are doing: who they are, what they want, and what is likely coming. He suggested organizations elsewhere need the same approach. Google faces this challenge internally too, because the technology is changing fast and it has to keep up.
Nakagawa cited reports that Google's security teams were already using Gemini to fix vulnerabilities. Hultquist said Google has been doing this kind of work for a few years. He described a program called Big Sleep, built to find vulnerabilities. It started on models he says we would now call "prehistoric," much less sophisticated than today's. What made it effective was a strong harness of code built around the model by vulnerability and exploit experts.
Google first used Big Sleep to review its own code, then open-source code, and began finding very serious vulnerabilities. When it reported them to open-source projects, it received a lot of negative feedback. The reason was that the more vulnerabilities Google found, the harder it became for maintainers to keep up. The lesson, Hultquist said, was that finding vulnerabilities isn't enough. AI also has to help fix them. That led to CodeMender, which Google runs internally and offers externally. It identifies vulnerabilities and proposes options for fixing them. "If you're not doing both sides," he said, you're not getting the full benefit of the technology.
Self-Replicating AI and Threats to Critical Infrastructure
Nakagawa asked about the risk that AI could launch attacks on its own or copy itself onto systems it has compromised, referring to a report Anthropic had published a few weeks earlier. Hultquist said he had not read the report. He acknowledged that self-replicating, worm-like malware has long been a concern and that intelligent capabilities make it a real threat. The hard part, in his view, is access to a model. Such malware can't operate independently without access to "the brain," and he said it remains to be seen how that plays out.
The conversation then turned to physical harm, prompted by a recent cyberattack on a French supermarket food logistics system. Hultquist explained that attackers who are genuinely skilled at targeting critical infrastructure are very few. "We know their names... we know where they live," he said. They come from around the world and generally have deep technical training, often in industrial control systems, which he described as obscure technology and "a whole another level of nerd."
His concern is that AI opens this obscure technology to many more people, because it can walk them through it. He gave a personal example. He is building a robot at home, something he says he has "no business" doing because it is far outside his expertise. Working with a model, he can do things he never could before. It isn't perfect, but he says he is making major progress. He expects the same to apply to complex critical infrastructure systems. Attackers who didn't understand these systems before can now use a model to guide them, so the threat to critical infrastructure "is absolutely growing."
His response is that these systems have relied heavily on "security through obscurity," meaning they were secure mainly because they were hard to understand. Stronger controls will need to be put in place and funded, including controls that never existed. He also called for a broad view of what counts as critical infrastructure. People think first of power plants, but the French case shows how essential systems like food logistics can be overlooked.
Who Has the Advantage
Asked whether attackers or defenders benefit more from AI, Hultquist said that in theory attackers always move first. They lack the bureaucracy and the safeguards that defenders have, which he noted exist for good reasons. For defenders, moving quickly requires money and investment, and he said that unfortunately this usually doesn't happen without serious incidents first.
Attackers also have a large freedom to experiment. A bank robber risks jail or injury if things go wrong, but a cyberattacker whose operation fails can just restart it. That flexibility helps them learn new technology first.
Still, Hultquist said he has been "pleasantly surprised" by how some organizations have responded on vulnerabilities. After major discussion of models' ability to find vulnerabilities, many organizations took it seriously and are already running their code through major models for review. He called this "absolutely fantastic" and a cornerstone of the future approach. The other challenge is what he called the "swarm effect" of scaled-up attacks. Defenders will need to raise their game, probably using AI themselves, and he said how fast that happens "is really up to us."
Five Years Out: Good and Bad Scenarios
In the good scenario, Hultquist said, systems are locked down and models and agents handle many security tasks that have long been done at human speed and scale. With those in place, he believes defenders will have "a fighting chance" to keep up with the latest models. Without them, "we're just going to be overwhelmed."
In the bad scenario, a massive series of intrusions leaves "all of our data out... outside of our own control." He called this very realistic if action isn't taken now.
Nakagawa asked what would happen if all systems were disrupted. Hultquist, who has spent much of his career on disruptive attacks, answered calmly: generally, "we'll be fine." It would be a massive nuisance, and the biggest effect of many disruptions is psychological. But people will find a way to get groceries where they need to go, the water will come back on, and the trains will run again. He added that it would still be bad for economic outcomes and for people's lives, and that this is why it should be prevented.
Oney
Music.
Okay, we have John Hultquist, chief analyst at Google Threat Intelligence Group. Thank you very much for your time today.
Thanks for having me.
You've been in the cybersecurity industry for more than 20 years.
That's right.
You've seen a lot of cyberattacks and defense, both sides. So I think you're the right person to ask questions about cyberattacks these days. But before we go into the details, could you tell us quickly about what Google Threat Intelligence Group does day to day?
Our job is to essentially track these threat actors all over the world through a variety of means. Mandiant is a company inside Google that does the incident response. When somebody has a major problem or incident, a cyber incident, they call us, and we go in and figure out what happened. There's a lot of intelligence you can glean from that process, especially if you're doing it all over the world all the time.
We look for threat actors in places like Gmail and increasingly now in Gemini. We look for them in the underground, the criminal underground. And we have all kinds of other tricks and ways to track them continuously. The point, though, is that if you're trying to do cybersecurity and you don't know what threat actors are actually doing, you're just kind of building walls for an enemy that's got airplanes, right? You can't possibly build a solution until you know what you're facing.
Okay. In terms of what's happening in Japan, Japanese companies have been reporting a lot about unauthorized access or data leaks or system disruptions. From what you can see, what is happening right now?
There have been a handful of incidents, or quite a few incidents lately, but it all looks not entirely orchestrated or coordinated across the different actors. Probably separate financial actors that are targeting Japan because there's a huge opportunity. There are a lot of major brands here, and there's a lot of valuable data, and that means they could probably extort some of these targets for payouts. And there's a lot of tech debt, right?
One of the interesting experiences that some countries go through that others don't is they go through a series of major incidents, and as a result they harden a lot of systems. Japan's not necessarily had that experience. In fact, the last time I visited, I mentioned that to some of my colleagues, that they've not had the same serious breaches that we've had. And I think that made people more reluctant to make some of those major investments in security. And threat actors are taking advantage of that lack of investment.
There's also room here for the possibility that models are somehow playing a role too. And I think that one of the things that has protected Japan for a very long time has been the language barrier. It's hard for outsiders to master the language or pretend to be Japanese. And I think that's been a huge advantage for a very long time, but that advantage might be eroding in an era when you can get translation services through a model.
How much of the attacks do you think are linked to AI as of now?
I think a lot of this would have happened anyway. Probably the best evidence I have of that is we can actually see threat activity from criminal actors dropping or plateauing in the United States while it's rising elsewhere. I think threat actors are always looking for new fertile ground, and because they can now work inside this language, this is just a new opportunity for them.
There's some other factors too. It's interesting, we see a lot of phishing kits, which are tools for compromising specifically Japanese persons. A lot of these Japanese-language phishing kits are coming out of the Chinese underground, which is on the uptick as well. And so you might have a regional threat as well adding to this problem.
So are Chinese doing that with AI?
I think that's probably a major part of it. There's not a threat actor out there now that's not using AI as part of their operations. There's too many things that it's good at that they do day to day, and there's too many opportunities for them to leverage it.
What has changed the most about cyberattacks because of AI? What was it like before AI, and what is it like now after AI?
When we first started seeing the technology being used in the midst of intrusions, the real earliest use case was social engineering, using it to fool somebody, using it through translation. And one of the real early players was North Koreans, and they are always targeting Japan. For instance, they have this scam where they're actually trying to get IT worker jobs in this country, and that involves a lot of them talking to people as part of the job-hunting process. And so we can see them using AI to conduct that process.
But in the last few years, new capabilities have come online. The biggest one was suddenly you could use a model to answer really technical questions and get really strong, accurate answers out of it. And they're particularly good at technical troubleshooting. Hacking is really just technical troubleshooting, right? And so we started to see actors in Gemini trying to get Gemini to give them answers in the midst of intrusions.
So you literally have a serious cyberespionage actor asking Gemini for help in the middle of an intrusion. 'Hey, I'm in this network. I need to steal these credentials.' They would say it like that, but they would say it in a clever way.
They pretend to be troubleshooting.
Yeah. And sometimes they would pretend to be red teamers or something like that, and they'd try to fool the model. But if the model could give the answer without the controls firing, it could give a really strong answer. Models are really good at hacking, unfortunately.
But the third piece, the era that we're in now, is the agentic piece, right? It's not just that it's good at this kind of work. It's now it can do this work without us being involved, or highly involved. And so that means threat actors can send models out into the world, or agents out into the world, on their behalf, and they only have to be minimally involved.
So you mean Gemini has been used to do the cyberattacks?
We have seen threat actors in Gemini attempting to leverage it to carry out cyberattacks on multiple occasions. We actually have an operation to keep threat actors out of there. That's a big part of what we do.
How do you do that?
The same way we keep them out of Gmail or any other Google property. We track them globally, and whenever they show up, we essentially disrupt their operations. It's really important to us that we keep this technology in the hands of the good guys, and we work really hard to do that.
Because you never know if this is an attacker or defender only by prompt. So you track them by IP address, or?
Exactly. That's a good example. You can look at what we call IOCs, or some of the attributes that we recognize from them, and we're always looking for them to enter our technology so that we can shut them down.
Okay. So besides Gemini, what models are they using right now?
I have seen them use just about every model you can imagine. In some cases they're using frontier labs. They do still use frontier labs.
GPT?
Yeah, the whole range. They still use the frontier labs whenever they can. But the problem with frontier labs is there are a lot of controls involved in those. And if you essentially orchestrate your activity through a commercial product, the downside is that somebody else gets to see your entire operation. And for spies and criminals, that's not ideal, right? And so many threat actors are also using open models or local models where they can carry out an operation without anybody else's visibility.
Increasingly, we can see them essentially stealing compute. So they'll go out and compromise a server and then drop a local model on that.
Steal from who?
It's anybody. You opportunistically get access to somebody's server, and they use their compute, their very expensive processing power, to run your own model. And that basically gives you the ability to use tokens for free. That's a big concern of ours, that while we're paying for tokens, threat actors are increasingly stealing their access, which may give them a bit of a token advantage.
Do the models have to be frontier level to do cyberattacks?
Not really. Not anymore. I think many of the open models are at the level where they can do different pieces of the attack. And it's rare to see an incident where it's one absolute soup-to-nuts intrusion, right from the very beginning all the way to the end. They're still carried out in portions, and so that's an opportunity to use different models and mix things up. It's not unusual for a threat actor to use all kinds of different models for that.
So there's no giant models or giant agents coordinating whole operations. It's more separated?
There have been some cases where we've seen pretty highly orchestrated things, but it's still probably used a lot of different models. We're moving in that direction, right? And we'll get there. But for different reasons, threat actors separate different pieces, probably. One of them is if you run the entire thing across one single model, then you have a single point of failure.
In some cases, state threat actors, like cyberespionage actors, want a lot of control over the operation because they don't want to necessarily light up a lot of detection on the way in, because then you get caught and your operation goes downhill.
It has been reported that attackers are using the Chinese agent attacks against Japan as well, but they've been used to attack Korean financial institutions last month. So how do you see the—
That's a tool that has actually been showing up quite a bit lately. We've been seeing it in incidents, and I've heard quite a bit of chatter around it. I think it's the tool du jour that's used in a lot of incidents. There's another one called HexStrike that we've seen used in Japan. Ultimately, as these tools coalesce, it's going to make a lot of this work really easy for just about any threat actor to really up-level their operations. It also means that we are going to be facing a much faster, scaled-up operator very soon.
As you said, there are a lot of open-weight models for attackers to use. For example, there's China's GLM 5.3, and AI labs like Anthropic said that it caused harm for us to put them into the hands of the attackers. Do you agree with that, and how do you see the risk of open-weight models?
Well—
It's basically available to anyone.
Yeah. I think obviously threat actors are going to go use a model that essentially allows them to carry out their operations without being monitored or without running into controls. And so they are probably going to prefer that option. But I can't judge all the other externalities and parts of that discussion. I will say that they are always looking to stay under the radar, and if an option is strong enough, they'll use it.
Okay. When it comes to Japanese companies facing a lot of challenges right now, how should the leaders prioritize their activities to defend themselves right now?
I think this is a once-in-a-lifetime opportunity to get ahead of this threat, right? It's only going to scale up, and it's only going to become more serious. The best time to deal with this was yesterday. The second best time is today. I believe in the 'never waste a crisis' philosophy here, that we can see how serious this threat is, and we can invest against it.
It's actually kind of ideal that we've seen this now, or we're dealing with this now, because this is just the beginning. There's an opportunity for us to get ahead of a threat that's going to grow much more serious and really address some of that tech debt that I mentioned earlier.
Okay. When it comes to individuals, what precautions should they take in this time of cyberattacks?
That's a good question. I think most of this is about the enterprise, really. This stuff targets the enterprise.
But enterprises' customers are individuals.
Yeah, but their information gets involved. But it's strange. Some of these things cross over strange places, right? A good example would be one of the ways that threat actors are carrying out these operations using models is they gain access to a random person's credentials. And then they use your account to carry out what could be a major incident. You don't want to be a part of that. So one of the things you can do is really lock down these accounts so that you're the only one using them.
One of the other interesting things that takes place right here in Japan is state threat actors very often come through small office and home office routers. And there's actually a role for everybody to play in literally just making sure our routers at home have been power cycled and the firmware is updated so that they can't be taken advantage of by a third party. So even though this affects us but isn't necessarily something that we can take a lot of action on, there are little things we can do to make it harder for the adversary.
So who are the attackers against Japan right now, anyways? Are they state-backed or more like individual hackers?
Yeah, there are definitely state-backed hackers that have always been interested in Japan. They come out of China, Russia, North Korea, and those actors are active all over the world, but most of these recent incidents are
Really are cyber criminals, financially motivated cyber criminals, and their business model is essentially extortion. They recognize that they can essentially directly monetize any kind of stress they can put on a target. And what I mean by that is because they can ask you to pay them through cryptocurrency directly. All they have to do is find some way to put enough pressure on you to pay out.
Now, historically, they actually started by targeting individuals. They would ransomware just a single computer, but over time they realized that companies actually cared more and they would pay more when it came to these incidents. And then they realized that leaking was actually potentially a more powerful tool when it came to getting somebody to pay out.
So increasingly they're leaking our data, not just disrupting our systems, but ultimately anything they can do to put pressure on a target, they will do. And that includes threats, which is something we've increasingly seen against the CEO, or other types of disruption like a DoS attack.
Yeah.
They will go across the entire range of activity to get somebody to pay.
Okay. And Google has been a target as well, right? Google has billions of users worldwide and a lot of information about individuals. So how does Google handle those scale of attacks, and how has it been changed?
Well, we're targeted like every major organization is targeted by threat actors. Everybody gets targeted by opportunistic threat actors regularly. But I'll tell you, one of the big things that we do is this threat protection mechanism, which is using intelligence to understand what we can expect and looking at our own infrastructure and making sure we're addressing all those problems.
Inside Google, you can't really have a discussion of what to do next with security without looking at what the bad guys are doing first, right? And we probably need to do that here as well, right? Who are these guys, and what are they after? And how can we address all those problems, and what can we anticipate? Because it's going to change very soon. That's a problem that we're facing internally because we recognize that the nature of the technology is changing rapidly, and we have to keep up.
Yeah. So Google said the security teams already used the Gemini 4 Argon to fix the problem, the vulnerabilities. So could you walk us through how Google defends itself by using its own models?
That's a great question. We've actually been doing this for a few years now. One of the really exciting programs, they started using, honestly, models that we would probably call prehistoric at this point, just very old models. They built a program that was basically designed to look for vulnerabilities called Big Sleep. And what it was was a much less sophisticated model than the ones we are used to now, but it had a really good harness built around it. And that's basically all this code that was designed by vulnerability and exploit experts.
And we went out and started looking at our code to review it. And we started looking at code in the open source, and we started finding very serious vulnerabilities. In fact, we started reporting it to some of these open-source places, and we were actually getting a lot of negative feedback. And the reason we were getting negative feedback is the more we found vulnerabilities, the harder it was for people to keep up.
And so the lesson we learned from that is it's not just about finding these vulnerabilities. You have to use AI to fix them as well. And so we have a program now called CodeMender that's running internally, and we're offering it externally. But the point is it essentially not only identifies these vulnerabilities, but it gives you options for fixing them as well, right? If you're not doing both sides, you're really not using the technology to the full benefit.
Okay. Well, as agents are doing more on their own, I think there's people talking about the risk AI could launch a cyber attack on its own or copy itself on the system it has hacked and keep running on those devices. I think I've seen one of the reports that Anthropic submitted a few weeks ago. So how real is that risk? How do you see that?
I haven't read the report myself. But I think obviously the idea, when you have capability that is intelligent, right? One of the concerns that we've always had is the ability to self-replicate, or what we call a worm, like worm malware. That is certainly a threat concern. The hard part, though, for that is always where is it going to get access to a model, right? You can't just operate independently without access to essentially the brain. And I think that we'll have to figure out how all that shakes out.
And we just witnessed the cyber attacks on the French supermarkets and refresh system a little while ago. So as AI evolves, are we seeing more physical harm like that? Then how do we get ready for that?
Well, that's another thing. So I will tell you that the threat actors that were generally good at targeting critical infrastructure, the ones that are really good at it, there's very few of them. We know their names. We know who they are. We know where they live. There's just not that many people who do that kind of work very well, right?
Who are they?
Well, they're from all over the world, but they're generally steeped in very, very technical training, right? A lot of them work on industrial control systems. It's very obscure technology. It's a whole other level of nerd that is good at this stuff, right? And there aren't that many of them. The issue with AI is that it really opens up these opportunities or this obscure technology to more people because it can walk you through that technology, right?
I'm at home. I'm building a robot now. I have no business building a robot. That's so far outside of my expertise, but I found that working with a model, it's able to walk me through things that I would have never been able to do before. It's not perfect, but I'm definitely making major progress.
The same is going to apply to those really complex critical infrastructure systems. Threat actors who didn't understand them before now can use a model to walk them through that. And that means the threat to critical infrastructure is absolutely growing.
That's a critical risk. So how should we get ready for that?
Well, we're going to have to do a better job of securing that technology. For a long time, it's really been what we call security through obscurity, which is it's just hard to understand, and so it's more secure. But we're going to have to actually put stronger controls on a lot of this technology and probably fund a lot of stronger controls that weren't there in the first place.
There are a lot of places that we probably think of first when we think of critical infrastructure, like the power plants, right? But there is a lot of other critical infrastructure we take for granted. A great example is what we saw in France. That was the food logistics system. Extremely important to our day-to-day life, but it's not something that we generally think of first. We'll have to take a broad view of that.
So, as AI strengthens both attackers and defenders, who has the advantage today?
Well, I will tell you, theoretically, I think attackers always move first, right? They don't necessarily have the same bureaucracy. They don't have all the safety safeguards that we do, which are there for a good reason. Right now, us moving quickly is going to take money. It's going to take investment. And that's not going to happen, I think, unfortunately, generally doesn't happen without some serious incidents first.
Threat actors also have a tremendous ability to experiment.
Yeah.
If you rob a bank, you could get caught. There's a good chance that you will get caught and you'll go to jail or you'll get hurt in the process, right? If one of their incidents goes wrong, they can just restart it, right? And that gives them the ability to be more flexible and learn to use this technology first.
We are going to have to work hard to close that gap. And I will tell you, I've been pleasantly surprised on the movement of some orgs on certain things. The vulnerability problem. There was a massive discussion about models and their ability to find vulnerabilities, and I can tell you a lot of organizations took that very seriously, and they're already putting their code through these major models and reviewing it. That's absolutely fantastic, and that's going to be a cornerstone of our approach to the future.
The other one's going to be dealing with this swarm effect, right? We're going to have to uplevel our game, and we're probably going to have to use AI to do that. And it's really up to us how fast that goes.
Yeah. Okay. Last question. When you look out for, I would say, the next five years, what is a good scenario and what is a bad scenario?
Yeah. I think the good scenario is that we've locked things down and that we are using models and agents to do a lot of the security tasks that we've been doing at human speed or at human scale for a very long time. If we can get those things in place, I think we'll have a fighting chance to keep up with the latest models. If we don't, I think we're just going to be overwhelmed.
What's a bad scenario?
Well, a bad scenario is, I think, a massive series of intrusions that essentially we end up with all of our data outside of our own control. And that's very realistic if we don't take action now.
And if all of our systems are disrupted, what will happen?
Well, listen, I spend a lot of time focused on cyber attacks and disruptive attacks. And I'll tell you, generally, we'll be fine. It's fine. We'll be fine, right? Our life, it's going to be a massive nuisance, but we'll get along just fine. Generally, the biggest threat when it comes to a lot of disruptions is psychological. But we'll figure out some way to get the groceries where the groceries need to go, and the water will come back on, and the trains will run again.
But it won't be good for our economic outcomes, and it certainly won't be good for our lives. So we want to prevent all that.
Yeah. Okay, great. I think I have to let you go. Thank you very much for your time.
Yeah.
Article published · Updated
