Behind the research: Why AI still needs human assistance to fix security vulnerabilities

The large language models (LLMs) powering your favorite AI tools can handle a growing range of tasks, from writing code to planning your next vacation. They're also incredibly capable of finding software vulnerabilities.

Companies already use AI to discover flaws in their products, systems, and infrastructure. Public debate has focused on who should have access to the models that perform best at bug hunting, and what happens when attackers get hold of them.

Just as important, though, is whether AI can patch the vulnerabilities it uncovers. Many teams hope AI will help them issue security fixes faster and keep pace with the growing number of vulnerability disclosures. The idea is simple: use AI to keep up with AI-assisted vulnerability discovery.

Off-by-1 Labs, 1Password's security research team, has found that AI models effectively patch only a narrow subset of vulnerabilities. Keith Hoodlet, director of security at 1Password, joined the Random but Memorable podcast to discuss the team’s first research paper and explain why human expertise remains essential to vulnerability remediation.

Think of the interview below as a director’s commentary on the research. You can also read the full paper and explore the team’s tooling and datasets.

Editor’s note: This interview has been lightly edited for clarity and brevity. The views and opinions expressed by the interviewee don’t represent the opinions of 1Password.

Wade Wells: What did you want to find out with this research?

Keith Hoodlet: The first question we asked as a research team was: How capable are large language models (LLMs) at producing security patches that don’t alter the application's behavior and don’t introduce a new vulnerability in the process? 

The reason we asked this question was that Anthorpic’s Claude Mythos had just broken onto the scene and made a pretty big splash. And Mozilla issued a massive Firefox patch the month after Mythos was released. 

I thought: Okay, we're all about to get hit by a tsunami of vulnerabilities. And people will naturally reach for large language models as a coding solution to this problem. 

Knowing some of the research presented by Veracode, which found a consistent 44% rate of new vulnerability introduction in AI-generated code, it made me wonder: How capable is AI of editing existing code rather than building something new? 

We found some interesting results that have changed a lot of people's opinions on what AI is capable of today.

WW: What did you expect to find?

KH: We hypothesized that for the open-source code bases we were using as part of our target data set, LLMs would have a roughly 67% true positive rate, or true patch might be the better term here, of producing something robust that actually solved the problem. 

The reason we hypothesized this was because, at least for the open-source code, it was part of the training dataset that went into building the large language models’ capabilities for writing software. 

But we looked at six complex vulnerabilities, meaning they touched multiple functions, in some cases multiple files, and were high impact. All of them were CVSS (Common Vulnerability Scoring System) 8.0 or above and new enough that they weren't part of the training data set overall. 

So these were novel vulnerabilities that weren’t included in the training data set. But the codebase itself was part of the training data set, so we hypothesized that we would see a decent patch-rate success of around 67%. We were very wrong.

“We hypothesized that we would see a decent patch-rate success of around 67%. We were very wrong.

WW: What was the headline result of the research?

KH: We were wrong. Very wrong.

I'll describe each scenario and then give you the percentages. So we looked at five different scenarios, S1 through S5, where the best case was S1 and the worst case was S5. 

Scenario one successfully patches the vulnerability and does not alter the application's behavior. That is the best-case outcome. That happened only 26% of the time out of the 6,000+ patches we generated.

Scenario two means the AI patches the vulnerability, but it alters the application's behavior. That could be like an allow list becomes a deny list, or the AI adds new parsing functionality locally to the application. That happened around 20% of the time. Still a successful patch, but the application's now acting a little weird, so not great.

Scenario three is that the AI doesn’t successfully patch the vulnerability. This is like rearranging deck chairs on the Titanic. The code may have changed a little, but the vulnerability is still there. That was 49.3% of the time. So by itself, nearly half the time, you generate a patch, and it's still exploitable for the same vulnerability you were trying to patch.

“Nearly half the time, you generate a patch, and it's still exploitable for the same vulnerability you were trying to patch.”

Scenario four is the AI patches the vulnerability by adding a new one. So you now have an off-by-one error, or you have an out-of-bounds read or write, or any number of vulnerabilities we observed. Thankfully, that only happened 2.3% of the time. 

The worst-case outcome was that the AI doesn’t patch the vulnerability and adds a new one. That’s like double or nothing. That happened 2.1% of the time, so when you add everything up and look at scenarios three through five, that was around 53.9% of the time. 

I will also add one caveat for scenario one. When you look closely at those patches, around 9% are patched very specifically to the proof of concept. So we would call that a fragile patch because if you change the proof of concept or the exploit even a little, you're still hitting the same exact vulnerability. So when you boil it down, robust security patching generated by an AI across more than 6,000 patches generated in this data set was only truly around 17%. That's a wide gulf from the 67% we expected.

WW: Which AI models did you test, and why did you choose them?

KH: We started testing on May 20th and used the frontier models that were widely available to defenders at the time. So we were using OpenAI’s ChatGPT 5.5 with Trusted Access for Cyber guardrails, as well as Anthropic’s Claude Opus 4.8 with Cyber Verification Program guardrails.

WW: How did you choose the vulnerabilities used in the study?

KH: We wanted a few characteristics. One, the vulnerability had to be new enough that it was unlikely to be part of the training data set of the models themselves. 

Second, the vulnerability had to be high impact. It had to be one that an entity, whether open-source or private, would want to patch quickly. So CVSS 8.0 or above.

The third characteristic was that it had to be complex enough that it wasn’t a single line or word change to patch the vulnerability. So we were looking for issues that were usually multi-file, multi-function. 

Copy Fail was one of the vulnerabilities that made it into our research set because it met all of the criteria. What’s interesting is that the correct way to patch Copy Fail was to remove the functionality entirely. They didn't add or change any code; they just removed it. And so it was an interesting test for AI.

Then it was a matter of deferring to my team and asking: Which vulnerabilities are you most interested in researching? Because as humans, we needed to get familiar with the vulnerability, how the exploit worked, and what the successful patch looked like. Because we weren't going to take the large language model's word for it. We manually validated a subset of the patches that were created to ensure the AI wasn't cheating and that the patches were being graded correctly by our various models.

“ We manually validated a subset of the patches that were created to ensure the AI wasn't cheating.”

WW: Where did the models struggle most when they tried to produce complete security fixes?

KH: It's becoming clearer, but it's not perfectly clear. 

Something we observed, which is fundamental to how large language models work, is the attention mechanisms they operate from. We found instances where it would patch a particular set of code when a character-for-character replica of that same vulnerability exists elsewhere in the file. It could be right next to where the AI was patching, and it would never patch that same character-for-character vulnerability elsewhere in the file.

“It would patch a particular set of the code when a character-for-character exact replica of that same vulnerability exists elsewhere in the file.”

This is where some of the struggles may exist and continue to persist. There are mechanisms and engineering that have gone into making these things operate the way that they do, such as attention mechanisms. If the attention mechanism is patching so strictly to a very specific area of a file, without also changing other areas that have the same code pattern and vulnerability, that's a really big problem. It means you're not patching categorically; you're patching in singletons. 

How many times do you have to do that until you get a fully clean code base? That’s the outstanding question that a lot of people are going to have to ask.

WW: What needs to happen before organizations can start trusting AI tools for vulnerability remediation?

KH: Chris Wysopal is a founder and chief security evangelist at Veracode. He’s talked about having a hardened, secure, known-good data set to train models to write only clean, secure, good code. Maybe that produces the outcomes we're looking for, but I suspect it'll be very expensive to curate and maintain that data set. I also think the capabilities of large language models will be stunted if they have a limited set of training data to work with compared to the world's available code bases. 

We plan to do some testing and eventually some reporting around harnesses. A lot of people ask the question: What if you take the code you've generated as a patch and send it back through a vulnerability discovery, critic judge, POC-exploit creation loop? Does that solve the problem? And the answer is we don't know. Maybe. But the other question I have is: How many times do you need to go through that process to produce a 99%, true-positive, robust patch outcome?

In the back of my mind, I also wonder whether some vulnerabilities are an infinite compute problem where you never get a good patch with today's large language models.

The larger lesson is that Secure by Design, which security professionals have been saying for over a decade, is now truly the most important investment you can make. That means thinking about your architectural and design principles so your applications are secure, regardless of how an attacker might interact with them. That approach matters far more than just patching the vulnerability in front of you.

“Secure by Design, which is something security professionals have been saying for over a decade, is now truly the most important investment you can make.”

WW: What should businesses do today to find and fix vulnerabilities before attackers can exploit them?

KH: I’ll use the analogy of medieval times. If you were being invaded by a Viking army, sometimes what you'd do is slash and burn your wheat fields so that the opposing army couldn't feed itself using your grain. But if you were a neighboring kingdom, you didn't care. You might think: That’s unfortunate for them, but we're just going to let our wheat grow and harvest it when it's ready. Now, unfortunately, those Viking armies are enhanced by AI systems, and they're attacking everybody all at once. 

The best thing you can do is slash and burn the vulnerability wheat fields growing in your company.

“Slash and burn the vulnerability wheat fields growing in your company.”

You have technical debt. If you know about it, burn it down because the attackers are at the door. The question I will often ask leaders is: How much time are you willing to invest in fixing the vulnerabilities uncovered by agentic vulnerability discovery loops? Nobody loves the answer to that question because sometimes it's a lot of time. But now people are saying: Well, can we apply AI to fix these problems? 

My other recommendation is the FLAWED tooling we released.Go back to a vulnerability you patched 6 to 12 months ago that was a known-good patch, perhaps reviewed and approved by a human. Run different AI models through it with FLAWED and determine how successful a patch outcome is going to be for that kind of vulnerability in your company’s code base.

Sometimes you might get really great results. Other times you might find 0% success rates. But you’re going to learn where you absolutely need to apply human knowledge, where you can take a little risk on an AI-generated patch, and how much time you're going to allocate for a human to review. 

But as Anthropic quoted in our blog post for the research: 

“Patch generation has outpaced patch verification, and the fix is to make verification execution-grounded rather than inspection-based, while keeping domain experts as the final reviewers at current model capabilities.”

A human reviewer has to be the final step of anything you're doing with agent-generated patching. We strongly agree with Anthropic’s recommendation.

So go and burn down the vulnerability backlog if you've got it, because the Vikings are coming.

WW: Can you tease what the team is working on next?

KH: We're continuing our research with AI-generated patching. We’re looking at the rest of the Patch the Planet patches that have been publicized by Trail of Bits because it's a publicly available data set and patches we can observe and run through our processes. We're looking at that and additional vulnerability disclosures from that work.

We have our own vulnerability harness called Vulnfinder or Vulnhunter, or V8R for short. And we are refining it to make it more efficient in its vulnerability discovery capabilities, as well as more performant from a scaling perspective.

We have high true-positive finding rates right now with some really impactful vulnerabilities at a pretty low price point. So we're continuing to refine and work on that. More vulnerability disclosures are forthcoming from that work, which we’re likely to release and present sometime next year.

WW: Finally, where can people go to read the research paper or learn more about the work your team is doing?

1Password.com/research brings you to our page for Off-by-1 Labs. You'll see some links for the latest research, and there are some Easter eggs if you scroll down the page. We’re also likely to be at some conferences this fall, so there’s more to come.