Prompting for Patches That Fix Vulns Without Adding New Ones - Keith Hoodlet - ASW #394

There's already an increase in volume of security flaws found by LLMs. And orgs are already turning to LLMs to write code. So, what happens when orgs lean on LLMs to create patches for those security flaws? Keith Hoodlet gives an exclusive early look at his team's recent research into the success, quality, and failures of LLM-generated security patches. Notably, they saw scenarios across a spectrum from robust, effective patches to patches that changed the software's behavior to patches that introduced new vulns to patches that didn't even fix the original vuln while also introducing a new vuln.

The research considers factors like quality and correctness of prompts, complexity of the target software, programming language, and expertise required to understand what a robust patch should look like. If you're going to spend tokens on fixing security flaws, you want a feedback loop that fixes them correctly -- not an infinite loop of new flaws creeping in with every LLM iteration.

Watch for this research, its toolset, and data to be released on Thursday August 6th during Black Hat.

Show Notes: https://securityweekly.com/asw-394