Come pulire una cosa sporcandone un’altra…But how effective are LLMs at producing patches without altering the application’s behavior? The paper’s title is Frontier Models’ Vulnerability Patches are Often F.L.A.W.E.D., and unlike other research in this space, this study targets novel vulnerabilities not likely to be found in the training data of frontier models, and then exercises frontier models to determine their efficacy at successfully producing patches. The average success rate for generating a patch that fully resolved the vulnerability (without materially changing application behavior) was just 26.0%. Patches that successfully resolved the vulnerability, but altered the application’s behavior in the process, occurred 20.1% of the time. Conversely, LLM-generated patches did not resolve the vulnerability, added a new vulnerability, or both, an average 53.9% of the time.