LLMs Are the Key to Mutation Testing and Better Compliance
Meta has developed an LLM-based tool called Automated Compliance Hardening (ACH) that enhances mutation testing to improve software compliance and security. By generating relevant code mutations and corresponding tests, ACH helps developers proactively identify compliance-related bugs and reduce manual effort. The tool simplifies risk assessment and supports scalable, continuous compliance in complex software systems.
- ▪Meta's Automated Compliance Hardening (ACH) tool uses large language models to automate mutation testing for compliance.
- ▪ACH generates both code mutants and tests to catch them, improving the effectiveness of software testing at scale.
- ▪The tool enables proactive identification of compliance risks, such as privacy violations, before code reaches production.
- ▪Meta has presented ACH at FSE 2025 and EuroSTAR 2025, highlighting LLMs' role in advancing automated software testing.
- ▪Mutation testing with ACH goes beyond traditional coverage methods by evaluating whether tests actually detect faulty code behavior.
Opening excerpt (first ~120 words) tap to expand
POSTED ON SEPTEMBER 30, 2025 TO AI Research, ML Applications, Security & Privacy LLMs Are the Key to Mutation Testing and Better Compliance By Mark Harman Following our keynote presentations at FSE 2025 and Eurostar 2025, we’re delving further into the development of Meta’s Automated Compliance Hardening (ACH) tool, an LLM-based tool for software testing that is automating aspects of compliance adherence at Meta, while accelerating developer and product velocity. By leveraging LLMs we’ve been able to overcome the barriers that have prevented mutation testing from being efficiently deployed at scale. This allows us to greatly simplify risk assessments, reduce cognitive load for developers, and, ultimately, create a safer online ecosystem by enabling continuous compliance.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Engineering at Meta.
Discussion
0 commentsMore from Engineering at Meta
