Dotnet-slopwatch – detect when AI coding agents "fix" problems by cheating
Slopwatch is a .NET tool designed to detect AI coding agents' shortcuts in code changes. It identifies patterns of 'reward hacking' where AI tools may disable tests or suppress warnings instead of addressing issues properly. By integrating with CI/CD pipelines, Slopwatch helps maintain code quality by catching these problematic behaviors before they enter the codebase.
- ▪Slopwatch detects LLM 'reward hacking' behaviors in code changes.
- ▪It can be installed globally or locally and requires a baseline to analyze new issues.
- ▪The tool integrates with Claude Code and CI/CD systems like GitHub Actions and Azure DevOps.
Opening excerpt (first ~120 words) tap to expand
Slopwatch // LLM anti-cheat A .NET tool that detects LLM "reward hacking" behaviors in code changes. Runs as a Claude Code hook or in CI/CD pipelines to catch when AI coding assistants take shortcuts instead of properly fixing issues. What is "Slop"? When LLMs generate code, they sometimes take shortcuts that make tests pass or builds succeed without actually solving the underlying problem. These patterns include: Disabling tests instead of fixing them ([Fact(Skip="flaky")]) Suppressing warnings instead of addressing them (#pragma warning disable) Swallowing exceptions with empty catch blocks Adding arbitrary delays to mask timing issues (Task.Delay(1000)) Project-level warning suppression (<NoWarn>, <TreatWarningsAsErrors>false</TreatWarningsAsErrors>) Bypassing Central Package…
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.