What the hell are we doing?
The article discusses the stagnation in fuzzing research, suggesting that contributions are often incremental and lack clarity in their utility. It highlights a recent fuzzing competition where two contestants demonstrated different approaches, yet the winner's methods were deemed less technically interesting. The author calls for a reevaluation of how fuzzers are assessed and emphasizes the need for more meaningful metrics in the field.
- ▪Fuzzing research has stalled due to a focus on general improvements rather than identifying specific areas for enhancement.
- ▪In a recent competition, two fuzzers showcased different strategies, with one winning despite being less technically advanced.
- ▪The author argues that current evaluation metrics may obscure the true contributions of fuzzers and calls for a reevaluation of assessment methods.
Opening excerpt (first ~120 words) tap to expand
I have come to realise---or rather, I have become more and more convinced that---fuzzing research has stalled not because we have no further contributions to make, but because the contributions that we are making are either incremental and merely sound impressive or presented in ways that obscure their utility. To be more concrete: we are spending time trying to "improve" fuzzing generally rather than identifying what can be improved; everyone is trying to be "the best" rather than trying to identify what is actually happening. This is not the first time that I have felt this, but perhaps my understanding of this problem has improved in the last two years. It's time for a revisit! Last year, I was involved in a paper which tried to standardise fuzzer evaluation.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Addisoncrump.
Discussion
0 commentsMore from Addisoncrump
