Resources

People often ask me "How did you learn how to hack?" The answer: by reading. This page is a collection of the blog posts and other articles that I have accumulated over the years of my journey. Enjoy!

Commonware is a set of primitives for building a blockchain, similar to that of the Cosmos SDK. In Commonware, fuzz tests are first class tests, treated as a source of truth like unit and integration tests, with over 60 confirmed bugs found via this. This article describes how they use fuzzing to keep bugs out of the code.

Most fuzz targets use LibFuzzer, and the arbitrary crate. Instead of completely random input being deserialized into types, arbitrary handles that for you. This allows for more time exploring meaningful test cases rather than simple deserialization failures.

Controlled randomness is a big part of fuzzing. Normally, it sort of sucks to deal with because the changes with random values won't be incremental, which breaks the coverage maps. To fix this, the randomness is effectively a tape that can get changed. When doing weird things, like forcing faults, this can be useful for good coverage of the system.

They have several types of fuzzers. The first one is primitive fuzzers, or writing fuzz tests for small parts in isolation. Parsers are a good use case for this. Next, they have protocol fuzzers, which are similar to integration tests but with fuzzing. These target harder to engineer tasks with more state.

They also have specification based testing. When writing fuzzers, it's common to write bugs in your fuzzer. So, how do you determine if something is a bug in your fuzzer, or the implementation itself? We need an oracle to do this. The author of the posts uses LLMs to create oracles of the expected functionality/specification to determine whether something is a bug or not. The LLM is not the authority; the written code from the LLM is.

Where do these tests actually live at? To start with, the fuzz tests are in experimental/security research branches on the repository. After being ran for long times on the fuzzing infrastructure, the tests are merged. They split the running tests into two groups: production, and research. The production fuzz tests are ran continuously, intending to find, reproduce, and regress bugs. The research ones are tests that are not in CI but are being considered for it.

Overall, a good blog post on fuzzing lifecycle from AR.