Resources

People often ask me "How did you learn how to hack?" The answer: by reading. This page is a collection of the blog posts and other articles that I have accumulated over the years of my journey. Enjoy!

Monad has engaged with many of its ecosystem partners to create a fuzzing framework for Solidity smart contracts. This came from an audit-readiness campaign that was trying to identify when a project is ready to be externally audited. Because of tight time constraints, most tests now-a-days are written with AI. After taking some time to review the blind spots of the tests, they decided that fuzzing was the answer to their problems.

In less than a week, a particular project was extended to support stateless fuzzing with invariant tests. From a top-tier auditing firm, this likely takes 4-8 weeks to do, depending on the size of the codebase. Fuzzing is a great way to catch low-hanging fruit, and for preventing stupid bugs from getting committed once they have been added. Why not just use Claude Skills to review the code? It's just a different workflow; different strategies find different bugs.

From doing this by hand on the initial project, they wanted to automate the writing of the fuzz tests in some capacity. So, they created various skills to do this to have a single pipeline that could create the fuzz testing framework. The simple steps were A) setup Foundry, B) understand the protocol, C) create fuzz tests based on this information, and D) dedup, triage, and report findings.

Unfortunately, this initial approach didn't pan out well. The skills always ended too early, not writing enough tests. In the end, there was just too much to consider. The different strategies, and steps made the context unmanageable; context rot was a major problem for quality. The next attempt was an autoresearch loop. This ran each iteration of the harness through a scoring system, and had the fuzzer get better, and better over time. Because of issues in scoring (lots of different tweaks didn't work), they determined this wasn't viable either.

The goal was a one-shot skill but this didn't work out: it turned into a single binary that people can run with the ability to configure the prompts, and goals. The UltraFuzz campaign has four phases: setup, properties (invariants), strategies, and a final review state. setup is learning about the project, preparing the Foundry harness, and a few other things to put the project into a good state.

The Properties enumerate what system invariants need to be upheld. This builds on other projects invariant findings, dedups them, and uses these. The third step strategies is the most important of all: actually writing the tests. There are five kinds of tests: parametric user flows, round-trip properties, differential tests, property-based, and stateful invariant fuzzing. From these five, there are about twenty strategies.

The harnessing around how to set this up is interesting. They ran the test generation three times in different worktrees; most of the time, only a single iteration found a particular bug. They use different models as well. In the final stage, deduplication, adjudication, and report findings are made.

From the fuzzing of various projects with this, they found 22 clear production bugs, and 5 underspecified findings. 2 high, 15 medium, and 10 low. From writing the strategies, they learned that many of them depend on protocol-specific properties. So, it's worth the time to create custom strategies tailored for the codebase, instead of just the generalized rules. The cost was about $230 and six hours to do this, much faster and cheaper than hiring a company.

Overall, a great article on the usage of AI alongside fuzzing to find bugs. I appreciate the process of failures shown as well. The knobs that they turn with multiple attempts, a single CLI, and other things is also very interesting. Good blog post!