Great, I’ll look for academic papers, blog posts, GitHub repos, and conference talks where Crytic’s Medusa has been used or referenced. I’ll also find projects or studies comparing it—especially in terms of speed and accuracy—with other popular EVM-compatible smart contract fuzzers like Echidna, Foundry, and Mythril.
I’ll update you once I’ve gathered a solid set of sources you can use for benchmarking.
-
Olympia (ASE 2024) – Benchmarking smart-contract fuzzers: Researchers introduced Olympia, a tool to generate Solidity benchmarks with known bugs, and evaluated four fuzzers (Echidna, Foundry, Medusa, ItyFuzz) under equal conditions (Olympia: Fuzzer Benchmarking for Solidity) (Olympia: Fuzzer Benchmarking for Solidity). In a 1-hour run per fuzzer on 200 generated contracts, ItyFuzz solved the most (147/200), significantly outperforming the others (Running Halmos by Default for Library Testing - All things fuzzy) (Olympia: Fuzzer Benchmarking for Solidity). Echidna found 83/200, Medusa 58/200, and Foundry 31/200 bugs within the time limit (Olympia: Fuzzer Benchmarking for Solidity). This indicates ItyFuzz’s superior bug-finding accuracy in that benchmark, while Medusa’s effectiveness was intermediate (between Echidna and Foundry). The study also noted variation in fuzzing speed: Echidna and Medusa took anywhere from seconds up to ~59 minutes to hit certain bugs, whereas Foundry found fewer bugs but all its findings occurred within ~20 minutes (Olympia: Fuzzer Benchmarking for Solidity). (Foundry’s faster runs might reflect limitations in deeper state exploration.) Overall, Olympia highlights how Medusa performs competitively but not yet better than Echidna in bug coverage on generated test cases (Olympia: Fuzzer Benchmarking for Solidity).
-
Viggiano (DeFi Security Summit 2023) – Foundry vs. Echidna on DeFi invariants: A comparative study by Antonio Viggiano focused on how quickly fuzzers breach invariants in mutated Uniswap v2 contracts (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness). Over 12 test cases (24-hour campaigns each), Foundry broke invariants faster than Echidna in 9 of 12 cases, while Echidna was faster in 1 case (the remaining 2 saw no significant difference) (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness). Foundry’s edge in speed for most tests suggests its design (Rust-based, integrated with Forge) can sometimes generate failing inputs more quickly (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness). However, the paper emphasized high variance between runs and called for including additional fuzzers in benchmarks (noting tools like Medusa and Harvey as promising contenders) (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness) (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness). In short, Foundry often showed faster time-to-bug (speed), but the study advocated evaluating newer fuzzers like Medusa to get a fuller picture of performance and accuracy (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness).
-
Wu et al. (ICSE 2024) – “Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers”: This comprehensive study reviewed the research landscape and empirically evaluated top fuzzers’ usability and performance ([2402.02973] Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers). The authors introduced pragmatic metrics covering throughput, code coverage, bug finding, oracle support, etc., and tested multiple tools. One finding was that current fuzzers are heavily CPU-bound – executing EVM bytecode in a single thread becomes a bottleneck (). Reported throughputs ranged from just 166 exec/sec for Echidna up to 33,000 exec/sec for ItyFuzz, with most others far behind (e.g. ILF ~57/sec, sFuzz ~512/sec) (). This aligns with the notion that increasing coverage requires exponentially more CPU power (). The study also surveyed practitioners: 14 of 16 auditors reported using Echidna in their workflow (praising its support for custom invariants), versus only 2 using Foundry and 1 using ItyFuzz ([2402.02973] Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers). Notably, auditors valued features like ease of use, good docs, and custom property support ([2402.02973] Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers) ([2402.02973] Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers). They highlighted a desire for “online fuzzing” (using live on-chain state) – at the time only ItyFuzz provided this, allowing detection of more real-world bugs via stateful, cross-contract interactions ([2402.02973] Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers). (Medusa’s designers appear to have addressed this by adding an on-chain fuzzing mode seeded from blockchain state (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog).) In summary, this 2024 study didn’t explicitly include Medusa (it was too new), but it set expectations for high throughput and better state-handling, areas where Medusa aims to improve the status quo.
-
GPU-Accelerated Fuzzing (2024) – Towards fuzzing on GPUs: Recognizing the execution bottleneck above, recent research has explored GPU parallelism to speed up smart contract fuzzing. Weimin Chen et al. (2024) observe that over 70% of a fuzz campaign’s time is spent in EVM execution (Step ➁), which all existing tools run in a single CPU thread () (). They developed MAU, the first GPU-based smart contract fuzzer, which compiles EVM bytecode to GPU kernels for massively parallel fuzzing (). In their experiments, MAU achieved 8.7× to 15.3× higher throughput than the fastest CPU fuzzer (ItyFuzz) (). This dramatic speedup translated to finding about 1.01×–2.50× more bugs and covering 1.03×–4.71× more code than the baseline fuzzers in the same time (). These results underline that speed (executions per second) has a direct impact on fuzzers’ bug-finding accuracy, and suggest that a GPU-enhanced version of Medusa could similarly uncover more vulnerabilities given the order-of-magnitude higher execution rate. (The need for such optimization is clear, as conventional fuzzers’ throughput is limited by the slow EVM interpreter and limited CPU parallelism () ().)
-
Trail of Bits Internal Benchmark (2025) – Medusa vs. Echidna: When Trail of Bits announced Medusa v1, they shared results from an extensive internal head-to-head with Echidna (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog). After fine-tuning Medusa’s mutational strategies, the two fuzzers showed comparable coverage and corpus growth rates over time (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog). A sample coverage chart (Medusa’s line vs. Echidna’s) indicated that Medusa could keep up with Echidna in exploring complex contracts and finding inputs that increase coverage. In other words, Medusa achieved parity in the core metrics of fuzzing effectiveness, while offering new advantages in scalability. Trail of Bits attributes this to Medusa’s design – written in Go and built on a Geth EVM engine – which allows parallel fuzzing across multiple workers and better maintenance as a modern codebase (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog) (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog). They’ve made Medusa their primary fuzzer going forward, noting that it “represents the state of the art in smart contract fuzzing” and will receive new features, whereas Echidna will only get minor fixes (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog). This indicates strong confidence that Medusa can match or exceed Echidna’s performance (accuracy) while improving speed via multi-threading (since Echidna runs single-threaded by design).
-
ConsenSys Diligence Benchmark (2023) – Harvey vs. Echidna vs. Foundry: ConsenSys Diligence (Valentin Wüstholz’s team) performed an 8-hour fuzzing benchmark on maze-like contracts to compare their in-house fuzzer Harvey (which powers Diligence Fuzzing service) against Echidna and Foundry (Benchmarking Smart-Contract Fuzzers | Consensys Diligence) (Benchmarking Smart-Contract Fuzzers | Consensys Diligence). The outcome: Harvey found ~43% more unique bug-triggering transactions than Echidna, and ~25% more than Foundry, on these challenges (Benchmarking Smart-Contract Fuzzers | Consensys Diligence). It also found them faster – Harvey needed only ~19 minutes to achieve the same number of invariant violations that Foundry found in a full 8 hours (Benchmarking Smart-Contract Fuzzers | Consensys Diligence). This highlights the gap in both speed and effectiveness: Harvey’s advanced engine (and presumably its coverage-guided strategies) outpaced the open-source fuzzers. Foundry and Echidna each caught many issues quickly at the start, but plateaued; Harvey continued to hit new violations later in the run (Benchmarking Smart-Contract Fuzzers | Consensys Diligence). These results suggest that Medusa, as a new tool from Trail of Bits, will be judged against not just Echidna but also services like Harvey. The Diligence study shows that a well-optimized fuzzer can significantly outperform others – e.g. one data point noted “19 minutes of fuzzing with Harvey are as effective as 8 hours with Foundry” (Benchmarking Smart-Contract Fuzzers | Consensys Diligence). Any Medusa benchmarking effort should consider including such state-of-the-art comparisons to gauge relative performance.
-
Badger DAO’s eBTC Fuzzing (2024) – 6-week real-world assessment: A team of auditors chronicled their experience fuzzing Badger DAO’s eBTC protocol using Echidna, Foundry, and Medusa in tandem. One major takeaway was that “Echidna is still the best tool for the job.” (Learnings from 6 weeks of fuzzing Badger DAO's eBTC protocol) Despite experimenting with all three fuzzers, the auditors found Echidna’s robustness and features gave it an edge for detecting complex logic issues. The report (on AllThingsFuzzy blog) noted a few best practices: use test case converters to translate failing inputs between frameworks (for example, turning a Medusa sequence into a Foundry test for easier debugging), and leverage each tool’s strengths (Feedback on fuzzer benchmarking setup · Issue #103 · crytic/optik · GitHub). Medusa and Foundry were certainly useful, but Echidna’s maturity (years of development and community use) meant it caught subtle bugs and was reliable for long-running invariant tests. This real-world evaluation underscores that accuracy and reliability can trump raw speed – a fuzzer needs to integrate well into an audit workflow. It also suggests that Medusa (at the time, in beta) hadn’t yet unseated its predecessor for deep protocol testing. However, the auditors were excited about using multiple fuzzers in combination. In fact, tools like Chimera have emerged to allow running Echidna, Medusa, and Foundry together on the same invariants (to cross-check and maximize coverage) (Dacian on X: "Fuzzing smart contracts with Foundry, Echidna or ...), which can increase overall bug-finding success.
-
Community Fuzzing Challenge (Dacian, 2024) – Foundry vs. Echidna vs. Medusa on specific targets: An independent security researcher (Dacian) created a series of Solidity fuzzing challenges to directly compare the three fuzzers’ capabilities in both “basic” (unguided) and “advanced” (with hints) modes. The results, posted in a GitHub repository, show that for many simple challenges all three tools were tied – each fuzzer could eventually find the intended bug and even produce minimal exploit transactions when guided properly (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa) (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa). However, certain harder cases differentiated the tools:
- In a token-sale challenge, Medusa was the only fuzzer that, in default mode, quickly broke both invariants (one leading to a Critical vulnerability and another High/Medium) – it did so almost immediately, whereas Foundry and Echidna in default settings could only find the easier issue and missed the second until given extra guidance (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa). Here Medusa’s out-of-the-box strategy proved superior in speed and depth, making it “the clear winner.” (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa)
- In a “Rarely False” challenge (a tricky edge case with no contract state), initially Medusa could find the bug instantly as well, but after a recent update it regressed and could no longer find it (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa). Echidna and Foundry also failed on this particular case. Interestingly, formal tools like Halmos (an SMT-based fuzzer) and Certora (formal verification) did manage to detect this issue, so they were declared winners for that round (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa). This pointed out that fuzzers sometimes miss certain bugs unless their heuristics are tuned, and even Medusa can temporarily lose ground due to changes – highlighting the importance of continuous benchmarking as tools evolve.
- In a DeFi protocol challenge (Beta Finance’s Omni) with 16 complex invariants, all fuzzers were run in guided mode. Medusa consistently found at least 2 invariants breaking within the first 5 minutes (often in seconds on subsequent runs), whereas Echidna would sometimes find 1 within 5 minutes, and Foundry found none in that short window (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa) (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa). Medusa was again deemed the clear winner for this real-world scenario, demonstrating a noticeable advantage in both speed and thoroughness when handling a complex, stateful system. Notably, all three fuzzers were able to eventually find more issues given more time and guidance, but Medusa’s head start means faster feedback for developers (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa).
These community-run comparisons suggest that Medusa can outperform Echidna and Foundry in certain scenarios, especially where rapid exploration of deep state or complex input sequences is needed. Its coverage-guided engine (inspired by afl-like techniques) and parallel execution likely contribute to this. At the same time, results varied per challenge – no single fuzzer won every test, and proper configuration mattered. This reinforces the idea that a comprehensive benchmarking effort should test multiple patterns (simple and complex invariants, with and without guidance) to measure each fuzzer’s accuracy (bugs found) and speed (time to find).
-
Integration in Fuzzing Platforms – Multi-fuzzer approaches: Rather than picking one winner, the latest trend is to combine fuzzing tools to maximize coverage. For example, Recon (a fuzzing-as-a-service platform by Zellic/A16z) integrates Echidna, Medusa, Foundry and more in one cloud solution (Best Smart Contract Auditing and Security Tools - Cyfrin). It allows running invariant tests in parallel across different fuzzers and consolidating the results. This approach has been used to secure projects like Centrifuge and Badger DAO (over $1B in total value) by uncovering issues that one tool alone might miss (Best Smart Contract Auditing and Security Tools - Cyfrin). Recon’s “Builder” even lets developers set up Medusa + Echidna campaigns with minimal effort, showing the demand for Medusa in real-world testing pipelines (Best Smart Contract Auditing and Security Tools - Cyfrin). The security community is acknowledging Medusa’s strengths: a February 2025 review by Cyfrin called Medusa “one of the most powerful publicly available smart contract fuzzers” (Best Smart Contract Auditing and Security Tools - Cyfrin) – albeit labeling it experimental – and praised its parallelism and coverage-guided design. Likewise, prominent auditors have noted that Echidna & Medusa are now paramount tools for Web3 security, often used together (Highlights by Antonio Viggiano (@agfviggiano) / X) (Antonio Viggiano on X: "Another benefit of transcribing Echidna ...). Finally, projects like Youdusa have emerged to translate Medusa’s findings into Foundry test cases (GitHub - perimetersec/evm-fuzzing-resources: A curated collection of tools, articles, research, and guides for fuzzing smart contracts on the Ethereum Virtual Machine (EVM).), and hybrid workflows use Scribble or other oracles across fuzzers. All of this indicates that Medusa is being actively evaluated against (and alongside) other popular fuzzers. Any benchmarking of a GPU-accelerated Medusa should therefore not only measure its raw speedup, but also compare its bug-finding capabilities to the likes of Echidna, Foundry, and other state-of-the-art tools across a variety of benchmarks – as recent papers and community trials have begun to do.
Sources:
- Feist et al., “Unleashing Medusa: Fast and scalable smart contract fuzzing” – Trail of Bits blog (Feb 2025) (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog) (Unleashing Medusa: Fast and scalable smart contract fuzzing - The Trail of Bits Blog)
- Chadt et al., “Olympia: Fuzzer Benchmarking for Solidity” – ASE 2024 (tool & evaluation) (Olympia: Fuzzer Benchmarking for Solidity) (Olympia: Fuzzer Benchmarking for Solidity)
- Viggiano, “A Comparative Analysis of Smart Contract Fuzzers’ Effectiveness” – DeFi Security Summit 2023 (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness) (A Comparative Analysis of Smart Contract Fuzzers' Effectiveness)
- Wu et al., “Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers” – ICSE 2024 () ([2402.02973] Are We There Yet? Unraveling the State-of-the-Art Smart Contract Fuzzers)
- Chen et al., “Towards Smart Contract Fuzzing on GPUs” – 2024 (preprint) () ()
- Wüstholz, “Benchmarking Smart-Contract Fuzzers” – ConsenSys Diligence blog (Apr 2023) (Benchmarking Smart-Contract Fuzzers | Consensys Diligence) (Benchmarking Smart-Contract Fuzzers | Consensys Diligence)
- AllThingsFuzzy blog, “Learnings from 6 weeks of fuzzing Badger DAO’s eBTC” (2024) (Learnings from 6 weeks of fuzzing Badger DAO's eBTC protocol)
- devdacian, Solidity Fuzzing Comparison – GitHub repo (2024) (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa) (GitHub - devdacian/solidity-fuzzing-comparison: A comparison of solidity fuzzing tools Foundry, Echidna & Medusa)
- Cyfrin Security, “8 Smart Contract Security Auditing Tools” (Feb 2025) (Best Smart Contract Auditing and Security Tools - Cyfrin) (Best Smart Contract Auditing and Security Tools - Cyfrin)
- Recon documentation/announcements (2024) (Best Smart Contract Auditing and Security Tools - Cyfrin) (GitHub - perimetersec/evm-fuzzing-resources: A curated collection of tools, articles, research, and guides for fuzzing smart contracts on the Ethereum Virtual Machine (EVM).)