The research team at Discreet Labs has been following up on new results in zero-knowledge proofs. Recently, there has been a lot of discussion surrounding HyperPlonks as a breakthrough for ZK-rollup technology; Discreet Labs researchers are highly skeptical, and this article is meant to explain the sources of our skepticism.
Our efforts in experimenting and deploying HyperPlonk suggest HyperPlonks are not as efficient as claimed. In terms of Rollups and ZK-EVM, HyperPlonk may actually be inferior to existing approaches. We feel it necessary to share these concerns with the blockchain and ZK community to conserve efforts toward more valuable directions.
We believe that peer review is important in ensuring the health of the research surrounding zero-knowledge proofs. Our concerns on HyperPlonk are centered around two issues, optimization and costs.
The primary motivation of HyperPlonk is to remove FFT since FFT contributes a significant overhead for large instances. Our research team finds this surprising because it is generally agreed that FFT is far from being the dominating cost.
Jump Crypto has published their team’s work on FFT acceleration in FPGA. In a less than three-month development period, Jump Crypto provided an implementation of FFT on a single FPGA card that can process 2^24 FFT in less than 6ms (https://eprint.iacr.org/2022/1657), a result which is several orders of magnitude smaller than that produced by Multiscalar Multiplication for these proof systems. These achievements were made using only one FPGA card. With more cards, the performance can improve further.
The assertion that FFT will be a bottleneck has been a misconception in the industry. Without doing the experiments themselves, people could easily speculate that the computation burden of FFTs would eventually surpass that of multiscalar multiplication (MSM) for large instances. But in practice, the instance size that FFT starts to surpass MSM is very large, which is beyond the working range of most applications for zero-knowledge proofs, even for the most complex ZK-EVM.
For this reason, the motivation to replace FFT in favor of HyperPlonks is, indeed, misguided and not supported by empirical evidence. FFT has never been the dominating cost of Plonk and related proof systems, and we are unsure why the team behind HyperPlonk has based their paper on this unsolid foundation.
The cost that HyperPlonk pays to eliminate FFT is very expensive and indeed contradictory to the goal of improving ZK-Rollup. This is not obvious because the HyperPlonk paper did not provide an empirical evaluation of its verification costs.
HyperPlonk requires log(N) pairings for the verifier. For a constraint system of size 2^25, the gas cost per verification is at least 895k for pairing and several hundred thousand for other operations, making the total verifier gas cost surpass 1.1m. This is already inferior to most proof systems used in production that require at most three pairings, and it almost prohibits the use of HyperPlonk in the Rollup setting. If we use HyperPlonk to roll up transactions, the L1 costs could be orders of magnitude higher than Groth16 and Plonk, making L1 verification extremely expensive, and if we use another proof system to roll up HyperPlonk, the other proof system has to spend a lot of the time just to verify these pairings. Adding that HyperPlonk’s proving time is not competitive with existing Plonk implementations, it is therefore not worthwhile to use HyperPlonks, as it comes with prohibitive verification costs.
We will be glad to hear from the team behind HyperPlonks if they have any clarifications. We believe that researchers and developers would appreciate seeing the verification costs, especially the gas costs if the proofs are verified on EVM. Without added clarification, and based on the empirical evidence discussed above, it is our contention that HyperPlonks are not useful for ZK-Rollups.
In the history of zero-knowledge proofs and their deployment in blockchain, this is not the first time that “different” and “novel” algorithms have not been competitive with those which are mature and mainstream.
Discreet Labs has previously worked on Supersonic Proofs which share an almost identical situation with Hyperplonk, but it soon became clear that Supersonic Proofs indeed had a poor performance that made it unusable in practice. Discreet Labs has abandoned its efforts in writing the corresponding Rust libraries.
We eventually need to see evidence that HyperPlonks could feasibly be deployed in the real world. Being transparent about the major limitations of one’s work is an important responsibility to the community and an important part of academic integrity and ethics.
This line of research seems like a distraction to the community and may become detrimental to researchers, manufacturers, and others investing in HyperPlonk technology. We know, for example, that a company has been working on and investing significant resources into ASICs optimized for HyperPlonks. Such efforts could have been used more meaningfully, given the findings presented here.
We hope that our findings create a healthy discussion around the limitations of HyperPlonks.
This kind of statement doesn't help the field.
If a paper made incorrect performance claims, it would be reasonable to call that out, but as far as I know the HyperPlonk paper doesn't.
If a paper made incorrect claims about asymptotic performance, that would also be reasonable to call out. As far as I know the HyperPlonk paper doesn't do that either.
The claim that FFTs eventually get more expensive than MSMs is correct. Yes, that might not happen for the range of circuit sizes your company is interested in, but so what? It's still a correct claim.
The HyperPlonk paper doesn't mention EVM gas costs or anything like that.
We all know that it's necessary to take performance claims of every paper with a pinch of salt, because benchmarking is hard and it's easy for authors to end up using a benchmark that favours their system, without any intent to mislead. This is a general problem and I'm not saying that it even applies to HyperPlonk.
There are no "concerns" here that justify describing the line of research in the HyperPlonk paper as a "distraction" or "detrimental" to researchers, manufacturers, or anyone else. The ideas of HyperPlonk are ideas that I expect to continue to be used in future systems with varying performance characteristics.
I have no direct connection to any of the authors of the HyperPlonk paper or to the company they work for (Espresso Systems); it just annoys me to see supposed critique that skirts the edges of accepted norms of scientific discourse, for no justified reason.
Apparently Discreet Labs spent a bunch of time and effort investigating SuperSonic and HyperPlonk, and neither of them turned out to have performance characteristics that work for your application. It happens; deal with it. There is nothing (to my knowledge) wrong with HyperPlonk or the claims its designers make for it.
--
Daira Emma Hopwood (speaking only for myself)