squashfs compression tests using mksquashfs - xz, gzip, zstd, lz4, lz4hc, lzo

squashfs.txt

Just a quick test of squashfs compression ratio using different settings.

I am looking for relatively good and fast compression, that also is quick to decompress.
I don't care about ultimate end size exactly tho.

Input (a Debian testing live build with 6240 installed packages):

$ sudo du -bs ./chroot
26566785410	./chroot   # 26.6GB
$
$ sudo find ./chroot -type f | wc -l
614635
$ sudo find ./chroot -type d | wc -l
55771
$

Input and output on tmpfs.

32 compression threads on AMD TR 2950X

Results:

mksquashfs ./chroot chroot_gzip9_default.squashfs -noappend -comp gzip  # 128kiB blocks, gzip-9           # 168s
mksquashfs ./chroot chroot_xz_default.squashfs -noappend -comp xz  # 128kiB blocks                        # 376s
mksquashfs ./chroot chroot_xz-x86_default.squashfs -noappend -comp xz -Xbcj x86  # 128kiB blocks          # 781s
mksquashfs ./chroot chroot_xz_block32k.squashfs -noappend -comp xz -b 32768  # 32 kiB blocks              # 336s
mksquashfs ./chroot chroot_zstd15_default.squashfs -noappend -comp zstd                                   # 215s
mksquashfs ./chroot chroot_zstd15_block32k.squashfs -noappend -comp zstd -b 32768                         # 159s
mksquashfs ./chroot chroot_zstd22_block32k.squashfs -noappend -b 32768 -comp zstd -Xcompression-level 22  # 650s
mksquashfs ./chroot chroot_zstd22_block128k.squashfs -noappend -comp zstd -Xcompression-level 22          # 934s
mksquashfs ./chroot chroot_lzo9_block32k.squashfs -noappend -b 32768 -comp lzo -Xcompression-level 9      # 180s
mksquashfs ./chroot chroot_lz4_block32k.squashfs -noappend -comp lz4 -b 32768                             #  21s (not a typo!)
mksquashfs ./chroot chroot_lz4hc_block32k.squashfs -noappend -comp lz4 -b 32768 -Xhc                      # 121s

-rw-r--r--  1 root root  7701307392 Jul 24 07:01 chroot_xz-x86_block128k.squashfs
-rw-r--r--  1 root root  7822753792 Jul 24 05:08 chroot_xz_default.squashfs
-rw-r--r--  1 root root  8473190400 Jul 24 06:37 chroot_zstd22_block128k.squashfs
-rw-r--r--  1 root root  8517169152 Jul 24 05:14 chroot_xz_block32k.squashfs
-rw-r--r--  1 root root  8553738240 Jul 24 05:44 chroot_zstd15_default.squashfs
-rw-r--r--  1 root root  9059016704 Jul 24 06:09 chroot_zstd22_block32k.squashfs
-rw-r--r--  1 root root  9148821504 Jul 24 05:53 chroot_zstd15_block32k.squashfs
-rw-r--r--  1 root root  9248616448 Jul 24 06:47 chroot_gzip9_block128k.squashfs
-rw-r--r--  1 root root 10710016000 Jul 24 06:19 chroot_lzo9_block32k.squashfs
-rw-r--r--  1 root root 11628195840 Jul 24 06:14 chroot_lz4hc_block32k.squashfs
-rw-r--r--  1 root root 13007339520 Jul 24 06:11 chroot_lz4_block32k.squashfs

I will stick with `-b 32768 -comp zstd -Xcompression-level 22`, because it is
reasonably fast to compress (but actually slower than xz), reasonably good
compression, and should be really fast to decompress. 32kiB blocks hopefully
reduce some redundant reads for some big files, which hopefully will make
latency lower and total transfered data (on read) smaller, for overall faster
load and better expirience, and slightly more optimized memory usage.

I did a bunch of benchmarks on my system and here are the results I got, if this can be helpful for anyone else, (best to copy into a .csv file and open in some spreadsheet software so you can sort by different columns and whatnot, this is currently sorted by the timeToRatioRatio which I calculated by dividing the compression ratio by (the time it took to compress * 10)):

benchmark,decompression,compression,size_bytes,compressionRatio,timeToRatioRatio
zstd lvl 3 | 512K,26.3867,39.5691,13069406208,2.554,0.00645453143993672
zstd lvl 6 | 1M,26.5293,42.184,12423729152,2.687,0.00636971363550161
zstd lvl 6 | 512K,26.3862,41.8544,12640718848,2.641,0.00630996980006881
zstd lvl 3 | 256K,26.0852,40.5409,13253533696,2.519,0.00621347824049293
zstd lvl 1 | 512K,26.5541,39.2732,13773877248,2.424,0.00617214792784902
zstd lvl 6 | 256K,26.3778,42.4718,12912058368,2.585,0.00608639144090903
zstd lvl 3 | 128K,26.4282,40.9465,13586948096,2.457,0.00600051286434738
zstd lvl 6 | 128K,26.3714,43.3861,13084012544,2.551,0.0058797633343398
zstd lvl 1 | 256K,26.281,41.6946,13737750528,2.43,0.00582809284655567
zstd lvl 1 | 128K,26.2708,40.795,14043029504,2.377,0.005826694447849
gzip lvl 1 | 512K,27.5735,42.1527,14243766272,2.344,0.00556073513677653
zstd lvl 9 | 512K,26.1858,47.8761,12568555520,2.656,0.00554765321318988
gzip lvl 1 | 1M,27.5551,42.6213,14233137152,2.345,0.00550194386374888
gzip lvl 1 | 256K,27.5261,44.3128,14266306560,2.34,0.00528064125941037
zstd lvl -3 | 256K,26.6092,40.3938,16001200128,2.086,0.00516415885606207
zstd lvl -3 | 1M,26.9118,40.396,16227000320,2.057,0.00509208832557679
gzip lvl 6 | 512K,27.5579,49.4579,13279518720,2.514,0.00508311109044258
gzip lvl 6 | 128K,27.3842,49.1227,13384765440,2.494,0.00507708248935828
gzip lvl 6 | 256K,27.8546,49.41,13314113536,2.507,0.00507387168589354
zstd lvl 9 | 1M,26.187,53.7327,12335816704,2.706,0.00503603950666912
zstd lvl -3 | 512K,26.217,40.6966,16299175936,2.048,0.00503236142577021
zstd lvl -3 | 128K,27.3453,40.434,16512106496,2.022,0.00500074194984419
gzip lvl 6 | 1M,27.0708,50.5391,13262741504,2.517,0.00498030237974163
lz4 | 1M,27.0353,40.6466,16982528000,1.966,0.00483681291916175
lz4 | 512K,26.8966,41.3424,17002967040,1.963,0.00474815201826696
lz4 | 256K,26.3701,42.0476,17045614592,1.958,0.00465662725102027
lz4 | 128K,26.5548,42.6559,17132654592,1.949,0.00456912173931391
zstd lvl -10 | 512K,27.4216,42.0507,19260829696,1.733,0.00412121558024004
zstd lvl -10 | 256K,27.1452,43.0123,18984116224,1.758,0.00408720296287341
zstd lvl -10 | 1M,27.2964,43.1011,19146797056,1.744,0.0040463004424481
zstd lvl -10 | 128K,27.6526,43.6764,19578130432,1.705,0.00390371001272999
zstd lvl 3 | 1M,26.2903,67.052,12885508096,2.591,0.00386416512557418
zstd lvl 9 | 256K,26.3742,70.2462,12705783808,2.627,0.00373970406940162
zstd lvl 9 | 128K,26.4958,70.3286,12993933312,2.569,0.00365285246684848
zstd lvl 15 | 1M,27.0082,96.1357,12247793664,2.726,0.00283557512973848
zstd lvl 1 | 1M,26.716,89.0552,13693243392,2.438,0.00273762789820246
gzip lvl 9 | 128K,27.6915,99.9294,13336002560,2.503,0.00250476836646673
gzip lvl 9 | 256K,27.6127,102.105,13263241216,2.517,0.00246510944615837
gzip lvl 9 | 1M,27.4221,103.894,13210427392,2.527,0.00243228675380676
gzip lvl 1 | 128K,27.6593,101.284,14312304640,2.332,0.00230243671261009
zstd lvl 15 | 512K,26.3681,119.657,12484366336,2.674,0.00223472091060281
zstd lvl 15 | 128K,26.6938,125.644,12481363968,2.675,0.00212903123109739
lz4 HC | 128K,26.7983,105.388,14962786304,2.231,0.00211693930997837
gzip lvl 9 | 512K,27.0981,124.358,13227642880,2.524,0.00202962414963251
lz4 HC | 256K,26.7159,111.686,14751711232,2.263,0.0020262163565711
zstd lvl 15 | 256K,26.4936,137.776,12200570880,2.736,0.00198583207525258
lz4 HC | 512K,27.3042,115.868,14646755328,2.279,0.00196689336141126
lz4 HC | 1M,27.3919,117.09,14594854912,2.287,0.00195319839439747
lzo | 256K,26.7627,130.175,14178951168,2.354,0.00180833493374304
lzo | 512K,27.1022,132.303,14119571456,2.364,0.00178680755538423
lzo | 1M,27.3225,133.453,14090256384,2.369,0.00177515679677489
lzo | 128K,26.731,142.768,14298914816,2.335,0.00163552056483246
xz | 256K,31.2603,221.312,11486687232,2.906,0.00131307836899942
xz | 128K,31.4704,223.256,11771940864,2.836,0.00127029060809116
xz | 512K,30.9808,237.558,11245805568,2.968,0.00124937909899898
xz | 1M,31.0173,258.026,11043774464,3.023,0.00117158735941339
xz BCJ x86 | 256K,31.4657,420.076,11353423872,2.94,0.000699873356249821
xz BCJ x86 | 128K,32.2735,420.627,11639312384,2.868,0.000681839254256146
xz BCJ x86 | 512K,31.0566,450.086,11113127936,3.004,0.000667428002648383
xz BCJ x86 | 1M,31.0959,495.412,10913026048,3.059,0.000617465866793699

I decided to go with zstd level 6 with a 1M block size (isoImage.squashfsCompression = "zstd -Xcompression-level 6 -b 1M";) for my iso builds because it got a ratio of 2.687 while only taking 42 seconds and the extra processing for options with better compression just doesn't seem worth it to me