-c option species decode method XX encode method YY where
00 is scalar
01 is SSE4
02 is AVX2
04 is AVX512
Input data is 10MB worth of NovaSeq quality values; approx 100k records. Performance is data specific, so these figures are purely a snapshot and not indicative of all data types. The test machine reports as:
Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz
The -o field is a bit field where.
0/1 Order 0 or 1
4 32-way variant (permits SIMD)
64 RLE
128 Bit packing (4 novaseq quals to a byte)
Hence -o133-c0202 is pack 4 quals to a bit and order-1 encode with AVX2 32-way encode/decode.
r4x8-o0 10000000 uncomp, 665848 comp 395.3 enc MB/s 718.3 dec MB/s
r4x16-o0 10000000 uncomp, 665415 comp 400.3 enc MB/s 716.1 dec MB/s
arith-o0 10000000 uncomp, 660701 comp 105.0 enc MB/s 86.3 dec MB/s
r4x8-o1 10000000 uncomp, 615304 comp 274.8 enc MB/s 385.1 dec MB/s
r4x16-o1 10000000 uncomp, 616134 comp 289.6 enc MB/s 536.6 dec MB/s
arith-o1 10000000 uncomp, 613736 comp 75.4 enc MB/s 87.1 dec MB/s
r4x16-o64 10000000 uncomp, 712335 comp 382.0 enc MB/s 749.2 dec MB/s
arith-o64 10000000 uncomp, 744000 comp 153.1 enc MB/s 112.2 dec MB/s
r4x16-o65 10000000 uncomp, 591457 comp 360.6 enc MB/s 705.8 dec MB/s
arith-o65 10000000 uncomp, 585233 comp 161.3 enc MB/s 117.7 dec MB/s
r4x16-o128 10000000 uncomp, 615915 comp 780.2 enc MB/s 2092.5 dec MB/s
arith-o128 10000000 uncomp, 609977 comp 257.4 enc MB/s 219.0 dec MB/s
r4x16-o129 10000000 uncomp, 553081 comp 645.1 enc MB/s 1394.1 dec MB/s
arith-o129 10000000 uncomp, 550377 comp 165.1 enc MB/s 180.7 dec MB/s
r4x16-o192 10000000 uncomp, 621771 comp 513.1 enc MB/s 1003.0 dec MB/s
arith-o192 10000000 uncomp, 621415 comp 217.9 enc MB/s 180.8 dec MB/s
r4x16-o193 10000000 uncomp, 550325 comp 474.1 enc MB/s 920.6 dec MB/s
arith-o193 10000000 uncomp, 543687 comp 195.7 enc MB/s 158.0 dec MB/s
r32x16-o4-c0000 10000000 uncomp, 665501 comp 399.0 enc MB/s 613.9 dec MB/s
r32x16-o4-c0101 10000000 uncomp, 665501 comp 402.1 enc MB/s 968.0 dec MB/s
r32x16-o4-c0202 10000000 uncomp, 665501 comp 690.8 enc MB/s 1796.0 dec MB/s
r32x16-o4-c0404 10000000 uncomp, 665501 comp 866.9 enc MB/s 2098.6 dec MB/s
r32x16-o5-c0000 10000000 uncomp, 616223 comp 274.6 enc MB/s 426.5 dec MB/s
r32x16-o5-c0101 10000000 uncomp, 616223 comp 274.1 enc MB/s 626.8 dec MB/s
r32x16-o5-c0202 10000000 uncomp, 616223 comp 391.8 enc MB/s 1472.8 dec MB/s
r32x16-o5-c0404 10000000 uncomp, 616223 comp 563.5 enc MB/s 1673.9 dec MB/s
r32x16-o68-c0000 10000000 uncomp, 712513 comp 363.8 enc MB/s 717.4 dec MB/s
r32x16-o68-c0101 10000000 uncomp, 712513 comp 384.7 enc MB/s 836.5 dec MB/s
r32x16-o68-c0202 10000000 uncomp, 712513 comp 438.8 enc MB/s 913.6 dec MB/s
r32x16-o68-c0404 10000000 uncomp, 712513 comp 450.8 enc MB/s 918.0 dec MB/s
r32x16-o69-c0000 10000000 uncomp, 591639 comp 369.7 enc MB/s 684.2 dec MB/s
r32x16-o69-c0101 10000000 uncomp, 591639 comp 370.2 enc MB/s 780.1 dec MB/s
r32x16-o69-c0202 10000000 uncomp, 591639 comp 408.5 enc MB/s 894.9 dec MB/s
r32x16-o69-c0404 10000000 uncomp, 591639 comp 431.6 enc MB/s 906.5 dec MB/s
r32x16-o132-c0000 10000000 uncomp, 615999 comp 659.2 enc MB/s 1861.9 dec MB/s
r32x16-o132-c0101 10000000 uncomp, 615999 comp 660.0 enc MB/s 2580.6 dec MB/s
r32x16-o132-c0202 10000000 uncomp, 615999 comp 971.6 enc MB/s 3679.2 dec MB/s
r32x16-o132-c0404 10000000 uncomp, 615999 comp 1050.6 enc MB/s 3947.9 dec MB/s
r32x16-o133-c0000 10000000 uncomp, 553181 comp 573.2 enc MB/s 848.8 dec MB/s
r32x16-o133-c0101 10000000 uncomp, 553181 comp 566.3 enc MB/s 1517.0 dec MB/s
r32x16-o133-c0202 10000000 uncomp, 553181 comp 759.1 enc MB/s 1923.8 dec MB/s
r32x16-o133-c0404 10000000 uncomp, 553181 comp 914.4 enc MB/s 1981.4 dec MB/s
r32x16-o194-c0000 10000000 uncomp, 621771 comp 558.0 enc MB/s 1085.0 dec MB/s
r32x16-o194-c0101 10000000 uncomp, 621771 comp 559.2 enc MB/s 1088.6 dec MB/s
r32x16-o194-c0202 10000000 uncomp, 621771 comp 552.9 enc MB/s 1091.2 dec MB/s
r32x16-o194-c0404 10000000 uncomp, 621771 comp 550.1 enc MB/s 1070.3 dec MB/s
r32x16-o197-c0000 10000000 uncomp, 550497 comp 484.2 enc MB/s 791.8 dec MB/s
r32x16-o197-c0101 10000000 uncomp, 550497 comp 487.2 enc MB/s 1004.4 dec MB/s
r32x16-o197-c0202 10000000 uncomp, 550497 comp 488.0 enc MB/s 1033.9 dec MB/s
r32x16-o197-c0404 10000000 uncomp, 550497 comp 502.0 enc MB/s 1027.6 dec MB/s
For completeness, a couple other tools are also shown below. Note fqzcomp here is slightly smaller as it has been trimmed to end on a whole line.
fqzcomp -s1 9999975 uncomp, 494485 comp 27.4 enc MB/s 27.1 dec MB/s
bsc -m3e1tT 10000000 uncomp, 553958 comp 43.7 enc MB/s 31.6 dec MB/s
bsc -m0e2tT 10000000 uncomp, 531536 comp 19.0 enc MB/s 25.5 dec MB/s
10MB worth of Illumina HiSeq data with 40 distinct quality values. Note this sequencing run had a few erratic cycles, leading to unusually good performance from fqzcomp. The bit-packing modes of rANS are not relevant (nor shown) here due to the cardinality of the data.
r4x8-o0 10000000 uncomp, 5092977 comp 303.9 enc MB/s 553.3 dec MB/s
r4x16-o0 10000000 uncomp, 5092608 comp 357.4 enc MB/s 579.8 dec MB/s
arith-o0 10000000 uncomp, 5079029 comp 51.9 enc MB/s 33.1 dec MB/s
r4x8-o1 10000000 uncomp, 4911113 comp 278.1 enc MB/s 356.4 dec MB/s
r4x16-o1 10000000 uncomp, 4918609 comp 290.5 enc MB/s 542.4 dec MB/s
arith-o1 10000000 uncomp, 4911347 comp 42.1 enc MB/s 32.3 dec MB/s
r4x16-o64 10000000 uncomp, 5092608 comp 215.5 enc MB/s 782.7 dec MB/s
arith-o64 10000000 uncomp, 5194241 comp 36.8 enc MB/s 26.6 dec MB/s
r4x16-o65 10000000 uncomp, 4918609 comp 167.0 enc MB/s 484.0 dec MB/s
arith-o65 10000000 uncomp, 4909925 comp 33.4 enc MB/s 23.8 dec MB/s
r32x16-o4-c0000 10000000 uncomp, 5092684 comp 367.2 enc MB/s 642.1 dec MB/s
r32x16-o4-c0101 10000000 uncomp, 5092684 comp 340.7 enc MB/s 1005.1 dec MB/s
r32x16-o4-c0202 10000000 uncomp, 5092684 comp 666.8 enc MB/s 1777.5 dec MB/s
r32x16-o4-c0404 10000000 uncomp, 5092684 comp 827.0 enc MB/s 2158.9 dec MB/s
r32x16-o5-c0000 10000000 uncomp, 4918685 comp 273.9 enc MB/s 391.5 dec MB/s
r32x16-o5-c0101 10000000 uncomp, 4918685 comp 268.5 enc MB/s 524.0 dec MB/s
r32x16-o5-c0202 10000000 uncomp, 4918685 comp 396.0 enc MB/s 1218.2 dec MB/s
r32x16-o5-c0404 10000000 uncomp, 4918685 comp 553.4 enc MB/s 1418.4 dec MB/s
r32x16-o68-c0000 10000000 uncomp, 5092684 comp 216.3 enc MB/s 646.6 dec MB/s
r32x16-o68-c0101 10000000 uncomp, 5092684 comp 235.2 enc MB/s 1016.3 dec MB/s
r32x16-o68-c0202 10000000 uncomp, 5092684 comp 336.4 enc MB/s 1804.4 dec MB/s
r32x16-o68-c0404 10000000 uncomp, 5092684 comp 376.5 enc MB/s 2162.2 dec MB/s
r32x16-o69-c0000 10000000 uncomp, 4918685 comp 194.3 enc MB/s 390.1 dec MB/s
r32x16-o69-c0101 10000000 uncomp, 4918685 comp 195.3 enc MB/s 593.4 dec MB/s
r32x16-o69-c0202 10000000 uncomp, 4918685 comp 251.6 enc MB/s 1212.7 dec MB/s
r32x16-o69-c0404 10000000 uncomp, 4918685 comp 306.3 enc MB/s 1415.6 dec MB/s
fqzcomp -s1 10000000 uncomp, 3196746 comp 16.6 enc MB/s 16.0 dec MB/s
bsc -m3e1tT 10000000 uncomp, 4762846 comp 12.9 enc MB/s 17.5 dec MB/s
bsc -m0e2tT 10000000 uncomp, 4477056 comp 6.1 enc MB/s 8.8 dec MB/s