Skip to content

blake2 simd benches produce unexpected results #407

Open
@nazar-pc

Description

@nazar-pc

I ran blake2 tests with and without simd feature enabled and found that behavior is strange and doesn't make a lot of sense.

Default (no simd):

$ cargo +nightly bench
...
running 8 tests
test blake2b512_10    ... bench:           8 ns/iter (+/- 0) = 1250 MB/s
test blake2b512_100   ... bench:          74 ns/iter (+/- 4) = 1351 MB/s
test blake2b512_1000  ... bench:         708 ns/iter (+/- 85) = 1412 MB/s
test blake2b512_10000 ... bench:       7,005 ns/iter (+/- 531) = 1427 MB/s
test blake2s256_10    ... bench:          13 ns/iter (+/- 0) = 769 MB/s
test blake2s256_100   ... bench:         121 ns/iter (+/- 7) = 826 MB/s
test blake2s256_1000  ... bench:       1,155 ns/iter (+/- 62) = 865 MB/s
test blake2s256_10000 ... bench:      11,791 ns/iter (+/- 1,005) = 848 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 28.22s

Simd:

$ cargo +nightly bench --features simd
...
running 8 tests
test blake2b512_10    ... bench:          11 ns/iter (+/- 0) = 909 MB/s
test blake2b512_100   ... bench:         102 ns/iter (+/- 0) = 980 MB/s
test blake2b512_1000  ... bench:       1,010 ns/iter (+/- 11) = 990 MB/s
test blake2b512_10000 ... bench:      10,076 ns/iter (+/- 68) = 992 MB/s
test blake2s256_10    ... bench:          16 ns/iter (+/- 0) = 625 MB/s
test blake2s256_100   ... bench:         152 ns/iter (+/- 1) = 657 MB/s
test blake2s256_1000  ... bench:       1,483 ns/iter (+/- 12) = 674 MB/s
test blake2s256_10000 ... bench:      14,750 ns/iter (+/- 344) = 677 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 2.43s

I expected simd version to produce better results and judging by the total execution time it is the case, but throughput reported by the benchmark says the opposite.

I suspect something is wrong with benchmarking code.

Using latest master (cc52373 right now) and rustc 1.65.0-nightly (20ffea693 2022-08-11).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions