Skip to content
This repository has been archived by the owner on Mar 1, 2022. It is now read-only.

Initialize benchmark package #15

Merged
merged 4 commits into from
Dec 22, 2021
Merged

Initialize benchmark package #15

merged 4 commits into from
Dec 22, 2021

Conversation

srid
Copy link
Member

@srid srid commented Dec 22, 2021

Add cabal bench package with a dummy benchmark, as base for #9

@TotallyNotChase Feel free to add your benchmarks here. If you are not using criterion feel free to remove it, but it would be nice to be able to generate the benchmark report in JSON (criterion does this automatically for us), so that we can integrate it to CI for a nicer report like IntersectMBO/plutus#3494 (comment) ... edit: actually, it may be a good idea to use criterion here because we get cheap comparison tools (see next paragraph).

@MatthewCroughan nix build '.#pluton:bench:perf' followed by running ./result/bin/perf generates a bench.json as well as bench.csv that can be used to generate the CI report (there is also bench.html for the corresponding HTML view). There is even criterion-compare which seems to do the heavy-lifting here. cf. haskell/criterion#151

@srid srid merged commit a2cab16 into main Dec 22, 2021
@srid srid deleted the bench/init branch December 22, 2021 21:32
@srid
Copy link
Member Author

srid commented Dec 22, 2021

bench.csv =>

Name,Mean,MeanLB,MeanUB,Stddev,StddevLB,StddevUB
fib/1,5.626615908983465e-9,5.204812201870928e-9,6.264731155976395e-9,1.716652388696088e-9,1.2457188183919467e-9,2.2170916103471158e-9
fib/5,2.6809657559565625e-8,2.645283350629125e-8,2.7510165335409422e-8,1.6655396284576668e-9,1.0559431569056852e-9,2.4737825591278294e-9
fib/9,2.0783884805815688e-7,2.058856748400889e-7,2.1078807548744367e-7,8.241848518035217e-9,5.6001125047839684e-9,1.0920654754077311e-8
fib/11,5.536281632540129e-7,5.48200847684166e-7,5.615283224083176e-7,2.1481856752110492e-8,1.5241171933881577e-8,2.797168473489698e-8

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant