Testing both debug and release builds in CI

[tarcieri]

Note: some past discussion in RustCrypto/MACs#186

Several of our projects test --release builds in CI: https://github.com/search?q=org%3ARustCrypto+cargo+test+--release+language%3AYAML&type=code&l=YAML

Many of our projects currently test release builds exclusively. I think where we're not testing debug builds, we probably should, in order to test debug assertions. We can probably do things like cargo hack and feature-by-feature testing in debug builds. But I think we should still include something like cargo test --release --all-features to test release builds, possibly as its own job which can run in parallel.

Debug versus release builds involve conditional compilation, so testing debug builds exclusively doesn't give you a full picture of what happens in a release build. Only testing the release build can do that, and IMO release builds are what actually matter. This is why I originally included testing --release builds in the CI boilerplate. In general we test many other combinations as we're attempting to produce high-assurance software, so testing both debug and release builds seems like standard practice to me.

Some examples of other Rust cryptography projects which test release builds in CI:

• ring: https://github.com/briansmith/ring/blob/main/.github/workflows/ci.yml
• rustls: https://github.com/rustls/rustls/blob/91e33db/.github/workflows/build.yml#L80
• graviola: https://github.com/ctz/graviola/blob/1a0facf/.github/workflows/build.yml#L56-L57

[newpavlov]

I am fine with adding one release test run (cargo test --release --all-features or equivalent) without splitting it into a separate job, but I don't think we should test all feature combinations with it. We also should be consistent and do it (gradually) in all repositories.

[newpavlov]

What should we do with build-only jobs? I think it's fine to use debug builds for them.

[tarcieri]

The build-only jobs are largely to test no_std compatibility (and most should probably be named no_std), which is largely to ensure the crate isn't linking std, so I think testing either profile is fine there