zcash_primitives: Refactor Transaction to permit omitting protocol-specific bundles

[str4d]

Draft because there are still some things that need doing:

• I want some feedback on the overall approach to check the direction makes sense.
• I haven't documented much yet (spent my time getting the refactor structured correctly so the commits make sense).
• I haven't checked all the various feature flag combinations yet.
• The Authorization-ish traits are still tied to the protocol-specific crates. I think I need to introduce intermediate traits with blanket impls.
• Factor out Sprout for completeness (even though we don't depend on anything for Sprout that we don't inherently need for basic Zcash support).
• Factor out ZFuture.
• Needs changelog updates.

[codecov]

Codecov Report

Attention: Patch coverage is 73.33333% with 68 lines in your changes missing coverage. Please review.

Project coverage is 56.49%. Comparing base (2df4497) to head (52bb40d).

Files with missing lines	Patch %	Lines
zcash_primitives/src/transaction/mod.rs	52.54%	28 Missing ⚠️
zcash_primitives/src/transaction/components/tze.rs	0.00%	18 Missing ⚠️
zcash_primitives/src/transaction/sighash_v4.rs	89.39%	7 Missing ⚠️
zcash_primitives/src/transaction/txid.rs	76.92%	6 Missing ⚠️
zcash_primitives/src/transaction/components.rs	50.00%	4 Missing ⚠️
...sh_primitives/src/transaction/components/sprout.rs	89.47%	2 Missing ⚠️
devtools/src/bin/inspect/transaction.rs	0.00%	1 Missing ⚠️
...h_primitives/src/transaction/components/sapling.rs	96.15%	1 Missing ⚠️
zcash_primitives/src/transaction/sighash_v5.rs	88.88%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1388      +/-   ##
==========================================
+ Coverage   56.39%   56.49%   +0.10%     
==========================================
  Files         148      149       +1     
  Lines       19232    19313      +81     
==========================================
+ Hits        10845    10911      +66     
- Misses       8387     8402      +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[daira]

LGTM so far. I was a little bit alarmed about the name UnauthorizedTransactionDataWithSaplingProofs (i.e. why so unorthogonal?), but then I saw the motivation and it's fine.

[daira]

I haven't done anything with ZFuture, and am not quite sure what to do there.

Can we just delete TZE support? I don't care about them and I very much doubt they are ever going to be deployed. We should ask on #libraries in the R&D Discord obviously. Deleting them should be easy given that they're feature flagged.

[str4d]

I was a little bit alarmed about the name UnauthorizedTransactionDataWithSaplingProofs (i.e. why so unorthogonal?), but then I saw the motivation and it's fine.

My hope is that this type goes away once I land the refactor that follows after this one (to restructure the zcash_primitives transaction builder to match Sapling and Orchard, by separating the building of the unauthorized bundle and its authorization). At that point, this type will be a well-defined intermediate state (as an explicit method output), instead of an ill-defined internal detail of Builder::build that requires type-pinning to help the compiler.

[str4d]

In e3a29b8 I've added Yet Another Intermediate Trait Bundles, which I think solves the "conditional compilation of bundles" that is necessary for merging code that is not currently part of consensus (as well as making the enabling of these bundles in consensus less of a breaking change to downstream code). If this approach looks good, I'll rebase the PR to merge the Bundles changes back into the earlier commits (simplifying them).

[str4d]

Force-pushed to merge the commit introducing the Bundles trait back into the rest of the commits, simplifying them.

[str4d]

Force-pushed to address some initial review comments from a pairing with @nuttycom.

[str4d]

Force-pushed to move Sprout onto the Bundles trait. It has no authorization traits because we only support already-authorized Sprout bundles (due to not supporting Sprout in transaction building).

[str4d]

Force-pushed to move TZEs onto the Bundles trait, as a proof-of-concept for other conditionally-compiled transaction format changes that we want to merge into the codebase before inclusion in consensus (e.g. ZSAs).

[str4d]

Force-pushed to adjust how the new traits work in order to eliminate the bounds on the protocol-crate-specific authorization traits.

[str4d]

Force-pushed with some further decoupling adjustments and trait renames. It also removes the NoTransparent etc. changes (and the TransactionWith refactor) as not currently necessary, because the dependency removal will be done by altering the Transparent<A> etc. types (in a way that is technically a breaking change, but in a way that should only be observable as purely additive in our API).

[str4d]

Rebased on main to fix a small merge conflict.

[str4d]

Rebased on main to fix merge conflicts.

[str4d]

Rebased on main to bring in recent changes.

[str4d]

Force-pushed to fix formatting.