feat!: Improved array lowering

[mark-koch]

Feature branch for improved array lowering.

• The old array type is now called value_array and lives in a separate extension
• The default array is now a linear type with additional clone and discard operations
• To avoid code duplication, array operations and values are now defined generically over a new ArrayKind trait that is instantiated with Array (the linear one) and VArray (the copyable one) to generate the array and value_array extensions
• An array<n, T> is now lowered to a fat pointer {ptr, usize} where ptr is a heap allocated pointer of size at least n * sizeof(T) and the usize is an offset pointing to the first element (i.e. the first element is at ptr + offset * sizeof(T)). The rational behind the additional offset is the pop_left operation which bumps the offset instead of mutating the pointer. This way, we can still free the original pointer when the array is discarded after a pop.

Tracked PRs:

• feat!: Make arrays linear and add copyable value arrays #2097
• feat: Add array clone and discard ops #2100
• feat: Add value_array to/from array conversion ops #2101
• feat!: Make array.get give back the passed array #2110
• feat!: Update array LLVM lowering to use malloc #2112
• feat!: Generalise array op builers to arbitrary array kinds #2119
• feat: Add array linearisation pass #2125

BREAKING CHANGE: std.collections.array is now a linear type, even if the contained elements are copyable. Use the new std.collections.value_array for an array with the previous copyable semantics.

BREAKING CHANGE: std.collections.array.get now also returns the passed array as an extra output

[hugrbot]

This PR contains breaking changes to the public Rust API.

cargo-semver-checks summary

--- failure enum_missing: pub enum removed or renamed ---

Description:
A publicly-visible enum cannot be imported by its prior path. A `pub use` may have been removed, or the enum itself may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/enum_missing.ron

Failed in:
enum hugr_core::std_extensions::collections::array::ArrayOpDef, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array/array_op.rs:27

--- failure struct_missing: pub struct removed or renamed ---

Description:
A publicly-visible struct cannot be imported by its prior path. A `pub use` may have been removed, or the struct itself may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/struct_missing.ron

Failed in:
struct hugr_core::std_extensions::collections::array::ArrayValue, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array.rs:41
struct hugr_core::std_extensions::collections::array::ArrayRepeat, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array/array_repeat.rs:107
struct hugr_core::std_extensions::collections::array::ArrayRepeatDef, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array/array_repeat.rs:22
struct hugr_core::std_extensions::collections::array::ArrayOp, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array/array_op.rs:236
struct hugr_core::std_extensions::collections::array::ArrayScanDef, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array/array_scan.rs:24
struct hugr_core::std_extensions::collections::array::ArrayOpDefIter, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array/array_op.rs:24
struct hugr_core::std_extensions::collections::array::ArrayScan, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-core/src/std_extensions/collections/array/array_scan.rs:139

--- failure function_parameter_count_changed: pub fn parameter count changed ---

Description:
A publicly-visible function now takes a different number of parameters.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/function_parameter_count_changed.ron

Failed in:
hugr_llvm::extension::collections::array::emit_repeat_op now takes 4 parameters instead of 3, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-llvm/src/extension/collections/array.rs:803
hugr_llvm::extension::collections::array::emit_scan_op now takes 6 parameters instead of 5, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-llvm/src/extension/collections/array.rs:831

--- failure function_missing: pub fn removed or renamed ---

Description:
A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.41.0/src/lints/function_missing.ron

Failed in:
function hugr_passes::replace_types::handlers::linearize_array, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/replace_types/handlers.rs:78

[codecov]

Codecov Report

Attention: Patch coverage is 82.19814% with 115 lines in your changes missing coverage. Please review.

Project coverage is 83.38%. Comparing base (a1cd051) to head (26a079c).

Files with missing lines	Patch %	Lines
...rc/std_extensions/collections/array/array_value.rs	61.84%	28 Missing and 1 partial ⚠️
...rc/std_extensions/collections/array/array_clone.rs	86.17%	11 Missing and 2 partials ⚠️
.../std_extensions/collections/array/array_discard.rs	85.05%	11 Missing and 2 partials ⚠️
...core/src/std_extensions/collections/value_array.rs	75.55%	10 Missing and 1 partial ⚠️
hugr-passes/src/replace_types/handlers.rs	60.71%	11 Missing ⚠️
hugr-py/src/hugr/std/collections/value_array.py	81.63%	9 Missing ⚠️
...src/std_extensions/collections/array/op_builder.rs	40.00%	1 Missing and 5 partials ⚠️
hugr-core/src/std_extensions/collections/array.rs	81.48%	4 Missing and 1 partial ⚠️
...c/std_extensions/collections/array/array_repeat.rs	77.27%	5 Missing ⚠️
...src/std_extensions/collections/array/array_scan.rs	78.26%	5 Missing ⚠️
... and 2 more

Additional details and impacted files

@@                  Coverage Diff                   @@
##           release-rs-v0.16.0    #2109      +/-   ##
======================================================
- Coverage               83.49%   83.38%   -0.11%     
======================================================
  Files                     219      225       +6     
  Lines                   42188    42335     +147     
  Branches                38290    38388      +98     
======================================================
+ Hits                    35225    35303      +78     
- Misses                   5150     5215      +65     
- Partials                 1813     1817       +4     

Flag	Coverage Δ
python	85.68% <82.00%> (-0.06%)	⬇️
rust	83.15% <82.21%> (-0.12%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.