0.19.0

What's Changed

• Support block_id for general_blockwise functions by @tomwhite in #593
• Run tests on Python 3.13 by @tomwhite in #503
• Fix dtypes for QR by @tomwhite in #594
• Change usages of map_direct in QR to general_blockwise by @tomwhite in #597
• Implement var and std using a numerically stable parallel algorithm by @tomwhite in #596
• Add map_selection primitive to replace map_direct by @tomwhite in #595
• Use unstack from array-api-compat by @tomwhite in #602
• Increase allowed_mem default to 2GB by @tomwhite in #599
• Implement structured arrays as Zarr group of arrays by @tomwhite in #603
• Implement int array indexing using map_selection by @tomwhite in #604
• Implement merge_chunks using map_selection by @tomwhite in #605
• Free memory earlier by changing function scope by @tomwhite in #606
• Fix groupby tests running on tensorstore by @tomwhite in #608
• Implement concat using general_blockwise by @tomwhite in #607
• Speed up test collection by @tomwhite in #609
• Update GH actions to macos-13 test runner and upload-artifact@v4 by @tomwhite in #611
• Implement repeat by @tomwhite in #610
• Map nested improvements by @tomwhite in #613
• SVD by @tomwhite in #598
• Unify chunks for concat, and check preconditions by @tomwhite in #616
• Defer to merge_chunks in special cases of rechunk by @tomwhite in #612
• Document map_selection core operation by @tomwhite in #617
• Change default max_total_num_input_blocks to 10 by @tomwhite in #615
• Update slides - November 2024 by @tomwhite in #620
• Change use_backups default to True only on cloud stores by @tomwhite in #619
• Various minor doc improvements by @tomwhite in #625
• Make threads the default executor by @tomwhite in #621
• Rewrite Cubed intro docs by @tomwhite in #622
• More doc improvements: simplify getting started page by @tomwhite in #629
• Add examples to docs by @tomwhite in #630

Full Changelog: 0.18.0...0.19.0

0.18.0

What's Changed

• Multiple outputs by @tomwhite in #419
• Support use of multiprocessing start methods other than "spawn" (e.g. "dragon") by @applio in #554
• Tensorstore compression fix by @tomwhite in #574
• Implement unstack using multiple outputs by @tomwhite in #575
• Handle more memray allocation types by @tomwhite in #578
• Drop Python 3.9 by @tomwhite in #576
• QR decomposition by @tomwhite in #577
• Expose the linalg namespace and include in status page by @tomwhite in #581
• Fix outer result dtype by @tomwhite in #582
• Add new 2023.12 elemwise functions by @tomwhite in #583
• Bring vecdot implementation in line with the one in array-api-compat by @tomwhite in #402
• Zarr v3 now requires at least Python 3.11 by @tomwhite in #585
• Debug failing straggler test by @tomwhite in #580
• Set object codec for object arrays by @tomwhite in #573
• Fix for flaky straggler test by @tomwhite in #587
• Use general_blockwise in case of chunk-aligned selections in index by @tomwhite in #586
• Remove deprecated code by @tomwhite in #588
• Documentation small fixes by @tomwhite in #591
• Add page on optimization to user guide by @tomwhite in #397
• Remove old reduction implementation by @tomwhite in #589
• Add array API inspection utilities by @tomwhite in #592

New Contributors

• @applio made their first contribution in #554

Full Changelog: 0.17.0...0.18.0

0.17.0

What's Changed

• Skip fewer array API tests by @tomwhite in #547
• Rechunk where dict has missing axes by @tomwhite in #546
• Adds logging [INFO] to TimelineVisualizationCallback and HistoryCallback by @norlandrhagen in #548
• More lenient dtype support by @tomwhite in #550
• Cast inputs to Cubed arrays in apply_ufunc by @tomwhite in #551
• Fix bug where newaxis with full slices doesn't add new axes by @tomwhite in #559
• Memray integration by @tomwhite in #558
• Add chunkmem to VirtualFullArray, which was missed in #447 by @tomwhite in #560
• Fix Array API Tests by @tomwhite in #562
• Memray callback by @tomwhite in #561
• Introduce FinalizedPlan by @tomwhite in #563
• Materialize arrays optimizer bugfix by @tomwhite in #564
• Align tqdm progress bars by @tomwhite in #567
• Coerce args to map_blocks to arrays by @tomwhite in #566
• Fail during planning if map_blocks drop_axis is for a chunked dimension by @tomwhite in #569
• Fix Rich progress bar flickering by limiting refreshes by @tomwhite in #571
• Allow Zarr compression to be set for intermediate files by @tomwhite in #572

Full Changelog: 0.16.0...0.17.0

0.16.0

What's Changed

• Improve documentation around Array API support by @tomwhite in #477
• Speed up tests by @JosepSampe in #478
• Add normalize_shape since it is not in Zarr's public API by @tomwhite in #479
• Implement virtual array indexing using ndindex by @tomwhite in #441
• Ndindex indexing by @tomwhite in #481
• Pluggable backend storage by @tomwhite in #480
• Ensure integer arrays used for indexing are NumPy arrays by @tomwhite in #485
• Indicate that we support both 2021.12 and 2022.12 versions of the Array API by @tomwhite in #487
• TensorStore storage by @tomwhite in #483
• Update slides - June 2024 by @tomwhite in #489
• Fail with NotImplementedError if backend Zarr is missing nchunks_initialized by @tomwhite in #491
• Zarr v3 by @tomwhite in #484
• Remove print for local executors by @tomwhite in #506
• document what needs to be installed to run climatology anomaly example by @rbavery in #501
• Document how to run examples on local machine using processes by @tomwhite in #507
• Check allowed_mem does not exceed total on machine for processes executor by @tomwhite in #517
• Warn if peak mem exceeds allowed_mem by @tomwhite in #516
• Remove max_tasks_per_child=1 limitation from processes executor by @tomwhite in #515
• Zarr Python v3 updates by @tomwhite in #523
• Modal progress fix by @tomwhite in #522
• Delete local context directories on exit by @tomwhite in #524
• Update demo notebooks by @tomwhite in #525
• Update CUBED_CONFIG path in example README.md by @thodson-usgs in #530
• Implement flip by @tomwhite in #528
• Add better test for local paths in delete_on_exit by @tomwhite in #532
• Deprecate cubed.extensions and move to cubed.diagnostics by @tomwhite in #533
• Turning on JAX tests for all PRs. by @alxmrs in #537
• Improve set-up instructions for examples by @tomwhite in #534
• Configuration documentation by @tomwhite in #535
• Adding compile_function as execute option. by @alxmrs in #536
• Diagnostics documentation by @tomwhite in #540

New Contributors

• @JosepSampe made their first contribution in #478
• @rbavery made their first contribution in #501
• @thodson-usgs made their first contribution in #530
• @alxmrs made their first contribution in #537

Full Changelog: 0.15.0...0.16.0

0.15.0

What's Changed

• Test on Python 3.12 by @tomwhite in #453
• Test NumPy 2 by @tomwhite in #454
• Expose Lithops wait_dur_sec to speed up tests by @tomwhite in #456
• Simplify executor naming by @tomwhite in #457
• Treat warnings as errors when running tests by @tomwhite in #458
• Re-interpret 'fusable' to mean with predecessor operations by @tomwhite in #460
• Don't include hidden arrays when counting max_total_source_arrays by @tomwhite in #463
• Limited implementation of map_overlap by @tomwhite in #462
• Limited implementation of pad by @tomwhite in #461
• Implement roll by @tomwhite in #465
• Fix map_overlap in case of single chunk in overlap dimension by @tomwhite in #466
• Fix case of Lithops 3.4.0 hanging by @tomwhite in #471
• Fix unify_chunks to return regular chunks in all cases by @tomwhite in #470
• Improve test coverage by @tomwhite in #472
• Change development status to beta by @tomwhite in #474
• Top level array namespace by @tomwhite in #473
• Add notebooks for Pangeo examples by @tomwhite in #476

Full Changelog: 0.14.3...0.15.0

0.14.3

What's Changed

• Take broadcast trick into account for array chunk memory by @tomwhite in #447
• Fix Documentation link by @norlandrhagen in #451
• Include j2 templates in the distribution by @tomwhite in #450
• Add links to blog posts by @TomNicholas in #452
• Add a groupby_blockwise function for use in Flox by @tomwhite in #448

New Contributors

• @norlandrhagen made their first contribution in #451

Full Changelog: 0.14.2...0.14.3

0.14.2

What's Changed

• Allow networkx 3.3 now that pydot issue has been fixed by @tomwhite in #445
• Change drop_axis to default to None rather than an empty list in map_blocks by @tomwhite in #446
• Add a groupby_reduction function for use in Flox by @tomwhite in #442

Full Changelog: 0.14.1...0.14.2

0.14.1

What's Changed

• Providing filesystem credentials through storage_options by @balanz24 in #436
• Only use oindex for advanced indexing (int arrays) by @tomwhite in #440
• Deprecate misspelled aggegrate_func in favour of aggregate_func by @tomwhite in #444

New Contributors

• @balanz24 made their first contribution in #436

Full Changelog: 0.14.0...0.14.1

0.14.0

What's Changed

• Handle zero-length slices in index function by @tomwhite in #401
• Run unit tests on Python 3.11 on Linux by @tomwhite in #405
• Update examples to use Python 3.11 by @tomwhite in #406
• Remove unused lazy_full function (replaced by virtual_full) by @tomwhite in #409
• Delegate dtype functions to backend array API by @tomwhite in #410
• Add a 'processes' executor that runs using local processes by @tomwhite in #411
• Mark stack operation as not fusable by @tomwhite in #415
• Track source array names in PrimitiveOperation by @tomwhite in #416
• Fix Modal deprecations by @tomwhite in #417
• Make the new multiple-inputs optimization function the default by @tomwhite in #412
• Add cloudpickle dependency following #411 ('processes' executor) by @tomwhite in #421
• Documentation corrections following optimization improvements by @tomwhite in #423
• Add name property to all executors by @tomwhite in #424
• Build Lithops images for every push to main by @tomwhite in #422
• Make Lithops example names consistent with benchmarks and other executors by @tomwhite in #425
• Delete existing Lithops image before rebuilding by @tomwhite in #427
• Rename block_function to key_function in blockwise by @tomwhite in #426
• Use executor name rather than class name by @tomwhite in #430
• Remove unused sync version of Modal executor by @tomwhite in #431
• Change total_nbytes to total_nbytes_written by @tomwhite in #433
• Fix deprecation warning on logger.warn by @tomwhite in #434

Full Changelog: 0.13.0...0.14.0

0.13.0

What's Changed

• Introduce finalize_dag method in Plan that caches DAG optimization by @tomwhite in #340
• Add VirtualInMemoryArray that keeps small arrays in memory by @tomwhite in #336
• Fix Modal test following #336 by @tomwhite in #341
• Change the DAG to have separate nodes for operations and arrays by @tomwhite in #337
• Allow more control over optimizations applied by @tomwhite in #342
• Handle the case where an array is created by calling map_blocks with no input arrays by @tomwhite in #343
• Add a test for visualizing the quad means computation by @tomwhite in #345
• Show nbytes for each array and total on visualization by @tomwhite in #344
• Fuse multiple inputs by @tomwhite in #346
• Upgrade to sphinx-notes/pages v3 by @tomwhite in #347
• Minor improvements to DAG visualization by @tomwhite in #348
• Remove deprecated code by @tomwhite in #349
• Add generated doc for primitive and core ops to design page by @tomwhite in #351
• Support arbitrary block index functions in blockwise by @tomwhite in #350
• Refactor pipeline to reduce coupling between runtime and primitive layers by @tomwhite in #352
• Implement stack using general_blockwise by @tomwhite in #353
• Implement reshape_chunks using general_blockwise by @tomwhite in #354
• More control over operation fusion by @tomwhite in #355
• Minor memory fixes by @tomwhite in #358
• Use Dask's 'broadcast trick' to save memory for single-valued arrays by @tomwhite in #359
• Add a show_hidden option to visualize by @tomwhite in #360
• Reinstate slow tests by @tomwhite in #361
• Add function_nargs to BlockwiseSpec by @tomwhite in #363
• Add missing api doc by @tomwhite in #364
• Implement partial_reduce and tree_reduce using generalized blockwise by @tomwhite in #365
• Quad means validation test by @tomwhite in #367
• Take allowed_mem into account when fusing primitive operations by @tomwhite in #366
• Fuse operations with different numbers of tasks by @tomwhite in #368
• Show number of tasks prominently on graph visualization by @tomwhite in #371
• Introduce event objects for all callback methods by @tomwhite in #373
• Minor Modal improvements by @tomwhite in #375
• Run tests using JAX as the backend array API (on CPU) by @tomwhite in #374
• Introduce max_total_num_input_blocks as a heuristic to control fusion by @tomwhite in #376
• Rename array_name to name in runtime following #337 by @tomwhite in #377
• Enable backup tasks by default for all executors that support them by @tomwhite in #378
• Remove unused array_names parameter in executors by @tomwhite in #380
• Introduce compute_id for each computation by @tomwhite in #382
• Add a Rich progress bar by @tomwhite in #383
• Change timeline visualization default format to SVG by @tomwhite in #384
• Preserve laziness in fuse_multiple to avoid materializing array blocks in _partial_reduce by @tomwhite in #386
• Fix Modal unit tests following change in #382 by @tomwhite in #387
• Add debug log statements to optimizer to see why nodes are fused by @tomwhite in #388
• Allow optional path in from_zarr and to_zarr by @tomwhite in #391
• Remove CUBED_MODAL_FORCE_BUILD env var … by @tomwhite in #393
• Add introductory slides to docs by @tomwhite in #394
• Use donfig to control default spec by @tomwhite in #392
• small changes to coiled/aws examples by @rsignell in #369
• Skip vecdot test due to zero-size array failures by @tomwhite in #399
• Show both operation ID and name in visualization by @tomwhite in #396
• Add support for minimum_workers in Coiled by @tomwhite in #398
• Create a single set of example scripts that can run on any executor by @tomwhite in #395

New Contributors

• @rsignell made their first contribution in #369

Full Changelog: 0.12.0...0.13.0