Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preliminary version of Blosc2 module #463

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

FrancescAlted
Copy link

This is a preliminary version of a blosc2.pyx module for numcodecs. It is essentially the same than blosc.pyx, but the API has been updated to use Blosc2 instead. A new C-Blosc2 submodule has been added too.

However, C-Blosc2 requires cmake to build, and we normally use something like scikit-build-core. Unfortunately this means quite a bit of changes into the current build procedure based on setuptools; would you be up to adopt these changes? If you don't, probably Blosc2 can be made to work with existing setuptools, but we still need to figure out how.

TODO:

  • Update setup.py / pyproject.toml to compile the C-Blosc2 library
  • Unit tests and/or doctests in docstrings
  • Tests pass locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • Docs build locally
  • GitHub Actions CI passes
  • Test coverage to 100% (Codecov passes)

@mkitti
Copy link
Contributor

mkitti commented Sep 20, 2023

I think this is an important step to take in the right direction.

Later, we should also consider a deprecation procedure for writing Blosc v1. Reading Blosc v1 should be able continue indefinitely via the c-blosc2 library

@FrancescAlted, do you have a timeline for how much longer c-blosc1 will be maintained?

@FrancescAlted
Copy link
Author

@FrancescAlted, do you have a timeline for how much longer c-blosc1 will be maintained?

Well, the Blosc team would be supporting C-Blosc1 for as long as possible. Having said that, all the action is happening in C-Blosc2 right now, and C-Blosc1 will only receive fixes in the future.

@dstansby dstansby added the New codec Suggestion for a new codec label Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs release notes New codec Suggestion for a new codec
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants