Skip to content

Releases: kedro-org/kedro

0.19.10

26 Nov 17:44
e2c241b
Compare
Choose a tag to compare

Major features and improvements

  • Add official support for Python 3.13.
  • Implemented dict-like interface for KedroDataCatalog.
  • Implemented lazy dataset initializing for KedroDataCatalog.
  • Project dependencies on both the default template and on starter templates are now explicitly declared on the pyproject.toml file, allowing Kedro projects to work with project management tools like uv, pdm, and rye.

Note: KedroDataCatalog is an experimental feature and is under active development. Therefore, it is possible we'll introduce breaking changes to this class, so be mindful of that if you decide to use it already. Let us know if you have any feedback about the KedroDataCatalog or ideas for new features.

Bug fixes and other changes

  • Added I/O support for Oracle Cloud Infrastructure (OCI) Object Storage filesystem.
  • Fixed DatasetAlreadyExistsError for ThreadRunner when Kedro project run and using runner separately.

Breaking changes to the API

Documentation changes

  • Added Databricks Asset Bundles deployment guide.
  • Added a new minimal Kedro project creation guide.
  • Added example to explain how dataset factories work.
  • Updated CLI autocompletion docs with new Click syntax.
  • Standardised .parquet suffix in docs and tests.

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

0.19.9

10 Oct 19:13
91468e6
Compare
Choose a tag to compare

Major features and improvements

  • Dropped Python 3.8 support.
  • Implemented KedroDataCatalog repeating DataCatalog functionality with a few API enhancements:
    • Removed _FrozenDatasets and access datasets as properties;
    • Added get dataset by name feature;
    • add_feed_dict() was simplified to only add raw data;
    • Datasets' initialisation was moved out from from_config() method to the constructor.
  • Moved development requirements from requirements.txt to the dedicated section in pyproject.toml for project template.
  • Implemented Protocol abstraction for the current DataCatalog and adding new catalog implementations.
  • Refactored kedro run and kedro catalog commands.
  • Moved pattern resolution logic from DataCatalog to a separate component - CatalogConfigResolver. Updated DataCatalog to use CatalogConfigResolver internally.
  • Made packaged Kedro projects return session.run() output to be used when running it in the interactive environment.
  • Enhanced OmegaConfigLoader configuration validation to detect duplicate keys at all parameter levels, ensuring comprehensive nested key checking.

Note: KedroDataCatalog is an experimental feature and is under active development. Therefore, it is possible we'll introduce breaking changes to this class, so be mindful of that if you decide to use it already. Let us know if you have any feedback about the KedroDataCatalog or ideas for new features.

Bug fixes and other changes

  • Fixed bug where using dataset factories breaks with ThreadRunner.
  • Fixed a bug where SharedMemoryDataset.exists would not call the underlying MemoryDataset.
  • Fixed template projects example tests.
  • Made credentials loading consistent between KedroContext._get_catalog() and resolve_patterns so that both use _get_config_credentials()

Breaking changes to the API

  • Removed ShelveStore to address a security vulnerability.

Documentation changes

  • Fix logo on PyPI page.
  • Minor language/styling updates.

Community contributions

0.19.8

22 Aug 13:46
ba98135
Compare
Choose a tag to compare

Major features and improvements

  • Made default run entrypoint in __main__.py work in interactive environments such as IPyhon and Databricks.

Bug fixes and other changes

  • Fixed a bug that caused tracebacks disappeared from CLI runs.
  • Moved _find_run_command() and _find_run_command_in_plugins() from __main__.py in the project template to the framework itself.
  • Fixed a bug where %load_node breaks with multi-lines import statements.
  • Fixed a regression where rich mark up logs stop showing since 0.19.7.

Breaking changes to the API

Documentation changes

  • Add clarifications in docs explaining how runtime parameter resolution works.

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

0.19.7

01 Aug 18:53
94ae223
Compare
Choose a tag to compare

Major features and improvements

  • Exposed load and save publicly for each dataset in the core kedro library, and enabled other datasets to do the same. If a dataset doesn't expose load or save publicly, Kedro will fall back to using _load or _save, respectively.
  • Kedro commands are now lazily loaded to add performance gains when running Kedro commands.
  • Implemented key completion support for accessing datasets in the DataCatalog.
  • Implemented dataset pretty printing.
  • Implemented DataCatalog pretty printing.
  • Moved to an opt-out model for telemetry, enabling it by default without requiring prior consent.

Bug fixes and other changes

  • Updated error message for invalid catalog entries.
  • Updated error message for catalog entries when the dataset class is not found with hints on how to resolve the issue.
  • Fixed a bug in the DataCatalog shallow_copy() method to ensure it returns the type of the used catalog and doesn't cast it to DataCatalog.
  • Made kedro-telemetry a core dependency.
  • Fixed a bug when OmegaConfigLoader is printed, there are few missing arguments.
  • Fixed a bug when where iterating OmegaConfigLoader's keys return empty dictionary.

Breaking changes to the API

Upcoming deprecations for Kedro 0.20.0

  • The utility method get_pkg_version() is deprecated and will be removed in Kedro 0.20.0.

Documentation changes

  • Improved documentation for configuring dataset parameters in the data catalog
  • Extended documentation with an example of logging customisation at runtime

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

0.19.6

27 May 16:32
011e3a8
Compare
Choose a tag to compare

Major features and improvements

  • Added raise_errors argument to find_pipelines. If True, the first pipeline for which autodiscovery fails will cause an error to be raised. The default behaviour is still to raise a warning for each failing pipeline.
  • It is now possible to use Kedro without having rich installed.
  • Updated custom logging behavior: conf/logging.yml will be used if it exists and KEDRO_LOGGING_CONFIG is not set; otherwise, default_logging.yml will be used.

Bug fixes and other changes

  • User defined catch-all dataset factory patterns now override the default pattern provided by the runner.

Breaking changes to the API

Upcoming deprecations for Kedro 0.20.0

  • All micro-packaging commands (kedro micropkg pull, kedro micropkg package) are deprecated and will be removed in Kedro 0.20.0.

Documentation changes

  • Improved documentation for custom starters
  • Added a new docs section on deploying Kedro project on AWS Airflow MWAA
  • Detailed instructions on using globals and runtime_params with the OmegaConfigLoader

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

0.19.5

22 Apr 14:50
e81dadb
Compare
Choose a tag to compare

Bug fixes and other changes

  • Fixed breaking import issue when working on a project with kedro-viz on python 3.8.

Documentation changes

  • Updated the documentation for deploying a Kedro project with Astronomer Airflow.
  • Used kedro-sphinx-theme for documentation.

0.19.4

17 Apr 17:31
b9e99dd
Compare
Choose a tag to compare

Major features and improvements

  • Kedro commands now work from any subdirectory within a Kedro project.
  • Kedro CLI now provides a better error message when project commands are run outside of a project i.e. kedro run
  • Added the --telemetry flag to kedro new, allowing the user to register consent to have user analytics collected at the same time as the project is created.
  • Improved the performance of Pipeline object creation and summing.
  • Improved suggestions to resume failed pipeline runs.
  • Dropped the dependency on toposort in favour of the built-in graphlib module.
  • Cookiecutter errors are shown in short format without the --verbose flag.

Bug fixes and other changes

  • Updated kedro pipeline create and kedro pipeline delete to read the base environment from the project settings.
  • Updated CLI command kedro catalog resolve to read credentials properly.
  • Changed the path of where pipeline tests generated with kedro pipeline create from <project root>/src/tests/pipelines/<pipeline name> to <project root>/tests/pipelines/<pipeline name>.
  • Updated .gitignore to prevent pushing Mlflow local runs folder to a remote forge when using mlflow and git.
  • Fixed error handling message for malformed yaml/json files in OmegaConfigLoader.
  • Fixed a bug in node-creation allowing self-dependencies when using transcoding, that is datasets named like name@format.
  • Improved error message when passing wrong value to node.

Breaking changes to the API

  • Methods _is_project and _find_kedro_project have been moved to kedro.utils. We recommend not using private methods in your code, but if you do, please update your code to use the new location.

Documentation changes

  • Added missing description for merge_strategy argument in OmegaConfigLoader.
  • Added documentation on best practices for testing nodes and pipelines.
  • Clarified docs around using custom resolvers without a full Kedro project.

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

0.19.3

27 Feb 12:44
d3e9027
Compare
Choose a tag to compare

Major features and improvements

  • Create the debugging line magic %load_node for Jupyter Notebook and Jupyter Lab.
  • Add better IPython, VSCode Notebook support for %load_node and minimal support for Databricks.
  • Add full Kedro Node input syntax for %load_node.

load-node-debug

Bug fixes and other changes

  • Updated CLI Command kedro catalog resolve to work with dataset factories that use PartitionedDataset.
  • Addressed arbitrary file write via archive extraction security vulnerability in micropackaging.
  • Added the _EPHEMERAL attribute to AbstractDataset and other Dataset classes that inherit from it.
  • Added new JSON Schema that works with Kedro versions 0.19.*

Breaking changes to the API

Documentation changes

  • Enable read-the-docs search when user presses Command/Ctrl + K.
  • Added documentation for kedro-telemetry and the data collected by it.

Community contributions

Many thanks to the following Kedroids for contributing PRs to this release:

0.19.2

22 Jan 11:10
1927f9b
Compare
Choose a tag to compare

Bug fixes and other changes

  • Removed example pipeline requirements when examples are not selected in tools.
  • Allowed modern versions of JupyterLab and Jupyter Notebooks.
  • Removed setuptools dependency
  • Added source_dir explicitly in pyproject.toml for non-src layout project.
  • MemoryDataset entries are now included in free outputs.
  • Removed black dependency and replaced it functionality with ruff format.
  • Added logging about not using async mode in SequentiallRunner and ParallelRunner.

Breaking changes to the API

  • Changed input format for tools option obtained from --config file from numbers to short names.

Documentation changes

  • Added documentation about bootstrap_project and configure_project.
  • Added documentation about kedro run and hook execution order.

0.19.1

13 Dec 13:48
99e194b
Compare
Choose a tag to compare

Release 0.19.1

What's Changed