Releases: kedro-org/kedro
0.19.10
Major features and improvements
- Add official support for Python 3.13.
- Implemented dict-like interface for
KedroDataCatalog
. - Implemented lazy dataset initializing for
KedroDataCatalog
. - Project dependencies on both the default template and on starter templates are now explicitly declared on the
pyproject.toml
file, allowing Kedro projects to work with project management tools likeuv
,pdm
, andrye
.
Note: KedroDataCatalog
is an experimental feature and is under active development. Therefore, it is possible we'll introduce breaking changes to this class, so be mindful of that if you decide to use it already. Let us know if you have any feedback about the KedroDataCatalog
or ideas for new features.
Bug fixes and other changes
- Added I/O support for Oracle Cloud Infrastructure (OCI) Object Storage filesystem.
- Fixed
DatasetAlreadyExistsError
forThreadRunner
when Kedro project run and using runner separately.
Breaking changes to the API
Documentation changes
- Added Databricks Asset Bundles deployment guide.
- Added a new minimal Kedro project creation guide.
- Added example to explain how dataset factories work.
- Updated CLI autocompletion docs with new Click syntax.
- Standardised
.parquet
suffix in docs and tests.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.9
Major features and improvements
- Dropped Python 3.8 support.
- Implemented
KedroDataCatalog
repeatingDataCatalog
functionality with a few API enhancements:- Removed
_FrozenDatasets
and access datasets as properties; - Added get dataset by name feature;
add_feed_dict()
was simplified to only add raw data;- Datasets' initialisation was moved out from
from_config()
method to the constructor.
- Removed
- Moved development requirements from
requirements.txt
to the dedicated section inpyproject.toml
for project template. - Implemented
Protocol
abstraction for the currentDataCatalog
and adding new catalog implementations. - Refactored
kedro run
andkedro catalog
commands. - Moved pattern resolution logic from
DataCatalog
to a separate component -CatalogConfigResolver
. UpdatedDataCatalog
to useCatalogConfigResolver
internally. - Made packaged Kedro projects return
session.run()
output to be used when running it in the interactive environment. - Enhanced
OmegaConfigLoader
configuration validation to detect duplicate keys at all parameter levels, ensuring comprehensive nested key checking.
Note: KedroDataCatalog
is an experimental feature and is under active development. Therefore, it is possible we'll introduce breaking changes to this class, so be mindful of that if you decide to use it already. Let us know if you have any feedback about the KedroDataCatalog
or ideas for new features.
Bug fixes and other changes
- Fixed bug where using dataset factories breaks with
ThreadRunner
. - Fixed a bug where
SharedMemoryDataset.exists
would not call the underlyingMemoryDataset
. - Fixed template projects example tests.
- Made credentials loading consistent between
KedroContext._get_catalog()
andresolve_patterns
so that both use_get_config_credentials()
Breaking changes to the API
- Removed
ShelveStore
to address a security vulnerability.
Documentation changes
- Fix logo on PyPI page.
- Minor language/styling updates.
Community contributions
0.19.8
Major features and improvements
- Made default run entrypoint in
__main__.py
work in interactive environments such as IPyhon and Databricks.
Bug fixes and other changes
- Fixed a bug that caused tracebacks disappeared from CLI runs.
- Moved
_find_run_command()
and_find_run_command_in_plugins()
from__main__.py
in the project template to the framework itself. - Fixed a bug where
%load_node
breaks with multi-lines import statements. - Fixed a regression where
rich
mark up logs stop showing since 0.19.7.
Breaking changes to the API
Documentation changes
- Add clarifications in docs explaining how runtime parameter resolution works.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.7
Major features and improvements
- Exposed
load
andsave
publicly for each dataset in the corekedro
library, and enabled other datasets to do the same. If a dataset doesn't exposeload
orsave
publicly, Kedro will fall back to using_load
or_save
, respectively. - Kedro commands are now lazily loaded to add performance gains when running Kedro commands.
- Implemented key completion support for accessing datasets in the
DataCatalog
. - Implemented dataset pretty printing.
- Implemented
DataCatalog
pretty printing. - Moved to an opt-out model for telemetry, enabling it by default without requiring prior consent.
Bug fixes and other changes
- Updated error message for invalid catalog entries.
- Updated error message for catalog entries when the dataset class is not found with hints on how to resolve the issue.
- Fixed a bug in the
DataCatalog
shallow_copy()
method to ensure it returns the type of the used catalog and doesn't cast it toDataCatalog
. - Made kedro-telemetry a core dependency.
- Fixed a bug when
OmegaConfigLoader
is printed, there are few missing arguments. - Fixed a bug when where iterating
OmegaConfigLoader
'skeys
return empty dictionary.
Breaking changes to the API
Upcoming deprecations for Kedro 0.20.0
- The utility method
get_pkg_version()
is deprecated and will be removed in Kedro 0.20.0.
Documentation changes
- Improved documentation for configuring dataset parameters in the data catalog
- Extended documentation with an example of logging customisation at runtime
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.6
Major features and improvements
- Added
raise_errors
argument tofind_pipelines
. IfTrue
, the first pipeline for which autodiscovery fails will cause an error to be raised. The default behaviour is still to raise a warning for each failing pipeline. - It is now possible to use Kedro without having
rich
installed. - Updated custom logging behavior:
conf/logging.yml
will be used if it exists andKEDRO_LOGGING_CONFIG
is not set; otherwise,default_logging.yml
will be used.
Bug fixes and other changes
- User defined catch-all dataset factory patterns now override the default pattern provided by the runner.
Breaking changes to the API
Upcoming deprecations for Kedro 0.20.0
- All micro-packaging commands (
kedro micropkg pull
,kedro micropkg package
) are deprecated and will be removed in Kedro 0.20.0.
Documentation changes
- Improved documentation for custom starters
- Added a new docs section on deploying Kedro project on AWS Airflow MWAA
- Detailed instructions on using
globals
andruntime_params
with theOmegaConfigLoader
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.5
Bug fixes and other changes
- Fixed breaking import issue when working on a project with
kedro-viz
on python 3.8.
Documentation changes
- Updated the documentation for deploying a Kedro project with Astronomer Airflow.
- Used
kedro-sphinx-theme
for documentation.
0.19.4
Major features and improvements
- Kedro commands now work from any subdirectory within a Kedro project.
- Kedro CLI now provides a better error message when project commands are run outside of a project i.e.
kedro run
- Added the
--telemetry
flag tokedro new
, allowing the user to register consent to have user analytics collected at the same time as the project is created. - Improved the performance of
Pipeline
object creation and summing. - Improved suggestions to resume failed pipeline runs.
- Dropped the dependency on
toposort
in favour of the built-ingraphlib
module. - Cookiecutter errors are shown in short format without the
--verbose
flag.
Bug fixes and other changes
- Updated
kedro pipeline create
andkedro pipeline delete
to read the base environment from the project settings. - Updated CLI command
kedro catalog resolve
to read credentials properly. - Changed the path of where pipeline tests generated with
kedro pipeline create
from<project root>/src/tests/pipelines/<pipeline name>
to<project root>/tests/pipelines/<pipeline name>
. - Updated
.gitignore
to prevent pushing Mlflow local runs folder to a remote forge when using mlflow and git. - Fixed error handling message for malformed yaml/json files in OmegaConfigLoader.
- Fixed a bug in
node
-creation allowing self-dependencies when using transcoding, that is datasets named likename@format
. - Improved error message when passing wrong value to node.
Breaking changes to the API
- Methods
_is_project
and_find_kedro_project
have been moved tokedro.utils
. We recommend not using private methods in your code, but if you do, please update your code to use the new location.
Documentation changes
- Added missing description for
merge_strategy
argument in OmegaConfigLoader. - Added documentation on best practices for testing nodes and pipelines.
- Clarified docs around using custom resolvers without a full Kedro project.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.3
Major features and improvements
- Create the debugging line magic
%load_node
for Jupyter Notebook and Jupyter Lab. - Add better IPython, VSCode Notebook support for
%load_node
and minimal support for Databricks. - Add full Kedro Node input syntax for
%load_node
.
Bug fixes and other changes
- Updated CLI Command
kedro catalog resolve
to work with dataset factories that usePartitionedDataset
. - Addressed arbitrary file write via archive extraction security vulnerability in micropackaging.
- Added the
_EPHEMERAL
attribute toAbstractDataset
and other Dataset classes that inherit from it. - Added new JSON Schema that works with Kedro versions 0.19.*
Breaking changes to the API
Documentation changes
- Enable read-the-docs search when user presses Command/Ctrl + K.
- Added documentation for
kedro-telemetry
and the data collected by it.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.2
Bug fixes and other changes
- Removed example pipeline requirements when examples are not selected in
tools
. - Allowed modern versions of JupyterLab and Jupyter Notebooks.
- Removed setuptools dependency
- Added
source_dir
explicitly inpyproject.toml
for non-src layout project. MemoryDataset
entries are now included in free outputs.- Removed black dependency and replaced it functionality with
ruff format
. - Added logging about not using async mode in
SequentiallRunner
andParallelRunner
.
Breaking changes to the API
- Changed input format for tools option obtained from --config file from numbers to short names.
Documentation changes
- Added documentation about
bootstrap_project
andconfigure_project
. - Added documentation about
kedro run
and hook execution order.