Releases: kedro-org/kedro
0.19.8
Major features and improvements
- Made default run entrypoint in
__main__.py
work in interactive environments such as IPyhon and Databricks.
Bug fixes and other changes
- Fixed a bug that caused tracebacks disappeared from CLI runs.
- Moved
_find_run_command()
and_find_run_command_in_plugins()
from__main__.py
in the project template to the framework itself. - Fixed a bug where
%load_node
breaks with multi-lines import statements. - Fixed a regression where
rich
mark up logs stop showing since 0.19.7.
Breaking changes to the API
Documentation changes
- Add clarifications in docs explaining how runtime parameter resolution works.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.7
Major features and improvements
- Exposed
load
andsave
publicly for each dataset in the corekedro
library, and enabled other datasets to do the same. If a dataset doesn't exposeload
orsave
publicly, Kedro will fall back to using_load
or_save
, respectively. - Kedro commands are now lazily loaded to add performance gains when running Kedro commands.
- Implemented key completion support for accessing datasets in the
DataCatalog
. - Implemented dataset pretty printing.
- Implemented
DataCatalog
pretty printing. - Moved to an opt-out model for telemetry, enabling it by default without requiring prior consent.
Bug fixes and other changes
- Updated error message for invalid catalog entries.
- Updated error message for catalog entries when the dataset class is not found with hints on how to resolve the issue.
- Fixed a bug in the
DataCatalog
shallow_copy()
method to ensure it returns the type of the used catalog and doesn't cast it toDataCatalog
. - Made kedro-telemetry a core dependency.
- Fixed a bug when
OmegaConfigLoader
is printed, there are few missing arguments. - Fixed a bug when where iterating
OmegaConfigLoader
'skeys
return empty dictionary.
Breaking changes to the API
Upcoming deprecations for Kedro 0.20.0
- The utility method
get_pkg_version()
is deprecated and will be removed in Kedro 0.20.0.
Documentation changes
- Improved documentation for configuring dataset parameters in the data catalog
- Extended documentation with an example of logging customisation at runtime
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.6
Major features and improvements
- Added
raise_errors
argument tofind_pipelines
. IfTrue
, the first pipeline for which autodiscovery fails will cause an error to be raised. The default behaviour is still to raise a warning for each failing pipeline. - It is now possible to use Kedro without having
rich
installed. - Updated custom logging behavior:
conf/logging.yml
will be used if it exists andKEDRO_LOGGING_CONFIG
is not set; otherwise,default_logging.yml
will be used.
Bug fixes and other changes
- User defined catch-all dataset factory patterns now override the default pattern provided by the runner.
Breaking changes to the API
Upcoming deprecations for Kedro 0.20.0
- All micro-packaging commands (
kedro micropkg pull
,kedro micropkg package
) are deprecated and will be removed in Kedro 0.20.0.
Documentation changes
- Improved documentation for custom starters
- Added a new docs section on deploying Kedro project on AWS Airflow MWAA
- Detailed instructions on using
globals
andruntime_params
with theOmegaConfigLoader
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.5
Bug fixes and other changes
- Fixed breaking import issue when working on a project with
kedro-viz
on python 3.8.
Documentation changes
- Updated the documentation for deploying a Kedro project with Astronomer Airflow.
- Used
kedro-sphinx-theme
for documentation.
0.19.4
Major features and improvements
- Kedro commands now work from any subdirectory within a Kedro project.
- Kedro CLI now provides a better error message when project commands are run outside of a project i.e.
kedro run
- Added the
--telemetry
flag tokedro new
, allowing the user to register consent to have user analytics collected at the same time as the project is created. - Improved the performance of
Pipeline
object creation and summing. - Improved suggestions to resume failed pipeline runs.
- Dropped the dependency on
toposort
in favour of the built-ingraphlib
module. - Cookiecutter errors are shown in short format without the
--verbose
flag.
Bug fixes and other changes
- Updated
kedro pipeline create
andkedro pipeline delete
to read the base environment from the project settings. - Updated CLI command
kedro catalog resolve
to read credentials properly. - Changed the path of where pipeline tests generated with
kedro pipeline create
from<project root>/src/tests/pipelines/<pipeline name>
to<project root>/tests/pipelines/<pipeline name>
. - Updated
.gitignore
to prevent pushing Mlflow local runs folder to a remote forge when using mlflow and git. - Fixed error handling message for malformed yaml/json files in OmegaConfigLoader.
- Fixed a bug in
node
-creation allowing self-dependencies when using transcoding, that is datasets named likename@format
. - Improved error message when passing wrong value to node.
Breaking changes to the API
- Methods
_is_project
and_find_kedro_project
have been moved tokedro.utils
. We recommend not using private methods in your code, but if you do, please update your code to use the new location.
Documentation changes
- Added missing description for
merge_strategy
argument in OmegaConfigLoader. - Added documentation on best practices for testing nodes and pipelines.
- Clarified docs around using custom resolvers without a full Kedro project.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.3
Major features and improvements
- Create the debugging line magic
%load_node
for Jupyter Notebook and Jupyter Lab. - Add better IPython, VSCode Notebook support for
%load_node
and minimal support for Databricks. - Add full Kedro Node input syntax for
%load_node
.
Bug fixes and other changes
- Updated CLI Command
kedro catalog resolve
to work with dataset factories that usePartitionedDataset
. - Addressed arbitrary file write via archive extraction security vulnerability in micropackaging.
- Added the
_EPHEMERAL
attribute toAbstractDataset
and other Dataset classes that inherit from it. - Added new JSON Schema that works with Kedro versions 0.19.*
Breaking changes to the API
Documentation changes
- Enable read-the-docs search when user presses Command/Ctrl + K.
- Added documentation for
kedro-telemetry
and the data collected by it.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
0.19.2
Bug fixes and other changes
- Removed example pipeline requirements when examples are not selected in
tools
. - Allowed modern versions of JupyterLab and Jupyter Notebooks.
- Removed setuptools dependency
- Added
source_dir
explicitly inpyproject.toml
for non-src layout project. MemoryDataset
entries are now included in free outputs.- Removed black dependency and replaced it functionality with
ruff format
. - Added logging about not using async mode in
SequentiallRunner
andParallelRunner
.
Breaking changes to the API
- Changed input format for tools option obtained from --config file from numbers to short names.
Documentation changes
- Added documentation about
bootstrap_project
andconfigure_project
. - Added documentation about
kedro run
and hook execution order.
0.19.1
0.19.0
🚀 Major Features and improvements
- Dropped Python 3.7 support.
- Introduced project tools and example to the kedro new CLI flow.
- The new spaceflights starters, spaceflights-pandas, spaceflights-pandas-viz, spaceflights-pyspark, and spaceflights-pyspark-viz can be used with the kedro new command with the
--starter
flag. - Added the
--conf-source
option to%reload_kedro
, allowing users to specify a source for project configuration. - Added the functionality to choose a merging strategy for config files loaded with OmegaConfigLoader.
- Modified the mechanism of importing datasets, raise more explicit error when dependencies are missing.
- Added validation for configuration file used to override run commands via the CLI.
- Moved the default environment base and local from config loader to
_ProjectSettings
. This enables the use of config loader as a standalone class without affecting existing Kedro Framework users.
🪲 Bug fixes and other changes
- Added a new field tools to pyproject.toml when a project is created.
- Reduced spaceflights data to minimise waiting times during tutorial execution.
- Added validation to node tags to be consistent with node names.
- Removed pip-tools as a dependency.
- Accepted path-like filepaths more broadly for datasets.
💥 Breaking changes
- Removed ConfigLoader and TemplatedConfigLoader.
- Removed kedro.extras.datasets and tests (use kedro-datasets instead)
- Removed PartitionedDataset and IncrementalDataset from
kedro.io
(import them from kedro-datasets instead) - logging is removed from OmegaConfigLoader in favour of the environment variable
KEDRO_LOGGING_CONFIG
. - Removed support for defining the layer attribute at top-level within DataCatalog.
- Renamed
data_set
and DataSet to dataset and Dataset everywhere. - Removed the
create_default_data_set()
method in the Runner in favour of using dataset factories to create default dataset instances. - The default project template now has only one pyproject.toml at the root of the project (containing both the packaging metadata and the Kedro build config).
✍️ Documentation changes
- Added new top navigation to easily switch between Framework, Viz, and Datasets.
- Added new search-as-you-type to improve the search experience.
New Contributors
- @MinuraPunchihewa made their first contribution in #3115
- @mustious made their first contribution in #3181
- @JayOaks made their first contribution in #3239
- @adamkells made their first contribution in #3203
- @HKABIG made their first contribution in #3270
- @pdave34 made their first contribution in #3213
- @hermlon made their first contribution in #3303
Full Changelog: 0.18.14...0.19.0
🚨 If you are upgrading from Kedro 0.18, have a look at the migration guide.
We welcome every community contribution, large or small. See what we're working on now and report bugs or suggest future features.
Until next time,
The Kedro Team 💛
0.18.14
Release 0.18.14
Major features and improvements
- Allowed using of custom cookiecutter templates for creating pipelines with
--template
flag forkedro pipeline create
or viatemplate/pipeline
folder. - Allowed overriding of configuration keys with runtime parameters using the
runtime_params
resolver withOmegaConfigLoader
.
Bug fixes and other changes
- Updated dataset factories to resolve nested catalog config properly.
- Updated
OmegaConfigLoader
to handle paths containing dots outside ofconf_source
. - Made
settings.py
optional.
Documentation changes
- Added documentation to clarify execution order of hooks.
- Added a notebook example for spaceflights to illustrate how to incrementally add Kedro features.
- Moved documentation for the
standalone-datacatalog
starter into its README file. - Added new documentation about deploying a Kedro project with Amazon EMR.
- Added new documentation about how to publish a Kedro-Viz project to make it shareable.
- New TSC members added to the page and the organisation of each member is also now listed.
- Plus some minor bug fixes and changes across the documentation.
Upcoming deprecations for Kedro 0.19.0
- All dataset classes will be removed from the core Kedro repository (
kedro.extras.datasets
). Install and import them from thekedro-datasets
package instead. - All dataset classes ending with
DataSet
are deprecated and will be removed in Kedro0.19.0
andkedro-datasets
2.0.0
. Instead, use the updated class names ending withDataset
. - The starters
pandas-iris
,pyspark-iris
,pyspark
, andstandalone-datacatalog
are deprecated and will be archived in Kedro 0.19.0. PartitionedDataset
andIncrementalDataset
have been moved tokedro-datasets
and will be removed in Kedro0.19.0
. Install and import them from thekedro-datasets
package instead.
Community contributions
Many thanks to the following Kedroids for contributing PRs to this release: