Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Kedro 19 #1680

Merged
merged 27 commits into from
Dec 15, 2023
Merged

Support for Kedro 19 #1680

merged 27 commits into from
Dec 15, 2023

Conversation

ravi-kumar-pilla
Copy link
Contributor

@ravi-kumar-pilla ravi-kumar-pilla commented Dec 12, 2023

Description

  • This PR resolves the compatibility issues of Kedro Viz with Kedro 19
  • Kedro 19 Changes

Development notes

  • Update demo project to kedro version 19 (This is a partial update to run kedro viz tests against demo project. kedro run might not work on demo-project and needs more refactoring)
  • Remove _reformat_load_versions callback and import (This function is not available at kedro.framework.cli.utils)
  • Update Dataset imports from kedro.io.core
  • Update e2e tests to support older kedro versions and latest version (Requirements file path change)
  • Update e2e tests with new starter template spaceflights-pandas for latest version of kedro
  • Fix lint issues related to Dataset imports
  • Update datasets property access via DataCatalog catalog._datasets
  • Remove try, except blocks in import statements for test files (eg., conftest) as we run pytests against latest kedro version
  • Update DataCatalog initialization renaming data_sets to datasets and removing layers as mentioned here
  • Update mock test api responses due to removal of layer and new id for TaskNode
  • Update new way of passing --params to kedro viz run using = as mentioned here

QA notes

  • All tests should pass
  • Create a new conda environment using conda create --name <env_name> python=3.8 or clean old env requirements using pip uninstall -y -r <(pip freeze)
  • Install requirements using pip install -r package/test_requirements.txt -r demo-project/src/docker_requirements.txt
  • Run make build
  • Run pip install -e package
  • cd demo-project
  • Run kedro viz run
  • Kedro Viz should start successfully

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added new entries to the RELEASE.md file
  • Added tests to cover my changes

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
…ro-viz into fix/kedro19-compat

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
…ro-viz into fix/kedro19-compat

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
@ravi-kumar-pilla ravi-kumar-pilla changed the title Fix for Kedro19 compatibility Support Kedro 19 Dec 13, 2023
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
@ravi-kumar-pilla ravi-kumar-pilla changed the title Support Kedro 19 Support for Kedro 19 Dec 13, 2023
ravi-kumar-pilla and others added 4 commits December 13, 2023 16:41
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
@jitu5 jitu5 self-requested a review December 14, 2023 19:26
Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one minor suggestion around using kedro-datasets 2.0.0 in the demo project, but otherwise looks great! 🎉

@@ -1,4 +1,4 @@
kedro~=0.18.0
kedro>=0.18.0
kedro-datasets[pandas.CSVDataSet,pandas.ExcelDataSet, pandas.ParquetDataSet, plotly.PlotlyDataSet, matplotlib.MatplotlibWriter]~=1.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would try to use kedro-datasets 2.0.0 if possible for the demo project so it's on the very latest release.

Suggested change
kedro-datasets[pandas.CSVDataSet,pandas.ExcelDataSet, pandas.ParquetDataSet, plotly.PlotlyDataSet, matplotlib.MatplotlibWriter]~=1.0
kedro-datasets[pandas.CSVDataset,pandas.ExcelDataset, pandas.ParquetDataset, plotly.PlotlyDataset, matplotlib.MatplotlibWriter]~=2.0

@rashidakanchwala rashidakanchwala merged commit 3227783 into main Dec 15, 2023
17 checks passed
@rashidakanchwala rashidakanchwala deleted the fix/kedro19-compat branch December 15, 2023 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants