Documentation for Preview Dataset #1757

rashidakanchwala · 2024-02-21T16:46:29Z

Description

Documentation for the new changes in 'Preview Datasets'

Development notes

QA notes

Checklist

Read the contributing guidelines
Opened this PR as a 'Draft Pull Request' if it is work-in-progress
Updated the documentation to reflect the code changes
Added new entries to the RELEASE.md file
Added tests to cover my changes

ravi-kumar-pilla · 2024-02-21T21:50:13Z

docs/source/preview_datasets.md


-This page describes how to preview data from different datasets in a Kedro project with Kedro-Viz. Dataset preview was introduced in Kedro-Viz version 6.3.0, which offers preview for `CSVDatasets` and `ExcelDatasets`.
+To provide users with a glimpse of their datasets within a Kedro project, Kedro-Viz offers a preview feature. This feature was introduced in Kedro-Viz version 6.3.0 and expanded upon in version 8.0.0. Initially, it supported `CSVDatasets` and `ExcelDatasets`, and later extended to encompass additional dataset types such as `PlotlyDatasets` and image datasets like `MatplotlibWriter`.


[Nit] ExcelDatasets, and was later extended to ..

ravi-kumar-pilla · 2024-02-21T21:53:55Z

docs/source/preview_datasets.md

-```{important}
-We recommend that you use the same version of Kedro that was most recently used to test this tutorial (0.19.0). To check the version installed, type `kedro -V` in your terminal window.
-```
+Whilst we currently support the above datasets. We are soon going to extend this functionality to other datasets. Users with custom datasets can also extend the preview functionality and we will cover that in the following sections. 


While we currently support the aforementioned datasets, we are soon going to extend this functionality to include other datasets. Users with custom datasets can also expand the preview functionality, and we will cover that in the following sections.

ravi-kumar-pilla · 2024-02-21T22:01:50Z

docs/source/preview_datasets.md

+
+**Extend Preview to Custom Datasets**
+
+The page titled [Extend Preview to Custom Datasets](./preview_custom_datasets.md) contains information on how you can set up preview for custom datasets and what types are supported by Kedro-viz.


The page titled Extend Preview to Custom Datasets contains information on how to set up previews for custom datasets and which types are supported by Kedro-Viz

ravi-kumar-pilla · 2024-02-21T23:37:18Z

docs/source/preview_datasets.md



-To enable dataset preview, add the `preview_args` attribute to the kedro-viz configuration under the `metadata` section in the Data Catalog. Within preview_args, specify `nrows` as the number of rows to preview for the dataset.
+To disable dataset previews for specific datasets, you need to set preview: false under the kedro-viz key within the metadata section of your conf.yml file. Here's an example configuration:


should we mention that previewing is made default from latest version of viz ? If someone is using old version of viz, it will be opt in ? Also, it will be a good idea to mention about the kedro-datasets version this new feature of kedro viz supports somewhere in the doc. Thank you

good point.

…kedro-viz into docs/preview-datasets

rashidakanchwala · 2024-02-22T10:41:23Z

docs/source/preview_datasets.md

 ```



-## Previewing Data on Kedro-viz


FYI - this entire section has now moved to Preview Tabular Data on Kedro-viz

rashidakanchwala · 2024-02-22T10:42:19Z

docs/source/preview_pandas_datasets.md

@@ -0,0 +1,74 @@
+# Preview Tabular Data in Kedro-viz


This content in this section is not new and it is just moved to a new page. Earlier it was a part of the 'Preview Datasets' page

merelcht

Some initial comments, but I'll have another look next week 🙂

docs/source/experiment_tracking.md

docs/source/preview_custom_datasets.md

merelcht · 2024-02-22T13:30:18Z

docs/source/preview_custom_datasets.md

+from kedro_datasets._typing import TablePreview
+
+class CustomDataset:
+    def preview(self, nrows: int = 5) -> TablePreview:


Can we maybe add an example that works, to give users a bit more guidance on how they should realistically implement a preview() method?

I will need some help on this as I can't think of a realistic CustomDataset that is not a part of kedro-datasets. @astrojuanlu -- do you have some ideas?

docs/source/preview_datasets.md

noklam

A few minor comments, I think the most important one is an example of CustomDataset. Other than that there are quite a few inconsistency uses of Dataset Datasets and DataSet.

noklam · 2024-02-22T23:43:45Z

docs/source/preview_pandas_datasets.md

+      preview_args:
+          nrows: 15
+```
+
+If no preview_args are specified, the default preview will show the first 5 rows.
+


This was asked in the Slack once, I think we should make it obvious that the preview_args is the argument that get pass into the preview function directly, and user can have arbitary arguments.

def preview(self, arg1, arg2): ...

So I think for pandas datasets .. it is specific to nrows as we wrote the preview() func.

But I have updated the custom dataset docs to include arguments. Thanks for highlighting this @noklam

docs/source/preview_custom_datasets.md

astrojuanlu · 2024-02-23T17:21:10Z

docs/source/preview_custom_datasets.md

+
+When creating a custom dataset, if you wish to enable data preview for that dataset, you must implement a `preview()` function within the custom dataset class. Kedro-Viz currently supports previewing tables, Plotly charts, images, and JSON objects.
+
+The return type of the `preview()` function should match one of the following types, as defined in the `kedro-datasets` source code (_typing.py file):


Any chance to add a link to such file?

is the github link fine ? - https://github.com/kedro-org/kedro-plugins/blob/main/kedro-datasets/kedro_datasets/_typing.py

I can't seem to find docs source code link for _typing.py

Yeah this is not documented, so a link to the source code is fine for now.

docs/source/preview_datasets.md

astrojuanlu · 2024-02-23T17:23:17Z

docs/source/preview_datasets.md


-In your terminal window, navigate to the folder you want to store the project. Generate the spaceflights tutorial project with all the code in place by using the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas):
+While we currently support the aforementioned datasets, we are soon going to extend this functionality to include other datasets. Users with custom datasets can also expand the preview functionality, , which is covered in the section [Extend Preview to Custom Datasets](./preview_custom_datasets.md).


Do we have Vale enabled on this repo? I get the sense that aforementioned would be flagged as too wordy

i don't think so.

docs/source/preview_datasets.md

astrojuanlu · 2024-02-23T17:44:31Z

docs/source/preview_pandas_datasets.md

+We recommend that you use the same version of Kedro that was most recently used to test this tutorial (0.19.0). To check the version installed, type `kedro -V` in your terminal window.
+```
+
+In your terminal window, navigate to the folder you want to store the project. Generate the spaceflights tutorial project with all the code in place by using the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas):


Probably capitalise Spaceflights?

Suggested change

In your terminal window, navigate to the folder you want to store the project. Generate the spaceflights tutorial project with all the code in place by using the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas):

In your terminal window, navigate to the folder you want to store the project. Generate the Spaceflights tutorial project with all the code in place by using the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas):

I noticed it's mostly lower-case elsewhere in the docs so I will leave it as is.

docs/source/preview_pandas_datasets.md

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

…kedro-viz into docs/preview-datasets

astrojuanlu

LGTM!

merelcht

I left some minor comments and a suggestion on adding an example preview() method. It's all non blocking though, so I'll approve and let you decide on whether to add it or not 🙂

merelcht · 2024-02-26T10:29:16Z

docs/source/preview_custom_datasets.md

@@ -0,0 +1,59 @@
+# Extend preview to Custom Datasets


Suggested change

# Extend preview to Custom Datasets

# Extend preview to custom datasets

merelcht · 2024-02-26T13:33:18Z

docs/source/preview_custom_datasets.md

+```
+
+
+## Examples of Previews


Is there way to show the JSONPreview as well?

so, the JSON preview is actually experiment tracking oriented hence I was hesitant to share as it might create some confusion. In the next couple of sprints, we will enable preview for a JSONDataset and then I could add that example then.

merelcht · 2024-02-26T13:34:07Z

docs/source/preview_custom_datasets.md

+
+class CustomDataset:
+    def preview(self, nrows, ncolumns, filters) -> TablePreview:
+        # Add logic for generating preview


When I suggested adding a working example, I didn't necessarily mean anything complex, just in this case some code that would produce a working TablePreview and demonstrates how nrows, ncolumns and filters would be used.

docs/source/preview_pandas_datasets.md

NeroOkwa

LGTM!

ravi-kumar-pilla

Awesome work with the docs 💯 ...LGTM

demo-project/conf/base/catalog_01_raw.yml

rashidakanchwala added 7 commits February 21, 2024 16:45

first draft

691c7ba

add to toctree

70fa78f

fix build errors

1b692f8

revise structure

9e93ca8

test layout

f57b1b6

test layout

2628396

typo in title

93b79ae

ravi-kumar-pilla reviewed Feb 21, 2024

View reviewed changes

rashidakanchwala and others added 4 commits February 22, 2024 10:37

doc fixes based on review

6e83fc5

Merge branch 'main' into docs/preview-datasets

fb2d6e7

typo

8c80933

Merge branch 'docs/preview-datasets' of https://github.com/kedro-org/…

2a9c923

…kedro-viz into docs/preview-datasets

rashidakanchwala commented Feb 22, 2024

View reviewed changes

edit disable preview sec

919ab73

rashidakanchwala self-assigned this Feb 22, 2024

rashidakanchwala marked this pull request as ready for review February 22, 2024 10:46

rashidakanchwala requested review from tynandebold and yetudada as code owners February 22, 2024 10:46

rashidakanchwala requested review from NeroOkwa, astrojuanlu, noklam and merelcht and removed request for tynandebold and yetudada February 22, 2024 10:46

merelcht reviewed Feb 22, 2024

View reviewed changes

Merge branch 'main' into docs/preview-datasets

b57b3d7

noklam reviewed Feb 22, 2024

View reviewed changes

rashidakanchwala added 3 commits February 23, 2024 10:28

fixes based on review

5d4963f

fix typo

401434e

fix based on reviews -2

c8e8f84

astrojuanlu reviewed Feb 23, 2024

View reviewed changes

rashidakanchwala and others added 5 commits February 26, 2024 10:01

fixes based on reviews

e296352

add link to typing file

ff7b87b

Update docs/source/preview_pandas_datasets.md

22809be

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

typo and spacing fix

fdd9613

Merge branch 'docs/preview-datasets' of https://github.com/kedro-org/…

9f67c74

…kedro-viz into docs/preview-datasets

rashidakanchwala requested review from astrojuanlu, merelcht, noklam and ravi-kumar-pilla February 26, 2024 10:14

non caps

1d89fed

astrojuanlu approved these changes Feb 26, 2024

View reviewed changes

merelcht approved these changes Feb 26, 2024

View reviewed changes

rashidakanchwala added 2 commits February 26, 2024 13:50

based on reviews

ed42972

fix docs lint

cf7a2cb

rashidakanchwala requested review from stephkaiser and jitu5 February 26, 2024 14:04

NeroOkwa approved these changes Feb 26, 2024

View reviewed changes

ravi-kumar-pilla approved these changes Feb 26, 2024

View reviewed changes

ravi-kumar-pilla reviewed Feb 26, 2024

View reviewed changes

demo-project/conf/base/catalog_01_raw.yml Outdated Show resolved Hide resolved

rashidakanchwala and others added 2 commits February 26, 2024 15:41

revert changes

ebf6f30

Merge branch 'main' into docs/preview-datasets

074c92b

rashidakanchwala merged commit 93fe1c8 into main Feb 26, 2024
5 checks passed

rashidakanchwala deleted the docs/preview-datasets branch February 26, 2024 17:55

ravi-kumar-pilla mentioned this pull request Mar 1, 2024

Release v8.0.0 #1781

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documentation for Preview Dataset #1757

Documentation for Preview Dataset #1757

rashidakanchwala commented Feb 21, 2024 •

edited

Loading

ravi-kumar-pilla Feb 21, 2024

ravi-kumar-pilla Feb 21, 2024

ravi-kumar-pilla Feb 21, 2024

ravi-kumar-pilla Feb 21, 2024

rashidakanchwala Feb 22, 2024

rashidakanchwala Feb 22, 2024 •

edited

Loading

rashidakanchwala Feb 22, 2024

merelcht left a comment

merelcht Feb 22, 2024

rashidakanchwala Feb 23, 2024 •

edited

Loading

noklam left a comment

noklam Feb 22, 2024

rashidakanchwala Feb 23, 2024

astrojuanlu Feb 23, 2024

rashidakanchwala Feb 26, 2024

astrojuanlu Feb 26, 2024

astrojuanlu Feb 23, 2024

rashidakanchwala Feb 26, 2024

astrojuanlu Feb 23, 2024

rashidakanchwala Feb 26, 2024

astrojuanlu left a comment

merelcht left a comment

merelcht Feb 26, 2024

merelcht Feb 26, 2024

rashidakanchwala Feb 26, 2024 •

edited

Loading

merelcht Feb 26, 2024

NeroOkwa left a comment

ravi-kumar-pilla left a comment


		This page describes how to preview data from different datasets in a Kedro project with Kedro-Viz. Dataset preview was introduced in Kedro-Viz version 6.3.0, which offers preview for `CSVDatasets` and `ExcelDatasets`.
		To provide users with a glimpse of their datasets within a Kedro project, Kedro-Viz offers a preview feature. This feature was introduced in Kedro-Viz version 6.3.0 and expanded upon in version 8.0.0. Initially, it supported `CSVDatasets` and `ExcelDatasets`, and later extended to encompass additional dataset types such as `PlotlyDatasets` and image datasets like `MatplotlibWriter`.


		Extend Preview to Custom Datasets

		The page titled [Extend Preview to Custom Datasets](./preview_custom_datasets.md) contains information on how you can set up preview for custom datasets and what types are supported by Kedro-viz.



		To enable dataset preview, add the `preview_args` attribute to the kedro-viz configuration under the `metadata` section in the Data Catalog. Within preview_args, specify `nrows` as the number of rows to preview for the dataset.
		To disable dataset previews for specific datasets, you need to set preview: false under the kedro-viz key within the metadata section of your conf.yml file. Here's an example configuration:


		When creating a custom dataset, if you wish to enable data preview for that dataset, you must implement a `preview()` function within the custom dataset class. Kedro-Viz currently supports previewing tables, Plotly charts, images, and JSON objects.

		The return type of the `preview()` function should match one of the following types, as defined in the `kedro-datasets` source code (_typing.py file):


		In your terminal window, navigate to the folder you want to store the project. Generate the spaceflights tutorial project with all the code in place by using the [Kedro starter for the spaceflights tutorial](https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pandas):
		While we currently support the aforementioned datasets, we are soon going to extend this functionality to include other datasets. Users with custom datasets can also expand the preview functionality, , which is covered in the section [Extend Preview to Custom Datasets](./preview_custom_datasets.md).

	# Extend preview to Custom Datasets
	# Extend preview to custom datasets

Documentation for Preview Dataset #1757

Documentation for Preview Dataset #1757

Conversation

rashidakanchwala commented Feb 21, 2024 • edited Loading

Description

Development notes

QA notes

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rashidakanchwala Feb 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

merelcht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rashidakanchwala Feb 23, 2024 • edited Loading

Choose a reason for hiding this comment

noklam left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

astrojuanlu left a comment

Choose a reason for hiding this comment

merelcht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rashidakanchwala Feb 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NeroOkwa left a comment

Choose a reason for hiding this comment

ravi-kumar-pilla left a comment

Choose a reason for hiding this comment

rashidakanchwala commented Feb 21, 2024 •

edited

Loading

rashidakanchwala Feb 22, 2024 •

edited

Loading

rashidakanchwala Feb 23, 2024 •

edited

Loading

rashidakanchwala Feb 26, 2024 •

edited

Loading