Introduce Data providers (mui#2644)

Signed-off-by: Jan Potoms <2109932+Janpot@users.noreply.github.com> Co-authored-by: Marija Najdova <mnajdova@gmail.com> Co-authored-by: Pedro Ferreira <10789765+apedroferreira@users.noreply.github.com>
Janpot · Oct 6, 2023 · c8aa712 · c8aa712
1 parent f7ec24a
commit c8aa712
Show file tree

Hide file tree

Showing 54 changed files with 1,426 additions and 189 deletions.
diff --git a/.eslintrc.js b/.eslintrc.js
@@ -34,6 +34,10 @@ module.exports = {
             message:
               'Avoid kitchensink libraries like lodash-es. We prefer a slightly more verbose, but more universally understood javascript style',
           },
+          {
+            name: 'react-query',
+            message: 'deprecated package, use @tanstack/react-query instead.',
+          },
         ],
         patterns: [
           {

diff --git a/docs/data/pages.ts b/docs/data/pages.ts
@@ -44,6 +44,10 @@ const pages: MuiPage[] = [
             pathname: '/toolpad/concepts/custom-functions',
             title: 'Custom functions',
           },
+          {
+            pathname: '/toolpad/concepts/data-providers',
+            title: 'Data providers',
+          },
         ],
       },
       {
@@ -163,14 +167,18 @@ const pages: MuiPage[] = [
             pathname: '/toolpad/reference/api/functions-group',
             subheader: 'Functions',
             children: [
-              {
-                title: 'createFunction',
-                pathname: '/toolpad/reference/api/create-function',
-              },
               {
                 title: 'createComponent',
                 pathname: '/toolpad/reference/api/create-component',
               },
+              {
+                title: 'createDataProvider',
+                pathname: '/toolpad/reference/api/create-data-provider',
+              },
+              {
+                title: 'createFunction',
+                pathname: '/toolpad/reference/api/create-function',
+              },
               {
                 title: 'getContext',
                 pathname: '/toolpad/reference/api/get-context',

diff --git a/docs/data/toolpad/concepts/data-providers.md b/docs/data/toolpad/concepts/data-providers.md
@@ -0,0 +1,119 @@
+# Data Providers
+
+<p class="description">Bring tabular data to the frontend with server-side pagination and filtering.</p>
+
+Toolpad functions are great to bring some backend state to the page, but they fall short when it comes to offering pagination and filtering capabilities from the server. Toolpad offers a special construct to enable this use case: Data providers. Data providers abstract server-side collections. They could be database tables, REST APIs, or any data that represents a set of records that share a common interface. Data providers are defined as server-side objects and can be directly connected to a data grid to make it fully interactive.
+
+Follow these steps to create a new data provider:
+
+1. Drag a data grid into the canvas
+
+2. Under its **Row Source** property, select the option **Data Provider**.
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/rows-source.png", "alt": "Select data provider row source", "caption": "Select data provider row source", "zoom": false, "width": 297}}
+
+3. Click the data provider selector and choose **Create new data provider**.
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/create-data-provider.png", "alt": "Create data provider", "caption": "Create data provider", "zoom": false, "width": 294}}
+
+4. Name the new data provider and click **Create**
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/create-data-provider-dialog.png", "alt": "Create data provider dialog", "caption": "Create data provider dialog", "zoom": false, "width": 490}}
+
+5. Use the code button to open your code editor with the data provider backend.
+
+{{"component": "modules/components/DocsImage.tsx", "src": "/static/toolpad/docs/concepts/data-providers/open-editor.png", "alt": "Open data provider editor", "caption": "Open data provider editor", "zoom": false, "width": 272}}
+
+6. A data provider that iterates over a static list could look as follows:
+
+   ```tsx
+   import { createDataProvider } from '@mui/toolpad-core/server';
+   import DATA from './movies.json';
+
+   export default createDataProvider({
+     async getRecords({ paginationModel: { start = 0, pageSize } }) {
+       const records = DATA.slice(start, start + pageSize);
+       return { records, totalCount: DATA.length };
+     },
+   });
+   ```
+
+## Pagination
+
+The data provider supports two styles of pagination. Index based, and cursor based pagination.
+
+### Index based
+
+This is the strategy your data is paginated by when it returns data based on a page number and page size. The `getRecords` method will receive `page` and `pageSize` values in it `paginationModel` parameter and returns a set of records representing the page. Index based pagination is the default but you can explicitly enable this by setting `paginationMode` to `'index'`.
+
+```tsx
+export default createDataProvider({
+  paginationMode: 'index',
+  async getRecords({ paginationModel: { start = 0, pageSize } }) {
+    const { page, totalCount } = await db.getRecords(start, pageSize);
+    return {
+      records: page,
+      totalCount,
+    };
+  },
+});
+```
+
+### Cursor based
+
+This is the strategy your data is paginated by when it returns data based on a cursor and a page size. The `getRecords` method will receive `cursor` and `pageSize` values in its `paginationModel` parameter and returns a set of records representing the page. You indicate the cursor of the next page with a `cursor` property in the result. Pass `null` to signal the end of the collection. You can enable Cursor based pagination by setting `paginationMode` to `'cursor'`.
+
+The `cursor` property of the `paginationModel` is `null` when Toolpad fetches the initial page. Any result set returned from the `getRecords` function must be accompanied with a `cursor` property, a string which contains a reference to the next page. This value will be passed as the `cursor` parameter in the `paginationModel` when fetching the subsequent page. Return `null` for this value to indicate the end of the sequence.
+
+```tsx
+export default createDataProvider({
+  paginationMode: 'cursor',
+  async getRecords({ paginationModel: { cursor = null, pageSize } }) {
+    const { page, nextPageCursor, totalCount } = await db.getRecords(
+      cursor,
+      pageSize,
+    );
+    return {
+      records: page,
+      cursor: nextPageCursor,
+      totalCount,
+    };
+  },
+});
+```
+
+## Filtering 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Sorting 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Row editing 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Row creation 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## Deleting rows 🚧
+
+:::warning
+This feature isn't implemented yet. It's coming.
+:::
+
+## API
+
+See the documentation below for a complete reference to all of the functions and interfaces mentioned in this page.
+
+- [`createDataProvider`](/toolpad/reference/api/create-data-provider/)
diff --git a/docs/data/toolpad/reference/api/create-data-provider.md b/docs/data/toolpad/reference/api/create-data-provider.md
@@ -0,0 +1,91 @@
+# createDataProvider API
+
+<p class="description">Define a backend to load server-side collections.</p>
+
+## Import
+
+```jsx
+import { createDataProvider } from '@mui/toolpad/server';
+```
+
+## Description
+
+```jsx
+import { createDataProvider } from '@mui/toolpad-core/server';
+import DATA from './movies.json';
+
+export default createDataProvider({
+  async getRecords({ paginationModel: { start = 0, pageSize } }) {
+    const records = DATA.slice(start, start + pageSize);
+    return { records, totalCount: DATA.length };
+  },
+});
+```
+
+Data providers expose collections to the Toolpad frontend. They are server-side data structures that abstract the loading and manipulation of a backend collection of records of similar shape. They can be directly connected to data grids to display the underlying data.
+
+## Parameters
+
+- `config`: [`DataProviderConfig`](#dataproviderconfig) An object describing the data provider capabilities
+
+## Returns
+
+An object that is recognized by Toolpad as a data provider and which is made available to the front-end.
+
+## Types
+
+### DataProviderConfig
+
+Describes the capabilities of the data provider.
+
+**Properties**
+
+| Name              | Type                                                   | Description                                             |
+| :---------------- | :----------------------------------------------------- | :------------------------------------------------------ |
+| `paginationMode?` | `'index' \| 'cursor'`                                  | Declares the pagination strategy of this data provider. |
+| `getRecords`      | `async (params: GetRecordsParams) => GetRecordsResult` | Responsible for fetching slices of underlying data.     |
+
+### GetRecordsParams
+
+**Properties**
+
+| Name               | Type              | Description                                              |
+| :----------------- | :---------------- | :------------------------------------------------------- |
+| `paginationModel?` | `PaginationModel` | The pagination model that describes the requested slice. |
+
+### PaginationModel
+
+- `IndexPaginationModel` when `paginationMode` is set to `'index'`.
+- `CursorPaginationModel` when `paginationMode` is set to `'cursor'`.
+
+### IndexPaginationModel
+
+**Properties**
+
+| Name       | Type     | Description                                             |
+| :--------- | :------- | :------------------------------------------------------ |
+| `start`    | `number` | The start index of the requested slice requested slice. |
+| `pageSize` | `number` | The length of the requested slice.                      |
+
+### CursorPaginationModel
+
+**Properties**
+
+| Name       | Type     | Description                                                             |
+| :--------- | :------- | :---------------------------------------------------------------------- |
+| `cursor`   | `number` | The cursor addressing the requested slice. `null` for the initial page. |
+| `pageSize` | `number` | The length of the requested slice.                                      |
+
+### GetRecordsResult
+
+| Name          | Type             | Description                                                                                                                                   |
+| :------------ | :--------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- |
+| `records`     | `any[]`          | The start index of the requested slice requested slice.                                                                                       |
+| `totalCount?` | `number`         | The length of the requested slice.                                                                                                            |
+| `cursor?`     | `string \| null` | Used when `paginationMode` is set to `cursor`. It addresses the next page in the collection. Pass `null` to signal the end of the collection. |
+
+## Usage
+
+:::info
+See [data providers](/toolpad/concepts/data-providers/)
+:::
diff --git a/docs/data/toolpad/reference/api/index.md b/docs/data/toolpad/reference/api/index.md
@@ -5,5 +5,6 @@
 ## Functions
 
 - [createComponent](/toolpad/reference/api/create-component/)
+- [createDataProvider](/toolpad/reference/api/create-data-provider/)
 - [createFunction](/toolpad/reference/api/create-function/)
 - [getContext](/toolpad/reference/api/get-context/)