Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

M31 rebase #1014

Open
wants to merge 49 commits into
base: master
Choose a base branch
from
Open

M31 rebase #1014

wants to merge 49 commits into from

Conversation

jezdez
Copy link

@jezdez jezdez commented Oct 15, 2020

This m31-rebase branch contains the almost working rebase on top of a recent-ish version of Redash.

I'm having trouble fixing the schema drawer integration into the data source edit interface and I wonder if you had time to look at it @emtwo?

I'm getting this error:

Warning: Failed prop type: The prop `record.key` is marked as required in `EditableCell`, but its value is `undefined`.
    in EditableCell (created by Cell)

when editing a data source locally.

CleanShot 2020-10-15 at 15 36 33@2x

gabrieldutra and others added 30 commits July 14, 2020 10:02
…dash#5045)

* Add loading button in UI

* Handle databricks schema requests without RQ

* Don't use gevent worker

* Revert "Don't use gevent worker"

This reverts commit 9704c70.

* Use eventlet

* Use first column instead of 'namespace' one

* Revert "Add loading button in UI"

This reverts commit c0e4dfb.

* Remove databricks tasks

* Update eventlet

* Add libevent

* Display logs on failure

* Revert "Add libevent"

This reverts commit a00d067.

* Test updating gunicorn

* Don't set eventlet as the default for Redash

Co-authored-by: Arik Fraimovich <arik@arikfr.com>

* Remove fetchDataFromJob usage

Co-authored-by: Arik Fraimovich <arik@arikfr.com>
…1009)

* on dashboard api calls - take the id from the beginning of the slug, unless there is no number in it - in that case, take the entire slug as id

* add dashboard id when showing links to dashboards

* change path to include new name when renaming dashboards

* move slug generation to backend

* redirect to new name after changing (this time with a proper promise)

* oh right, we already have a slug function

* add spec that makes sure that renamed dashboards are redirected to the
url which contains their new name

* use id-slug in all Cypress specs

* move dashboards from /dashboard/:slug to /dashboards/:id-:name_as_slug

* Update dashboard url as its name changes

* Update separator to be "/"

* Update missing dashboard urls

* Update api not to depend on int id

* Use '-' instead of '/' as separator and update Dashboard.get calls

* slug -> name_as_slug

* Keep slug urls on cypress

* Update route path

* Use legacy attr for GET

* Use getter for urlForDashboard

* Update dashboard url when loaded by slug

* Update Dashboard routes to use id instead of slug

* Update Dashboard handler tests

* Update Cypress tests

* Fix create new dashboard spec

* Use axios { params }

* Drop Ternary operator

* Send updated slug directly in 'slug' attr

* Update multiple urls Dashboard test name

* Update route names

Co-authored-by: Levko Kravets <levko.ne@gmail.com>

Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
Co-authored-by: Levko Kravets <levko.ne@gmail.com>
* TASK Add typescript dependencies to package.json

* TASK Add typescript to build process and npm scripts and TASK Move example components to typescript and add an example definition file.

* TASK Move back to ts-loader instead of babel typescript preset

* FIX Remove unnecessary changes

* FIX Explicitly mention tsconfig file in webpack.config.js to avoid `error while parsing tsconfig.json, The 'files' list in config file 'tsconfig.json' is empty`
See (TypeStrong/ts-loader#405 (comment))

* FIX Move tsconfig to client subdirectory to make it accessible in docker container (only webpack.config.js is copied over from root folder in Dockerfile)

* TASK Move from ts-loader to babel to reduce compatibility issues between ES6/7 and typescript compilation.

* TASK Add types for classnames, hoist-non-react-statics and lodash. Fix default export of DashboardList and run prettier on eslintrc

* Run npm install

* Trigger tests

* Run npm install 2

* Trigger tests
* eager load outdated queries

* explicitly use .all() instead of list()
Bumps [lodash](https://github.com/lodash/lodash) from 4.17.15 to 4.17.19.
- [Release notes](https://github.com/lodash/lodash/releases)
- [Commits](lodash/lodash@4.17.15...4.17.19)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* getredash#5026 Fix wrong Y-axis range for stacked bar chart

* Update tests

* Use Plotly's built-in algorinthm to compute Y-axis range

* Update tests

* Revert previous solution (yRange-related code)

* Revert other unrelated changes

* Revert other unrelated changes

* Move chart rendering to own file and ensure that rendering steps will occur in necessary order

* Reduce amount of plot updates by mergin separate updates into a sigle cumulative update

* Give better names for several functions
* Only try to create tables and stamp DB if not tables exist already.

* load extensions when creating the database
* Add Column Type to Databricks schema browser

* Map schema columns to be an object

* Format pg with Black

* Add data_type for Postgres
* Add refresh button in the bottom

* Add caching

* Drop allSettled

* Simplify refresh button

* Update error to return 500

* Load tables before loading columns

* Don't mutate schema

* Reset db name and schemas when changing data source

* Load both tables and columns

* Return error with code 200

* Code review updates

* Add expiration time to the cache Keys

* Back with RQ
When using some of the customized login flows such as `REMOTE_USER` the deployed site breaks due to not finding template files. This change updated the app default to use the existing Flask templates directory rather than the compiled static assets directory which only contains an index.html file.
Signed-off-by: koooge <koooooge@gmail.com>
* add flask-wtf

* add CSRF tokens to all static forms

* add CSRF tokens to all axios requests

* disable CSRF validation in unit tests

* support CSRF-protected requests in *most* cypress tests

* don't enfroce CSRF checks by default

* avoid CSRF enforcement in unit tests

* remove redundant spread

* some camel casing hiccups

* always yield the CSRF cookie, but avoid enforcing it if CSRF toggle is off

* Restyled by prettier (getredash#5056)

Co-authored-by: Restyled.io <commits@restyled.io>

* set a CSRF header only if cookie is present

* enforce CSRF in CI

* install lodash directly for Cypress

* install request-cookies directly for Cypress. We should probably start loading package.json deps

* enable CSRF support when logout and login happen within the same spec

Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
* add lock table header

* Move styling to a new class

* Update renderer.less

* Move class to table and fix top border

* Update renderer.less

* Update viz-lib/src/visualizations/table/renderer.less

Thanks, this change is good to me.

Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>

Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
* if present, always convert CSRF cookies to headers

* exempt auth blueprints from CSRF protection

* respect CSRF exempts
* Update Organization Settings

* Cypress: Update tab naming
* allow non-sequential IDs for DataSources in Cypress tests

* refactor redash-api to a set of Cypress commands

* support mounting Redash endpoints in Cypress routes

* fix some parameter specs by waiting for schema to load

* extract baseUrl from cypress.json

* Restyled by prettier (getredash#5110)

Co-authored-by: Restyled.io <commits@restyled.io>

Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <commits@restyled.io>
* Remove content width limit on all pages

* Update client/app/assets/less/inc/base.less

Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>

* Remove content limit; limit sidebar width

Co-authored-by: Gabriel Dutra <nesk.frz@gmail.com>
gabrieldutra and others added 16 commits August 21, 2020 16:47
* Fix create link on data sources page

* Cypress: Add test that the source dialog opens
* Set corejs version in .babelrc so Jest doesn't complain.

* Rewrite services/routes in TypeScript.

* Add TypeScript definitions for DialogComponent.

* Make image paths more portable

* Add current route context and hook.

* Make EmptyState more flexible by being able to pass in getSteps function.

* Rewrite ItemsList in TypeScript.

* Introduce the possibility to add custom sorters for a column.

* Rearrange props to be friendly to TypeScript.

* Type definitions for NotificationApi.

* Use Databricks query editor components for databricks_internal type of query runner.

* URL Escape password in Alembic configuration.

* Compare types in migrations.
…redash#5130)

* Auth: make login url configurable.

* More portable image url.

* durationHumanize: support for milliseconds.

* Sorter: support for custom sort.
- Use new master / rc release release strategy (#440)
- Migrate Circle CI 2.0 (#488, #502, #923)
- Install redash-stmo.

In the long run we'll be able to install additional dependencies by
having an own Dockerfile to build images based on the Redash image
but that installs additional Python dependencies. But until we have
a fork with lots of changes ourselves we need to do it this way.

Redash-stmo contains the ability to hook up our own Dockerflow
library.

Disable handlers test that tests the login page and the existence of the
remote auth link there. We override this functionality for user
experience reasons via redash-stmo and redirect from the login page to
the remote auth URL.

Update metrics test since our auth changes add a database call which
also gets timed, and therefore assert this was called only once fails.

Refs #13
Refs #37
* Process extra column metadata for a few sql-based data sources.

* Add Table and Column metadata tables.

* Periodically update table and column schema tables in a celery task.

* Fetching schema returns data from table and column metadata tables.

* Add tests for backend changes.

* Front-end shows extra table metadata and uses new schema response.

* Delete datasource schema data when deleting a data source.

* Process and store data source schema when a data source is first created or after a migration.

* Tables should have a unique name per datasource.

* Addressing review comments.

* Update migration file for mixins.

* Appease PEP8

* Upgrade migration file for rebase.

* Cascade delete.

* Adding org_id

* Remove redundant column and table prefixes.

* Non-existing tables and columns should be filtered out on the server side not client side.

* Fetching table samples should be optional and should happen in a separate task per table.

* Allow users to force a schema refresh.

* Use updated_at to help prune old schema metadata periodically.

* Using settings.SCHEMAS_REFRESH_QUEUE

* fix for getredash#2426 test

* more stable test_interactive_new

* Closes #927, #928: Schema refresh improvements.

* Closes #934, #935: Remove type from schema browser and don't show empty example column in schema drawer (#936)

* Speed up schema fetch requests with fewer postgres queries.

* Add column metadata to Athena glue processing.

* Fix bug assuming 'metadata' exists for every table.

* Closes #939: Persisted, existing table metadata should be updated.

* Sample processing should be rate-limited.

* Add cli command for refreshing data samples.

* Schema refreshes should not overwrite column 'example' field.

* refresh_samples() should filter tables_to_sample on the datasource's id being sampled

* Correctly wrap long text in schema drawer.

Schema Improvements Part 2: Add data source config options.

Adding BigQuery schema drawer with data types and samples.

Add empty migration to replace the removed schedule_until migration

Add merge migration.

Fix spacing issue with data scanned value in query execution metadata.

Increase schema refresh timeout.

Remove old migrations.

Co-authored-by: Alison <github@bankofknowledge.net>
Co-authored-by: Jannis Leidel <jannis@leidel.info>
@jezdez jezdez requested a review from emtwo October 15, 2020 13:36
@emtwo
Copy link

emtwo commented Oct 15, 2020

This m31-rebase branch contains the almost working rebase on top of a recent-ish version of Redash.

I'm having trouble fixing the schema drawer integration into the data source edit interface and I wonder if you had time to look at it @emtwo?

I'm getting this error:

Warning: Failed prop type: The prop `record.key` is marked as required in `EditableCell`, but its value is `undefined`.
    in EditableCell (created by Cell)

when editing a data source locally.

CleanShot 2020-10-15 at 15 36 33@2x

Thanks for doing the rebase, Jannis! I will try to take a look today or if not, tomorrow. Just wondering, does the page not load at all? What else happens aside from seeing the error? Thanks!

@jezdez
Copy link
Author

jezdez commented Oct 15, 2020

@emtwo thank you! the page loads, and I can see the datasource edit form, but for some reason I can't update any of the columns metadata, the form input can't be selected

@emtwo
Copy link

emtwo commented Oct 19, 2020

@jezdez I pushed my in-progress commit where I've updated the schema table based on the example of editable tables made by antd here: https://ant.design/components/table/#components-table-demo-edit-row

It's kind of in a rough state right now unfortunately. It loads and you can type in the description when you hit edit (that's an improvement over what was there before!) But saving doesn't work yet.

I think the example in the link should hopefully be helpful enough to get the rest of it sorted out. Let me know if there is anything else I can help with.

}

onChange = () => {
this.setState(prevState => ({ visible: !prevState.visible }));
};

onChangeDesc = (e) => {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might not be necessary, I was testing some things out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.