Merge remote-tracking branch 'upstream/master' into multi-queries

* upstream/master: (55 commits) feat(explore): time picker enhancement (apache#11418) feat: update alert/report icons and column order (apache#12081) feat(explore): metrics and filters controls redesign (apache#12095) feat(alerts/reports): add refresh action (apache#12071) chore: add latest tag action (apache#11148) fix(reports): increase crontab size and alert fixes (apache#12056) Small typo fix in Athena connection docs (apache#12099) feat(queries): security perm simplification (apache#12072) feat(databases): security perm simplification (apache#12036) feat(dashboards): security permissions simplification (apache#12012) feat(logs): security permissions simplification (apache#12061) chore: Remove unused CodeModal (apache#11972) Fix typescript error (apache#12074) fix: handle context-dependent feature flags in CLI (apache#12088) fix: Fix "View in SQLLab" bug (apache#12086) feat(alert/report): add 'not null' condition option to modal (apache#12077) bumping superset ui to 15.18 and deckgl to 0.3.2 (apache#12078) fix: Python dependencies in apache#11499 (apache#12079) reset active tab on open (apache#12048) fix: improve import flow UI/UX (apache#12070) ...
simcha90 · Dec 18, 2020 · a61c985 · a61c985
2 parents 8428c07 + b592cc7
commit a61c985
Show file tree

Hide file tree

Showing 226 changed files with 8,530 additions and 2,293 deletions.
diff --git a/.github/workflows/latest-release-tag.yml b/.github/workflows/latest-release-tag.yml
@@ -0,0 +1,32 @@
+name: Tags
+on:
+  release:
+    types: [published] # This makes it run only when a new released is published
+
+jobs:
+  latest-release:
+    name: Add/update tag to new release
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v2
+
+    - name: Check for latest tag
+      id: latest-tag
+      run: |
+        source ./scripts/tag_latest_release.sh $(echo ${{ github.event.release.tag_name }}) --dry-run
+
+        if ${SKIP_TAG}
+        then
+          echo "::set-env name=skip_tag::true"
+        fi
+
+    - name: Run latest-tag
+      uses: EndBug/latest-tag@latest
+      if: (! env.skip_tag )
+      with:
+        description: Superset latest release
+        tag-name: latest
+      env:
+        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
diff --git a/.github/workflows/superset-e2e.yml b/.github/workflows/superset-e2e.yml
@@ -1,13 +1,18 @@
 name: E2E
 
-on: [push, pull_request]
+on: [push, pull_request_target]
 
 jobs:
-  Cypress:
+  cypress-matrix:
     runs-on: ubuntu-18.04
     strategy:
-      fail-fast: true
+      # when one test fails, DO NOT cancel the other
+      # containers, because this will kill Cypress processes
+      # leaving the Dashboard hanging ...
+      # https://github.com/cypress-io/github-action/issues/48
+      fail-fast: false
       matrix:
+        containers: [1, 2, 3]
         browser: ["chrome"]
     env:
       FLASK_ENV: development
@@ -34,8 +39,14 @@ jobs:
         uses: styfle/cancel-workflow-action@0.6.0
         with:
           access_token: ${{ github.token }}
-      - name: Checkout code
+      - name: Checkout code (push)
+        if: github.event_name == 'push'
+        uses: actions/checkout@v2
+      - name: Checkout code (pull_request)
+        if: github.event_name == 'pull_request' || github.event_name == 'pull_request_target'
         uses: actions/checkout@v2
+        with:
+          ref: 'refs/pull/${{ github.event.number }}/merge'
       - name: Setup Python
         uses: actions/setup-python@v2
         with:
@@ -89,3 +100,12 @@ jobs:
         with:
           name: screenshots
           path: ${{ github.workspace }}/superset-frontend/cypress-base/cypress/screenshots
+  Cypress:
+    if: ${{ always() }}
+    name: Cypress (chrome)
+    runs-on: ubuntu-18.04
+    needs: cypress-matrix
+    steps:
+      - name: Check build matrix status
+        if: ${{ needs.cypress-matrix.result != 'success' }}
+        run: exit 1
diff --git a/.gitignore b/.gitignore
@@ -81,6 +81,7 @@ ghostdriver.log
 testCSV.csv
 .terser-plugin-cache/
 apache-superset-*.tar.gz*
+release.json
 
 # Translation binaries
 messages.mo

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -672,6 +672,28 @@ CYPRESS_BASE_URL=<your url> npm run cypress open
 
 See [`superset-frontend/cypress_build.sh`](https://github.com/apache/incubator-superset/blob/master/superset-frontend/cypress_build.sh).
 
+As an alternative you can use docker-compose environment for testing:
+
+Make sure you have added below line to your /etc/hosts file:
+```127.0.0.1 db```
+
+If you already have launched Docker environment please use the following command to assure a fresh database instance:
+```docker-compose down -v```
+
+Launch environment:
+
+```CYPRESS_CONFIG=true docker-compose up```
+
+It will serve backend and frontend on port 8088.
+
+Run Cypress tests:
+
+```bash
+cd cypress-base
+npm install
+npm run cypress open
+```
+
 ### Storybook
 
 Superset includes a [Storybook](https://storybook.js.org/) to preview the layout/styling of various Superset components, and variations thereof. To open and view the Storybook:

diff --git a/README.md b/README.md
@@ -78,11 +78,11 @@ A modern, enterprise-ready business intelligence web application.
 
 Superset provides:
 * An intuitive interface to explore and visualize datasets, and
-    create interactive dashboards.
+    create interactive dashboards
 * A wide array of beautiful visualizations to showcase your data.
 * Easy, code-free, user flows to drill down and slice and dice the data
     underlying exposed dashboards. The dashboards and charts act as a starting
-    point for deeper analysis.
+    point for deeper analysis
 * A state of the art SQL editor/IDE exposing a rich metadata browser, and
     an easy workflow to create visualizations out of any result set.
 * An extensible, high granularity security model allowing intricate rules
@@ -133,7 +133,6 @@ A complete list of supported databases can be found
 
 [See in the documentation](https://superset.incubator.apache.org/docs/installation/installing-superset-using-docker-compose)
 
-
 ## Get Involved
 
 * Ask and answer questions on [StackOverflow](https://stackoverflow.com/questions/tagged/apache-superset)
@@ -152,14 +151,16 @@ how to set up a development environment.
 
 ## Resources
 
-* Superset 101 -- Getting Started Guide (From [Preset](https://preset.io) [Blog](https://preset.io/blog/))
+* Superset 101 -- Getting Started Guide (From [Preset Blog](https://preset.io/blog/))
   * [Installing Apache Superset Locally](https://preset.io/blog/2020-05-11-getting-started-installing-superset/)
   * [Installing Database Drivers](https://preset.io/blog/2020-05-18-install-db-drivers/)
   * [Connect Superset To Google Sheets](https://preset.io/blog/2020-06-01-connect-superset-google-sheets/)
   * [Create Your First Chart](https://preset.io/blog/2020-06-08-first-chart/)
   * [Create Time Series Charts](https://preset.io/blog/2020-06-26-timeseries-chart/)
-* [Docker image](https://hub.docker.com/r/preset/superset/)
+* [Docker image](https://hub.docker.com/r/apache/incubator-superset)
 * [Youtube Channel](https://www.youtube.com/channel/UCMuwrvBsg_jjI2gLcm04R0g)
   * [May 2020: Virtual Meetup Recording. Topics: 0.36 Overview, Committers Self-Intro, Roadmap](https://www.youtube.com/watch?v=tXGDmqjmcTs&t=20s)
   * [July 2020: Virtual Meetup Recording. Topics: Visualization Plugins, 0.37 Preview, Demo](https://www.youtube.com/watch?v=f6up5x_iRbI)
-* [So, You Want to Build a Superset Viz Plugin...](https://preset.io/blog/2020-07-02-hello-world/)
+  * [November 2020: Superset 1.0](https://www.youtube.com/watch?v=GwtWRUSEjk4)
+* [Building Custom Viz Plugins](https://superset.apache.org/docs/installation/building-custom-viz-plugins)
+* [Superset API](https://superset.apache.org/docs/rest-api)
diff --git a/RELEASING/README.md b/RELEASING/README.md
@@ -329,3 +329,5 @@ Finally, so the Github UI reflects the latest release, you should create a relea
 tag corresponding with the new version. Go to https://github.com/apache/incubator-superset/tags,
 click the 3-dot icon and select `Create Release`, paste the content of the ANNOUNCE thread in the
 release notes, and publish the new release.
+
+At this point, a GitHub action will run that will check whether this release's version number is higher than the current 'latest' release. If that condition is true, this release sha will automatically be tagged as `latest` so that the most recent release can be referenced simply by using the 'latest' tag instead of looking up the version number. The existing version number tag will still exist, and can also be used for reference.
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -24,6 +24,7 @@ x-superset-volumes: &superset-volumes
   - ./superset:/app/superset
   - ./superset-frontend:/app/superset-frontend
   - superset_home:/app/superset_home
+  - ./tests:/app/tests
 
 version: "3.7"
 services:
@@ -57,6 +58,8 @@ services:
     user: "root"
     depends_on: *superset-depends-on
     volumes: *superset-volumes
+    environment:
+      CYPRESS_CONFIG: "${CYPRESS_CONFIG}"
 
   superset-init:
     image: *superset-image
@@ -66,6 +69,8 @@ services:
     depends_on: *superset-depends-on
     user: "root"
     volumes: *superset-volumes
+    environment:
+      CYPRESS_CONFIG: "${CYPRESS_CONFIG}"
 
   superset-node:
     image: node:12

diff --git a/docker/.env b/docker/.env
@@ -42,3 +42,4 @@ REDIS_PORT=6379
 FLASK_ENV=development
 SUPERSET_ENV=development
 SUPERSET_LOAD_EXAMPLES=yes
+CYPRESS_CONFIG=false
diff --git a/docker/docker-bootstrap.sh b/docker/docker-bootstrap.sh
@@ -19,7 +19,13 @@
 set -eo pipefail
 
 REQUIREMENTS_LOCAL="/app/docker/requirements-local.txt"
-
+# If Cypress run – overwrite the password for admin and export env variables
+if [ "$CYPRESS_CONFIG" == "true" ]; then
+    export SUPERSET_CONFIG=tests.superset_test_config
+    export SUPERSET_TESTENV=true
+    export ENABLE_REACT_CRUD_VIEWS=true
+    export SUPERSET__SQLALCHEMY_DATABASE_URI=postgresql+psycopg2://superset:superset@db:5432/superset
+fi
 #
 # Make sure we have dev requirements installed
 #

diff --git a/docker/docker-init.sh b/docker/docker-init.sh
@@ -37,22 +37,29 @@ Init Step ${1}/${STEP_CNT} [${2}] -- ${3}
 
 EOF
 }
-
+ADMIN_PASSWORD="admin"
+# If Cypress run – overwrite the password for admin and export env variables
+if [ "$CYPRESS_CONFIG" == "true" ]; then
+    ADMIN_PASSWORD="general"
+    export SUPERSET_CONFIG=tests.superset_test_config
+    export SUPERSET_TESTENV=true
+    export ENABLE_REACT_CRUD_VIEWS=true
+    export SUPERSET__SQLALCHEMY_DATABASE_URI=postgresql+psycopg2://superset:superset@db:5432/superset
+fi
 # Initialize the database
 echo_step "1" "Starting" "Applying DB migrations"
 superset db upgrade
 echo_step "1" "Complete" "Applying DB migrations"
 
 # Create an admin user
-echo_step "2" "Starting" "Setting up admin user ( admin / admin )"
+echo_step "2" "Starting" "Setting up admin user ( admin / $ADMIN_PASSWORD )"
 superset fab create-admin \
               --username admin \
               --firstname Superset \
               --lastname Admin \
               --email admin@superset.com \
-              --password admin
+              --password $ADMIN_PASSWORD
 echo_step "2" "Complete" "Setting up admin user"
-
 # Create default roles and permissions
 echo_step "3" "Starting" "Setting up roles and perms"
 superset init
@@ -61,6 +68,12 @@ echo_step "3" "Complete" "Setting up roles and perms"
 if [ "$SUPERSET_LOAD_EXAMPLES" = "yes" ]; then
     # Load some data to play with
     echo_step "4" "Starting" "Loading examples"
-    superset load_examples
+    # If Cypress run which consumes superset_test_config – load required data for tests
+    if [ "$CYPRESS_CONFIG" == "true" ]; then
+        superset load_test_users
+        superset load_examples --load-test-data
+    else
+        superset load_examples
+    fi
     echo_step "4" "Complete" "Loading examples"
 fi
diff --git a/docs/README.txt → docs/README.md b/docs/README.txt → docs/README.md
diff --git a/docs/gatsby-node.js b/docs/gatsby-node.js
@@ -634,4 +634,34 @@ exports.createPages = ({ actions }) => {
     toPath: '/docs/creating-charts-dashboards/first-dashboard',
     isPermanent: true,
   });
+  createRedirect({
+    fromPath: '/issue_code_reference.html#issue-1000',
+    toPath: '/docs/miscellaneous/issue-codes#issue-1000',
+    isPermanent: true,
+  });
+  createRedirect({
+    fromPath: '/issue_code_reference.html#issue-1001',
+    toPath: '/docs/miscellaneous/issue-codes#issue-1001',
+    isPermanent: true,
+  });
+  createRedirect({
+    fromPath: '/issue_code_reference.html#issue-1002',
+    toPath: '/docs/miscellaneous/issue-codes#issue-1002',
+    isPermanent: true,
+  });
+  createRedirect({
+    fromPath: '/issue_code_reference.html#issue-1003',
+    toPath: '/docs/miscellaneous/issue-codes#issue-1003',
+    isPermanent: true,
+  });
+  createRedirect({
+    fromPath: '/issue_code_reference.html#issue-1004',
+    toPath: '/docs/miscellaneous/issue-codes#issue-1004',
+    isPermanent: true,
+  });
+  createRedirect({
+    fromPath: '/issue_code_reference.html#issue-1005',
+    toPath: '/docs/miscellaneous/issue-codes#issue-1005',
+    isPermanent: true,
+  });
 };
diff --git a/docs/src/pages/docs/Connecting to Databases/athena.mdx b/docs/src/pages/docs/Connecting to Databases/athena.mdx
@@ -11,7 +11,7 @@ version: 1
 ### PyAthenaJDBC
 
 [PyAthenaJDBC](https://pypi.org/project/PyAthenaJDBC/) is a Python DB 2.0 compliant wrapper for the
-[Amnazon Athena JDBC driver](https://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html).
+[Amazon Athena JDBC driver](https://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html).
 
 The connection string for Amazon Athena is as follows:
 

diff --git a/docs/src/pages/docs/Miscellaneous/issue_codes.mdx b/docs/src/pages/docs/Miscellaneous/issue_codes.mdx
@@ -73,3 +73,14 @@ The table was deleted or renamed in the database.
 Your query failed because it is referencing a table that no longer exists in
 the underlying database. You should modify your query to reference the correct
 table.
+
+## Issue 1006
+
+```
+One or more parameters specified in the query are missing.
+```
+
+Your query was not submitted to the database because it's missing one or more
+parameters. You should define all the parameters referenced in the query in a
+valid JSON document. Check that the parameters are spelled correctly and that
+the document has a valid syntax.
diff --git a/docs/src/pages/docs/frequently-asked-questions-page.mdx b/docs/src/pages/docs/frequently-asked-questions-page.mdx
@@ -287,3 +287,11 @@ guarantees and are not recommended but may fit your use case temporarily:
 - using the ORM (SQLAlchemy) directly
 - using the internal FAB ModelView API (to be deprecated in Superset)
 - altering the source code in your fork
+
+
+### What Does Hours Offset in the Edit Dataset view do?
+
+
+In the Edit Dataset view, you can specify a time offset. This field lets you configure the
+number of hours to be added or subtracted from the time column.
+This can be used, for example, to convert UTC time to local time.
diff --git a/docs/src/pages/docs/installation/building-viz-plugins.mdx b/docs/src/pages/docs/installation/building-viz-plugins.mdx
@@ -207,3 +207,11 @@ to your plugin. These do a quick sanity check that the plugin actually loads cor
 through the basics of making sure that your controls are properly respected by modifying the
 resulting data and/or props of the plugin. Running `yarn test` from the root directory of
 `superset-ui` will run all the tests for plugins/packages, including your Hello World.
+
+### Deploying Custom Visualization to Production
+
+To deploy plugins to a production environment, you must have additional code
+inside Superset that includes the npm packages of your plugins so they can be installed in the frontend.
+
+
+One option is to build your Dockerfile so it contains your custom visualization packages.
diff --git a/docs/src/pages/docs/installation/index.mdx b/docs/src/pages/docs/installation/index.mdx
@@ -51,6 +51,12 @@ $ git clone https://github.com/apache/incubator-superset.git
 Once that command completes successfully, you should see a new `incubator-superset` folder in your
 current directory.
 
+We recommend that you check out and run the code from the last tagged release:
+
+```bash
+$ git checkout latest
+```
+
 ### 3. Launch Superset Through Docker Compose
 
 Navigate to the folder you created in step 1:
@@ -68,7 +74,7 @@ $ docker-compose up
 You should see a wall of logging output from the containers being launched on your machine. Once
 this output slows, you should have a running instance of Superset on your local machine!
 
-### 4. Login to Superset
+### 4. Log in to Superset
 
 Your local Superset instance also includes a Postgres server to store your data and is already
 pre-loaded with some example datasets that ship with Superset. You can access Superset now via your

diff --git a/docs/src/pages/docs/installation/installing_scratch.mdx b/docs/src/pages/docs/installation/installing_scratch.mdx
@@ -63,7 +63,7 @@ pip install --upgrade setuptools pip
 
 ### Python Virtual Environment
 
-We highly recommend installing Superset inside of a virtual environment. Python 3 ships with
+We highly recommend installing Superset inside of a virtual environment. Python ships with
 `virtualenv` out of the box but you can install it using:
 
 ```

diff --git a/docs/src/pages/docs/roadmap-page.mdx b/docs/src/pages/docs/roadmap-page.mdx
@@ -5,14 +5,14 @@ route: /docs/roadmap
 ---
 
 ## Roadmap
+
+You can find the approximate public roadmap for Superset [here](https://github.com/apache-superset/superset-roadmap/projects/1).
+
 <a href="https://github.com/apache-superset/superset-roadmap/projects/1">
 <img src="/images/roadmap.png" style="max-width:100%; box-shadow: rgb(170, 170, 170) 0px 0px 3px 1px" />
 </a>
 
-We try our best to keep the
-[Superset Roadmap](https://github.com/apache-superset/superset-roadmap/projects/1) updated.
-
 Keep in mind that the roadmap represents only a subset of what's
 ahead. Many contributions come directly without ever making it onto the roadmap.
 Find out more about how the roadmap is managed in
-[SIP (Superset Improvement Proposal) 53 - Proposal to create a Superset public roadmap ](https://github.com/apache/incubator-superset/issues/10894)
+[SIP (Superset Improvement Proposal) 53](https://github.com/apache/incubator-superset/issues/10894)