test: fix flaky tests

[kstrz]

SUMMARY

Fix a few flaky tests:

• test_create_database_fail_sqllite
• test_time_grain_addons
• email_tests.py
• queries/api_tests.py
• queries/saved_queries/api_tests.py
• test_set_perm_druid_datasource

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

• Has associated issue:
• Changes UI
• Requires DB Migration.
• Confirm DB Migration upgrade and downgrade tested.
• Introduces new feature or API
• Removes existing feature or API

[codecov-io]

Codecov Report

Merging #12253 (e8a432c) into master (f3ab1f4) will decrease coverage by 3.15%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #12253      +/-   ##
==========================================
- Coverage   66.20%   63.04%   -3.16%     
==========================================
  Files         996     1002       +6     
  Lines       49174    49217      +43     
  Branches     4993     5000       +7     
==========================================
- Hits        32554    31028    -1526     
- Misses      16476    17989    +1513     
- Partials      144      200      +56     

Flag	Coverage Δ
cypress	?
javascript	61.25% <ø> (+0.14%)	⬆️
python	64.20% <ø> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset-frontend/src/explore/App.jsx	0.00% <0.00%> (-100.00%)	⬇️
superset-frontend/src/dashboard/App.jsx	0.00% <0.00%> (-100.00%)	⬇️
superset-frontend/src/setup/setupColors.js	0.00% <0.00%> (-100.00%)	⬇️
superset-frontend/src/chart/ChartContainer.jsx	0.00% <0.00%> (-100.00%)	⬇️
superset-frontend/src/explore/reducers/index.js	0.00% <0.00%> (-100.00%)	⬇️
...et-frontend/src/dashboard/containers/Dashboard.jsx	0.00% <0.00%> (-100.00%)	⬇️
...t-frontend/src/explore/reducers/getInitialState.js	0.00% <0.00%> (-100.00%)	⬇️
...-frontend/src/visualizations/presets/MainPreset.js	0.00% <0.00%> (-100.00%)	⬇️
...tend/src/dashboard/containers/DashboardBuilder.jsx	0.00% <0.00%> (-100.00%)	⬇️
...c/visualizations/FilterBox/FilterBoxChartPlugin.js	0.00% <0.00%> (-100.00%)	⬇️
... and 162 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3ab1f4...e8a432c. Read the comment docs.

[villebro]

If it's possible to patch the app.config dict I'd prefer that approach. Other than that very nice!

[villebro]

To make these even more resilient to flakery, I believe patching the dict would be the best option here, like here:
https://github.com/apache/incubator-superset/blob/164db3e5a13c21137afb56a3044ef3f1aaf89e11/tests/charts/api_tests.py#L1108-L1110

[kstrz]

Sure, good idea. Done