Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/lens/smokescreen·ts - lens app lens smokescreen tests click on the bar in XYChart adds proper filters/timerange in dashboard #67838

Closed
kibanamachine opened this issue May 31, 2020 · 12 comments · Fixed by #68677 or #69519
Closed

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/lens/smokescreen·ts - lens app lens smokescreen tests click on the bar in XYChart adds proper filters/timerange in dashboard #67838

kibanamachine opened this issue May 31, 2020 · 12 comments · Fixed by #68677 or #69519
Assignees
@flash1293
Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test Feature:Lens skipped-test Team:Visualizations Visualization editors, elastic-charts and infrastructure v7.9.0

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented May 31, 2020 •
edited
Loading

A test failed on a tracked branch

Error: expected 'Sep 19, 2015 @ 06:31:44.000' to equal 'Sep 21, 2015 @ 09:00:00.000'
    at Assertion.assert (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.equal (/dev/shm/workspace/kibana/packages/kbn-expect/expect.js:227:8)
    at assertExpectedTimerange (test/functional/apps/lens/smokescreen.ts:57:27)
    at process._tickCallback (internal/process/next_tick.js:68:7)

First failure: Jenkins Build
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label May 31, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-test-triage (failed-test)

spalger added a commit that referenced this issue Jun 3, 2020
@spalger
skip flaky suite (#67838)
ce6b61e
@spalger
Copy link
Contributor

spalger commented Jun 3, 2020 •
edited
Loading

This test has failed 12 times in PRs (all PRs were to master) and once on master in the last week.

image

Skipped

master: ce6b61e
7.x/7.9: c9ccb7d
7.8: 9990814

@spalger spalger added blocker Feature:Lens skipped-test v8.0.0 Team:Visualizations Visualization editors, elastic-charts and infrastructure labels Jun 3, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app (Team:KibanaApp)

@spalger spalger mentioned this issue Jun 3, 2020
Merged
jloleysens added a commit to jloleysens/kibana that referenced this issue Jun 4, 2020
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into im/data-strea…
7f7b959 
…ms-column

* 'master' of github.com:elastic/kibana: (63 commits)
  remove scripts. prettire update has been done (elastic#68130)
  Closes elastic#68055 by detecting the local Kibana version and using that as (elastic#68198)
  [apm] docs: add deployment annotation example (elastic#67408)
  [ML] Extend population preview chart to show actual and typical value (elastic#67569)
  Refactor index management client integration tests for scalability (elastic#67917)
  Add generator function that creates multiple alerts (elastic#67713)
  chore(NA): remove config arg from os packages (elastic#67871)
  [Reporting] Move code out of Legacy (elastic#67904)
  [Metrics UI] Add overrides to Snapshot API to support alert previews (elastic#68125)
  [Security] [Cases] Manage timeline UI API (elastic#67719)
  [ENDPOINT][INGEST]Task/endpoint ingest update (elastic#67234)
  Fix code coverage for jest, upload merged reports (elastic#68149)
  Update documentation/examples of deprecated namespaceAgnostic field (elastic#68039)
  [DOCS] Updates Canvas docs with new menus (elastic#66061)
  chore(NA): avoids imports of server or public code into common (elastic#67231)
  [SIEM] Fix GetOneTimeline graphql type (elastic#68137)
  skip flaky suite (elastic#67838)
  [Uptime] Add loading message for monitor list no items (elastic#67378)
  [Ingest Manager] Update indexing strategy docs to use dataset.* (elastic#68068)
  [Ingest Manager] Fix datasource validation for streams without vars (elastic#67950)
  ...

# Conflicts:
#	x-pack/plugins/index_management/__jest__/client_integration/helpers/index.ts
#	x-pack/plugins/index_management/__jest__/client_integration/home.test.ts
#	x-pack/plugins/index_management/__jest__/client_integration/home/index_templates_tab.helpers.ts
@wylieconlon wylieconlon self-assigned this Jun 9, 2020
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@wylieconlon
Copy link
Contributor

I think the failure reported is from my testing. I have only been able to reproduce the issue using the flaky test runner, and not locally. I think I can fix the issue by waiting for the chart to be fully rendered.

@spalger
Copy link
Contributor

spalger commented Jun 10, 2020 •
edited
Loading

It's from the 7.8 branch, and we've had 9 other failures in PRs in the last 7 days.

image

Skipping in 7.x, 7.8 too

(turns out 7.x was skipped almost a month ago c9ccb7d)

@spalger spalger added v7.8.0 and removed v8.0.0 labels Jun 10, 2020
spalger added a commit that referenced this issue Jun 10, 2020
@spalger
skip flaky suite (#67838)
9990814 
(cherry picked from commit ce6b61e)
@wylieconlon
Copy link
Contributor

Got it. I'm uploading this screenshot from my functional test runner, which shows what I think the issue is: I think the test environment is getting dirty with Endpoint functional tests, which have affected the saved objects that we expected. So this doesn't look like a flaky test, but a real failure.

lens app lens smokescreen tests click on the bar in XYChart adds proper filters_timerange in dashboard

@wylieconlon wylieconlon mentioned this issue Jun 10, 2020
Merged
1 task
mbondyra added a commit to mbondyra/kibana that referenced this issue Jun 11, 2020
@mbondyra
Update smokescreen.ts
ead4b47 
added a changes from elastic#67838
@kibanamachine kibanamachine reopened this Jun 16, 2020
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@wylieconlon
Copy link
Contributor

I just looked into this failure again. It looks different than last time: the "apply filters" button shown in this modal is being found and clicked, but somehow over the next 1000ms it does not get closed. The modal staying open then causes a failure in the next step of the test. This looks like the definition of a flaky test, not a legit failure- it's timing related.

lens app lens smokescreen tests click on the bar in XYChart adds proper filters_timerange in dashboard 2

@timroes timroes added v7.9.0 and removed v7.8.0 labels Jun 18, 2020
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@wylieconlon
Copy link
Contributor

@flash1293 thinks that this is caused by an infrequent issue where modals need to be clicked twice for the close event to happen: this can be solved by adding a retry.

@flash1293
Copy link
Contributor

Next thing on my list.

@flash1293 flash1293 assigned flash1293 and unassigned wylieconlon Jun 18, 2020
@flash1293 flash1293 mentioned this issue Jun 18, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@flash1293 flash1293

Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test Feature:Lens skipped-test Team:Visualizations Visualization editors, elastic-charts and infrastructure v7.9.0
Projects
None yet
Milestone
No milestone
6 participants
@wylieconlon @timroes @spalger @flash1293 @elasticmachine @kibanamachine