[Flaky Test] Unstable test result of internalusers_spec.js for security during artifact build

[RyanL1997]

What is the bug?

A clear and concise description of the bug.
The functional test case internalusers_spec.js [1] for security is unstable, specifically during the building of release candidates.

How can one reproduce the bug?

Run internalusers_spec.js with security enable. However, it is kinda hard to reproduce it on local. But this issue does consecutively happen during previous releases on 1.3 branch [2].

What is the expected behavior?

Stable results.

What is your host/environment?

1.3 branch

Do you have any additional context?

[1]: Test file - https://github.com/opensearch-project/opensearch-dashboards-functional-test/blob/1.3/cypress/integration/plugins/security/internalusers_spec.js
[2]: Comments from previous release - opensearch-project/opensearch-build#3331 (comment)

[kavilla]

@RyanL1997 will you be addressing this?

[peternied]

@RyanL1997 Are you working on this?

[RyanL1997]

Hello folks, sorry for the late response. I'm currently testing this on my local setup. For main, I wasn't able to reproduce the error on my local by constantly passing it.

Here are the result on main:

• minimum build of OS + OSD
• security plugin installed
• run cypress with security enabled/configured
• here is the video recording of one of my runs:

Screen.Recording.2024-01-16.at.2.04.01.PM.mov

At the same time, I'm also checking the 1.3 branch and I will post the result here. Since this issue didn't happen for a while in the past couple releases of both 2.x and 1.x, I will close this issue if there is nothing broken on 1.3 branch.

[RyanL1997]

Here is the passing result of 1.3 branch, with the same setup as main:

• security plugin installed
• run cypress with security enabled/configured
• here is the video recording of 3 of my runs (I re-run it twice in the following video and all of them are passing):

Screen.Recording.2024-01-16.at.5.56.07.PM.mov

According to the above, lets close this issue for now.