Rework Browserstack service, improve Cucumber integration #5271
Conversation
|
@nextlevelbeard I can provide you an example curl request that will return a session
I realize their own documentation says |
|
@nextlevelbeard Is this issue here also related with the problem that all API-Status marked with Status
|
nextlevelbeard
changed the title
I can but it should be out of scope of this PR, don't you agree? |
Not quite sure but this PR is redefining failure, shouldn't report any false failures like that, no. |
|
@christian-bromann tests run locally just fine, do you have an idea why CI is failing here? |
@nextlevelbeard the error is caused by this line: https://github.com/webdriverio/webdriverio/pull/5271/files#diff-540108e09662df242d6404527d1597ddR96 |
Should be taken care of. |
nextlevelbeard
changed the title
| } | ||
| } | ||
|
|
||
| async onReload(oldSessionId, newSessionId) { | ||
| this.sessionId = newSessionId | ||
| await this._update(oldSessionId, this._getBody()) | ||
| this.failures = 0 | ||
| await this._update(oldSessionId, { name: this.fullTitle }) |
There was a problem hiding this comment.
how are the status and reason updated in this case?
There was a problem hiding this comment.
@erwinheitzman @christian-bromann
We should update everything, ideally it would execute after for oldSessionId.
I took a look at the Sauce service:
apparently the whole raison d'être for the failure counter (which this PR removes) is so that onReload hook can report on the ending session' status
We don't have any session results exposed on onReload, we would ideally use what is later given in after hook. Would it make sense to expose some results on the onReload hook along with oldSessionId and newSessionId? Something like
onReload: function(oldSessionId, result, newSessionId) {
...
},Illustrating the problem:
it("Test that runs first", () => throw new Error('I fail!')) // sessionId 1 will fail
it("Test that runs second", () => {
console.log("I am running on session 1")
browser.reloadSession() // We should update session 1 with status "failed"
// but there's no session 1 results exposed in `onReload`
// every service has to count failures so far (with varying degrees of success)
return true // session 2 passes and `after` will update the status accordingly
})
There was a problem hiding this comment.
This looks good to me, please address @erwinheitzman comments and we can ship this. Thanks!
|
^ Making this more visible.
|
nextlevelbeard
changed the title
|
Should be good to go, OP updated. |
|
Addressed one last thing about destructing parameters. |
christian-bromann
added
the
PR: Polish 💅
Proposed changes
Reworks BrowserStack service way of handling failures
afterhooks'resultcode to measure failure as it is more accurateonReloadwhich has no choice but to update sessionstatusby evaluating number of failure reasonsMocha / Jasmine
sessionIdImproves Cucumber integration:
strictoption provided incucumberOptspreferScenarioNameoption for service which updates the session name at the end end to the scenario name if only a single scenario ran on that session. Useful for parallel runs with wdio-cucumber-parallel-execution.For 2 Sessions, same Cucumber feature, different Scenarios:
preferScenarioName = falsepreferScenarioName = true, on session start has feature namepreferScenarioName = true, on session end has scenario nameSingle reasons for failure (i.e. 1 test failed)
Reason when multiple tests fail
Types of changes
Checklist
Further comments
Reviewers: @webdriverio/project-committers