Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add JSON reporter for Sauce Orchestrate #849

Merged
merged 15 commits into from
Nov 3, 2023
Merged

feat: Add JSON reporter for Sauce Orchestrate #849

merged 15 commits into from
Nov 3, 2023

Conversation

tianfeng92
Copy link
Contributor

@tianfeng92 tianfeng92 commented Nov 2, 2023 •
edited
Loading

Proposed changes

Add JSON reporter for Sauce Orchestrate.
The report body will be like

[
  {
    "name": "WebdriverIO in ImageRunner",
    "duration": 797308834,
    "startTime": "2023-11-03T10:11:50.528903-07:00",
    "endTime": "2023-11-03T10:11:51.326215-07:00",
    "status": "Failed",
    "platform": "Linux"
  },
  {
    "name": "WebdriverIO in ImageRunner2",
    "duration": 46294290083,
    "startTime": "2023-11-03T10:11:50.528719-07:00",
    "endTime": "2023-11-03T10:12:36.823238-07:00",
    "status": "Succeeded",
    "platform": "Linux",
    "artifacts": [
      {
        "filePath": "artifacts/WebdriverIO_in_ImageRunner2/app/artifacts/screenshots/example-screenshot.png"
      }
    ],
    "runID": "c07e80f2b41e4b6e823c6c4fb0d2630b"
  }
]

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • I have updated the json schema (if appropriate)

Further comments

@tianfeng92 tianfeng92 added the enhancement New feature or request label Nov 2, 2023
alexplischke
alexplischke reviewed Nov 2, 2023
View reviewed changes
Copy link
Contributor

@alexplischke alexplischke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MR is missing an update to the imagerunner config spec ⚠️

internal/saucecloud/imagerunner.go Outdated Show resolved Hide resolved
@tianfeng92
Copy link
Contributor Author

MR is missing an update to the imagerunner config spec ⚠️

Good call!

@tianfeng92 tianfeng92 marked this pull request as ready for review November 2, 2023 22:45
@tianfeng92 tianfeng92 requested a review from a team as a code owner November 2, 2023 22:45
alexplischke
alexplischke reviewed Nov 3, 2023
View reviewed changes
Copy link
Contributor

@alexplischke alexplischke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I ran these changes against our imagerunner example repo and this is the output I get:

  {
    "name": "Playwright in ImageRunner",
    "duration": 121427749250,
    "startTime": "2023-11-03T07:25:32.411675-07:00",
    "endTime": "2023-11-03T07:27:33.839834-07:00",
    "status": "Succeeded",
    "platform": "",
    "url": "",
    "origin": ""
  }

As a user, I'd be confused as to what platform, url and origin represent and why they are empty.

internal/saucecloud/imagerunner.go Outdated Show resolved Hide resolved
api/v1alpha/framework/imagerunner.schema.json Outdated Show resolved Hide resolved
alexplischke
alexplischke reviewed Nov 3, 2023
View reviewed changes
internal/saucecloud/imagerunner.go Outdated Show resolved Hide resolved
internal/report/json/json.go Outdated Show resolved Hide resolved
internal/report/json/json.go
}
}
}

// cleanup removes any information that isn't relevant in the rendered report. Particularly when it comes to
// artifacts, this reporter is only interested in those that have been persisted to the file system.
func (r *Reporter) cleanup() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this behavior not needed anymore?

Copy link
Contributor Author

@tianfeng92 tianfeng92 Nov 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not needed anymore do to the changes in https://github.com/saucelabs/saucectl/pull/849/files#diff-ea42af98511d2bfb326add8027a845ef0e7cfe15c81221258fdf22fd7367df6fR367 I don't think there is empty FilePath anymore.

RDC doesn't have this issue because it appends artifacts after the error handling, which means there is no empty FilePath. https://github.com/saucelabs/saucectl/blob/main/internal/http/rdcservice.go#L396

tianfeng92 and others added 2 commits November 3, 2023 13:45
@tianfeng92 @alexplischke
Update internal/saucecloud/imagerunner.go
3955c56 
Co-authored-by: Alex Plischke <alex.plischke@saucelabs.com>
@tianfeng92 @alexplischke
Update internal/report/json/json.go
131e396 
Co-authored-by: Alex Plischke <alex.plischke@saucelabs.com>
tianfeng92 and others added 2 commits November 3, 2023 14:30
@tianfeng92 @alexplischke
Update api/v1alpha/framework/imagerunner.schema.json
6cb591c 
Co-authored-by: Alex Plischke <alex.plischke@saucelabs.com>
@tianfeng92
sync schema
e9548db
@tianfeng92 tianfeng92 merged commit 5b64dc6 into main Nov 3, 2023
18 checks passed
@tianfeng92 tianfeng92 deleted the DEVX-2559 branch November 3, 2023 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexplischke alexplischke alexplischke approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tianfeng92 @alexplischke