Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor CI artifacts for release improvements #10704

Merged
merged 17 commits into from
Jan 24, 2025
Merged

Refactor CI artifacts for release improvements #10704

merged 17 commits into from
Jan 24, 2025

Conversation

jviau
Copy link
Contributor

@jviau jviau commented Dec 18, 2024

Issue describing the changes in this PR

resolves #10762

Pull request checklist

IMPORTANT: Currently, changes must be backported to the in-proc branch to be included in Core Tools and non-Flex deployments.

  • Backporting to the in-proc branch is not required
    • Otherwise: Link to backporting PR -- TODO
  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • My changes do not require diagnostic events changes
    • Otherwise: I have added/updated all related diagnostic events and their documentation (Documentation issue linked to PR)
  • I have added all required tests (Unit tests, E2E tests)

Additional information

This PR refactors our build artifacts to support the upcoming release improvements. The changes are:

  1. SiteExtension and Symbols are no longer zipped.
    • The release pipeline will be responsible for zipping after it has made any changes (such as inserting jittraces)
  2. PrivateSiteExtension removed. A new script is now included which will transform site extension into the private site extension.
    • This reduces our artifact size significantly. Especially since private site extension is only occasionally needed.
  3. Windows artifacts consolidated under a single drop_windows artifact.
    • this is so the release pipeline can download only a single artifact.
  4. Release notes added to artifacts so the release pipeline can use them when creating a GitHub release
  5. We produce a new file release.json as a build artifact in drop_windows. This is used by the release pipeline for creating the GitHub release.
    • This is created by the site extension project via a New-ReleaseJson.ps1 script
  6. Linux artifacts renamed to drop_linux for consistency with windows drop.
  7. Adds a custom GetFileHas_Custom msbuild task to work around an issue with file path length in the existing msbuild task.

@jviau jviau requested a review from a team as a code owner December 18, 2024 21:33
@jviau jviau changed the title Refactor CI artifacts to no longer include Refactor CI artifacts for release improvements Dec 18, 2024
@jviau jviau added the area: engineering Engineering improvements label Dec 18, 2024
@jviau jviau force-pushed the jviau/eng/no-zip branch from 2c2640c to 9adf37f Compare January 8, 2025 20:55
@jviau jviau requested a review from liliankasem January 22, 2025 23:21
@kshyju
Copy link
Member

kshyju commented Jan 23, 2025

zipping Symbols don't need JITTrace insertion. Do they? why we don't zip it anymore?

@jviau
Copy link
Contributor Author

jviau commented Jan 23, 2025

zipping Symbols don't need JITTrace insertion. Do they? why we don't zip it anymore?

The way zipping worked it is easier to not zip all instead of only some. We are going to have the release pipeline perform zipping. But I am separately looking into uploading symbols directly to the public symbol server, which will remove the need for this artifact entirely.

@jviau jviau requested a review from kshyju January 24, 2025 20:00
Copy link
Member

@liliankasem liliankasem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm; just need to fix the default bitness in the script

Co-authored-by: Lilian Kasem <likasem@microsoft.com>
@jviau jviau merged commit e6b3d16 into dev Jan 24, 2025
9 checks passed
@jviau jviau deleted the jviau/eng/no-zip branch January 24, 2025 23:01
jviau added a commit that referenced this pull request Jan 24, 2025
* Consolidate artifacts

* Disable sdl spotBugs

* Remove PrivateSiteExtension

* stage release_notes.md as artifact

* Add script to generate private site extension

* Copy release_notes.md in site extension

* Flatten SiteExtension artifacts, custom hash task

* Generate release.json during build

* modifyOutputPath: false

* Use SHA256 for the hash

* Default private siteext to 64bit

* Default private siteext to zipping

* Remove binlog artifact to save space

* Update SiteExtension readme

* Update src/WebJobs.Script.SiteExtension/New-PrivateSiteExtension.ps1

Co-authored-by: Lilian Kasem <likasem@microsoft.com>

---------

Co-authored-by: Lilian Kasem <likasem@microsoft.com>
jviau added a commit that referenced this pull request Jan 28, 2025
* Refactor CI artifacts for release improvements (#10704)

* Consolidate artifacts

* Disable sdl spotBugs

* Remove PrivateSiteExtension

* stage release_notes.md as artifact

* Add script to generate private site extension

* Copy release_notes.md in site extension

* Flatten SiteExtension artifacts, custom hash task

* Generate release.json during build

* modifyOutputPath: false

* Use SHA256 for the hash

* Default private siteext to 64bit

* Default private siteext to zipping

* Remove binlog artifact to save space

* Update SiteExtension readme

* Update src/WebJobs.Script.SiteExtension/New-PrivateSiteExtension.ps1

Co-authored-by: Lilian Kasem <likasem@microsoft.com>

---------

Co-authored-by: Lilian Kasem <likasem@microsoft.com>

* Fix MinorVersionPrefix

* Fix nuget_drop_path

* Fix modifyOutputPath: false

* Publish site-ext to sub-folders per tfm

---------

Co-authored-by: Lilian Kasem <likasem@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: engineering Engineering improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor build artifacts to support new release pipelines
3 participants