Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[build] pack .NET .nupkg files in parallel #7648

Closed
wants to merge 1 commit into from

Conversation

jonathanpeppers
Copy link
Member

@jonathanpeppers jonathanpeppers commented Dec 20, 2022

Previously, we were calling dotnet pack serially for many projects, such as:

<Exec Command="&quot;$(DotNetPreviewTool)&quot; pack ... &quot;$(MSBuildThisFileDirectory)Microsoft.Android.Ref.proj&quot;" />

This ends up taking a while:

Target CreateAllPacks 52.127s
    Task Exec 3.712s
    Task Exec 3.569s
    Task Exec 3.432s
    Task Exec 3.434s
    Task Exec 24.741s
    Task Exec 7.257s
    Task Exec 2.507s
    Task Exec 3.468s

Instead, we can create a @(_ProjectsToPack) item group, set %(AdditionalProperties), and call the <MSBuild/> task:

<MSBuild
    Projects="@(_ProjectsToPack)"
    Targets="Pack"
    Properties="@(_GlobalProperties)"
    BuildInParallel="$(BuildInParallel)"
/>

This is faster because:

  • We don't shell out to a new dotnet.exe each time.

  • <MSBuild/> task knows how to run in parallel, managing multiple MSBuild nodes.

This also has the benefit of including these dotnet pack commands in the .binlog. Previously, we would just have console output for them.

After these changes, I instead see:

Target CreateAllPacks 5.857s
    Task MSBuild 5.857s

From my numbers before, I am unsure why one of the dotnet pack commands took ~24 seconds. But I believe this change should reduce the time to the slowest individual dotnet pack now.

It could save up to ~1 min of time in the CreateAllPacks target.

Previously, we were calling `dotnet pack` serially for many projects,
such as:

    <Exec Command="&quot;$(DotNetPreviewTool)&quot; pack ... &quot;$(MSBuildThisFileDirectory)Microsoft.Android.Ref.proj&quot;" />

This ends up taking a while:

    Target CreateAllPacks 52.127s
        Task Exec 3.712s
        Task Exec 3.569s
        Task Exec 3.432s
        Task Exec 3.434s
        Task Exec 24.741s
        Task Exec 7.257s
        Task Exec 2.507s
        Task Exec 3.468s

Instead, we can create a `@(_ProjectsToPack)` item group, set
`%(AdditionalProperties)`, and call the `<MSBuild/>` task:

    <MSBuild
        Projects="@(_ProjectsToPack)"
        Targets="Pack"
        Properties="@(_GlobalProperties)"
        BuildInParallel="$(BuildInParallel)"
    />

This is faster because:

* We don't shell out to a new `dotnet.exe` each time.

* `<MSBuild/>` task knows how to run in parallel, managing multiple
  MSBuild nodes.

This also has the benefit of including these `dotnet pack` commands in
the `.binlog`. Previously, we would just have console output for them.

After these changes, I instead see:

    Target CreateAllPacks 5.857s
        Task MSBuild 5.857s

From my numbers before, I am unsure why one of the `dotnet pack`
commands took ~24 seconds. But I believe this change should reduce the
time to the *slowest` individual `dotnet pack` now.

It could save up to ~1 min of time in the `CreateAllPacks` target.
@jonathanpeppers
Copy link
Member Author

I might come back to this later, this hits:

(_GenerateMsxDocXmls target) -> 
     C:\a\_work\1\s\build-tools\installers\create-installers.targets(55,5): error MSB3073: The command " "C:\a\_work\1\s\bin\Release\lib\packs\Microsoft.Android.Sdk.Windows\34.0.0\\tools\mdoc.exe" --debug export-msxdoc -o "C:\a\_work\1\s\bin\Release\lib\xamarin.android\xbuild-frameworks\MonoAndroid\v13.0\Mono.Android.xml" "C:\a\_work\1\s\external\android-api-docs\docs\Mono.Android\en"" exited with code 1. [C:\a\_work\1\s\build-tools\create-packs\Microsoft.Android.Sdk.proj]

@github-actions github-actions bot locked and limited conversation to collaborators Jan 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant