Add spec for Trusted Publishers (OIDC auth for NuGet)

[joelverhagen]

See the rendered functional description (main spec) and rendered technical description (supporting document).

This is a spec for NuGet/NuGetGallery#9332.

It brings OpenSSF's https://repos.openssf.org/trusted-publishers-for-all-package-repositories document to NuGet.org.

Please 👍 or 👎 this comment to help us with the direction of this feature & leave as much feedback/questions/concerns as you'd like on this issue itself and we will get back to you shortly.

Thank You 🎉

[JonDouglas]

Here are some additional responses of ~n=100-200 on later questions on the recent survey we ran:

[JonDouglas]

Do we need a branch item here? i.e.

[joelverhagen]

The branch component of the sub identifier gets overridden when someone sets the environment in their YAML. But the environment isn't there by default. I'm not sure if we should only allow folks to specify which "channel" or "track" the packages come from via environment. It seems more git flavored to also allow a branch filter which will be possible on nearly all workflows out of the box vs. an environment specifier.

See https://docs.github.com/en/actions/managing-workflow-runs-and-deployments/managing-deployments/managing-environments-for-deployment

We could have a series of checkboxes. "Filter by environment: ___________", "Filter by branch: __________", "Filter by workflow file: __________". At least one would be required but you could filter by all 3 if you want.

[joelverhagen]

I added clarification about the 3 filtering options (branch vs environment vs workflow file name)

[JonDouglas]

I assume we will want to have a NuGet GH action template on the marketplace to simplify this beyond docs?

i.e. rubygems/rubygems.org#4286

[joelverhagen]

Are you referring to a workflow template like this?
https://docs.github.com/en/actions/using-workflows/creating-starter-workflows-for-your-organization
Or a GitHub Action to do pack + auth + push all in one?

[JonDouglas]

More so on the public marketplace:

https://github.com/rubygems/release-gem?tab=readme-ov-file#requirements -> https://github.com/marketplace/actions/release-gem

[joelverhagen]

The nuget/login action in the design would be on the public marketplace. But it would only do the OIDC -> short-lived API key trade, and not the push step. ... but maybe it could? It would need a parameter to specify which .nupkgs to push.

I think it's better to keep the public marketplace action to be minimal, and re-usable for multiple use cases. If the action does nuget push internally, then we may end up needing all of the CLI parameters that go into nuget push, and add them as push gets more complex.

I would prefer nuget/login public marketplace action and not nuget/release for now. This also makes using the short-lived API key for unlist/relist and even deprecate API easier since all nuget/login does is produce an API key. It's not hardcode to always push.

[JonDouglas]

There are no desired UI elements to indicate to end-users that a package has enabled this functionality today right?

Would we want to consider parts of the Build Provenance UI elements here?

i.e. Source Commit and Build File of the "trusted publisher"

[joelverhagen]

As part of this design, no.

In your example screenshot there is a green check mark and the word "Signed" as well as a ledger entry. The green check mark only makes sense if there is some tamper protection or strong build provenance consideration (SLSA build level 2+ IMO).

We could perhaps show "Published from GitHub Actions", with the source commit and build file links, but it's not strong in the same sense that npm build provenance is strong where the artifacts are signed inside the workload, rather than just published.

See the latter paragraphs in the Auditing usage on NuGet.org section to see what I think we should commit to in this spec. Essentially, it's just recording the token (or some useful part of it) for later mining.

[JonDouglas]

+1 to AzDo based on usage and qualitative data posted in this PR thread as a comment screenshot.

[joelverhagen]

I hope we can get Azure DevOps integration. That would help many Microsoft teams who publish from Azure DevOps to NuGet.org (e.g. NuGet client team, NuGet.* packages!).

[JonDouglas]

Looks great. Awesome details and data driving this! Let me know when we can amplify and get more eyes on it. The functional bits look good to me and added comments as consideration points.

[dtivel]

You should be explicit on whether a repository can be private.

[joelverhagen]

Added! As long as the token is valid and issued from GitHub Actions (the known issuer base URL https://token.actions.githubusercontent.com), we will accept it. This can include private repositories.

[dtivel]

Are there any minimum expectations on a forge, in terms of security, transparency, or whatever?

[joelverhagen]

I updated the future possibilities section with some vague details. I think the reality is that we will evaluate it on case-by-case basis. GitHub is a trusted, industry wide service so trusting their OIDC tokens is straight forward. If there is a request for a Trusted Publisher we, nor many of our users, know anything about, we will probably be a little skeptical and ask a lot of questions.

[dtivel]

Suppose you transfer ownership of a package to me. (You make me a coowner, and then I remove you.) How would I remove your ability to continue publishing packages with your trusted publisher policy?

[joelverhagen]

The moment you are removed as an owner, your trust policy will no longer work. The trust policy is moreso associated with a package owner rather than a package. In fact, the only this package-specific about a trust policy (or long lived API key) is a package ID scope.

Each time you upload a package version (via web, API key, or Trusted Publisher), whether or not the Package owner in question (the package owner property listed here) is a current, direct owner of the package. If it is not, the upload will be blocked.

I updated the spec to clarify this.

[dtivel]

Is this NuGet.org's responsibility? Is it as simple as replacing all \ with /?

[joelverhagen]

I think it can be NuGet.org's responsibility, as a nicety. It would be very annoying to get auth errors, just because you copied the relative path from a Windows machine. Yes, it should be that simple.

[dtivel]

It's a bit difficult for me to understand what's variable and what's constant. For example, this might be reworded to:

sub claim matches the regular expression pattern {repo owner}/{repo name}:.* where {repo owner} is the repository owner's name and {repo name} is the repository's name. Then, I would understand dtivel/myrepo:somepath as a matching pattern.

(I inferred that * is something of a wildcard or maybe .* is a regular expression pattern, but I can't be certain from the spec.)

What would an actual sub look like?

[joelverhagen]

I expressed it as a regex. You're right, that's confusing. I'll clarify that and add a sample.

A branch sub would look like this: repo:octo-org/octo-repo:ref:refs/heads/demo-branch.

[dtivel]

I don't grok how this limit aligns with someone publishing many packages in a short period of time. Consider the .NET team publishing packages in bulk to NuGet.org. Will that translate to hundreds or thousands of API calls within a short timeframe? Or, is the API key cached and reused on the client side?

It would be good to discuss here how you see this limitation not being a problem for various publishing scenarios.

[joelverhagen]

Will that translate to hundreds or thousands of API calls within a short timeframe? Or, is the API key cached and reused on the client side?

I expect the API key to be cached on the client side. In most workflows (no custom scripting), this will be done by a single nuget/login workflow step with a single output value, which is the short lived API key. This could be used one or more push operations (all referring to the single step output == single API key). I imagine most users will do dotnet nuget push *.nupkg which internally does multiple push operations with a single API key provided via CLI arg.

[dtivel]

Suppose dotnet nuget push *.nupkg takes longer than 15 minutes. Imagine lots of packages, slow network, whatever. Will NuGet client automatically re-request a new token via nuget/login midstream, as necessary?

[joelverhagen]

Not with this version of the proposal. See the https://github.com/NuGet/Home/blob/jver-oidc/accepted/2024/trusted-publishers-oidc-for-nuget-push.md#enable-nuget-authentication-provider-integration-to-avoid-token-expiration-for-long-workflows.

The user will be able to work around the issue with multiple nuget/login GitHub Actions steps and breaking up the push steps. It's not a great solution, but the data suggests we can pick a single short lifetime (15 minutes) and cover the vast majority of cases.

[dtivel]

I didn't see any explicit mention of how long the short-lived API key will be valid. I think that's intentional, since we should have some flexibility in adjusting its lifespan. However, for the sake of argument, what is a reasonable starting point? 30 minutes?

What is the user experience if the short-lived API key expires before they're done publishing and how do they recover? Or, is the client (NuGet client, here) supposed to automatically reacquire a new API key transparently so that the user never notices the expiration?

I'm trying to understand where this automagical flow might actually break down for users.

[joelverhagen]

I didn't see any explicit mention of how long the short-lived API key will be valid. I think that's intentional, since we should have some flexibility in adjusting its lifespan. However, for the sake of argument, what is a reasonable starting point? 30 minutes?

I actually thought I documented. I did not 😂. I was planning on using 15 minutes. This is what PyPI uses and it should have good coverage for our push patterns. I will add supporting data.

[dtivel]

This numeric owner ID is forge-assigned (here GitHub-assigned)?

[joelverhagen]

Yes, GitHub assigned.

I should note that other forges may not have a numeric owner ID. We will need to assess resurrection attacks or any other impersonation risks on a case by case basis.

[haydentherapper]

This is a fantastic write up! Excited to see this spec!

[haydentherapper]

It might be helpful to also note which of these platforms support OIDC for their workflows.

[joelverhagen]

Great point. I'll do that research and include a little note in the table.

[haydentherapper]

Thoughts on if you would let a package "downgrade" to move off trusted publishing? And a similar question, would you allow both to be configured, or only one?

[joelverhagen]

Hmm good question! I think starting out we would allow full flexibility in a "walk-crawl-run" approach. As we mature, we see how the feature goes in our ecosystem, we can lock things down more. We took a similar approach with requiring 2FA on our website. At first it was opt in, then opt out (new accounts had it enabled), then it was mandated. This allowed us to look at customer feedback as we went and not move so quickly that folks got stuck.

[joelverhagen]

I added a section about this.

[haydentherapper]

This comment is orthogonal for trusted publishing, but I might avoid discussing SLSA Build L1 - It doesn't add much given it's not a signed attestation and if the registry was constructing the provenance, then it requires trust in the registry that it properly recorded the claims, rather than the builder.

[joelverhagen]

That's a fair point. I do want to mention SLSA Build levels to come extent, at least to say this does NOT provided L2+! I talked to Zach S. on the OpenSSF Slack. He said using this information would be L1 but it's not great since it is tamperable. I would say this metadata is better than package author-attested repo/project URL since the consumer needs to just trust the registry and GitHub Actions, but it's not as good as a proper signed provenance thing.

[skofman1]

I recommend investigating the current usage of API Key scopes to understand how they are used. For example, are globs commonly used? Are there API keys that only allow pushing new versions? Are they common? I have a suspicion that this functionality isn't used, so we should check before carrying it forward. We might want to make changes / simplify the MVP.

[joelverhagen]

Great point. I was assuming parity would be the easiest since we would be stamping out short-lived API keys from the OIDC token, so all of the existing API key scoping can apply. But there is still cost to adding these parameters to the trust policy definition and making the UI work and look pretty+accessible. I will check the usage and modify the spec if adoption is "low".

[joelverhagen]

I looked at the data. I think we can definitely get by without package ID and allowed action scoping, especially since the API keys are short-lived.

[dtivel]

I'm partial to calling this "OIDC Publishers" over "Trusted Publishers", but otherwise this spec looks good.

[martincostello]

Will this also support tags? For example, when a tag is created v1.2.3 and a workflow is kicked off automatically on tag creation to build that version and then publish it.

[joelverhagen]

Hmmm generally a tag trust policy should be possible: https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/about-security-hardening-with-openid-connect#filtering-for-a-specific-tag

Question for you: would you want this feature? It seems like you would need to update the trust policy for every release as your version tag increments.

[joelverhagen]

I'll note the existing filters are meant to be "invariants" you set once and change infrequently (branch name, env name, workflow path). Tag seems different in this regard.

[martincostello]

I guess more what I'd be after is a policy that matches a tag with pattern (e.g. :ref:refs/tags/v*) rather than exact match.

My typical release process is to draft a release for a tag in GitHub that has a specific version (e.g. v1.2.3) with the release notes prepared etc. Then when I'm ready to ship, I publish that release which creates the tag. The triggers on my CI workflow are then setup like this:

on:
  push:
    branches: [ main ]
    tags: [ v* ]
  pull_request:
    branches: [ main ]
  workflow_dispatch:

Publishing then happens based on a condition like this:

if: |
  github.event.repository.fork == false &&
  startsWith(github.ref, 'refs/tags/v')

The net effect is that the "real" build happens after tagging, and that packages get shipped off to NuGet.org once produced.

So in a vacuum, I would match on the tag rather than a branch as I wouldn't want any push to main to be allowed to publish.

If tags aren't supported, then I would use an environment rather than a branch, as I currently use environments to scope the NuGet API key secret to that environment so it's not available to all the workflow runs (it also lets me manually approve the push job).

[baronfel]

I heartily endorse @martincostello's comments above - this is how a very large set of OSS projects are versioned, not just in the .NET ecosystem but broadly across other ecosystems as well. Not supporting pattern-based, tag-based workflows would be a huge miss IMO.

[baronfel]

It may be useful looking into branch wildcard support as well - I know that e.g. in dotnet we use pattern-based rules in many other places to signify some special status - e.g. for the SDK ^release/\d\.\d\.\d\d\d$ is a stable release branch and may be able to trigger certain workflows.

[joelverhagen]

I updated the document with wildcard support on tags and branches. This aligns with the pattern matching on package ID scopes. I am not sure if a regex is the right UX but I'm happy to hear other opinions on that!

[jimmylewis]

I am not sure if a regex is the right UX but I'm happy to hear other opinions on that!

Glob patterns vs. regex have both been used as examples here, so I think it's worth clarifying which since the syntax is so similar for trivial cases. A lot of places use glob patterns for branch matching, so there may be some confusion if regex is used.

Glob pattern above: :ref:refs/tags/v* (typically converted to regex: :ref:refs/tags/v.* though this is not precisely the same; it would be more accurate as :ref:refs/tags/v[!/]* though that's hardly intuitive)
Regex pattern above: ^release/\d\.\d\.\d\d\d$ (converted to glob: release/[0-9].[0-9].[0-9][0-9][0-9])

Worth noting in the glob pattern example, typically glob expansion only happens within the current directory level, so v* will match v1.2.3 but not version/1.2.3. The simple regex conversion .* does not respect path delimiters, so v.* would match both v1.2.3 and version/1.2.3.

[baronfel]

to clarify - I'd be perfectly happy for just globs, the regex was just meant to be a related example of branch and ref names being signals based on structure. I'd expect users to use release/* for example.

[joelverhagen]

Good point @jimmylewis, in my experience the term globbing is most often used for file system/file paths. I am not sure if the concept cleanly translated in users' minds to tag/branch names with slashes. IMO we can do something simple where * expands to .* regex and not worry about including slashes in that that capture.

Today "globbing" on NuGet.org API key scopes is like this:
https://github.com/NuGet/NuGetGallery/blob/a4b3c64acb65ca68eb67ac6b51242356aa5e4da7/src/NuGetGallery.Services/Extensions/ScopeExtensions.cs#L54-L58

[di]

Note that although it's not true for GHA, with other IdPs it's possible to generate/provide an OIDC token with multiple aud claims which is supported by the OIDC spec: https://openid.net/specs/openid-connect-core-1_0.html#:~:text=REQUIRED.%20Audience(s,case%2Dsensitive%20string.

You may want to additionally verify that only the nuget audience is included in the token, to ensure that users aren't potentially sharing nuget-enabled OIDC tokens with additional audiences.

[joelverhagen]

Ah, nice call out. I didn't know multiple audiences was possible. Yes, there should be exactly one and it should be a well-defined "nuget" value.

[di]

I think this should be case sensitive? The workflow path at least is definitely case sensitive.

[joelverhagen]

I chose to relax case sensitivity here due to Windows being a case insensitive file system. I don't feel too strongly, but it seems like a silly reason to reject a token JUST because the case was wrong, assuming everything else matches.

[di]

Since GitHub is case sensitive, it does mean that more than one workflow could gain the ability to publish. In practice I think that would be fairly uncommon (probably mostly indicative of a typo) but might be a surprising side effect.

[joelverhagen]

Yes, that could be weird. I am not sure if GH has mitigations in place for multiple files of the same case. I'll listen for other input on this, and we can make a call during implementation.

[di]

Mostly curious: why is username necessary here? Shouldn't the OIDC token always map 1:1 with a configuration/policy?

[joelverhagen]

In this design a trust policy is attached to a user. Inside the trust policy, the user specifies which user/org the trust policy is scoped to (to handle the “Pending” Trusted Publishers problem mentioned in Seth's doc) -- a user can publish packages as themselves or to an organization they are a memeber of. So this username parameter specifies the user profile than holds the trust policy.

There is no enforcement of 1:1 mapping planned. If we did want to enforce a 1:1 mapping we'd have to decide who (which NuGet.org user profile) gets preference. A first come first service basis would be weird too -- who would win?

I think is it better to allow users to set up whatever trust policy they want and require this additional parameter during token trade. The token trade would first fetch all trust policies for that username and see if any of them apply to the incoming token.

As an added benefit this would allow a future model where the trust policy is attached to an organization directly (instead of one of the admins). But this is not planned yet and we're more approaching this as a way to mint short-lived API keys which are only owned by users today.

[di]

Got it, so the configuration of the nuget/login would need to include a single username as well, correct? I'm imagining that it might be hard to determine who this user would be in the case of a project with multiple maintainers. Does this also mean that the projects other maintainers have no way to administer the publisher?

(For context, in PyPI's implementation, publishers map to projects, not users, and all owners/maintainers of a given project can change/configure publishers for that project. But that does require 'pending publishers', which is a little awkward UX-wise, but seems to work)

[joelverhagen]

Thank you for testing my logic here! I really appreciate it!

Yes, nuget/login would have a username provided. I have a bit of sample YAML in the spec (exact shape TBD):

      - name: NuGet login
        uses: nuget/login@v1
        id: login
        with:
          user: joelverhagen # this is the user profile which has the GitHub Actions trust policy
          source: https://api.nuget.org/v3/index.json

I'm imagining that it might be hard to determine who this user would be in the case of a project with multiple maintainers.

Yes, this is not great. But it is the current problem we have with "who owns the long lived" API key, so it's not worse than what we have today. If a user is concerned about sharing their user profile name it could be a GitHub secret variable NUGET_USERNAME.

Does this also mean that the projects other maintainers have no way to administer the publisher?

The term publisher is not well defined in NuGet world, but when used that typically refers to the visible owner (user/org) of the package. But I think based on that PyPI definition I think that a PyPI project is equivalent to a NuGet.org package ID (sometimes called package registration). I don't think we would want to attach trust policies to a package ID since orgs have many many package IDs (hundreds) and they would need to set the policy on all those places (painful). I guess .NET people like a bunch of little packages for some reason 😆.

Suppose a core contributor on a GitHub repo disappears and another contributor needs to publish a new version of the package(s). They would add an equivalent trust policy to their user account scoped to the org (or user co-owner) that owns the GitHub repo's packages and then specify their username in the token trade as we discussed above.

I think this is the PyPI + NuGet term mapping (did this mainly to get my head straight)

PyPI	NuGet.org	Notes
Project	Package ID / Package Registration
Release	Package version	Sometimes just called "package" in NuGet, confusingly
Collaborator / Maintainer (?)	Owner	Can be a user or organization on NuGet.org, not sure about PyPI
Organization	Organization	Looks like creating these has an approval flow on PyPI, unlike NuGet.org
User	User	These seem to have equivalent usages

[di]

I'm partial to calling this "OIDC Publishers" over "Trusted Publishers", but otherwise this spec looks good.

This was actually the original name planned for the PyPI implementation of this spec, but my concern was that "OIDC" as a term was too unfamiliar and low-level for the average user, which might discourage more widespread adoption.

Note that RubyGems also calls this Trusted Publishing so there is some consistency there as well. I'm definitely biased though, since I came up with the name. 🙂