How to rebuild past tagged builds as public releases? / Add BuildingRef override

[georg-jung]

My use case: I have a GitHub Actions cron job that checks if my docker base images have updated. If they have, it checks out the latest release tag according to GitHub Releases and rebuilds my Docker images.

My version.json looks like this:

{
  "$schema": "https://raw.githubusercontent.com/dotnet/Nerdbank.GitVersioning/master/src/NerdBank.GitVersioning/version.schema.json",
  "version": "0.7",
  "publicReleaseRefSpec": [
    "^refs/heads/release/v\\d+(?:\\.\\d+)?$",
    "^refs/tags/v\\d+\\.\\d+"
  ],
  "cloudBuild": {
    "buildNumber": {
      "enabled": true
    }
  },
  "release": {
    "branchName": "release/v{version}"
  }
}

As I'm checking out a different commit than the one chosen by GitHub (which defaults to the newest commit on the default branch for a cron job), the information in GITHUB_REF is stale. Now, I've seen #795 and adapted my workflow config accordingly:

(Assume a tag e.g. v0.7.3 as inputs.build-ref)

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
      with:
        fetch-depth: 0 # avoid shallow clone so nbgv can do its work.
        ref: ${{ inputs.build-ref }}
    - uses: dotnet/nbgv@v0.4
      id: nbgv
      with:
        # see https://github.com/dotnet/nbgv/issues/50
        # remove this as soon as nbgv 3.6.x+ is released
        toolVersion: 3.6.65-alpha # or later
        toolFeed: https://pkgs.dev.azure.com/andrewarnott/OSS/_packaging/PublicCI/nuget/v3/index.json
      env:
        # if we checkout a build ref from input above, this is important
        IGNORE_GITHUB_REF: true # see https://github.com/dotnet/Nerdbank.GitVersioning/blob/274a42ed3321bcc43632dc970d65677f99a486a9/src/NerdBank.GitVersioning/CloudBuildServices/GitHubActions.cs#L32

As expected, the resulting BuildingRef does not point to the stale information from GITHUB_REF anymore, which is great.

However, nbgv also tells that I'm not building a public release now. I am, however, at a commit that is part of a release/v0.7 branch and also at the commit where v0.7.3 points. This is easy to reproduce locally:

git checkout refs/tags/v0.7.3 results in a detached head state. Executing nbgv get-version -f json afterwards also shows a non-public release. Now I think it might make sense for detached heads to assume they are non-public. Also, looking at what I believe is the relevant code, this makes sense.

To summarize, my intention is to build from a tagged past commit again in exactly the same way as when the tag was created. GitHub Actions does not allow changing the GITHUB_REF, so changing the environment accordingly is not possible.

Options that came to my mind:

• add a BuildingRef override by env variable
  • probably change this line to something similar to this.BuildingRef = buildingRefOverrideFromEnv ?? cloudBuild?.BuildingTag ?? cloudBuild?.BuildingBranch ?? context.HeadCanonicalName;
  • This also adds a lot more flexibility for a lot of other use cases than this specific one.
• add an equivalent of git tag --points-at HEAD to GitRepository.GetHeadAsReferenceOrSha()

Is there any smarter way around this? What do you think?

Complete example workflow runs:

• Original run triggered by publishing a new github release, works
• Re-run on same commit, checked out by tag, does not work (non-public release)

[AArnott]

Thanks for this excellent write-up. To get you unblocked, I want to first offer you the available workaround: Set PublicRelease=true as an environment variable (or msbuild property).

Now as to your suggestions...

I take it GitHub doesn't set any environment variable with the name of the tag or branch that was checked out when you override the default commit to checkout. Is that correct? If no env var exists, allowing the user to set one manually makes sense IMO.

I like the git tag --points-at idea as a means to automatically consider something a public release if any tag that points at a commit conforms to the publicReleaseRefSpec. Doing the right thing automatically sounds great. Possibly in addition to your env var override idea.

[georg-jung]

Thanks for providing the unblocker and reacting so fast!

I take it GitHub doesn't set any environment variable with the name of the tag or branch that was checked out when you override the default commit to checkout. Is that correct?

If I understand it correctly, every workflow run has a commit it belongs to. See here, the GITHUB_REF cells in all those small tables per trigger. A workflow does not necessarily check anything out from git though and what it checks out can be overriden by using ref with checkout

    - uses: actions/checkout@v3
      with:
        fetch-depth: 0 # avoid shallow clone so nbgv can do its work.
        ref: ${{ inputs.build-ref }}

What is checked out is then in no way related to the commit the current workflow run is assigned to. Still every workflow is always assigned to a commit and that assignment is available in GITHUB_REF.

My idea was to provide a BuildingRef override, in the same way the checkout itself can be overriden. Usage could work like this:

    - uses: actions/checkout@v3
      with:
        fetch-depth: 0 # avoid shallow clone so nbgv can do its work.
        ref: ${{ inputs.build-ref }}
    - uses: dotnet/nbgv@v0.4
      id: nbgv
      with:
        buildingRef: ${{ inputs.build-ref }}

[AArnott]

Your proposal would require adding an input to the nbgv Github Action. And it would only apply there. What about builds in that commit, where the Nerdbank.GitVersioning package applies? We'll presumably want to impact builds too, whereas the nbgv GA mainly produces other version-related environment variables. If we teach Nerdbank.GitVersioning to honor some env var override, then nbgv will pick it up 'for free' because it uses Nerdbank.GitVersioning internally.

[georg-jung]

You are right, of course. I've been experimenting so much with the GitHub Action that I've lost sight of that. Yes, adding the option explicitly to the nbgv action to pass it on to Nerdbank.GitVersioning doesn't give too tangible an advantage.

I thought some more about my two original ideas.

Adding a BuildingRef env var override probably offers the most flexibility, which is nice of course (it could probably even replace #795, hmm; and it is cloud build system agnostic).

On the other hand, if I had to choose one option I'd probably take extending GitRepository.GetHeadAsReferenceOrSha() / doing something like git tag --points-at HEAD:

• It would just work without any configuration needed (one might argue it's a breaking change though; could be opt-in to mitigate, wouldn't just work then).
• It hits the notch that git is the only source of truth.
• With more complicated pipelines and changing environments (e.g. the nbgv step vs. inside a docker build container), it is still straight forward (if .git is available, it works) and we don't need to worry about is this env var for the override available inside the container that performs the build?
• It feels natural, because it does what I specified in version.json and what I specified there does work in the typical case, where GITHUB_REF is not stale.
• It makes local builds where the tag is checked out equivalent to cloud builds initiated by tag push. Currently "^refs/tags/v\\d+\\.\\d+" doesn't have any effect locally if I get it right - it does in CI environments though.

[georg-jung]

Also interesting:

If a tag is checked out, using managed git the BuildingRef contains the commit sha. When opting for LibGit2, it is "BuildingRef": "(no branch)"

$Env:NBGV_GitEngine = "LibGit2"
nbgv get-version -f json