feat: macos signing and notarization #367

lidel · 2021-07-21T14:50:56Z

This is a streamlined refactor of #362 on road to fix issues described in #8240

This PR adds sign-macos job to CI workflow, adding a transparent signing step.

The main reason is to sign repo migrations and avoid issue from ipfs/kubo#8240 globally (outside of Brave). It also provides us with a template if we wish to add signing for other platforms.

How this works?

Technical details

This PR splits make publish performed in main.yml into three stages:

build executes ./dockerized make all_dists and produces any missing binaries in ./releases
sign-macos signs and notarizes every new binary from ./releases and updates .tar.gz packages and relevant hashes in-place
publish
- executes make publish and adds any new and signed packages from ./releases to DIST_ROOT
  (/ipns/dist.ipfs.io by default)
- pins produced CID to ipfs-cluster for persistence

Flow for adding new dist or version

Open a PR against this repo with new entry for one of ./dists/{name}/versions
Wait for CI to build signed dist, put it on IPFS and pin updated DIST_ROOT to ipfs-cluster
Preview link will be published on PR and printed at the end of publish step of CI build
If everything looks good, write down the CID from preview link, and update the DNSlink of dist.ipfs.io

Content routing optimization

IPFS node running on CI manually pre-connects to ipfs-websites cluster peers, which are known to already have all blocks required during dist.ipfs.io build. This way we skip DHT lookup.

Website preview on PRs

Every PR now gets a link to website preview at the dweb.link IPFS gateway:

TODO

sign darwin binaries and put them in the same place as unsigned ones
ensure all hashes and manifests match signed binaries
ensure the root CID for updated dist.ipfs.io website is persisted after CI job ends
- we use /dnsaddr/ipfs-websites.collab.ipfscluster.io but it errors right now
gateway link with preview of updated website
remove fake DIST_ROOT and ensure CI is no-op when no new binaries are added
squash & merge

#367 (comment)

lidel

Ready for review (silence is compliance – aiming for merging in the beginning of the next week).

lidel · 2021-07-29T13:11:26Z

scripts/ci/github-preview-link.sh

+    '{ state: $state, target_url: $target_url, description: $description, context: $context }' )
+curl --output /dev/null --silent --show-error \
+    -X POST -H "Authorization: Bearer $GITHUB_TOKEN" -H 'Content-Type: application/json' \
+    --data "$API_PARAMS" 'https://api.github.com/repos/ipfs/distributions/statuses/${GIT_REVISION}'


ℹ️ we could replace this script with JS call and actions/github-script but I don't want to sink any more time into refactoring this PR ;)

lidel · 2021-07-29T13:14:22Z

dockerized

+docker build . -t distributions \
+    --build-arg CACHEBUST=`date --iso-8601=date` \
+    --build-arg USER_UID=$(id -u "$USER") \
+    --build-arg GO_IPFS_VER=${GO_IPFS_VER:-$(curl -s https://dist.ipfs.io/go-ipfs/versions | tail -n 1)} # match http api client version on CI


ℹ️ this controls the version of the CLI client inside ./dockerized build, which talks to go-ipfs running outside on the host. Here, we ensure it uses the same version (with fallback to the latest one if GO_IPFS_VER is undefined).

Should this comment live in the source file?

I believe the comment on the very right is enough, more context can be found on this PR.

lidel · 2021-07-29T13:16:38Z

scripts/ci/pin-to-cluster.sh

+    if [[ $(jq '.peer_map[].status' cluster-pin-status | grep '"pinned"' | wc -l) -ge 2 ]]; then
+        echo "Got 2 pin confirmations, finishing the workflow"


ℹ️ to make CI faster and avoid hanging forever if parts of cluster are out of sync, cluster pinning is considered "done" after we have 2 copies.

Here, I do it manually in userland, but filled ipfs-cluster/ipfs-cluster#1427 to add this sort of thing to ipfs-cluster-ctl CLI tool.

Similar to above: when do we decide to put this in the PR vs. the code itself? Do folks tend to do a git blame and look for comments to find more context?

I often use git blame, but in this specific case those comments are provided mostly to make PR review easier.

Ok. I'll defer to the team for the standard. I was bringing this up because it was hard for me to imagine a case where a comment is useful during PR where it isn't also useful 6 months in the future when looking at the code with fresh eyes. Anyways, sounds good.

lidel · 2021-07-29T13:26:50Z

scripts/ci/setup-ipfs.sh

+# fix resolv - DNS provided by Github is unreliable for DNSLik/dnsaddr
+sudo sed -i -e 's/nameserver 127.0.0.*/nameserver 1.1.1.1/g' /etc/resolv.conf


ℹ️ This was pretty annoying. Entire build failed because Githubs DNS proxying cache at 127.0.0.* randomly timeouted while resolving _dnslink or _dnsaddr. Switching to Cloudflare here make the problem go away.

lidel · 2021-07-29T13:29:26Z

scripts/ci/sign-new-macos-releases.sh

+            EXECUTABLES=$(jq -nc '$ARGS.positional' --args $(find "./tmp/${DIST_NAME}_${DIST_VERSION}_${arch}-unsigned/" -perm +111 -type f -print))
+            echo "{
+                \"source\" : $EXECUTABLES,
+                \"bundle_id\" : \"io.ipfs.dist.${DIST_NAME}\",


ℹ️ I registered the wildcard bundle_id=io.ipfs.dist.* at Apple, which enables us to have each package "isolated" from each other. This is important in case Apple decides to block something from running on macOS – we don't have all egs in one basket.

lidel · 2021-07-29T13:31:07Z

.github/workflows/main.yml

+          brew install ipfs coreutils gawk gnu-sed jq mitchellh/gon/gon
+          ipfs init --profile test # needed for calculating NEW_CID later
+      - name: Import Keychain Certs
+        uses: apple-actions/import-codesign-certs@253ddeeac23f2bdad1646faac5c8c2832e800071 # v1@2020-02-03


ℹ️ 👮‍♂️ I hardcoded specific revision because this is a third-party action and in theory someone could swap-out git tag and steal our signing keys.

lidel · 2021-07-29T13:32:28Z

.github/workflows/main.yml

+          brew tap mitchellh/gon
+          brew install ipfs coreutils gawk gnu-sed jq mitchellh/gon/gon


ℹ️ gon package is used by Terraform project, so multiple eyes are looking at this

lidel · 2021-07-29T13:49:20Z

README.md

+To produce a signed, **official build** for use in DNSLink at `dist.ipfs.io`:
+
+1. Run `./dist.sh add-version` locally.
+2. Commit created changes to `dists/<dist>` and open a PR against `ipfs/distributions`.
+3. Wait for Github Action to finish PR build. It runs `./dockerized` build, then signs macOS binaries and spits out updated root CID at the end.
+4. If everything looks good, write down the CID from the preview link on the PR, and update the DNSlink at `dist.ipfs.io`.


ℹ️ I believe this is the TLDR of this entire PR :)

aschmahmann

LGTM, although I mostly reviewed the README. We may learn more as we start using it for nightlies though ❤️

README.md

BigLep

Great work!
Only comments were around inlining comments PR into the code for discoverability. Your call.

BigLep · 2021-07-29T22:01:11Z

dockerized

+docker build . -t distributions \
+    --build-arg CACHEBUST=`date --iso-8601=date` \
+    --build-arg USER_UID=$(id -u "$USER") \
+    --build-arg GO_IPFS_VER=${GO_IPFS_VER:-$(curl -s https://dist.ipfs.io/go-ipfs/versions | tail -n 1)} # match http api client version on CI


Should this comment live in the source file?

BigLep · 2021-07-29T22:03:55Z

scripts/ci/pin-to-cluster.sh

+    if [[ $(jq '.peer_map[].status' cluster-pin-status | grep '"pinned"' | wc -l) -ge 2 ]]; then
+        echo "Got 2 pin confirmations, finishing the workflow"


Similar to above: when do we decide to put this in the PR vs. the code itself? Do folks tend to do a git blame and look for comments to find more context?

+ add ability to run workflow against arbitrary path - until we have signed nightlies

lidel · 2021-08-03T22:42:13Z

As agreed during stewards call, I'm starting the merge dance starting with this PR

lidel force-pushed the feat/signed-ci-build branch 7 times, most recently from d891669 to a06abf7 Compare July 21, 2021 16:46

lidel mentioned this pull request Jul 21, 2021

feat: add ability to add-version via CI #362

Closed

11 tasks

lidel changed the title ~~feat: macos signing and notarization (refactor)~~ feat: macos signing and notarization Jul 21, 2021

lidel force-pushed the feat/signed-ci-build branch 2 times, most recently from 057619f to 2ba7177 Compare July 21, 2021 18:12

wip: signing poc with hardcoded dist details

cd78bdf

lidel force-pushed the feat/signed-ci-build branch 18 times, most recently from 7183e8a to 92386a4 Compare July 23, 2021 21:30

lidel added 4 commits July 27, 2021 13:47

docs: DNSLink discussion

f115f93

refactor: simplify

9acff7e

#367 (comment)

refactor: scripts/ci/setup-ipfs.sh

87c0b11

refactor: scripts/ci/pin-to-cluster.sh

9544654

lidel force-pushed the feat/signed-ci-build branch from 2852464 to 9544654 Compare July 28, 2021 13:16

lidel added 4 commits July 28, 2021 15:17

fix: quic perf

c239cfb

refactor: sign-new-macos-releases.sh

ef7843a

refactor: github-preview-link.sh

835df84

test: signed release built for ipfs-update@v1.7.1

1c9b839

lidel mentioned this pull request Jul 29, 2021

feat: create nightly builds #369

Merged

7 tasks

lidel added 2 commits July 29, 2021 15:07

docs: notes on reproducible builds

3a3e3d8

fix(ci): pin name for signed build

84af7e2

lidel commented Jul 29, 2021

View reviewed changes

docs: improved ./dist add-version

79f20c2

lidel commented Jul 29, 2021

View reviewed changes

lidel mentioned this pull request Jul 29, 2021

Apple Notarization of migration assets ipfs/kubo#8240

Closed

aschmahmann approved these changes Jul 29, 2021

View reviewed changes

README.md Show resolved Hide resolved

lidel mentioned this pull request Jul 29, 2021

Automate copying of signed binaries from dist.ipfs.io/go-ipfs ipfs/kubo#8316

Closed

BigLep approved these changes Jul 29, 2021

View reviewed changes

This was referenced Jul 30, 2021

feat: Add deep diffs to PRs #366

Merged

Add instructions how to get unsigned binaries #377

Open

chore: restore workflow targets

2d131bd

+ add ability to run workflow against arbitrary path - until we have signed nightlies

lidel merged commit 25aade5 into master Aug 3, 2021

lidel deleted the feat/signed-ci-build branch August 3, 2021 22:44

This was referenced Aug 3, 2021

Make nightlies signed #378

Open

MacOS Notarization #273

Closed

Build fails on macos due to unsupported comm flag --nocheck-order #350

Closed

Windows signing #384

Open

lidel mentioned this pull request Apr 15, 2022

Cannot verify ipfs on Mac when downloading binary file ipfs/kubo#8132

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: macos signing and notarization #367

feat: macos signing and notarization #367

lidel commented Jul 21, 2021 •

edited

Loading

lidel left a comment •

edited

Loading

lidel Jul 29, 2021

lidel Jul 29, 2021

BigLep Jul 29, 2021

lidel Aug 3, 2021 •

edited

Loading

lidel Jul 29, 2021

BigLep Jul 29, 2021

lidel Aug 3, 2021

BigLep Aug 3, 2021

lidel Jul 29, 2021

lidel Jul 29, 2021

lidel Jul 29, 2021

lidel Jul 29, 2021

lidel Jul 29, 2021

aschmahmann left a comment

BigLep left a comment

BigLep Jul 29, 2021

BigLep Jul 29, 2021

lidel commented Aug 3, 2021

		if [[ $(jq '.peer_map[].status' cluster-pin-status \| grep '"pinned"' \| wc -l) -ge 2 ]]; then
		echo "Got 2 pin confirmations, finishing the workflow"

		# fix resolv - DNS provided by Github is unreliable for DNSLik/dnsaddr
		sudo sed -i -e 's/nameserver 127.0.0.*/nameserver 1.1.1.1/g' /etc/resolv.conf

		brew tap mitchellh/gon
		brew install ipfs coreutils gawk gnu-sed jq mitchellh/gon/gon

feat: macos signing and notarization #367

feat: macos signing and notarization #367

Conversation

lidel commented Jul 21, 2021 • edited Loading

How this works?

Technical details

Flow for adding new dist or version

Content routing optimization

Website preview on PRs

TODO

lidel left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lidel Aug 3, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aschmahmann left a comment

Choose a reason for hiding this comment

BigLep left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lidel commented Aug 3, 2021

lidel commented Jul 21, 2021 •

edited

Loading

lidel left a comment •

edited

Loading

lidel Aug 3, 2021 •

edited

Loading