Public release of fv3net

[nbren12]

We can use this issue for tracking the process of preparing a public release of fv3net.

Related Issues:

• fv3fit tests require GCS access #902

[nbren12]

Here is the checklist: https://paper.dropbox.com/doc/Checklist-to-make-repo-public--BDq0i0~OjRKq4_u8n2m74rIHAg-IZYSz3BuQqT5XJPvIr3hF

[nbren12]

@oliverwm1 What were you thinking needed to be done here? In terms of house-keeping the only other thing I can think of is to clean up the top-level README file. I don't think it needs to be perfect or entirely reproducible outside of our environment before we open it up.

[oliverwm1]

There are still some tests that depend on GCS data. I suppose this isn't an absolute blocker but does mean tests could not be run locally by other users. Some that come to mind (probably not a complete list):

• e2e integration test
• prognostic report integration test
• fregrid test (since fregrid python interface downloads from GCS)

I also wonder if the many links to our GCS buckets are an issue? In particular w.r.t. to uploading reports to the public bucket.

[nbren12]

I think it could take a fair amount of effort to allow people to run the tests locally (we often struggle to do this), although I agree we should long-term try to remove the need for private data in our tests.

I'm similarly not concerned about the links to GCS buckets in the short-term, since I don't really consider them to be "secrets".

Just don't want "perfect" to be the enemy of "good".

[nbren12]

I expect at first most will just lightly peruse the code, and not actually try to run anything.

[nbren12]

• Repository has a license
• Repository is cleared for public release by legal (Matt Talpis)
• Disclaimer about work in progress, if relevant
• GitHub repository configuration has been reviewed (and adjusted), e.g.
  • Actions? In the repository, click the Settings tab followed by Actions in the left hand menu.
    • Change from Allow all actions to Allow local actions only
    • You must click the Save button at the bottom
    • Refresh the page and ensure the “Allow local actions only” item is still selected
  • Ensure “Run workflows from fork pull requests” is disabled
    • Make sure to hit Save and refresh the page to confirm the setting went through
  • Who can push (write)?
    • Restrict editing to users in teams with push access only is checked
    • @coreteam and external contributers who need it have write access
• Make sure no one ever checked in any secrets into the repo (fingerprints are not secrets)
  • Check git grep "PRIVATE KEY" $(git rev-list --all) and ensure that any checked-in keys have been invalidated
• Make sure CI is safe
  • If CircleCI is enabled, make sure “Build forked pull requests” and “Pass secrets to builds from forked pull requests” are disabled.
  • Make sure build logs of CI do not contain any secrets
    • mypy
    • lint
    • build_default-fv3fit
    • build_default-prognostic-run
    • build_default-fv3net
    • build_default-post-process
    • argo
    • test dataflow
    • integration-tests
      • Check if any environment variables are set for the project. If none are set, you should be done (since no secrets are checked in to the repository, right?)

[nbren12]

@ofuhrer @rheacangeo Could you comment on the need for "Remove circle CI deploy key". SSH fingerprints aren't private data are they?

[nbren12]

Regarding secrets. The phrase "PUBLIC KEY" does appear in git grep history, but it is checked into the binary file kustomize and does not contain any of our actual keys. I am surprised that kustomize was checked into version control.

[nbren12]

I reviewed the CI logs and didn't see any secrets.

[ofuhrer]

Sorry, only see this now. Got dropped in the flood of GitHub emails. I agree that SSH fingerprints aren't public data and we weren't too sure about CircleCI secrets. Please adapt the "making public repo checklist" according to newest insights!

[frodre]

I also reviewed the CI logs and didn't see any secrets.