Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor2 the proxying #2

Merged
merged 17 commits into from
Jun 19, 2022
Merged

Refactor2 the proxying #2

merged 17 commits into from
Jun 19, 2022

Conversation

briantist
Copy link
Owner

A lot of big changes here:

  • properly license as GPLv3
  • setting the API key for Artifactory is not possible directly, optionally via the galaxy token sent with a request, or both
  • all requests to artifactory now include the API key if one is available; that makes it possible to use this with repos that are not open anonymously for reads
  • all downloads are now proxied through galactory (previously we sent the galaxy client the direct artifactory URL), which should have only a negligible performance impact, but now also enables authenticated downloads possible, and proxying upstreams (next item)
  • support for proxying to an upstream galaxy server
    • only tested with the Galaxy, a server that doesn't match its endpoints exactly may not work
    • an upstream that requires authorization to read will not work at this time (no way to configure it, and we strip all authorization from upstream requests to avoid leaking the artifactory key)
    • ensures upstream results are combined with locally available results (not for the general "tell me all the collections path")
    • to make this feasible, we need to cache the upstream results; rather than cache them locally, I've chosen to cache them inside artifactory, to fit the use case of multiple instances of galactory (short lived instances running in CI for example) not hitting upstream throttling
    • when proxying to an upstream, because normal reads now need to write to artifactory, an API key is needed even if the repository is readable anonymously
    • namespaces that should never be proxied upstream can be configured (don't want to be sending a bunch of API requests to galaxy for your_company.internal_collection all the time)
    • when collections are downloaded from an upstream, they get uploaded to artifactory first, making them local in subsequent requests (although that should all be transparent)
      • this can enable a scenario where a "trusted" instance, configured with an artifactory API key, and knows all the upstream collections needed, can ansible-galaxy install or ansible-galaxy download them to a throwaway location, in order to ensure they're available from artifactory
      • other instances can be configured not to proxy upstream, and they will only ever download from what's available in artifactory
      • if they aren't proxying, and the artifactory repo allows anonymous downloads, then no API key will be needed for this
  • all uploads (direct or via proxying) now calculate all 3 hashes (MD5, SHA1, SHA256) so that they can be properly verified in Artifactory

@briantist briantist self-assigned this Jun 18, 2022
@briantist briantist merged commit b718e25 into main Jun 19, 2022
@briantist briantist deleted the refactor2-the-proxying branch June 21, 2022 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant