Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker layer caching #5882

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Docker layer caching #5882

wants to merge 12 commits into from

Conversation

galvana
Copy link
Contributor

@galvana galvana commented Mar 12, 2025

Closes []

Description Of Changes

Write some things here about the changes and any potential caveats

Code Changes

  • list your code changes here

Steps to Confirm

  1. list any manual steps for reviewers to confirm the changes

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Followup issues:
    • Followup issues created (include link)
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

Copy link

vercel bot commented Mar 12, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
fides-privacy-center ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 13, 2025 4:16pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
fides-plus-nightly ⬜️ Ignored (Inspect) Visit Preview Mar 13, 2025 4:16pm

Copy link
Contributor

@NevilleS NevilleS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits and questions - but overall this seems like a slam dunk

@@ -21,7 +21,8 @@ Changes can also be flagged with a GitHub label for tracking purposes. The URL o

## [Unreleased](https://github.com/ethyca/fides/compare/2.57.0...main)


### Developer experience
- Adding Docker layer caching for the backend check's build step [#5882](https://github.com/ethyca/fides/pull/5882)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's start nudging our CHANGELOG entries to be more clear. We should be able to write some cursor rules to help with that too 👍.

I think GitLab's thoughts on this are pretty good: https://docs.gitlab.com/development/changelog/#writing-good-changelog-entries

In general I'd say the CHANGELOG should look like: <what was improved> + <how> + <when (if relevant)>

So in your case I'd do this:

Suggested change
- Adding Docker layer caching for the backend check's build step [#5882](https://github.com/ethyca/fides/pull/5882)
- Improved CI build times for backed checks by adding Docker layer caching to GHA workflow [5882](https://github.com/ethyca/fides/pull/5882)

Comment on lines -35 to -39
# Install Python Dependencies

COPY dev-requirements.txt .
RUN pip install --user -U pip --no-cache-dir -r dev-requirements.txt

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You sure about this one?


COPY dev-requirements.txt .
RUN pip install --user -U pip --no-cache-dir -r dev-requirements.txt

# Activate a Python venv
RUN python3 -m venv /opt/fides
ENV PATH="/opt/fides/bin:${PATH}"

# Install Python Dependencies
RUN pip --no-cache-dir --disable-pip-version-check install --upgrade pip setuptools wheel
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're optimizing, you should be able to include this in the pip install - requirements.txt below too, right?

- name: Cache Docker layers
uses: actions/cache@v3
with:
path: /tmp/.buildx-cache${{ matrix.python_version == '3.9.18' && '-py39' || '' }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are concurrent GHA jobs both running at the same time, is this cache shared between them? In other words, could this lead to strange situations where code from one PR gets into another one?

Or is this caching strictly tied to the particular workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants