Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: resolve error in highly concurrent by singleflight #143

Merged
merged 2 commits into from
May 31, 2023

Conversation

Desiki-high
Copy link
Collaborator

@Desiki-high Desiki-high commented May 29, 2023

When acceld handles highly concurrent conversion tasks, the content store will be locked because of fetching the same layer.

Changes:

  1. Add singleflight in LocalAdapter.Dispatch. If the image ref is the same, we only convert once simultaneously.
  2. Add singleflight in fetchHandler. If the desc.Digest is the same, we only fetch once simultaneously.

Github Action:

In Integration Test, we can move accel check out of build, it will make ci faster.

resolve #46.

@Desiki-high Desiki-high changed the title [WIP]feat: resolve error in highly concurrent by singleflight feat: resolve error in highly concurrent by singleflight May 30, 2023
Signed-off-by: Yadong Ding <ding_yadong@foxmail.com>
Copy link
Collaborator

@imeoer imeoer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

others LGTM!

pkg/adapter/adapter.go Outdated Show resolved Hide resolved
pkg/adapter/adapter.go Outdated Show resolved Hide resolved
pkg/content/ported.go Outdated Show resolved Hide resolved
Fixes: goharbor#46

Signed-off-by: Yadong Ding <ding_yadong@foxmail.com>
@Desiki-high
Copy link
Collaborator Author

@imeoer All fixed. PTAL.

@imeoer imeoer merged commit 54049ff into goharbor:main May 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handle highly concurrent conversion tasks
2 participants