Migrate github.com/docker/docker/pkg/fileutils and remove docker dependency from main go.mod #105

thaJeztah · 2021-06-09T12:54:48Z

depends on moby/moby#42492
relates to moby/buildkit#2150 (comment)

See individual commits for details

git clone https://github.com/thaJeztah/docker.git ~/Projects/temp_docker_fsutils
cd ~/Projects/temp_docker_fsutils
git remote set-url --push origin no_pushing
git fetch origin -v
git checkout remove_fsutil_debug_log

# remove all code, except for the files we need, and rename the files
git filter-repo \
    --force \
    --path-match pkg/fileutils/fileutils.go \
    --path-match pkg/fileutils/fileutils_test.go \
    --path-rename pkg/fileutils/fileutils.go:filematch/patternmatcher.go \
    --path-rename pkg/fileutils/fileutils_test.go:filematch/patternmatcher_test.go


# integrate the files in the docs repository
cd ~/projects/fsutil
git checkout -b migrate_fileutil
git remote add temp_docker_fsutils ~/projects/temp_docker_fsutils
git fetch temp_docker_fsutils

git merge --allow-unrelated-histories --signoff -S temp_docker_fsutils/remove_fsutil_debug_log

This is the second of two steps to break the archive package's dependence on utils so that archive may be moved into pkg. `Matches()` is also a good candidate pkg in that it is small, concise, and not specific to docker internals Signed-off-by: Rafe Colton <rafael.colton@gmail.com>

Fixed #8761 Signed-off-by: Alexandr Morozov <lk4d4@docker.com>

Bump version to 1.4.0

Merge release v1.4.0

Signed-off-by: Antonio Murdaca <me@runcom.ninja>

Signed-off-by: Dave Goodchild <buddhamagnet@gmail.com>

Should fix #11598 Signed-off-by: Vincent Demeester <vincent@sbr.pm>

Change CLI error msg because it was too specific and didn't make sense when there were errors not related to inaccessible files. Removed some log.Error() calls since they're not really errors we should log. Returning the error will be enough. Closes: #13417 Signed-off-by: Doug Davis <dug@us.ibm.com>

Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>

linting changes

Signed-off-by: Lei Jitang <leijitang@huawei.com>

Signed-off-by: Vincent Demeester <vincent@sbr.pm>

Signed-off-by: John Howard <jhoward@microsoft.com>

Windows: Fix error in daemon log on info

Closes #13113 Signed-off-by: Doug Davis <dug@us.ibm.com>

Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>

Bugfix - Typos

Signed-off-by: John Howard <jhoward@microsoft.com>

Signed-off-by: allencloud <allen.sun@daocloud.io>

Signed-off-by: Elena Morozova <lelenanam@gmail.com>

.dockerignore pattern of **/.foo incorrectly matched **/bar.foo because **/.foo was getting converted into a .*\.foo regex instead of (.*/)*\.foo Closes #29014 Signed-off-by: Doug Davis <dug@us.ibm.com>

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>

Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>

Signed-off-by: Daniel Nephin <dnephin@docker.com>

Fixes case sensitivity issue Signed-off-by: Derek McGowan <derek@mcgstyle.net>

Signed-off-by: Daniel Nephin <dnephin@docker.com>

gty-migrate-from-testify --ignore-build-tags Signed-off-by: Daniel Nephin <dnephin@docker.com>

Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>

Signed-off-by: Vincent Demeester <vincent@sbr.pm>

Update gotestyourself to gotest.tools

Signed-off-by: mooncake <xcoder@tenxcloud.com>

``` builder/remotecontext/remote.go:48: G107: Potential HTTP request made with variable url (gosec) builder/remotecontext/git/gitutils.go:145: G107: Potential HTTP request made with variable url (gosec) builder/remotecontext/git/gitutils.go:147: G107: Potential HTTP request made with variable url (gosec) pkg/fileutils/fileutils_test.go:185: G303: File creation in shared tmp directory without using ioutil.Tempfile (gosec) pkg/tarsum/tarsum_test.go:7: G501: Blacklisted import `crypto/md5`: weak cryptographic primitive (gosec) pkg/tarsum/tarsum_test.go:9: G505: Blacklisted import `crypto/sha1`: weak cryptographic primitive (gosec) ``` Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Replace gometalinter with golangci-lint

full diff: gotestyourself/gotest.tools@v2.3.0...v3.0.1 Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

These tests were no longer valid on Go 1.16; related to https://tip.golang.org/doc/go1.16#path/filepath > The Match and Glob functions now return an error if the unmatched part of > the pattern has a syntax error. Previously, the functions returned early on > a failed match, and thus did not report any later syntax error in the pattern. Causing the test to fail: === RUN TestMatches fileutils_test.go:388: assertion failed: error is not nil: syntax error in pattern: pattern="a\\" text="a" --- FAIL: TestMatches (0.00s) Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Trying to avoid logging code in "libraries" used elsewhere. If this debug log is important, it should be easy to add in code that's calling it. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

This package only contains testing / benchmarking code, so dependencies needed for this package should not be in the main go.mod. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

It was only used for very minimal checks, so removing it to not have this as a dependency. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

aledbf · 2022-03-18T15:37:08Z

@thaJeztah any chance to get this change merged? Thanks!

estroz · 2022-04-14T22:31:35Z

The dependency on docker here currently pins docker in any consumer of docker/buildx, so it would be nice to see this PR merged. I can help if needed!

estroz · 2022-04-14T22:33:02Z

cc @tonistiigi

thaJeztah · 2022-09-30T20:19:29Z

I opened #136 to replace this one; that migrates to a separate module

rafecolton and others added 30 commits September 29, 2014 23:21

Use logrus everywhere for logging

6997122

Fixed #8761 Signed-off-by: Alexandr Morozov <lk4d4@docker.com>

Merge pull request #9345 from jfrazelle/bump_v1.4.0

3f34172

Bump version to 1.4.0

Merge pull request #9623 from jfrazelle/merge_release_v1.4.0

c40acfb

Merge release v1.4.0

Replace aliased imports of logrus, fixes #11762

adf480f

Signed-off-by: Antonio Murdaca <me@runcom.ninja>

Refactor ultis/utils_daemon, fixes #11908

729de31

Signed-off-by: Antonio Murdaca <me@runcom.ninja>

Refactor utils/utils, fixes #11923

0a27ce5

Signed-off-by: Antonio Murdaca <me@runcom.ninja>

add support for exclusion rules in dockerignore

efa3920

Signed-off-by: Dave Goodchild <buddhamagnet@gmail.com>

Add coverage on pkg/fileutils

12de81b

Should fix #11598 Signed-off-by: Vincent Demeester <vincent@sbr.pm>

linting changes

c1c4092

Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>

Merge pull request #13961 from unclejack/linting_changes

bcd8472

linting changes

Fix copy from a "created" container. Fixes #14420

b1e4b70

Signed-off-by: Lei Jitang <leijitang@huawei.com>

Add missing tests and docs for pkg/fileutils

0ef7a97

Signed-off-by: Vincent Demeester <vincent@sbr.pm>

Windows: Fix warning on info

7a793b5

Signed-off-by: John Howard <jhoward@microsoft.com>

Merge pull request #16407 from Microsoft/10662-fixinfoprocerror

5874684

Windows: Fix error in daemon log on info

Support multi-dir wildcards in .dockerignore

0fc3ada

Closes #13113 Signed-off-by: Doug Davis <dug@us.ibm.com>

Fix typos found across repository

a2acf9a

Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>

Merge pull request #18623 from jutaz/bugfix/typos

35f2440

Bugfix - Typos

Windows CI: Fixes panic in test-unit for FileUtils

84a541e

Signed-off-by: John Howard <jhoward@microsoft.com>

Windows CI: Turn off failing unit test pkg\fileutils

78e8dc8

Signed-off-by: John Howard <jhoward@microsoft.com>

fix typos

357fbde

Signed-off-by: allencloud <allen.sun@daocloud.io>

fix typos

ab6836d

Signed-off-by: allencloud <allen.sun@daocloud.io>

all: use strings.Contains instead Index

92cb27b

Signed-off-by: Elena Morozova <lelenanam@gmail.com>

Fix use of **/ in .dockerignore

94abe55

.dockerignore pattern of **/.foo incorrectly matched **/bar.foo because **/.foo was getting converted into a .*\.foo regex instead of (.*/)*\.foo Closes #29014 Signed-off-by: Doug Davis <dug@us.ibm.com>

Fix inefficient file paths filter

05da6ee

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>

pkg/file{utils,notify}: don't compare to bool

3590d84

Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>

Enable a unit test on windows.

e1cdf57

Signed-off-by: Daniel Nephin <dnephin@docker.com>

Update logrus to v1.0.1

32ab3cb

Fixes case sensitivity issue Signed-off-by: Derek McGowan <derek@mcgstyle.net>

Add goimports to linters.

ddcb242

Signed-off-by: Daniel Nephin <dnephin@docker.com>

dnephin and others added 17 commits February 5, 2018 16:51

Add canonical import comment

cd692c7

Signed-off-by: Daniel Nephin <dnephin@docker.com>

Automated migration using

aeae3e8

gty-migrate-from-testify --ignore-build-tags Signed-off-by: Daniel Nephin <dnephin@docker.com>

Fix typos

6b3b3d2

Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>

Update tests to use gotest.tools 👼

bf06bf6

Signed-off-by: Vincent Demeester <vincent@sbr.pm>

Merge pull request #37243 from vdemeester/gotestyourself-with-tools

31861b4

Update gotestyourself to gotest.tools

Remove duplicated words in pkg files

84a1066

Signed-off-by: mooncake <xcoder@tenxcloud.com>

Merge pull request #39668 from thaJeztah/replace_gometalinter

a3c9386

Replace gometalinter with golangci-lint

bump gotest.tools v3.0.1 for compatibility with Go 1.14

e67d934

full diff: gotestyourself/gotest.tools@v2.3.0...v3.0.1 Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

pkg/fileutils: PatternMatcher.Matches(): remove debug logging

b340094

Trying to avoid logging code in "libraries" used elsewhere. If this debug log is important, it should be easy to add in code that's calling it. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Migrate github.com/docker/docker/pkg/fileutils

35f6224

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

filematch: update package name

a8a9249

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

filematch: remove utilities we don't use

85eadbf

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

replace github.com/docker/docker/pkg/fileutils with filematch

cab31cd

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

bench: make separate module

eaffca0

This package only contains testing / benchmarking code, so dependencies needed for this package should not be in the main go.mod. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

filematch: remove use of gotest.tools

82bb6d3

It was only used for very minimal checks, so removing it to not have this as a dependency. Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

thaJeztah mentioned this pull request Jun 9, 2021

bench: make separate module #106

Merged

thaJeztah mentioned this pull request Aug 18, 2021

Update fsutil to latest master and remove deprecated fileutil.Matches moby/moby#42758

Closed

aledbf mentioned this pull request Aug 23, 2022

Error linking to v0.10.4 moby/buildkit#3055

Closed

This was referenced Sep 30, 2022

pkg/fileutils: remove gotest.tools as dependency, use t.TempDir() moby/moby#44223

Merged

replace pkg/fileutils Matching funcs with github.com/moby/patternmatcher moby/moby#44226

Merged

Replace docker/pkg/fileutils with github.com/moby/patternmatcher #136

Merged

tonistiigi closed this in #136 Sep 30, 2022

thaJeztah deleted the migrate_fileutil branch September 30, 2022 23:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate github.com/docker/docker/pkg/fileutils and remove docker dependency from main go.mod #105

Migrate github.com/docker/docker/pkg/fileutils and remove docker dependency from main go.mod #105

thaJeztah commented Jun 9, 2021

aledbf commented Mar 18, 2022

estroz commented Apr 14, 2022 •

edited

Loading

estroz commented Apr 14, 2022

thaJeztah commented Sep 30, 2022

Migrate github.com/docker/docker/pkg/fileutils and remove docker dependency from main go.mod #105

Migrate github.com/docker/docker/pkg/fileutils and remove docker dependency from main go.mod #105

Conversation

thaJeztah commented Jun 9, 2021

aledbf commented Mar 18, 2022

estroz commented Apr 14, 2022 • edited Loading

estroz commented Apr 14, 2022

thaJeztah commented Sep 30, 2022

estroz commented Apr 14, 2022 •

edited

Loading