Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LocationResolver can leak goroutines #2487

Closed
wagoodman opened this issue Jan 11, 2024 · 3 comments · Fixed by #2518
Closed

LocationResolver can leak goroutines #2487

wagoodman opened this issue Jan 11, 2024 · 3 comments · Fixed by #2518
Assignees
Labels
bug Something isn't working
Milestone

Comments

@wagoodman
Copy link
Contributor

Today the LocationResolver returns a channel of locations:

type LocationResolver interface {
	AllLocations() <-chan Location
}

However, while iterating though the channel if you return early you will leak a go routine. This should probably be updated to allow for cancellation via context and follow an iterator pattern (soft suggestion):

type LocationResolver interface {
	AllLocations() LocationIterator
}

type LocationIterator interface {
	Next(context.Context) (Location, bool)
}

Where the caller would now be able to coordinate cancellation for correct usage:

ctx, cancel := context.WithCancel(ctx)
defer cancel()

iter := resolver.AllLocations()
for {
	location, ok := iter.Next(ctx)
	if !ok {
		break
	}
	// process the location...
}

Alternatively AllLocations can be a slice, which is simpler, but may cause memory pressures for images with many files:

type LocationResolver interface {
	AllLocations() [] Location
}
@wagoodman wagoodman added the bug Something isn't working label Jan 11, 2024
@wagoodman wagoodman added this to the Syft 1.0 milestone Jan 11, 2024
@willmurphyscode willmurphyscode self-assigned this Jan 18, 2024
@willmurphyscode
Copy link
Contributor

We should also pass context down via the cataloger interface: https://github.com/anchore/syft/blob/main/syft/pkg/cataloger.go#L15.

@willmurphyscode
Copy link
Contributor

willmurphyscode commented Jan 19, 2024

I have a question about the suggested implementation: Why are we making a separate iterator? Why no let AllLocations accept a context, like this (example from the directory iterator):

func (r *Directory) AllLocations(ctx context.Context) <-chan file.Location {
	results := make(chan file.Location)
	go func() {
		defer close(results)
		for _, ref := range r.tree.AllFiles(stereoscopeFile.AllTypes()...) {
			select {
			case <-ctx.Done():
				return
			case results <- file.NewLocationFromDirectory(r.responsePath(string(ref.RealPath)), ref):
				continue
			}
		}
	}()
	return results
}

Testing this with "go.uber.org/goleak" seems to work fine:

func TestAllLocationsDoesNotLeakGoRoutine(t *testing.T) {
	defer goleak.VerifyNone(t)
	resolver, err := NewFromDirectory("./test-fixtures/symlinks-from-image-symlinks-fixture", "")
	require.NoError(t, err)
	ctx, cancel := context.WithCancel(context.Background())
	for _ = range resolver.AllLocations(ctx) {
		break
	}
	cancel()
}

This seems simpler than having a struct carry around the state of the iteration and implement a new interface. What do you think @wagoodman?

@wagoodman
Copy link
Contributor Author

wagoodman commented Jan 19, 2024

I'm down with not needing to bring another interface along for the ride -- overall I think this looks good (and is a simpler change) 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants