-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trufflehog hangs while scanning archived files #884
Comments
trufflehog 3.16.1 Looks like I have the same issue: TRAC[0003] scanning file file_path=/Users/some_path/charts/kube-prometheus-stack-39.11.0.tgz and here it hangs. |
I am experiencing this issue as well |
Same thing happened here using I downgraded to I'll start narrowing down the version and reproducability... |
Confirmed that the first version I see this with is Nothing in the changelog description between Will try to bisect the changes next. |
I also was able to reproduce this one but was not able to figure out what’s wrong Even so I believe than one more check shouldn’t do it, but indeed, you are correct my changes were in last release so I did tried to comment out sql server detector but nothing changed, so not sure if it may be a root cause, even more I reproducing this locally while commenting absolutely all detectors From what I understand problem is somewhere in channels, archive module seems to be working but extracted file does not being passed further via pipeline At moment it seems that for now workaround might be to add archive files to exclusions (but it works only for git check) |
BTW very good idea about bisect 👍 (I do not remember how to do it so did it semi manually) from my experiments the last commit when everything was fine is: 034ca4f Add bytes counter to scans (#876) and after next commit problem occur: ab71b93 Add context to handler (#877) @bill-rich if I understand everything right you have the most context around #877 and it seems that after that change issue occur (at least from my experiments but I might be wrong) will be so nice if you have a look at the problem |
I confirm that if I’d change handlers.go back to original version with range everything starts work but have no idea of what’s need to be done to this select to start working 🤷♂️ We need help of experienced go land engineers here |
Thanks for digging in! I got distracted. I know nothing about debugging Golang code, so by the time-honored method of sprinkling printfs and running strace, I can see that for me, it consistently hangs inside the for...select loop in
It'll get stuck with a bunch of threads endlessly reporting ready, read; ready, read; ready, read forever. Neither the According to strace, basically nothing is happening between those loop iterations except some According to |
While digging in I checked the latest commits and I think the refactoring that occurred in ab54ec4 had a side effect of fixing the problem. When I try using the tip of |
confirming (I'm on ecd2578) everything works 🎉 @mcastorina 💪 thanks |
yes, it works |
Awesome investigative work in this thread, thanks everyone! |
Thanks everyone! Looks like that fix made it into v3.16.4 |
Community Note
TruffleHog Version
3.16.1
Trace Output
hangs with
RAC[0002] Remaining buffer capacity: 43586
TRAC[0002] Handling extracted file. filename=wipefs
TRAC[0002] Remaining buffer capacity: 8130
DEBU[0002] Max archive size reached.
DEBU[0002] Error unarchiving chunk. error="archive/tar: invalid tar header"
Remaining
Expected Behavior
Should cleanly exit
Actual Behavior
hangs foreever
Steps to Reproduce
Environment
kali linux
Additional Context
References
The text was updated successfully, but these errors were encountered: