-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: log dandischema version and ensure we log (consistently) path for log messages in download #1499
base: master
Are you sure you want to change the base?
Conversation
…r log messages in download
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1499 +/- ##
==========================================
- Coverage 88.51% 88.17% -0.35%
==========================================
Files 77 77
Lines 10572 10671 +99
==========================================
+ Hits 9358 9409 +51
- Misses 1214 1262 +48
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
…with or without exception etc
…message while at it
else: | ||
lgr.debug( | ||
"Download directory found, but digests do not match; starting new download" | ||
"%s - download directory found, but digests do not match;" | ||
" starting new download", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would digests.keys() and self. digests.keys() be useful to know here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I felt like I wondered the same and then dismissed as it would be "too much noise". but now not sure -- it might be... yet to decide
Also shortened the log line to not include traceback
…(via inode matching)
not sure why was not failing for me locally but fails on CI
That is my guess for what is happening in ________________________ test_DownloadDirectory_basic _________________________ dandi\tests\test_download.py:1048: in test_DownloadDirectory_basic with DownloadDirectory(tmp_path, digests={}) as dl: dandi\download.py:889: in __exit__ self.writefile.replace(self.filepath) C:\hostedtoolcache\windows\Python\3.10.11\x64\lib\pathlib.py:1247: in replace self._accessor.replace(self, target) E PermissionError: [WinError 5] Access is denied: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_DownloadDirectory_basic0.dandidownload\\file' -> 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pytest-of-runneradmin\\pytest-0\\test_DownloadDirectory_basic0'
@@ -853,12 +869,35 @@ def __exit__( | |||
exc_tb: TracebackType | None, | |||
) -> None: | |||
assert self.fp is not None | |||
if exc_type is not None or exc_val is not None or exc_tb is not None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
exc_tb
is not used in the log here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
first I had it but then kicked out from actual log -- the point is that if any of them provided, exception happened.
Very unlikely but it could be that directory already existed but without checksum file for some reason.
no v prefix since is not providing any information on top; sorting for deterministic order
…th was set to default to True
…IVE_RETRY mode This is all to address that odd case with 000026 where connection keeps interrupting. Unclear why so adding more specific cases handling and allowing for such an aggressive retrying where we would proceed as long as we are getting something (but sleep would also increase)
hopefully would be in assist to troubleshoot issues with downloads, like the one now reported by @Kevancic on slack