use --json-lines mode for list archive results for better parsing of unusual filenames #885
Conversation
rblenis
force-pushed
the
issue_883
branch
3 times, most recently
from
8fdcc0d to
d9ca64c
Compare
|
Neat 👍 Actually we already have a way to use different borg features, depending on version. Base class is here: https://github.com/borgbase/vorta/blob/master/src/vorta/borg/_compatibility.py See it in action: https://github.com/borgbase/vorta/blob/master/src/vorta/borg/create.py#L115 |
|
ok, awesome. I also saw in BorgThread that at least borg 1.1.0 is required to do anything, so I don't need to support the non "--json-lines" archive list output. I also just fixed issue of datetime for python < 3.7 |
Codecov Report
@@ Coverage Diff @@
## master #885 +/- ##
==========================================
- Coverage 70.32% 70.28% -0.04%
==========================================
Files 56 56
Lines 3791 3793 +2
==========================================
Hits 2666 2666
- Misses 1125 1127 +2
Continue to review full report at Codecov.
|
rblenis
force-pushed
the
issue_883
branch
2 times, most recently
from
38cb192 to
47e8b7c
Compare
…n handle unusual filenames.
|
I'll do some edits, as usual. E.g. would like to add this to the compatibility class, since it needs a newer Borg. |
|
Does this ('list --json-lines') need compatibility check? It seems that the bare minimum borg version for vorta (because of --json-logs' support, which is cheecked in base borg_thread) is borg 1.1.0, which also supports 'list --json-lines'. |
|
You're right. We require >1.1.0 already vorta/src/vorta/borg/borg_thread.py Line 132 in 8da81e7 |
| # modified = datetime.fromisoformat(data["mtime"]).ctime() | ||
| # python < 3.7 | ||
| try: | ||
| modified = datetime.strptime(data["mtime"], "%Y-%m-%dT%H:%M:%S.%f").ctime() |
There was a problem hiding this comment.
How about using the date string as-is? Else this could get slow for many entries. (hundreds of thousands of files are common) Back when I added the feature, I tried hard to keep it fast.
There was a problem hiding this comment.
Actually it's still fast even when parsing the date. Tried with 40k files.
There was a problem hiding this comment.
I don't recall now, but I think I was trying to get it into a datetime object so it would be easier to manipulate how it was displayed on the gui, if ever needed.
There was a problem hiding this comment.
The time string from json is a bit ugly with MS included. So let's just use ctime() for now. Maybe in the future there could be a setting for a default time format or something.
This change relies on using borg >= 1.1.0. Do we need to support older versions? If so, I can add checks for borg version and fallback to old text parsing. This fixes one of the 2 cases in issue #883.