-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor FileType
so compressed and archived file handling is transparent
#285
Labels
difficult
A difficult problem; a major coding effort or difficult algorithm to perfect
enhancement
New feature or request
P1
important
Comments
jtmoon79
added
enhancement
New feature or request
P1
important
difficult
A difficult problem; a major coding effort or difficult algorithm to perfect
labels
Apr 20, 2024
jtmoon79
changed the title
refactor
refactor Apr 20, 2024
FileType
to make compressed and archived files handled transparentenum FileType
to make compressed and archived file handling transparent
jtmoon79
changed the title
refactor
refactor Apr 20, 2024
enum FileType
to make compressed and archived file handling transparentFileType
, BlockReader
so compressed and archived file handling is transparent
This was referenced Apr 21, 2024
jtmoon79
changed the title
refactor
refactor Apr 22, 2024
FileType
, BlockReader
so compressed and archived file handling is transparentFileType
so compressed and archived file handling is transparent
jtmoon79
added a commit
that referenced
this issue
Apr 30, 2024
refactor `enum FileType` to embed archive and storage information in field variant `archival_type` Add variant `encoding_type` for `FileType::Text` refactor `pathbuf_to_filetype` to be more straightforward and recursive entirely remove `Mimeguess` Issue #15 (completed) This part 1 of completing the following issues: Issue #257 Issue #285
jtmoon79
added a commit
that referenced
this issue
Apr 30, 2024
refactor `enum FileType` to embed archive and storage information in field variant `archival_type` Add variant `encoding_type` for `FileType::Text` refactor `pathbuf_to_filetype` to be more straightforward and recursive entirely remove `Mimeguess` Issue #15 (completed) This part 1 of completing the following issues: Issue #257 Issue #285
jtmoon79
added a commit
that referenced
this issue
May 4, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
difficult
A difficult problem; a major coding effort or difficult algorithm to perfect
enhancement
New feature or request
P1
important
Summary
Currently
enum FileType
notion of compressed and archived files presumes those to be plain-text ad-hoc log files ("syslog" files).Refactor
enum FileType
to have variants to signify the other file types,FixedStruct
,Journal
, etc., may also be a compressed or archived.Current behavior
Currently, a compressed file,
log.gz
, is givenFileType::Gz
. It is presumed to be an ad-hoc text log ("syslog" file). This presumption means a compressed fixed struct file,wtmp.gz
, is presumed to be a compressed text log file and parsing fails.Suggested behavior
enum FileType
andBlockReader
should be refactored to more transparently handle files that are compressed or archived. Compression and archival should be generic sub-variants for primaryFileType
files.Issue #283 adds file-searching "seeking modes" (sequential seeking or random seeking). That "seeking mode" determination should be based on these proposed enum values.
Other Considerations
FileType::Journal
A
FileType::Journal
that is compressed or archived (archival_type != Normal
) would need to be handled by extracting to a named temporary file (Issue #284).Multiple levels of archiving or compression
This proposed design presumes "one level" of archiving or compression, e.g. can process file
log.tar
but cannot process filelog.tar.xz
. Chained Block Readers (Issue #14) implementation would follow this implementation. Chained Block Readers would require a stack ofFileArchivalType
held or referenced by theFileType
value, e.g. can process filelog.tar.xz
.Encoding
While working on this issue, can also cover Issue #16; add to
FileType::Text
a sub-varianttext_encoding
.The text was updated successfully, but these errors were encountered: