Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fast mmap()able model type #641

Closed
exaexa opened this issue Jul 26, 2022 · 1 comment · Fixed by #642
Closed

fast mmap()able model type #641

exaexa opened this issue Jul 26, 2022 · 1 comment · Fixed by #642
Labels
performance reduces time and resource consumption
Milestone
v1.5.x

Comments

@exaexa
Copy link
Collaborator

exaexa commented Jul 26, 2022 •
edited
Loading

Target functionality

provide a very quickly loading model type suitable for huge data, preferably mmap()able from disk.

Desired output

load_model preferably causes only a few mmap() syscalls

Optional: Suggestions for implementation

  • HDF5 should be able to make this work, if not we can build the binary data manually.
  • SparseMatrixCSC can work upon the mmapped data
  • is there any way to make mmapped stirngs?
@exaexa exaexa added the performance reduces time and resource consumption label Jul 26, 2022
@exaexa exaexa added this to the v1.4 milestone Jul 26, 2022
@exaexa exaexa linked a pull request Jul 27, 2022 that will close this issue
HDF5 model storage and fast memory mapping support #642
Merged
4 tasks
@exaexa
Copy link
Collaborator Author

exaexa commented Aug 1, 2022

closed by #642

@exaexa exaexa closed this as completed Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance reduces time and resource consumption
Projects
None yet
Milestone
v1.5.x
Development

Successfully merging a pull request may close this issue.

HDF5 model storage and fast memory mapping support LCSB-BioCore/COBREXA.jl
1 participant
@exaexa