Unable to cache files on separate drives

[vkee]

Hi,

I just started using vmtouch and it's been working great for the most part. However, I am unable to cache files that are stored on a separate drive using the 9P filesystem. The files either are cached and then immediately removed when touching them or FATAL (Cannot allocate memory) errors are thrown when trying to lock the files in memory.

Do you have any idea of workarounds?

Thanks,
Vincent

[hoytech]

Hi thanks for writing. I have no experience with the 9P filesystem. Which OS are you on, and which driver are you using? Does it use FUSE? Are there any special mount options on the FS?

I have however used S3FS before which is a FUSE mount for amazon's S3 system. It interacts with the page cache in a strange way: It creates a special temp directory on the filesystem and these are the files that back the page cache for the filesystem, so vmtouch needs to be run on them, not the files that appear in the mounted directory.

Regarding the FATAL errors when locking, does this occur when run as the root user as well? Most OSs have resource limits and/or system-wide limits on "wired memory". You can read more about those here: https://github.com/hoytech/vmtouch/blob/master/TUNING.md

[vkee]

I'm running Ubuntu 14.04 LTS on QEMU host with VIRTIO transport. I don't think I am using FUSE or any special mount options either. I am just using the 9p with the 9P2000.L protocol.

I see there are some options regarding cache:
cache=mode specifies a caching policy. By default, no caches are used.
none = default no cache policy, metadata and data
alike are synchronous.
loose = no attempts are made at consistency,
intended for exclusive, read-only mounts
fscache = use FS-Cache for a persistent, read-only
cache backend.
mmap = minimal cache that is only used for read-write
mmap. Northing else is cached, like cache=none

I am not using this flag, so I guess this is cache=none. Perhaps this flag needs to be set?

Also, I tried running as root and it was successful (and vmtouch showed that the file was cached completely). However, when I ran my programs using the file, the performance was not as good as when the file was on the local SSD so I don't think it quite worked as intended.

[hoytech]

Ahh interesting. I would try to set the cache to "fscache" if you want vmtouch to be able to control it. Hopefully you can run vmtouch directly on the files instead of the temporary backing files like with S3FS.

Please let us know what you find out. :)

[vkee]

It seems to have fixed the issue, thanks a lot for all your help!

[hoytech]

Thanks for letting me know. I've added a brief note about 9P to the TUNING.md file: https://github.com/hoytech/vmtouch/blob/master/TUNING.md