A buffer pool file I/O library for Go.
The purpose of the library is to provide I/O mechanisms for copying data to and from a buffer in user space and maintain complete control over how and when it transfers pages.
This is an alternative to using mmap on large DBMS like files.
go get github.com/tidwall/bfile
Use the NewPager function to create a new Pager, which includes three
functions: ReadAt, WriteAt, and Flush.
The Pager reading and writing works like an os.File, but with an
automatically maintained pool of buffered pages.
// Open a file for read/write
f, err := os.OpenFile("bigfile.dat", os.O_RDWR, 0)
// Create a new Pager for accessing the data in opened file.
p := bfile.NewPager(f)
// Read some data at a specific offset.
data := make([]byte, 50)
n, err := p.ReadAt(data, 827364)
// Write some data at the same offset.
n, err := p.WriteAt([]byte("hello"), c)
// Flush unwritten data to the underlying os.File
err = p.Flush()There's also a Stream object for sequentially reading and writing data, which
includes the three functions: Read, Write, and Flush.
// Create a new Pager
p := bfile.NewPager(f)
// Create a Stream that is backed by the Pager starting at a specific offset
s := p.Stream(827364)
// Read 50 bytes data at the offset 827364.
data := make([]byte, 50)
n, err := s.Read(data)
// Write the string "hello" at 827414, which is after the previous read.
n, err := s.Write([]byte("hello"))
// Flush unwritten data to the underlying os.File
err = p.Flush()The default page size is 4096 and the default size of all buffered pages will
not exceed 8 MB.
These defaults can be changed by using the NewPagerSize function.
All operations are thread-safe.
Josh Baker @tidwall
Source code is available under the MIT License.