-
Notifications
You must be signed in to change notification settings - Fork 10
Skyhook Architecture
Skyhook's architecture includes 3 layers:
- Database application layer (PostgreSQL)
- External table interface layer (PostgreSQL's foreign data wrapper)
- Storage layer (Ceph with programmable storage functions)
Skyhook's design is based on partitioning database table data and storing each partition into a Ceph object and the partition's metadata stored in the local RocksDB instance on the Ceph server that stores the object. The diagram below indicates data partitioning and placement. Replication is transparently provided by Ceph and not shown here.
Skyhook's development is based on
- PostgreSQL's mature external table interface ("foreign data wrapper") which enables read/write and limited pushdown processing to the external source.
- Ceph's extensible object classes ("cls") interface which enables developers to write custom extensions to Ceph's object methods.
Concepts
> Architecture
> Data formats
> > Flatbuffers
> > Arrow
> Test Integration
Tutorials
> Build
> Dev/Test environment
> Run test queries
> Ceph-SkyhookDM cluster setup
Technical Reports
> Google Summer of Code 2019 Report
> Google Summer of Code 2020 Report
> Flatbuffers and Flexbuffers access experiments
Archives
> CloudLab Ceph Deployment Notes
> Deploy Notes
> Running CloudLab Experiments
> Installing a Non Release Fork on CloudLab
> Installing with Skyhook-Ansible
> FBU Queries (PDSW19)
> Paper Experiments
> Skyhook Development on CloudLab
> Building Deb Files for Cloudlab Installs (Sp2019)
> CephFS FUSE on CloudLab