Skip to content

Latest commit

 

History

History
21 lines (15 loc) · 856 Bytes

disaggregated-storage.md

File metadata and controls

21 lines (15 loc) · 856 Bytes

Building an elastic query engine on disaggregated storage

Source: https://blog.acolyer.org/2020/03/09/snowflake/

Cloud-native design principles:

  1. scalability of independent resources - disaggregation to allow resources to scale independently based on pre-determined resourcec units
  2. elasticity with declarable units - quantamize resources to allow expand or contract resources in those unit steps.
  3. multi-tenancy with isolation

Current trends:

  1. Variety of data sources to support is ever increasing.
  2. Most workloads vary in system resource (cpu, mem, i/o) demands based on traffic

Data processing guidelines:

  1. decouple compute and storage resource management
  2. an emphemeral storage for transient usecases like number crunching, union of results/queries, caching
  3. isolation of storage and compute for workloads

Observations:

  1. ...