Skip to content

the Clymene is time-series data and Logs collection platform for distributed systems.

License

Notifications You must be signed in to change notification settings

Clymene-project/Clymene

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Clymene

CII Best Practices CodeQL License GitHub release (latest by date) Go Reference
Go Docker Kubernetes ElasticSearch

clymene_logo

The Clymene is a time-series data and logs collection platform for distributed systems inspired by Prometheus and Jaeger. Time-series data and logs from various environments can be collected and stored in different types of databases. It can be configured in a variety of architectures. Choose the storage that users can use easily and build a monitoring system with dashboards that users can use easily. Also, Clymene Agent's time series data collection uses fewer resources than Prometheus' remote_write. Various time series data analysis functions will be added.

Architecture

The Clymene project is a platform for collecting time series data and log data. So there are two pipelines: timeseries and logs. All agents can collect data and store it directly in the database, Or, depending on the situation, you can use the Clymene components to configure a variety of architectures. It can be configured with an architecture that can be used with a lot of traffic. The Clymene Project provides various storage types, so choose a storage type you are familiar with. And build great monitoring systems using familiar dashboards.

architecture_v2.1.0

architecture_v2.2.2

Components

Pipeline 1. TimeSeries data collection

Clymene Agent(Getting Started)

dockerhub : bourbonkk/clymene-agent:v2.x.x
redhatQuay: quay.io/clymene/clymene-agent:v2.x.x
Docker Pulls Docker Repository on Quay
The Clymene-agent is service that collects time series data(does not use disks)

  1. Service Discovery
  2. scrape time series data
  3. Time-series data transfer to gateway(gRPC) (Optional)
  4. Time-series data transfer to kafka (Optional)
  5. Time-series data insert to Database(supported DB) (Optional)

Clymene Ingester(Optional) (Getting Started)

dockerhub : bourbonkk/clymene-ingester:v2.x.x
redhatQuay: quay.io/clymene/clymene-ingester:v2.x.x
Docker Pulls Docker Repository on Quay
The Clymene ingester is an optional service responsible for insert time series data loaded on kafka into the database.

  1. Kafka message consume
  2. Time-series data insert to Database(supported DB) (Optional)

Clymene Gateway(Optional) (Getting Started)

dockerhub : bourbonkk/clymene-gateway:v2.x.x
redhatQuay: quay.io/clymene/clymene-gateway:v2.x.x
Docker Pulls Docker Repository on Quay
The Clymene Gateway is an optional service that can receive metric data from the another component through gRPC or HTTP communication.

  1. gRPC, HTTP Service
  2. Time-series data insert to Database(supported DB) (Optional)

Pipeline 2. Logs collection

Clymene Promtail(Getting Started)

dockerhub : bourbonkk/clymene-promtail:v2.x.x
redhatQuay: quay.io/clymene/clymene-promtail:v2.x.x
Docker Pulls Docker Repository on Quay
The Clymene-promtail customized loki's log collection agent for the Clymene project.

  1. Service Discovery
  2. log collection
  3. log data transfer to gateway(gRPC or HTTP)
  4. log data transfer to kafka
  5. log data insert to Database(supported DB) (Optional)

Promtail Ingester(Optional) (Getting Started)

dockerhub : bourbonkk/promtail-ingester:v2.x.x
redhatQuay: quay.io/clymene/promtail-ingester:v2.x.x
Docker Pulls Docker Repository on Quay
Promtail ingester is an optional service responsible for insert log data loaded on kafka into the database.

  1. Kafka message consume
  2. Time-series data insert to Database(supported DB) (Optional)

Promtail Gateway(Optional) (Getting Started)

dockerhub : bourbonkk/promtail-gateway:v2.x.x
redhatQuay: quay.io/clymene/promtail-gateway:v2.x.x
Docker Pulls Docker Repository on Quay
The Promtail Gateway is an optional service that can receive log data from the another component through gRPC or HTTP communication.

  1. gRPC, HTTP Service
  2. Time-series data insert to Database(supported DB) (Optional)

How to work it on k8s

For example, see the k8s example

How to work it on docker-compose

For example, see the docker-compose example

Roadmap

  1. AI/ML platform for clymene(https://github.com/Clymene-project/clymene-analyzer)
  2. push-type agent(node-exporter, cadvisor, process-exporter, etc)
  3. Add new pipeline(trace or Something - discussions)
  4. eBPF trace agent(for v3.0.0 release)

Get in Touch

Have questions, suggestions, bug reports? Reach the project community via these channels:

Further reading

To learn more about Clymene, consult the following talks and articles.

Recent talks and articles

License

Apache License 2.0, see LICENSE.

Code Style

supported

The Clymene project is being developed with the support of Open Source development license of Jetbrains'.