README.md

Benchmark Output Handling

┌────────────────────────────────────────────────┐ 
|                 CPE System                     |
|  ┌──────────────────────╮   ┌─────────────╮    |   ┏━━━━━━━━━━┓
|  │ benchmark_controller ├─ >┤ job_tracker |────── >┃   COS    ┃
|  └──────────────────────┘   └──────┬──────┘    |   ┗━━━━━━━━━━┛ 
|                                    V           |  
|                             ┏━━━━━━━━━━━━┓     |  
|                             ┃ CPE Parser ┃     |  
|                             ┗━━━━━━┬━━━━━┛     |  
|                                    V           |  
|                          ┌─────────────────╮   |  ┏━━━━━━━━━━━━━━━━━━━┓ 
|                          |    collector    |─────>┃ Prometheus Server ┃
|                          └─────────────────┘   |  ┗━━━━━━━━━━━━━━━━━━━┛
└────────────────────────────────────-───────────┘ 

This module is implemented in job_tracker.go

Default Mapping from Benchmark to Pod Log

Benchmark -- (.spec.operator) --> Benchmark Operator 
           -- (.spec.apiVersion,.spec.kind) --> Benchmark's Job GVK 
            -- (labeled cpe-benchmark && .status.conditions[-1].type=Complete) --> JobTracker 
             -- (job-name=[JobName]) --> Completed Pods

Custom Mapping

• Implement OperatorAdaptor abstraction operator_adaptor.go,check example from RipsawAdaptor

type OperatorAdaptor interface {
	CheckComplete(jobObject map[string]interface{}) bool
	GetPodList(jobObject map[string]interface{}, clientset *kubernetes.Clientset) (*corev1.PodList, error)
}

// operator_adaptor.go

type CustomAdaptor struct {
	*BaseOperatorAdaptor
}

func NewCustomAdaptor() *CustomAdaptor {
	custom := &CustomAdaptor{}
	abs := &BaseOperatorAdaptor{
		OperatorAdaptor: custom,
	}
	custom.BaseOperatorAdaptor = abs
	return custom
}

func (a *CustomAdaptor) CheckComplete(jobObject map[string]interface{}) bool {
	// point to status complete
}

func (a *CustomAdaptor) GetPodList(jobObject map[string]interface{}, clientset *kubernetes.Clientset) (*corev1.PodList, error) {
	// use clientset to list related pods from job object
}

var custom OperatorAdaptor = NewCustomAdaptor()

var OperatorAdaptorMap map[string]OperatorAdaptor = map[string]OperatorAdaptor{
  // append your custom adaptor
	"customKey":  custom,
}

• add .spec.adaptor in BenchmarkOperator, see cpe_v1_benchmarkoperator_helm.yaml

apiVersion: cpe.cogadvisor.io/v1
kind: BenchmarkOperator
metadata:
  name: ripsaw
  namespace: default
spec:
  apiVersion: ripsaw.cloudbulldozer.io/v1alpha1
  kind: Benchmark
  adaptor: ripsaw
  ...

Requirements for Custom Job Resource generated by Custom Operator

• update job status when complete as .status.conditions[i].type=Complete && .status.conditions[i].status=True
• label pod with job-name=[JobName]

Raw Output Collection

┌──────────────────────────────────────────────┐ 
|                      CPE System              |
|  ┌──────────────────────╮   ┌─────────────╮  |   ┏━━━━━━━━━━┓
|  │ benchmark_controller ├─ >┤ job_tracker |──── >┃   COS    ┃
|  └──────────────────────┘   └─────────────┘  |   ┗━━━━━━━━━━┛
└──────────────────────────────────────────────┘                              

1. Prepare cloud object storage bucket for raw output

2. Create secret yaml file to specify the prepared bucketname and secret key to connect to IBM Cloud Object Storage

   2.1. create a template file

    # cpe-cos-key-template.yaml
    apiVersion: v1
    kind: Secret
    metadata:
      name: cpe-cos-key
    type: Opaque
    stringData:
      rawBucketName: ${BUCKET_NAME}
      apiKey: ${APIKEY}
      serviceInstanceID: "${COS_ID}"
      authEndpoint: ${AUTH_ENDPOINT}
      serviceEndpoint: ${SERVICE_ENDPOINT}

   2.2. Update the value to cpe-cos-key.yaml with envsubst

    export BUCKET_NAME=[your bucket to store log]
    export APIKEY=[api key]
    export COS_ID=[instance ID] # crn:v1:...
    export AUTH_ENDPOINT=[authentication endpoint] # https://iam.cloud.ibm.com/identity/token
    export SERVICE_ENDPOINT=[service endpoint] # e.g., s3.jp-tok.cloud-object-storage.appdomain.cloud
    envsubst < cpe-cos-key-template.yaml > cpe-cos-key.yaml

3. Modify kustomization for default (manager) and parser by uncommenting lines of section [LOG-COS].

4. After the job is completed, log will be collected with this object key format: [benchmarkName]/[jobName]/[podName].log

Parsed Output Visualization

┌────────────────────────────────────────────────┐ 
|                 CPE System                     |
|  ┌──────────────────────╮   ┌─────────────╮    |  
|  │ benchmark_controller ├─ >┤ job_tracker |    |  
|  └──────────────────────┘   └──────┬──────┘    |  
|                                    V           |  
|                             ┏━━━━━━━━━━━━┓     |  
|                             ┃ CPE Parser ┃     |  
|                             ┗━━━━━━┬━━━━━┛     |  
|                                    V           |  
|                          ┌─────────────────╮   |  ┏━━━━━━━━━━━━━━━━━━━┓ 
|                          |    collector    |─────>┃ Prometheus Server ┃
|                          └─────────────────┘   |  ┗━━━━━━━━━━━━━━━━━━━┛
└────────────────────────────────────-───────────┘