Skip to content

Commit

Permalink
benchmark: add a testcase for creating a pod and a container
Browse files Browse the repository at this point in the history 
Add a testcase for measuring the time of creating a pod
and container.

Signed-off-by: Fupan Li <lifupan@gmail.com>
  • Loading branch information
@lifupan
lifupan committed Dec 21, 2018
1 parent 5f10256 commit 9381fbd
Show file tree
Hide file tree
Showing 2 changed files with 280 additions and 0 deletions.
206 changes: 206 additions & 0 deletions pkg/benchmark/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
# pod and container lifecycle benchmark for kata

This repocitory is mainly focused on the pod and container's lifecircle benchmark for kata. The way to run a kata container is
used cri + containerd + containerd-shim-kata-v2.

## Install

### Install kata components
```sh
ARCH=$(arch)
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/home:/katacontainers:/releases:/${ARCH}:/master/xUbuntu_$(lsb_release -rs)/ /' > /etc/apt/sources.list.d/kata-containers.list"
curl -sL http://download.opensuse.org/repositories/home:/katacontainers:/releases:/${ARCH}:/master/xUbuntu_$(lsb_release -rs)/Release.key | sudo apt-key add -
sudo -E apt-get update
sudo -E apt-get -y install kata-runtime kata-proxy kata-shim
```
### Install kata shimv2
```sh
go get github.com/kata-containers/runtime
pushd $GOPATH/src/github.com/kata-containers/runtime
git remote add hyper https://github.com/hyperhq/kata-runtime
git fetch hyper
git checkout -b benchmark hyper/benchmark
make
sudo -E PATH=$PATH make install
popd

sed -i 's/image =/#image =/' /usr/share/defaults/kata-containers/configuration.toml
```
### Install critest

```sh
go get https://github.com/lifupan/cri-tools.git
pushd $GOPATH/src/github.com/lifupan/cri-tools
git checkout kata_benchmark
make;make install
```

### Install containerd
```sh
sudo apt-get update
sudo apt-get install libseccomp-dev btrfs-tools git -y
go get github.com/containerd/containerd
pushd $GOPATH/src/github.com/containerd/containerd
make
sudo -E PATH=$PATH make install
popd
```

### Install cni plugin
```sh
go get github.com/containernetworking/plugins
pushd $GOPATH/src/github.com/containernetworking/plugins
./build_linux.sh
mkdir /opt/cni
cp -r bin /opt/cni/
popd
```

## Setup cni plugin
```sh
sudo mkdir -p /etc/cni/net.d
cat >/etc/cni/net.d/10-mynet.conf <<EOF
{
"cniVersion": "0.2.0",
"name": "mynet",
"type": "bridge",
"bridge": "cni0",
"isGateway": true,
"ipMasq": true,
"ipam": {
"type": "host-local",
"subnet": "172.19.0.0/24",
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
}
EOF
```

## Setup containerd
### create containerd configure file
```sh
sudo mkdir -p /etc/containerd/
cat > /etc/containerd/config.toml <<EOF
[plugins]
[plugins.cri]
sandbox_image = "mirrorgooglecontainers/pause-amd64:3.1"
[plugins.cri.containerd]
[plugins.cri.containerd.default_runtime]
runtime_type = "io.containerd.kata.v2"
[plugins.cri.cni]
# conf_dir is the directory in which the admin places a CNI conf.
conf_dir = "/etc/cni/net.d"
EOF
```

### create containerd systemd service
```sh
sudo cat > /etc/systemd/system/containerd.service <<EOF
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target
[Service]
ExecStartPre=/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerd --config /etc/containerd/config.toml
Restart=always
RestartSec=5
Delegate=yes
KillMode=process
OOMScoreAdjust=-999
LimitNOFILE=1048576
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
[Install]
WantedBy=multi-user.target
EOF
```
### start containerd
```sh
systemctl enable containerd
systemctl start containerd
```

## Run benchmark test
```sh
critest -benchmark --runtime-endpoint /run/containerd/containerd.sock
```
Then you will get some benchmark outputs as below on the screen
```sh
• [MEASUREMENT]
[k8s.io] PodSandbox
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/framework/framework.go:72
benchmark about operations on PodSandbox
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/benchmark/pod.go:41
benchmark about lifecycle of PodSandbox
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/benchmark/pod.go:42

Ran 20 samples:
create PodSandbox:
Fastest Time: 2.234s
Slowest Time: 2.625s
Average Time: 2.499s ± 0.083s
PodSandbox status:
Fastest Time: 0.000s
Slowest Time: 0.002s
Average Time: 0.001s ± 0.000s
stop PodSandbox:
Fastest Time: 0.287s
Slowest Time: 0.355s
Average Time: 0.321s ± 0.019s
remove PodSandbox:
Fastest Time: 0.008s
Slowest Time: 0.015s
Average Time: 0.009s ± 0.002s

• [MEASUREMENT]
[k8s.io] Container
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/framework/framework.go:72
benchmark about operations on Container
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/benchmark/container.go:39
benchmark about basic operations on Container
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/benchmark/container.go:54

Ran 20 samples:
create Container:
Fastest Time: 0.036s
Slowest Time: 0.070s
Average Time: 0.053s ± 0.010s
start Container:
Fastest Time: 0.144s
Slowest Time: 0.222s
Average Time: 0.165s ± 0.016s
Container status:
Fastest Time: 0.001s
Slowest Time: 0.005s
Average Time: 0.001s ± 0.001s
stop Container:
Fastest Time: 0.149s
Slowest Time: 0.220s
Average Time: 0.185s ± 0.019s
remove Container:
Fastest Time: 0.007s
Slowest Time: 0.014s
Average Time: 0.008s ± 0.001s

• [MEASUREMENT]
[k8s.io] PodSandbox
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/framework/framework.go:72
benchmark about start a container from scratch
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/benchmark/pod_container.go:47
benchmark about start a container from scratch
/root/gopath/src/github.com/kubernetes-sigs/cri-tools/pkg/benchmark/pod_container.go:48

Ran 20 samples:
create PodSandbox and container:
Fastest Time: 2.695s
Slowest Time: 2.963s
Average Time: 2.841s ± 0.067s

```
74 changes: 74 additions & 0 deletions pkg/benchmark/pod_container.go
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
/*
Copyright 2018 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package benchmark

import (
"github.com/kubernetes-sigs/cri-tools/pkg/framework"
internalapi "k8s.io/kubernetes/pkg/kubelet/apis/cri"
runtimeapi "k8s.io/kubernetes/pkg/kubelet/apis/cri/runtime/v1alpha2"

. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
)

var _ = framework.KubeDescribe("PodSandbox", func() {
f := framework.NewDefaultCRIFramework()

var rc internalapi.RuntimeService
var ic internalapi.ImageManagerService
var podID string

BeforeEach(func() {
rc = f.CRIClient.CRIRuntimeClient
ic = f.CRIClient.CRIImageClient
})

AfterEach(func() {
By("stop PodSandbox")
rc.StopPodSandbox(podID)
By("delete PodSandbox")
rc.RemovePodSandbox(podID)
})

Context("benchmark about start a container from scratch", func() {
Measure("benchmark about start a container from scratch", func(b Benchmarker) {
var err error

podSandboxName := "PodSandbox-for-creating-pod-and-container-performance-test-" + framework.NewUUID()
uid := framework.DefaultUIDPrefix + framework.NewUUID()
namespace := framework.DefaultNamespacePrefix + framework.NewUUID()

config := &runtimeapi.PodSandboxConfig{
Metadata: framework.BuildPodSandboxMetadata(podSandboxName, uid, namespace, framework.DefaultAttempt),
Linux: &runtimeapi.LinuxPodSandboxConfig{},
}

operation := b.Time("create PodSandbox and container", func() {
By("run PodSandbox")
podID, err = rc.RunPodSandbox(config, framework.TestContext.RuntimeHandler)
framework.ExpectNoError(err, "failed to create PodSandbox: %v", err)
By("create container in PodSandbox")
containerID := framework.CreateDefaultContainer(rc, ic, podID, config, "Pod-Container-for-creating-benchmark-")
By("start container in PodSandbox")
err = rc.StartContainer(containerID)
})

framework.ExpectNoError(err, "failed to start Container: %v", err)
Expect(operation.Seconds()).Should(BeNumerically("<", 5), "create PodSandbox shouldn't take too long.")
}, defaultOperationTimes)
})
})

0 comments on commit 9381fbd

Please sign in to comment.