Support for cpu_shares

[sstarcher]

It would be nice to have resources limits for cpu_shares and mem_limits.

[concaf]

It would be great to have support for mem_limit which should translate to a pod's spec like -

spec:
  containers:
  - image: nginx
    imagePullPolicy: Always
    name: nginx
    resources:
      limits:
        memory: 200Mi
    terminationMessagePath: /dev/termination-log
    volumeMounts:

@kadel imo, mem_limit can be mapped directly, not sure about cpu_shares.

[surajssd]

@containscafeine 👍

[cdrage]

I'll start working on this. I'll assign myself to this issue!

[cdrage]

Assigning to @containscafeine instead.

[concaf]

Sigh, okay, so taking a look at the mem_limit part, and having implemented it, turns out that if the docker-compose.yaml looks like -

web:
  image: tuna/docker-counter23
  ports:
    - "5000:5000"
  links:
    - redis
  mem_limit: 500m

i.e., has a non integer value, then, kompose convert fails with -

ERRO[0000] Could not parse config for project tmp : strconv.ParseInt: parsing "500m": invalid syntax 
FATA[0000] Failed to load compose file: strconv.ParseInt: parsing "500m": invalid syntax 

Tracking this down further, this is a libcompose issue and libcompose up fails with -

ERRO[0000] Failed to unmarshall: strconv.ParseInt: parsing "500m": invalid syntax
web:
  image: tuna/docker-counter23
  links:
  - redis
  mem_limit: 500m
  ports:
  - 5000:5000 
ERRO[0000] Could not parse config for project tmp : strconv.ParseInt: parsing "500m": invalid syntax 
FATA[0000] Failed to read project: strconv.ParseInt: parsing "500m": invalid syntax 

However, trying libcompose up with the latest release works fine.

So, for this to work fine, libcompose version has to be bumped, which is being tracked at #374

[kadel]

@containscafeine can you test it with #356?

[concaf]

@cdrage just tried this with #356, and mem_limit fails. This is fixed in the latest release of libcompose, can we upgrade to that?

[surajnarwade]

@cdrage @surajssd we can close this issue, as mem_limit is now supported

[cdrage]

@surajnarwade cpu_shares is still an issue. So let's leave this open.

[surajnarwade]

@cdrage oh my bad, got it

[surajssd]

@cdrage mem_limit is supported right now so edited the title

[surajssd]

@cdrage @surajnarwade we can officially declare that we don't support cpu_shares becasue there is no way we can map it to anything in k8s.

cpu_shares talks in terms of percentage of cpu to give to any container, this is feasible on a single machine but does not make sense on multi-node cluster where we dont know the entire cpu number.

TLDR: this is not easy to map to k8s cpu resources.

more info about cpu shares: https://docs.docker.com/engine/admin/resource_constraints/#cpu

[surajssd]

So the PR that closes this issue should remove all the code which tries to read this info from docker compose file and adds it to ServiceConfig and also give warning as this is unsupported.

[cdrage]

@surajssd Adding this to unsupportedkeys sounds like the best viable option considering the complexity of implementing cpu_shares. I agree.

[surajnarwade]

@surajssd @cdrage , I am onto this.

[surajnarwade]

we can close this now

[gitlawr]

Hope not to be pedantic here, but I think for cpu_shares and cpu_quota there are highly relevant concepts in Kubernetes: see.
These two keys in compose file are analogous to docker run counterpart, while in k8s requests.cpu maps to docker run --cpu-shares and limits.cpu maps to --cpu-quota(involving simple computations and pre-condition that --cpu-period=100000 though)
A user case I can come up with is that we may use daemonset or pod affinity to run one instance on every node and keep dedicated share/quota of CPU for it.
As for cpuset key I agree that it is pointless in a cluster environment as we usually don't care which cpu is used.
WDYT? @surajssd @cdrage @surajnarwade

[surajnarwade]

@gitlawr , but there is no direct mapping for cpu_shares and quotas

[gitlawr]

@surajnarwade For example,setting cpu resource such as:

apiVersion: v1
kind: Pod
metadata:
  name: one
spec:
  volumes:
  containers:
  - name: busysleep
    image: busybox
    command: ["sleep","3000"]
    resources:
      requests:
        cpu: 100m
      limits:
        cpu: 0.5

Then the container will be run with CpuShares==102(which is 0.1*1024) and CpuQuota=50000(which is half of CpuPeriod given CpuPeriod= 100000). Is it kind of direct mapping I think?