feat: add a function to get number of logical cpu cores

[Lloyd-Pottiger]

Feature Request

Is your feature request related to a problem? Please describe:

Now we can only get number of physical cpu cores by dbms/src/Common/getNumberOfPhysicalCPUCores.cpp.

1. getNumberOfPhysicalCPUCores() will return number of logical cpu cores in some cases which is misleading.
2. In most cases, number of logical cpu cores matters rather than physical cpu cores especially in cloud env.

Describe the feature you'd like:

Add a function to get number of logical cpu cores which support different env like docker/k8s, cgroup subsystem.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

[fuzhe1989]

Does std::thread::hardware_concurrency() fit your demand?

[Lloyd-Pottiger]

Does std::thread::hardware_concurrency() fit your demand?

Normally, it fits well. I am still investigating whether it works in docker and subsystem.

[Lloyd-Pottiger]

#include <iostream>
#include <thread>

using namespace std;

int main()
{
    cout << thread::hardware_concurrency() << endl;
}

test by

docker run -it --rm --cpus=2 docker-test /bin/bash
clang++ main.cc -o main && ./main
# output: 40

[Lloyd-Pottiger]

Get Number of Logical CPU Cores

Ways to limit cpu usage by using cgroups

1. cpuset subsystem

2. cpu subsystem

cgroup v1

Linux manual page: Over time, various cgroup controllers have been added to allow the management of various types of resources. However, the development of these controllers was largely uncoordinated, with the result that many inconsistencies arose between controllers and management of the cgroup hierarchies became rather complex.

Firstly, we can obtaining information about control groups by:

$ cat /proc/self/cgroup
11:cpuset:/
10:hugetlb:/
9:blkio:/user.slice
8:memory:/user.slice
7:freezer:/
6:net_prio,net_cls:/
5:devices:/user.slice
4:pids:/user.slice
3:cpuacct,cpu:/user.slice
2:perf_event:/
1:name=systemd:/user.slice/user-1013.slice/session-32587.scope

The cpuset subsystem can allocate independent CPU and memory nodes to cgroups. cpuset.cpus set the CPU that the cgroup task can access.

We can obtain the information of cpuset is in $group_name group, by

$ cat /sys/fs/cgroup/cpuset/{$group_name}/cpuset.cpus
0-2,16

which means CPU 0/1/2/16 are available.

The cpu subsystem can schedule CPU acquisition by cgroup. The following two schedulers are available to manage the acquisition of CPU resources:

1. CFS

2. RT

we only consider CFS here. cpu.cfs_period_us can set the interval at which available CPU resources are reallocated to the cgroup in microseconds(us). cpu.cfs_quota_us can be set to the total amount of time that all tasks in a cgroup can run during a period(equal tocpu.cfs_period_us). cpu.shares use an integer to set the relative proportion of CPU time available for tasks in the cgroup. cpu.shares only limit the percent of cpus that can use when cpus are busy, and there are no limitation when cpus are free. Theoretically, we still can use all the cpus when there are no other processes with share limitation. Therefore, we can ignore cpu.shares limitation.

We can obtain the information of cpu is in $group_name group, by

$ cat /sys/fs/cgroup/cpu/{$group_name}/cpu.cfs_period_us
> 100000
$ cat /sys/fs/cgroup/cpu/{$group_name}/cpu.cfs_quota_us
> 200000

which means cgroup {$group_name} can use two full cpus.

cgroup v2

design documentation of v2: [Control Group v2 — The Linux Kernel documentation](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html)

v2 simplifies v1. For cpuset subsystem, we can obtain the information of cpuset is in $group_name group, by

$ cat /sys/fs/cgroup/{$group_name`}/cpuset.cpus
0-1,6,8-10

which means CPU 0/1/6/8/9/10 are available.

For cpu subsystem, we can obtain the information of cpu is in $group_name group, by

$ cat /sys/fs/cgroup/{$group_name`}/cpu.max
$MAX $PERIOD

which indicates that the group may consume upto $MAX in each $PERIOD duration. "max" for $MAX indicates no limit. If only one number is written, $MAX is updated.

There are other limitations like cpu.weight and cpu.weight.nice which is not important and rarely used, we just ignore.

Docker

Docker use cgroup to implement resource isolation. There are three ways to limit CPU usage of a container.

1. --cpus limit the number of CPU can be used.

2. --cpuset-cpus set the specified CPU can be used.

3. --cpu-shares set the CPU weight.

Essentially, --cpus and --cpu-shares use the cgroup cpu subsystem, and --cpuset-cpus uses the cgroup cpuset subsystem.

In docker

$ cat /proc/self/cgroup 
11:cpuset:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
10:hugetlb:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
9:blkio:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
8:memory:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
7:freezer:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
6:net_prio,net_cls:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
5:devices:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
4:pids:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
3:cpuacct,cpu:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
2:perf_event:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740
1:name=systemd:/docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740

which seems we are under a cgroup named /docker/5aeb23b2170fd1681448df3729213f4727507b0c49760bd325d885cda1ab4740

$ ls /sys/fs/cgroup/cpu
cgroup.clone_children  cgroup.procs       cpu.cfs_quota_us  cpu.rt_runtime_us  cpu.stat      cpuacct.usage         notify_on_release
cgroup.event_control   cpu.cfs_period_us  cpu.rt_period_us  cpu.shares         cpuacct.stat  cpuacct.usage_percpu  tasks

However, there are no cgroup under subsytems. The /docker/xxx is just a hint that the process is running in the docker. We can just obtain the information from root cgroup.

$ docker run -it --rm --cpus=21 tiflash-llvm-base:amd64 /bin/bash
# cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us
2100000
# cat /sys/fs/cgroup/cpu/cpu.cfs_period_us
100000

$ docker run -it --rm --cpuset-cpus="0-1,4-6" tiflash-llvm-base:amd64 /bin/bash
# cat /sys/fs/cgroup/cpuset/cpuset.cpus
0-1,4-6