Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(config): allow percentage value for workers option #5982

40 changes: 20 additions & 20 deletions docs/config/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -654,17 +654,17 @@ export default defineConfig({

##### poolOptions.threads.maxThreads<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Maximum number of threads. You can also use `VITEST_MAX_THREADS` environment variable.
Maximum number or percentage of threads. You can also use `VITEST_MAX_THREADS` environment variable.

##### poolOptions.threads.minThreads<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Minimum number of threads. You can also use `VITEST_MIN_THREADS` environment variable.
Minimum number or percentage of threads. You can also use `VITEST_MIN_THREADS` environment variable.

##### poolOptions.threads.singleThread

Expand Down Expand Up @@ -726,17 +726,17 @@ export default defineConfig({

##### poolOptions.forks.maxForks<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Maximum number of forks.
Maximum number or percentage of forks.

##### poolOptions.forks.minForks<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Minimum number of forks.
Minimum number or percentage of forks.

##### poolOptions.forks.isolate

Expand Down Expand Up @@ -789,17 +789,17 @@ export default defineConfig({

##### poolOptions.vmThreads.maxThreads<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Maximum number of threads. You can also use `VITEST_MAX_THREADS` environment variable.
Maximum number or percentage of threads. You can also use `VITEST_MAX_THREADS` environment variable.

##### poolOptions.vmThreads.minThreads<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Minimum number of threads. You can also use `VITEST_MIN_THREADS` environment variable.
Minimum number or percentage of threads. You can also use `VITEST_MIN_THREADS` environment variable.

##### poolOptions.vmThreads.memoryLimit<NonProjectOption />

Expand Down Expand Up @@ -870,17 +870,17 @@ export default defineConfig({

##### poolOptions.vmForks.maxForks<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Maximum number of threads. You can also use `VITEST_MAX_FORKS` environment variable.
Maximum number or percentage of threads. You can also use `VITEST_MAX_FORKS` environment variable.

##### poolOptions.vmForks.minForks<NonProjectOption />

- **Type:** `number`
- **Type:** `number | string`
- **Default:** _available CPUs_

Minimum number of threads. You can also use `VITEST_MIN_FORKS` environment variable.
Minimum number or percentage of threads. You can also use `VITEST_MIN_FORKS` environment variable.

##### poolOptions.vmForks.memoryLimit<NonProjectOption />

Expand Down Expand Up @@ -914,15 +914,15 @@ This option doesn't affect tests running in the same file. If you want to run th

### maxWorkers<NonProjectOption /> {#maxworkers}

- **Type:** `number`
- **Type:** `number | string`

Maximum number of workers to run tests in. `poolOptions.{threads,vmThreads}.maxThreads`/`poolOptions.forks.maxForks` has higher priority.
Maximum number or percentage of workers to run tests in. `poolOptions.{threads,vmThreads}.maxThreads`/`poolOptions.forks.maxForks` has higher priority.

### minWorkers<NonProjectOption /> {#minworkers}

- **Type:** `number`
- **Type:** `number | string`

Minimum number of workers to run tests in. `poolOptions.{threads,vmThreads}.minThreads`/`poolOptions.forks.minForks` has higher priority.
Minimum number or percentage of workers to run tests in. `poolOptions.{threads,vmThreads}.minThreads`/`poolOptions.forks.minForks` has higher priority.

### testTimeout

Expand Down
20 changes: 10 additions & 10 deletions docs/guide/cli-table.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,27 +61,27 @@
| `--pool <pool>` | Specify pool, if not running in the browser (default: `threads`) |
| `--poolOptions.threads.isolate` | Isolate tests in threads pool (default: `true`) |
| `--poolOptions.threads.singleThread` | Run tests inside a single thread (default: `false`) |
| `--poolOptions.threads.maxThreads <workers>` | Maximum number of threads to run tests in |
| `--poolOptions.threads.minThreads <workers>` | Minimum number of threads to run tests in |
| `--poolOptions.threads.maxThreads <workers>` | Maximum number or percentage of threads to run tests in |
| `--poolOptions.threads.minThreads <workers>` | Minimum number or percentage of threads to run tests in |
| `--poolOptions.threads.useAtomics` | Use Atomics to synchronize threads. This can improve performance in some cases, but might cause segfault in older Node versions (default: `false`) |
| `--poolOptions.vmThreads.isolate` | Isolate tests in threads pool (default: `true`) |
| `--poolOptions.vmThreads.singleThread` | Run tests inside a single thread (default: `false`) |
| `--poolOptions.vmThreads.maxThreads <workers>` | Maximum number of threads to run tests in |
| `--poolOptions.vmThreads.minThreads <workers>` | Minimum number of threads to run tests in |
| `--poolOptions.vmThreads.maxThreads <workers>` | Maximum number or percentage of threads to run tests in |
| `--poolOptions.vmThreads.minThreads <workers>` | Minimum number or percentage of threads to run tests in |
| `--poolOptions.vmThreads.useAtomics` | Use Atomics to synchronize threads. This can improve performance in some cases, but might cause segfault in older Node versions (default: `false`) |
| `--poolOptions.vmThreads.memoryLimit <limit>` | Memory limit for VM threads pool. If you see memory leaks, try to tinker this value. |
| `--poolOptions.forks.isolate` | Isolate tests in forks pool (default: `true`) |
| `--poolOptions.forks.singleFork` | Run tests inside a single child_process (default: `false`) |
| `--poolOptions.forks.maxForks <workers>` | Maximum number of processes to run tests in |
| `--poolOptions.forks.minForks <workers>` | Minimum number of processes to run tests in |
| `--poolOptions.forks.maxForks <workers>` | Maximum number or percentage of processes to run tests in |
| `--poolOptions.forks.minForks <workers>` | Minimum number or percentage of processes to run tests in |
| `--poolOptions.vmForks.isolate` | Isolate tests in forks pool (default: `true`) |
| `--poolOptions.vmForks.singleFork` | Run tests inside a single child_process (default: `false`) |
| `--poolOptions.vmForks.maxForks <workers>` | Maximum number of processes to run tests in |
| `--poolOptions.vmForks.minForks <workers>` | Minimum number of processes to run tests in |
| `--poolOptions.vmForks.maxForks <workers>` | Maximum number or percentage of processes to run tests in |
| `--poolOptions.vmForks.minForks <workers>` | Minimum number or percentage of processes to run tests in |
| `--poolOptions.vmForks.memoryLimit <limit>` | Memory limit for VM forks pool. If you see memory leaks, try to tinker this value. |
| `--fileParallelism` | Should all test files run in parallel. Use `--no-file-parallelism` to disable (default: `true`) |
| `--maxWorkers <workers>` | Maximum number of workers to run tests in |
| `--minWorkers <workers>` | Minimum number of workers to run tests in |
| `--maxWorkers <workers>` | Maximum number or percentage of workers to run tests in |
| `--minWorkers <workers>` | Minimum number or percentage of workers to run tests in |
| `--environment <name>` | Specify runner environment, if not running in the browser (default: `node`) |
| `--passWithNoTests` | Pass when no tests are found |
| `--logHeapUsage` | Show the size of heap for each test when running in node |
Expand Down
1 change: 1 addition & 0 deletions packages/vitest/rollup.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ const external = [
'worker_threads',
'node:worker_threads',
'node:fs',
'node:os',
'node:stream',
'node:vm',
'inspector',
Expand Down
8 changes: 4 additions & 4 deletions packages/vitest/src/node/cli/cli-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,11 @@ const poolThreadsCommands: CLIOptions<ThreadsOptions & WorkerContextOptions> = {
description: 'Run tests inside a single thread (default: `false`)',
},
maxThreads: {
description: 'Maximum number of threads to run tests in',
description: 'Maximum number or percentage of threads to run tests in',
argument: '<workers>',
},
minThreads: {
description: 'Minimum number of threads to run tests in',
description: 'Minimum number or percentage of threads to run tests in',
argument: '<workers>',
},
useAtomics: {
Expand All @@ -84,11 +84,11 @@ const poolForksCommands: CLIOptions<ForksOptions & WorkerContextOptions> = {
description: 'Run tests inside a single child_process (default: `false`)',
},
maxForks: {
description: 'Maximum number of processes to run tests in',
description: 'Maximum number or percentage of processes to run tests in',
argument: '<workers>',
},
minForks: {
description: 'Minimum number of processes to run tests in',
description: 'Minimum number or percentage of processes to run tests in',
argument: '<workers>',
},
execArgv: null,
Expand Down
58 changes: 55 additions & 3 deletions packages/vitest/src/node/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ import {
} from '../constants'
import { benchmarkConfigDefaults, configDefaults } from '../defaults'
import { isCI, stdProvider, toArray } from '../utils'
import type { BuiltinPool } from '../types/pool-options'
import type { BuiltinPool, ForksOptions, PoolOptions, ThreadsOptions } from '../types/pool-options'
import { getWorkersCountByPercentage } from '../utils/workers'
import { VitestCache } from './cache'
import { BaseSequencer } from './sequencers/BaseSequencer'
import { RandomSequencer } from './sequencers/RandomSequencer'
Expand Down Expand Up @@ -97,6 +98,15 @@ export function resolveApiServerConfig<Options extends ApiConfig & UserConfig>(
return api
}

function resolveInlineWorkerOption(value: string | number): number {
if (typeof value === 'string' && value.trim().endsWith('%')) {
return getWorkersCountByPercentage(value)
}
else {
return Number(value)
}
}

export function resolveConfig(
mode: VitestRunMode,
options: UserConfig,
Expand Down Expand Up @@ -176,11 +186,11 @@ export function resolveConfig(
}

if (resolved.maxWorkers) {
resolved.maxWorkers = Number(resolved.maxWorkers)
resolved.maxWorkers = resolveInlineWorkerOption(resolved.maxWorkers)
}

if (resolved.minWorkers) {
resolved.minWorkers = Number(resolved.minWorkers)
resolved.minWorkers = resolveInlineWorkerOption(resolved.minWorkers)
}

resolved.browser ??= {} as any
Expand Down Expand Up @@ -436,6 +446,48 @@ export function resolveConfig(
}
}

const poolThreadsOptions = [
['threads', 'minThreads'],
['threads', 'maxThreads'],
['vmThreads', 'minThreads'],
['vmThreads', 'maxThreads'],
] as const satisfies [keyof PoolOptions, keyof ThreadsOptions][]

for (const [poolOptionKey, workerOptionKey] of poolThreadsOptions) {
const workerInlineOption = resolved.poolOptions?.[poolOptionKey]?.[workerOptionKey]

if (workerInlineOption) {
resolved.poolOptions = {
...resolved.poolOptions,
sheremet-va marked this conversation as resolved.
Show resolved Hide resolved
[poolOptionKey]: {
...resolved.poolOptions?.[poolOptionKey],
[workerOptionKey]: resolveInlineWorkerOption(workerInlineOption),
},
}
}
}

const poolForksOptions = [
['forks', 'minForks'],
['forks', 'maxForks'],
['vmForks', 'minForks'],
['vmForks', 'maxForks'],
] as const satisfies [keyof PoolOptions, keyof ForksOptions][]

for (const [poolOptionKey, workerOptionKey] of poolForksOptions) {
const workerInlineOption = resolved.poolOptions?.[poolOptionKey]?.[workerOptionKey]

if (workerInlineOption) {
resolved.poolOptions = {
...resolved.poolOptions,
[poolOptionKey]: {
...resolved.poolOptions?.[poolOptionKey],
[workerOptionKey]: resolveInlineWorkerOption(workerInlineOption),
},
}
}
}

if (resolved.workspace) {
// if passed down from the CLI and it's relative, resolve relative to CWD
resolved.workspace
Expand Down
15 changes: 9 additions & 6 deletions packages/vitest/src/types/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import type { SnapshotStateOptions } from './snapshot'
import type { Arrayable, ParsedStack } from './general'
import type { BenchmarkUserOptions } from './benchmark'
import type { BrowserConfigOptions, ResolvedBrowserOptions } from './browser'
import type { Pool, PoolOptions } from './pool-options'
import type { Pool, PoolOptions, ResolvedPoolOptions } from './pool-options'

export type { BrowserScript, BrowserConfigOptions } from './browser'
export type { SequenceHooks, SequenceSetupFiles } from '@vitest/runner'
Expand Down Expand Up @@ -343,13 +343,13 @@ export interface InlineConfig {
poolOptions?: PoolOptions

/**
* Maximum number of workers to run tests in. `poolOptions.{threads,vmThreads}.maxThreads`/`poolOptions.forks.maxForks` has higher priority.
* Maximum number or percentage of workers to run tests in. `poolOptions.{threads,vmThreads}.maxThreads`/`poolOptions.forks.maxForks` has higher priority.
*/
maxWorkers?: number
maxWorkers?: number | string
/**
* Minimum number of workers to run tests in. `poolOptions.{threads,vmThreads}.minThreads`/`poolOptions.forks.minForks` has higher priority.
* Minimum number or percentage of workers to run tests in. `poolOptions.{threads,vmThreads}.minThreads`/`poolOptions.forks.minForks` has higher priority.
*/
minWorkers?: number
minWorkers?: number | string

/**
* Should all test files run in parallel. Doesn't affect tests running in the same file.
Expand Down Expand Up @@ -969,7 +969,7 @@ export interface ResolvedConfig

browser: ResolvedBrowserOptions
pool: Pool
poolOptions?: PoolOptions
poolOptions?: ResolvedPoolOptions

reporters: (InlineReporter | ReporterWithOptions)[]

Expand Down Expand Up @@ -1009,6 +1009,9 @@ export interface ResolvedConfig
enabled: boolean
}
runner?: string

maxWorkers: number
minWorkers: number
}

export type ProjectConfig = Omit<
Expand Down
25 changes: 21 additions & 4 deletions packages/vitest/src/types/pool-options.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,19 @@ export interface PoolOptions extends Record<string, unknown> {
vmForks?: ForksOptions & VmOptions
}

export interface ResolvedPoolOptions extends PoolOptions {
threads?: ResolvedThreadsOptions & WorkerContextOptions
forks?: ResolvedForksOptions & WorkerContextOptions
vmThreads?: ResolvedThreadsOptions & VmOptions
vmForks?: ResolvedForksOptions & VmOptions
}

export interface ThreadsOptions {
/** Minimum amount of threads to use */
minThreads?: number
minThreads?: number | string

/** Maximum amount of threads to use */
maxThreads?: number
maxThreads?: number | string

/**
* Run tests inside a single thread.
Expand All @@ -66,12 +73,17 @@ export interface ThreadsOptions {
useAtomics?: boolean
}

export interface ResolvedThreadsOptions extends ThreadsOptions {
minThreads?: number
maxThreads?: number
}

export interface ForksOptions {
/** Minimum amount of child processes to use */
minForks?: number
minForks?: number | string

/** Maximum amount of child processes to use */
maxForks?: number
maxForks?: number | string

/**
* Run tests inside a single fork.
Expand All @@ -81,6 +93,11 @@ export interface ForksOptions {
singleFork?: boolean
}

export interface ResolvedForksOptions extends ForksOptions {
minForks?: number
maxForks?: number
}

export interface WorkerContextOptions {
/**
* Isolate test environment by recycling `worker_threads` or `child_process` after each test
Expand Down
8 changes: 8 additions & 0 deletions packages/vitest/src/utils/workers.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import { cpus } from 'node:os'

export function getWorkersCountByPercentage(percent: string) {
const maxWorkersCount = cpus().length
sheremet-va marked this conversation as resolved.
Show resolved Hide resolved
const workersCountByPercentage = Math.round((Number.parseInt(percent) / 100) * maxWorkersCount)

return Math.max(1, Math.min(maxWorkersCount, workersCountByPercentage))
}
5 changes: 5 additions & 0 deletions test/config/fixtures/workers-option/example.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { expect, test } from 'vitest'

test('it works', () => {
expect(true).toBe(true)
})
1 change: 1 addition & 0 deletions test/config/fixtures/workers-option/vitest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export default {}
Loading
Loading