Skip to content

Commit

Permalink
Improve Batch documentation (#5700)
Browse files Browse the repository at this point in the history 
- Improved application package references documentation.
  - Improved max tasks per node documentation.
  - Improved spread vs pack documentation.
  - Improved documentation about task idempotency.
  • Loading branch information
@matthchr @sarangan12
matthchr authored and sarangan12 committed Apr 22, 2019
1 parent a6dff7a commit 721e63f
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 18 deletions.
35 changes: 21 additions & 14 deletions specification/batch/data-plane/Microsoft.Batch/stable/2018-12-01.8.0/BatchService.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11715,7 +11715,7 @@
},
"required": [ "commandLine" ],
"title": "A task which is run when a compute node joins a pool in the Azure Batch service, or when the compute node is rebooted or reimaged.",
"description": "Batch will retry tasks when a recovery operation is triggered on a compute node. Examples of recovery operations include (but are not limited to) when an unhealthy compute node is rebooted or a compute node disappeared due to host failure. Retries due to recovery operations are independent of and are not counted against the maxTaskRetryCount. Even if the maxTaskRetryCount is 0, an internal retry due to a recovery operation may occur. Because of this, all tasks should be idempotent. This means tasks need to tolerate being interrupted and restarted without causing any corruption or duplicate data. The best practice for long running tasks is to use some form of checkpointing."
"description": "Batch will retry tasks when a recovery operation is triggered on a compute node. Examples of recovery operations include (but are not limited to) when an unhealthy compute node is rebooted or a compute node disappeared due to host failure. Retries due to recovery operations are independent of and are not counted against the maxTaskRetryCount. Even if the maxTaskRetryCount is 0, an internal retry due to a recovery operation may occur. Because of this, all tasks should be idempotent. This means tasks need to tolerate being interrupted and restarted without causing any corruption or duplicate data. The best practice for long running tasks is to use some form of checkpointing. In some cases the start task may be re-run even though the node was not rebooted. Special care should be taken to avoid start tasks which create breakaway process or install/launch services from the start task working directory, as this will block Batch from being able to re-run the start task."
},
"CertificateReference": {
"properties": {
Expand Down Expand Up @@ -11836,11 +11836,12 @@
"type": "integer",
"format": "int32",
"title": "The maximum number of tasks that can run concurrently on a single compute node in the pool.",
"description": "The default value is 1. The maximum value of this setting depends on the size of the compute nodes in the pool (the vmSize setting)."
"description": "The default value is 1. The maximum value is the smaller of 4 times the number of cores of the vmSize of the pool or 256."
},
"taskSchedulingPolicy": {
"$ref": "#/definitions/TaskSchedulingPolicy",
"title": "How tasks are distributed across compute nodes in a pool."
"title": "How tasks are distributed across compute nodes in a pool.",
"description": "If not specified, the default is spread."
},
"resizeTimeout": {
"type": "string",
Expand Down Expand Up @@ -11902,7 +11903,8 @@
"items": {
"$ref": "#/definitions/ApplicationPackageReference"
},
"title": "The list of application packages to be installed on each compute node in the pool."
"title": "The list of application packages to be installed on each compute node in the pool.",
"description": "Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. There is a maximum of 10 application package references on any given pool."
},
"applicationLicenses": {
"type": "array",
Expand Down Expand Up @@ -13367,7 +13369,8 @@
"items": {
"$ref": "#/definitions/ApplicationPackageReference"
},
"title": "The list of application packages to be installed on each compute node in the pool."
"title": "The list of application packages to be installed on each compute node in the pool.",
"description": "Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. There is a maximum of 10 application package references on any given pool."
},
"applicationLicenses": {
"type": "array",
Expand All @@ -13380,11 +13383,13 @@
"maxTasksPerNode": {
"type": "integer",
"format": "int32",
"title": "The maximum number of tasks that can run concurrently on a single compute node in the pool."
"title": "The maximum number of tasks that can run concurrently on a single compute node in the pool.",
"description": "The default value is 1. The maximum value is the smaller of 4 times the number of cores of the vmSize of the pool or 256."
},
"taskSchedulingPolicy": {
"$ref": "#/definitions/TaskSchedulingPolicy",
"title": "How tasks are distributed across compute nodes in a pool."
"title": "How tasks are distributed across compute nodes in a pool.",
"description": "If not specified, the default is spread."
},
"userAccounts": {
"type": "array",
Expand Down Expand Up @@ -13496,7 +13501,8 @@
"items": {
"$ref": "#/definitions/ApplicationPackageReference"
},
"title": "The list of application packages to be installed on each compute node in the pool."
"title": "The list of application packages to be installed on each compute node in the pool.",
"description": "Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. There is a maximum of 10 application package references on any given pool."
},
"applicationLicenses": {
"type": "array",
Expand All @@ -13510,11 +13516,12 @@
"type": "integer",
"format": "int32",
"title": "The maximum number of tasks that can run concurrently on a single compute node in the pool.",
"description": "The default value is 1. The maximum value of this setting depends on the size of the compute nodes in the pool (the vmSize setting)."
"description": "The default value is 1. The maximum value is the smaller of 4 times the number of cores of the vmSize of the pool or 256."
},
"taskSchedulingPolicy": {
"$ref": "#/definitions/TaskSchedulingPolicy",
"title": "How tasks are distributed across compute nodes in a pool."
"title": "How tasks are distributed across compute nodes in a pool.",
"description": "If not specified, the default is spread."
},
"userAccounts": {
"type": "array",
Expand Down Expand Up @@ -14982,8 +14989,8 @@
"items": {
"$ref": "#/definitions/ApplicationPackageReference"
},
"title": "A list of application packages to be installed on each compute node in the pool.",
"description": "The list replaces any existing application package references on the pool. Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. If omitted, or if you specify an empty collection, any existing application packages references are removed from the pool."
"title": "The list of application packages to be installed on each compute node in the pool.",
"description": "The list replaces any existing application package references on the pool. Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. There is a maximum of 10 application package references on any given pool. If omitted, or if you specify an empty collection, any existing application packages references are removed from the pool."
},
"metadata": {
"type": "array",
Expand Down Expand Up @@ -15017,8 +15024,8 @@
"items": {
"$ref": "#/definitions/ApplicationPackageReference"
},
"title": "A list of application packages to be installed on each compute node in the pool.",
"description": "Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. If this element is present, it replaces any existing application package references. If you specify an empty collection, then all application package references are removed from the pool. If omitted, any existing application package references are left unchanged."
"title": "The list of application packages to be installed on each compute node in the pool.",
"description": "The list replaces any existing application package references on the pool. Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. There is a maximum of 10 application package references on any given pool. If omitted, any existing application package references are left unchanged."
},
"metadata": {
"type": "array",
Expand Down
11 changes: 7 additions & 4 deletions specification/batch/resource-manager/Microsoft.Batch/stable/2019-04-01/BatchManagement.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2772,11 +2772,13 @@
"maxTasksPerNode": {
"type": "integer",
"format": "int32",
"title": "The maximum number of tasks that can run concurrently on a single compute node in the pool."
"title": "The maximum number of tasks that can run concurrently on a single compute node in the pool.",
"description": "The default value is 1. The maximum value is the smaller of 4 times the number of cores of the vmSize of the pool or 256."
},
"taskSchedulingPolicy": {
"$ref": "#/definitions/TaskSchedulingPolicy",
"title": "How tasks are distributed across compute nodes in a pool."
"title": "How tasks are distributed across compute nodes in a pool.",
"description": "If not specified, the default is spread."
},
"userAccounts": {
"type": "array",
Expand Down Expand Up @@ -2812,7 +2814,7 @@
"$ref": "#/definitions/ApplicationPackageReference"
},
"title": "The list of application packages to be installed on each compute node in the pool.",
"description": "Changes to application packages affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged."
"description": "Changes to application package references affect all new compute nodes joining the pool, but do not affect compute nodes that are already in the pool until they are rebooted or reimaged. There is a maximum of 10 application package references on any given pool."
},
"applicationLicenses": {
"type": "array",
Expand Down Expand Up @@ -3404,7 +3406,8 @@
"description": "When this is specified, all directories recursively below the AZ_BATCH_NODE_ROOT_DIR (the root of Azure Batch directories on the node) are mapped into the container, all task environment variables are mapped into the container, and the task command line is executed in the container."
}
},
"title": "A task which is run when a compute node joins a pool in the Azure Batch service, or when the compute node is rebooted or reimaged."
"title": "A task which is run when a compute node joins a pool in the Azure Batch service, or when the compute node is rebooted or reimaged.",
"description": "In some cases the start task may be re-run even though the node was not rebooted. Due to this, start tasks should be idempotent and exit gracefully if the setup they're performing has already been done. Special care should be taken to avoid start tasks which create breakaway process or install/launch services from the start task working directory, as this will block Batch from being able to re-run the start task."
},
"TaskContainerSettings": {
"properties": {
Expand Down

0 comments on commit 721e63f

Please sign in to comment.