Skip to content

Commit

Permalink
Corrected links and titles (#248)
Browse files Browse the repository at this point in the history
* Corrected links and titles

* Replaced job scheduler to manager
  • Loading branch information
satr authored Oct 5, 2023
1 parent ee592f7 commit b398b65
Show file tree
Hide file tree
Showing 6 changed files with 33 additions and 30 deletions.
15 changes: 8 additions & 7 deletions public-site/docs/src/guides/jobs/configure-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ prev: .
next: job-manager-and-job-api
---

# Configuring Jobs
## Job configuration

Jobs are configured in [`radixconfig.yaml`](../../references/reference-radix-config/#jobs), similar to how components are configured.

Expand Down Expand Up @@ -44,6 +44,7 @@ spec:
gpuCount: 2
```
## Options
They share many of the same configuration options with a few exceptions.
A job does not have `publicPort`, `ingressConfiguration`, `replicas`, `horizontalScaling` and `alwaysPullImageOnDeploy`
Expand Down Expand Up @@ -80,28 +81,28 @@ The compute job in the example above has `payload.path` set to `/compute/args`.

### resources

The resource requirement for a job can be sent in the request body to the job scheduler as a JSON document with an element named `resources`.
The resource requirement for a job can be sent in the request body to the job manager as a JSON document with an element named `resources`.
The content of the resources will be used to set the resource definition for the job [`radixconfig.yaml`](../../references/reference-radix-config/#resources-common).
The data type of the `resources` is of type `ResourceRequirements` an requires this specific format.

The etl job in the example above has `resource` configured.

### node

The node requirerement for a job can be sent in the request body to the job scheduler as a JSON document with an element named `node`.
The node requirerement for a job can be sent in the request body to the job manager as a JSON document with an element named `node`.
The content of the node will be used to set the node definition for the job [`radixconfig.yaml`](../../references/reference-radix-config/#node).
The data type of the `node` is of type `RadixNode` an requires this specific format.

The etl job in the example above has `node` configured.

### timeLimitSeconds

The maximum running time for a job can be sent in the request body to the job scheduler as a JSON document with an element named `timeLimitSeconds`.
The maximum running time for a job can be sent in the request body to the job manager as a JSON document with an element named `timeLimitSeconds`.

The etl job in the example above has `timeLimitSeconds` configured in its [`radixconfig.yaml`](../../references/reference-radix-config/#timelimitseconds). If a new job is sent to the job scheduler without an element `timeLimitSeconds`, it will default to the value specified in radixconfig.yaml. If no value is specified in radixconfig.yaml, it will default to 43200 (12 hours).
The etl job in the example above has `timeLimitSeconds` configured in its [`radixconfig.yaml`](../../references/reference-radix-config/#timelimitseconds). If a new job is sent to the job manager without an element `timeLimitSeconds`, it will default to the value specified in radixconfig.yaml. If no value is specified in radixconfig.yaml, it will default to 43200 (12 hours).

### backoffLimit

The maximum number of restarts if the job fails can be sent in the request body to the job scheduler as a JSON document with an element named `backoffLimit`.
The maximum number of restarts if the job fails can be sent in the request body to the job manager as a JSON document with an element named `backoffLimit`.

The etl job in the example above has `backoffLimit` configured in its [`radixconfig.yaml`](../../references/reference-radix-config/#backofflimit). If a new job is sent to the job scheduler without an element `backoffLimit`, it will default to the value specified in radixconfig.yaml.
The etl job in the example above has `backoffLimit` configured in its [`radixconfig.yaml`](../../references/reference-radix-config/#backofflimit). If a new job is sent to the job manager without an element `backoffLimit`, it will default to the value specified in radixconfig.yaml.
24 changes: 12 additions & 12 deletions public-site/docs/src/guides/jobs/job-manager-and-job-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The Job Manager exposes the following methods for managing jobs:
- `POST /api/v1/batches/{batchName}/stop` Stop a named batch
- `POST /api/v1/batches/{batchName}/jobs/{jobName}/stop` Stop a named job of a batch

### Create a single job
## Create a single job

- `POST /api/v1/jobs` Create a new job using the Docker image that Radix built for the job. Job-specific arguments can be sent in the request body

Expand Down Expand Up @@ -54,7 +54,7 @@ The Job Manager exposes the following methods for managing jobs:

`imageTagName` field allows to alter specific job image tag. In order to use it, the `{imageTagName}` need to be set as described in the [`radixconfig.yaml`](../../references/reference-radix-config/#imagetagname-2)

### Create a batch of jobs
## Create a batch of jobs

- `POST /api/v1/batches` Create a new batch of single jobs, using the Docker image, that Radix built for the job component. Job-specific arguments can be sent in the request body, specified individually for each item in `jobScheduleDescriptions` with default values defined in `defaultRadixJobComponentConfig`.

Expand Down Expand Up @@ -112,7 +112,7 @@ The Job Manager exposes the following methods for managing jobs:
}
```

### Starting a new job
## Starting a new job

The example configuration at the top has component named `backend` and two jobs, `compute` and `etl`. Radix creates two job-schedulers, one for each of the two jobs. The job-scheduler for `compute` listens to `http://compute:8000`, and job-scheduler for `etl` listens to `http://etl:9000`.

Expand Down Expand Up @@ -141,7 +141,7 @@ Once the job has been created successfully, the `job-scheduler` responds to `bac
- `ended` is the date and time the job successfully ended. Also represented in RFC3339 form and is in UTC. This value is only set for `Successful` jobs.
- `status` is the current status of the job container. Possible values are `Running`, `Successful` and `Failed`. Status is `Failed` if the container exits with a non-zero exit code, and `Successful` if the exit code is zero.

### Getting the status of all existing jobs
## Getting the status of all existing jobs

Get a list of all single jobs with their states by sending a `GET` request to `http://compute:8000/api/v1/jobs`. The response is an array of job state objects, similar to the response received when creating a new job. Jobs that have been started within a batch are not included in this list

Expand Down Expand Up @@ -173,7 +173,7 @@ To get state for a specific job (single or one within a batch), e.g. `batch-comp
}
```

### Deleting an existing job
## Deleting an existing job

The job list in the example above has a job named `batch-compute-20230220101417-idwsxncs-rkwaibwe`. To delete it, send a `DELETE` request to `http://compute:8000/api/v1/jobs/batch-compute-20230220101417-idwsxncs-rkwaibwe`. A successful deletion will respond with result object. Only single job can be deleted with this method

Expand All @@ -185,7 +185,7 @@ The job list in the example above has a job named `batch-compute-20230220101417-
}
```

### Stop a job
## Stop a job

The job list in the example above has a job named `batch-compute-20230220100755-xkoxce5g-mll3kxxh`. To stop it, send a `POST` request to `http://compute:8000/api/v1/jobs/batch-compute-20230220100755-xkoxce5g-mll3kxxh/stop`. A successful stop will respond with result object. Only single job can be stopped with this method. Stop of a job automatically deletes corresponding Kubernetes job and its replica, as well as its log. The job will get the status "Stopped".

Expand All @@ -205,7 +205,7 @@ The job list in the example above has a job named `batch-compute-20230220100755-
}
```

### Starting a new batch of jobs
## Starting a new batch of jobs

To start a new batch of jobs, send a `POST` request to `http://compute:8000/api/v1/batches` with request body set to

Expand Down Expand Up @@ -312,7 +312,7 @@ Once the batch has been created, the `job-scheduler` responds to `backend` with
- `ended` is the date and time the batch successfully ended (empty when not completed). The value is represented in RFC3339 form and is in UTC. This value is only set for `Successful` batches. Batch is ended when all batched jobs are completed or failed.
- `status` is the current status of the batch. Possible values are `Running`, `Successful` and `Failed`. Status is `Failed` if the batch fails for any reason.

### Get a list of all batches
## Get a list of all batches

Get a list of all batches with their states by sending a `GET` request to `http://compute:8000/api/v1/batches`. The response is an array of batch state objects, similar to the response received when creating a new batch

Expand All @@ -334,7 +334,7 @@ Get a list of all batches with their states by sending a `GET` request to `http:
]
```

### Get a state of a batch
## Get a state of a batch

To get state for a specific batch, e.g. `batch-compute-20220302155333-hrwl53mw`, send a `GET` request to `http://compute:8000/api/v1/batches/batch-compute-20220302155333-hrwl53mw`. The response is a batch state object, with states of its jobs

Expand Down Expand Up @@ -368,7 +368,7 @@ To get state for a specific batch, e.g. `batch-compute-20220302155333-hrwl53mw`,
}
```

### Delete a batch
## Delete a batch

The batch list in the example above has a batch named `batch-compute-20220302155333-hrwl53mw`. To delete it, send a `DELETE` request to `http://compute:8000/api/v1/batches/batch-compute-20220302155333-hrwl53mw`. A successful deletion will respond with result object. Deleting of a batch job automatically deletes all jobs, belonging to this batch job.

Expand All @@ -380,7 +380,7 @@ The batch list in the example above has a batch named `batch-compute-20220302155
}
```

### Stop an existing batch
## Stop an existing batch

The batch list in the example above has a batch named `batch-compute-20220302155333-hrwl53mw`. To stop it, send a `POST` request to `http://compute:8000/api/v1/batches/batch-compute-20220302155333-hrwl53mw/stop`. A successful stop will respond with result object. Stop of a batch automatically deletes all batch Kubernetes jobs and their replicas, belonging to this batch job, as well as their logs. All not completed jobs will get the status "Stopped".

Expand All @@ -392,7 +392,7 @@ The batch list in the example above has a batch named `batch-compute-20220302155
}
```

### Stop a jobs in a batch
## Stop a jobs in a batch

The batch list in the example above has a batch named `batch-compute-20220302155333-hrwl53mw` and jobs, one of whicvh has name `batch-compute-20220302155333-hrwl53mw-fjhcqwj7`. To stop this job, send a `POST` request to `http://compute:8000/api/v1/batches/batch-compute-20220302155333-hrwl53mw/jobs/batch-compute-20220302155333-hrwl53mw-fjhcqwj7/stop`. A successful stop will respond with result object. Stop of a batch job automatically deletes corresponding Kubernetes job and its replica, as well as its log. The job will get the status "Stopped".

Expand Down
Binary file modified public-site/docs/src/guides/jobs/job-scheduler-diagram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion public-site/docs/src/guides/jobs/jobs-in-web-console.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ next: openapi-swagger

## Managing Jobs in Web Console

Batches and jobs can be found in the Web console under the "Batches" and "Jobs" tabs. Job scheduler manager pod status and its log can be found in the "Job manager" section
Batches and jobs can be found in the Web console under the "Batches" and "Jobs" tabs. Job manager pod status and its log can be found in the "Job manager" section
![Batch job status](./batch-job-manager-status.png)
* It shows batch and job states. When all jobs in a batch are completed, the batch is completed. If any job fails, the batch is failed.
![Batch and job statuses](./batch-and-job-statuses.png)
Expand Down
4 changes: 2 additions & 2 deletions public-site/docs/src/guides/jobs/notifications.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ prev: job-manager-and-job-api
next: environment-variables
---

## Job notifications
## Notifications

When job and batch status needs to be monitored, their statuses can be [pulled](./#get-a-state-of-a-batch). Another option is to use job component [notifications](../../references/reference-radix-config/#notifications), posted to one of Radix application components.

Expand Down Expand Up @@ -37,7 +37,7 @@ The URL to the Radix application component or job component endpoint can be spec
```
:::

### Radix batch event
## Radix batch event
```json
{
"name": "batch-compute-20220302155333-hrwl53mw",
Expand Down
18 changes: 10 additions & 8 deletions public-site/docs/src/references/reference-radix-config/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,20 +52,22 @@ spec:

The `build` section of the spec contains configuration needed during build (CI part) of the components. In this section you can specify build secrets, which is needed when pulling from locked registries, or cloning from locked repositories.

### `useBuildKit`
`useBuildKit` - (optional, default `false`) build a component with Docker BuildKit. Read [more](../../guides/build-secrets/#build-secrets-with-buildkit) in the guide.

:::tip
When the option `useBuildKit` is set to `true`, Radix will use [buildah](https://www.redhat.com/en/topics/containers/what-is-buildah) to build the components. Buildah requires the `Dockerfile` instruction `FROM` to have a repository prefixing the docker image name.
Otherwise there will be an error during the docker image build:
```bash
Error: creating build container: short-name resolution enforced but cannot prompt without a TTY
```
Otherwise, there will be an error during the docker image build:
> Error: creating build container: short-name resolution enforced but cannot prompt without a TTY

E.g. instead of `FROM alpine` use `FROM docker.io/alpine`, as this `alpine` image is located in the [Docker Hub](https://hub.docker.com/) repository.
Example: instead of `FROM alpine` use `FROM docker.io/alpine`, as this `alpine` image is located in the [Docker Hub](https://hub.docker.com/) repository.
:::
`secret` - (optional) add secrets to Radix config `radixconfig.yaml` in the branch defined as `Config Branch` for your application. This will trigger a new build. This build will fail as no specified build secret has been set. You will now be able to set the secret **values** in the configuration section of your app in the Radix Web Console. These secrets also can be used in the [sub-pipelines](../../guides/sub-pipeline).

`variables` - (optional) environment variable names and values (currently available only in [sub-pipelines](../../guides/sub-pipeline)), provided for all build Radix environments in [sub-pipelines](../../guides/sub-pipeline). These common environment variables are overridden by environment-specific environment variables with the same names.
### `secrets`
`secrets` - (optional) add secrets to Radix config `radixconfig.yaml` in the branch defined as `Config Branch` for your application. This will trigger a new build. This build will fail as no specified build secret has been set. You will now be able to set the secret **values** in the configuration section of your app in the Radix Web Console. These secrets also can be used in the [sub-pipelines](../../guides/sub-pipeline).

### `variables`
`variables` - (optional, available only in [sub-pipelines](../../guides/sub-pipeline)) environment variables names and values, provided for all build Radix environments in [sub-pipelines](../../guides/sub-pipeline). These common environment variables are overridden by environment-specific environment variables with the same names.

:::tip
* When an option `useBuildKit: false`, to ensure that multiline build secrets are handled correct by the build, **all** build secrets are passed as `ARG`-s during container build, base-64 encoded (they need to be decoded before use).
Expand Down Expand Up @@ -716,7 +718,7 @@ spec:
webhook: http://api:8080/monitor-batch-status
```

`webhook` is an optional URL to the Radix application component or job component which will be called when any of the job-component's running jobs or batches changes states. Only changes are sent by POST method with a `application/json` `ContentType` in a [batch event format](../../guides/jobs/#get-a-state-of-a-batch).
`webhook` is an optional URL to the Radix application component or job component which will be called when any of the job-component's running jobs or batches changes states. Only changes are sent by POST method with a `application/json` `ContentType` in a [batch event format](../../guides/jobs/notifications/#radix-batch-event). Read [more](../../guides/jobs/notifications)

### `monitoringConfig`

Expand Down

0 comments on commit b398b65

Please sign in to comment.