Skip to content
This repository has been archived by the owner on Sep 30, 2024. It is now read-only.

Docs: update docs for src validate command to add new install subcommand #46397

Merged
merged 4 commits into from
Jan 13, 2023
Merged

Docs: update docs for src validate command to add new install subcommand #46397

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
32dc6ac
Update YAML and JSON config specs
jdpleiness Jan 12, 2023
223cbd8
Update examples with new `src validate` command
jdpleiness Jan 12, 2023
462ef90
Fix formatting issues
jdpleiness Jan 12, 2023
e9b1bfa
Fix spelling
jdpleiness Jan 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
205 changes: 103 additions & 102 deletions doc/admin/validation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,22 @@
# Sourcegraph Instance Validation

>NOTE: **Sourcegraph Instance Validation is currently experimental.** We're exploring this feature set.
>🚨 WARNING 🚨: **Sourcegraph Instance Validation is currently experimental.** We're exploring this feature set.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use emoji more 😆

>Let us know what you think! [File an issue](https://github.com/sourcegraph/sourcegraph/issues/new/choose)
>with feedback/problems/questions, or [contact us directly](https://about.sourcegraph.com/contact).

Instance validation provides a quick way to check that a Sourcegraph instance functions properly after a fresh install
or an update.

The [`src` CLI](https://github.com/sourcegraph/src-cli) has an experimental command `validate` which drives the
validation from a user-provided configuration file with a validation specification (in JSON or YAML format). if no validation specification file is provided it will execute the following:
The [`src` CLI](https://github.com/sourcegraph/src-cli) has an experimental command `validate install` which drives the
validation from a user-provided configuration file with a validation specification (in JSON or YAML format). If no validation specification file is provided it will execute the following:

* temporarily add an external service
* wait for a repository to be cloned
* perform a search on the cloned repo
* perform a search on a non-indexed branch of the cloned repo
* temporarily adds an external service
* waits for a repository to be cloned
* performs a search on the cloned repo
* performs a search on a non-indexed branch of the cloned repo
* creates basic code insight
* remove the added external service
* removes the added external service
* removes the added code insight

### Validation specification

Expand All @@ -24,152 +25,152 @@ Validation specifications can be provided in either a YAML or JSON format. The b
#### YAML File Specification

```yaml
# creates the first admin user on a fresh install (skips creation if user exists)
firstAdmin:
email: foo@example.com
username: foo
password: "{{ .admin_password }}"

# adds the specified code host
externalService:
config:
url: https://github.com
token: "{{ .github_token }}"
orgs: []
repos:
- sourcegraph-testing/zap
kind: GITHUB
displayName: footest
displayName: srcgraph-test
# set to true if this code host config should be deleted at the end of validation
deleteWhenDone: true

# checks maxTries if specified repo is cloned and waits sleepBetweenTriesSeconds between checks
waitRepoCloned:
repo: github.com/sourcegraph-testing/zap
maxTries: 5
sleepBetweenTriesSeconds: 2
# maxRetries amount of retries for cloning repo
maxRetries: 5
# retryTimeoutSeconds wait in seconds between retries
retryTimeoutSeconds: 5
config:
gitHub:
url: https://github.com
orgs: []
repos:
- sourcegraph-testing/zap

# performs the specified search and checks that at least one result is returned
searchQuery:
- repo:^github.com/sourcegraph-testing/zap$ test
- repo:^github.com/sourcegraph-testing/zap$@v1.14.1 test

# checks to see if instance can create code insights
createInsight:
title: "Javascript to Typescript migration"
insight:
title: "Javascript to Typescript migration"
dataSeries:
[ {
"query": "lang:javascript",
"label": "javascript",
"repositoryScope": [
"github.com/sourcegraph/sourcegraph"
],
"lineColor": "#6495ED",
"timeScopeUnit": "MONTH",
"timeScopeValue": 1
},
{
"query": "lang:typescript",
"label": "typescript",
"lineColor": "#DE3163",
"repositoryScope": [
"github.com/sourcegraph/sourcegraph"
],
"timeScopeUnit": "MONTH",
"timeScopeValue": 1
}
]
```
#### JSON File Specification

```json
{
"externalService": {
"kind": "GITHUB",
"displayName": "srcgraph-test",
"deleteWhenDone": true,
"maxRetries": 5,
"retryTimeoutSeconds": 5,
"config": {
"gitHub": {
"url": "https://github.com",
"orgs": [],
"repos": [
"sourcegraph-testing/zap"
]
}
}
},

"searchQuery": [
"repo:^github.com/sourcegraph-testing/zap$ test",
"repo:^github.com/sourcegraph-testing/zap$@v1.14.1 test"
],

"insight": {
"title": "Javascript to Typescript migration",
"dataSeries": [
{
"query": "lang:javascript",
"label": "javascript",
"repositoryScope": [], # e.g. ["github.com/sourcegraph-testing/zap"]
"repositoryScope": [
"github.com/sourcegraph/sourcegraph"
],
"lineColor": "#6495ED",
"timeScopeUnit": "MONTH",
"timeScopeValue": 1
},
{
{
"query": "lang:typescript",
"label": "typescript",
"lineColor": "#DE3163",
"repositoryScope": [
"github.com/sourcegraph/sourcegraph"
],
"github.com/sourcegraph/sourcegraph"
],
"timeScopeUnit": "MONTH",
"timeScopeValue": 1
}
]
```
#### JSON File Specification
}

```json
{
"firstAdmin": {
"email": "foo@example.com",
"username": "foo",
"password": "{{ .admin_password }}"
},
"externalService": {
"config": {
"url": "https://github.com",
"token": "{{ .github_token }}",
"orgs": [],
"repos": [
"sourcegraph-testing/zap"
]
},
"kind": "GITHUB",
"displayName": "footest",
"deleteWhenDone": true
},
"waitRepoCloned": {
"repo": "github.com/sourcegraph-testing/zap",
"maxTries": 5,
"sleepBetweenTriesSeconds": 5
},
"searchQuery": [
"repo:^github.com/sourcegraph-testing/zap$ test",
"repo:^github.com/sourcegraph-testing/zap$@v1.14.1 test"
],

"createInsight": {
"title": "Javascript to Typescript migration",
"dataSeries" : [ {
"query": "lang:javascript",
"label": "javascript",
"repositoryScope": [],
"lineColor": "#6495ED",
"timeScopeUnit": "MONTH",
"timeScopeValue": 1
},
{
"query": "lang:typescript",
"label": "typescript",
"lineColor": "#DE3163",
"repositoryScope": [],
"timeScopeUnit": "MONTH",
"timeScopeValue": 1
}
]
}
```

With this configuration, the validation command executes the following steps:

* create the first admin user
* add an external service
* wait for a repository to be cloned
* perform a search
* creates a code insight ( will need to be manually removed )
* adds an external service
* waits for a repository to be cloned
* performs a search
* removes the external service
* creates a code insight
* removes the code insight

>NOTE: Every step is optional (if the corresponding top-level key is not present then the step is skipped).
>

### Passing in secrets

It is often the case that the config file with the validation specification needs to declare passwords, tokens, or other
secrets and these secrets should not be exposed or committed to a git repo.
Secrets are handled via environment variables. `src validate install` requires an environment variables to be set to authenticate to your code host.

The validation specification can refer to string values that come from a context specified outside the config file
(see the `Usage` section below). References to string values from this outside context are specified like so:
`{{ .some_key }}`. The context will have a string value defined under the key `some_key` and the validation execution will
use that.
>NOTE: Currently only GitHub is supported as a code host.
>

`src validate install` requires an environment variable to be setup to authenticate against your code host.

* `SRC_GITHUB_TOKEN` - defines the GitHub access token used to authenticate to the GitHub API. This must be set to use a GitHub code host.


Example:
```bash
export SRC_GITHUB_TOKEN=token
```

### Usage

Use the [`src` CLI](https://github.com/sourcegraph/src-cli) to validate with a validation specification file:
```shell script
src validate -context github_token=$GITHUB_TOKEN validate.yaml
src validate install validate.yaml
```
```shell script
src validate -context github_token=$GITHUB_TOKEN validate.json
src validate install validate.json
```
To execute default validation checks:

```shell script
src validate -context github_token=$GITHUB_TOKEN
src validate install
```

The `src` binary finds the Sourcegraph instance to validate from the environment variables
[`SRC_ENDPOINT` and `SRC_ACCESS_TOKEN`](https://github.com/sourcegraph/src-cli#setup-with-your-sourcegraph-instance).

> Note: The `SRC_ACCESS_TOKEN` is not needed when a first admin user is declared in the validation specification.