diff --git a/jsonschema/docs/.snapshots/TestAWS.md b/jsonschema/docs/.snapshots/TestAWS.md
index 612cabc..ec0d376 100644
--- a/jsonschema/docs/.snapshots/TestAWS.md
+++ b/jsonschema/docs/.snapshots/TestAWS.md
@@ -209,12 +209,12 @@
This will override any regions specified in the provider block.
You can specify all regions by using the `*` character as the only argument in the array.
-* `organization_units` (`[]string`) (nullable)
+* `organization_units` (`[]string`) (nullable) ([pattern](https://json-schema.org/draft/2020-12/json-schema-validation#section-6.3.3): `^((ou-[0-9a-z]{4,32}-[a-z0-9]{8,32})|(r-[0-9a-z]{4,32}))$`)
List of Organizational Units that CloudQuery should use to source accounts from.
If you specify an OU, CloudQuery will also traverse nested OUs.
-* `skip_organization_units` (`[]string`) (nullable)
+* `skip_organization_units` (`[]string`) (nullable) ([pattern](https://json-schema.org/draft/2020-12/json-schema-validation#section-6.3.3): `^((ou-[0-9a-z]{4,32}-[a-z0-9]{8,32})|(r-[0-9a-z]{4,32}))$`)
List of Organizational Units to skip.
This is useful in conjunction with `organization_units` if there are child OUs that should be ignored.
diff --git a/jsonschema/docs/.snapshots/TestGCP.md b/jsonschema/docs/.snapshots/TestGCP.md
new file mode 100644
index 0000000..0cbdec9
--- /dev/null
+++ b/jsonschema/docs/.snapshots/TestGCP.md
@@ -0,0 +1,147 @@
+# Table of contents
+
+* [`Spec`](#Spec)
+ * [`Strategy`](#Strategy)
+ * [`CredentialsConfig`](#CredentialsConfig)
+
+## Spec
+
+ Spec defines GCP source plugin Spec
+
+* `project_ids` (`[]string`) (nullable)
+
+ Specify projects to connect to.
+ If either `folder_ids` or `project_filter` is specified,
+ these projects will be synced in addition to the projects from the folder/filter.
+
+ Empty or `null` value will use all projects available to the current authenticated account.
+
+* `folder_ids` (`[]string`) (nullable) ([pattern](https://json-schema.org/draft/2020-12/json-schema-validation#section-6.3.3): `^(folders|organizations)/(.)+$`)
+
+ CloudQuery will sync from all the projects in the specified folders, recursively.
+ `folder_ids` must be of the format `folders/` or `organizations/`.
+ This feature requires the `resourcemanager.folders.list` permission.
+
+ By default, CloudQuery will also sync from sub-folders recursively (up to depth `100`).
+ To reduce this, set `folder_recursion_depth` to a lower value (or to `0` to disable recursion completely).
+
+ Mutually exclusive with `project_filter`.
+
+* `folder_recursion_depth` (`integer`) (nullable) (range: `[0,+∞)`) (default: `100`)
+
+ The maximum depth to recurse into sub-folders.
+ `0` means no recursion (only the top-level projects in folders will be used for sync).
+
+* `organization_ids` (`[]string`) (nullable)
+
+ Specify organizations to use when syncing organization level resources (e.g.
+ [folders](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/resourcemanager/folders_fetch.go#L23)
+ or
+ [security findings](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/securitycenter/organization_findings.go#L43)).
+
+ If `organization_filter` is specified, these organizations will be used in addition to the organizations from the filter.
+
+ Empty or `null` value will use all organizations available to the current authenticated account).
+
+* `project_filter` (`string`)
+
+ A filter to determine the projects that are synced, mutually exclusive with `folder_ids`.
+
+ For instance, to only sync projects where the name starts with `how-`, set `project_filter` to `name:how-*`.
+
+ More examples:
+
+ - `"name:how-* OR name:test-*"` matches projects starting with `how-` or `test-`
+ - `"NOT name:test-*"` matches all projects _not_ starting with `test-`
+
+ For syntax and example queries refer to API References
+ [here](https://cloud.google.com/resource-manager/reference/rest/v1/projects/list#google.cloudresourcemanager.v1.Projects.ListProjects)
+ and
+ [here](https://cloud.google.com/sdk/gcloud/reference/topic/filters).
+
+* `organization_filter` (`string`)
+
+ A filter to determine the organizations to use when syncing organization level resources (e.g.
+ [folders](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/resourcemanager/folders_fetch.go#L23)
+ or
+ [security findings](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/securitycenter/organization_findings.go#L43)).
+
+ For instance, to use only organizations from the `cloudquery.io` domain, set `organization_filter` to `domain:cloudquery.io`.
+
+ For syntax and example queries refer to API Reference [here](https://cloud.google.com/resource-manager/reference/rest/v1/organizations/search#google.cloudresourcemanager.v1.SearchOrganizationsRequest).
+
+* `service_account_key_json` (`string`)
+
+ GCP service account key content.
+
+ Using service accounts is not recommended, but if it is used it is better to use
+ [environment or file variable substitution](/docs/advanced-topics/environment-variable-substitution).
+
+* `backoff_delay` (`integer`) (range: `[0,+∞)`) (default: `30`)
+
+ If specified APIs will be retried with exponential backoff if they are rate limited.
+ This is the max delay (in seconds) between retries.
+
+* `backoff_retries` (`integer`) (range: `[0,+∞)`) (default: `0`)
+
+ If specified APIs will be retried with exponential backoff if they are rate limited.
+ This is the max number of retries.
+
+* `enabled_services_only` (`boolean`)
+
+ If enabled CloudQuery will skip any resources that belong to a service that has been disabled or not been enabled.
+
+ If you use this option on a large organization (with more than `500` projects)
+ you should also set the `backoff_retries` to a value greater than `0`, otherwise you may hit the API rate limits.
+
+ In `>=v9.0.0` if an error is returned then CloudQuery will assume that all services are enabled
+ and will continue to attempt to sync all specified tables rather than just ending the sync.
+
+* `concurrency` (`integer`) (range: `[1,+∞)`) (default: `50000`)
+
+ The best effort maximum number of Go routines to use.
+ Lower this number to reduce memory usage.
+
+* `discovery_concurrency` (`integer`) (range: `[1,+∞)`) (default: `100`)
+
+ The number of concurrent requests that CloudQuery will make to resolve enabled services.
+ This is only used when `enabled_services_only` is set to `true`.
+
+* `scheduler` ([`Strategy`](#Strategy))
+
+ The scheduler to use when determining the priority of resources to sync.
+
+ For more information about this, see [performance tuning](/docs/advanced-topics/performance-tuning).
+
+* `service_account_impersonation` ([`CredentialsConfig`](#CredentialsConfig)) (nullable)
+
+ Service Account impersonation configuration.
+
+### Strategy
+
+CloudQuery scheduling strategy
+
+(`string`) (possible values: `dfs`, `round-robin`, `shuffle`) (default: `dfs`)
+
+### CredentialsConfig
+
+* `target_principal` (`string`) (required) ([format](https://json-schema.org/draft/2020-12/json-schema-validation#section-7): `email`)
+
+ The email address of the service account to impersonate.
+
+* `scopes` (`[]string`) (nullable) ([pattern](https://json-schema.org/draft/2020-12/json-schema-validation#section-6.3.3): `^https://www.googleapis.com/auth/(.)+$`) (default: `[https://www.googleapis.com/auth/cloud-platform]`)
+
+ Scopes that the impersonated credential should have.
+
+ See available scopes in the [documentation](https://developers.google.com/identity/protocols/oauth2/scopes).
+
+* `delegates` (`[]string`) (nullable) ([format](https://json-schema.org/draft/2020-12/json-schema-validation#section-7): `email`)
+
+ Delegates are the service account email addresses in a delegation chain.
+ Each service account must be granted `roles/iam.serviceAccountTokenCreator` on the next service account in the chain.
+
+* `subject` (`string`)
+
+ The subject field of a JWT (`sub`).
+ This field should only be set if you wish to impersonate a user.
+ This feature is useful when using domain wide delegation.
diff --git a/jsonschema/docs/docs.go b/jsonschema/docs/docs.go
index d0eec41..5188e41 100644
--- a/jsonschema/docs/docs.go
+++ b/jsonschema/docs/docs.go
@@ -147,6 +147,11 @@ func writeDescription(sc *jsonschema.Schema, buff *strings.Builder) {
}
func writeValueAnnotations(sc *jsonschema.Schema, buff *strings.Builder) {
+ if sc.Type == "array" {
+ // tricky, we will traverse the items first
+ writeValueAnnotations(sc.Items, buff)
+ }
+
if len(sc.Format) > 0 {
_, _ = fmt.Fprintf(buff, " ([format](https://json-schema.org/draft/2020-12/json-schema-validation#section-7): `%s`)", sc.Format)
}
@@ -186,6 +191,12 @@ func anyValue(a any) string {
if float64(int64(a)) == a {
return fmt.Sprintf("%d", int64(a))
}
+ case []any:
+ elems := make([]string, len(a))
+ for i, v := range a {
+ elems[i] = anyValue(v)
+ }
+ return fmt.Sprintf("%+v", elems)
}
return fmt.Sprintf("%v", a)
}
diff --git a/jsonschema/docs/docs_test.go b/jsonschema/docs/docs_test.go
index 0aa6070..87b69e2 100644
--- a/jsonschema/docs/docs_test.go
+++ b/jsonschema/docs/docs_test.go
@@ -33,6 +33,10 @@ func TestAWS(t *testing.T) {
genSnapshot(t, "testdata/aws.json")
}
+func TestGCP(t *testing.T) {
+ genSnapshot(t, "testdata/gcp.json")
+}
+
func TestClickHouse(t *testing.T) {
genSnapshot(t, "testdata/clickhouse.json")
}
diff --git a/jsonschema/docs/testdata/gcp.json b/jsonschema/docs/testdata/gcp.json
new file mode 100644
index 0000000..198b178
--- /dev/null
+++ b/jsonschema/docs/testdata/gcp.json
@@ -0,0 +1,211 @@
+{
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
+ "$id": "https://github.com/cloudquery/cloudquery/plugins/source/gcp/client/spec/spec",
+ "$ref": "#/$defs/Spec",
+ "$defs": {
+ "CredentialsConfig": {
+ "properties": {
+ "target_principal": {
+ "type": "string",
+ "format": "email",
+ "description": "The email address of the service account to impersonate."
+ },
+ "scopes": {
+ "oneOf": [
+ {
+ "items": {
+ "type": "string",
+ "pattern": "^https://www.googleapis.com/auth/(.)+$"
+ },
+ "type": "array",
+ "description": "Scopes that the impersonated credential should have.\n\nSee available scopes in the [documentation](https://developers.google.com/identity/protocols/oauth2/scopes).",
+ "default": [
+ "https://www.googleapis.com/auth/cloud-platform"
+ ]
+ },
+ {
+ "type": "null"
+ }
+ ]
+ },
+ "delegates": {
+ "oneOf": [
+ {
+ "items": {
+ "type": "string",
+ "format": "email"
+ },
+ "type": "array",
+ "description": "Delegates are the service account email addresses in a delegation chain.\nEach service account must be granted `roles/iam.serviceAccountTokenCreator` on the next service account in the chain."
+ },
+ {
+ "type": "null"
+ }
+ ]
+ },
+ "subject": {
+ "type": "string",
+ "minLength": 1,
+ "description": "The subject field of a JWT (`sub`).\nThis field should only be set if you wish to impersonate a user.\nThis feature is useful when using domain wide delegation."
+ }
+ },
+ "additionalProperties": false,
+ "type": "object",
+ "required": [
+ "target_principal"
+ ]
+ },
+ "Spec": {
+ "not": {
+ "properties": {
+ "project_filter": {
+ "type": "string",
+ "minLength": 1,
+ "description": "A filter to determine the projects that are synced, mutually exclusive with `folder_ids`.\n\nFor instance, to only sync projects where the name starts with `how-`, set `project_filter` to `name:how-*`.\n\nMore examples:\n\n- `\"name:how-* OR name:test-*\"` matches projects starting with `how-` or `test-`\n- `\"NOT name:test-*\"` matches all projects _not_ starting with `test-`\n\nFor syntax and example queries refer to API References\n[here](https://cloud.google.com/resource-manager/reference/rest/v1/projects/list#google.cloudresourcemanager.v1.Projects.ListProjects)\nand\n[here](https://cloud.google.com/sdk/gcloud/reference/topic/filters)."
+ },
+ "folder_ids": {
+ "items": {
+ "type": "string",
+ "minLength": 1
+ },
+ "type": "array",
+ "minItems": 1,
+ "description": "CloudQuery will sync from all the projects in the specified folders, recursively.\n`folder_ids` must be of the format `folders/\u003cfolder_id\u003e` or `organizations/\u003corganization_id\u003e`.\nThis feature requires the `resourcemanager.folders.list` permission.\n\nBy default, CloudQuery will also sync from sub-folders recursively (up to depth `100`).\nTo reduce this, set `folder_recursion_depth` to a lower value (or to `0` to disable recursion completely).\n\nMutually exclusive with `project_filter`."
+ }
+ },
+ "required": [
+ "project_filter",
+ "folder_ids"
+ ]
+ },
+ "properties": {
+ "project_ids": {
+ "oneOf": [
+ {
+ "items": {
+ "type": "string",
+ "minLength": 1
+ },
+ "type": "array",
+ "description": "Specify projects to connect to.\nIf either `folder_ids` or `project_filter` is specified,\nthese projects will be synced in addition to the projects from the folder/filter.\n\nEmpty or `null` value will use all projects available to the current authenticated account."
+ },
+ {
+ "type": "null"
+ }
+ ]
+ },
+ "folder_ids": {
+ "oneOf": [
+ {
+ "items": {
+ "type": "string",
+ "pattern": "^(folders|organizations)/(.)+$"
+ },
+ "type": "array",
+ "description": "CloudQuery will sync from all the projects in the specified folders, recursively.\n`folder_ids` must be of the format `folders/\u003cfolder_id\u003e` or `organizations/\u003corganization_id\u003e`.\nThis feature requires the `resourcemanager.folders.list` permission.\n\nBy default, CloudQuery will also sync from sub-folders recursively (up to depth `100`).\nTo reduce this, set `folder_recursion_depth` to a lower value (or to `0` to disable recursion completely).\n\nMutually exclusive with `project_filter`."
+ },
+ {
+ "type": "null"
+ }
+ ]
+ },
+ "folder_recursion_depth": {
+ "oneOf": [
+ {
+ "type": "integer",
+ "minimum": 0,
+ "description": "The maximum depth to recurse into sub-folders.\n`0` means no recursion (only the top-level projects in folders will be used for sync).",
+ "default": 100
+ },
+ {
+ "type": "null"
+ }
+ ]
+ },
+ "organization_ids": {
+ "oneOf": [
+ {
+ "items": {
+ "type": "string",
+ "minLength": 1
+ },
+ "type": "array",
+ "description": "Specify organizations to use when syncing organization level resources (e.g.\n[folders](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/resourcemanager/folders_fetch.go#L23)\nor\n[security findings](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/securitycenter/organization_findings.go#L43)).\n\nIf `organization_filter` is specified, these organizations will be used in addition to the organizations from the filter.\n\nEmpty or `null` value will use all organizations available to the current authenticated account)."
+ },
+ {
+ "type": "null"
+ }
+ ]
+ },
+ "project_filter": {
+ "type": "string",
+ "description": "A filter to determine the projects that are synced, mutually exclusive with `folder_ids`.\n\nFor instance, to only sync projects where the name starts with `how-`, set `project_filter` to `name:how-*`.\n\nMore examples:\n\n- `\"name:how-* OR name:test-*\"` matches projects starting with `how-` or `test-`\n- `\"NOT name:test-*\"` matches all projects _not_ starting with `test-`\n\nFor syntax and example queries refer to API References\n[here](https://cloud.google.com/resource-manager/reference/rest/v1/projects/list#google.cloudresourcemanager.v1.Projects.ListProjects)\nand\n[here](https://cloud.google.com/sdk/gcloud/reference/topic/filters)."
+ },
+ "organization_filter": {
+ "type": "string",
+ "description": "A filter to determine the organizations to use when syncing organization level resources (e.g.\n[folders](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/resourcemanager/folders_fetch.go#L23)\nor\n[security findings](https://github.com/cloudquery/cloudquery/blob/0e384a84d1c9545b24c2eda9af00f111bab79c36/plugins/source/gcp/resources/services/securitycenter/organization_findings.go#L43)).\n\nFor instance, to use only organizations from the `cloudquery.io` domain, set `organization_filter` to `domain:cloudquery.io`.\n\nFor syntax and example queries refer to API Reference [here](https://cloud.google.com/resource-manager/reference/rest/v1/organizations/search#google.cloudresourcemanager.v1.SearchOrganizationsRequest)."
+ },
+ "service_account_key_json": {
+ "type": "string",
+ "description": "GCP service account key content.\n\nUsing service accounts is not recommended, but if it is used it is better to use\n[environment or file variable substitution](/docs/advanced-topics/environment-variable-substitution)."
+ },
+ "backoff_delay": {
+ "type": "integer",
+ "minimum": 0,
+ "description": "If specified APIs will be retried with exponential backoff if they are rate limited.\nThis is the max delay (in seconds) between retries.",
+ "default": 30
+ },
+ "backoff_retries": {
+ "type": "integer",
+ "minimum": 0,
+ "description": "If specified APIs will be retried with exponential backoff if they are rate limited.\nThis is the max number of retries.",
+ "default": 0
+ },
+ "enabled_services_only": {
+ "type": "boolean",
+ "description": "If enabled CloudQuery will skip any resources that belong to a service that has been disabled or not been enabled.\n\nIf you use this option on a large organization (with more than `500` projects)\nyou should also set the `backoff_retries` to a value greater than `0`, otherwise you may hit the API rate limits.\n\nIn `\u003e=v9.0.0` if an error is returned then CloudQuery will assume that all services are enabled\nand will continue to attempt to sync all specified tables rather than just ending the sync."
+ },
+ "concurrency": {
+ "type": "integer",
+ "minimum": 1,
+ "description": "The best effort maximum number of Go routines to use.\nLower this number to reduce memory usage.",
+ "default": 50000
+ },
+ "discovery_concurrency": {
+ "type": "integer",
+ "minimum": 1,
+ "description": "The number of concurrent requests that CloudQuery will make to resolve enabled services.\nThis is only used when `enabled_services_only` is set to `true`.",
+ "default": 100
+ },
+ "scheduler": {
+ "$ref": "#/$defs/Strategy",
+ "description": "The scheduler to use when determining the priority of resources to sync.\n\nFor more information about this, see [performance tuning](/docs/advanced-topics/performance-tuning)."
+ },
+ "service_account_impersonation": {
+ "oneOf": [
+ {
+ "$ref": "#/$defs/CredentialsConfig",
+ "description": "Service Account impersonation configuration."
+ },
+ {
+ "type": "null"
+ }
+ ]
+ }
+ },
+ "additionalProperties": false,
+ "type": "object",
+ "description": "Spec defines GCP source plugin Spec"
+ },
+ "Strategy": {
+ "type": "string",
+ "enum": [
+ "dfs",
+ "round-robin",
+ "shuffle"
+ ],
+ "title": "CloudQuery scheduling strategy",
+ "default": "dfs"
+ }
+ }
+}