From 42fedb2631b96f4e0b1c4509931df7b215688e95 Mon Sep 17 00:00:00 2001 From: apopova <61460069+AnastasiaTWW@users.noreply.github.com> Date: Fri, 24 Nov 2023 14:27:26 +0400 Subject: [PATCH] Docs/apif 0.6.14 (#70) * describe the new graphql params * describe the changed api mode behavior * fixes to the main guide * add the changelog on the apif 0.6.14 * update the tags to 0.6.14 * add the command for mkdocs serve to the dockerfile to quickly copy it * fixes after review * actualize the list of system varuable for the api mode * fix the description of db update process --- docs.Dockerfile | 4 +- docs/configuration-guides/system-settings.md | 10 +- docs/installation-guides/api-mode.md | 131 ++++++++++++++++-- docs/installation-guides/docker-container.md | 8 +- .../graphql/docker-container.md | 6 +- .../graphql/limit-compliance.md | 4 +- docs/release-notes.md | 5 + 7 files changed, 141 insertions(+), 27 deletions(-) diff --git a/docs.Dockerfile b/docs.Dockerfile index 69304fc..d2e6805 100644 --- a/docs.Dockerfile +++ b/docs.Dockerfile @@ -12,4 +12,6 @@ RUN rm -rf docs EXPOSE 8000 ENTRYPOINT ["mkdocs"] -CMD ["serve", "--dev-addr=0.0.0.0:8000", "--config-file=mkdocs.yml"] \ No newline at end of file +CMD ["serve", "--dev-addr=0.0.0.0:8000", "--config-file=mkdocs.yml"] + +# docker run --rm -it -p 8000:8000/tcp -v ${PWD}:/docs docs-apifirewall:latest \ No newline at end of file diff --git a/docs/configuration-guides/system-settings.md b/docs/configuration-guides/system-settings.md index 09171fe..e68b058 100644 --- a/docs/configuration-guides/system-settings.md +++ b/docs/configuration-guides/system-settings.md @@ -6,10 +6,10 @@ To fine-tune system API Firewall settings, use the following optional environmen | -------------------- | ----------- | | `APIFW_READ_TIMEOUT` | The timeout for API Firewall to read the full request (including the body) sent to the application URL. The default value is `5s`. | | `APIFW_WRITE_TIMEOUT` | The timeout for API Firewall to return the response to the request sent to the application URL. The default value is `5s`. | -| `APIFW_SERVER_MAX_CONNS_PER_HOST` | The maximum number of connections that API Firewall can handle simultaneously. The default value is `512`. | -| `APIFW_SERVER_READ_TIMEOUT` | The timeout for API Firewall to read the full response (including the body) returned to the request by the application. The default value is `5s`. | -| `APIFW_SERVER_WRITE_TIMEOUT` | The timeout for API Firewall to write the full request (including the body) to the application. The default value is `5s`. | -| `APIFW_SERVER_DIAL_TIMEOUT` | The timeout for API Firewall to connect to the application. The default value is `200ms`. | -| `APIFW_SERVER_CLIENT_POOL_CAPACITY` | Maximum number of the fasthttp clients. The default value is `1000`. | +| `APIFW_SERVER_MAX_CONNS_PER_HOST`
(for [`PROXY`](../installation-guides/docker-container.md) and [`graphql`](../installation-guides/graphql/docker-container.md) modes) | The maximum number of connections that API Firewall can handle simultaneously. The default value is `512`. | +| `APIFW_SERVER_READ_TIMEOUT`
(for [`PROXY`](../installation-guides/docker-container.md) and [`graphql`](../installation-guides/graphql/docker-container.md) modes) | The timeout for API Firewall to read the full response (including the body) returned to the request by the application. The default value is `5s`. | +| `APIFW_SERVER_WRITE_TIMEOUT`
(for [`PROXY`](../installation-guides/docker-container.md) and [`graphql`](../installation-guides/graphql/docker-container.md) modes) | The timeout for API Firewall to write the full request (including the body) to the application. The default value is `5s`. | +| `APIFW_SERVER_DIAL_TIMEOUT`
(for [`PROXY`](../installation-guides/docker-container.md) and [`graphql`](../installation-guides/graphql/docker-container.md) modes) | The timeout for API Firewall to connect to the application. The default value is `200ms`. | +| `APIFW_SERVER_CLIENT_POOL_CAPACITY`
(for [`PROXY`](../installation-guides/docker-container.md) and [`graphql`](../installation-guides/graphql/docker-container.md) modes) | Maximum number of the fasthttp clients. The default value is `1000`. | | `APIFW_HEALTH_HOST` | The host of the health check service. The default value is `0.0.0.0:9667`. The liveness probe service path is `/v1/liveness` and the readiness service path is `/v1/readiness`. | diff --git a/docs/installation-guides/api-mode.md b/docs/installation-guides/api-mode.md index cfc9a2d..f361743 100644 --- a/docs/installation-guides/api-mode.md +++ b/docs/installation-guides/api-mode.md @@ -5,27 +5,130 @@ If you need to validate individual API requests based on a given OpenAPI specifi !!! info "Feature availability" This feature is available for the API Firewall versions 0.6.12 and later, and it is tailored for REST API. -To do so: +## Requirements -1. Instead of [mounting the OpenAPI specification](../installation-guides/docker-container.md) file to the container, mount the [SQLite database](https://www.sqlite.org/index.html) containing one or more OpenAPI 3.0 specifications to `/var/lib/wallarm-api/1/wallarm_api.db`. The database should adhere to the following schema: +* [Installed and configured Docker](https://docs.docker.com/get-docker/) +* [SQLite database](https://www.sqlite.org/index.html) with the `openapi_schemas` table containing one or more [OpenAPI 3.0 specifications](https://swagger.io/specification/). The table should adhere to the following schema: * `schema_id`, integer (auto-increment) - ID of the specification. * `schema_version`, string - Specification version. You can assign any preferred version. When this field changes, API Firewall assumes the specification itself has changed and updates it accordingly. * `schema_format`, string - The specification format, can be `json` or `yaml`. * `schema_content`, string - The specification content. -1. Run the container with the environment variable `APIFW_MODE=API` and if needed, with other variables that specifically designed for this mode: - | Environment variable | Description | - | -------------------- | ----------- | - | `APIFW_MODE` | Sets the general API Firewall mode. Possible values are [`PROXY`](docker-container.md) (default), [`graphql`](graphql/docker-container.md), and `API`. | - | `APIFW_SPECIFICATION_UPDATE_PERIOD` | Determines the frequency of specification updates. If set to `0`, the specification update is disabled. The default value is `1m` (1 minute). | - | `APIFW_API_MODE_UNKNOWN_PARAMETERS_DETECTION` | Specifies whether to return an error code if the request parameters do not match those defined in the the specification. The default value is `true`. | - | `APIFW_PASS_OPTIONS` | When set to `true`, the API Firewall allows `OPTIONS` requests to endpoints in the specification, even if the `OPTIONS` method is not described. The default value is `false`. | +## Running the API Firewall container -1. When evaluating whether requests align with the mounted specifications, include the header `X-Wallarm-Schema-ID: ` to indicate to API Firewall which specification should be used for validation. +To use the API Firewall for request validation without further proxying, you need to mount the [SQLite database containing OpenAPI 3.0 specifications](#requirements) to `/var/lib/wallarm-api/1/wallarm_api.db` inside the API Firewall Docker container. The path can be changed using the `APIFW_API_MODE_DEBUG_PATH_DB` variable. -API Firewall validates requests as follows: +Use the following command to run the API Firewall container: -* If a request matches the specification, an empty response with a 200 status code is returned. -* If a request does not match the specification, the response will provide a 403 status code and a JSON document explaining the reasons for the mismatch. -* If it is unable to handle or validate a request, an empty response with a 500 status code is returned. +``` +docker run --rm -it -v :/var/lib/wallarm-api/1/wallarm_api.db \ + -e APIFW_MODE=API -p 8282:8282 wallarm/api-firewall:v0.6.14 +``` + +You can pass to the container the following variables: + +| Environment variable | Description | Required? | +| -------------------- | ----------- | --------- | +| `APIFW_MODE` | Sets the general API Firewall mode. Possible values are [`PROXY`](docker-container.md) (default), [`graphql`](graphql/docker-container.md), and `API`.

The appropriate value for this case is `API`. | Yes | +| `APIFW_URL` | URL for API Firewall. For example: `http://0.0.0.0:8088/`. The port value should correspond to the container port published to the host.

If API Firewall listens to the HTTPS protocol, please mount the generated SSL/TLS certificate and private key to the container, and pass to the container the [API Firewall SSL/TLS settings](../configuration-guides/ssl-tls.md).

The default value is `http://0.0.0.0:8282/`. | No | +| `APIFW_API_MODE_DEBUG_PATH_DB` | Sets a path to a specification database inside the Docker container.

The default value is `/var/lib/wallarm-api/1/wallarm_api.db`. | No | +| `APIFW_SPECIFICATION_UPDATE_PERIOD` | Determines the frequency of fetching updates from the mounted database. If set to `0`, the update is disabled. The default value is `1m` (1 minute). | No | +| `APIFW_API_MODE_UNKNOWN_PARAMETERS_DETECTION` | Determines if requests with undefined parameters, as per the specification, are blocked.

When set to `true`, requests with any non-required, undefined parameters are rejected (e.g., `GET test?a=123&b=123` is blocked if `b` is undefined in the `/test` endpoint specification). If set to `false`, such requests are allowed, provided they contain all required parameters.

The default vaue is `true`. | No | +| `APIFW_PASS_OPTIONS` | When set to `true`, the API Firewall allows `OPTIONS` requests to endpoints in the specification, even if the `OPTIONS` method is not described. The default value is `false`. | No | +| `APIFW_READ_TIMEOUT` | The timeout for API Firewall to read the full request (including the body). The default value is `5s`. | No | +| `APIFW_WRITE_TIMEOUT` | The timeout for API Firewall to return the response to the request. The default value is `5s`. | No | +| `APIFW_HEALTH_HOST` | The host of the health check service. The default value is `0.0.0.0:9667`. The liveness probe service path is `/v1/liveness` and the readiness service path is `/v1/readiness`. | No | + +## Evaluating requests against the specification + +When evaluating requests against the mounted specification, include the header `X-Wallarm-Schema-ID: ` to indicate to API Firewall which specification should be used for validation: + +=== "Single specification" + ``` + curl http://0.0.0.0:8282/path -H "X-Wallarm-Schema-ID: " + ``` +=== "Multiple specifications" + You can evaluate requests against multiple specifications simultaneously. To do this, include the relevant list of specification IDs in the `X-Wallarm-Schema-ID` header, separated by commas. For instance, to assess a request against specifications with IDs 1 and 2, use the following format: + + + ``` + curl http://0.0.0.0:8282/path -H "X-Wallarm-Schema-ID: 1, 2" + ``` + +## Understanding API Firewall responses + +API Firewall responds with the `200` HTTP code and JSON with details on request validation: + +=== "Request matches the specification" + ```json + { + "summary": [ + { + "schema_id": 1, + "status_code": 200 + } + ] + } + ``` +=== "Request does not match the specification" + ```json + { + "summary": [ + { + "schema_id":1, + "status_code":403 + } + ], + "errors": [ + { + "message":"method and path are not found", + "code":"method_and_path_not_found", + "schema_id":1 + } + ] + } + ``` +=== "Unable to validate a request" + ```json + { + "summary": [ + { + "schema_id": 0, + "status_code": 500 + } + ] + } + ``` + +| JSON key | Description | +| -------- | ----------- | +| `summary` | Array with a request validation summary. | +| `summary.schema_id` | The ID of the specification against which the API Firewall performed the request validation. | +| `summary.status_code` | Request validation status code. Possible values:
  • `200` if a request matches the specification.
  • `403` if a request does not match the specification.
  • `500` if it is unable to handle or validate a request.
| +| `errors` | Array containing details about the reasons why a request does not match the specification. | +| `errors.message` | Explanation for the request's dismatch with the specification. | +| `errors.code` | Code indicating the reason for a request's mismatch with the specification. [Possible values](https://github.com/wallarm/api-firewall/blob/50451e6ae99daf958fa75e592d724c8416a098dd/cmd/api-firewall/internal/handlers/api/errors.go#L14). | +| `errors.schema_version` | The version of the specification against which the API Firewall performed the request validation. | +| `errors.related_fields` | An array of parameters that violated the specification. | +| `errors.related_fields_details` | Details on parameters that violated the specification. | +| `errors.related_fields_details.name` | Parameter name. | +| `errors.related_fields_details.expected_type` | Expected parameter type (if the type is wrong). | +| `errors.related_fields_details.current_value` | Parameter value passed in a request. | +| `errors.related_fields_details.pattern` | Parameter value pattern specified in the specification. | + +## Database issues + +### Handling invalidity in an already mounted SQLite database + +The API Firewall automatically retrieves specification updates from the mounted database at intervals defined by the `APIFW_SPECIFICATION_UPDATE_PERIOD` variable. If the database structure or specifications become invalid, or if the database file disappears post-update, the Firewall maintains the last valid specification file and pauses further updates. This method guarantees continuous operation with the most recent valid specifications until a correct database file is reestablished in the API Firewall. + +In cases where the database file is valid but contains an invalid specification, the API Firewall will disregard the faulty specification and proceed to load all valid specifications. + +**Example** + +Suppose the API Firewall has loaded two specifications, labeled 1 and 2. If specification 1 is modified and becomes invalid (due to syntax errors or parsing issues), the API Firewall will then only load and use specification 2. It will log an error message indicating the issue and will operate with only specification 2. + +### Mounting empty SQLite database + +If the API Firewall is initiated with an empty, invalid, or non-existent database file, it will start and log errors if updates fail. In this state, the API Firewall will not have any specification, thus unable to validate requests, and will respond with a 500 status code. Note that the readiness probe will fail until a valid database is loaded. diff --git a/docs/installation-guides/docker-container.md b/docs/installation-guides/docker-container.md index 0920a1f..1b8e92e 100644 --- a/docs/installation-guides/docker-container.md +++ b/docs/installation-guides/docker-container.md @@ -27,13 +27,13 @@ networks: services: api-firewall: container_name: api-firewall - image: wallarm/api-firewall:v0.6.13 + image: wallarm/api-firewall:v0.6.14 restart: on-failure volumes: - : environment: APIFW_API_SPECS: - APIFW_URL: + APIFW_URL: http://0.0.0.0:8088/ APIFW_SERVER_URL: APIFW_REQUEST_VALIDATION: APIFW_RESPONSE_VALIDATION: @@ -83,7 +83,7 @@ Pass API Firewall configuration in **docker-compose.yml** → `services.api-fire | Environment variable | Description | Required? | |-----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| | `APIFW_API_SPECS` | Path to the OpenAPI 3.0 specification. There are the following ways to specify the path:
  • Path to the specification file mounted to the container, for example: `/api-firewall/resources/swagger.json`. When running the container, mount this file with the `-v :` option.
  • URL address of the specification file, for example: `https://example.com/swagger.json`. When running the container, omit the `-v :` option.
| Yes | -| `APIFW_URL` | URL for API Firewall. For example: `http://0.0.0.0:8088/`. The port value should correspond to the container port published to the host.

If API Firewall listens to the HTTPS protocol, please mount the generated SSL/TLS certificate and private key to the container, and pass to the container the **API Firewall SSL/TLS settings** described below. | Yes | +| `APIFW_URL` | URL for API Firewall. For example: `http://0.0.0.0:8088/`. The port value should correspond to the container port published to the host.

If API Firewall listens to the HTTPS protocol, please mount the generated SSL/TLS certificate and private key to the container, and pass to the container the [API Firewall SSL/TLS settings](../configuration-guides/ssl-tls.md).

The default value is `http://0.0.0.0:8282/`. | Yes | | `APIFW_SERVER_URL` | URL of the application described in the mounted OpenAPI specification that should be protected with API Firewall. For example: `http://backend:80`. | Yes | | `APIFW_REQUEST_VALIDATION` | API Firewall mode when validating requests sent to the application URL:
  • `BLOCK` to block and log the requests that do not match the schema provided in the mounted OpenAPI 3.0 specification (the `403 Forbidden` response will be returned to the blocked requests). Logs are sent to the [`STDOUT` and `STDERR` Docker services](https://docs.docker.com/config/containers/logging/).
  • `LOG_ONLY` to log but not block the requests that do not match the schema provided in the mounted OpenAPI 3.0 specification. Logs are sent to the [`STDOUT` and `STDERR` Docker services](https://docs.docker.com/config/containers/logging/).
  • `DISABLE` to disable request validation.
| Yes | | `APIFW_RESPONSE_VALIDATION` | API Firewall mode when validating application responses to incoming requests:
  • `BLOCK` to block and log the request if the application response to this request does not match the schema provided in the mounted OpenAPI 3.0 specification. This request will be proxied to the application URL but the client will receive the `403 Forbidden` response. Logs are sent to the [`STDOUT` and `STDERR` Docker services](https://docs.docker.com/config/containers/logging/).
  • `LOG_ONLY` to log but not block the request if the application response to this request does not match the schema provided in the mounted OpenAPI 3.0 specification. Logs are sent to the [`STDOUT` and `STDERR` Docker services](https://docs.docker.com/config/containers/logging/).
  • `DISABLE` to disable request validation.
| Yes | @@ -153,6 +153,6 @@ To start API Firewall on Docker, you can also use regular Docker commands as in -v : -e APIFW_API_SPECS= \ -e APIFW_URL= -e APIFW_SERVER_URL= \ -e APIFW_REQUEST_VALIDATION= -e APIFW_RESPONSE_VALIDATION= \ - -p 8088:8088 wallarm/api-firewall:v0.6.13 + -p 8088:8088 wallarm/api-firewall:v0.6.14 ``` 4. When the environment is started, test it and enable traffic on API Firewall following steps 6 and 7. diff --git a/docs/installation-guides/graphql/docker-container.md b/docs/installation-guides/graphql/docker-container.md index ea2e694..8ff8aa4 100644 --- a/docs/installation-guides/graphql/docker-container.md +++ b/docs/installation-guides/graphql/docker-container.md @@ -29,7 +29,7 @@ networks: services: api-firewall: container_name: api-firewall - image: wallarm/api-firewall:v0.6.13 + image: wallarm/api-firewall:v0.6.14 restart: on-failure volumes: - : @@ -96,7 +96,9 @@ Pass API Firewall configuration in **docker-compose.yml** → `services.api-fire | `APIFW_GRAPHQL_MAX_QUERY_COMPLEXITY` | [Defines](limit-compliance.md) the maximum number of Node requests that might be needed to execute the query. Setting it to `0` disables the complexity check. The default value is `0`. | Yes | | `APIFW_GRAPHQL_MAX_QUERY_DEPTH` | [Specifies](limit-compliance.md) the maximum permitted depth of a GraphQL query. A value of `0` means the query depth check is skipped. | Yes | | `APIFW_GRAPHQL_NODE_COUNT_LIMIT` | [Sets](limit-compliance.md) the upper limit for the node count in a query. When set to `0`, the node count limit check is skipped. | Yes | +| `APIFW_GRAPHQL_MAX_ALIASES_NUM` | Sets a limit on the number of aliases that can be used in a GraphQL document. If this variable is set to `0`, it implies that there is no limit on the number of aliases that can be used. | Yes | | `APIFW_GRAPHQL_INTROSPECTION` | Allows introspection queries, which disclose the layout of your GraphQL schema. When set to `true`, these queries are permitted. | Yes | +| `APIFW_GRAPHQL_FIELD_DUPLICATION` | Defines whether to allow or prevent the duplication of fields in a GraphQL document. The default value is `false` (prevent). | No | | `APIFW_LOG_LEVEL` | API Firewall logging level. Possible values:
  • `DEBUG` to log events of any type (INFO, ERROR, WARNING, and DEBUG).
  • `INFO` to log events of the INFO, WARNING, and ERROR types.
  • `WARNING` to log events of the WARNING and ERROR types.
  • `ERROR` to log events of only the ERROR type.
  • `TRACE` to log incoming requests and API Firewall responses, including their content.
The default value is `DEBUG`. Logs on requests and responses that do not match the provided schema have the ERROR type. | No | | `APIFW_SERVER_DELETE_ACCEPT_ENCODING` | If it is set to `true`, the `Accept-Encoding` header is deleted from proxied requests. The default value is `false`. | No | | `APIFW_LOG_FORMAT` | The format of API Firewall logs. The value can be `TEXT` or `JSON`. The default value is `TEXT`. | No | @@ -182,6 +184,6 @@ To start API Firewall on Docker, you can also use regular Docker commands as in -e APIFW_GRAPHQL_MAX_QUERY_COMPLEXITY= \ -e APIFW_GRAPHQL_MAX_QUERY_DEPTH= -e APIFW_GRAPHQL_NODE_COUNT_LIMIT= \ -e APIFW_GRAPHQL_INTROSPECTION= \ - -p 8088:8088 wallarm/api-firewall:v0.6.13 + -p 8088:8088 wallarm/api-firewall:v0.6.14 ``` 4. When the environment is started, test it and enable traffic on API Firewall following steps 6 and 7. diff --git a/docs/installation-guides/graphql/limit-compliance.md b/docs/installation-guides/graphql/limit-compliance.md index 88460ef..2a532c0 100644 --- a/docs/installation-guides/graphql/limit-compliance.md +++ b/docs/installation-guides/graphql/limit-compliance.md @@ -8,7 +8,9 @@ When [running](docker-container.md) the API Firewall Docker container for a Grap | -------------------- | ----------- | | `APIFW_GRAPHQL_MAX_QUERY_COMPLEXITY` | Defines the maximum number of Node requests that might be needed to execute the query. Setting it to `0` disables the complexity check. | | `APIFW_GRAPHQL_MAX_QUERY_DEPTH` | Specifies the maximum permitted depth of a GraphQL query. A value of `0` means the query depth check is skipped. | -| `APIFW_GRAPHQL_NODE_COUNT_LIMIT` | Sets the upper limit for the node count in a query. When set to `0`, the node count limit check is skipped. | +| `APIFW_GRAPHQL_NODE_COUNT_LIMIT` | Sets the upper limit for the node count in a query. When set to `0`, the node count limit check is skipped. | +| `APIFW_GRAPHQL_MAX_ALIASES_NUM` | Sets a limit on the number of aliases that can be used in a GraphQL document. If this variable is set to `0`, it implies that there is no limit on the number of aliases that can be used. | Yes | +| `APIFW_GRAPHQL_FIELD_DUPLICATION` | Defines whether to allow or prevent the duplication of fields in a GraphQL document. The default value is `false` (prevent). | ## How limit calculation works diff --git a/docs/release-notes.md b/docs/release-notes.md index 6796c9c..8ac4b09 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -2,6 +2,11 @@ This page describes new releases of Wallarm API Firewall. +## v0.6.14 (2023-11-23) + +* Introduced new environment variables to limit GraphQL queries: `APIFW_GRAPHQL_MAX_ALIASES_NUM` and `APIFW_GRAPHQL_FIELD_DUPLICATION`. +* Implemented more [detailed responses](installation-guides/api-mode.md#understanding-api-firewall-responses) for requests that do not match mounted specifications in the **API non-proxy mode**. + ## v0.6.13 (2023-09-08) * [Support for GraphQL API requests validation](installation-guides/graphql/docker-container.md)