feat: preview_persisted_queries w/opt-in safelisting

[EverlastingBugstopper]

Persisted Queries w/opt-in safelisting

⚠️ This is an Enterprise feature of the Apollo Router. It requires an organization with a GraphOS Enterprise plan and the feature to be enabled for your account.

If your organization doesn't currently have an Enterprise plan, you can test out this functionality by signing up for a free Enterprise trial and reaching out to enable the feature for your account.

Overview

The persisted queries feature allows you to pre-register operations, allowing clients to send an operation ID over the wire and execute the associated operation. Each operation defines the exact shape of a GraphQL operation that the router expects clients to send. In its simplest form, Persisted Queries (PQ’s) can be used like Automatic Persisted Queries (APQ’s) with one key difference: sending an operation body is never allowed for a PQ. Registering persisted operations allows locking down the router to log unregistered operations, or to reject them outright.

Main Configurations

• Unregistered operation monitoring
  • Your router can allow all GraphQL operations, while emitting structured traces containing unregistered operation bodies.
• Operation safelisting
  • Reject unregistered operations
  • Require all operations to be sent as an ID

Usage

preview_persisted_queries:
  enabled: true

This enables additive PQs.

Requires APOLLO_KEY and APOLLO_GRAPH_REF to start up properly (to fetch the license key and the persisted queries themselves), and the graph variant must be linked to a persisted query list. This is only available in preview right now and has to be enabled for a graph.

The router will not start up until all persisted queries have been read into a std::collections::HashMap<String, String> mapping ID to their body. Additionally, just the bodies are stored in a std::collections::HashSet.

After the router starts, persisted queries can be sent over the wire like so:

curl http://localhost:4000/ -X POST --json \
'{"extensions":{"persistedQuery":{"version":1,"sha256Hash":"dc67510fb4289672bea757e862d6b00e83db5d3cbbcfb15260601b6f29bb2b8f"}}}'

2. ./examples/persisted-queries/safelist_pq_log_only.yaml

preview_persisted_queries:
  enabled: true
  log_unpersisted_queries: true

Starting the router with this configuration logs freeform GraphQL operations that do not match a persisted query.

3. ./examples/persisted-queries/safelist_pq.yaml

preview_persisted_queries:
  enabled: true
  safelist:
    enabled: true
apq:
  enabled: false

Starting the router with this configuration will require all operations sent over the wire to match either the ID (O(1) retrieval from HashMap) or the body (O(1) retrieval from HashSet). APQ is enabled by default, and is incompatible with the persisted queries feature (clients are not allowed to register their own persisted queries, they must be pre-published), therefore it must be disabled to start properly. An error is returned if APQ is not explicitly disabled in router.yaml.

4. ./examples/persisted-queries/safelist_pq_require_id.yaml

preview_persisted_queries:
  enabled: true
  safelist:
    enabled: true
    require_id: true
apq:
  enabled: false

This configuration is a stricter version of safelisting that rejects all freeform GraphQL requests, even if they match the body of a persisted query.

[router-perf]

CI performance tests

• events_without_dedup - Stress test for events with a lot of users and deduplication DISABLED
• const - Basic stress test that runs with a constant number of users
• no-graphos - Basic stress test, no GraphOS.
• events_big_cap_high_rate - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity
• events - Stress test for events with a lot of users and deduplication ENABLED
• xxlarge-request - Stress test with 100 MB request payload
• reload - Reload test over a long period of time at a constant rate of users
• xlarge-request - Stress test with 10 MB request payload
• step - Basic stress test that steps up the number of users over time
• large-request - Stress test with a 1 MB request payload

[o0Ignition0o]

Preapproving, I still have two questions, one around an assert! and the other one about the snapshot.

Great work! 🚀

[o0Ignition0o]

is this intentional? iirc we d rather pub use (like at line 92 for example)

[EverlastingBugstopper]

I import the mocks with use crate::test_harness::mocks::persisted_queries::*; and kind of like that structure. I don't want to expose individual mock structs for something specific to the PQ layer.

[o0Ignition0o]

I don't think this is what is expected.
This often means the query planner was updated and the mock was used to work with {errorField id}

⚠️ i think this needs to be addressed before merging

[EverlastingBugstopper]

agree, not sure how this made it in - thought i fixed that up.

[BrynCooke]

If possible can we make the test panic if it detects a missing mock?

[EverlastingBugstopper]

what do you mean? this file is just around because of the router-bridge update and a local test i ran w/this file accidentally committed.

[BrynCooke]

If the snapshot contains couldn't find mock for query it should hard fail the test.

[BrynCooke]

I'm going to fix the remaining test failures, make some minor modifications and then merge this.

[o0Ignition0o]

I'm going to bypass branch protections and merge this so we can cut an alpha release today, for testing purposes.

[glasser]

(The only branch protection bypassed was the Performance Tests.)