Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: refocus fly-replay documentation #1884

Merged
merged 2 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
145 changes: 69 additions & 76 deletions networking/dynamic-request-routing.html.markerb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Dynamic Request Routing
title: Dynamic Request Routing with fly-replay
layout: docs
nav: firecracker
redirect_from:
Expand All @@ -8,127 +8,120 @@ redirect_from:
- /docs/reference/dynamic-request-routing/
---

Traffic sent to multi-region Fly.io apps is automatically routed to the region nearest to the client, thanks to Anycast-enabled IP addresses. [Read more about Anycast on Wikipedia](https://en.wikipedia.org/wiki/Anycast).
The `fly-replay` header is a powerful feature that gives you fine-grained control over request routing in your Fly.io applications. It allows you to dynamically route requests between regions, specific Machines, or even different apps within your organization.

Sometimes it's useful, or necessary, to route requests to other regions, other apps, or both. You can use custom request and response HTTP headers to achieve this.
## How fly-replay Works

## The `fly-replay` response header
When your app adds a `fly-replay` header to a response, [Fly Proxy](/docs/reference/fly-proxy) will automatically replay the original request according to your specified routing rules. This enables advanced patterns like:

Apps may append a `fly-replay` header to responses. This instructs [Fly Proxy](/docs/reference/fly-proxy) to redeliver (replay) the original request in another region, or to replay it to another app in the same organization.
- Routing write operations to primary database regions
- Load balancing between specific Machines
- Cross-app request routing within your organization
- Implementing sticky sessions

<%= youtube "https://www.youtube.com/watch?v=riCh9Xeuf0s" %>

The content of the `fly-replay` header fields tells Fly Proxy which magic to perform, and the proxy takes it from there. If the target region doesn't host any healthy Machines for the target app, the proxy throws an error.
## Header Fields Reference

The `fly-replay` header accepts the following fields:

|Field |Description |
|---|---|
|`region` | The 3-letter code for the [region](/docs/reference/regions/) to which the request should be routed. |
|`instance` | The ID of the specific Machine to which the request should be routed. |
|`app` | The name of the app within the same organization, to which the request should be routed.<br>Fly Proxy will choose the nearest Machine if no region is specified.|
|`state` | Optional arbitrary string to include in the `fly-replay-src` header appended to the request being replayed. |
|`elsewhere` | Boolean. If `true`, the responding Machine will be excluded from the next round of load-balancing. |

### Limitations

Attempting to replay requests larger than 1MB will throw an error. If you need certain requests - like file uploads - to be handled by a specific region, consider the following workarounds.

If your service is simply proxying uploads to object storage, some services like S3 allow [uploading directly from the browser](https://aws.amazon.com/blogs/compute/uploading-to-amazon-s3-directly-from-a-web-or-mobile-application/+external). Rails supports this [out of the box](https://guides.rubyonrails.org/active_storage_overview.html#direct-uploads+external).

If you can perform uploads via `fetch` or `XMLHttpRequest`, you can prepend the [fly-prefer-region](#the-fly-prefer-region-request-header) or [fly-force-instance-id](#the-fly-force-instance-id-request-header) header to send the request directly to a specific region.

Your app should use the [http handler](/docs/networking/services/#http-handler) in order to use fly-replay.
|`region` | The 3-letter code for the [region](/docs/reference/regions/) to route the request to |
|`instance` | The ID of a specific Machine to route to |
|`app` | The name of another app (in same organization) to route to |
|`state` | Optional string included in `fly-replay-src` header on replay |
|`elsewhere` | If `true`, excludes responding Machine from next load-balance |

## `fly-replay` use cases

Here are a few example use cases for the `fly-replay` response header.

### Replaying writes in secondary regions

Fly.io Postgres supports [global read replicas](/docs/postgres/high-availability-and-global-replication) for speeding up reads in regions close to users.

Replay a request to the region where the HA database leader lives:
### Example Usage

Route to specific region:
```
fly-replay: region=sjc
```

Fly Proxy will get the request to a Machine in that region and let the instances in the cluster take care of getting it to the writeable leader.

Check out our blueprint for [Multi-region databases and fly-replay](/docs/blueprints/multi-region-fly-replay/).

### Replay requests to other apps

`fly-replay` can replay requests across apps in the same organization. Think of a router app for a FaaS that wants to spin up a customer [Machine](/docs/machines/) on demand.

To send the request to a different app in the same organization, in the nearest region that has a Machine:

```
fly-replay: app=app-in-same-org
```

### Replay requests to specific Machines

Replay the request to a specific Machine by ID:

Route to specific Machine:
```
fly-replay: instance=00bb33ff
```

### Pass state to replay targets

The request replay target may need to know why the request was routed to it:

Route to another app:
```
fly-replay: region=sjc;state=captured_write
fly-replay: app=app-in-same-org
```

Fields can be stacked; for instance, to send the request on to an Machine in the app "app-in-same-org" in the sjc region:

You can combine multiple fields:
```
fly-replay: region=sjc;app=app-in-same-org
```

Some combinations of fields can conflict. For example, don't specify an app name and a Machine ID that doesn't belong to that app.
## Implementation Details

### Requirements and Limitations

Related: [Multi-region PostgreSQL](/docs/postgres/high-availability-and-global-replication); [Run Ordinary Rails Apps Globally](/blog/run-ordinary-rails-apps-globally/)
- Your app must use the [http handler](/docs/networking/services/#http-handler)
- Requests larger than 1MB cannot be replayed
- Field combinations must be logically valid (e.g., don't specify both app and instance if instance isn't in that app)

## The `fly-replay-src` request header
For large uploads that exceed the 1MB limit, consider:
- Using direct-to-storage uploads where possible
- Using the [fly-prefer-region](#the-fly-prefer-region-request-header) header instead

When replaying an HTTP request, Fly Proxy appends the `fly-replay-src` header with information about the Machine that sent the `fly-replay`.
### The fly-replay-src Header

When a request is replayed, Fly Proxy adds a `fly-replay-src` header containing metadata about the original request:

|Field |Description |
|---|---|
|`instance` | The ID of the Machine emitting `fly-replay`. |
|`region` | The region `fly-replay` was sent from. |
|`t` | A timestamp: microseconds since the Unix epoch. |
|`state` | The contents of the `state` supplied by the `fly-replay` header, if any. |

See how the [official Fly.io Ruby client](https://github.com/superfly/fly-ruby/blob/main/lib/fly-ruby/regional_database.rb#L74-L76+external) uses the `state` and `t` fields to prevent [read-your-own-write](https://jepsen.io/consistency/models/read-your-writes+external) inconsistency.
|`instance` | ID of Machine that sent fly-replay |
|`region` | Region request was replayed from |
|`t` | Timestamp (microseconds since Unix epoch) |
|`state` | Contents of original state field, if any |

## The `fly-prefer-region` request header
This header is useful for tracking request paths and implementing consistency patterns. See the [official Ruby client](https://github.com/superfly/fly-ruby/blob/main/lib/fly-ruby/regional_database.rb#L74-L76+external) for an example of using these fields to prevent read-your-write inconsistency.

Clients accessing Fly.io apps may set the `fly-prefer-region` header to attempt sending the request directly to a desired target region. This is useful for cases where `fly-replay` isn't practical, such as large file uploads which can't be replayed once buffered by Fly Proxy.
## Alternative Routing Headers

If the target region has no healthy Machines, the region nearest to the client with healthy Machines will field the request.
For cases where `fly-replay` isn't suitable, Fly.io provides two alternative request headers:

Example:
### The fly-prefer-region Header

```
fly-prefer-region: ams
```

## The `fly-force-instance-id` request header

Clients accessing Fly.io apps may set the `fly-force-instance-id` header to ensure that the request is sent to only a specific Machine.
Attempts to route directly to a specific region. Falls back to nearest region with healthy Machines if target region is unavailable. Useful for large uploads that can't be replayed.

If the Machine is deleted or not found, no other Machines will be tried.

Example:
### The fly-force-instance-id Header

```
fly-force-instance-id: 90801679a10038
```

Forces routing to a specific Machine. No fallback if Machine is unavailable.

<div class="note icon">
**Note**: The value of this header is the Machine ID, which you can get by running `fly status` or `fly machines list`.
**Note**: Get Machine IDs using `fly status` or `fly Machines list`.
</div>

## Common Use Cases

### Multi-Region Databases

When using [global read replicas](/docs/postgres/high-availability-and-global-replication), use `fly-replay` to ensure write operations go to the primary region:

```
fly-replay: region=sjc
```

See our blueprint for [Multi-region databases and fly-replay](/docs/blueprints/multi-region-fly-replay/) for a complete implementation guide.

### Cross-App Routing

Use `fly-replay` to implement routing layers or FaaS-style architectures:

```
fly-replay: app=customer-function-app
```

This allows you to build router apps that can dynamically route requests to other apps in your organization.
2 changes: 2 additions & 0 deletions styles/words2ignore.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ v1
v2
vm
vms
http
youtube
Loading