|
1 | 1 | # Registry
|
2 | 2 |
|
3 |
| -Feast uses a registry to store all applied Feast objects (e.g. Feature views, entities, etc). The registry exposes methods to apply, list, retrieve and delete these objects, and is an abstraction with multiple implementations. |
| 3 | +Feast uses a registry to store all applied Feast objects (e.g. Feature views, entities, etc). The registry exposes |
| 4 | +methods to apply, list, retrieve and delete these objects, and is an abstraction with multiple implementations. |
4 | 5 |
|
5 | 6 | ### Options for registry implementations
|
6 | 7 |
|
7 |
| -By default, the registry Feast uses a file-based registry implementation, which stores the protobuf representation of the registry as a serialized file. This registry file can be stored in a local file system, or in cloud storage (in, say, S3 or GCS). |
| 8 | +#### File-based registry |
| 9 | +By default, Feast uses a file-based registry implementation, which stores the protobuf representation of the registry as |
| 10 | +a serialized file. This registry file can be stored in a local file system, or in cloud storage (in, say, S3 or GCS). |
8 | 11 |
|
9 |
| -However, there's inherent limitations with a file-based registry, since changing a single field in the registry requires re-writing the whole registry file. With multiple concurrent writers, this presents a risk of data loss, or bottlenecks writes to the registry since all changes have to be serialized (e.g. when running materialization for multiple feature views or time ranges concurrently). |
| 12 | +The quickstart guides that use `feast init` will use a registry on a local file system. To allow Feast to configure |
| 13 | +a remote file registry, you need to create a GCS / S3 bucket that Feast can understand: |
| 14 | +{% tabs %} |
| 15 | +{% tab title="Example S3 file registry" %} |
| 16 | +```yaml |
| 17 | +project: feast_demo_aws |
| 18 | +provider: aws |
| 19 | +registry: s3://[YOUR BUCKET YOU CREATED]/registry.pb |
| 20 | +online_store: null |
| 21 | +offline_store: |
| 22 | + type: file |
| 23 | +``` |
| 24 | +{% endtab %} |
10 | 25 |
|
| 26 | +{% tab title="Example GCS file registry" %} |
| 27 | +```yaml |
| 28 | +project: feast_demo_gcp |
| 29 | +provider: gcp |
| 30 | +registry: gs://[YOUR BUCKET YOU CREATED]/registry.pb |
| 31 | +online_store: null |
| 32 | +offline_store: |
| 33 | + type: file |
| 34 | +``` |
| 35 | +{% endtab %} |
| 36 | +{% endtabs %} |
| 37 | +
|
| 38 | +However, there are inherent limitations with a file-based registry, since changing a single field in the registry |
| 39 | +requires re-writing the whole registry file. With multiple concurrent writers, this presents a risk of data loss, or |
| 40 | +bottlenecks writes to the registry since all changes have to be serialized (e.g. when running materialization for |
| 41 | +multiple feature views or time ranges concurrently). |
| 42 | +
|
| 43 | +#### SQL Registry |
11 | 44 | Alternatively, a [SQL Registry](../../tutorials/using-scalable-registry.md) can be used for a more scalable registry.
|
12 | 45 |
|
| 46 | +This supports any SQLAlchemy compatible database as a backend. The exact schema can be seen in [sql.py](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/registry/sql.py) |
| 47 | +
|
13 | 48 | ### Updating the registry
|
14 | 49 |
|
15 |
| -We recommend users store their Feast feature definitions in a version controlled repository, which then via CI/CD automatically stays synced with the registry. Users will often also want multiple registries to correspond to different environments (e.g. dev vs staging vs prod), with staging and production registries with locked down write access since they can impact real user traffic. See [Running Feast in Production](../../how-to-guides/running-feast-in-production.md#1.-automatically-deploying-changes-to-your-feature-definitions) for details on how to set this up. |
| 50 | +We recommend users store their Feast feature definitions in a version controlled repository, which then via CI/CD |
| 51 | +automatically stays synced with the registry. Users will often also want multiple registries to correspond to |
| 52 | +different environments (e.g. dev vs staging vs prod), with staging and production registries with locked down write |
| 53 | +access since they can impact real user traffic. See [Running Feast in Production](../../how-to-guides/running-feast-in-production.md#1.-automatically-deploying-changes-to-your-feature-definitions) for details on how to set this up. |
16 | 54 |
|
17 | 55 | ### Accessing the registry from clients
|
18 | 56 |
|
19 |
| -Users can specify the registry through a `feature_store.yaml` config file, or programmatically. We often see teams preferring the programmatic approach because it makes notebook driven development very easy: |
| 57 | +Users can specify the registry through a `feature_store.yaml` config file, or programmatically. We often see teams |
| 58 | +preferring the programmatic approach because it makes notebook driven development very easy: |
20 | 59 |
|
21 | 60 | #### Option 1: programmatically specifying the registry
|
22 | 61 |
|
23 | 62 | ```python
|
24 | 63 | repo_config = RepoConfig(
|
25 | 64 | registry=RegistryConfig(path="gs://feast-test-gcs-bucket/registry.pb"),
|
26 | 65 | project="feast_demo_gcp",
|
27 |
| - provider="gcp",thon |
| 66 | + provider="gcp", |
28 | 67 | offline_store="file", # Could also be the OfflineStoreConfig e.g. FileOfflineStoreConfig
|
29 | 68 | online_store="null", # Could also be the OnlineStoreConfig e.g. RedisOnlineStoreConfig
|
30 | 69 | )
|
|
0 commit comments