ADR 3 revision #27
ADR 3 revision #27
Conversation
| ``` | ||
|
|
||
| This template contains four keys: | ||
| - `prefix`: the root location for the target. This could just be a bucket name, | ||
| or it could be a sub-path within a bucket. | ||
| **Question: should prefix include the protocol, e.g. s3:// ?** | ||
| For now I am assuming no. | ||
| - `feedstock_name`: the name of the repository that holds the feedstock with | ||
| the recipe that generated the dataset. | ||
| - `subcatalog_name`: [**optional**] the name of a subcatalog to which the dataset belongs. If `subcatalog_name` is not provided, the feedstock will be included in the top-level Pangeo Forge STAC Catalog. |
There was a problem hiding this comment.
I agree it would be useful to allow a given dataset to be included in arbitrary catalogs. However, I'm not convinced it needs to be in the storage path name. There are advantages to having the storage path name map closely to feedstocks and recipes.
Here is an alternative way we could support having recipes appear in different catalogs. Rather than having subcatalog_name be part of the dataset path, we could allow datasets to be added to any catalog, via a new keyword in meta.yaml, or even an external database. We could even imagine giving users the ability to define their own catalogs and share them with others, e.g. "my favorite datasets". None of this requires us to change the storage path naming conventions.
|
closing as stale |
Following discussion in pangeo-forge/staged-recipes#67 and pangeo-forge/pangeo-forge-catalog#1, here is a first draft of some possible revisions to ADR 3, as a basis for conversation.
cc @rabernat @sharkinsspatial