Skip to content

Commit

Permalink
Updated based on feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
joereuss12 committed Apr 3, 2024
1 parent d5312b9 commit c90d95c
Show file tree
Hide file tree
Showing 2 changed files with 7 additions and 3 deletions.
4 changes: 2 additions & 2 deletions docs/pages/client_usage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ which should output a path to the executable. If there is no output to this comm
### Useful Terminology

**Notes on Federations:**
Below are some notes on federations on top of their terminology. Here is the example URL we used previously:
Below are some notes on federations on top of their terminology. Here is the example URL we used previously in the quick start guide:
```console
/ospool/uc-shared/public/OSG-Staff/validation/test.txt
```
Expand All @@ -45,7 +45,7 @@ When using the `osdf:///` scheme, pelican will load in the federation metadata a
```bash
pelican object get osdf:///<namespace-prefix></path/to/file> <local/path/to/file>
```
>This scheme has three slashes (`///`) after the `osdf` because the hostname is left empty to be automatically populated therefore, just start the URL with the namespace prefix. Pelican also supports the `osdf://` scheme (with two slashes) in case the user forgets to pass the third slash, but this is not recommended.
>This scheme has three slashes (`///`) after the `osdf` because the hostname is left empty to be automatically populated. Therefore, just start the URL with the namespace prefix. Pelican also supports the `osdf://` scheme (with two slashes) in case the user forgets to pass the third slash, but this is not recommended.
### The `stash:///` URL Scheme:
This URL scheme comes from legacy code called `stashcp` that Pelican was built off of and is still accepted for backwards compatibility. This URL scheme functions identically to the `osdf:///` URL scheme, loading in OSDF defaults. Here is an example:
Expand Down
6 changes: 5 additions & 1 deletion docs/pages/quickstart_guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ which should output a path to the executable. If there is no output to this comm

### Useful Terminology
**Objects:**
Pelican should be thought of as a tool that works with federated *objects* as opposed to *files*. The reason for this is that calling something a file carries with it the connotation of a filesystem. Filesystems have extra features such as owners, permissions, etc. and Pelican does not work with any of that. In addition, a *file* gives the connotation that the file is mutable, i.e. its contents can change without requiring a new name. Objects in a Pelican federation, however, should be treated as **immutable**, especially in any case where objects are pulled through a cache (which will be the case for almost all files in the [Open Science Data Federation](https://osg-htc.org/services/osdf), or OSDF). This is because the underlying cache mechanism, powered by [XRootD](https://xrootd.slac.stanford.edu/), will deliver whatever object it already has access to; if an object's contents change at the origin, the cache will remain unaware and continue to deliver the old object. In the worst case, when the cache only has a partial object, it may attempt to combine its stale version with whatever exists at the origin. Use object names wisely!
Pelican should be thought of as a tool that works with federated *objects* as opposed to *files*. The reason for this is that calling something a file carries with it the connotation of a filesystem. Filesystems have extra features such as owners, permissions, etc. which Pelican does not implement. In addition, a *file* gives the connotation that the file is mutable, i.e. its contents can change without requiring a new name. Objects in a Pelican federation, however, should be treated as **immutable**, especially in any case where objects are pulled through a cache (which will be the case for almost all files in the [Open Science Data Federation](https://osg-htc.org/services/osdf), or OSDF). This is because the underlying cache mechanism, powered by [XRootD](https://xrootd.slac.stanford.edu/), will deliver whatever object it already has access to; if an object's contents change at the origin, the cache will remain unaware and continue to deliver the old object. In the worst case, when the cache only has a partial object, it may attempt to combine its stale version with whatever exists at the origin. Use object names wisely!

**Federations:**
Objects in Pelican belong to *federations*, which are aggregations of data that are exposed to other individuals in the federation. Each Pelican federation constitutes its own global namespace of objects and each object within a federation has its own path, much like files on a computer. Fetching any object from a federation requires at minimum two pieces of information: a URL indicating the object's federation and the path to the object within that federation (there is the potential that some objects require access tokens as well, but more on that later). For example, the OSDF’s federation URL is https://osg-htc.org and an example object from the federation can be found at the following path:
Expand All @@ -40,6 +40,10 @@ However, things can get a little tricky when you add a longer namespace prefix a
```
In this example, the namespace prefix is `/ospool/uc-shared/public/`, and the actual object is named `/OSG-Staff/validation/test.txt`. While the namespace prefix always comes before the object name, it can get a little confusing once paths and names get longer.

**How to differenciate namespace prefix and object name?**
To discover where the namespace prefix ends and the object name begins, there are a few options. One way is to access your federation's registry (i.e. [https://osdf-registry.osg-htc.org/view/registry/](https://osdf-registry.osg-htc.org/view/registry/)) which should list the namespaces available in the federation. Another option is also asking your federation administrator for how to access the registry.
>We are currently working to improve user experience exploring federations which includes better ways to differentiate namespace prefix and object name.

## Your First Pelican Client Command
Accessing objects from a Pelican federation requires the use of the Pelican client commands. Here we will be using the `pelican object get` command to get an object from a federation. This command is formatted like so:
Expand Down

0 comments on commit c90d95c

Please sign in to comment.