Skip to content

Latest commit

 

History

History
30 lines (21 loc) · 2.12 KB

README.md

File metadata and controls

30 lines (21 loc) · 2.12 KB
Raw

Data Receiver Sync Instructions

This readme covers how to use rclone to interact with data stored in the bucket related to this repository.

Definitions

  • Object Storage: "Object Storage (also known as object-based storage) is a computer data storage that manages data as objects, ..."
  • Bucket: "Buckets are the basic containers that hold your data [on Google Cloud Storage]."
  • Objects: "Objects are the individual pieces of data that you store in Cloud Storage."
  • rclone: "Rclone is a command-line program to manage files on cloud storage."

Procedures

The general procedure for gaining access to data stored within a Cloud Storage bucket may be segmented into steps for a Terraform Administrator and a Data receiver (roles are outlined under the root README.md Roles section).

1.) Terraform Administrator Procedure

  1. Communicate with Data receiver about need for Google account.
  2. Add Data receiver Google account to bucket access as a principal.
    • The principal should have Storage Object Viewer and Storage Legacy Bucket Reader roles assigned for data full data read access.
  3. Communicate with Data receiver about data access readiness with their account and the bucket name (for example gc-bucket-alsf-predicting-nuclear-speckles).

2.) Data receiver Procedure

  1. Install rclone.
  2. Configure rclone with Google Cloud Storage.
  3. Test access to Google Cloud Storage bucket with the configured rclone access using, for example, rclone ls <configured_name>:gc-bucket-alsf-predicting-nuclear-speckles
  4. Download or synchronize data found within bucket by using rclone commands.
    • Please note: many rclone commands are recursive by default with options to disable.
    • An example with the copy command: rclone copy <configured_name>:gc-bucket-alsf-predicting-nuclear-speckles