fix file management policy

Overview

Any text fix files (less than 1.5MB) will be tracked by git, just like script/config/etc files in the rrfs_workflow repo.

Any binary or large text file will be tracked through a link (not the file content itself).
Binary/large files themselves are stored at a fixed location (FIX_RRFS2/) and available on Jet, Hera, Orion, WCOSS in a similar setting. Synchronization from Jet to other HPCs may be done every week automatically (or on demand)

FIX_RRFS Locations at different HPCs:

• Jet: /lfs5/BMC/nrtrr/FIX_RRFS2
• Hera: /scratch2/BMC/rtrr/FIX_RRFS2
• Orion: /work/noaa/rtrr/FIX_RRFS2

The "init.sh" script under rrfs-workflow/ush will create correct links based on the current HPC platform.

• The build step will run "init.sh" automatically.
• If cloning the "rrfs-workflow" independently, one needs to run "init.sh" first
• If fix/INIT_DONE exists, it means the local work copy has already initialized fix/.
• setup_exp.py will also run "init.sh".

How to commit text fix files?

git add   
git commit   
git push   

How to commit binary/large fix files?

Follow the following steps:

1. put the binary/large file under FIX_RRFS2/ using the role.rrfsfix role account
   FIX_RRFS2/meshes/conus12km.grid.nc

2. under rrfs-workflow/fix/meshes
   ln -snf ../.agent/meshes/conus12km.grid.nc .

git add fix/meshes/conus12km.grid.nc
git commit
git push

Suggestions:

1. all binary/large files under FIX_RRFS should be read-only so as to prevent accidental deletion or overwrite.
   chmod -w -R meshes/conus12km.grid.nc

2. Add a descriptive suffix to the file name so as to self-describe it as much as possible. For example:
   3drtma_berror_stats_heightDepedent_20200528

Possible situations when modifying fix files:

1. cp: try to overwrite 'testlink', overriding mode 0444 (r--r--r--)?
2. E45: 'readonly' option is set (add ! to override)

Change the FIX_RRFS location

1. modify ush/fix_rrfs_locations.sh
2. ln -snf /to/your/FIX_RRFS fix/.agent (this will be overwritten when init.sh runs)

Don't use a linked directory as we cannot track individual files inside a linked directory.
Instead, create a corresponding directory and link files under that directory.

If we will not track individual files for a group of fix files in rrfs-workflow anyway, we can create a directory link (such as crtm)

How to track binary files

.agent --> a link to FIX_RRFS