Skip to content

How to run NEUS Atlantis

Jump to bottom
andybeet edited this page Feb 22, 2024 · 31 revisions

Before you attempt run NEUS-Atlantis please ensure that the example model is running on your Windows machine (Please make sure you have followed the instructions in Getting Started before you read further). The example program will run in seconds, NEUS-Atlantis may take 10-12 hours.

Note: In all following content assume docker = podman. They can be used interchangeably

Login to server

  • Log on to the server using windows powershell (recommended on windows) or PuTTY
  • For windows powershell type ssh username@servername at the prompt and enter your password.
  • Navigate to EDAB/atlantis directory.

Running Atlantis in a container (on server)

Setup

Existing versions of the atlantis source code can be found in home/EDAB/atlantis/atlantisCode. Each version will reside in its own folder titled vxxxx where xxxx is the version number. If a version you want to run doesn't exist then you will need to create an image from the source code by following the following steps. Otherwise skip to Run Model

  • Login to server

  • Make a new directory with the name vxxxx, eg. v6536 for atlantis version 6536

  • Copy the atlantis and the .svn directories from the atlantis SVN repository into this new directory.

  • Navigate to the root directory (EDAB/atlantis/)

  • Edit the Dockerfile. Replace all instances of vxxxx with the name of the new directory

  • Create the docker file for this version (don't forget the period!):

    podman build -t atlantis:xxxx .

  • This creates an image called atlantis:xxxx where xxxx is the atlantis version number (tag).

Run Model

  • Login to server

  • Build the image (based on the appropriate code base. Follow the setup guide.

  • Run the container and export model output to folder, for example

    podman run -rm --name user_taskName
    --mount "type=bind,src=$PWD/currentVersion,dst=/app/model"
    --mount "type=bind,src=$PWD/out,dst=/app/model/output"
    atlantis:xxxx

This docker command will mount the model files (in the currentVersion directory) to the app/model directory in the container (called atlantis:xxxx, where xxxx is the version tag). The model will run and the output will be exported to the directory out. Note: the output folder defined in the RunAtlantis.sh shell script must match the name defined in the dst argument. If you encounter problems getting the container to run see common issues section.

Note: The flag -rm removes the container after exiting. This is good practice to clean up exited containers. $PWD$ can be substituted for the full path. Use docker ps -a to view all exited containers remaining in your environment. The flag --name is followed by a text string provided by the user to name the container. Useful for identifying your containers if you have several containers running at once or if you are on a network with others running containers

Running Atlantis in a container (windows PC)

Setup

  • Clone this repository (neus-atlantis)

  • Copy the source code (atlantis folder) from the CSIRO SVN into the root of the cloned repo

  • Copy the .svn folder into the root of the cloned repo

  • Do not copy the Dockerfile or makefile (they already exist in the repo)

NOTE: Ensure that you get the appropriate revision of Atlantis (from the SVN repo). NEUS-Atlantis may or may not utilize the latest version, please check the latest release

Run Model

  • Start docker desktop

  • Open Windows PowerShell and navigate to the root directory of the cloned repo

  • Build the docker image using the Dockerfile (already present in the root) using the following command (don't forget the trailing period!):

    docker build -t atlantis:xxxx .

    where xxxx is the version of Atlantis you intend to use. For example:

    docker build -t atlantis:6536 .

This creates an image called atlantis:6536, with a tag of 6536 (This assumes that the source code used in this build is v6536)

  • Run the container and export model output to folder, for example:

    docker run -rm --name user_taskName
    --mount "type=bind,src=$pwd/currentVersion,dst=/app/model"
    --mount "type=bind,src=$pwd/out,dst=/app/model/output"
    atlantis:6536

  • This will mount the model files to the app/model directory in the container. The model will run and the output will be exported to the out folder. Note: the output folder defined in the RunAtlantis.sh shell script must match the name defined in the dst argument.

    Note: $pwd is a variable that stores the present working directory. It should contain the path to the root directory of the repo. You can replace with the full path if desired

Transferring Files to Server

  • Connect to the server. For windows powershell users type sftp username@servername at the prompt and enter your password. This uses ssh ftp (a secure file transfer protocol).

  • Navigate to directory on the server where you wish to copy from/to using cd command

  • Navigate to directory on the local machine where you wish to copy from/to using lcd command (Note lpwd will print the local working directory)

  • To copy a file from the local machine to the server put filename

  • To download a file from the server to local machine get filename

  • To copy ALL CONTENT from a directory on your local machine to the server use the -r flag. eg. put -r localFolder remoteFolder. The folder has to exist on the remote server for this to copy correctly. All CONTENT in localFolder will be copied to remoteFolder.

  • Conversely to copy from the server to your local machine get -r remoteFolder localFolder

  • Rather than navigating to respective folders on the server and your local machine you can define the full path in your get or put command. eg. get -r remotePath localPath or put localPath remotePath

Note: The order of the argument matters (Always source-destination, from-to)

For additional sftp commands see here

RunAtlantis.sh

  • Contents of RunAtlantis.sh script (v6536, Nov 2020)

    #!/bin/bash

    find /app/model -type f | xargs dos2unix

    atlantisMerged -i neus_init.nc 0 -o neus_output.nc -r at_run.prm -f at_force_LINUX.prm -p at_physics.prm -b at_biology.prm -h at_harvest.prm -e at_economics.prm -s neus_groups.csv -q neus_fisheries.csv -t . -d output

The second line deals with line ending issues. It converts all windows line endings (files found in the app/model folder) to linux format. The third line runs the Atlantis model. The output folder is specified after the -d flag. In this case it is called output (this must match the name specified in the dst argument at run time). The other file names (.prm, .csv, .nc) are input files all found in the currentVersion folder

  • Contents of RunAtlantis.sh script (v6626, May 2022)

    #!/bin/bash

    find /app/model -type f | xargs dos2unix

    atlantisMerged -i neus_init.nc 0 -o neus_output.nc -r at_run.prm -f at_force_LINUX.prm -p at_physics.prm -b at_biology.prm -h at_harvest.prm -e at_economics.prm -m neus_migration.prm -s neus_groups.csv -q neus_fisheries.csv -t . -d output

Note: the addition of the migration.csv term

PLEASE SAVE RunAtlantis.sh WITH UNIX LINE ENDINGS . You can do this using notepad++ (Edit -> EOL Conversions -> Unix (LF)).

Running on windows locally

As of version 6536 of Atlantis, it is not possible to compile the code on Windows. All efforts are now pursued via Containers.

Legacy code:

> atlantismain -i neus_init.nc 0 -o neus_output.nc -r at_run.prm -f at_force_WIN.prm -p at_physics.prm -b at_biology.prm -h at_harvest.prm -e at_economics.prm -s neus_groups.csv -q neus_fisheries.csv -t path-to-repo\currentVersion -d outputFolder 1\>1a.junk 2\>2a.junk

Note: replace outputFolder with the full path to where output from the model should be saved

Running Atlantis

Calibration

Model Components

Model Criteria

Documentation

Clone this wiki locally