Changes to support NUOPC in CESM and OMP bug fixes.

[dabail10]

For detailed information about submitting Pull Requests (PRs) to the CICE-Consortium,
please refer to: https://github.com/CICE-Consortium/About-Us/wiki/Resource-Index#information-for-developers

PR checklist

• [ X] Short (1 sentence) summary of your PR:
  This is the latest NUOPC updates for CESM/CMEPS.
• Developer(s):
  mvertens (Mariana Vertenstein)
• Suggest PR reviewers from list in the column to the right.
• Please copy the PR test results link or provide a summary of testing completed below.
  https://github.com/CICE-Consortium/Test-Results/wiki/cice_by_mach_forks#cheyenne
• How much do the PR code changes differ from the unmodified code?
  • bit for bit
  • different at roundoff level
  • more substantial
• Does this PR create or have dependencies on Icepack or any other models?
  • Yes
  • No
• Does this PR add any new test cases?
  • Yes
  • No
• Is the documentation being updated? ("Documentation" includes information on the wiki or in the .rst files from doc/source/, which are used to create the online technical docs at https://readthedocs.org/projects/cice-consortium-cice/. A test build of the technical docs will be performed as part of the PR testing.)
  • Yes
  • No, does the documentation need to be updated at a later time?
    • Yes
    • No
• Please provide any additional information or relevant details below:
  This is only answer changing in CESM with CMEPS/NUOPC. These changes do not affect CICE standalone ... except a print statement in ice_init.F90 and some changes to the Makefile and cice.build file. Please let me know if these were just changes that were removed from CICE standalone and are creeping back in with the NUOPC stuff. It looks suspicious to me, but I'd like you to review it.
  Update: I have tested within standalone CICE and the NUOPC related changes to not impact the rest of the standalone model. However, I have added some OMP bug fixes to this tag. This now allows three PGI BGC tests to run and pass. For some reason, this does not change answers with intel and gnu.

[apcraig]

Has this been tested in standalone CICE? We want to make sure the Makefile changes (and non cmeps driver changes) don't cause problems in standalone.

[dabail10]

I believe the changes to the Makefile and cice.build were introduced by you for UFS? There is also some ufsapps changes.

[apcraig]

That is correct that I did a bunch of the work, but we should still retest before merging just to be sure there aren't any issues.

[phil-blain]

usually that's ar -rcs no ?

[dabail10]

This is ready to be merged once approved.

[eclare108213]

Maybe this is something that was added in a previous PR and I missed it, but I don't recognize the forapps directory. What is that for? If it's needed, should it be kept under scripts instead of the main directory? Or maybe a driver?

[apcraig]

I am running a full test suite on gordon now.

The forapps directory is something that I added for UFS. I agree it should be under scripts. @dabail10, could you do the following

git mv forapps scripts/

or similar. We'll need to let UFS know of this change so they can also change their scripts. The forapps area is similar to drivers in the sense that it provides a link between the coupled model and CICE for building and/or running the model. CESM doesn't need it, but other models may depending how they're implemented. I don't think we want this in the source code area under drivers.

[apcraig]

Test results are all bit-for-bit on gordon with 4 compilers. https://github.com/CICE-Consortium/Test-Results/wiki/cice_by_hash_forks#73e77746d8204c181a311be8e51c6b3edec75dea