Merge branch 'dev_14_15_to_release' into release-clm5.0

From dev014 & dev015: CMIP6 compset modifiers, output usermods & fixes

Bring in all changes from ctsm1.0.dev014 and ctsm1.0.dev015:

From ctsm1.0.dev015:

(1) Support %BGC-CROP-CMIP6DECK and %BGC-CROP-CMIP6WACCMDECK compset
    modifiers, so that we can turn on the necessary options
    (output-related and others) via new CMIP6-specific compsets.

(2) Turn on carbon isotopes in CMIP6 runs (from Erik Kluzek)

(3) Remove setting of CCSM_BGC=CO2A in the cmip6 usermods

(4) Add usermods directories for getting typical extra output that's
    wanted in many cases: output_crop, output_crop_highfreq, output_bgc,
    output_bgc_highfreq, output_sp, and output_sp_highfreq. These can be
    enabled by adding something like '--user-mods-dir output_crop' on
    the create_newcase line (that short-hand works for an I compset; for
    F or B compsets, you need to provide the full path to the usermod
    directory).

(4) Allow holes in the number of history tapes. Holes are cases where,
    for example, we have h0, h1 and h3 tapes, but no h2 tape (because
    there are no fields on the h2 tape). (This is needed for (3).)

(5) Fix reading and writing of 1-d logical global arrays. This fixes
    #24 for real (rather than just preventing an attempt to
    read/write 1-d logical arrays, as was done in the previous 'fix').

(6) Add C13_NBP and C14_NBP diagnostic fields (from Keith Oleson)

(7) Make a bunch of carbon isotope diagnostic fields inactive by default

(8) Don't allow interpolation (use_init_interp) from a case without
    carbon isotopes to a case with carbon isotopes: Due to
    #67, interpolation from a case
    without carbon isotopes to a case with carbon isotopes yields
    incorrect initialization values for the carbon isotopes. Now that
    we're turning carbon isotopes on via some semi-out-of-the-box
    usermods (for cmip6), it is becoming more important to check to make
    sure someone doesn't shoot themselves in the foot this way.

(9) Add tests of the new output usermods as well as of the CMIP6 compset
    modifiers

From ctsm1.0.dev014: Four miscellaneous minor, bit-for-bit bug fixes:

(1) Py3 pylint check and address cime issue ESMCI/cime#2822 (from Jim
    Edwards: #526)

(2) Change uppercase DEBUG variables to lowercase debug (requested by
    Jim Edwards to avoid conflicting with the DEBUG CPP token)
    (Fixes #534)

(3) Remove unnecessary line of code in LunaMod.F90 that was causing
    problems with some compilers due to an uninitialized variable
    (Fixes #322)

(4) Add r8 to 0 constant to fix build issue with XLF compiler (from Jim
    Edwards: #531)

bld/namelist_files/namelist_definition_clm4_5.xml

@@ -1973,6 +1973,16 @@ Flag to use the atmospheric time series of C13 concentrations from natural abund
 Filename with time series of atmospheric Delta C13 data, which use CMIP6 format.  variables in file are "time" and "delta13co2_in_air".  time variable is in format: years since 1850-01-01 0:0:0.0. units are permil.
 </entry>
 
+<!-- BUG(wjs, 2018-10-25, ESCOMP/ctsm#67) -->
+<entry id="for_testing_allow_interp_non_ciso_to_ciso" type="logical" category="clm_isotope"
+       group="clm_inparm" valid_values="" >
+There is a bug that causes incorrect values for C isotopes if running
+init_interp from a case without C isotopes to a case with C isotopes
+(https://github.com/ESCOMP/ctsm/issues/67). Normally, an error-check
+prevents you from doing this interpolation (until we have fixed that
+bug). However, we sometimes want to bypass this error-check in system
+tests. This namelist flag bypasses this error-check.
+</entry>
 
 <!-- permfrost-specific flags    -->
 

cime_config/buildlib

@@ -3,7 +3,7 @@
 """
 build clm library
 """
-import sys, os, time, filecmp, shutil, imp
+import sys, os, shutil, imp
 
 _CIMEROOT = os.environ.get("CIMEROOT")
 if _CIMEROOT is None:

cime_config/buildnml

@@ -3,7 +3,7 @@
 """
 CLM namelist creator
 """
-import sys, os, time, shutil, re, imp, filecmp
+import sys, os, shutil, imp, filecmp
 
 _CIMEROOT = os.environ.get("CIMEROOT")
 if _CIMEROOT is None:
@@ -46,7 +46,6 @@ def buildnml(case, caseroot, compname):
     clm_force_coldstart = case.get_value("CLM_FORCE_COLDSTART")
     lnd_tuning_mode = case.get_value("LND_TUNING_MODE")
     clm_accelerated_spinup = case.get_value("CLM_ACCELERATED_SPINUP")
-    comp_glc = case.get_value("COMP_GLC")
     comp_atm = case.get_value("COMP_ATM")
     lnd_grid = case.get_value("LND_GRID")
     lnd_ncpl = case.get_value("LND_NCPL")
@@ -62,14 +61,6 @@ def buildnml(case, caseroot, compname):
     glc_nec = case.get_value("GLC_NEC")
     mask = case.get_value("MASK_GRID")
 
-    # -----------------------------------------------------
-    # Clear out old data
-    # -----------------------------------------------------
-
-    input_data_list = os.path.join(caseroot,"Buildconf","clm.input_data_list")
-    if os.path.exists(input_data_list):
-        os.remove(input_data_list)
-
     # -----------------------------------------------------
     # Set clmconf
     # -----------------------------------------------------
@@ -133,10 +124,10 @@ def buildnml(case, caseroot, compname):
     if run_type == "branch":
         startfile_type = "nrevsn"
         if clm_force_coldstart == "on":
-           clm_force_coldstart = "off"
-           logger.warning( "WARNING: You've turned on CLM_FORCE_COLDSTART for a branch run_type, which is a contradiction, the coldstart will be ignored\n" +
-                           "  turn off CLM_FORCE_COLDSTART, or set RUN_TYPE=hybrid to get rid of this warning"
-                         )
+            clm_force_coldstart = "off"
+            logger.warning( "WARNING: You've turned on CLM_FORCE_COLDSTART for a branch run_type, which is a contradiction, the coldstart will be ignored\n" +
+                            "  turn off CLM_FORCE_COLDSTART, or set RUN_TYPE=hybrid to get rid of this warning")
+
 
     if (clm_force_coldstart == "on"):
         logger.warning( "WARNING: CLM is starting up from a cold state" )
@@ -168,9 +159,9 @@ def buildnml(case, caseroot, compname):
         usecase = ""
 
     if ( (mask != "null") and (mask != "UNSET") ):
-       gridmask = "-mask %s" %mask
+        gridmask = "-mask %s" %mask
     else:
-       gridmask = ""
+        gridmask = ""
 
     start_ymd = run_startdate.replace('-','')
 
@@ -191,6 +182,13 @@ def buildnml(case, caseroot, compname):
 
     config_cache_file = os.path.join(caseroot,"Buildconf","clmconf","config_cache.xml")
 
+    # -----------------------------------------------------
+    # Clear out old data
+    # -----------------------------------------------------
+
+    if os.path.exists(inputdata_file):
+        os.remove(inputdata_file)
+
     # -----------------------------------------------------
     # loop over instances
     # -----------------------------------------------------

cime_config/config_component.xml

@@ -14,9 +14,9 @@
 
   <!-- Descriptions of all the different valid configurations for different model versions -->
   <description modifier_mode="1">
-    <desc lnd="CLM40[%SP][%CN][%CNDV][%CN-CROP][%CNDV-CROP]"                                                        >clm4.0:</desc>
-    <desc lnd="CLM45[%SP][%SP-VIC][%CN][%CNDV][%CN-CROP][%CNDV-CROP][%BGC][%BGC-CROP][%FATES][%BGCDV][%BGCDV-CROP]" >clm4.5:</desc>
-    <desc lnd="CLM50[%SP][%SP-VIC][%CN][%BGC][%BGC-CROP][%FATES][%BGCDV][%BGCDV-CROP]"                              >clm5.0:</desc>
+    <desc lnd="CLM40[%SP][%CN][%CNDV][%CN-CROP][%CNDV-CROP]"                                                                         >clm4.0:</desc>
+    <desc lnd="CLM45[%SP][%SP-VIC][%CN][%CNDV][%CN-CROP][%CNDV-CROP][%BGC][%BGC-CROP][%FATES][%BGCDV][%BGCDV-CROP]"                  >clm4.5:</desc>
+    <desc lnd="CLM50[%SP][%SP-VIC][%CN][%BGC][%BGC-CROP][%FATES][%BGCDV][%BGCDV-CROP][%BGC-CROP-CMIP6DECK][%BGC-CROP-CMIP6WACCMDECK]">clm5.0:</desc>
     <desc option="SP"              >Satellite phenology:</desc>
     <desc option="CN"              >CN: Carbon Nitrogen model</desc>
     <desc option="CNDV"            >CNDV: CN with Dynamic Vegetation</desc>
@@ -29,6 +29,8 @@
     <desc option="FATES"           >FATES (Functionally Assembled Terrestrial Ecosystem Simulator) Ecosystem Demography model: (experimental)</desc>
     <desc option="BGCDV"           >BGC (vert. resol. CN and methane) with dynamic vegetation:</desc>
     <desc option="BGCDV-CROP"      >BGC (vert. resol. CN and methane) with dynamic vegetation and prognostic crop:</desc>
+    <desc option="BGC-CROP-CMIP6DECK">BGC (vert. resol. CN and methane) with prognostic crop, with modifications appropriate for CMIP6 DECK experiments:</desc>
+    <desc option="BGC-CROP-CMIP6WACCMDECK">BGC (vert. resol. CN and methane) with prognostic crop, with modifications appropriate for CMIP6 WACCM DECK experiments:</desc>
   </description>
 
   <entry id="COMP_LND">
@@ -224,6 +226,19 @@
       precedence over any settings for finidat in the $CASEROOT/user_clm_clm file.</desc>
   </entry>
 
+  <entry id="CLM_USER_MODS">
+    <type>char</type>
+    <valid_values></valid_values>
+    <default_value></default_value>
+    <values match="last">
+      <value compset="_CLM50%[^_]*CMIP6DECK[%_]">$COMP_ROOT_DIR_LND/cime_config/usermods_dirs/cmip6_deck</value>
+      <value compset="_CLM50%[^_]*CMIP6WACCMDECK[%_]">$COMP_ROOT_DIR_LND/cime_config/usermods_dirs/cmip6_waccm_deck</value>
+    </values>
+    <group>run_component_clm</group>
+    <file>env_case.xml</file>
+    <desc>User mods to apply to specific compset matches. </desc>
+  </entry>
+
   <help>
     =========================================
     CLM naming conventions

cime_config/config_compsets.xml

@@ -96,6 +96,18 @@
     <science_support grid="f19_g17"/>
   </compset>
 
+  <!-- Primarily for testing the CMIP6DECK compset option -->
+  <compset>
+    <alias>I1850Clm50BgcCropCmip6</alias>
+    <lname>1850_DATM%GSWP3v1_CLM50%BGC-CROP-CMIP6DECK_SICE_SOCN_MOSART_CISM2%NOEVOLVE_SWAV</lname>
+  </compset>
+
+  <!-- Primarily for testing the CMIP6WACCMDECK compset option -->
+  <compset>
+    <alias>I1850Clm50BgcCropCmip6waccm</alias>
+    <lname>1850_DATM%GSWP3v1_CLM50%BGC-CROP-CMIP6WACCMDECK_SICE_SOCN_MOSART_CISM2%NOEVOLVE_SWAV</lname>
+  </compset>
+
   <compset>
     <alias>I1850Clm50BgcCropCru</alias>
     <lname>1850_DATM%CRUv7_CLM50%BGC-CROP_SICE_SOCN_MOSART_CISM2%NOEVOLVE_SWAV</lname>

cime_config/testdefs/ExpectedTestFails.xml

@@ -5,9 +5,10 @@
     <entry issue="#404"            >FAIL ERP_D_Ld10_P36x2.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-ciso_decStart RUN</entry>
     <entry issue="#158"            >FAIL ERS_Lm20_Mmpi-serial.1x1_smallvilleIA.I2000Clm50BgcCropGs.cheyenne_gnu.clm-monthly RUN</entry>
     <entry issue="#158"            >FAIL ERS_Lm20_Mmpi-serial.1x1_smallvilleIA.I2000Clm50BgcCropGs.cheyenne_intel.clm-monthly RUN</entry>
+    <entry issue="#384"            >FAIL ERP_D_Ld5.f09_g17.I2000Clm50Vic.cheyenne_intel.clm-vrtlay RUN</entry>
     <entry issue="#442"            >FAIL SMS.f10_f10_musgs.I2000Clm50BgcCrop.hobart_pgi.clm-crop RUN</entry>
     <entry issue="#442"            >FAIL SMS_D.f10_f10_musgs.I2000Clm50BgcCrop.hobart_pgi.clm-crop RUN</entry>
-    <entry issue="ESMCI/cime#2804" >FAIL ERS_D_Ln9_P480x3.f19_g16.I2000Clm50SpGs.cheyenne_intel.clm-waccmx_offline MODEL_BUILD</entry>
+    <entry issue="#550"            >FAIL ERS_D_Ln9_P480x3.f19_g16.I2000Clm50SpGs.cheyenne_intel.clm-waccmx_offline COMPARE_base_rest</entry>
   </category>
   <category name="fates">
     <entry issue="NGEET/fates#315">FAIL ERS_Ld60.f45_f45_mg37.I2000Clm45Fates.hobart_nag.clm-Fates COMPARE_base_rest</entry>

cime_config/testdefs/testlist_clm.xml

@@ -984,13 +984,13 @@
       <option name="tput_tolerance">0.5</option>
     </options>
   </test>
-  <test name="ERS_Ly3" grid="f10_f10_musgs" compset="I1850Clm50BgcCrop" testmods="clm/cmip6">
+  <test name="ERS_Ly3" grid="f10_f10_musgs" compset="I1850Clm50BgcCropCmip6" testmods="clm/basic">
     <machines>
       <machine name="cheyenne" compiler="intel" category="aux_clm"/>
     </machines>
     <options>
       <option name="wallclock">01:40:00</option>
-      <option name="comment"  >Include a long ERS test of the cmip6 configuration, though at coarse resolution</option>
+      <option name="comment"  >Include a long ERS test of the cmip6 configuration, though at coarse resolution. This gives a year+ test covering the output_crop usermod, which is something we want: if this is removed, we should add a test of at least a year duration covering the output_crop usermod. This test needs to use init_interp to work, because of adding virtual Antarctica columns (currently the default out-of-the-box setting uses init_interp for this).</option>
     </options>
   </test>
   <test name="ERS_Ly3" grid="f10_f10_musgs" compset="I1850Clm50BgcCrop" testmods="clm/clm50KSinkMOut">
@@ -1492,13 +1492,13 @@
       <option name="wallclock">00:20:00</option>
     </options>
   </test>
-  <test name="SMS_Ld5_D" grid="f09_g16" compset="I1850Clm50BgcCrop" testmods="clm/cmip6">
+  <test name="SMS_Ld5_D" grid="f09_g17" compset="I1850Clm50BgcCropCmip6" testmods="clm/basic">
     <machines>
       <machine name="cheyenne" compiler="intel" category="aux_clm"/>
     </machines>
     <options>
       <option name="wallclock">00:20:00</option>
-      <option name="comment"  >Include a short debug test of the cmip6 configuration at the production resolution</option>
+      <option name="comment"  >This gives a short debug test of the cmip6 configuration as well as a test of the cmip6 configuration at the production resolution, both of which we want. This test needs to use init_interp to work, because of adding virtual Antarctica columns (currently the default out-of-the-box setting uses init_interp for this).</option>
     </options>
   </test>
   <test name="SMS_Ld5_D_P48x1" grid="f10_f10_musgs" compset="IHistClm50Bgc" testmods="clm/monthly">
@@ -1926,4 +1926,43 @@
       <option name="wallclock">00:40:00</option>
     </options>
   </test>
+  <test name="SMS_Lm1" grid="f10_f10_musgs" compset="I1850Clm50BgcCropCmip6waccm" testmods="clm/basic">
+    <machines>
+      <machine name="cheyenne" compiler="gnu" category="aux_clm"/>
+    </machines>
+    <options>
+      <option name="wallclock">00:20:00</option>
+      <option name="comment"  >The main point of this test is simply to make sure that the CMIP6WACCMDECK moifierd works. (This configuration is basically the same as I1850Clm50BgcCropCmip6, but without cmip6_glaciers_virtual_antarctica - so we don't need huge coverage of this.) Month-long so that we actually get some history output (because this test exercises a usermods directory with only monthly and yearly output).</option>
+    </options>
+  </test>
+  <test name="SMS_Lm1_D" grid="f10_f10_musgs" compset="I1850Clm50BgcCrop" testmods="clm/output_crop_highfreq">
+    <machines>
+      <machine name="cheyenne" compiler="intel" category="aux_clm">
+        <options>
+          <option name="wallclock">00:20:00</option>
+          <option name="comment">Want at least a month-long debug test covering the output_crop usermod, as well as a test covering the output_crop_highfreq usermod. (Note that we already have a year+ test of output_crop via a cmip6 test, so having this test just be a month, rather than a year, seems good enough.)</option>
+        </options>
+      </machine>
+    </machines>
+  </test>
+  <test name="SMS_Ly1_Mmpi-serial" grid="1x1_brazil" compset="IHistClm50BgcQianGs" testmods="clm/output_bgc_highfreq">
+    <machines>
+      <machine name="cheyenne" compiler="gnu" category="aux_clm">
+        <options>
+          <option name="wallclock">00:20:00</option>
+          <option name="comment">Want a year-long test covering the output_bgc and output_bgc_highfreq usermods; don't want a highfreq, year-long global test because of the output volume, so this is single-point.</option>
+        </options>
+      </machine>
+    </machines>
+  </test>
+  <test name="SMS_Ly1_Mmpi-serial" grid="1x1_vancouverCAN" compset="I1PtClm50SpGs" testmods="clm/output_sp_highfreq">
+    <machines>
+      <machine name="cheyenne" compiler="gnu" category="aux_clm">
+        <options>
+          <option name="wallclock">00:10:00</option>
+          <option name="comment">Want a year-long test covering the output_sp and output_sp_highfreq usermods; don't want a highfreq, year-long global test because of the output volume, so this is single-point.</option>
+        </options>
+      </machine>
+    </machines>
+  </test>
 </testlist>

cime_config/testdefs/testmods_dirs/clm/basic/README

@@ -0,0 +1,8 @@
+This testmods directory is for tests where we want to use the
+out-of-the-box output options, but still want the minimal changes that
+we apply for nearly all tests.
+
+("default" would probably be a good name for this, but we already use
+"default" to mean "the standard output changes that are applied for most
+tests" (adding some streams and fields, double precision, etc.). We may
+want to consider renaming that one and using "default" for this one.)

cime_config/testdefs/testmods_dirs/clm/ciso/user_nl_clm

@@ -1,2 +1,10 @@
  use_c13        = .true.
  use_c14        = .true.
+
+ ! BUG(wjs, 2018-10-25, ESCOMP/ctsm#67) There is a bug that causes incorrect values for C
+ ! isotopes if running init_interp from a case without C isotopes to a case with C
+ ! isotopes (https://github.com/ESCOMP/ctsm/issues/67). Normally, an error-check prevents
+ ! you from doing this interpolation (until we have fixed that bug). However, we
+ ! sometimes want to bypass this error-check in system tests. This namelist flag bypasses
+ ! this error-check.
+ for_testing_allow_interp_non_ciso_to_ciso = .true.

cime_config/testdefs/testmods_dirs/clm/ciso_flexCN_FUN/user_nl_clm

@@ -2,3 +2,11 @@
  use_c14        = .true.
  use_fun        = .true.
  use_flexibleCN = .true.
+
+ ! BUG(wjs, 2018-10-25, ESCOMP/ctsm#67) There is a bug that causes incorrect values for C
+ ! isotopes if running init_interp from a case without C isotopes to a case with C
+ ! isotopes (https://github.com/ESCOMP/ctsm/issues/67). Normally, an error-check prevents
+ ! you from doing this interpolation (until we have fixed that bug). However, we
+ ! sometimes want to bypass this error-check in system tests. This namelist flag bypasses
+ ! this error-check.
+ for_testing_allow_interp_non_ciso_to_ciso = .true.

cime_config/testdefs/testmods_dirs/clm/cmip6_deck/include_user_mods

@@ -0,0 +1 @@
+../../../../usermods_dirs/cmip6_deck

cime_config/testdefs/testmods_dirs/clm/cmip6_deck/shell_commands

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+./xmlchange BFBFLAG="TRUE"

cime_config/testdefs/testmods_dirs/clm/output_bgc_highfreq/README

@@ -0,0 +1 @@
+Tests the output_bgc_highfreq usermods

cime_config/testdefs/testmods_dirs/clm/output_bgc_highfreq/include_user_mods

@@ -0,0 +1,2 @@
+../../../../usermods_dirs/output_bgc_highfreq
+../basic

cime_config/testdefs/testmods_dirs/clm/output_crop_highfreq/README

@@ -0,0 +1 @@
+Tests the output_crop_highfreq usermods

cime_config/testdefs/testmods_dirs/clm/output_crop_highfreq/include_user_mods

@@ -0,0 +1,2 @@
+../../../../usermods_dirs/output_crop_highfreq
+../basic

cime_config/testdefs/testmods_dirs/clm/output_sp_highfreq/README

@@ -0,0 +1 @@
+Tests the output_sp_highfreq usermods

cime_config/testdefs/testmods_dirs/clm/output_sp_highfreq/include_user_mods

@@ -0,0 +1,2 @@
+../../../../usermods_dirs/output_sp_highfreq
+../basic

cime_config/usermods_dirs/_includes/README

@@ -0,0 +1,2 @@
+This directory contains usermods directories that are meant to be
+included in other usermods directories, not used directly

cime_config/usermods_dirs/_includes/cmip6_carbon_isotopes/user_nl_clm

@@ -0,0 +1,12 @@
+!----------------------------------------------------------------------------------
+! Settings from cmip6_carbon_isotopes
+!----------------------------------------------------------------------------------
+
+use_c13 = .true.
+use_c14 = .true.
+use_c13_timeseries = .true.
+use_c14_bombspike = .true.
+
+! h3 stream (yearly average, gridcell-level)
+hist_fincl4 += 'C14_SOIL1C_vr', 'C14_SOIL2C_vr', 'C14_SOIL3C_vr'
+

cime_config/usermods_dirs/cmip6_glaciers/user_nl_clm → cime_config/usermods_dirs/_includes/cmip6_glaciers_virtual_antarctica/user_nl_clm

@@ -1,5 +1,9 @@
+!----------------------------------------------------------------------------------
+! Settings from cmip6_glaciers_virtual_antarctica
+!----------------------------------------------------------------------------------
 
 ! This differs from the default in that it turns on virtual columns over Antarctica
 ! This is desired so that we have the output needed to drive a later offline CISM Antarctica simulation
 ! However, this increases the cost of CLM by about 10%
 glacier_region_behavior = 'single_at_atm_topo', 'virtual', 'virtual', 'virtual'
+

cime_config/usermods_dirs/_includes/output_base/README

@@ -0,0 +1,6 @@
+This usermods directory provides a common base on which other usermods
+can build for changing the out-of-the-box output to something more
+generally useful.
+
+This should not be used directly. Instead, use one of the other usermods
+directories that builds on this.