
======================
**** INSTALLATION ****
======================
Very little needs to be done to get this set up. The major thing it needs is to know
who you are, along with any extra information your job scheduler might need to know
about you. The most basic installation is to simply run:

    python buildhpc.py

This will get your username from the operating system, and write a file called identity.py
which stores that in a variable for later use. Job submission scripts will use this information.

NOTE: This will use default module versions for openmpi, python, gcc, and the intel compiler.
The assumption is you will use python-2.7.9, gcc-4.9.1, intel-17, and openmpi v2.0.1-intel-17. 
IF THIS IS NOT YOUR ENVIRONMENT, you will need to specify that. Individual module versions can
be specified directly:

    python buildhpc.py gcc=4.9.1 python=2.7.9 intel=intel-17 openmpi=2.0.1-intel-17
    
But they can also be generated automatically if you already have your modules loaded. A "sane"
installation workflow would therefore be to FIRST compile PlaSim, using your desired build
environment, and then without unloading any modules, run buildhpc.py with the 'automod' flag:

    python buildhpc.py automod
    
This will pull the list of loaded modules, and use the versions of GCC, Python, Intel, and 
OpenMPI that you're actually using. 

If you use a system like Slurm which also requires an account name and an email address, you
can build the system with that information by running:

    python buildhpc.py email=you@mail account=whospayingforthis
    
Including these fields is optional and harmless, so you can do this even if your system does
*not* require this information: in that case, those variables will simply never be used. If
you do not provide this fields, the variables will still be defined, but they will contain
empty strings.

-------------------------------------------
* Configuring your PlaSim HPC environment *
-------------------------------------------

Once you've set up your identity information with the above steps, there's one more thing
you'll need to do for basic operation: define a template folder for PlaSim runs. You can specify
any directory you wish for the template as a job-specific argument, but if that's not defined,
the default is to check for a folder called 'clean' (as in, a clean PlaSim configuration). 
Therefore it's a good idea to create such a folder, in the same directory as launcher.py, and 
populate it with all the files you would need for a basic, vanilla, modern Earth PlaSim run.
The files and namelists in this directory will serve as the default configuration, so any 
parameter *not* specified will assume the value it has in this template folder.

If there are special run scripts you want to use beyond those provided, you should put them in
the top-level directory (the same directory as launcher.py). If however there are *other* files
your job requires, such as specific restart files or boundary condition files, you should place 
those in a folder called 'hopper'. If this folder does not exist, it will be automatically created, even if it is not needed.


=================
***** USAGE *****
=================

The major strength of this program is that you can specify the parameters for a large number of
PlaSim runs in a single file. That file is tasks.crwl. The basic format of this file is a
header, followed by a list of jobs which use that header. Examples are given in the default
tasks.crwl file. 

Each header line must begin with a #. After that, the first 4 fields *must* be, in this order,
JOBNAME, STATUS, NCORES, and QUEUE. JOBNAME is the name you want to give your job; every job's
name should be unique. The job will be run in a folder with that name (you don't need to create
it yourself). STATUS is whether or not the job has been initialized--0 if no, 1 if yes. When
you run launcher.py, it will find every job whose status is 0 and either set it up or set it
up and submit it. NCORES is the number of MPI threads the job will use. QUEUE is the name of
the job queue to which you want to eventually or immediately submit the job. After these four
fields, the order of the other fields is mostly irrelevant, though the program will iterate 
through them from left to right, so if you put in any conflicting parameters, only the rightmost
will apply.

To run the program and generate all available jobs, run:

    python launcher.py

If you want to generate the job folders and configuraiton files but not yet submit the jobs,
you can simply run:

    python launcher.py nosub
    
While you can include any namelist parameter with the header field syntax KEY@namelist, there
are a large number of predefined arguments. These are described in the next section.

---------------------------------------------
* Pre-defined parameters and configurations *
---------------------------------------------

* source
    - This should be a string, and it should give the path to the template directory you want
      to use. If left undefined, it will default to "clean"
      
* noutput
    - Switch for writing model output. 0=no, 1=yes
    
* flux
    - Incident stellar flux (insolation) at the planet, in W/m^2.
    
* startemp
    - Mostly unused. Effective temeprature of the parent star, in Kelvins
    
* pCO2
    - CO2 partial pressure in ubars (10^-6 bars)

* pN2
    - N2 partial pressure in bars
    
* year
    - Number of days per year
    
* sidyear
    - Number of days in a sidereal year
    
* solyear
    - Number of days in a solar year

* sidday
    - Number of hours in a sidereal day
    
* solday
    - Number of hours in a solar day
    
* nlevs
    - Number of vertical model levels to use
    
* vtype
    - Which vertical scheme to use. Options are:
        -1: User-defined, through the SIGH parameter in plasim_namelist.
         0: Linear-like (actually a polynomial, but nonlinear behavior is only at bottom and top)
         1: Linear, equidistant
         2: Logarithmic, equidistant
         3: Pseudo-logarithmic; top is pinned and there's extra resolution near the surface
         4: Like 0, but with the top pinned at PTOP.
         
* rotspd
    - Inverse of the planet's period. I.e. 3-day rotation period means ROTSPD=0.3333333.
    
* lockedyear
    - A shortcut for creating a tidally-locked planet. Use the format "DAY/LON0" where DAY is the
      length of the sidereal day (in days) (and sidereal year) and LON0 is the substellar 
      longitude. For example, 15.0/0.0 would mean the orbital and rotational periods are 15 days,
      and the substellar point is at 0 degrees longitude.
      
* restart
    - If this is "none", delete any restart file that was copied over from the template folder.
    - Otherwise, assume that the value is the name of a restart file, and copy it from 'hopper'
      to 'plasim_restart' in the job folder.
      
* gravity
    - The planet's surface gravity, in m/s^2.
    
* radius
    - The planet's radius in Earth radii.
    
* eccen
    - The plane'ts orbital eccentricity.
    
* obliq
    - The planet's axial tilt, in degrees.
    
* vernlon
    - The planet's longitude of vernal equinox (MVELP). This is only relevant if eccen or obliq
      are nonzero. Even then you can probably ignore it.

* nfixorb
    - (0/1 = no/yes) Treat the orbit as fixed, and ignore any precession effects that would 
      otherwise be computed (i.e. Milankovitch cycles)
      
* oroscale
    - Multiplicative factor by which to scale topography; also turns on glacier module
    
* nglacier
    - (1/0 = yes/no) turn on/off the glacier module
    
* glacelim
    - Thickness Limit at which glaciers are formed/removed, in meters

* icesheets
    - Wipe clean some of the surface boundary conditions, and prescribe an initial ice sheet whose
      height is this parameter's argument given in meters.
      
* lowerANTGRN
    - (CURRENTLY BROKEN; requires additional files) Substitute in boundary condition files that
      don't assume that Antarctica and Greenland are covered in km-thick ice sheets.
      
* ncarbon
    - (1/0=yes/no) Switch for the carbon evolution model.
    
* co2evolve
    - (1/0=yes/no) Whether or not CO2 should be evolved year-to-year or handled externally by the
      user (1 = internally, year-to-year).
      
* nsupply
    - (1/0=yes/no) Institute a supplit limit on the weathering rate, due to erosion.
    
* wmax
    - Maximum weathering rate to be used as the supply limit, in nbars/yr (bars/Gyr)

* volcanCO2
    - Volcanic CO2 outgassing rate in nbars/yr (bars/Gyr)
    
* snowmax
    - Maximum surface snow thickness in meters of liquid h2o equivalent (-1 = no limit)
    
* soilalbedo
    - Default albedo to use for land surface (removes albedo boundary condition file)
    
* wetsoil
    - (1/0=yes/no) Make soil albedo dependent on moisture content
    
* soilh2o
    - Soil water capacity in meters of liquid water equivalent

* naqua
    - (1/0=yes/no) Initialize the planet as an aquaplanet (flat surface, no land)
    
* ndesert
    - (1/0=yes/no) Initialize the planet as a desert planet (flat surface, no oceans, no soil h2o)
    
* drycore
    - (1/0=yes/no) Turn off evaporation, so the atmosphere has no water vapor
    
* soilheat
    - Heat capacity of soil, in J/m^3/K. 1 J/kg/K is approximately 1 mJ/m^3/K. The ocean is 4.18e6.
    
* soildepth
    - Multiplicative factor by which to scale the depth of the soil model, which defaults to 5
      layers of 0.4, 0.8, 1.6, 3.2, and 6.4 meters.
      
* nwpd
    - Number of writes per day.
    
* months
    - Number of months to run each PlaSim instance (each time it loops)
    
* rhum_c
    - Multiplicative factor by which to scale the critical relative humidity for cloud formation.
    
* rhum_m
    - The slope to use for the mostly-linear function that describes how the critical relative
      humidity changes throughout the atmosphere
      
* rcrit
    - List with NLEVS elements, giving the critical relative humidity at each model layer.
    
* days
    - Number of days to run each PlaSim instance

* steps
    - Number of timesteps to run each PlaSim instance
    
* o3
    - (1/0=yes/no) Whether not to include the effects of ozone (O3)
    
* ptop
    - Pressure in hPa at which to pin the top of the atmosphere. NOTE THIS ENABLES NEQSIG=3. If you
      want to use a different vertical scheme, you should specify ptop FIRST and then vtype.
      
* timestep
    - Timestep in minutes. This will scale the number of steps per write and the the number of
      steps per snapshot (it will turn snapshots on) so that I/O does not increase with
      smaller timesteps.
    
* rayleigh
    - (1/0=yes/no) Switch for layer-by-layer Rayleigh scattering, as opposed to doing it all in the
      bottom layer.
      
* script
    - The run script to be called from the job submission script. Must be in the top-level
      directory.
      
* postprocessor
    - The burn7 .nl configuration file to use with the postprocessor.
    
* notify
    - Email notification flags. For PBS/Torque, valid arguments are combinations of 'a' (abort),
      'b' (begin), and 'e' (end).
      
* columnmode
    - Up to two arguments, separate by |. Arguments can be:
        static: Turn off horizontal advection
        clear: Turn off cloud diagnostics and feedback
        
* snapshots
    - Number of timesteps per snapshot write.
    
* extra
    - Any additional file in the toplevel directory that should be included.
    
* runyears
    - Number of years to run PlaSim if using most_plasim_run.
    
* fixedlon
    - Fix the sun in the sky at a longitude given in degrees.
    

------------
* Examples *
------------

Below are 3 examples of configurations one may want to use. Each example assumes the source is
in a folder called 'clean'. Each example consists of a header line, followed by a job line.
In actual practice, you would probably have many job lines following the header line, perhaps
varying pN2 or pCO2 or flux or something.

1. Modern Earth, with 410 ubars of CO2, run to energy-balance equilibrium, and notifying only on
   abort. Run with 8 cores on hpq.

# JOBNAME STATUS NCORES QUEUE pCO2 script extra notify
modern-earth 0 8 hpq 410.0 super-equil.sh super_relax.py a

2. Aquaplanet with 8 bars of N2 with a pinned linear-like scheme, layer-by-layer Rayleigh 
   scattering, run to energy-balance, notifying on start, abort, or end, run on 16 cores on sandyq.
   We'll pin the top at 50 hPa and use a 15-minute timestep. pCO2 is 360 ubars.
   
# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep naqua pN2 pCO2 rayleigh script extra notify
aqua-8.0bar 0 16 sandyq 50.0 4 10 15.0 1 8.5 360.0 1 super-equil.sh super_relax.py abe

3. Tidally-locked aquaplanet, 1.3 Earth radii, 4.5 Earth masses, 12-day orbital period, 2.5 bars of 
   pN2 and 10 mbar of CO2, with 800 W/m^2 insolation, no axial tilt, and circular orbit; run to 
   energy-balance equilibrium, and notifying only on abort or end. Using 32 cores on greenq.
   30-minute timestep.
   
# JOBNAME STATUS NCORES QUEUE ptop vtype nlevs timestep radius gravity nfixorb eccen obliq lockedyear year naqua flux pCO2 pN2 script extra notify
locked_superearth 0 32 greenq 100.0 4 10 30.0 1.3 20.086447 1 0.0 0.0 12.0/0.0 36 1 800.0 1000.0 2.5 super-equil.sh super_relax.py ae

        We had to do a little bit of math here, because we need to know the surface 
        gravity, not the mass. The gravity is M/R^3*g_earth, where g_earth in PlaSim 
        is 9.80665 W/m^2. For 4.5 Earth masses and 1.3 Earth radii, we get 20.086447 
        W/m^2.
        
        