Micro and Nano Mechanics Group
Revision as of 16:07, 4 February 2008 by Caiwei (Talk)

This document is taken from ParaDiSInfo.txt in ParaDiS.v2.2 distribution. Written by Gregg Hommes, LLNL.

  1.  Introduction

  2.  Building ParaDiS
      2.1 Directory Structure
      2.2 Compiling
      2.3 Make Files

  3.  Executing ParaDiS
      3.1 Command Line
      3.2 Examples

  4.  Inputs
      4.1 Input Files
      4.2 Control Parameter File
      4.3 Nodal Data File
          4.3.1 Data File Format
          4.3.2 Data File Segments
          4.3.3 Data File Parameters
          4.3.4 Domain Decomposition
          4.3.5 Nodal Data

  5.  Cell and Domain Structure

  6.  Domain Decomposition
      6.1 Selecting a Domain Decomposition Method
      6.2 Recursive Sectioning
      6.3 Recursive Bisection

  7.  Dislocation Mobility
      7.1 Selecting a Mobility Module
      7.2 BCC_0 Mobility
      7.3 BCC_0b Mobility
      7.4 -- TBD
      7.5 FCC_0 Mobility  -- TBD
      7.6 FCC_2 Mobility  -- TBD

  8.  Material Properties 
      8.1 Specifying Material Properties

  9.  Far-Field Dislocation Interactions
      9.1 Fast Multipole for Far-Field Interactions
      9.2 Non-FMM Far-Field Interactions

  10. Discretization (Remesh)
      10.1 Selecting a Discretization Method
      10.2 Remesh version 2
      10.3 Remesh version 3

  11. Simulation Timestepping
      11.1 Timestep Controls
      11.2 Timestep Integrators
           10.2.1  Trapezoid
           10.2.2  Forward Euler

  12. Visualization
      12.1 X-window display
      12.2 Gnuplot
      12.3 Tecplot
      12.4 Povray
      12.5 Terascale Browser  -- TBD
      12.6 Postscript

  13. Output
      13.1 Restart files
      13.2 Property Outputs
           13.2.1 Enabling Properties Output
           13.2.2 Property Files
                13.2.2.1 density
                13.2.2.2 time_Plastic_strain
                13.2.2.3 stress_Plastic_strain
                13.2.2.4 stress_Total_strain
                13.2.2.5 alleps
                13.2.2.6 epsdot
      13.3 Flux decomposition
      13.4 Pole figures  -- TBD
      13.5 Velocity files
      13.6 Segment (arm) files
      13.7 Density Field file
 
  14. Utilities
      14.1 Creating Initial Dislocations with 'paradisgen'
      14.2 Recomputing Domain Boundaries with 'paradisrepart'
      14.3 Converting Control and Data File with 'paradisconvert'
      14.4 Creating the FMM Image Correction Table with 'ctablegen'
      14.5 Creating Far-Field Stress Tables with 'stresstablegen'
 
  15. Tools
      15.1 genPovrayFrames
      15.2 gnuplot2povray
      15.3 stitch
 
  Appendix 1.  Control Parameters



---------------------------------------------------------------------

  1.  Introduction
  ----------------

      The Parallel Dislocation Simulator (ParaDiS) was written
      and developed at LLNL by a team of physicists and computer
      scientists.  ParaDiS is a code for doing Dislocation Dynamics
      simulations and was specifically written to perform well on
      massively parallel computers.  The computational approach to
      Dislocation Dynamics used in ParaDiS is fairly simple.  In
      it, one introduces dislocation lines into the computational
      volume and lets them interact and move in response to the
      forces imposed by external stress and inter-dislocation
      interactions. 


  2.  Building ParaDiS
  --------------------

      2.1 Directory Structure
      ------------------------

          The ParaDiS release consists of a file which, when unzipped
          and untarred creates a primary directory and a structure
          of subdirectories.  For convenience, this primary directory
          will hereafter be referred to as <ParadisDir>.  The directory
          structure of the release will look like:

              <ParadisDir>/            Contains primary 'make' files for
                                       controlling build of all executables

              <ParadisDir>/bin/        This directory will be created during
                                       the 'make' process.  All executables
                                       will be placed in this directory
                                       during compilation

              <ParadisDir>/docs/       Contains any additional documentation

              <ParadisDir>/include/    Contains all C and C++ header files

              <ParadisDir>/inputs/     FMM and non-FMM correction tables,
                                       X-display defaults file, gnuplot
                                       command file, etc.

              <ParadisDir>/materials/  Sample files containing material
                                       specific tables and constants

              <ParadisDir>/src/        All C and C++ source code modules
                                       related to the paradis executable

              <ParadisDir>/tools/      Miscellaneous support scripts

              <ParadisDir>/utilities/  Source code modules pertaining to 
                                       the various support tools
    

      2.2 Compiling
      -------------

          The current ParaDiS 'make' files support compilation for
          a number of pre-defined system types that are defined in
          the file 'makefile.sys'. In order to compile the code on
          one of these pre-defined system types, simply edit
          'makefile.setup' and set the "SYS=..." value to the
          desired system type.  For other system types, add a new
          system type into 'makefile.sys' following the format used
          for the existing systems and, as above, set the "SYS=..."
          value in 'makefile.setup'.

          Executing 'gmake' with no options in <ParadisDir> will
          build the following executables:

            <ParadisDir>/bin/paradis         -- Main parallel application

            <ParadisDir>/bin/paradisgen      -- Utility for creating initial
                                                dislocation configurations
                                                (See Utilities section for
                                                details)

            <ParadisDir>/bin/paradisrepart   -- Utility for generating a new
                                                domain decomposition for an
                                                existing dislocation
                                                configuration (See Utilities
                                                section for details)

            <ParadisDir>/bin/paradisconvert  -- Utility for converting older
                                                format ParaDiS control
                                                parameter files to the current
                                                format (See Utilities section
                                                for details)
 
            <ParadisDir>/bin/ctablegen       -- Utility for creating image
                                                correction tables needed when
                                                FMM code is enabled.  (See
                                                Utilities section for details)

            <ParadisDir>/bin/ctablegenp      -- Parallel version of the 
                                                ctablegen utility.  See above.

            <ParadisDir>/bin/stresstablegen  -- Utility for creating tables
                                                used in calculating far-field
                                                stress if the FMM code is
                                                not enabled.  (See Utilities
                                                section for details)

      2.3 Make files
      --------------

          Compilation of the code is done via 'gmake' and depends
          on the following 'make' files located in the <ParadisDir>
          directory.  (See the individual make files for details)

          makefile        -- This is the primary make file controlling
                             the build of the parallel executable and
                             associated utilities.  

          makefile.sys    -- This file contains, for each supported system
                             type, a set of 'make' macros, definitions,
                             compiler selection, library and include file
                             paths, etc.
                             
          makefile.setup  -- Contains numerous 'make' settings and flags
                             that are not system specific, including
                             settings for system type, optimization level,
                             debugging flags, etc.

  3.  Executing ParaDiS
  ---------------------

      3.1  Command line
      -----------------

           The ParaDiS command line format is:

               paradis [-r <numCycles>] [-d dataFile] <controlFile>

           where:

               <ctrlFile>      Specifies the name of the ParaDiS control
                               parameter file

               -d <dataFile>   Specifies the base name of the file(s)
                               containing the nodal data for the run.
                               If this file name is not supplied, the
                               code looks for a data file named the same
                               as the control file with the file name
                               suffix (if any) replaced with ".data"

               -r <numCycles>  Causes the code to execute a series of
                               <numCycles> cycles during which no
                               force calculations or dislocation movement
                               will occur.   These cycles will be used
                               strictly for load-balancing purposes and
                               will be done before the normal cycles.
                               This can be useful when a simulation
                               is started with a uniform domain decomposition
                               and needs time to converge on an optimal
                               decomposition as would occur when
                               restarting a simulation on a different
                               number of cpus
                            
      3.2  Examples
      -------------

           For example, the following are all valid execution lines with the
           first two being equivalent, the third version using the same control
           file as the first two, but explicitly selecting a different nodal
           data file.

           For example, assuming you had a the control parameter file
           'testrun.ctrl' and the two different nodal data files 'testrun.data'
           and 'testrun2.data', all of the following command lines formats
           are valid.  The first two commands are equivalent, the third
           uses the same control parameter file as the first two, but
           explicitly selects a different nodal data file.
           
               mpirun -np 8 bin/paradis testrun.ctrl
               mpirun -np 8 bin/paradis -d testrun.data testrun.ctrl
               mpirun -np 8 bin/paradis -d testrun2.data testrun.ctrl


           NOTE: The number of processors specified to mpirun (or other
           parallel job initiator applicable to the execting system)
           must match the number of domains specified in the control
           file (i.e. np = numXdoms * numYdoms * numZdoms).


           Several small example problems are included with the
           ParaDiS release.  The control and data files for these
           examples are:

               <ParadisDir>/tests/frank_read_src.ctrl
               <ParadisDir>/tests/frank_read_src.data

               <ParadisDir>/tests/form_binaryjunc.ctrl
               <ParadisDir>/tests/form_binaryjunc.data

               <ParadisDir>/tests/fmm_8cpu.ctrl
               <ParadisDir>/tests/fmm_8cpu.data

               NOTE: These tests require the following tables used
                     for calculating the far-field stresses from distant
                     cells and periodic images of the problem.

                     <ParadisDir>/inputs/Rijm.cube.out
                     <ParadisDir>/inputs/RijmPBC.cube.out
                     <ParadisDir>/inputs/fm-ctab.Ta.600K.0GPa.m2.t5.dat

                     These file *may* have been provided with the source
                     code release.  If not, see the README file in the
                     <ParadisDir>/tests directory for instructions on
                     generating these files.

           The 'form_binaryjunc*' files contain a small configuration that
           will demonstrate the formation of a binary junction from two
           dislocation lines, while the 'frank_read_src*' files contain a
           configuration that demonstrates the behaviour of a frank-read
           source.  Both these examples are single cpu examples.  To run
           these tests, from the main <ParadisDir> directory execute:

               mpirun -np 1 ./bin/paradis ./tests/form_binaryjunc.ctrl
           or
               mpirun -np 1 ./bin/paradis ./tests/frank_read_src.ctrl
          
           The 'fmm_8cpu*' files are just a general demonstration of the
           behavior of a small number of screw dislocations.  This is
           an 8 processor simulation using the Fast Multipole Method
           (FMM) for calculating stress from far-field interactions.

               mpirun -np 8 ./bin/paradis ./tests/fmm_8cpu.ctrl


           Any output from these runs will be placed into corresponding
           subdirectories under the "tests" directory.


  4.  Inputs
  ----------

      4.1  Input Files
      ----------------

           A ParaDiS simulation may be started from scratch, or 
           terminated and then later restarted if the user requested
           periodic dumps of restart files.  In either case, the initial
           or restart data data consists of two files, a control
           parameter file and a nodal data file.  (See section on
           Output for details on restart files)


      4.2  Control Parameter File
      ---------------------------

           The control parameter file consists of data of the form:

               identifier = value
               identifier = [value_list]
           or  identifier = string

           where
               identifier   is the name of a control parameter
               value        is a numeric value associated with the parameter
                            specified by <identifier>
               value_list   is a list of numeric values to be associated
                            with the array specified by <identifier>.
                            This values in this list must be delimited by
                            white-space or line-feeds.
               string       a character string enclosed within either single
                            or double quotes.  The string may not contain
                            a line-feed character.
        
           The identifier names are case-insensitive and may be specified
           as any mixture of upper and lower case.  If the code encounters
           an identifier it does not recognize, the identifier and associated
           value(s) will be ignore and a warning message displayed.

           Any blank lines in the control parameter file will will be
           ignored and additionally, any '#' not contained within quotes
           is treated as the beginning of a comment and causes the remainder
           of the current line to be ignored.

           See the Appendix for a complete list of the recognized control
           parameters as well as a brief description of each.
 

      4.3  Nodal Data File
      --------------------

           4.3.1  Data File Format
           -----------------------

                  The nodal data will be contained in 1 or more file
                  segments, with the information being broken into
                  three sections (described below). The first section
                  is the data file parameters, the second section the
                  domain decomposition, and the third section the nodal
                  data.  The first two sections will be included only
                  in the first file segment.

           4.3.2  Data File Segments
           -------------------------

                  Given a control parameter file 'restart.cn', the
                  associated nodal data file(s) would be named:

                      restart.data[.SEQ] 

                  where  SEQ    is a sequence number included only
                                when the nodal data was written in
                                parallel (i.e. the <numIOGroups> control
                                parameter was greater than 1). <SEQ> will
                                range from zero to <numIOGroups>-1.

                  For example, suppose the <numIOGroups> control parameter
                  was set to 4 during a simulation executed on 16 processors.
                  If the simulation created a restart file set at termination,
                  the following files would be generated and could be used
                  as input to continue the simulation at a later time:

                      <outputDir>/restart/restart.cn
                      <outputDir>/restart/restart.data.0
                      <outputDir>/restart/restart.data.1
                      <outputDir>/restart/restart.data.2
                      <outputDir>/restart/restart.data.3

           4.3.3  Data File Parameters
           ---------------------------

                  The data file parameters make up the first section of
                  the nodal data file and are parsed in the same manner
                  as specified above for the control file parameters.
                  These parameters must preceed the other two types of
                  information in the nodal data file.  NOTE: The values
                  of these parameters are updated within the ParaDiS
                  code as necessary and should not be changed by the user.

                  The recognized data file parameters are:

                      dataFileVersion
                      numFileSegments
                      minCoordinates
                      maxCoordinates
                      nodeCount
                      dataDecompType
                      dataDecompGeometry

           4.3.4  Domain Decomposition
           ---------------------------

                  TBD

           4.3.5  Nodal Data
           -----------------

                  The raw nodal data comprises the third section
                  of the nodal data file.  For each node, there will
                  be a single line of node specific data followed by
                  several lines of segment specific data for each
                  segment associated with the node.  The nodal data
                  consists of:

                      node_tag x_coord y_coord z_coord num_arms constraint

                  The segment specific data consists of:

                      nbr_tag burg_x burg_y, burg_z
                      norm_x norm_y norm_z

                      where
                          *_tag      is a comma delimited pair of numbers
                                     uniquely identifying a node.  The first
                                     number is the ID of the domain owning
                                     the node, the second is the index number
                                     of the node within the owning domain.

                          x_coord,
                          y_coord,
                          z_coord    coordinates of the node

                          num_arms   Number of segments associated with the node

                          constraint Integer value indicating any constraints
                                     placed on the node (i.e.  a constraint
                                     of 7 implies a node fixed in place and
                                     unmovable)

  5.  Cell and Domain Structure
  -----------------------------

      The ParaDiS code is used to simulate cubic systems which
      are simultaneously partitioned into a uniform mesh of
      cubic 'cells' and spatially decomposed into 'domains'.

      The cellular structure is defined by the <nXcells>, <nYcells>
      and <nZcells> control parameters, and is used to determine
      the cut-off distance between direct segment to segment
      dislocation interactions and remote (or far-field) interactions.
      In particular, for a given dislocation segment, the interaction
      between the segment and any other segments in the same cell
      or any of the immediately neighboring 26 cells are calculated
      directly.  Interactions with all segments ooutside that range
      are calculated via a hierarchical Fast Multipole Method or
      by lumping all segments in the remote cell into a
      'super-dislocation' where the group of remote dislocations
      are represented as an expansion of dislocation multipoles.
      (See section on Far-Field Forces for details)

      The type of spatial decomposition used for the simulation is
      selected by the <decompType> control parameter, along with
      the <nXdoms>, <nYdoms> and <nZdoms> parameters defining
      the number of spatial domains in each dimension.  Each 'domain'
      is assigned to a single task (processor) within the simulation.
      (See section on Domain Decomposition for details on the 
      supported types of spatial decomposition)


  6.  Domain Decomposition
  ------------------------

      6.1 Selecting a Domain Decomposition Method
      -------------------------------------------

          ParaDiS simulations are spatially decomposed into a number of
          domains equal to the number of processors on which the simulation
          is being executed.  The code curently supports two types of
          domain decomposition and is selected via the <decompType>
          control parameter.  The type 1 decomposition is a Recursive
          Sectioning decomposition which is the default if no decomposition
          type is explicitly requested.  The type 2 is a Recursive
          Bisection decomposition.  (See descriptions of decomposition
          algorithms below.)

          Since the ParaDiS simulations tend to grow in size and are
          spatially heterogeneous, it is preferable to dynamically
          recalculate the domain decomposition at intervals during the
          simulation in order to rebalance the workload more efficiently.
          The frequency with which the domain decomposition will be
          recomputed is specified by the <DLBfreq> control parameter.

          The domain decomposition will be included in the restart files.
          If the simulation is restarted and the <decompType> control
          parameter selects a decomposition type *other* than the
          decomposition that was used to generate the restart file, the
          old domain decomposition from the restart file will be ignored
          and a new domain decomposition of the selected type will be used.

          Additionally, if a simulation is restarted with a different number
          of domains or a different domain geometry that that which was used
          to create the restart file, the domain decomposition from the
          restart file will be ignored and a new decomposition will be
          generated.  Note: When restarting a large simulation (i.e. thousands
          of processors) in such a way that the old domain decomposition must
          be discarded, it may take some time for the simulation to converge
          on an optimal domain decomposition. (The type 1 decomposition is
          particularly susceptible to this.)  There are two ways to mitigate
          this effect.  The first is to use the 'paradisrepart' utility, the
          second through the use of the "-r" command line option to ParaDiS.
          See comments in other sections pertaining to these capabilities.

      6.2 Recursive Sectioning
      ------------------------

          The Recursive Sectioning algorithm performs a domain
          decomposition over a 3-timestep period.  During the
          first timestep, the entire problem space will be
          sectioned along the X dimension into <nXdoms> slabs
          such that the computational cost of each slab is roughly
          equivalent.  The next timestep, each slab will be
          sectioned alng the Y axis into <nYdoms> columns
          such that the computational cost of each column in a
          slab is roughly equivalent.  On the thrid timestep,
          every column will be sectioned along the Z axis
          into <nZdoms> chunks such that the computational
          cost of each chunk in a column is roughy equivalent.

          As stated above, the frequency with which the domain boundaries
          will be recalulated is controlled by the <DLBfreq> parameter.
          If this parameter is not explicity set, the default frequency
          for recalculating this type of domain decomposition is every
          third timestep.

          NOTE: Due to the fact that this algorithm requires 3
                timesteps to complete a new decomposition, the
                <DLBfreq> value must be no less than 3.


      6.3 Recursive Bisection
      -----------------------

          WARNING: The implementation of this algorithm requires the
                   number of domains in each dimension (as specified
                   by the <nXdoms>, <nYdoms>, and <nZdoms> control
                   parameters) must be a power of 2.

          The Recursive Bisectioning algorithm begins with the entire
          cubic problem space and bisects the space in the X, Y and/or
          Z dimensions into octants quarters or halves (depending on
          the number of domains specified per dimension) such that 
          computational cost of each subpartition is roughly the same.
          The decomposition is then recursively applied to each of the
          subpartitions until no further decomposition is necesssary.

          As stated above, the frequency with which the domain boundaries
          will be recalulated is controlled by the <DLBfreq> parameter.
          If this parameter is not explicity set, the default frequency
          for recalculating this type of domain decomposition is every
          timestep.


  7.  Dislocation Mobility
  ------------------------

      7.1  Selecting a Mobility Module
      ---------------------------------

           One of the crucial aspects of a ParaDiS simulation is the
           selection of the set of rules governing the material specific
           physics such as glide climb, and cross-slip with respect to
           crystallographic constraints.  ParaDiS provides multiple sets
           of rules or "mobility laws", each implemented in a separate
           module, that may be selected via the <mobilityLaw> control
           file parameter.  The currently support modules are:

               "BCC_0"
               "BCC_0b"

           There are additional modules for dealing with FCC type materials
           however these are neither fully developed nor supported at this
           point.  These FCC mobility functions are:

               "FCC_0"
               "FCC_2"

           For a list of the control parameters related to the mobility
           of dislocations, see the "Material and Mobility Parameters"
           section of the Appendix detailing the control file parameters.
           Currently the default values of all mobility related parameters
           correspond to tantalum at a temperature of 600 K and a
           pressure of 0GPa.
    

      7.2  BCC_0 Mobility:
      --------------------

           In BCC metals, screw dislocations do not dissociate into partial
           dislocations the same way they do in FCC metals, therefore, for
           BCC crystals we do not assign glide plane normals to screw
           dislocations.  Instead, screw dislocations are given the same
           mobility in all directions perpendicular to the line.  This
           isotropic mobility for screws mimics the "pencil-glide" behavior
           of dislocations observed in BCC metals at elevated temperatures.
           At the same time, the drag coefficient for non-screw segments
           will remain anisotropic with respect to glide and climb.

           For details on this mobility module, refer to the paper:

               <ParadisDir>/Docs/ParaDiSAlgorithm.pdf


      7.3  BCC_0b Mobility:
      ---------------------

           The BCC_0b mobility module is nearly a duplicate of BCC_0 with
           the exception that the movement of discretization nodes along
           the dislocation lines has been dampened.


      7.4  TBD
      ---------------------


      7.5  FCC_0 Mobility:
      --------------------

           This module attempts to simulate easy glide in FCC systems.  The
           glide plane is limited to one of the [111] planes, and cross-slip
           is not permitted.  Also, no crystallographic information is
           used in the dislocation core reactions and hence junction formation
           can take place even slightly off the zone axis.

           Reminder:  This mobility module is not fully developed or supported.


      7.6  FCC_2 Mobility:
      --------------------

           TBD


  8.  Material Properties
  -------------------------------

      8.1 Specifying Material Properties
      ----------------------------------

          All material specific properties are set via the control
          file parameters.  If not specified, defaults will correspond
          to tantalum at a temperatue of 300 K and pressure of 0GPa.

          See the "Material and mobility parameters" section of 
          Appendix 1 for a list of the material related parameters.


  9.  Far-Field Dislocation Interactions
  --------------------------------------

     9.1 Fast Multipole for Far-Field Interactions
     ---------------------------------------------

         The FMM algorithm is enabled within the code via the <fmEnabled>
         toggle in the control file.  The order of the multipole and
         taylor expansions used by the FMM algorithm are set via the
         <fmMPOrder> and <fmTaylorOrder> control parameters respectively.
         As a general rule, the taylor expansion order should be around
         twice that of the multipole expansion order.  One additional
         control parameter <fmCorrectionTbl> specifies the name of the
         file containing the table used by the FMM code to adjust the
         stress calculated by the FMM for multiple periodic images.

         It is important to note that the file indicated by <fmCorrectionTbl>
         must be built with the same multipole order, taylor expansion order,
         and poisson value (<pois> control parameter) specified for a
         simulation.  An appropriate correction table may be generated
         via the 'ctablegen' utility (See the section on Utilities for
         details on the use of 'ctablegen'), although a correction table
         matching the code's default control parameter settings *may* have
         been provided in the code release as:

             <ParadisDir>/inputs/fm-ctab.Ta.600k.0GPa.m2.t5.dat


         The use of a Fast Multipole Method (FMM) for calculating the
         far-field forces is based on formulae for generating and evaluating
         multipole expansions and a few translation theorems.  A very
         general description can be found below, but for details refer
         to the following paper included with the source release.

             <ParadisDir>/docs/ParaDiSAlgorithm.pdf


         The general FMM algorithm consists of the following steps:

           1: Construct multipole moments: Each domain (CPU) calculates
              the contributions of its dislocation segments to the
              multipole moments of the FMM subcells to which those
              segments belong, and communicates those contributions to the
              domain that owns the FMM subcell.   Each domain that owns
              an FMM subcell sums all the contributions for that subcell
              together.
    
           2: Upward pass: Starting at the lowest layer of the FMM
              hierarchy, each domain collects and sums the contributions
              to the multipole moments for each of the FMM subcells it
              owns from the eight child cells and the calculates the
              upward pass translation of its multipole moments and
              communicates the result to the domain owing the subcell's
              parent, until the top of the hierarchy is reached.

           3: Transverse translations: The multipole moments from 189
              cells that are outside the nearest neighbor distance of
              the target cell but inside the nearest neaighbor distance
              of its parent are collected by the domain owning the target
              cell and their contributions to the Taylor series expansion
              of the stress filed in target is calculated.

           4: PBC correction: The domain that owns the FMM cell at the
              highest level of the FMM hierarchy calculates the Taylor
              series expansion of the stress state due to periodic images
              of the system (using table specified in the <fmCorrectionTbl>
              parameter).

           5: Downward pass: Starting with the highest level of the FMM
              hierarchy, each domain that owns an FMM subcell sums the
              contributions from its parent to its Taylor series expansion
              of the stress from step 3 and then calculates the downward
              pass translation of the stress for each one of its child
              cells and sends the results to the domain owning the child
              subcells until the bottom of the hierarchy is reached.

           6: Each domain that owns a subcell at the lowest level of the
              FMM hierarchy communicates the taylor series expansions of
              the stress field to the domains intersecting the subcell.

    
     9.2 Non-FMM Far-Field Interactions
     ----------------------------------

         The use of the FMM in ParaDiS can be disable by setting the control
         parameter <fmEnabled> to 0.  When FMM is disabled, ParaDiS requires
         additional tables used to factor in the far-field stresses from
         distant cells and periodic images of the system.  The names of the
         files containing the tables are specified via the <Rijmfile> and
         <RijmPBCfile> control parameters.  These tables can be generated
         using the 'stresstablegen' utility (see Utilities section for
         details), although copies *may* have been provided in the code
         release as:

             <ParadisDir>/inputs/Rijm.cube.out
             <ParadisDir>/inputs/RijmPBC.cube.out


         This method essentially lumps all dislocations in a cell into a 
         'super-dislocation' where the cell's dislocations are represented
         as an expansion of the dislocation multipoles.  The remote stress
         for a segment in a given cell is then calculated from two
         components.  The first is the sum of the stresses resulting from
         expansions from all remote cells (i.e. neither the current cell
         nor any of its 26 immediate neighbors) and the stress from all
         periodic images of those remote cells.  The second component
         consists of the stress from only the periodic images of the local
         cells (i.e. the current cell and its immediate neighbors).

         This method is can be faster for smaller simulations but unlike
         the FMM, its performance does not scale well as the size of
         a simulation increases in both dislocation density and number
         of cells.


 10. Discretization (Remesh)
 ---------------------------

     10.1 Selecting a Discretization Method
     --------------------------------------

         The nature of DD simulations is such that the total length
         of dislocation lines can increase several orders of magnitude.
         Hence, rediscretization of the dislocations is an absolutely necessary
         components of ParaDiS simulations.  The goal of the rediscretization
         is to optimize the numerical description of the continuous dsilocation
         line geometry so that a given level of accuracy is achieved with the
         fewest degrees of freedom.  For regions of high curvature, an optimal
         distribution of nodes will be place nodes more closely together than
         in regions of lower curvature.

         The level of accuracy is tied to the control parameters <maxSeg>
         and <minSeg> which define the maximum and minimum segment lengths (in
         units of b) respectively.  The smaller the maximum segment length, the
         higher the accuracy.

         A number of rediscretization version have been developed and
         tested, although only versions 2 and 3 are currently supported.
         The rediscretization version to use can be selected through
         the control parameter <remeshRule>.

     10.2 Remesh Version 2
     ---------------------

         This method achieves rediscrtization through two types of
         operations; mesh coarsening and mesh refinement (removing and
         adding discretization nodes respectively).

         this involved defining both minimum and maximum discretization areas
         (A_min and A_max) based on the simulation minimum and maximum
         segment lengths <minSeg> and <maxSeg>.  Then for each discretization
         node (i.e. a node with no more than two associated segments) a
         discretization area (A_node) is calculated.  This area is defined
         as the triangle with vertices at the node and its two neighbors.
         When the discretization area associated with a node A_node < A_min
         the node is removed (coarsened out).  Conversely, if the area
         A_node > A_max, the local discretization is refined by bisecting
         both segments attached to the node.

         For additional information, refer to the following paper included
         with the ParaDiS source release:
 
             <ParadisDir>/docs/ParaDiSAlgorithm.pdf

     10.3 Remesh Version 3
     ---------------------

         This rediscretization method is identical to type 2 with
         the exception that during mesh refinement the inserted
         nodes are not placed at the exact center of the segment
         being bisected.  Instead, the algorithm treats the three
         initial nodes as if they were on an arc and places the
         new nodes at the center of the segments of the arc.

         The rationale behind this is that in simulations using
         long segments and/or high stress, a new node added at the
         center of a segment in a region of high curvature will
         not be optimally placed.  This new node may immediately
         accelerate quickly toward its optimal position then
         decelerate as it nears that position.  This behavior
         can severly impact the simulatuion timesteps.  By placing
         the new nodes on an arc, the initial stress on the new
         nodes will not be as high, keeping the motion of the node
         similar to that of the surrounding nodes.  Hence, no
         detrimental effects on the timestep.


 11. Simulation Timestepping
 ---------------------------

     11.1 Timestep Controls
     ----------------------

          Two timestepping algorithms are provided in the ParaDiS code.
          The <timestepIntegrator> control parameter is used to select
          the desired algorithm.  The currently supported algorithms
          are:

              "trapezoid"
              "forward-euler"

          The default timestep integrator is the "trapezoid" integrator.
          There are several general control parameters used in conjunction
          with any selected timestep intergrator, and several that are unique
          to specific integrators.  The general control parameters are
          <deltaTT> which specifies the duration of the previous timestep,
          <nextDT> which indicates the desired timestep to attempt the next
          timestep, and <maxDT> which specifies the maximum timestep duration
          permitted.

     11.2 Timestep Integrators
     -------------------------

          11.2.1 Trapezoid
          ----------------

                 This implicit integrator is a mix of the Euler-Forward and
                 Euler-Backward methods.  This method is unconditionally
                 stable, but requires an iterative process that may involve
                 multiple nodal force and velocity calculations.  The
                 additional expense of the multiple calculations, however,
                 is usually offset by the gains from the larger timesteps
                 compared to explicit methods such as the forward euler
                 integrator.

                 The algorithm uses the current nodal velocities to reposition
                 each node, then recalculates forces and velocities for the
                 nodes at their new positions.  A positioning 'error' is
                 calculated for each node based on the current and previous
                 nodal position and velocity.  If the positioning error of
                 any node is greater than the maximum permitted positioning
                 error as defined by the <rTol> control parameter, the
                 timestep will be decremented by a factor specified in the
                 <dtDecrementFact> control parameter and the process starts
                 again.  On the otehr hand, if the positioning error of all
                 nodes is within the tolerance, the timestep is accepted
                 and the current timestep is multipled by the factor in the
                 <dtIncrementFactor> to be used as the initial timestep to
                 attempt the next cycle.

          11.2.2 Forward Euler
          --------------------

                 The forward euler timestep integrator is relatively simple
                 and inexpensive in that it requires only a single calculation
                 of nodal force and velocities per timestep.  Unfortunately,
                 the algorithm is subject to the Courant condition for
                 numerical stability and is limited to relatively small
                 timesteps.  The size of the timestep is controlled by the
                 ratio between the length of the shortest segment and the
                 velocity of the fastest moving node.  Additionally, the
                 <rmax> control parameter defines the maximum distance
                 any node is permitted to move in a single timestep, which
                 further limits the timestep duration.  This 'rmax' distance
                 should be set such that no dislocation segment crosses
                 multiple simulation cells in a single timestep.

                 Note: This timestep integrator is NOT recommended.


 12. Visualization
 -----------------

     12.1 X-window display
     ---------------------

          ParaDiS provides a simple X-window display capability for
          visualization and debugging of small scale simulations as well
          as obtaining certain types of nodal data via the display window.

          Unlike some of the other visualization capabilities, this one 
          must be enabled/disabled via the compile time flag "XLIB_MODE"
          in the file <ParadisDir>/makefile.setup.  The X-window support
          is enabled by setting:

              XLIB_MODE = ON

          and disabled by:

              XLIB_MODE = OFF

          The default behavior is to have the X-window support enabled.

          The <winDefaultsFile> control parameter can be used to specify default
          visualization options and attributes such as view perspective, colors
          and so on.  Unless otherwise specified, this control parameter will
          point to the following defaults file provided with the source release:

             <ParadisDir>/inputs/paradis.xdefaults


          Once the X-window display is initiated, the view can be controlled
          through the following single-key commands:

          Key            Command Description
          ---            -------------------
          <Home>         Restores image to default view
          <Esc>          Terminates X-window display without terminating
                         the simulation
          a              Enable/disable aspect ratio changes.  When enabled,
                         the Arrow keys alter the aspect ratio
          c              Enable/disable slice view of image.  When enabled,
                         Up/Down Arrows control slice position, Left/Right
                         Arrows control slice thickness.
          f              Turn display frame on/off
          p              Pause/restart simulation
          r              Enable image rotation.  When enabled, rotation can
                         be controlled via the mouse or Arrow keys.
          s              Enable display scaling.  When enabled, the Arrow
                         keys control scaling size.
          t              Enable image translation.  When enable,
                         the Arrow keys control translation direction.
          <F10>          Generate a postscript image of the display window.
                         File will be called <outputDir>/YshotNNNN where
                        'NNNN' is a sequence number incremented each time
                         a dump of the display window is generated and
                         <outputDir> is the directory specified by the
                         <dirname> control file parameter.

          Additionally, clicking the mouse on a nodal point in the X-window
          image will cause the following information to be written to the
          terminal device (not the X-window display):

              - relative position (x,y) on the X-window display
              - node ID (domainID,nodeindex)
              - number of segments (arms) associated with the node
              - simulation coordinates (x,y,z) of the node


     12.2 Gnuplot
     ------------

          ParaDiS is capable of producing output files formatted for use
          with the gnuplot visualization package.  This capability is
          enabled via the <gnuplot> toggle in the control parameter file.
          The frequency with which gnuplot files will be generated is
          controlled by the settings of the <gnuplotfreq> and <gnuplotdt>
          parameters.

          When this capability is enabled, the code will periodically
          create a set of gnuplot output files in the directory
          <outputDir>/gnuplot where <outputDir> is the directory
          specified by the <dirname> control file parameter.  The
          naming convention used for these files is:

              <outputDir>/gnuplot/box.in
              <outputDir>/gnuplot/0tNNNN[.SEQ] 

              where NNNN is a sequence number begining at '0001' and 
                         incremented each time a gnuplot file set is
                         written.
                    SEQ  is a sequence number included only when gnuplot
                         files are being written in parallel (i.e. the
                         <numIOGroups> control parameter is greater than 1)
                         <SEQ> will range from zero to <numIOGroups>-1.

          In addition, at program termination, an extra set of gnuplot
          files will be created.  This set will be generated regardless
          of the specified dump frequency in order to provide an image
          of the final configuration. This set will be named with the
          slightly different naming convention:
 
              <outputDir>/gnuplot/gnuplot.final[.SEQ]


          The 'box.in' data file will contain data for plotting the boundaries
          of the simulation space, while the '0t*' files contain the coordinate
          pairs defining each unique dislocation segment.

          NOTE: When gnuplot data is being written in parallel
          (i.e. <numIOGroups> is larger than 1), the dislocation segment
          data is spread over the file segments.  These file segments
          can be combined into a single file for display in gnuplot
          via the <ParadisDir>/tools/stitch tool.  (See description of
          'stitch' tool for details.)


          To aid in visualizing the dislocation configuration, a file
          containing commands to set some useful gnuplot options has
          been provided;  This file is located at:

              <ParadisDir>/inputs/gnuplot.cmds

          For example, to view the dislocation structure contained in the
          gnuplot data file 0t0001 you could start gnuplot interactively
          and execute the commands:

              gnuplot> load '<ParadisDir>/inputs/gnuplot.defaults'
              gnuplot> splot 'box.in' with lines, '0t0001' w lines


          A second gnuplot command file has been provided as an example
          of displaying a sequence of gnuplot files.  This file is
          located at:

              <ParadisDir>/inputs/gnuplot.movie

          This file has the neccesary commands to plot the sequence
          of files beginning with 0t0001 and ending with 0t1000.  After
          starting gnuplot, simply load this file with:

              gnuplot> load '<ParadisDir>/inputs/gnuplot.movie'

          This will initiate the plot of the first file, and thereafter,
          simply hitting return will cause gnuplot to move to the next
          file in the sequence.
          

          For details on using gnuplot, refer to the gnuplot manual or
          enter 'help' from the gnuplot interactive prompt.


     12.3 Tecplot
     ------------

          ParaDiS is capable of producing output files formatted for use
          with the commercial visualization package Tecplot.  This
          capability is enabled via the <tecplot> toggle in the control
          parameter file.  The frequency with which tecplot files will
          be generated is controlled by the settings of the <tecplotfreq>
          and <tecplotdt> parameters.

          When this capability is enabled, the code will periodically
          create a set of tecplot output files in the directory
          <outputDir>/tecplot where <outputDir> is the directory
          specified by the <dirname> control file parameter.  The
          naming convention used for these files is:

              <outputDir>/tecplot/tecdataNNNN[.SEQ] 

              where NNNN is a sequence number begining at '0001' and 
                         incremented each time a tecplot file set is
                         written.
                    SEQ  is a sequence number included only when tecplot
                         files are being written in parallel (i.e. the
                         <numIOGroups> control parameter is greater than 1)
                         <SEQ> will range from zero to <numIOGroups>-1.

          In addition, at program termination, an extra set of tecplot
          files will be created.  This set will be generated regardless
          of the specified dump frequency in order to provide an image
          of the final configuration. This set will be named with the
          slightly different naming convention:

              <outputDir>/tecplot/tecdata.final[.SEQ]


          The generated tecplot output has specific headers to assign the
          variables and time frames (zones) and should look something like
          this:

          ...
          variables = X,Y,Z,V1,V2,V3,V4,V5,V6,V7,V8
          zone i = 54  F=POINT
          -4000.0  500.0  6000.0  133.1  612.4 -612.3 0.0000  0.0000 0.0000 1 3
          -3866.9 1112.4  5387.7 -133.1 -612.4  612.3 0.0000  0.0000 0.0000 1 3
          471.0  -5132.6 11632.3 -1712.7 -48.8  48.9  0.0090 -0.0918 0.0918 2 3
          ...

          The first line defines the variables to reconstruct the dislocation
          configuration.  X Y and Z specify the nodal coordinates,  V1 V2 and
          V3 define arm vectors to the neighboring nodes.  These 6 variable
          are used to represent dislocation segments as combinations of points
          and vectors in tecplot.  The V4 V5 and V6 represent the nodal velocity
          vector, V7 indicates the number of segments associated with the node
          and V8 indicates a burgers vector type.


          NOTE: When tecplot data is being written in parallel
          (i.e. <numIOGroups> is larger than 1), the headers will only be
          included in the first file segment with the remainder of the data
          spread over the file segments.  These file segments can be combined
          into a single file for display in tecplot via the
          <ParadisDir>/tools/stitch tool.  (See description of 'stitch'
          tool for details.)


          For details on using tecplot, refer to the tecplot manual or select
          the 'Help' menu in tecplot.


     12.4 Povray
     -----------

          If the <povray> toggle is set in the control file,
          ParaDiS will periodically generate files containing
          dislocation segment data formatted for use with the POVRAY
          (Persistence of Vision(tm) Ray Tracer) tool.  The frequency
          with which these files will be generated is controlled by
          the settings of the <povrayfreq> and <povraydt> parameters.

          Note:  These files contain the main data to be processed by
                 povray, but must be post-processed via the
                 'genPovrayFrames' tool (See section on Tools)
                 which will create the final povray input file
                 containing the segment data embedded within a 
                 proper framework of povray settings and commands.

          When this capability is enabled, the code will periodically
          create a set of files in the directory <outputDir>/povray
          where <outputDir> is the directory specified by the <dirname>
          control file parameter.  The naming convention used for
          these files is:

              <outputDir>/povray/povframeNNNN[.SEQ] 

              where NNNN is a sequence number begining at '0001' and 
                         incremented each time a povray frame file
                         set is written.
                    SEQ  is a sequence number included only when povray
                         files are being written in parallel (i.e. the
                         <numIOGroups> control parameter is greater than 1)
                         <SEQ> will range from zero to <numIOGroups>-1.

          In addition, at program termination, an extra set of povray
          frame files will be created.  This set will be generated
          regardless of the specified dump frequency in order to provide
          an image of the final configuration. This set will be named
          with the slightly different naming convention:
 
              <outputDir>/povray/povray.final[.SEQ]


          NOTE: When povray data is being written in parallel
          (i.e. <numIOGroups> is larger than 1), the dislocation segment
          data is spread over the file segments.  If desired, these file
          segments may be combined into a single file for post-processing
          via the <ParadisDir>/tools/stitch tool.  (See description
          of 'stitch' tool for details.)


          As stated above, these files must be post-processed via the
          'genPovrayFrames' tool (See section on Tools) which will
          create the final povray input file containing the segment
          data embedded within a proper framework of povray settings
          and commands.  This tool is capable of dealing with single
          or segmented 'povframe*' files.


          For details on using povray, refer to the povray manual.


     12.5 Terascale Browser
     ----------------------

          TBD

     12.6 Postscript
     ---------------

          If the <psfile> toggle is set in the control file,
          ParaDiS will periodically generate a postscript file
          containing an image of the current state of the system.
          The frequency with which the postscript files are
          written is determined by the settings of the
          <psfilefreq> and <psfiledt> control file parameters.
          All files will be located in the directory <outputDir>
          where <outputDir> is the directory specified by the
          <dirname> control file parameter.

          All files will be named with the following convention:

              <outputDir>/YshotNNNN.ps

          where NNNN is a sequence number beginning with zero and incremented
                     each time another postscript file is written.


 13. Output
 ----------

     13.1 Restart files
     ------------------

          Periodic creation of ParaDiS restart files is enabled via
          the <savecn> toggle in the control file.  The frequency with
          which restart files are written is determined by the settings
          of the <savecnfreq> and <savecndt> control file parameters.

          When enabled, the code will periodically generate restart
          file which will be located in the directory <outputDir>/restart
          where <outputDir> is the directory specified by the <dirname>
          control file parameter.
           
          The naming convention used for these restart files is:

              <outputDir>/restart/rsNNNN             # control parameter file
              <outputDir>/restart/rsNNNN.data[.SEQ]  # nodal data file

              where NNNN is a sequence number begining at '0001' and 
                         incremented each time a restart file set is
                         written.
                    SEQ  is a sequence number included only when restart
                         files are being written in parallel (i.e. the
                         <numIOGroups> control parameter is greater than 1)
                         <SEQ> will range from zero to <numIOGroups>-1.

          Each time a restart file set is written, the name of the
          control parameter file will also be written into the file:

              <outputDir>/latest_restart

          Additionally, when creation of restart files is enabled, an
          additional restart file pair named 'restart.cn' and restart.data'
          will be written at program termination even if the cycle is not
          a multiple of the <savecnfreq> control parameter.

     13.2 Property Outputs
     ----------------------

          13.2.1 Enabling Properties Output
          -------------------------------

               The code has the capability to produce files containing
               various dislocation properties.  This capability is enabled
               via the <saveprop> toggle in the control file.  The
               frequency with which the property data is written is
               determined by the settings of the <savepropfreq> and
               <savepropdt> control file parameters. 

               When this capability is enabled, the code will generate 
               and periodically append the properties data to one or more of
               the following files which will be located in the directory
               <outputDir>/properties, where <outputDir> is the directory
               specified by the <dirname> control file parameter.

          13.2.2 Property Files
          ---------------------

               13.2.2.1 density
               ----------------

                      This file contains the following columns of data:

                      1:   plastic strain
                      2:   strain
                      3:   dislocation density
                      4:   deleted dislocation density (lost through
                           annihilation, junction formation, etc)
                           (density lost since last dump of data)
                      5:   average dislocation velocity++
                      6:   std. deviation of dislocation velocities++
                      7:   density file version number
                      8-*: dislocation density for segments of specific
                           groupings of burgers vectors (see below)

                      ++ These items will be calculated only if
                         the "VEL_STATISTICS" pre-processor macro was
                         defined during compilation.  Otherwise these
                         values will be zeroes.

                      For BCC simulations:
                        Column     Burgers vector types
                          8         [ 1 1 1] [-1-1-1]
                          9         [-1 1 1] [ 1-1-1]
                          10        [ 1-1 1] [-1 1-1]
                          11        [ 1 1-1] [-1-1 1]
                          12        [ 1 0 0] [-1 0 0] [ 0 1 0]
                                    [ 0-1 0] [ 0 0 1] [ 0 0-1]

                      For FCC simulations:
                        Column     Burgers vector types
                          8         [ 1 1 0] [-1-1 0]
                          9         [-1 1 0] [ 1-1 0]
                          10        [ 1 0 1] [-1 0-1]
                          11        [-1 0 1] [ 1 0-1]
                          12        [ 0 1 1] [ 0-1-1]
                          13        [ 0-1 1] [ 0 1-1]
                          14        all others

               13.2.2.2 time_Plastic_strain
               ----------------------------

                      This file contains the following two columns of data:

                        1: elapsed simulation time
                        2: plastic strain

               13.2.2.3 stress_Plastic_strain
               ------------------------------


                      This file will only be generated if the user specified
                      a loadType of 1 (constant strain rate) or 4 (cyclic
                      loading).  The file contains the following two columns
                      of data:

                        1: plastic strain
                        2: stress

               13.2.2.4 stress_Total_strain
               ----------------------------

                      This file will only be generated if the user specified
                      a loadType of 1 (constant strain rate) or 4 (cyclic
                      loading).  If the user specified a constant strain
                      rate, this file contains:

                        1: strain
                        2: stress

                      If the user specified cyclic loading, this file contains:

                        1:  net accumulated strain
                        2:  stress
                        3:  elapsed simulation time
                        4:  number of loading cycles

               13.2.2.5 alleps
               ---------------

                      This file contains the following 9 columns of data:
  
                        1:   simulation timestep number
                        2:   elapsed simulation time
                        3-8: plastic strain components*
                        9:   dislocation density

                        * from plastic strain tensor matrix, elements [0][0],
                          [1][1], [2][2], [1][2], [0][1], and [0][2]

               13.2.2.6 epsdot
               ---------------

                      This file contains the 2 columns of data:

                        1: elapsed simulation time
                        2: plastic strain rate


     13.3 Flux Decomposition
     -----------------------

          ParaDiS has the capability to produce files containing
          flux decomposition information for the simulation.
          This capability is enabled via the <fluxfile> control
          file toggle.  The frequency with which the flux
          data is written is determined by the settings of
          the <fluxfreq> and <fluxdt> control file parameters.

          When this capability is enabled the code will generate
          and periodically append the flux data to two sets of
          files that will be located in the directory
          <outputDir>/fluxdata where <outputDir> is the directory
          specified by the <dirname> control file parameter.

          Each set contains four files, one file per burgers vector.
          Files *b1, *b2, *b3, and *b4 contain statistics for burgers
          vector types [111], [-111], [1-11], and [11-1] respectively.


          The first set of files (Ltot_b1, Ltot_b2, Ltot_b3 and
          Ltot_b4) consist of the following 9 columns of data:

              1: Plastic strain
              2: Strain,
              3. Screw density
              4. Edgedensity1
              5. Edgedensity2
              6. Edgedensity3
              7. Sum of edge Densities (4+5+6)
              8. Total system EdgeDensity from all Ltot files
              9. Total system screw density from all Ltot files


          The second set of files (fluxtot_b1, fluxtot_b2, fluxtot_b3 and
          fluxtot_b4) consist of the following 9 columns of data:

              1:   Plastic Strain
              2:   Strain
              3:   Flux due to climb motion
              4-6: Flux created by edge components on three planes
              7-9: Flux created by screw dislocation on three planes

              for b1 = 1/2[111]
                4: (0 1 -1),[-2 1 1]
                5: (-1 0 1),[1 -2 1]
                6: (1 -1 0),[1 1 -2]
                7: (0 1 -1),[-2 1 1]
                8: (-1 0 1),[1 -2 1]
                9: (1 -1 0),[1 1 -2]

              for b2 = 1/2[-111]
                4: (0 1 -1),[2 1 1]
                5: (1 0 1),[1 2 -1]
                6: (1 1 0),[1 -1 2]
                7: (0 1 -1),[2 1 1]
                8: (1 0 1),[1 2 -1]
                9: (1 1 0),[1 -1 2]

              for b = 1/2[1-11]
                4: (0 1 1),[2 1 -1]
                5: (1 0 -1),[1 2 1]
                6: (1 1 0), [-1 1 2]
                7: (0 1 1),[2 1 -1]
                8: (1 0 -1),[1 2 1]
                9: (1 1 0), [-1 1 2]

              for b = 1/2[11-1]
                4:(0 1 1), [2 -1 1]
                5:(1 0 1), [-1 2 1]
                6:(1 -1 0), [1 1 2]
                7:(0 1 1), [2 -1 1]
                8:(1 0 1), [-1 2 1]
                9:(1 -1 0), [1 1 2]

     13.4 Pole figures
     -----------------

          TBD

     13.5 Velocity files
     -------------------

          If the <velfile> toggle is set in the control file,
          ParaDiS will periodically generate a set of files containing
          velocity information about each unique dislocation node 
          in the system.  The frequency with which the velocity
          data is written is determined by the settings of the
          <velfilefreq> and <velfiledt> control file parameters.
          All files will be located in the directory <outputDir>/velocity
          where <outputDir> is the directory specified by the
          <dirname> control file parameter.

          All files will be named with the following convention:

              <outputDir>/velocity/velNNNN[.SEQ]

          where NNNN is a sequence number based on the <velfilecounter>
                     control parameter and is incremented each time another
                     velocity file is written.
                SEQ  is a sequence number included only when velocity data
                     is being written in parallel (i.e. <numIOGroups>
                     control parameter is greater than 1.  SEQ will
                     range from zero to <numIOGroups>-1.

          In addition, at program termination, an extra set of velocity
          files will be created.  This set will be generated regardless
          of the specified dump frequency in order to provide an image
          of the final configuration. This set will be named with the
          slightly different naming convention:
 
              <outputDir>/velocity/vel.final[.SEQ]


          NOTE: When velocity data is being written in parallel
          the dislocation segment data is spread over the file segments.
          These file segments can be combined into a single file via
          the <ParadisDir>/tools/stitch tool.  (See section on Tools
          for details)


          The contents of the velocity files consist of 4 columns
          of data for each unique dislocation segment.  Those
          columns are:

              1-3:  Velocity components (x,y,z)
              4:    +1 if contributing to strain rate
                    -1 if moving in opposite direction


     13.6 Segment files
     ------------------

          If the <armfile> toggle is set in the control file,
          ParaDiS will periodically generate a set of files containing
          information about each unique dislocation segment 
          in the system.  The frequency with which the segment
          (or arm) files are written is determined by the settings
          of the <armfilefreq> and <armfiledt> control file parameters.
          All files will be located in the directory <outputDir>/armdata
          where <outputDir> is the directory specified by the
          <dirname> control file parameter.

          All files will be named with the following convention:

              <outputDir>/armdata/armNNNN[.SEQ]

          where NNNN is a sequence number based on the <armfilecounter>
                     control parameter and is incremented each time another
                     arm file is written.
                SEQ  is a sequence number included only when arm data
                     is being written in parallel (i.e. <numIOGroups>
                     control parameter is greater than 1.  SEQ will
                     range from zero to <numIOGroups>-1.

          In addition, at program termination, an extra set of arm
          files will be created.  This set will be generated regardless
          of the specified dump frequency in order to provide an image
          of the final configuration. This set will be named with the
          slightly different naming convention:
 
              <outputDir>/armdata/arm.final[.SEQ]


          NOTE: When arm data is being written in parallel
          the dislocation segment data is spread over the file segments.
          These file segments can be combined into a single file via
          the <ParadisDir>/tools/stitch tool.  (See section on Tools
          for details)


          The contents of the arm files consist of 10 columns
          of data for each unique dislocation segment.  Those
          columns are:

              1-3:  Burgers vector components
              4-6:  Line direction vector
              7:    Segment length
              8-10: Coordinates of node 'owning' the segment


     13.6 Density Field file
     -----------------------

          If all components of the <savedensityspec> control parameter
          are set to positive values in the control file, ParaDiS
          will (at program termination) create a file containing a
          3D dislocation density field formatted for use with the
          VASP Data Viewer (vaspview).  The three components of
          the <savedensityspec> parameter specify the granularity
          of the density field in the X, Y and Z dimensions
          respectively.   The density field will be written to the
          file:

              <outputDir>/densityfield.out

          Warning: This will overwrite any existing density field
                   file of the same name!


          The VASP Data Viewer is publicly available on the web
          for non-commercial use.  For details on the product,
          download the VASP Data Viewer from the web and refer
          to the accompanying documentation.


 14. Utilities
 -------------

     14.1 Creating Initial Dislocations with 'paradisgen'
     ----------------------------------------------------

          The paradisgen utility is capable of generating initial
          dislocations suitable for a paradis simulation.  The command
          line options for paradisgen control the type of dislocations,
          number of dislocations, size of the simulation box, and so
          on.  See "Examples" below for sample executions of this utility.


          The command line format for paradisgen is:

              paradisgen [-cubel <cubelength>] [-help] [-hexsize <size>]  \
                         [-maxseg <maxseglen] [-nchains <numchains>]      \
                         [-nloops <numloops>] [-outfile <filename>        \
                         [-radius <radius>] [-seed <seedval>]             \
                         [-type <configtype>]


          All command line options may be abbreviated to the shortest
          non-ambiguous abbreviation of the option.  Descriptions of
          the command lines options are as follows:

            -cubel <cubelength>  Defines the size of the simulation cube
                                 in units of b.  Minimum and maximum problem
                                 space coordinates will be 0.5*cubel and
                                 -0.5*cubel respectively.  If not specified
                                 this will default to 35000b (approx.
                                 10 micron).

            -help                Causes the utility to display the command line
                                 format and option descriptions then terminate.

            -hexsize <size>      Defines the radius (in units of b) for the
                                 hexagonal interstitial loops.  This option is
                                 used only when the selected dislocation type
                                 is 'fcc-irrad'.

            -looptype <ltype>    Specifies the types of prismatic loops to
                                 generate.  If not specified, this value
                                 will default to zero.  Valid values are:

                                   0 == mixture of [1 1 1] and [1 0 0] types
                                   1 == all [1 1 1] type loops
                                   2 == all [1 0 0] type loops

                                 This option is only used when the selected
                                 dislocation type is 'prismatic-loop'.

            -maxseg <length>     Defines the maximum desired segment length
                                 in units of b.  Initial dislocations will
                                 be discretized so no segment is longer than
                                 this value.  If not specified this value
                                 defaults to 500.

            -nchains <numchain>  Indicates the number of dislocation lines
                                 to create.  This value is ignored when the
                                 selected dislocation type is 'prismatic-loop',
                                 and when the type is 'fcc-perfect-loop',
                                 this value must be a multiple of 12.  If
                                 not specified, this value defaults to 2.

            -nloops <numloops>   Specifies the number of loops to create when
                                 the selected dislocation type is
                                 'prismatic-loop' or 'fcc-irrad'.  If not
                                 specified, this value defaults to 1000.

            -nsegs <numsegs>     Specifies the number of segments into which
                                 a prismatic loop will be discretized.  This
                                 option is only used when the selected
                                 dislocation type is 'prismatic-loop'.  If
                                 not specified, this value defaults to
                                 (2*PI*radius)/(maxseg/2).

            -outfile <filename>  Name of the ParaDiS data file to be created.
                                 If not specified, data will be written
                                 into a file called 'paradis.data'.

            -radius <radius>     Radius (in units of b) of prismatic loops.
                                 This option is only used when the selected
                                 dislocation type is 'prismatic-loop'.  If
                                 not specified, this value defaults to maxseg/2.

            -seed <seed>         Specifies a value to seed the random number
                                 generator.  Using the same seed value for
                                 multiple executions of paradis (for which
                                 all other options are identical) insures
                                 creation of an identical configuration.  

            -type <type>         Specifies the type of dislocations to create.
                                 This type can be a name or the integer value
                                 corresponding to the name.  Valid types are:

                                 0 or 'screw'        BCC screw dislocations.
                                                     Assumes periodic boundary
                                                     conditions are enabled.
                                 1 or 'finite-mixed' A mixture of BCC screw and
                                                     edge dislocations
                                                     terminating at the edges
                                                     of the cubic problem
                                                     space.  This should not
                                                     be used with periodic
                                                     boundary conditions.
                                 2 or 'prismatic-loop'   BCC prismatic loops.
                                                     Assumes periodic boundary
                                                     conditions are enabled.
                                 3 or 'fcc'          FCC dislocations. Assumes
                                                     periodic boundary
                                                     conditions are enabled.
                                 4 or 'fcc-irrad'    Mixture of FCC
                                                     dislocations and hexagonal
                                                     interstitial loops.
                                                     Assumes periodic boundary
                                                     conditions are enabled.
                                 5 or 'fcc-perfect-loop' Not fully implemented.

          Examples:
          ---------

              TBD


     14.2 Recomputing Domain Boundaries with 'paradisrepart'
     -------------------------------------------------------

          The paradisrepart utility provides a mechanism by which to
          replace the domain decomposition/partitioning in an
          existing nodal data file with a new domain decomposition.
          The utility will read the current nodal data, and use the
          specified domain/cell geometries to attempt to partition
          the domains such that the computational cost for each domain
          will be roughly equivalent.

          This utility is primarily used when it is necessary alter
          the domain geometry or domain count for a simulation in
          order to continue.  In such a situation, the existing
          domain decomposition would be thrown away and the ParaDiS
          code would generate an initial uniform decomposition
          and then over time converge on a more optimal decomposition.
          Using this utility instead to generate a new decomposition
          provides a much more reasonable starting point from allowing
          the ParaDiS simulation to converge on an optimal decomposition
          much more quickly.


          The command line format for the paradisrepart utility is:

              paradisrepart -infile <inputFile>                           \
                            -cells <xcells[,ycells,zcells]>               \
                            -domains <xdoms[,ydoms,zdoms]>                \
                            [-outfile <outputFile>]  [-decompType <type>] \
                            [-help]
    

          All command line arguments and options may be abbreviated to the
          shortest non-ambiguous abbreviation of the option.  Descriptions
          of the command lines options are as follows:


              -infile <inputFile>    Specifies the name of the nodal data file.
                                     On success, the first nodal data file
                                     segment will be copied to <inputFile>.bkup.
                                     This command line argument is not optional.

              -cells <xcells[,ycells,zcells]>  Specifies the number of
                                     cells in each dimension.  If the
                                     number of cells in the Y and Z dimensions
                                     are not provided, they will default to the
                                     same value as <xcells>. This command line
                                     argument is not optional.

              -domains <xdoms[,ydoms,zdoms]> Specifies the number of domains
                                     in each dimension.  If the number of
                                     domains in the Y and Z dimensions are
                                     not provided, they will default to the
                                     same value as <xdoms>.  This command line
                                     argument is not optional.

              -decompType <type>     Specifies type of domain decomposition:
                                     If not specified, defaults to 1.  Valid
                                     types are:

                                       1 == Recursive Sectioning
                                       2 == Recursive Bisectioning

              -help                  Causes the utility to display the command
                                     line format and option descriptions then
                                     terminate.

              -outfile <outputFile>  Specifies the name of the file into which
                                     to write the new domain decomposition.  If
                                     not specified, new domain decomposition
                                     will be written into the file specified
                                     by <inputFile>.

          Examples:
          ---------


     14.3 Converting Control and Data File with 'paradisconvert'
     -----------------------------------------------------------

          The paradisconvert utility provides a mechanism by which
          older format ParaDiS control and data files (and restart
          files) may be converted to the current file formats.
          This utility does recognize and handle segmented data files.

          In most cases this utility will not be required since
          ParaDiS is still able to recognize and handle *most* of 
          the older file formats.  The only format the ParaDiS code
          no longer recognizes is the truly ancient format associated
          with the earliest incarnations of the code in which the
          control file parameters and nodal data were included in
          a single file.  For these control files, the paradisconvert
          utility will be needed.


          The command line format for the paradisconvert utility is:

              paradisconvert  <controlFile> [dataFile]

          where:

              controlFile   Specifies the name of the control parameter file
                            to be updated to the current format.  

              dataFile      If the specified control file is the original
                            ancient format including control parameters and
                            nodal data in the single file, this argument is 
                            ignored.  Otherwise, this specifies the base name
                            of the nodal data file(s) to be converted.  If not
                            this argument is not provided, it will default to 
                            the same name as <controlFile> with any file name
                            suffix removed and the '.data' suffix added.

          On success, the utility will rename the original files by appending
          a '.bkup' suffix to the names, and create new control and data files
          with the specified names.  Note: for control parameter files which
          include nodal data, a new data file will be generated under the
          the name as <controlFile> with any file name suffix removed and
          a '.data' suffix added.


          Examples:
          ---------

          To convert the original format file 'restart.cn' containing
          both control parameters and nodal data, execute:

              paradisconvert restart.cn

          The utility will generate the following files:

              restart.cn.bkup
              restart.cn
              restart.data


          To convert a newer control parameter and data file pair with
          the names 'rs0100' and 'rs0100.data' respective, execute either
          of the following equivalent command lines:

              paradisconvert rs0100
              paradisconvert rs0100 rs0100.data

          In both cases, the utility would generate the files:

              rs0100.bkup
              rs0100.data.bkup

              rs0100
              rs0100.data


          To convert a newer control file <rs0100> and a set of segmented
          data files with the names <rs0100.data.0>, <rs0100.data.1>, ...
          <rs0100.data.7>, execute any of the following equivalent commands:

              paradisconvert rs0100
              paradisconvert rs0100 rs0100.data
              paradisconvert rs0100 rs0100.data.0

          In all of the above cases, the utility would generate the files

              rs0100.bkup
              rs0100.data.0.bkup
              rs0100.data.1.bkup
              ...
              rs0100.data.7.bkup

              rs0100
              rs0100.data.0
              rs0100.data.1
              ...
              rs0100.data.7


     14.4 Creating the FMM Image Correction Table with 'ctablegen'
     -------------------------------------------------------------

          The 'ctablegen' utility is used to create an image correction
          table needed when the FMM (Fast Multipole Method) has been enabled
          by setting the <fmEnabled> toggle on in the control file.  Since
          the data in this file is dependent on the poisson ratio, shear
          modulus, and orders of the multipole and taylor expansions, the
          file must be created for the particular values of these items
          used in the simulation.

          Given the current defaults used by ParaDiS (as set in Param.c):

              poisson ratio:          3.327533e-01
              shear modulus:          6.488424e+10
              multipole order:        2
              taylor expansion order: 5

          To create the FMM image correction table, you could execute
          ctablegen from the main <ParadisDir> directory using the
          following command line:

              bin/ctablegen -nu 3.327533e-01 -mu 6.488424e+10 -mporder 2 -torder 5 -outfile inputs/fm-ctab.Ta.600K.0GPa.m2.t5.dat

          NOTE:  The generation of this table can take a significant
          amount of time, therefore, a parallel version of the utility
          (called ctablegenp) has also been provided.  To execute the
          table generation in parallel on 8 processors using mpirun as
          a parallel program initiator, one could execute:

              mpirun -np 8 bin/ctablegen -nu 3.327533e-01 -mu 6.488424e+10 -mporder 2 -torder 5 -outfile inputs/fm-ctab.Ta.600K.0GPa.m2.t5.dat

          The locations of this file is specified in the control
          parameter file via the <fmCorrectionTbl> parameter.


     14.5 Creating Far-Field Stress Tables with 'stresstablegen'
     -----------------------------------------------------------

          The 'stresstablegen' utility is used to create tables needed
          for calculating stress from distant cells and periodic images
          of the system when FMM is disabled (i.e. <fmEnabled> is set 
          to zero).  There are two required tables, one which factors
          in stress from periodic images only and the other which factors
          in stress from both the primary and period images.

          To create the needed stress tables, execute the stesstablegen
          utility twice from the main <ParadisDir> directory using the
          following command lines:

               bin/stresstablegen -nopbc -outfile inputs/Rijm.cube.out
          and
               bin/stresstablegen -pbc -outfile inputs/RijmPBC.cube.out

          The locations of these files are specified in the control
          parameter file via the <Rijmfile> and <RijmPBCfile> parameters
          respectively.


 15. Tools
 ---------

     15.1 genPovrayFrames
     --------------------

          The genPovrayFrames tool is provided to post-process the
          povray data files generated by ParaDiS when the <povray>
          control parameter toggle has been set.  This tool will
          create an '*.pov' file containing the ParaDiS generated
          data embedded in a proper framework of povray settings 
          and commands.  The tool is located in the <ParadisDir>/tools
          directory.  For details on the use of this tool, see
          the comments at the beginning of the script or execute:

              <ParadisDir/tools/genPovrayFrames -help
          

     15.2 gnuplot2povray
     -------------------
         
          The gnuplot2povray tool is provided to post-process the
          gnuplot data files generated by ParaDiS when the <gnuplot>
          control parameter toggle has been set.  This tool will
          convert the data in the gnuplot files into a format
          suitable for povray, and create an '*.pov' file containing
          the converted data embedded in a proper framework of povray
          settings and commands.  The tool is located in the
          <ParadisDir>/tools directory.  For details on the use of
          this tool, see the comments at the beginning of the script
          or execute:

              <ParadisDir/tools/gnuplot2povray -help

     15.3 stitch
     -----------

          When ParaDiS is configured to enable parallel I/O
          (i.e. the <numIOGroups> control parameter is greater
          than 1), each of the following types of output will
          will be generated as a set of files rather than a
          single output file.  

              gnuplot files
              tecplot files
              segment/arm data
              povray files
              velocity data
              terascale browser files
              pole figures

          Each file 'segment' will contain a portion of the full
          data and will have a sequence number appended to the file
          name.  

          Most utilities for processing these types of output, however,
          expect the data in a single file, so the 'stitch' tool
          has been provided in order to recombine these data file
          segments into a single usable file as a post-processing
          step.  The tool can be found in the source release as:

              <ParadisDir>/tools/stitch

          The command line for 'stitch is as follows:

              stitch [-h] [-d dir | -f file]

          where
 
              -h          prints the usage information to stdout

              -d <dir>    specifies a directory the utility will
                          scan for any segmented output files that
                          need to be stitched together.

              -f <file>   specifies a base file name and the utility will
                          scan for the corresponding file segments (files
                          named <file>.N where N ranges from zero on up)
                          and stitch the segments into the single specified
                          base file name.

              NOTE: If neither a directory nor file name was provided
              on the command line, the tool will behave as though
              the caller specified the current directory on the command
              line and perform as stated above.


  Appendix 1.  Control Parameters
  -------------------------------

  The following is a brief description of the valid control
  file parameters.  Also see the example control files contained
  within the <ParadisDir>/tests directory.

  The parameters have been grouped into the following categories, and
  unless otherwise specified, units are in SI, and lengths are
  normalized by burgers vector magnitude:

    - Simulation cell and processor setup
    - Simulation time and timestepping controls
    - Discretization controls
    - FMM controls
    - Tables for non-FMM far-field forces
    - Loading condition parameters
    - Material and mobility parameters
    - Velocity statistics and controls
    - I/O controls and parameters
    - Miscellaneous parameters




                Simulation cell and processor setup:
                ------------------------------------

Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

numXdoms,
numYdoms,
numZdoms                  integer    Defines the number of computational
                                     domains into which the problem space is
                                     partitioned in the corresponding
                                     dimensions.
numXcells,
numYcells,
numZcells                 integer    Defines the number of cells in the
                                     corresponding dimension of the problem
                                     space.  Cells are independent of the
                                     domain geometry and are used to
                                     determine boundaries at which far-field
                                     forces are computed rather than direct
                                     segment to segment forces.
xBoundType,
yBoundType,
zBoundType                integer    Defines the type of problem space
                                     boundaries in the corresponding
                                     dimension.  Currently supported types
                                     are 0, and 1 for periodic and free
                                     surfaces respectively.
xBoundMax,
yBoundMax,
xBoundMax                 double     If periodic boundaries are not enabled,
                                     defines the upper limit on coordinates
                                     of any dislocation nodes in the
                                     corresponding dimension.  Value must
                                     be <= the respective maximum problem
                                     space coordinate specified in the
                                     nodal data file.
xBoundMin,
yBoundMin,
zBoundMin                 double     If periodic boundaries are not enabled,
                                     defines the lower limit on coordinates
                                     of any dislocation nodes in the
                                     corresponding dimension.  Value must
                                     be >= the respective minimum problem
                                     space coordinate specified in the
                                     nodal data file.

decompType                Integer    Specifies the type of domain decomp-
                                     osition to be used for a run. A value
                                     of one selects the old Recursive
                                     Sectioning (RS) algorithm, a value of
                                     2 selects the Recursive Bisectioning (RB)
                                     decomposition algorithm.  The default
                                     is currently the RS decomposition.

DLBfreq                   integer    Indicates the frequency (in cycles) at
                                     which Dynamic Load-Balancing is to be
                                     attempted.  A value of zero indicates
                                     no load-balancing is to be done.
                                     Default is 3.



               Simulation time and timestepping controls
               -----------------------------------------


Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

cycleStart                integer    Starting cycle number for the simulation.

maxstep                   integer    Indicates the number of timesteps to
                                     execute before terminating.

timeNow                   double     Current simulation time (in seconds).

timeStart                 double     Initial simulation time (in seconds).

timestepIntegrator        string     Selects a timestep integration method.
                                     Valid methods are "trapezoid" and
                                     "forward-euler".  The default value
                                     is "trapezoid".

deltaTT                   double     Specifies the duration of the previous
                                     timestep in units of seconds.

maxDT                     double     Specifies the maximum timestep permitted.
                                     Default is 1.0e-07.

nextDT                    double     Specifies the timestep duration to
                                     attempt on the next cycle.  The timestep
                                     integrator will adjust this value
                                     dynamically.  Default is <maxDT>.

dtDecrementFact           double     Factor by which <delatTT> is mutiplied
                                     when cutting the timestep duration down.
                                     Must be between 0.0 and 1.0.  Currently
                                     only used with the "trapezoid"
                                     integrator.  Default is 0.5.

dtExponent                double     Exponent used in calculation of timestep
                                     increment when <dtVariableAdjustment>
                                     toggle is enabled.  Currently used only
                                     with "trapezoid" timestep integrator.

dtIncrementFact           double     Maximum factor by which <deltaTT> is
                                     multiplied when incrementing the
                                     timestep.  Must be at least 1.0.  Default
                                     is 1.2.

dtVariableAdjustment      integer    Toggles ability to vary the increment by
                                     which the timestep is adjusted when the
                                     current timestep is determined to be too
                                     small.  This will permit the timestep to
                                     be adjusted to a value between 1.0 and
                                     <dtIncrementFact> * <deltaTT>.  Toggle
                                     is zero (off) by default.  Used only
                                     with "trapezoid" integrator.

rTol                      double     Maximum position error (in units of b)
                                     tolerated in the timestep integration.
                                     Only applies to the "trapezoid"
                                     integrator.  Default is 0.25 * <rc>.

rmax                      double     Maximum distance (in units of b) a
                                     node is permitted to move in a single
                                     timestep.  Not used if "trapezoid"
                                     timestep integrator used.  Default is
                                     0.5 * <minSeg>.




                      Discretization controls:
                      ------------------------


Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

maxSeg                    double     Sets the maximum permitted length (in
                                     units of b) of a dislocation segment.
                                     Primarily used for determining when
                                     segments are to be rediscretized during
                                     remesh operations.  This value must be
                                     less than 9/10 the size of a cell.
                                     There is no default value, and must
                                     be specified in the control file.

minSeg                    double     Sets the minimum desired length (in units
                                     of b) of a dislocation segment.
                                     Primarily used for determining when nodes
                                     are to be removed during remesh
                                     operations.  Default is: 
                                     sqrt(remeshAreaMin*(4/sqrt(3))) where
                                     remeshAreaMin = 2*rTol*maxSeg

remeshRule                int        Specifies by number the set of rules
                                     governing rediscretization of 
                                     dislocations.  Currently the only
                                     supported remesh modules are versions
                                     2 and 3.  Default is 2.  (See section
                                     on Rediscretization for details on
                                     remesh versions.)


                          FMM controls
                          ------------


Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

fmEnabled                 integer    Toggle controlling use of a Fast
                                     Multipole Method (FMM) for computing
                                     force contributions from remote
                                     dislocation segments.  A value of zero
                                     disables the use of FMM, any other value
                                     enables FMM.  Default value is zero.

fmCorrectionTbl           string     Name of the image correction table used
                                     by the FMM.  This correction table must
                                     correspond to the specified <fmMPOrder>,
                                     <fmTaylorOrder> and <shearModulus>(?)
                                     control parameters.  See the 'ctablegen'
                                     utility for information on creating these
                                     tables.  This value is ignored if
                                     <fmEnabled> is zero.

fmMPOrder                 integer    Defines the multipole expansion order
                                     used by the FMM.  This value is ignored
                                     if <fmEnabled> is zero.  Default is 2.

fmTaylorOrder             integer    Defines the order of taylor expansions
                                     used by the FMM.  This values is ignored
                                     if <fmEnabled> is zero.  Default is 4.


                   Tables for non-FMM far-field forces
                   -----------------------------------


Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------
Rijmfile                  string     Name of file containing the RIJM table
                                     to be used for far-field stress
                                     calculations.  This parameter is
                                     ignored if the <fmEnabled> parameter
                                     is non-zero.  Default is
                                     "inputs/Rijm.cube.out".

RijmPBCfile               string     Name of file containing the RIJM table
                                     to be used for far-field stress
                                     calculations with periodic boundary
                                     conditions.  This parameter is ignored
                                     if the <fmEnabled> parameter is non-zero.
                                     Default is "inputs/RijmPBC.cube.out".



                     Loading condition parameters:
                     -----------------------------


Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

loadType                  integer    Defines the type of load on the system.
                                     Valid types are:
                                      0:  Creep test
                                      1:  Constant strain rate test
                                      2:  Displacement-controlled test
                                      3:  Load-controlled, load vs. time curve
                                      4:  Cyclic loading condition

appliedStress             double[6]  External stress specified in units of Pa
                                     as [sigma11, sigma22, sigma33, sigma23,
                                     sigma31, sigma12] when <loadType> == 0.
                                     Default is [0.0, 0.0, 0.0, 0.0, 0.0, 0.0].
                            
eRate                     double     Strain rate.  Applicable only when
                                     <loadType> == 1.  Default is 1.0/sec.

indxErate                 integer    Index to indicate normal or shear
                                     deformation.  Used only when
                                     <loadType> == 1.
                                     Valid values are:
                                      1: normal
                                      2: shear
                                     If not specified, defaults to 1.
                                 

edotdir                   double[3]  Uniaxial loading direction accompanying
                                     <eRate>.  Default is [1.0 0.0 0.0].

cTimeOld                  integer    Timestep related to cyclic loading.
                                     Only used when <loadType> == 4.

dCyclicStrain             double     Incremental strain under cyclic load.
                                     Only used when <loadType> == 4.

netCyclicStrain           double     Net accumulated strain under cyclic
                                     load.  Only used when <loadType> == 4.

numLoadCycle              integer    Number of cyclic cycles.  Only used 
                                     when <loadType> == 4.

eAmp                      double     Strain amplitude.  Only used when
                                     <loadType> == 4.


                   Material and mobility parameters:
                   ---------------------------------

    ** Note: Default values for all material and mobility related
             parameters correspond to Tantalum at a temperature of 
             600 degress K and pressure of 0 GPa.

Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

mobilityLaw               string     Specifies by name the set of rules
                                     governing dislocation motion for
                                     the simulation conditions. Default
                                     is "BCC_0".
                                     
shearModulus              double     Shear modulus in units of Pa.  Default
                                     is 6.488424e+10 

pois                      double     Poisson ratio.  Default is 3.327533e-01 

burgMag                   double     Magnitude of the burgers vector (b) in
                                     units of meters.  Default is 2.875401e-10

YoungModulus              double     Young's modulus (in units of Pa).
                                     calculated from E = 2G(1+pois).
                                     Default is 200.0e+09.

rc                        double     Core radius (in units of b) for self-
                                     force calculations.  No default value;
                                     must be specified in control file.

Ecore                     double     Core energy used for self-force
                                     calculation.  Default is
                                     (<shearModulus>/(4*PI)) * log(<rc>/0.1)

MobScrew                  double     Mobility of screw dislocations in 
                                     units of 1/(Pa*sec).  Default is 10.0.

MobEdge                   double     Mobility of edge dislocations in units
                                     of Pa/s.  Default is 10.0.

MobClimb                  double     Climb mobility of dislocations in
                                     units of 1/(Pa*sec).  Default is 1.0e-02.

sessileburgspec           double[30] Array of burgers vectors to be
                                     considered sessile.  First element of the
                                     array contains the number of burgers
                                     vectors specified, the remaining elements
                                     specify the X, Y, then Z components of
                                     each sessile burgers vector.  Maximum
                                     sessile burgers vectors allowed is 9.  No
                                     burgers vectors are sessile by default.

sessilelinespec           double[30] Array of line directions related to
                                     sessileburgspec.  First element is
                                     ignored, remaining elements specify the
                                     X, Y, then Z components of each sessile
                                     line.  The number of lines is assumed
                                     to be the same as the number of sessile
                                     burgers vectors.


                        Flux Decomposition:
                        -------------------

Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

totpSpn                   double[6]  Plastic strain tensor

totpStn                   double[6]  Plastic spin tensor

totstraintensor           double[6]  Strain rate tensor with respect to
                                     global coordinate system.

Ltot                      double[4][4] Decomposed density per burgers vector
                                       for screw and three edges (for BCC
                                       slip systems only).

fluxtot                   double[4][7] For each burgers vector (for BCC slip
                                       systems only), contains:
                                         1:   flux due to climb
                                         2-4: flux due to edge components
                                         5-7: flux due to screw components



                      Velocity statistics:
                      --------------------

  ** Note: these statistics will only be used if the VEL_STATISTICS
           pre-processor macro has been defined during compilation.

Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

vAverage                  double     Average dislocation velocity.

vStDev                    double     Standard deviation of dislocation
                                     velocities. 



                   I/O controls and parameters
                   ---------------------------

  ** Note: A number of the supported output forms are controlled by
     very similar control parameters.  The general descriptions below
     apply to all I/O control parameters of like names:

     *freq    Sets the frequency (in cycles) at which the associated data
              will be written to disk.  If the corresponding <*dt> parameter
              is greater than 0, this parameter will be ignored. 
              Default for all such values is 100.

     *dt      The *dt variables specify the simulation delta time that will
              control the frequency at which the associated output will be
              written to disk. A positive value is interpreted as a delta
              time and will take precedence over any frequency specified by
              the corresponding <*freq> value.  A value <= zero indicates
              write frequency will not be determined by delta times.
              Default value is -1.0.

     *time    Specifies the simulation time at which the associated data
              was last written to disk.  This value will be automatically
              updated during the simulation.  If the corresponding <*dt>
              parameter is <= 0.0 this parameter will be ignored.

     *counter Sequence number of the previously written file of the
              corresponding type.  Default is 0.

Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

dirname                   string     Base output directory name.

skipIO                    integer    Toggle for disabling generation of all
                                     output types other than timing files.
                                     Overrides output-specific toggles if set.
                                     Default is 0.

numIOGroups               integer    Sets the number of groups into which the
                                     domains will be separated for doing
                                     parallel IO.  All files generated in
                                     parallel will be created with this
                                     number of segments.  This value must
                                     be at least one and no more than
                                     the total number of domains used.
                                     default is 1.

armfile                   integer    Toggle enabling/disabling generation of
                                     files identifying each unique dislocation
                                     segment.  Toggle is zero (off) by default.
armfilecounter            integer    See description of <*counter> above.
armfiledt                 double     See description of <*dt> above.
armfilefreq               integer    See description of <*freq> above.
armfiletime               double     See description of <*time> above.


fluxfile                  integer    Toggle enabling/disabling generation of
                                     flux decomposition files.  Toggle is
                                     zero (off) by default.
fluxcounter               integer    See description of <*counter> above.
fluxdt                    double     See description of <*dt> above.
fluxfreq                  integer    See description of <*freq> above.
fluxtime                  double     See description of <*time> above.


gnuplot                   integer    Toggle enabling/disabling generation of
                                     files formatted for use with gnuplot.
                                     Toggle is zero (off) by default.
gnuplotcounter            integer    See description of <*counter> above.
gnuplotdt                 double     See description of <*dt> above.
gnuplotfreq               integer    See description of <*freq> above.
gnuplottime               double     See description of <*time> above.


polefigfile               integer    Toggle enabling/disabling generation of
                                     <111> type burgers vector pole figures.
                                     Toggle is zero (off) by default.
polefilecounter           integer    See description of <*counter> above.
polefigdt                 double     See description of <*dt> above.
polefigfreq               integer    See description of <*freq> above.
polefigtime               double     See description of <*time> above.


povray                    integer    Toggle enabling/disabling generation of
                                     files with nodal data and domain
                                     boundaries for use with povray.  Toggle
                                     is zero (off) by default.
povraycounter             integer    See description of <*counter> above.
povraydt                  double     See description of <*dt> above.
povrayfreq                integer    See description of <*freq> above.
povraytime                double     See description of <*time> above.


psfile                    integer    Toggle enabling/disabling generation of
                                     postscript files containing nodal
                                     data and domain boundaries.   Toggle
                                     is zero (off) by default.
psfiledt                  double     See description of <*dt> above.
psfilefreq                integer    See description of <*freq> above.
psfiletime                double     See description of <*time> above.


savecn                    integer    Toggle enabling/disabling writing of
                                     restart files.  Toggle is zero (off)
                                     by default.
savecncounter             integer    See description of <*counter> above.
savecndt                  double     See description of <*dt> above.
savecnfreq                integer    See description of <*freq> above.
savecntime                double     See description of <*time> above.


saveprop                  integer    Toggle enabling/disabling writing of
                                     various properties files.  Toggle is
                                     zero (off) by default.
savepropdt                double     See description of <*dt> above.
savepropfreq              integer    See description of <*freq> above.
saveproptime              double     See description of <*time> above.


savetimers                integer    Toggle enabling/disabling generation of
                                     timing data files.  Toggle is zero (off)
                                     by default. 
savetimerscounter         integer    See description of <*counter> above.
savetimersdt              double     See description of <*dt> above.
savetimersfreq            integer    See description of <*freq> above.
savetimerstime            double     See description of <*time> above.


savedensityspec           integer[3] Specifies the granularity of the
                                     3D density field written to the 
                                     density field file in X, Y and Z
                                     dimensions.  If any element of this
                                     array is zero, this capability is
                                     disabled.
                                     

tecplot                   integer    Toggle enabling/disabling generation of
                                     output files formatted for use with
                                     tecplot.  Toggle is zero (off) by
                                     default.
tecplotcounter            integer    See description of <*counter> above.
tecplotdt                 double     See description of <*dt> above.
tecplotfreq               integer    See description of <*freq> above.
tecplottime               double     See description of <*time> above.


tsbfile                   integer    Toggle enabling/disabling generation of
                                     output files formatted for use with
                                     the TeraScale Browser.  Toggle is
                                     zero (off) by default.
writetsbspec              integer[3] TBD
tsbfilecounter            integer    See description of <*counter> above.
tsbfiledt                 double     See description of <*dt> above.
tsbfilefreq               integer    See description of <*freq> above.
tsbfiletime               double     See description of <*time> above.


velfile                   integer    Toggle enabling/disabling generation of
                                     output files containing velocity data
                                     for all nodes in the simulation.  Toggle
                                     is zero (off) by default.
velfilecounter            integer    See description of <*counter> above.
velfiledt                 double     See description of <*dt> above.
velfilefreq               integer    See description of <*freq> above.
velfiletime               double     See description of <*time> above.


winDefaultsFile           string     Name of file containing default
                                     options and attributes for the X-window
                                     display.  This is ignore if the
                                     X display support was not enabled
                                     at compile time.  Defaults to
                                     "./inputs/paradis.xdefaults".


                    Miscellaneous parameters
                    ------------------------


Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

elasticinteraction        integer    Toggles between explicit calculation of
                                     elastic interaction and simple line
                                     tension calculations.  Toggle is
                                     one (on) by default.

TensionFactor             double     Factor used for simple line tension
                                     force calculations when the
                                     <elasticinteraction> parameter is zero.
                                     Default is 1.0.




                        Under development
                        -----------------

Parameter                 Type       Description                   
------------------------- ---------- -----------------------------------------

TempK                     double     Simulation temperature (in Kelvin).  For
                                     use in FCC cross-slip.

enforceGlidePlanes        int        If non-zero, forces remesh, collision
                                     handling, etc. to use glide-plane
                                     constraints.  Default is zero.

springConst               double     For use with FCC_0 mobility

mob_compute_escaig        integer    TBD

mob_cross_slip            integer    TBD