This section goes into more detail about how to build the ESMF software, and how to customize the build if your platform is not supported by the makefiles packaged with the system.
Major releases of the ESMF software can be downloaded by following the instructions on the the Downloads & Documentation link on the ESMF website, http://www.esmf.ucar.edu.
The ESMF is distributed as a full source code tree. You will need to compile the code into the libesmf.a library. On some platforms a shared library, libesmf.so, is also created. Follow the instructions in the following sections to build the library and link it with your application.
The following compilers and utilities are required for compiling and linking the ESMF software:
An alternative to the MPI library is provided with the ESMF, a single-process MPI-bypass library. It allows applications which use MPI to be linked but only run single process.
In order to build html and pdf version of the ESMF documentation, LATEX, the latex2html conversion utility, and the Unix/Linux dvipdf utility must be installed.
The following is a full alphabetical list of all environment variables which are used by the ESMF build system. In many cases only ESMF_DIR must be set, and on Linux clusters ESMF_COMPILER must also be set to select which vendor's Fortran compiler is installed. The other variables have defaults which work for most systems.
Not normally used. ESMF auto-generates subroutine interfaces for a wide variety of data arrays of different ranks, shapes, and types. If no data of rank greater than 4D will be used, setting this variable to any value will prevent ESMF from generating interfaces for 5D to 7D arrays. This will shrink the amount of autogenerated code.
Variable that has the value of uname -s. For example, this will be AIX for IBM RS6000's. There should be no reason for the user to set ESMF_ARCH since the proper value should be determined automatically.
Variable specifying that the ESMF tests should use an alternative batch system for submitting jobs. The valid values are system dependent. Currently, only the value lsf is valid for AIX systems which are using the Platform LSF (Load Sharing Facility) to schedule jobs instead of POE.
Build option value of g for debug mode or O (capital oh) for optimized mode. Default value is O. See the related ESMF_OPTLEVEL entry.
Defines which MPI communications library to use. Many larger machines will come with a vendor-supplied MPI library and in those cases the default setting will be the native MPI. Otherwise the default setting will be mpiuni so that the mpi stub bypass library will be used. With this library ESMF programs can be executed but only single process. This is the default case for Linux and Darwin systems. To run multiprocess an MPI implementation must be installed, and this variable indicates which one it is. Generally, the value will be either mpich or lam.
Variable specifying which compiler to use. Value can be default, absoft, g95, intel, lahey, nag, pgi, or xlf. If the value is default or ESMF_COMPILER is left unset, then the default compiler will be used. On systems which usually come with a single vendor-supplied compiler, the default is to use this compiler. On systems like Linux clusters where there is no single vendor compiler installed on all systems, you will generally want to set this. The default for Linux systems is lahey and on Darwin (Mac OS X) systems it is absoft.
Variable specifying which C/C++ compiler to use. In most cases this should not be set; the default is to use the vendor-supplied compilers on those systems which normally come with a single development environment; for Fortran compilers which do not normally come with a companion C++ compiler, the default is to use the GNU C compilers. However, some Fortran compilers support linking with either their own C++ compiler or code compiled with the GNU compilers; in these cases, you can set this variable to the value gnu in order to compile with the vendor Fortran compiler and the GNU C/C++. This option is only supported for ESMF_ARCH=Darwin, ESMF_COMPILER=xlf, and ESMF_ARCH=Linux, ESMF_COMPILER=intel.
Variable specifying which C/C++ libraries to link with. In most cases this should not be set; the default is to use the vendor-supplied libraries on those systems which normally come with a single development environment; on other systems like Linux clusters, the default is to use the ESMF_COMPILER setting. However, some vendor compilers support compiling with their compilers but linking with the GNU C/C++ libraries. In these cases, you can set this variable to the value gnu in order to link with the GNU C/C++ libraries. This option is only supported for ESMF_ARCH=Linux, ESMF_COMPILER=intel.
Variable specifying a colon-separated list of directories to search for C++ related library files. These are typically needed when linking an F90 program; by default the F90 linker includes the location of the F90 system libraries needed, but the C++ libraries must be added explicitly. If the directories are not on the default search path already, this variable can be set to add them.
Variable specifying how to compile the unit tests. If set to the value ON, then all unit tests will be compiled and will be executed when the test is run. If unset or set to any other value, only a subset of the unit tests will be included to verify basic functions. Note that this is a compile-time selection, not a run-time option.
Variable specifying a colon-separated list of directories to search for F90 related library files. These are typically needed when linking an C++ program; by default the C++ linker includes the location of the C++ system libraries needed, but the F90 libraries must be added explicitly. If the directories are not on the default search path already, this variable can be set to add them.
Not normally used. At least one hardware platform (NEC) does not support declaring a data array of 1-byte integers. Setting this variable to any value will prevent ESMF from generating data array interfaces for data types of integer, 1-byte. For completeness, 2-byte integers can be avoided the same way by setting the corresponding variable to any value.
This version of the framework is prepared to use the netCDF I/O library. However, because the location of the library and include files varies widely from system to system the support for I/O is disabled by default. To enable support, edit build/common.mk and comment out the two lines which set ESMF_NO_IOCODE to ON and set the CPP flag, and recompile.
Variable specifying the size of an address on systems which can build either 32 or 64 bit executables. When possible the default value will be 64, otherwise it will be 32. Note that this is executable size (pointer size), and in no way affects default data sizes.
This compile-time option controls ESMF's dependency on a functioning Pthreads library. The default option is set to ON. Setting this variable to OFF will turn ESMF's pthreads feature set off and a stubs header file will be used instead of the Pthreads header during library compilation. This may be necessary on systems that do not supply a working Pthreads implementation. It may also be desirable to disable ESMF's pthreads features for purpose of debugging ESMF applications. Linking an ESMF application against a pthreads-disabled ESMF library will result in run-time ESMF errors if the application makes use of any ESMF pthreads features. The features offered by a pthreads-enabled ESMF library form a proper superset of the pthreads-disabled version. Specifically, a pthreads-disabled ESMF library does not support ESMF multi-threading and concurrent execution of components with overlapping PET lists. It also limits the communication API as well as place some restrictions on where Components may be created. See the VM section of the reference document for more details on Pthreads in ESMF. (Notice that on some platforms even a pthreads-disabled version of the ESMF library will need to be linked against a functioning Pthreads library. In those cases this dependency originates from the used compiler or the MPI implementation.)
If this environment variable is set to ON before the ESMF system tests are build they will activate ESMF threading in their code. Specifically each component will be executed using ESMF single threading instead of the default non-threaded mode. The difference between non-threaded and ESMF single threaded execution should be completely transparent. Notice that the setting of ESMF_TESTWITHTHREADS does not alter ESMF's dependency on Pthreads but tests ESMF threading features during the system tests. An ESMF library that was compiled with disabled Pthread features (via the ESMF_PTHREADS variable) will produce ESMF error messages during system test execution if the system tests were compiled with ESMF_TESTWITHTHREADS set to ON.
On Alpha machines an additional environment variable needs to be set:
On an Alpha machine, test and demo applications are run using the bsub command. The value of ESMF_PROJECT is used as the argument for bsub's -P option. The -P option assigns a job to a specific project.
Environment variables must be set in the user's shell and not inside an ESMF makefile or build system file. Here is an example of setting an environment variable in tcsh and csh shells:
setenv ESMF_PREC 32
In ksh shell environment variables are set this way:
export ESMF_PREC=32
Environment variables can also be set from the gmake command line:
gmake ESMF_PREC=32
The following two tables list various combinations of environment variable settings used by the ESMF build system. A default value in the compiler column indicates the vendor compiler.
The first table lists the exact combinations which are tested regularly and are fully supported. The second table lists all possible combinations which are included in the build system.
Fully tested combinations:
| ESMF_ARCH | ESMF_COMPILER | ESMF_COMM | ESMF_PREC | |
| Alpha | OSF1 | default | mpi | 64 |
| Cray X1 | Unicos | default | mpi | 64 |
| IBM RS6000 | AIX | default | mpi | 32 |
| IBM RS6000 | AIX | default | mpi | 64 |
| Linux | Linux | absoft | mpich | 32 |
| Linux | Linux | intel | mpich | 32 |
| Linux | Linux | intel | lam | 64 |
| Linux | Linux | lahey | mpich | 32 |
| Linux | Linux | lahey | mpiuni | 32 |
| Linux | Linux | nag | mpich | 32 |
| Linux | Linux | pgi | mpich | 32 |
| Linux | Linux | pgi | mpiuni | 32 |
| Mac OS X | Darwin | absoft | lam | 32 |
| Mac OS X | Darwin | absoft | mpiuni | 32 |
| Mac OS X | Darwin | xlf | lam | 32 |
| Mac OS X | Darwin | xlf | mpiuni | 32 |
| SGI | IRIX64 | default | mpi | 32 |
| SGI | IRIX64 | default | mpi | 64 |
| SGI Altix | Linux | intel | mpi | 64 |
All possible options, by platform/compiler. Where multiple options exist the default value is in bold:
| ESMF_ARCH | ESMF_COMPILER | ESMF_COMM | ESMF_PREC | |
| Alpha | OSF1 | default | mpi, mpiuni | 64 |
| Cray X1 | Unicos | default | mpi, mpiuni | 64 |
| IBM RS6000 | AIX | default | mpi, mpiuni | 32, 64 |
| Linux | Linux | absoft | lam, mpich, mpiuni | 32 |
| Linux | Linux | intel | lam, mpi, mpich, mpiuni | 32, 64 |
| Linux | Linux | lahey | lam, mpich, mpiuni | 32 |
| Linux | Linux | nag | lam, mpich, mpiuni | 32 |
| Linux | Linux | pathscale | lam, mpich, mpiuni | 64 |
| Linux | Linux | pgi | lam, mpich, mpiuni | 32, 64 |
| Blue Gene | Linux | xlf | mpi | 32 |
| Mac OS X | Darwin | absoft | lam, mpich, mpiuni | 32 |
| Mac OS X | Darwin | nag | lam, mpich, mpiuni | 32 |
| Mac OS X | Darwin | xlf | lam, mpich, mpiuni | 32 |
| SGI | IRIX64 | default | mpi, mpiuni | 32, 64 |
| SGI Altix | Linux | intel | mpi, mpiuni | 64 |
Building the library for multiple architectures or options at the same time is supported; building or running the tests or examples is restricted to one platform/architecture at a time. The output from the test cases will be stored in a separate directories so the results will be kept separate for different architectures or options.
GNU make is required to build the library. On some systems this will be just the command make. On others it might be installed as gmake or even gnumake. In any event, use the -version option with the make command to determine if it is GNU make.
Build the library with the command:
gmake
Makefiles throughout the framework are configured to allow users to compile files only in the directory where gmake is entered. Shared libraries are rebuilt only if necessary. In addition the entire ESMF framework may be built from any directory by entering gmake all, assuming that all the environmental variables are set correctly as described in Section 6.2.2.
Users may also run examples or execute unit tests of specific classes by changing directories to the desired class examples or tests directories and entering gmake run_examples or gmake run_unit_tests, respectively. For non-multiprocessor machines, uni-processor targets are available as gmake run_examples_uni or gmake run_unit_tests_uni.
The documentation consists of an ESMF User's Guide, ESMF Requirements Document, and ESMF Reference Manual for Fortran. To build documentation:
gmake doc ! Builds the manuals, including pdf and html.
The resulting documentation files will be located in the top level directory $ESMF_DIR/doc.
The ESMF build system is divided into two parts. The first is the series of makefiles located with the source code. The second is a set of makefile fragment files designed to be used by the source code makefiles. Makefile fragment files are files that contain makefile syntax defining build rules and actions, but do not constitute a complete build system.
The main components of the make system are:
There are two directories containing makefile fragment files used by the make system.
The build directory contains the generic makefile fragment file common.mk that is included by the top level makefile in the source tree. common.mk contains generic build system settings and build rules used across all platforms. A user should have no reason to edit common.mk.
The build_config directory contains makefile fragments for each supported platform defining compilers, compiler flags, and the various other definitions that are necessary to build on each platform. Files can also be added to this directory for specific machines where the build settings are different from the standards of the architecture. One of the files in this directory will be included by the build/common.mk file depending the values of the environment variables ESMF_ARCH, ESMF_COMPILER and ESMF_SITE. See below for more details on environment variables.
The three sets of source codes that the build system supports all need environment variables set to point to their top level source code directories.
To build the ESMF library, ESMF_DIR needs to be set to the top level ESMF library source code directory.
The build system needs ESMF_IMPL_DIR set to the top level source code directory of the Implementation Report source tree to build the report and to build and run the examples.
An EVA source code tree does not contain a copy of the ESMF build system. Instead it uses a copy found in an ESMF library source code tree. Building the EVA applications requires that ESMF_EVA_DIR and ESMF_DIR be set. ESMF_EVA_DIR has to be set to the top directory of the EVA source code. ESMF_DIR has to be set to the top directory of an ESMF source code tree.
Every source tree contains a makefile in its top level directory. This makefile includes the common.mk file from the build directory. The top level makefile contains makefile settings specific for the source code that it is found in.
Each directory in the source tree contains a makefile which includes the top level makefile. These local makefiles include definitions that allow the local files and documents to be built.
A single makefile or makefile fragment from the build system never constitutes a complete set of build rules and settings. Starting from the local makefile, successive include commands are used to string together makefiles and makefile fragments to create a complete system of build rules and settings. Configuration of the build system is done by including a configuration makefile fragment. The build system can be configured for a machine's architecture or, if needed, for a particular machine and its compiler. A configuration for a specific machine or compiler is referred to as a site configuration.
The string of files included is fairly short. Makefiles below the top level makefile include the top level makefile. The top level makefile includes build/common.mk and then build/common.mk includes a configuration file from the build_config directory. The configuration files in the build_config directory contain the architecture and site specific build settings. The architecture, compiler and site that a file configures is determined by its name. The configuration makefile fragments follow this naming convention:
ESMF_ARCH.ESMF_COMPILER.ESMF_SITE/build_rules.mk
Where ESMF_ARCH, ESMF_COMPILER, and ESMF_SITE are environment variables either set by the user or given default values by the build system. ESMF_ARCH is the current architecture and will have the value returned by the command uname -s. ESMF_COMPILER is the compiler name. ESMF_SITE, if set, is generally the current machine name, the location, or the organization (e.g. mit, cola). If there are no site specific files for a particular architecture, then ESMF_COMPILER and ESMF_SITE will be set to default. Examples:
AIX.default.default/build_rules.mk ! Default configuation for RS6000.
Linux.lahey.default/build_rules.mk ! Linux configuation using lahey compilers.
C++ and C source code written to build on a range of platforms many times require preprocessor directives to configure the source code for specific platforms. The directives are included in the source code and are processed by the C preprocessor (cpp) before the source code is compiled. The directives are used to determine among other things, the memory requirements of variable types and the system resources that are available.
The ESMF build system provides preprocessor directives in ESMC_Conf.h and ESMF_Conf.inc files that are included in the source code. The path to these files is
build_config/ESMF_ARCH.ESMF_COMPILER.ESMF_SITE
Where ESMF_ARCH, ESMF_COMPILER, and ESMF_SITE are environment variables set by the user or given default values be the build system. Based on the settings of these environment variables, the build system provides a path to the correct files during source code compiles.
The build system can be ported to other Unix platforms by adding new makefile fragments and configuration files. The new makefile fragment file has to follow the naming convention used by the existing makefile fragment files and be created in the directory build_config. The new file will also have to define the same makefile variables as the existing makefile fragment files. Example:
build_config/Linux.pgi.mysite/build_rules.mk
This fragment is included after the default fragment, so it need only contain overrides and additional definitions.
See the README file in the build_config directory in the source tree for more detailed information on what files will generally need to be customized to support a new architecture.