Research Note #19 - Installing prep_chem_sources Utility for WRF-Chem

This has been probably one of the trickiest installations I've ever experienced, thanks to the lack of necessary library, limited references and bug in the source code. 

In order to install (or rather, compile) prep_chem_sources tool (I'll call it PCS from now on), HDF5, NetCDF and ZLIB libraries are necessary. Since NetCDF and ZLIB have been already available from previous WRF installation, one should only care about HDF5. Be advised though, HDF5 library of Oakforest super computer module was not compatible with the installation script. Thus, newly fresh HDF5 library is necessary (latest version might be compatible, thus keep checking updates of the super computer).

1. HDF5 Library Installation

Download the latest version from HDF group website. For this case, the version used is HDF5-1.10.4. Extract the file in a directory and execute installation script. Since PCS needs ZLIB and Fortran library, do not forget to mention them during the configuration.

$ ./configure --prefix=/work/gi55/c24223/libs/hdf5 --enable-fortran --enable-cxx --with-zlib=/work/gi55/c24223/libs/grib2

$ ./make

$ ./make check

$ ./make install

Make sure no errors occurred during the installation. After installing HDF5, put its path and LD_LIBRARY_PATH in the .bash_profile and re-load the file.

export HDF5=$PACSLIB/hdf5

export PATH=$HDF5/bin:$PATH

export LD_LIBRARY_PATH= ... :/work/gi55/c24223/libs/hdf5/lib

2. Download PCS tool

Download the tool from this ftp site directory: ftp://aftp.fsl.noaa.gov/divisions/taq/global_emissions

By the time this note was written, the latest version of PCS tool is ver. 1.5. Extract the tools inside any desired directory. Once extracted, there will a directory PREP-CHEM-SRC-1.5 with several sub-directories such as: /bin, /src, /extra etc.

3. Edit  convert_edgar_to_RELACS_REAC.f90 Script

This step should be done, otherwise PCS installation will fail with following messages:

convert_edgar_to_RELACS_REAC.f90(17): error #6405: The same named entity from different modules and/or program units cannot be referenced. [CO]
,CO &
-----------------------------^
convert_edgar_to_RELACS_REAC.f90(20): error #6405: The same named entity from different modules and/or program units cannot be referenced. [CH4]
,CH4 &
-----------------------------^
convert_edgar_to_RELACS_REAC.f90(21): error #6405: The same named entity from different modules and/or program units cannot be referenced. [SO2]
,SO2 &
-----------------------------^
compilation aborted for convert_edgar_to_RELACS_REAC.f90 (code 1)

make: *** [convert_edgar_to_RELACS_REAC.o] Error 1

The cause of this error is ironically simple: the absence of remark symbol ('!') in front of the 9th line of script convert_edgar_to_RELACS_REAC.f90 in the /src sub-directory:

8 subroutine convert_edgar_to_relacs_reac(isp,iespc,ident,spc_name_dummy)  !kml
9 use chem1_list
10 !use chem1_list, only : alke, bio,ora2,aro,ket,alka,ald

Thus, edit the script by adding '!' symbol in front of the said line using vi or other text editor.

4. Edit include.mk.intel.wrf Option Script and Compile PCS

This step is briefly explained on emission guide of WRF-Chem, however no further explanation about the script options available since it will strictly depend on the machine/system architecture. Anyway, one useful hint is: use the same options with WRF-Chem installation used, which are stored inside file configure.wrf inside /WRFV3 sub-directory.

Go to /bin/build sub-directory of PCS, and edit include.mk.intel.wrf option script (since the compiler used is intel compiler). Change these settings as follows:

update 2019/01/31:
Changed F_OPTS because the "longer" version caused crash during execution of utility for GOCART background data.

NETCDF=/work/gi55/c24223/libs/netcdf

HDF5=/work/gi55/c24223/libs/hdf5

HDF5_LIB=-L$(HDF5)/lib -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -L/work/gi55/c24223/libs/grib2/lib -lz -ldl

F_COMP=mpif90 -f90=ifort

C_COMP=mpicc -cc=icc -DMPI2_SUPPORT

LOADER=$(F_COMP)

C_LOADER=$(C_COMP) -DFSEEKO64_OK

F_OPTS=-fpp -D$(CHEM) -O3 -convert big_endian
C_OPTS= -O2

LOADER_OPTS= -O2

Save the script and compile PCS (for GOCART chem option):

$ make OPT=intel.wrf CHEM=RADM_WRF_FIM

If everything's going well, an executable prep_chem_sources_RADM_WRF_FIM_.exe will be created in /bin sub-directory. Check with ldd command to make sure there are no missing shared library links of the executable.

$ ldd prep_chem_sources_RADM_WRF_FIM_.exe

        linux-vdso.so.1 =>  (0x00007ffc35bbe000)

        libnetcdff.so.6 => /work/gi55/c24223/libs/netcdf/lib/libnetcdff.so.6 (0x00007ff8b2825000)

        libnetcdf.so.11 => /work/gi55/c24223/libs/netcdf/lib/libnetcdf.so.11 (0x00007ff8b23cf000)

        libhdf5hl_fortran.so.100 => /work/gi55/c24223/libs/hdf5/lib/libhdf5hl_fortran.so.100 (0x00007ff8b21a8000)

        libhdf5_fortran.so.100 => /work/gi55/c24223/libs/hdf5/lib/libhdf5_fortran.so.100 (0x00007ff8b1f45000)

        libhdf5_hl.so.100 => /work/gi55/c24223/libs/hdf5/lib/libhdf5_hl.so.100 (0x00007ff8b1d1c000)

        libhdf5.so.103 => /work/gi55/c24223/libs/hdf5/lib/libhdf5.so.103 (0x00007ff8b1679000)

        libz.so.1 => /work/gi55/c24223/libs/grib2/lib/libz.so.1 (0x00007ff8b145a000)

        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff8b1237000)

        libmpifort.so.12 => /home/opt/local/cores/intel/impi/2018.1.163/intel64/lib/libmpifort.so.12 (0x00007ff8b0e8d000)

        libmpi.so.12 => /home/opt/local/cores/intel/impi/2018.1.163/intel64/lib/libmpi.so.12 (0x00007ff8b0208000)

        librt.so.1 => /lib64/librt.so.1 (0x00007ff8b0000000)

        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff8afde3000)

        libm.so.6 => /lib64/libm.so.6 (0x00007ff8afae1000)

        libc.so.6 => /lib64/libc.so.6 (0x00007ff8af71e000)

        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff8af507000)

        libhdf5_hl.so.10 => /work/opt/local/apps/intel/2017.1.132/hdf5/1.8.17/lib/libhdf5_hl.so.10 (0x00007ff8af2db000)

        libhdf5.so.10 => /work/opt/local/apps/intel/2017.1.132/hdf5/1.8.17/lib/libhdf5.so.10 (0x00007ff8aebb3000)

        libifport.so.5 => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libifport.so.5 (0x00007ff8ae984000)

        libifcoremt.so.5 => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libifcoremt.so.5 (0x00007ff8ae5f0000)

        libimf.so => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libimf.so (0x00007ff8ae061000)

        libsvml.so => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libsvml.so (0x00007ff8ac9ae000)

        libintlc.so.5 => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libintlc.so.5 (0x00007ff8ac740000)

        libirng.so => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libirng.so (0x00007ff8ac3cc000)

        /lib64/ld-linux-x86-64.so.2 (0x00005572db0f7000)

   

Research Note #18 - Installing WRF-Chem with Kinetic Pre-Processor (KPP)

The installation process is basically similar with typical WRF-Chem, with the additional steps to set KPP environment variables and flex library installation. To make it simple, these are the steps to install WRF-Chem with KPP. It assumes that 'yacc' (yet another compiler) has been installed into the system.

1. Install Flex Library

Flex library could be free-downloaded easily from many sources on internet. For this case, GNU flex used could be obtained from this link.

After downloading, extract the files into any desired directory. Next, run configuration script, and compile as usual. Simple as that.

$ ./configure --prefix=$PACSLIB/flex

$ make

$ make check

$ make install

Make sure that file libfl.a exists in $PACSLIB/flex/lib directory after compilation.
More detailed steps could be found here.

2. Setting the Environment Variables

This step is available on WRF-Chem manual. Just set the environment variables on bash shell for these values:

$ export WRF_KPP=1

$ export YACC="/usr/bin/yacc -d"

$ export FLEX_LIB_DIR=$PACSLIB/flex/lib

The most important thing, add the path of flex shared library (libfl.so.2) to the LD_LIBRARY_PATH of .bash_profile, otherwise, the installation of WRF-Chem will fail. For example:

$ export LD_LIBRARY_PATH=/home/opt/local/cores/intel/impi/2018.1.163/intel64/lib:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64_lin:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/mpi/intel64/lib:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/mpi/mic/lib:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/ipp/lib/intel64:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/tbb/lib/intel64/gcc4.7:/home/opt/local/cores/intel/debugger_2018/libipt/intel64/lib:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/daal/lib/intel64_lin:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/daal/../tbb/lib/intel64_lin/gcc4.7:/home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/daal/../compiler/lib/intel64_lin:/work/gi55/c24223/libs/grib2/lib:/work/gi55/c24223/libs/netcdf/lib:/work/gi55/c24223/libs/flex/lib

Re-run the bash_profile script:

$ . ~/.bash_profile

3. Install the WRF-Chem

Finally, install WRF-Chem as described on this previous post. If the bash_profile file has already listed netcdf libraries and impi libraries, just go straight to step #3.

The compilation of WRF-Chem's executables with KPP is somehow longer than typical model. New updates and details will be added after WPS compilation.