Research Note #33 - Instalasi WRF-Chem 4.7.1 + KPP di HPC BRIN (Mahameru)

Kinetic PreProcessor (KPP) adalah tool untuk menerjemahkan reaksi kimia kompleks ke dalam kode program (misal Fortran atau C) dan umumnya digunakan pada pemodelan kimia atmosfer, termasuk WRF-Chem. Dengan sekian banyaknya pilihan opsi mekanisme kimia yang dapat digunakan pada WRF-Chem, KPP membantu developer untuk mengaplikasikan skema kimia kompleks yang mereka bangun tanpa harus 'mengganggu' kode program WRF-Chem itu sendiri. Beberapa mekanisme kimia yang menerapkan reaksi kimia kompleks, misalnya untuk simulasi fase gas pada secondary aerosol, antara lain: MOZART, MOZCART, RADM, MOSAIC dll. Dengan kata lain, bila ingin menggunakan mekanisme-mekanisme tersebut untuk simulasi, WRF-Chem yang digunakan wajib hukumnya dikompilasi dengan dukungan KPP.

Tulisan ini berasumsi bahwa environment yang digunakan adalah HPC BRIN, dengan bash shell, gcc 12.2.0 dan pembaca telah mengetahui dasar-dasar kompilasi WRF-Chem ver. 4.7.1 (misal, sudah memiliki library-library yang dibutuhkan, seperti NetCDF, MPICH dll). Bila belum, silakan membaca tulisan berikut: https://sketsa-langit.blogspot.com/2025/10/research-note-29-instalasi-wrf-chem-di.html.

Berikut langkah-langkah kompilasi WRF-Chem dengan KPP pada HPC BRIN. 

Temukan path library flex di sistem. Library flex ini akan digunakan untuk kompilasi KPP:

module load gcc flex

module show flex

------------------------------------------------------------------------------------------------------------------------------------------------------------

   /mgpfs/shared/apps/mod/library/flex/2.6.4:

------------------------------------------------------------------------------------------------------------------------------------------------------------

load("gcc/12.2.0")

whatis("flex 2.6.4 - The fast lexical analyzer generator")

setenv("FLEXHOME","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0")

prepend_path("PATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/bin")

prepend_path("CPATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/include")

prepend_path("INCLUDE_PATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/include")

prepend_path("INCLUDE_LIBRARY_PATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/include")

prepend_path("LIBRARY_PATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/lib")

prepend_path("LD_LIBRARY_PATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/lib")

prepend_path("INFOPATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/share/info")

prepend_path("MANPATH","/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/share/man")

Deklarasikan WRF_CHEM, WRF_KPP dan FLEX_LIB_DIR di environment bash:

export WRF_CHEM=1

export WRF_KPP=1

export FLEX_LIB_DIR=/mgpfs/shared/apps/app/flex/2.6.4/GCC/12.2.0/lib

Masuk ke interactive-node HPC karena proses kompilasi ini akan memakan memori sistem yang besar. Proses kompilasi pada login-node hampir bisa dipastikan akan gagal (akan di-kill oleh sistem).

srun --partition=interactive --pty /bin/bash

Setelah masuk ke interactive-node, masuk ke direktori WRF dan jalankan konfigurasi kompilasi wrf.

./clean -aa

./configure

Pilih 34 (dmpar) untuk GCC compiler. Untuk nesting, pilih 1 (basic).

Pastikan proses konfigurasi berjalan lancar, tanpa ada pesan masalah pada compiler. Bila ada masalah, bereskan dulu masalah compiler, lalu lakukan konfigurasi ulang. Bila tak ada masalah, buka dan edit file configure.wrf.

Hilangkan 'time' pada deklarasi fortran compile 'FC':

FC = time $(DM_FC)  menjadi  FC = $(DM_FC)

Simpan file dan keluar dari configure.wrf. Selanjutnya, pada direktori WRF, buka dan edit file scan.h dan scan.y yang terdapat pada direktori chem/KPP/kpp/kpp-2.1/src/. 

Pada file scan.h (dan scan.y), temukan kode baris void yyerror( char * str ) ; pada baris ke-101 (390 pada scan.y), dan ubah menjadi void yyerror( const char * str ) ;. Simpan perubahan dan keluar dari kedua file tersebut.

Buka file compile_wkc pada direktori chem/KPP dan ubahlah baris kode pada baris 121:

 $WKC_HOME/util/wkc/tuv_kpp FIRST ../../inc/

menjadi:

 $WKC_HOME/util/wkc/tuv_kpp FIRST ../../../../inc/

Simpan perubahan dan keluar dari file. Sekarang anda bisa melakukan kompilasi wrf.

./compile -j 2 em_real >& compile.log

Proses kompilasi akan memakan waktu kurang lebih 30 menit - 1 jam. Pastikan tidak ada Error yang tercatat pada file compile.log dan file-file exe berhasil dibuat.

--->                  Executables successfully built                  <---

-rwxr-xr-x 1 ardh003 researcher 63915712 Apr 30 10:35 main/ndown.exe

-rwxr-xr-x 1 ardh003 researcher 64055224 Apr 30 10:35 main/real.exe

-rwxr-xr-x 1 ardh003 researcher 57765544 Apr 30 10:35 main/tc.exe

-rwxr-xr-x 1 ardh003 researcher 85490080 Apr 30 10:29 main/wrf.exe

==========================================================================

Research Note #25 - Reading a Text File Line-by-Line with Fortran

Again, this is actually a very simple task which I usually deal with ... and forget, most of the time! That's why this post exists. The following is a basic Fortran code to enable you reading the contents of a text file, line-by-line. 

As an example, let's make a text file (named 'list.txt') containing a list of files with their complete paths on Linux:

$ ls ../gcom-c/aerosol/0524/pl/* > list.txt

$ cat list.txt

../gcom-c/aerosol/0524/pl/GC1SG1_20190901D01D_T0524_L2SG_ARPLK_1001.h5

../gcom-c/aerosol/0524/pl/GC1SG1_20190902D01D_T0524_L2SG_ARPLK_1001.h5

../gcom-c/aerosol/0524/pl/GC1SG1_20190903D01D_T0524_L2SG_ARPLK_1001.h5

../gcom-c/aerosol/0524/pl/GC1SG1_20190904D01D_T0524_L2SG_ARPLK_1001.h5

Now, this is the Fortran code to read file 'list.txt':

PROGRAM READTEXT

IMPLICIT NONE

CHARACTER(LEN=8) :: listfile

CHARACTER(LEN=70) :: filecontents

INTEGER :: io

listfile="list.txt"

OPEN(10, FILE=listfile, FORM="FORMATTED", STATUS="OLD",&

        ACTION="READ")

DO

   READ(10,"(A70)",IOSTAT=io) filecontents

   IF (io/=0) EXIT

   PRINT *, filecontents

END DO

CLOSE(10)

END PROGRAM READTEXT

How it works?

The program will open file 'list.txt', then read the first line of the file and put the (string) contents into variable 'filecontents'. It will then print the contents of variable onto the screen. Since there's a DO.. END DO statement, this process will repeat, with the next line being read, replacing the contents of variable 'filecontents'. This looping process will end once there are no more lines found in the file, invoking IOSTAT to a value other than 0 (0 means no I/O error occurred, other values mean there are I/O errors), then program will exit. That's all!

Some important notices:

• The absence of 'DO ... END DO' will make the program only reads the first line of the file.
• The absence of 'IF (io/=0) EXIT' will make the program runs endlessly, with the last line of the file kept being showed on the screen.
• If the output format is not determined (for example, FMT=* instead of A70), the program will not correctly show the file's contents. The (character) length of this format should be same or more than the one at variable declaration. Why 70? Because in this case, the full path of file list (in one line) has 70 characters. 

Research Note #22 - Reading GCOM-C 250m-Res HDF5 Data with Fortran

There were not so many references about 'simple' ways to open and read HDF5 data file with Fortran as well as compiling the codes. I said 'simple', because there are actually manuals and tutorials about that purpose, however, most of them were too complicated and refer to other sub-references, thus blah .. blah ... it takes too much time, especially if you just simply want to open, read and compile a HDF dataset. Here, I posted a very basic way to access HDF5 data, using very minimal Fortran code. Before starting, here's the convention:

• I used Linux OS. 
• I used Fortran 90 for programming.
• HDF5 (ver 1.8.17) libraries has been already installed in my system as module.
• I used JAXA's GCOM-C NDVI Monthly statistics (Level-2 data) with 250m spatial resolution with Sinusoidal/EQA projection. This data contains 4800x4800 pixels. For exemplary purpose, I renamed it to "ndvi.h5".

So, the following is a very basic example of Fortran code to open and read HDF5 data:

PROGRAM HDF5test

   USE HDF5

   IMPLICIT NONE

   INTEGER :: hdferr, i, j

   INTEGER(HID_T) :: fileid, dset

   CHARACTER(LEN=7), PARAMETER :: filename = "ndvi.h5"

   CHARACTER(LEN=19), PARAMETER :: dataset = "Image_data/NDVI_AVE"

   INTEGER, PARAMETER :: xdim = 4800

   INTEGER, PARAMETER :: ydim = 4800

   INTEGER, DIMENSION(1:xdim, 1:ydim) :: ndvi_ave

   INTEGER(HSIZE_T), DIMENSION(1:2) :: dims = (/xdim, ydim/)

   

   CALL h5open_f(hdferr)

   CALL h5fopen_f(filename, H5F_ACC_RDONLY_F, fileid, hdferr)

   CALL h5dopen_f(fileid, dataset, dset, hdferr)

   CALL h5dread_f(dset, H5T_NATIVE_INTEGER, ndvi_ave, dims, hdferr)

    DO j = 1, ydim

      DO i = 1, xdim

         print *, ndvi_ave(i,j)

      ENDDO

   ENDDO

  CALL h5dclose_f(dset, hdferr)

  CALL h5fclose_f(fileid, hdferr)

  CALL h5close_f(hdferr)

END PROGRAM HDFtest

The most important lines are:

• CALL h5open_f(hdferr). This function initializes HDF5 library for Fortran uses. Returns 0 (to hdferr variable) if it's successful, and -1 if it's failed.
• CALL h5fopen_f(filename, H5F_ACC_RDONLY_F, fileid, hdferr). This function opens an existing HDF5 file. In this example, I just want to read it, hence H5F_ACC_RDONLY_F was used as argument. This function will return integers for file identifier and status to fileid and hdferr variables, respectively.
• CALL h5dopen_f(fileid, dataset, dset, hdferr). This function opens an existing dataset in the opened file. Take notice that it needs file identifier (from h5fopen_f), dataset name, dataset identifier and status (as previous functions).
• CALL h5dread_f(dset, H5T_NATIVE_INTEGER, ndvi_ave, dims, hdferr). This function reads raw data from dataset of the file, into a buffer. The buffer means a space in the computer memory, this could be a variable or array. Take notice that it need dataset identifier (from h5dopen_f), memory data type (Native integer), buffer, dimension (to store buffer dimension) and as always, status (hdferr).

To make it simple, the program will firstly initialize HDF5 library, opens an existing HDF5 datafile, opens a dataset from that file (a file could consist of many datasets), read the raw data from the dataset, put the data into an array buffer, then prints the contents of the buffer into the screen. Finally, it closes the dataset, closes the file and ultimately deactivates HDF5 library (flush and cleans the memory). That's all.

Another important thing is variable declaration. HID_T and HSIZE_T are parameters for predefined datatype used by HDF5 library. So, don't be confused with them (for details, just read: https://support.hdfgroup.org/HDF5/doc/fortran/index.html).

Finally, save and give the codes a name (for example: test.f90). To compile the code, just use this command on (bash) shell:

$ h5fc -o test.exe test.f90

It's just similar with a normal Fortran compiler command. If you don't specify a name for the output, a file named 'a.out' will be generated by the compiler.

Research Note #21 - Compiling Fortran Script for NetCDF Data Processing

NetCDF data is become more and more important for my research, thus it's necessary to work with it with my own program, in Fortran (F90). The compilation anyway, a little bit tricky and slightly different with ordinary Fortran compilation. Here are the simple steps to compile it, using Intel Fortran compiler:

1. Make sure NetCDF module has been installed and declared in the system path.

The path for NetCDF module is : $NETCDF/include

If the path has not yet existed, make a soft link of the module file to current directory:

ln -sf $NETCDF/include/netcdf.mod

2. Compile the Fortran code to make an object file

Let's say the code file name is 'test.f90', then execute:

$ ifort -c test.f90 -I$NETCDF/include

Or, if the path of NetCDF module has existed, just simply:

$ ifort -c test.f90

Make sure no problems occur.

3. Finally, link the object file to an executable file:

$ ifort -o test.exe test.o -I$NETCDF/include -L$NETCDF/lib -lnetcdff -lnetcdf

Same as step 2, if the module path has existed:

$ ifort -o test.exe test.o -L$NETCDF/lib -lnetcdff -lnetcdf

If everything's OK, then an exe file will be created in the current directory. The most important thing, don forget to include '-lnetcdff', because since release 4.1.3, the Fortran library is no longer combined with C library. One hint is by checking if there are file named 'libnetcdff' in the $NETCDF/lib directory. If they are there, the argument '-lnetcdff' should be added in the compile command. 

Research Note #19 - Installing prep_chem_sources Utility for WRF-Chem

This has been probably one of the trickiest installations I've ever experienced, thanks to the lack of necessary library, limited references and bug in the source code. 

In order to install (or rather, compile) prep_chem_sources tool (I'll call it PCS from now on), HDF5, NetCDF and ZLIB libraries are necessary. Since NetCDF and ZLIB have been already available from previous WRF installation, one should only care about HDF5. Be advised though, HDF5 library of Oakforest super computer module was not compatible with the installation script. Thus, newly fresh HDF5 library is necessary (latest version might be compatible, thus keep checking updates of the super computer).

1. HDF5 Library Installation

Download the latest version from HDF group website. For this case, the version used is HDF5-1.10.4. Extract the file in a directory and execute installation script. Since PCS needs ZLIB and Fortran library, do not forget to mention them during the configuration.

$ ./configure --prefix=/work/gi55/c24223/libs/hdf5 --enable-fortran --enable-cxx --with-zlib=/work/gi55/c24223/libs/grib2

$ ./make

$ ./make check

$ ./make install

Make sure no errors occurred during the installation. After installing HDF5, put its path and LD_LIBRARY_PATH in the .bash_profile and re-load the file.

export HDF5=$PACSLIB/hdf5

export PATH=$HDF5/bin:$PATH

export LD_LIBRARY_PATH= ... :/work/gi55/c24223/libs/hdf5/lib

2. Download PCS tool

Download the tool from this ftp site directory: ftp://aftp.fsl.noaa.gov/divisions/taq/global_emissions

By the time this note was written, the latest version of PCS tool is ver. 1.5. Extract the tools inside any desired directory. Once extracted, there will a directory PREP-CHEM-SRC-1.5 with several sub-directories such as: /bin, /src, /extra etc.

3. Edit  convert_edgar_to_RELACS_REAC.f90 Script

This step should be done, otherwise PCS installation will fail with following messages:

convert_edgar_to_RELACS_REAC.f90(17): error #6405: The same named entity from different modules and/or program units cannot be referenced. [CO]
,CO &
-----------------------------^
convert_edgar_to_RELACS_REAC.f90(20): error #6405: The same named entity from different modules and/or program units cannot be referenced. [CH4]
,CH4 &
-----------------------------^
convert_edgar_to_RELACS_REAC.f90(21): error #6405: The same named entity from different modules and/or program units cannot be referenced. [SO2]
,SO2 &
-----------------------------^
compilation aborted for convert_edgar_to_RELACS_REAC.f90 (code 1)

make: *** [convert_edgar_to_RELACS_REAC.o] Error 1

The cause of this error is ironically simple: the absence of remark symbol ('!') in front of the 9th line of script convert_edgar_to_RELACS_REAC.f90 in the /src sub-directory:

8 subroutine convert_edgar_to_relacs_reac(isp,iespc,ident,spc_name_dummy)  !kml
9 use chem1_list
10 !use chem1_list, only : alke, bio,ora2,aro,ket,alka,ald

Thus, edit the script by adding '!' symbol in front of the said line using vi or other text editor.

4. Edit include.mk.intel.wrf Option Script and Compile PCS

This step is briefly explained on emission guide of WRF-Chem, however no further explanation about the script options available since it will strictly depend on the machine/system architecture. Anyway, one useful hint is: use the same options with WRF-Chem installation used, which are stored inside file configure.wrf inside /WRFV3 sub-directory.

Go to /bin/build sub-directory of PCS, and edit include.mk.intel.wrf option script (since the compiler used is intel compiler). Change these settings as follows:

update 2019/01/31:
Changed F_OPTS because the "longer" version caused crash during execution of utility for GOCART background data.

NETCDF=/work/gi55/c24223/libs/netcdf

HDF5=/work/gi55/c24223/libs/hdf5

HDF5_LIB=-L$(HDF5)/lib -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -L/work/gi55/c24223/libs/grib2/lib -lz -ldl

F_COMP=mpif90 -f90=ifort

C_COMP=mpicc -cc=icc -DMPI2_SUPPORT

LOADER=$(F_COMP)

C_LOADER=$(C_COMP) -DFSEEKO64_OK

F_OPTS=-fpp -D$(CHEM) -O3 -convert big_endian
C_OPTS= -O2

LOADER_OPTS= -O2

Save the script and compile PCS (for GOCART chem option):

$ make OPT=intel.wrf CHEM=RADM_WRF_FIM

If everything's going well, an executable prep_chem_sources_RADM_WRF_FIM_.exe will be created in /bin sub-directory. Check with ldd command to make sure there are no missing shared library links of the executable.

$ ldd prep_chem_sources_RADM_WRF_FIM_.exe

        linux-vdso.so.1 =>  (0x00007ffc35bbe000)

        libnetcdff.so.6 => /work/gi55/c24223/libs/netcdf/lib/libnetcdff.so.6 (0x00007ff8b2825000)

        libnetcdf.so.11 => /work/gi55/c24223/libs/netcdf/lib/libnetcdf.so.11 (0x00007ff8b23cf000)

        libhdf5hl_fortran.so.100 => /work/gi55/c24223/libs/hdf5/lib/libhdf5hl_fortran.so.100 (0x00007ff8b21a8000)

        libhdf5_fortran.so.100 => /work/gi55/c24223/libs/hdf5/lib/libhdf5_fortran.so.100 (0x00007ff8b1f45000)

        libhdf5_hl.so.100 => /work/gi55/c24223/libs/hdf5/lib/libhdf5_hl.so.100 (0x00007ff8b1d1c000)

        libhdf5.so.103 => /work/gi55/c24223/libs/hdf5/lib/libhdf5.so.103 (0x00007ff8b1679000)

        libz.so.1 => /work/gi55/c24223/libs/grib2/lib/libz.so.1 (0x00007ff8b145a000)

        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff8b1237000)

        libmpifort.so.12 => /home/opt/local/cores/intel/impi/2018.1.163/intel64/lib/libmpifort.so.12 (0x00007ff8b0e8d000)

        libmpi.so.12 => /home/opt/local/cores/intel/impi/2018.1.163/intel64/lib/libmpi.so.12 (0x00007ff8b0208000)

        librt.so.1 => /lib64/librt.so.1 (0x00007ff8b0000000)

        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff8afde3000)

        libm.so.6 => /lib64/libm.so.6 (0x00007ff8afae1000)

        libc.so.6 => /lib64/libc.so.6 (0x00007ff8af71e000)

        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff8af507000)

        libhdf5_hl.so.10 => /work/opt/local/apps/intel/2017.1.132/hdf5/1.8.17/lib/libhdf5_hl.so.10 (0x00007ff8af2db000)

        libhdf5.so.10 => /work/opt/local/apps/intel/2017.1.132/hdf5/1.8.17/lib/libhdf5.so.10 (0x00007ff8aebb3000)

        libifport.so.5 => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libifport.so.5 (0x00007ff8ae984000)

        libifcoremt.so.5 => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libifcoremt.so.5 (0x00007ff8ae5f0000)

        libimf.so => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libimf.so (0x00007ff8ae061000)

        libsvml.so => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libsvml.so (0x00007ff8ac9ae000)

        libintlc.so.5 => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libintlc.so.5 (0x00007ff8ac740000)

        libirng.so => /home/opt/local/cores/intel/compilers_and_libraries_2018.1.163/linux/compiler/lib/intel64/libirng.so (0x00007ff8ac3cc000)

        /lib64/ld-linux-x86-64.so.2 (0x00005572db0f7000)

   

Research Note #10 - Installing NetCDF on FX10

Got a very good lesson today. I've just realized that the NetCDF package I installed for WRF on FX10 few months ago was in fact version 4.1.3 and not 4.4.1.1. I found that 4.1.3 package from WRF online tutorial web page contains more source codes, not only C but fortran and C++, where 4.4.1.1 downloaded  directly from NetCDF website only has either C or fortran codes. In order to install NetCDF libraries for both C and fortran from the latest packages, one should download and install the NetCDF-C first, then install NetCDF-fortran. It was quite confusing since NetCDF-C package's name doesn't contain "C" on it (just NetCDF).

The followings are the steps to install the latest NetCDF-C (4.4.1.1) and NetCDF-fortran (4.2) on FX10 Oakleaf super computer of University of Tokyo. The compilers used for these installations are cross-compilers of FX10 (frtpx, fccpx and FCCpx) on login node. While some resources I found on internet use quite complicated compiler options and optimizations, I used very basic one instead. So far, I found no problems for the compilations.  

1. Environment variables (bash)

$ export CC=fccpx

$ export CXX=FCCpx

$ export FC=frtpx

$ export F77=frtpx

2. Install NetCDF-C

Because it's cross-compiling, we need to add option "--host" for configure script. For FX10, the host name is "sparc64v-sparc-linux-gnu". For other machine, it should be "x86_64" (just run uname command to make it sure).

./configure --prefix=$INSTALL_DIR/netcdf --disable-dap --disable-netcdf-4 --disable-shared --host=sparc64v-sparc-linux-gnu

make

make install

Since it's cross-compilation, we cannot run "make check" on login mode of FX10. In order to check the installed binaries, we should use Interactive mode of FX10 to run them.

3. Install NetCDF-fortran

This is basically the same with its C counterpart. One of differences is the option for configure script where there are no "--disable-dap" and "--disable-netcdf-4" for fortran version. One important thing for this installation is, we should set CPPFLAGS and LDFLAGS on environment variables before running configure script, otherwise the configuration will be failed.

export CPPFLAGS=-I$INSTALL_DIR/netcdf/include

export LDFLAGS=-L$INSTALL_DIR/netcdf/lib

./configure --prefix=$INSTALL_DIR/netcdf --disable-shared --host=sparc64v-sparc-linux-gnu

make

make install

  

After  installation, check the contents of netcdf directory, especially /lib and /include, make sure all libraries are there. Also, don't forget to test "ncdump" on interactive mode to make sure it could be run well.

    

Research Note #8 - Running WRF Simulation (No Chem WPS)

To run WRF model (for this case, v.3.8.1), there are basically 3 main steps: Pre-processing/WPS (geogrid, ungrib, metgrib), WRF simulation (real, wrf) and post-processing (UPP, grib2ctl, gribmap). This post assumes that :

• Only single/coarse domain created for this simulation.
• Meteorological simulation only, no chem wps included.
• ARW core used.
• GrADS will be used to plot the output and UPP is used for post-processing (because ARWpost is .. ehmm .. troublesome) 
• All utilities (UPP, wgrib, and gribmap) have been installed and paths are set
• WRF history interval is 1 hour because UPP could only process output interval >= 1 hour (for now).

For more detail info:

• WRF ARW user guide
• WRF Chem tutorial and manual 
• UPP manual for WRF ARW and NMM

WPS --> Prepare data for WRF
-----------------------------
1. GEOGRID --> Setting domain with geogrid
   - Edit namelist.wps
       &share  :    wrf_core (core model)
                         max_dom (coarse=1, nested > 1)
                      io_form_geogrid (output format, 2 for NC format)
      &geogrid:  e_we, e_sn (grid numbers in w-e and s-n components),
                        dx, dy (grid res in meters)
                     map_proj (projection type) --> low latitude --> mercator
                     ref_lat, ref_lon (center coordinates of domain)
                     truelat1 (true latitude) --> 30 for mercator
                     geog_data_path (path of geo data)
   - Run geogrid.exe --> make sure 'successful ..' message shown.
   - Output file --> geo_em_dxx.nc in NC format(xx --> domain number id)
 
2. UNGRIB --> Extracting meteorological fields from met data into intermediate data
   - Download met data files and put in a directory
   - Make symbolic link of Vtable --> e.g. GFS data --> ln -sf ungrib/Variable_Tables/Vtable.GFS Vtable
   - Run link_grib.csh to the met data --> ./link_grib.csh /home/data/gfs/gfs.t00z (put parts of filename, DONT PUT ONLY path there)
   - Edit namelist.wps
        &share  : start_date (start date of unpacking, not related to any domain hence only 1st column will be processed)
                         end_date (idem)
                         interval_seconds (frequency of data in seconds, hourly --> 3600, 3hourly --> 10800, 6hrly --> 21600 and so on)
         &ungrib : out_format (WPS)
   - Run ungrib.exe --> make sure 'successful ..' message shown.
   - Output files --> FILE:YYYY-MM-DD_HH
 
3. METGRIB --> Horizontally interpolates extracted met data onto model domain 
   - Edit namelist.wps
       &metgrid : io_form_metgrid (output format, 2 for NC format)
   - Run metgrid.exe --> make sure 'successful ..' message shown.
   - Output files --> met_em.dxx.YYYY-MM-DD_hh:mm:ss.nc

 
WRF --> Simulate/Run the Model
------------------------------
1. REAL --> Vertically interpolates met_em data (METGRIB's output), creates boundary and initial condition files and does some consistency checks.
   - CD to WRFVx/run
   - Make symbolic links to the met_em files
   - Edit namelist.input
       &time_control : run_* (simulation length, will override end dates if less than end dates)
                                  start_* (start time of simulation)
                                  end_* (end time of simulation, could be overridden by run_*)
                                 interval_seconds (data frequency, must be same with namelist.wps)
                                  history_interval (frequency to write data to wrfout file in minutes, e.g. 60 --> hourly)
                                  frames_per_outfile (how many time periods must be written in a single wrfout file, e.g. 1440 --> could contain 24 hourly history).
                             io_form* (should be 2 for NC)
   &domains      :      time_step (time step for model simulation/integration)
                                 max_dom, e_we, e_sn, dx, dy, grid_id (must be same with namelist.wps)
                                  num_metgrid_levels (number of vertical levels, data-dependent, should be 32 for GFS. Check met_em files with ncdump -h)
   - Run real.exe --> check rsl.out.0000 for the process --> tail -f rsl.out.0000
   - For multi-processor run --> MPI -np x ./real.exe (x = number of processor. e.g 4)
   - Output files --> wrfinput_dxx and wrfbdy_dxx
 
2. WRF --> Generates the model forecast
   - Run wrf.exe --> check rsl.out.0000 for the process --> tail -f rsl.out.0000
   - For multi-processor run --> MPI -np x ./wrf.exe (x = number of processor. e.g 4)
   - Output files --> wrfout_dxx_[initial time], one for each domain
 
 
UPP (Unipost Post Processing) --> Convert wrfout NC file into GRIB data
-------------------------------------------------------------------------------------------
1. RUN_UNIPOST_FRAMES --> Convert single wrfout NC with forecast time frames into GRIB data
   - CD to DOMAINS/postprd/ directory
   - Make symbolic link to wrfout data to DOMAINS/wrfprd/ directory
   - Edit run_unipost_frames
            TOP_DIR (top directory of WRFV3 and UPP)
           DOMAINPATH (domain path)
           WRF_PATH (WRF path)
           UNIPOST_HOME (UPP path)
           POSTEXEC, SCRIPTS (path of UPP binaries and scripts)
           modelDataPath (wrfout file/symbolic link directory --> domains/wrfprd/)
           dyncore (WRF solver, should be 'ARW')
           informat (wrfout format, should be 'netcdf')
           outformat (upp output format, should be 'grib')
           startdate (forecast initial time --> YYYYMMDDHH, should be same as namelist.input)
           fhr (forecast start hour, should be same as namelist.input)
           lastfhr (forecast end hour, should be same as namelist.input)
           incrementhr (history interval, should be same as namelist.input)
   - Run run_unipost_frames --> confirm the messages in case there are errors
   - Output files --> WRFPRS_dxx.hh, one for each history interval
 

GRIB2CTL --> Creates control file for the GRIB data (UPP output) to be read by GrADS
--------------------------------------------------------------------------------------
   - Run grib2ctl.pl script --> perl grib2ctl.pl -verf WRFPRS_d01.%f2 > test.ctl (for domain 01, each forecast hour into a file named test.ctl)
   - Output file --> test.ctl, make sure the t-def parameter in control file corrects with forecast history interval
 
 
GRIBMAP --> Creates GRIB index file to be read by GrADS
--------------------------------------------------------
   - Run gribmap --> gribmap -i test.ctl
   - Output file --> WRFPRS_dxx.00.idx (1 index file for all forecast history interval)

 Once the idx file created, you can open the data with GrADS.

--------------------------------------------------------------------------------------------------------

Some errors which could occurs:

1. WRF crash

The simulation abruptly stopped with messages such as below:

BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 139 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES

Possible causes:

• Time step is too large compared with the horizontal resolution. Recommended time step is 6*dx in km. For example: dx = 10000 (10 km), then time step should be 6*10 = 60s.

2. WRF freeze

The simulation abruptly stopped without any messages. Possible causes:

• Memory consumption is too large. Use less processors and set bigger stack size. For example : ulimit -s 20000 or unlimited (not recommended for some cases) 

Research Note #7 - WRF Chem Installation

Finally getting face to face with this model once more. Here comes summary for its installation. I had quite problems while installing it, but overall, it was an easy procedure.

Library Needed
--------------
1. NetCDF --> for WRF/WPS, NetCDF Fortran, version >=4.4
2. MPICH --> for multiprocessor
3. zlib --> for WPS
4. libpng --> for WPS
5. Jasper --> for WPS 

Download: http://www2.mmm.ucar.edu/wrf/OnLineTutorial/compilation_tutorial.php

Environment Variables
---------------------
1. Before Any/NetCDF Installation
   DIR=/home/ardhi/Build_WRF/LIBRARIES
   CC=gcc
   CXX=g++
   FC=gfortran
   FCFLAGS=-m64
   F77=gfortran
   FFLAGS=-m64
   
2. After NetCDF Installation
   PATH=$DIR/netcdf/bin:$PATH
   NETCDF=$DIR/netcdf
   
3. After MPICH Installation
   PATH=$DIR/mpich/bin:$PATH
   
4. Before zlib Installation
   LDFLAGS=-L$DIR/grib2/lib 
   CPPFLAGS=-I$DIR/grib2/include

Library Compatibility Tests --> http://www2.mmm.ucar.edu/wrf/OnLineTutorial/compilation_tutorial.php
----------------------------------------------------------------------------------------------------
   
Building WRF
------------
1. Without Chem
   setenv WRF_CHEM=0
   setenv J "-j 2"
   setenv LD_LIBRARY_PATH $DIR/mpich/lib64:$DIR/netcdf/lib64 (for 64bit)
   ./configure --> Serial (32)/ dmpar (34), nesting 0 or 1
   configure.wrf --> LIB_EXTERNAL --> -L/home/ardhi/Build_WRF/LIBRARIES/netcdf/lib64 -lnetcdff -lnetcdf (for 64bit)

2. With Chem
   Same WRF version with Chem (e.g. both V.3.8.1)
   setenv WRF_KPP 0
   setenv WRF_CHEM 1
   setenv EM_CORE 1
   setenv NMM_CORE 0
   setenv J "-j 2"
   setenv LD_LIBRARY_PATH $DIR/mpich/lib64:$DIR/netcdf/lib64 (for 64bit)
   ./configure --> Serial (32)/ dmpar (34), nesting 0 or 1 
   configure.wrf --> LIB_EXTERNAL --> -L/home/ardhi/Build_WRF/LIBRARIES/netcdf/lib64 -lnetcdff -lnetcdf (for 64bit)  

Building WPS
-----------
   setenv JASPERLIB $DIR/grib2/lib
   setenv JASPERINC $DIR/grib2/include
   ./configure --> Serial, GRIB2 (1) or dmpar, GRIB2 (3)
   configure.wps --> WRF_LIB -->  -L$(NETCDF)/lib64  -lnetcdff -lnetcdf (for 64bit)  


Some useful LINUX/UNIX commands during installation:

• setenv (csh,tcsh), export (bash) : sets environment variables
• setenv (csh,tcsh), env | more/less : shows environment variables
• . ~/.bashrc (bash) : run configuration script/permanent environment variables 
• echo $0 : shows the type of current shell 
• lscpu : shows cpu, core and thread numbers (total cpu=socket x core x thread)
• uname -r : shows kernel release version
• uname -m, arch : shows whether linux is 32bit or 64bit
• lsb_release -a : shows distribution release name

Research Note #6 - Beberapa Script Fortran Serba Guna

Berikut ini adalah beberapa fungsi/statement Fortran yang biasa digunakan untuk berbagai keperluan (termasuk riset). Daftar script ini bisa diperbaharui kapan saja, sesuai kebutuhan. Yang perlu diperhatikan, untuk script Fortran pada tulisan ini, saya menggunakan gfortran compiler (GCC) versi 4.8.1 di Windows (MinGW).

Catatan:
(Last updated: Dec 14, 2016)
Judul tulisan ini sedikit diubah agar tidak bias. Versi GCC juga ditambahkan.

Selamat menikmati.

1. Deklarasi variabel

implicit none

character :: line,the_str,inputfile,outputfile

character(len=100) :: thn,yearstr

integer :: year,month,day,maxyear,maxmonth,maxday,obstime

integer*2 :: tmi_tot_px,tmi_rain_px,pr_tot_px,pr_rain_px

integer*4 :: comb_tot_px, comb_rain_px

real :: tmi_mean_rain,pr_mean_rain,comb_mean_rain

parameter (XDIM=1440,YDIM=720,undef=-999)

Catatan:

• Karakter '::' d antara jenis variabel dan nama variabel bisa diabaikan.
• 'Implicit none' diletakkan sebelum deklarasi variabel. Tanpa 'implicit none', variabel dengan nama i, j, k, l, m dan n akan dianggap integer, dan lainnya dianggap real. Statement ini harus digunakan untuk menghindari kebingungan yg akan muncul nantinya.
• Data character tanpa opsi 'len' hanya memuat 1 karakter.
• Integer *2 untuk integer 2 byte, integer*4 untuk integer 4 byte.
• Parameter digunakan untuk menentukan konstanta.

2. Deklarasi array

real, dimension (1:XDIM,1:YDIM) :: grid_val 

atau

real grid_val(XDIM, YDIM)

Catatan:

• Dimension berguna untuk menentukan awal indeks array (misalnya 0, 1 atau 2 dll). Tanpa dimension, indeks array akan dimulai dari 1.

3. Membuka file binary

open(unit_no,file=[filename],form='unformatted', access='direct',recl=[byte]*[xdim]*[ydim],status=[fstatus])
...
close(unit_no)

Catatan:

• Unit_no adalah integer antara 1 sampai 99. Hindari 5 dan 6 karena merupakan reserved number (5=standard input, 6=standard output).
• Filename harus berjenis karakter. 
• Recl harus ada. Byte bisa 2 (2 byte) atau 4 (4 byte). 
• Fstatus bisa 'old' bila hanya membuka, 'replace' bila ingin memodifikasi file.  

4. Membuka file teks/ASCII

open(unit_no,file=[filename],form='formatted',status=[status])

...
close(unit_no)

Catatan:

• Opsi form bisa diabaikan.
• Status sama aturannya dengan file binary.
• Aturan lain sama dengan file binary.

5. Looping

Ada beberapa cara. Misalnya:

do n=[start],[end] 

 ...

end do 

atau,

n=[start]

do while(n<=[end])

   ...

  n=n+1

end do

Catatan:

• Looping dengan 'while' harus menyertakan persamaan untuk counter (misal n=n+1) untuk menghindari looping tanpa akhir, kecuali pada kasus-kasus tertentu.

6. Komentar
Komentar bisa menggunakan 1 baris penuh (dengan 'c') atau pada baris yang sama di belakang script (dengan '!').

c --------------- The start of MAJOR comments #1 ----------------------------------
c This part will determine the starting and ending time of processed data files
c and observation time/type

write(*,'(a)',advance='no') ' Enter the starting year: '


day=1    ! Change this only if you want to start day other than 1. 

7. Kondisional

Ada dua jenis statement kondisional yang biasa digunakan. Yang pertama dan paling umum digunakan adalah (if .. then .. else .. end if) untuk dua atau tiga pilihan. Bila pilihannya banyak, bisa menggunakan (select .. case .. end select).

if ([cond1] [operator1] [cond2] [operator2] ... [condn]) then
 ...
else
 ...
end if

Catatan:

• Operator bisa berupa boolean (misal: '.and.', 'or' dll) atau matematika (misal: '==', '=/', '<=' dll)

select case ([var_name])
case ([cond1,cond2:cond3])
 ...
case ([condA,condB,...,condn])
 ...
case default
 ...
end select

Catatan:

• Koma digunakan untuk memisahkan kondisi tunggal, colon (:) digunakan untuk kondisi dengan selang tertentu.
• 'Case default' kurang lebih sama dengan 'else'.

8. Membaca file teks/ASCII baris demi baris sampai akhir file

do while (1.eq.1)                       

read(unit_no,END=500) line 

 ... 

end do

 ...

500 [CONTINUE or some script]
end

Catatan:

• Unit_no (unit number) harus sama dengan unit_no file yang hendak dibaca .
• Line number tidak harus 500. 

9. Memasukkan data file binary ke dalam array
Untuk array 2 dimensi (misalnya matriks array 10x10):

implicit none
parameter (XDIM=10,YDIM=10)
 ... 
do y=1,10
 do x=1,10
  read([unit_no],rec=1) ((rain(x,y),x=1,XDIM),y=1,YDIM)
 end do
end do  

Catatan:

• Rec=1 untuk akses direct.
• Untuk array 3 dimensi dst, tinggal tambahkan kurung di belakang statement read. Jangan lupa deklarasikan komponen arraynya. Contoh:

  read([unit_no],rec=1) (((rain(x,y,z),x=1,XDIM),y=1,YDIM),z=1,ZDIM)
     
 (Bersambung ...)