Thursday, August 23, 2018

Research Note #16 - WRF Errors Showcase

I've been working with WRF model for about 4 years, which is actually not so long for someone who has been studying atmospheric science for almost 20 years. During that time, I've switched from WRF-EMS (currently known as STRC-UEMS), WRF-ARW and finally WRF-Chem. While it's quite a short period, I've experienced the mess with WRF so many times, and I think it's time to recap those issues, not just for assisting my own future research, but for others who are studying in the same field.

FYI, I've run WRF model using supercomputer of UTokyo, Oakforest PACS, with the total 8208 computation nodes and peak performance of 25 PFLOPS. Anyway, I usually only use 16 nodes with 32 MPI processes for each simulation batch jobs, because I think it's more than enough.

Without further ado, here are the most common errors I've ever experienced with WRF model. Oh yeah, I will not list errors due to typos in the namelist.input (e.g. you forgot to match the grid resolution between namelist.input and namelist.wps, forget to put comma, period etc.).

1. CFL Error


Symptoms: 
  • WRF generates messages such as : "x points exceeded cfl=x in domain d0x at time ...
  • Simulation speed degrades or simulation completely stops.
Causes:
  • Model becomes unstable, mostly because the time-step used is too large for stable solution, especially while using high-res simulation grids.
  • Conflicts among model physics/dynamics/domains configuration.
Solutions:
  • Decrease the time-step (namelist.input > &domain > time_step).  The most common used convention is 6*DX in kilometers. That means, if the grid resolution is 10 km, then use at least 60 seconds time step. If the messages still appear, decrease the time step to 30 or 10 seconds.
  • Check the parameterization/configuration used in namelist.input which could potentially cause conflicts or model crash. I usually discard some parameterization schemes, and check them individually to see if I they are the causes of the crash.        
2. Flerchinger Messages

Symptoms:
  • WRF generates messages such as: "Flerchinger USEd in NEW version. Iterations= x"
  • Simulation still runs but the speed degrades.
  • WRF restart files are not generated.
Causes:
  • It's basically not an error, but a message generates by Noah LSM (namelist.input > &physics > sf_surface_physics), when the model output soil temperature which is super low/negative soil moisture. I experienced this while running simulation over Russia, with lon and lat > 55 degree. 
Solutions:
  • Change the surface physics to other options
  • Change the input data. I changed from GFS FNL ds083.2 into ECMWF ERA-Interim, and I've never met such messages ever again, even if I still use Noah LSM.


3. No WRF Simulation Log (rsl.out or rsl.error) Generated

Symptoms:
  • While running real.exe or wrf.exe, there are no logs of simulation. Instead of log file, the steps are shown on the screen (stdout).
Causes:
  • Well ... while it's quite strange and stupid (which I experienced), but this is absolutely not an error. It happens when you compiled WRF binaries for serial computation, instead of parallel ones (dmpar or smpar).
Solutions:
  • Recompile the model using dmpar or smpar or both, then you'll get your logs back.

4. Real.exe Error : Interpolation Order Error


Symptoms:
  • While running real.exe, the process is stopped at certain point, indicating that there are to few data for the interpolation order with Real.exe
Causes:
  • I'm not so sure about this, but I think that was caused when real.exe finds that the data for vertical interpolation is inadequate for model run. This was happened when I use GFS FNL ds083.2 for simulation over Russia, and I got so many warnings about missing level data, before this error occurred.
Solutions:
  • Change the data. I used ECMWF Era-Interim and the errors were gone.

5. WRF Simulation Sudden Death

Symptoms:
  • Model crashed. Real.exe and Wrf.exe are abruptly stopped, without any error messages in log files. It just stop.
Causes:
  • I hate this error because it might caused by many factors, but mostly because of the conflicts within the model configuration. For example, I was using WSM-3 MP parameterization schemes, with RRTM schemes for LW and SW, with domain over high latitude and complex terrain, 10 km resolution, using several computation nodes, then many strange things happened: the model crashed many times, could only stable while running on single node, etc.  
  • On several cases, it's also caused by too large time-step similar with CFL error.
  • Sometimes, it also occurs if the domain is too large, in particular when grid size < 10 km with complex terrain.
 Solutions:
  • Change the model configuration. For my case, I used Lin MP scheme with new RRTM schemes, and the error was gone.
  • Reduce the time-step in the factor of 2 (half of time-step first, if still not works, try 0.25 of the original time-step, and so on).
  • Reduce the domain size. 

6. Mismatch Landmask ivgtyp
Symptoms:
  • Real.exe crashed, giving error message: 
-------------- FATAL CALLED ---------------
FATAL CALLED FROM FILE: LINE: 2963
mismatch_landmask_ivgtyp
-------------------------------------------

Causes:
  • By far, only occurred while using ECMWF Era-Interim dataset. The reason is unknown. It's documented on WRF-ARW website.  
Solutions:
  • Change the value of 'surface_input_source' on &physics parameter of namelist.input from '3' to '1'

7. Ungrib.exe Segmentation Fault (End Date)

Symptoms:
  • Ungrib.exe crashes at the ungribbing process at the end date, giving error message: "Segmentation fault ..."
Causes:
  • Probably, it has something with computer memory, because when I set ulimit to unlimited, the problem was solved.
Solution:
  • ulimit -s unlimited 

8. Metgrid.exe error in ext_pkg_write_field

Symptoms:
  • Metgrid.exe crashes at the beginning of the process with messages: 'ERROR: Error in ext_pkg_write_field'.
Causes:
  • This will happen when new NCEP GFS data (Version 15.1 or higher) was process using old version of ungrib.exe (< Ver. 4).
Solution:
  • Install Ungrib from WPS Ver.4. (the old geogrid/metgrid still could be used). 

Thursday, August 16, 2018

Gunpla #38 - RG Sazabi WIP and Review


Sazabi will always be one of my favorite mobile suits in Gundam history. Therefore, when Bandai announced they would release it as an RG, I just felt nothing by hype!

On Saturday, August 11, 2018, it's finally released in Japan. I woke up early and rushed to one of the retail stores next to Kashiwa station to check it out. It was a pretty quiet Saturday, probably because most of people in the city traveled back to their hometown to celebrate Obon. Once I got to the store, I checked the shelves only to become disappointed to find no RG Sazabi there. I moved to another store, and again, no Sazabi. I wondered why I couldn't find it, was it sold out already? Here, at 11 AM?  I took a bus to another store, the one which has the most complete gunpla collection in the city, and finally I saw it. I saw the box.





Man, the box is so huge! It just like twice as big as an typical RG box, definitely an MG size. I took one of them, and grab some cash to the cashier. I bought it for 3600 yen, even though the original price is 4500 yen. In Japan, you will get 20-30% discount if you buy gunpla at retail stores, and even more if you buy the newly released kits.

The construction

Now, lets start building this monster. There are 17 runners in the box, quite normal for an RG kit. So why does it has a huge box? I was pretty sure that was because Sazabi itself is a huge mobile suit, and it should be the largest RG ever released to date. Anyway, I noticed something strange with one of the runners.


It's the B runner, the advanced MS joint, which serves as the inner frame for every RG kit. For a massive kit such as Sazabi though, I guess the parts were too few. Only 5 parts on it, and by the looks alone, I could tell that they should be the inner frame for 3 body limbs.

There were nothing special during the build process, except for the inner frame I mentioned above. I found out that Sazabi has 'another' inner frame for the feet and legs which are constructed from other runners, not from B runner. The inner frame was somehow similar with an MG kits. Interesting. The last RG I built was Banshee Norn, thus I don't know if this kind of inner frame has been already applied for RG Talgeese or just started with Sazabi. Nevertheless, the extra inner frames were more solid than the advance MS joint, hence making the feet and legs so sturdy. This is a good sign for the future of RG kits which are infamous for their fragility.




     
The next part to build was the waist unit, and there it went for the first part of B runner. It's one of the biggest parts of the kit, and as MG Sazabi ver Ka, it was packed with so many details on it. I found some sliding mechanisms for the back skirts, but not as extensive as its MG counterpart. The front and side skirts also have rotated clipping connectors which made them articulated really well. One should pay an extra attention while snap-fitting these parts, especially the back skirt, because sometimes, you will find that the parts are seemingly not fitted enough (even though, they are correctly fitted), and start to push or snap the parts too hard. This leads to stress marks or even breaking the parts. I suggest you check the manual carefully and just continue snap fitting other parts if you experience such situations. 



   
Next is the chest unit. Again, it uses an MG-like inner frame as feet and legs. Anyway, I had a small problem while building this part. The body-arm connectors were constructed on vertical pegs, and one should push them down tightly, as far as the parts went (as the manual said). This was where the problem started. It's so difficult to push the parts, and I just didn't want to break them, thus I just pushed them necessarily. Later, I found that the back part could not tightly cover the body because the arm connectors were not tightly pushed down enough. Anyway, problem solved by pushing the arm connectors carefully using a dull surface of my screwdriver. It was scary though.


  
Now, we get to the arms unit. This unit is arguably the most sophisticated part of the kit, because it houses both two main gimmicks of RG Sazabi: open hatch and wide-movement articulation. The remaining pairs of parts of B runner went for the arms unit. As MG ver. Ka, the arms looks very detailed with piston-like parts which extended if when the arms were bent. Similar with the waist unit, the shoulders have clipping connectors which enable the front and back parts to open, hence featuring the open-hatch gimmick.




The last body part is the head, which was I personally thought had the least feature of the kit. It also had open hatch gimmick by moving up and down the face part.



Next, let's build the back pack and funnels. If you have previously built MG Sazabi Ver. Ka, you might experience 'nightmare' while building those parts, especially the funnels. Well, for RG Sazabi, the build process was much more simpler. I could complete the whole parts in less than one hour. The most complicated part to snap-fit was probably the back covers of the back pack. Those parts were needed to be inserted first than rotated to fit in place. I was scared that I will broke the parts, but eventually, it just worked fine. So just stick on the manual and you'll be fine.



The backpack also features open hatch gimmicks by sliding the back covers and funnels launchers.




And finally, the weapons and shield were the last parts to put together. The construction for those parts was somehow very simple. Nothing special here. Ah, and don't forget about that tiny Char Aznable figure ^^.



The Aesthetics 

And now, time for the review.

Speaking of design aesthetics, probably nothing can beat Sazabi, both in its MG (Ver. Ka) and RG forms. This could probably bias since Sazabi is one of my favorite mobile suit. But, come on ... just look at those details, the colors and badassery! I personally prefer the proportion of RG version over MG ver. Ka. While MG looks so bulky, the RG looks much more balanced and muscular, even when open-hatch gimmick is applied. I also love the shield of RG version which has some sharp edges than the more rounded MG one. Anyway, I feel a little bit off with the chest of RG version. I think it's too small compared to other limbs.





There are 3 tones of red with this RG kit, and all three is blending just right. Watching the kit standing with all weapons and decal applied brought really pleasant feeling to the eyes!

Oh yeah, the decal and foil sticker for this kit are pretty good. I was quite anxious before applying them, but it turned out to be working really well, even though I have to trim some quite annoying decal stickers, especially for the front skirt and legs. The markings were quite thin, so if you apply them carefully (and cleanly), you will barely see the edges.    

The Articulation and gimmicks
This is probably where RG Sazabi stands out among the other RG kits. We knew that RG line is infamous due its fragility, even though some kits are proven to be sturdier than the others (e.g. Mk-II, Exia, 00, Qant or Unicorn/Banshee). Despite the massive size, the articulation of Sazabi is amazing. The legs and arms could bent really far, and it still feels sturdy and strong while putting the kit into many kind of pose. I believe this is the work of the new 'unique' inner-frame.








This kit features wide-range movement gimmick, which enables some parts of the body to be extended to improve their movement to certain degree. One clear example of this is the arms. Just look at this:


By opening and re-snapping some 'hatch' points, the hands and arms could extend a little bit further. Finally, the base of hands (trigger and weapon-handle hands) also has special articulation. This is really useful for posing purpose, especially with the beam riffle. Therefore, in contrast with its MG counterpart, there's exactly no problem with holding the beam rifle on the RG version whatsoever.



And of course, there is also the widely anticipated open-hatch gimmick. This gimmick on RG is basically the simplified version of the MG ver. Ka. It applied to fewer parts but much more easier to pull than MG ver. Ka, and I personally felt quite safe to do it without any feels to broke the kit. The most difficult part to open is probably the side skirts, but I think that's not a big deal.

The Accessories
This kit gives you a beam rifle, shield, 4 beam saber hilts (2 are stored inside the arms), 1 beam axe hilt which could be stored in the shield, 3 missile/rocket (in the shield, could be detached), 2 beam saber clear effects, 2 short beam axe effects and 2 long beam axe effects. There are also 2 standard close hands, 1 trigger right hand, 2 melee weapon holding hands and 2 open hands. There are also spare parts for hand guard and hand joint.

And also, don't forget about the tiny Char's figure^^

Verdicts
I will make it short. This is the best RG I've ever built. It's beautiful, massive, badass, sturdy, sophisticated and fully articulated, really easy for posing. The 3-tones red color really stands out, the gimmicks are great and doing just fine. The only thing I don't like is the small chest, but overall, I prefer this version over the MG Ver.ka.

This is arguably the one of the best (if not the best) RG to date, and you should have it! Hands down!






--------------------

MSN-04 Sazabi

Pros:
  • Highly detail design
  • Great articulation gimmicks
  • Sturdy build

Cons:
  • Small proportioned chest