Improved diagnostics for NWP verification in the tropics
Koh, Tieh Yong.
Ng, Jun Siang.
Date of Issue2009
The root-mean-square error (RMSE) is often used to verify forecasts. However, its strong dependence on the observation variability makes it unsuitable for comparing model performance between regions where observation variability is much different, e.g., across vertical levels or between the midlatitudes and the tropics. The alpha index based on the tensor variance of forecast-observation discrepancy was formulated to improve on RMSE (and the closely related bias-corrected RMSE). An “error ellipse” was used to represent the random error in vector wind, yielding two other diagnostics: eccentricity and orientation. These diagnostics were applied to verify Naval Research Laboratory's limited-area model, Coupled Ocean/Atmospheric Mesoscale Prediction System (COAMPS), for the first time in Southeast Asia. COAMPS forecasts were verified against radiosonde data from South China Sea Monsoon Experiment (SCSMEX), May–June 1998. Results revealed falling model performance as forecast time increases but little difference between forecasts at 18-km and 54-km resolution. Systematic errors in the model dynamics were suggested from the biases. The alpha indices show that (after bias correction) COAMPS performs best for wind, followed by temperature and then by dew point depression. In this tropical region, 1-day persistence forecasts were only outperformed by the model for wind predictions between 400 mb and 850 mb at forecast times less than 24 hr. The RMSE diagnostic was shown to sometimes yield misleading evaluation of the model's performance. The wind error ellipses revealed that the random error tended to align more with the background flow than with the model bias, possibly indicating a dynamical reason for its existence.
Journal of geophysical research
© 2009 American Geophysical Union.This paper was published in Journal of Geophysical Research and is made available as an electronic reprint (preprint) with permission of American Geophysical Union. The paper can be found at the following official URL: [http://dx.doi.org/10.1029/2008JD011179]. One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper is prohibited and is subject to penalties under law.