A diagnostic suite to assess NWP performance.
Koh, Tieh Yong.
Bhatt, Bhuwan Chandra.
Date of Issue2012
Earth Observatory of Singapore
A suite of numerical weather prediction (NWP) verification diagnostics applicable to both scalar and vector variables is developed, highlighting the normalization and successive decomposition of model errors. The normalized root-mean square error (NRMSE) is broken down into contributions from the normalized bias (NBias) and the normalized pattern error (NPE). The square of NPE, or the normalized error variance α, is further analyzed into phase and amplitude errors, measured respectively by the correlation and the variance similarity. The variance similarity diagnostic is introduced to verify variability e.g. under different climates. While centered RMSE can be reduced by under-prediction of variability in the model, α penalizes over- and under-prediction of variability equally. The error decomposition diagram, the correlation-similarity diagram and the anisotropy diagram are introduced. The correlation-similarity diagram was compared with the Taylor diagram: it has the advantage of analyzing the normalized error variance geometrically into contributions from the correlation and variance similarity. Normalization of the error metrics removes the dependence on the inherent variability of a variable and allows comparison among quantities of different physical units and from different regions and seasons. This method was used to assess the Coupled Ocean/Atmospheric Mesoscale Prediction System (COAMPS). The NWP performance degrades progressively from the midlatitudes through the sub-tropics to the tropics. But similar cold and moist biases are noted and position and timing errors are the main cause of pattern errors. Although the suite of metrics is applied to NWP verification here, it is generally applicable as diagnostics for differences between two data sets.
Journal of geophysical research
© 2012 American Geophysical Union. This paper was published in Journal of Geophysical Research and is made available as an electronic reprint (preprint) with permission of American Geophysical Union. The paper can be found at: [DOI: http://dx.doi.org/10.1029/2011JD017103]. One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper is prohibited and is subject to penalties under law.