Feature 1853 rps doc (#1861)

* Per #1853, adding RPS docs from Eric.

* Per #1853, work in progress getting the formatting correct.

* Per #1853, work in progress getting the formatting correct.

* Per #1853, work in progress getting the formatting correct.

* Per #1853, work in progress getting the formatting correct.

* Per #1853, adding a manual page break after the ROC image to get it to stay in the right spot.

* Per #1853, backing out last change which didn't make the ROC image stay in the right spot.

met/docs/Users_Guide/appendixC.rst

@@ -939,6 +939,24 @@ The area under the curve can be estimated in a variety of ways. In MET, the simp
 MET verification measures for ensemble forecasts
 ________________________________________________
 
+RPS
+~~~
+
+Called "RPS" in RPS output :numref:`table_ES_header_info_es_out_ECNT`
+
+While the above probabilistic verification measures utilize dichotomous observations, the Ranked Probability Score (RPS, :ref:`Epstein, 1969 <Epstein-1969>`, :ref:`Murphy, 1969 <Murphy-1969>`) is the only probabilistic verification measure for discrete multiple-category events available in MET.  It is assumed that the categories are ordinal as nominal categorical variables can be collapsed into sequences of binary predictands, which can in turn be evaluated with the above measures for dichotomous variables (:ref:`Wilks, 2011 <Wilks-2011>`). The RPS is the multi-category extension of the Brier score (:ref:`Tödter and Ahrens, 2012<Todter-2012>`), and is a proper score (:ref:`Mason, 2008<Mason-2008>`).
+
+Let :math:`\text{J}` be the number of categories, then both the forecast, :math:`\text{f} = (f_1,…,f_J)`, and observation, :math:`\text{o} = (o_1,…,o_J)`, are length-:math:`\text{J}` vectors, where the components of :math:`\text{f}` include the probabilities forecast for each category :math:`\text{1,…,J}` and :math:`\text{o}` contains 1 in the category that is realized and zero everywhere else. The cumulative forecasts, :math:`F_m`, and observations, :math:`O_m`, are defined to be:
+
+:math:`F_m = \sum_{j=1}^m (f_j)` and :math:`O_m = \sum_{j=1}^m (o_j), m = 1,…,J`.
+
+
+To clarify, :math:`F_1 = f_1` is the first component of :math:`F_m`, :math:`F_2 = f_1+f_2`, etc., and :math:`F_J = 1`.  Similarly, if :math:`o_j = 1` and :math:`i < j`, then :math:`O_i = 0` and when :math:`i≥j`, :math:`O_i = 1`, and of course, :math:`O_J = 1`.  Finally, the RPS is defined to be:
+
+.. math:: \text{RPS} = \sum_{m=1}^J (F_m - O_m)^2 = \sum_{m=1}^J BS_m,
+
+where :math:`BS_m` is the Brier score for the m-th category (:ref:`Tödter and Ahrens, 2012<Todter-2012>`).  Subsequently, the RPS lends itself to a decomposition into reliability, resolution and uncertainty components, noting that each component is aggregated over the different categories; these are written to the columns named "RPS_REL", "RPS_RES" and "RPS_UNC" in RPS output :numref:`table_ES_header_info_es_out_ECNT`.
+
 CRPS
 ~~~~
 

met/docs/Users_Guide/refs.rst

@@ -103,7 +103,13 @@ References
 
 | Efron, B. 2007: Correlation and large-scale significance testing. *Journal*
 |   of the American Statistical Association,* 102(477), 93-103.
-| 
+|
+
+.. _Epstein-1969:
+
+| Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories.
+|   *J. Appl. Meteor.*, 8, 985–987, 10.1175/1520-0450(1969)008<0985:ASSFPF>2.0.CO;2.
+|
 
 .. _Gilleland-2010:
 
@@ -158,6 +164,13 @@ References
 |   132, 1891-1895.
 | 
 
+.. _Mason-2008:
+
+| Mason, S. J., 2008: Understanding forecast verification statistics.
+|   *Meteor. Appl.*, 15, 31–40, doi: 10.1002/met.51.
+|
+
+
 .. _Mittermaier-2014:
 
 | Mittermaier, M., 2014: A strategy for verifying near-convection-resolving
@@ -170,6 +183,13 @@ References
 |   *Theory of Statistics*, McGraw-Hill, 299-338.
 | 
 
+.. _Murphy-1969:
+
+| Murphy, A.H., 1969: On the ranked probability score. *Journal of Applied*
+|   *Meteorology and Climatology*, 8 (6), 988 – 989,
+|   doi: 10.1175/1520-0450(1969)008<0988:OTPS>2.0.CO;2.
+|
+
 .. _Murphy-1987:
 
 | Murphy, A.H., and R.L. Winkler, 1987: A general framework for forecast
@@ -224,6 +244,13 @@ References
 |   *Meteorological Applications* 15, 41-50.
 | 
 
+.. _Todter-2012:
+
+| Tödter, J. and B. Ahrens, 2012: Generalization of the Ignorance Score:
+|   Continuous ranked version and its decomposition. *Mon. Wea. Rev.*,
+|   140 (6), 2005 – 2017, doi: 10.1175/MWR-D-11-00266.1.
+|
+
 .. _Weniger-2016:
 
 | Weniger, M., F. Kapp, and P. Friederichs, 2016: Spatial Verification Using