Improve forecast trial docs

usecases.md

@@ -110,11 +110,19 @@ The Solar Forecast Arbiter is primarily designed to support the evaluation of so
 
 **Use case narrative**: The framework administrator, in consultation with trial participants, defines trial period, metrics, locations, forecast quantities, time horizons, time resolutions, etc. of a forecast trial. The trial may be retrospective or live, and may involve a debugging period followed by an evaluation period. Separate trials may be administered for multiple forecast parameters (e.g. hour ahead and day ahead forecast evaluations for the same set of locations).
 
-**Example 1**: Forecast User 1 would like to evaluate the operational performance of many forecast providers. Forecast User 1 and the Framework Administrator announce the opportunity to Forecast Providers A-F. User 1 may announce its intention to contract with one or more Providers at the conclusion of the trial, but this is outside the scope of the Solar Forecast Arbiter.
+**Example 1**: An electric utility (the Utility) uses an in-house forecast model to predict the power generation of its fleet of solar photovoltaic (PV) power plants. The Utility is interested in replacing or supplementing its in-house forecast model with a commercially-available forecast, and therefore would like to evaluate the operational performance of commercial forecast providers. The Utility and the Framework Administrator announce the opportunity to Forecast Providers A-F. The Utility may announce its intention to contract with one or more Forecast Providers at the conclusion of the trial, but this is outside the scope of the Solar Forecast Arbiter.
 
-The User, Administrator, and Providers discuss the many possible forecasts that could be evaluated and determine that two trials will be conducted to support particular business needs. Trial 1 is a 1-hour ahead, 1-hour interval average trial and Trial 2 is day ahead as of 13:00, 1-hour interval trial. The full forecast evaluation time series attributes are defined for each trial. For both trials, the start date is Jan 1, 2020, the end date is March 31, 2020, the evaluation metrics are average hourly MAE and RMSE, and missing forecasts will be assumed to be 0. Trial 1 will include a persistence benchmark, and Trial 2 will include a benchmark based on transparent processing of the NOAA NAM model.
+The Utility, Administrator, and Providers discuss the many possible forecasts that could be evaluated and determine that two trials will be conducted to support particular business needs. Both trials will evaluate deterministic forecasts of power generated from the Utility's PV plants. Trial 1 will evaluate 1-hour ahead, 1-hour interval average forecasts that needs to be submitted at the prior hour. Trial 2 will evaluate day-ahead, 1-hour interval average forecasts that are issued once per day by 10:00 on the prior day and forecast for 00:00 to 23:00 for the next day (a lead time of 14 hours). The full forecast evaluation time series attributes are defined for each trial. For both trials, the start date is Jan 1, 2020, the end date is March 31, 2020, the evaluation metrics are average hourly MAE and RMSE, and missing forecasts will be assumed to be 0. Trial 1 will include a persistence benchmark, and Trial 2 will include a benchmark based on transparent processing of the NOAA NAM model, with solar irradiance from the NAM model transformed to power using pvlib.
 
-Forecast User 1 creates the Sites, Observations, and Aggregate, uploads 12 months of training data, and shares the data with Forecast Providers A-F. Forecast Providers A-F download the training data and create their models. An anonymous user is automatically generated for each Provider and the Framework Administrator does not keep a record of the user mapping. Separately for each trial, Forecast Providers upload their forecast runs during the debugging period, fix any issues, and continue to regularly upload runs during the evaluation period. Forecasters that fail to submit a forecast run are penalized according to predefined rules. At the conclusion of the trials, reports are automatically generated for all participants.
+Quality of the observation data provided by the Utility are also discussed and it is agreed that if nighttime hours will be ignored, which the Administrator will set the when performing the reporting on the metrics. In addition, if the observation data is identified to have quality issues for any periods, those values will be flagged in the data either on upload using the quality flag input or by the Administrator and any metric reports will be recalculated.
+
+The Utility creates the Sites and Observations for the chosen PV plants, uploads 12 months of training data, and shares the data with Providers A-F. Providers A-F download the training data and create their models. An anonymous user is created for each Provider by the Administrator, which helps ensure an unbiased evaluation of the forecasts by masking the identity of the Providers from the Utility and each other. The Providers are granted a debugging period prior to the official start of the trial, where they can test that their forecast generation and submission processes are working properly. Then the trial starts officially and the Providers submit forecasts separately to the two trials for the duration of the evaluation period. Missing forecasts are penalized according to predefined rules. At the conclusion of the trials, reports are automatically generated for all participants.
+
+Based on the results of the trial, the Utility can request discussions with all or a subset of the anonymized Forecast Providers. The Administrator notifies the selected Providers to contact the Utility for follow-on discussions.
+
+**Example 2**: Similar to Example 1, except the Utility wishes to evaluate probabilistic forecasts instead of deterministic. The same general process applies, with the key differences being (1) the Forecast Providers submit probabilistic forecasts for predefined probabilities, e.g., 5th, 10th, ..., 95th percentiles, and (2) the forecasts are evaluated using probabilistic forecast error metrics, e.g., CRPS.
+
+**Example 3**: Similar to Example 1, except the Utility wishes to evaluate sub-hourly time resolution forecasts, e.g., 1-hour ahead, 15-minute interval average forecasts instead of 1-hour ahead, 1-hour interval average forecasts. The same general process applies, with the key differences being (1) the Utility providers Observation data as 15-minute interval averages instead of 1-hour and (2) the Forecast Providers submit four 15-minute interval forecast values per hour instead of one 1-hour interval. Note that the Solar Forecast Arbiter can handle resampling Observation data and therefore the Utility could provide, e.g., 1-minute interval averages, with the Solar Forecast Arbiter then automatically resampling the 1-minute data to 15-minute when evaluating the forecasts.
 
 
 **Requirements**:
@@ -125,6 +133,7 @@ Forecast User 1 creates the Sites, Observations, and Aggregate, uploads 12 month
 - Framework accepts uploads of regularly updated data and allows for download of regularly updated data ([4.C](#uc4C)).
 - Framework generates intermediate reports during the trial ([3.D](#uc3D)).
 
+
 #### 1.G. Compare multiple overlapping forecast runs to measurements (stretch) {#uc1G}
 {: .anchor}