UNCERTAINTY MAPPING FOR STATE-OF-THE-ART DNI DAY

AHEAD FORECASTS

Marion Schroedter-Homscheidt
1

1
Dr. Marion Schroedter-Homscheidt, Deutsches Zentrumfr Luft- und Raumfahrt e.V. (DLR), Deutsches
Fernerkundungsdatenzentrum, D-82234 Oberpfaffenhofen, Tel. +49 8153 282896, marion.schroedter-homscheidt@dlr.de
Abstract
The successful grid integration requires day-ahead irradiance forecasts typically obtained from numerical
weather prediction. Nowadays the accuracy of irradiance predictions is not known in advance for a potential
power plant location. A continental map of uncertainty metrics can help to answer the question if there are
areas in Europe and Northern Africa with more favorable conditions for successful GHI and DNI forecasts
than other regions. It is therefore a help to identify regions providing a more reliable and less intermittent PV,
CPV and CSP electricity production with respect to meteorological conditions.
In this study we evaluate the global horizontal (GHI) and direct normal irradiance (DNI) forecast
uncertainties over Europe and Northern Africa for the European Centre for Medium-Range Weather
Forecasts (ECMWF) irradiance forecasts. The selection of metrics consist of classical parameters as bias or
root mean square errors, but is extended to typical meteorological verification parameters as skill scores to
evaluate against persistence approaches or to spatial error correlation maps. The uncertainty is quantified
through comparisons versus ground and satellite-based irradiance observations.
Keywords: irradiance forecasts, numerical weather forecast, ECMWF, SOLEMI, BSRN, spatial error
mapping
1. Introduction
The need to decarbonize the energy system for environmental and economic reasons will lead to larger shares
of renewable energies in the energy system. One of the major challenges is the integration of renewable
electricity generation into existing energy grids. Solar energy is an important example for an intermittent
energy resource providing a high potential for the replacement of fossil fuels and therefore environmental
impact reductions.
Compared to the nowadays state the future electricity grid will rely on the extension of the power lines on the
transmission network level and smart grid technologies making use of enhanced information availability on
all grid levels, but especially on the distribution network level. Additionally, the use of heat and/or electricity
storage technologies is promising in order to provide base power capabilities. For these purposes additional
information layers about the solar resource are needed: The extension of the transmission network needs
mainly information about the temporal and spatial distribution of renewable energy sources and their
interaction. Besides this request for historical databases, the daily operation of the grid requires solar
production forecasts and therefore meteorological forecasts of global and direct irradiances for photovoltaic
and concentrating technologies from the photovoltaic and solar thermal sector, respectively. This holds
especially also for the distribution network level and the large (e.g. concentrating solar thermal) power plant
operators acting either on the distribution or the transmission network level. It has also been shown in case
studies using data from individual days, that the combination of solar power prediction and storage
capabilities at concentration power plants will increase solar energy yield integrated into the electricity
supply [1].

Techniques as solar power prediction for the upcoming few hours up to two days for the day-ahead markets
are currently under development ([2]; [3]). Such irradiance forecasts are certainly a subject to errors in
forecasts as any other meteorological forecast. They have to be specified as basic information for these user
communities. In this study, we focus on the use of solar forecasts for the intra-day and day-ahead electricity
market and therefore, hourly forecasts are assessed. The transmission network level will mainly make profit
from knowledge about both the regional and the continental scale spatial correlation of solar forecasting
errors, while the distribution network planning and operations is mainly interested in regionally dependent
characterization of typical forecast error statistics.
Having such an error characterization would open the floor to a spatially dependent quantification of base
and reserve power capacities needed while taking the spatially different reliability of solar productions
forecasts into account. Regions with a high reliability are able to accept lower short-term reserve power
capacities than regions with a low reliability where a high solar production prediction might be wrong and
cause sudden or at least short-term needs for the provision of extra reserve capacities, resulting in increased
costs of the solar electricity use.
At the downstream end of energy system transition modeling, environmental impact assessment as a result of
an energy system transition process is performed. Changes in conventional reserve power needs can be
transferred into environmental impact studies. Such studies provide e.g. information about reduced
greenhouse gas emissions [4], about reduced loss of life due to decreased air pollution levels ([5]; [6]), about
ecosystem states with respect to acidification or eutrophication [7], biodiversity changes, soil and water
quality, land use or degradation, or the fragmentation of ecosystems.
Having this large application range in mind, this paper deals with the quantification of spatial error
correlations and regionally dependent error statistics for the European Centre for Middle-Range Weather
Forecasting (ECMWF) operational forecast scheme. For large-scale grid planning purposes the availability of
information at least on a continental scale is required, therefore only a global scale forecasting scheme is of
interest for this study. ECMWFs forecasts have been found to be of leading quality in comparisons of
nowadays operational irradiance forecasting schemes on a global scale (e.g. in [8]). Nevertheless, the existing
verification studies rely on ground measurements, which are not suited to provide spatially dependent
information due to the sparse data availability. This holds especially for publicly available ground
measurements of direct irradiances. Based on our knowledge this is the first study aiming at a spatially
resolved error characterization. It has to be noted that we aim at analyzing the currently available state of the
art as a basis for further model improvements by the meteorological developers community with respect to
solar energy needs.
The paper focuses on Europe and the Northern African region. On the one hand it makes use of Meteosat
Second Generation (MSG) based satellite observations covering Europe, Africa and the Middle East, on the
other hand it is meant as a contribution to the discussion about providing larger solar electricity generation as
regional source in Northern Africa for regional electricity and drinking water supply needs as well as
providing solar electricity as an export opportunity towards the European market (Desertec concept).
Section 2 introduces ECMWF forecasts as well as ground and satellite observations used as verification
reference, while section 3 focuses on the assessment methodology. Section 4 provides example results for
Europe and Northern Africa, while section 5 provides a conclusion and an outlook about further work
possibilities.


2. Data
2.1. ECMWF IFS forecasts
This study makes use of ECMWF (European Centre for Middle Range Weather Forecasting) deterministic
irradiance forecasts. Forecasts have been obtained on a global scale, but the study will focus on a regional
subset covering Europe and Northern Africa in a 0.25 spatial resolution. The study evaluates forecasts from
2006. An hourly temporal resolution is required following the requirements of the day-ahead and intraday
electricity markets under evaluation. Therefore and even before any direct or indirect normal irradiance is
derived, a temporal interpolation from the original 3-hourly irradiation sum towards an hourly global
irradiance forecast is performed following [9]. Due to the strong non-linear behavior of the DNI during the
day, the interpolation is done by using the clear sky index. Using a clear sky model as described in [10] the
clear sky index - defined as ratio global irradiance divided by the clear sky global irradiance - is calculated
for every third hour provided by ECMWF. The cloud index is then interpolated linearly for each hour. A
correction is applied if low morning and afternoon clear sky indices are calculated due to the 3-hourly
interval during a clear day in order to avoid large errors in clear sky cases Finally, by using the clear sky
model again, the global irradiance is calculated in hourly averages resolution. By using the Skartveith/Olseth
scheme [11] the direct irradiance is finally calculated in this hourly resolution and transferred to the direct
normal irradiance by using the cosine of the sun zenith angle.
2.2. Ground data
Global and direct irradiance measurements from the Baseline Surface Radiation Network (BSRN,
http://www.bsrn.awi.de/) have been used. They are provided in a minute average temporal resolution and
afterwards aggregated to hourly mean values. This reflects the user needs in intra-day and day-ahead
electricity markets. The study uses the stations Cabauw, Camborne, Carpentras, Lerwick, Lindenberg,
Palaiseau_Cedex, Payerne, Sede_Boquer, Tamanrasset, and Toravere. Additionally, the DLR pyrheliometer
measurements at the Plataforma Solar de Almeria, Spain, are included.
2.3. Satellite observations
Meteosat Second Generation satellite based surface solar irradiance observations are taken from the SOLEMI
database derived with the Heliosat-2 algorithm. The Heliosat method was originally proposed by [12] and
later modified by [13] and [14]. The basic idea is a two-step approach: In the first step, a relative normalized
cloud reflectivity the cloud index n is derived from METEOSAT images. The derived cloud index is
correlated to the clear-sky index k, which relates the actual ground irradiance G to the irradiance of the
cloud-free case Gclear sky. Consequently, in addition to the cloud index derived from the satellite signal, a
clear-sky model, providing Gclear sky, is necessary for the estimation of the actual ground irradiance. This
clear sky model includes knowledge on the sun geometry and atmospheric turbidity to describe aerosol and
water extinction. The nk relation is leads to a root mean square deviation (RMSD) between measured and
calculated solar irradiance for almost homogenous cloud situation of 13-14% for hourly global irradiance
values [14]. For all sky situations, RMSD values between 20 and 30% are found ([10]; [3]), while for direct
normal irradiances RMSD values between 20 and 60% are found depending on the region ([15]; [3]).
This study uses data for Europe and Northern Africa, restricting the analysis to 55N in order to avoid too
large satellite zenith angles and to 20N in order to keep the BSRN station Tamanrasset in the subset. In
longitudinal direction the subset is defined from 12 W to 40 E.
3. Methodology
Statistical parameters for the comparison of hourly values include mean bias deviation (MBD), root mean
square error (RMSE), linear correlation coefficients, and distribution parameters as the Kolmogorov-
Smirnov-Index describing the comparability of number distributions ([16]; [10]). The skill score applied is
based on the RMSD and defined as the ratio:

skill scorc =
(RHS
LCMwP
- RHS
2d_pcs
)
RHS
2d_pcs


It compares the RMSD against the RMSD of the persistence approach. The persistence approach in this case
is a two-day persistence. This is a specific feature required by the day-ahead electricity market: Typically,
any day-ahead forecast has to be provided to the market operator during the late morning hours. This restricts
the persistence to the values obtained on the previous day as the best estimate for tomorrow without using
any weather forecast.

All results are either derived spatially for a 1x1 latitude-longitude grid or at the location of ground-based
observations. Nearest neighbor ECMWF grid points or satellite pixels are selected. All parameters are
available for each forecast hour up to 48 hours, but only daytime values are evaluated following the MESOR
standard. In this paper we focus on relative RMSE and skill score values at 12 UTC of the day ahead. One
might argue that choosing a constant universal time step includes the analysis of solar energy at different
local solar times resulting in the assessment of different energy amounts due to the daily solar cycle. But on
the other hand the electricity grid at least in Europe is interconnected and electricity will flow through the
network at any time depending on demand and production without any means for the grid operator to control
that. Therefore, and for any spatial interdependence or balancing study, the information at the same time step
in UTC is more relevant.

Additionally to the mapping of statistical parameters at each grid location, also spatial error correlation maps
are generated. In this case, the error is defined as the difference of the forecasted minus the observed hourly
GHI or DNI value. Time series of errors are generated over a year and at all grid points. A single grid point is
selected as reference point. The linear correlation of the error time series at any point in the grid versus the
error time series at the reference point is calculated and plotted in a map at the location of any grid point.


Fig. 1. Map of relative RMSE values at 12 UTC for the BSRN and PSA ground stations, the year 2006
and hourly GHI (upper) and DNI (lower) forecasts. Left side maps are derived for the 2-day
persistence, right side maps are derived for the ECMWF.

4. Results
First, a map of relative RMSE values at 12 UTC at the ground measurement stations is derived both for GHI
and DNI (fig 1). It provides the most accurate assessment of forecast hours as BSRN ground measurements
have a higher accuracy. But it also illustrates the coarse data point availability in ground networks, if both
GHI and DNI measurements are requested at a ground measurement station. The value of a numerical
weather prediction scheme like ECMWF is quantified by the lower relative RMSE if compared to the two-
day persistence forecast as reference.

Spatially resolved information can only be derived on the basis of space-based observations as provided by
the SOLEMI database. Fig. 2 for GHI and Fig. 3 for DNI provide relative RMSE maps for 12 UTC both for
the two-day persistence approach and the ECMWF deterministic forecasts. It can be noticed that for the GHI
maps the reduction of relative RMSE is large due to the use of the ECMWF model. Relative RMSE values
show a nearly latitudinal distribution. Less accuracy for the ECMWF is observed only over the Alps an
area with a strong orography not fully described in the coarse ECMWF model.



Fig. 2. Map of relative RMSE values at 12 UTC for the whole grid, the year 2006, hourly GHI
forecasts based on a 2-day persistence (upper) and based on ECMWF forecasts (lower).
For the ECMWF generally a larger relative RMSE is found over most European land surfaces if compared to
the Mediterranean Sea, while such a feature is visible in the 2d-persistence map only in the Eastern
Mediterranean. For the DNI maps in most areas the relative RMSE is reduced when using the ECMWF
forecast. There are exceptions over the Alps, in Turkey, Libya, Egypt, and on the Arabian Peninsula. In the
alpine region the lower forecast accuracy again can be explained by the low spatial resolution of 0.25 in the
ECMWF model in a highly variable terrain, while there is no obvious reason over the other regions with less
accuracy.



Fig. 3. Map of relative RMSE values at 12 UTC for the whole grid, the year 2006, hourly DNI
forecasts based on a 2-day persistence (upper) and based on ECMWF forecasts (lower).

This distribution of increased or decreased RMSE values can be more clearly illustrated by a map of skill
scores. For the 12 UTC DNI map, negative skill scores up to -0.5 can be found in Egypt and the Alps, while
positive skill scores up to 0.5 occur in most regions.

Fig. 4. Map of RMSE-based skill scores at 12 UTC for the whole grid, the year 2006, and hourly GHI
(upper) and DNI (lower panel) forecasts based on a 2-day persistence as reference for the ECMWF
forecast.
Spatial correlation coefficients of forecast errors (ECMWF observation differences) can be derived with
respect to a reference station. A single grid point is chosen as reference site, being marked with a black cross.
The maps provide the correlation coefficients found for a time series of all hourly daytime forecast error
within a year, here 2006. Only the use of satellite observations reveals details about the spatial distribution of
correlations and their partly strong non-Gaussian shape.

Fig. 5 and 6 provide maps for the PSA (upper left plot), but add the same maps for arbitrary chosen locations
in Eastern Germany, South of Sicily and at the Libyan border. The only intention of this map collection is to
illustrate that the spatial correlation function results in a variety of shapes. Some locations show a Gaussian
shape, while others are characterized by a strong linearly shaped correlation function. In principle such a map
can and needs to be generated for each location in the grid as an input to further energy system analysis.
Additionally, the stability of such patterns for different years needs to assessed further.


Fig. 5. Spatial error correlation map for the whole grid, the year 2006, four locations and hourly GHI
forecasts based on the ECMWF forecast.

Fig. 6. Spatial error correlation map for the whole grid, the year 2006, four locations and hourly DNI
forecasts based on the ECMWF forecast.

5. Conclusion
An assessment of spatial error characteristics for the state-of-the-art ECMWF deterministic forecast has been
presented. The original hypothesis of the existence of a spatial dependency of error characteristics in day-
ahead global and direct normal irradiance forecasts has been confirmed. Such information can only be
derived on the basis on spatially available observations as provided from geostationary satellites, as
especially for the direct irradiances the public ground measurement network is too coarse.
It has to be further evaluated how variable these results are with respect to many years or a multi-annual
average. Additionally, the influence of the less accurate satellite observations with respect to ground
measurements with a higher accuracy at the point, but larger spatial representativeness error if compared to
model grid data, has to be further discussed.
Spatial error statistics and correlations can be calculated for any point within the satellite field of view. This
might be a CSP power plant location or a grid knot in a future electricity super grid.
Acknowledgements
We thank all BSRN station providers and the DLR Almeria colleagues for the provision of ground data and
the ECMWF centre for the access to historical forecasts for this study. The study was performed within
European Commissions FP7 EnerGEO project (grant agreement 226364).
References
[1] Wittmann, M., H. Breitkreuz, M. Schroedter-Homscheidt, M. Eck, 2008, Case Studies on the Use of Solar
Irradiance Forecast for Optimized Operation Strategies of Solar Thermal Power Plants IEEE Journal of
Selected Topics in Applied Earth Observations and Remote Sensing, 1, 1, 18 - 27, ISSN 1939- 1404, DOI:
10.1109/J STARS.2008.2001152
[2] Girodo, M. (2006). Solarstrahlungsvorhersage auf der Basis numerischer Wettermodelle. Dissertation,
Carl von Ossietzky Universitt Oldenburg
[3] Breitkreuz, H., M. Schroedter-Homscheidt, T. Holzer-Popp, S. Dech, 2009, Short range direct and diffuse
irradiance forecasts for solar energy applications based on aerosol chemical transport and numerical weather
modeling, Journal of Applied Meteorology and Climatology, Vol 48, 1766-1779,
doi:10.1175/2009JAMC2090.1
[4] Rao , S. and K. Riahi, 2006, The role of non-CO2 greenhouse gases in climate change mitigation: Long-
term scenarios for the 21st century, Energy J ournal 27(SPEC. ISS. NOV.): 177-200
[5] Amann, M., J . Cofala, C Heyes, Z. Klimont, R. Mechler, M. Posch, W. Schpp, 2004, The RAINS
model: Documentation of the model approach prepared for the RAINS peer review 2004. International
Institute for Applied Systems Analysis (IIASA), Laxenburg, Austria. (http://www.iiasa.ac.at/rains/review/
review-full.pdf)
[6] UN ECE, WHO, 2004, Modelling and assessment of the health impact of particulate matter and ozone.
United Nations Economic Commission for Europe and World Health Organization: EB.AIR/WG.1/2004/11,
Geneva, Switzerland.
[7] Amann M., et al., 2009: Air pollutants and greenhouse gases -- Options and benefits from control. In:
Air Pollution & Climate Change -- Two Sides of the Same Coin?, H. Pleijel, P.E. Karlsson, D. Simpson
(eds), Swedish Environmental Protection Agency, Stockholm,
Sweden, pp. 99-108, ISBN 978-91-620-1278-6

[8] Perez, R., S. Kivalov, J. Schlemmer, K. Hemker J r., D. Renne, T.E. Hoff, 2010, Validation of short and
medium term operational solar radiation forecasts in the US, Solar Energy, 84, 2161-2172
[9] Breitkreuz, H., Solare Strahlungsvorhersagen fr energiewirtschaftliche Anwendungen Der Einfluss
von Aerosolen auf das solare Strahlungsangebot in Europa, DLR Forschungsbericht 2008-24, 2008
[10] Hoyer-Klick, C., M. Lefvre, A. Oumbe, M. Schroedter-Homscheidt, L. Wald, Users Guide to the Soda
and Solemi services, MACC project report D_R-RAD_2.2.1, 2010
[11] Skartveit, A., J . A. Olseth and M. E. Tuft, 1998: An Hourly Diffuse Fraction Model with Correction for
Variability and Surface Albedo. Sol. Energy, 63, 173-183
[12] Cano, D., Monget, J ., Albuisson, M., Guillard, H., Regas, N., & Wald, L.(1986). A method for the
determination of the global solar radiation from meteorological satellite data. Solar Energy, 37, 3139
[13] Beyer, H., Costanzo, C., Heinemann, D. (1996). Modifications of the HELIOSAT procedure for
irradiance estimates from satellite images. Solar Energy, 56, 207 21
[14] Hammer, A., 2000, Anwendungsspezifische Solarstrahlungsinformation aus METEOSAT Daten,
Dissertation, Universitt Oldenburg, http://oops.ibit.unioldenburg.de/volltexte/ incoming/2001/347/
[15] Beyer, H.G., J . Polo Martinez, M. Suri, J . Luis Torres, E. Lorenz, S.C. Mller, C. Hoyer-Klick, P.
Ineichen, 2009, MESOR Management and Exploitation of Solar Resource Knowledge D1.1.3 Report on
Benchmarking of Radiation Products, EC project, grant 038 665
[16] Beyer, H.G.,J. Poo Martinez, M. Suri, J . Luis Torres, E. Lorenz, C. Hoyer-Klick, P. Ineichen, 2008,
MESOR Management and Exploitation of Solar Resource Knowledge D1.1.1 Handbook on
Benchmarking, EC project, grant 038 665