Prediction is difficult, even when it's about the past: A hindcast experiment using Res-IRF, an integrated energy-economy model

Highlights: • We perform a retrospective simulation of an energy-economy model over 1984-2012. • The model qualitatively replicates most observed trends. • Yet some observed evolutions are under-estimated: energy consumption per m² and the switch from fuel-oil to natural gas. • We discuss possible explanations for the discrepancies and whether they are problematic for long-term projections. • We conclude that hindcast experiments are useful to assess model performance. • Understanding the causes of discrepancies with observations is essential to improve models performance. -- Abstract: Model-based projections of energy demand are hardly ever confronted with observations. This shortfall threatens the credibility policy-makers might attach to integrated energy-economy models. One reason for it is the lack of historical data against which to calibrate models, a prerequisite for attempting to replicate past trends. In this paper, we (i) assemble piecemeal historical data to reconstruct the energy performance of the residential building stock of 1984 in France; (ii) calibrate Res-IRF, a bottom-up model of residential energy demand in France, against these data and run it to 2012. In a preliminary simulation with model parameters based only on the data that were known at the beginning of the simulated period, we find that the model accurately predicts energy consumption per m² aggregated over all dwelling types: the Mean Absolute Percentage Error is below 1.5% and 85% of the variance is explained, which builds confidence in the general accuracy of the Res-IRF model. Then we run 1920 simulations covering the uncertainty surrounding the parameters of the initial year. Even in simulations which fit the data best, energy demand is unevenly well replicated across fuels, which reveals some limitations in the ability of the model to capture politically-driven policies such as the expansion of the natural-gas distribution network. We discuss the directions for data collection which would ease comparison between simulations and observations in future hindcast experiments.