A comparison of six metamodeling techniques applied to building performance simulations

Highlights: • Linear regression (OLS), support vector regression (SVR), regression splines (MARS). • Random forest (RF), Gaussian processes (GPR), neural network (NN). • Accuracy, time, interpretability, ease-of-use, model selection, and robustness. • 13 problems modelled for 9 training set sizes spanning from 32 to 8192 simulations. • Methodology for comparison using exhaustive grid searches and sensitivity analysis. - Abstract: Building performance simulations (BPS) are used to test different designs and systems with the intention of reducing building costs and energy demand while ensuring a comfortable indoor climate. Unfortunately, software for BPS is computationally intensive. This makes it impractical to run thousands of simulations for sensitivity analysis and optimization. Worse yet, millions of simulations may be necessary for a thorough exploration of the high-dimensional design space formed by the many design parameters. This computational issue may be overcome by the creation of fast metamodels. In this paper, we aim to find suitable metamodeling techniques for diverse outputs from BPS. We consider five indicators of building performance and eight test problems for the comparison six popular metamodeling techniques – linear regression with ordinary least squares (OLS), random forest (RF), support vector regression (SVR), multivariate adaptive regression splines, Gaussian process regression (GPR), and neural network (NN). The methods are compared with respect to accuracy, efficiency, ease-of-use, robustness, and interpretability. To conduct a fair and in-depth comparison, a methodological approach is pursued using exhaustive grid searches for model selection assisted by sensitivity analysis. The comparison shows that GPR produces the most accurate metamodels, followed by NN and MARS. GPR is robust and easy to implement but becomes inefficient for large training sets compared to NN and MARS. A coefficient of determination, R², larger than 0.9 have been obtained for the BPS outputs using between 128 and 1024 training points. In contrast, accurate metamodels with R² values larger than 0.99 can be achieved for all eight test problems using only 32–256 training points.