Performance evaluation of BDI forecasting models: Cross efficiency, the directional distance function and the AVS utility function

Abstract

In the existing literature, the methodology for assessing the relative performance of competing forecasting models is unidimensional since the forecasting results are compared to each other by considering a single criterion at a time. Consequently, conflicting results about the performance of specific forecasting approaches are often reported because some of them perform better than others with respect to a different criterion. In this study, we propose a nonparametric efficiency measurement approach for the forecasting model selection problem. Three autoregressive models and three fuzzy time series approaches are employed for the calibration of the data structure to depict the trend of Baltic Dry Index (BDI). A directional distance function is defined that looks for possible increases in accuracy, skewness and decreases in variance obtained by cross efficiencies of those forecasting models. We also establish a link to the proper indirect accuracy-variance-skewness (AVS) utility function for various users in various utilities. An empirical section on a set of BDI forecasting models serves as an illustration