The Symmetric Mean Absolute Percentage Error: Unnecessary or Dangerous

The symmetric Mean Absolute Percentage Error (sMAPE) is a forecast error metric that has been proposed as an alternative to the more common Mean Absolute Percentage Error (MAPE), which is undefined whenever an actual is zero; the sMAPE does not have this problem. Thus, the sMAPE at first glance appears to be more suitable for evaluating forecasts of low volume or intermittent count demand time series. However, the sMAPE suffers from a number of other shortcomings; e.g., it is 2 for a zero actual regardless of the forecast, it always rewards (elicits) integer forecasts 0, 1, 2, …, if actuals are counts, and it elicits a (typically useless) zero forecast for sufficiently intermittent actuals. This paper collects such properties and discusses their real-world implications so the forecaster can make an informed decision as to whether to use the sMAPE or an alternative. In our opinion, the sMAPE is either unnecessary or dangerous; it should not be used.