Abstract
Regression analysis makes up a large part of supervised machine learning, and consists of the prediction of a continuous independent target from a set of other predictor variables. The difference between binary classification and regression is in the target range: in binary classification, the target can have only two values (usually encoded as 0 and 1), while in regression the target can have multiple values. Even if regression analysis has been employed in a huge number of machine learning studies, no consensus has been reached on a single, unified, standard metric to assess the results of the regression itself. Many studies employ the mean square error (MSE) and its rooted variant (RMSE), or the mean absolute error (MAE) and its percentage variant (MAPE). Although useful, these rates share a common drawback: since their values can range between zero and +infinity, a single value of them does not say much about the performance of the regression with respect to the distribution of the ground truth elements. In this study, we focus on two rates that actually generate a high score only if the majority of the elements of a ground truth group has been correctly predicted: the coefficient of determination (also known as R -squared or R 2 ) and the symmetric mean absolute percentage error (SMAPE). After showing their mathematical properties, we report a comparison between R 2 and SMAPE in several use cases and in two real medical scenarios. Our results demonstrate that the coefficient of determination ( R -squared) is more informative and truthful than SMAPE, and does not have the interpretability limitations of MSE, RMSE, MAE and MAPE. We therefore suggest the usage of R -squared as standard metric to evaluate regression analyses in any scientific domain.
Keywords
Mean squared errorStatisticsMathematicsLinear regressionRegression analysisRegressionMetric (unit)Mean absolute percentage errorRange (aeronautics)Binary numberGround truthCoefficient of determinationArtificial intelligenceComputer scienceArithmetic
Affiliated Institutions
Related Publications
performance: An R Package for Assessment, Comparison and Testing of Statistical Models
A crucial part of statistical analysis is evaluating a model's quality and fit, or performance.During analysis, especially with regression models, investigating the fit of model...
Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis
Statistical heterogeneity and small-study effects are 2 major issues affecting the validity of meta-analysis. In this article, we introduce the concept of a limit meta-analysis,...
R-Squared Measures for Count Data Regression Models with Applications to Health-Care Utilization
For regression models other than the linear model, R-squared type goodness-to-fit summary statistics have been constructed for particular models using a variety of methods. The ...
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression
Bounding box regression is the crucial step in object detection. In existing methods, while ℓn-norm loss is widely adopted for bounding box regression, it is not tailored to the...
Least Median of Squares Regression
Abstract Classical least squares regression consists of minimizing the sum of the squared residuals. Many authors have produced more robust versions of this estimator by replaci...
Publication Info
- Year
- 2021
- Type
- article
- Volume
- 7
- Pages
- e623-e623
- Citations
- 3896
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
3896
OpenAlex
Cite This
Davide Chicco,
Matthijs J. Warrens,
Giuseppe Jurman
(2021).
The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation.
PeerJ Computer Science
, 7
, e623-e623.
https://doi.org/10.7717/peerj-cs.623
Identifiers
- DOI
- 10.7717/peerj-cs.623