Score obtained from cross_val_score is RMSE or MSE?

Question

I am using following code:-

from sklearn.model_selection import cross_val_score
accuracies = cross_val_score(estimator = regressor, X=X,y=y, cv =10)
accuracies.mean()

This mean value is RMSE or MSE ?

EDIT:- I am using random forest regression. In Scikit learn documentation they describe it as accuracy. How can i relate it with RMSE or MSE

Answer 1

It is actually neither RMSE nor MSE. If you look into the documentation of cross_val_score, you can see that it has a parameter scoring for which it says:

If None, the estimator’s default scorer (if available) is used.

In your case, this means it will use the default scorer of the RandomForestRegressor. When you look up the documentation for its .score() method, it tells you:

Return the coefficient of determination R^2 of the prediction.

This means you computed the mean R^2. If you want to change this behavior, you have to specify the scoring parameter of cross_val_score. Options can be found here.

Answer 2

Scores obtained from cross_val_score regressor are by default 'r2' (R-squared), if you want to get RMSE you can use 'neg_root_mean_squared_error' and then change the sign. Sklearn sets a negative score because an optimization process usually seeks to maximize the score. But in this case, by maximizing it, we would be seeking to increase the error. I think, this was resolved by changing the sign of the score. If you would like to use other metrics, you can find them at scikit-learn page.

Here is how you could get the RMSE scores

from sklearn.model_selection import cross_val_score
accuracies = cross_val_score(estimator = regressor, X=X,y=y,scoring='neg_mean_squared_error', cv =10)
accuracies.mean()