'How to select the best hyperparameters based on the accuracy standard deviation generated by repeated k-fold cross validation using caret::train()

We have trained a polynomial kernal SVM classifier using repeated k-fold cross validation. We would like to use the hyperparameters that generate the lowest Accuracy standard deviation rather than those that generate the highest average accuracy.

The Accuracy standard deviation can be accessed in the the results of the trained model and we can choose the row from this with the lowest AccuracySD:

   SVM.polyclassifier$results %>% 
   filter(AccuracySD == min(SVM.polyclassifier$results$AccuracySD)) 

We have no idea how to pass this information to the final model to make predictions using the selected hyperparameters. Passing them (or anything) to the bestTune object of the model has no effect on the outcome of the prediction

   SVM.polyclassifier$bestTune
   SVM.updated <- SVM.polyclassifier 
   SVM.updated$bestTune <- SVM.polyclassifier$results %>% 
    filter(AccuracySD == 
    min(SVM.polyclassifier$results$AccuracySD)) %>% 
    select(-c("Accuracy", "Kappa", "AccuracySD",    "KappaSD"))
  
    y_pred_updated <- predict(SVM.updated, type = 'raw', newdata = test_set[-1])
    y_pred <- predict(SVM.polyclassifier, type = 'raw', newdata = test_set[-1])




   
   


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source