'How to get precision and recall from gridsearch results?
So I ran the following code to tune a model (XGBoost). All I am looking for is how to calculate the precision and recall for the best model below.
How exactly do I implement the recall_score and precision_score from best model below? Do I need anything else besides what I have?
from sklearn.model_selection import PredefinedSplit
import numpy as np
# Combine the feature matrix in the training and validation data
X_train_valid = np.vstack((X_train, X_valid))
# Combine the target vector in the training and validation data
y_train_valid = np.append(y_train, y_valid)
# Get the indices of training and validation data
train_valid_idxs = np.append(np.full(X_train.shape[0], -1), np.full(X_valid.shape[0], 0))
# The PredefinedSplit
ps = PredefinedSplit(train_valid_idxs)
from xgboost.sklearn import XGBClassifier
models = {'xgbc': XGBClassifier(seed=42)}
###Pipeline Dictionary
from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
pipes = {}
for acronym, model in models.items():
pipes[acronym] = Pipeline([('model', model)])
param_grids = {}
##XGBoost
# The grids for eta LEARNING RATE
eta_grids = [.1]
# The grids for gamma
gamma_grids = [0.5]
# The grids for lambda
lambda_grids = [10 ** i for i in range(-1, 2)]
# Update param_grids
param_grids['xgbc'] = [{'model__eta': eta_grids,
'model__gamma': gamma_grids,
'model__lambda': lambda_grids}]
# The list of [best_score_, best_params_, best_estimator_] obtained by GridSearchCV
best_score_param_estimator_gs = []
for acronym in pipes.keys():
# GridSearchCV
gs = GridSearchCV(estimator=pipes[acronym],
param_grid=param_grids[acronym],
scoring='roc_auc',
n_jobs=-1,
cv=ps,
return_train_score=True)
# Fit the pipeline
gs = gs.fit(X_train_valid, y_train_valid)
# Update best_score_param_estimator_gs
best_score_param_estimator_gs.append([gs.best_score_, gs.best_params_, gs.best_estimator_])
Given this best performing model, all I am seeking is how I would now calculate precision and accuracy on this best model.
If it makes it any easier, let's say that the best predictor was:
- ETA: .1
- GAMMA: .5
- LAMBDA: 10
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
