how to do a gridsearch through a for loop and a function?

Question

I have defined a function taking different ML models as arguments, such as Support Vector Machine or XGBoost. As it is necessary to have optimum values of each method, I have to use the grid search method but I do not know how to embed the grid search in my function.

Here is my whole function:

def define_model(model_type):
    if model_type == "linear":
        model = LinearRegression()
    elif model_type == "mlp":
        model = MLPRegressor(
        hidden_layer_sizes=(2),
        activation='identity',
        solver='lbfgs',
        max_iter=100000000000)
    elif model_type == 'KN':
        model = KNeighborsRegressor(
        n_neighbors=6, weights='distance', 
        algorithm='ball_tree', leaf_size=60, p=2,
        metric='minkowski', metric_params=None,
        n_jobs=None)
    elif model_type == 'Dec_tree':
        model = DecisionTreeRegressor( criterion='squared_error', splitter='best', max_depth=33, 
                              min_samples_split=2,
                              min_samples_leaf=1, min_weight_fraction_leaf=0.0,max_features=None, 
                              random_state=None, max_leaf_nodes=None, 
                              min_impurity_decrease=0.0, ccp_alpha=0.0)
    elif model_type == 'SV':
        model = SVR( kernel='rbf', degree=5, gamma='scale', coef0=0.0, tol=0.001, C=10.0, 
                     epsilon=0.1, shrinking=True, cache_size=200, verbose=False, max_iter=- 1)
        
    elif model_type == 'XGBoost':
        model = xg.XGBRegressor(objective ='reg:linear',
        n_estimators = 600, seed = 123)

        
    return model

now if I want to do grid search for example for SV or mlp, should written something like below:

    elif model_type == 'clr_svr':
        svr = SVR()

        parameters_svr = {'kernel': ('linear', 'rbf','poly'), 'C':[1.5, 10],'gamma': [1e-7, 1e-4],'epsilon':[0.1,0.2,0.5,0.3]}

        clr_svr = GridSearchCV(svr, parameters_svr)
        
    return model

But it did not work, and it gives this error

UnboundLocalError: local variable 'parameters_svr' referenced before assignment

while I have given the param list before and after the function. Could you please help me out?

Answer 1

It is simply done by defining the parameter list in a separate cell and calling it like this:

KNN = KNeighborsRegressor()

param_list = {'n_neighbors':[5,10,15,20,25,30,40,50],'weights': ('uniform','distance'),'algorithm':('ball_tree','kd_tree','brute'),'leaf_size':[30,35,40,45,50],'p':[2,3,4,5],'n_jobs':[1,2,3]}

then once you were defining the function for the models it is just a matter of calling gridsearch. Like below:

....
    elif model_type == 'KN':
        model = GridSearchCV(estimator=KNN,  param_grid=param_list)
....