Value errors in KerasRegressor using GridSearhCV

Question

I need to find the best parameters that will ultimately inform the best price for a certain number automobile characteristics. The *csv file is attached, so as you can have the exact picture of the problems: DataAutomobile.csv

When I compile, I get the following error:

File "C:\Users\jetta\anaconda3\lib\site-packages\scikeras\wrappers.py", line 1127, in set_params raise ValueError(

ValueError: Invalid parameter activ1 for estimator KerasRegressor. This issue can likely be resolved by setting this parameter in the KerasRegressor constructor:KerasRegressor(activ1=relu). Check the list of available parameters with estimator.get_params().keys()

The other messages if I force activ1 = relu. Below is just a sample of several similar errors

ValueError: Invalid parameter neurons1 for estimator KerasRegressor. This issue can likely be resolved by setting this parameter in the KerasRegressor constructor:KerasRegressor(neurons1=100)

Code:

import pandas as pd
import tensorflow as tf 
from tensorflow.keras.models import Sequential
from tensorflow.keras import backend as k
from sklearn.model_selection import GridSearchCV
from tensorflow.keras.wrappers.scikit_learn import KerasRegressor
from scikeras.wrappers import KerasRegressor

base = pd.read_csv('autos.csv', encoding = 'ISO-8859-1')

base = base.drop('dateCrawled', axis = 1)
base = base.drop('dateCreated', axis = 1)
base = base.drop('nrOfPictures', axis = 1)
base = base.drop('postalCode', axis = 1)
base = base.drop('lastSeen', axis = 1)
base = base.drop('name', axis = 1)
base = base.drop('seller', axis = 1)
base = base.drop('offerType', axis = 1)

base = base[base.price > 10]
base = base.loc[base.price < 350000]

auto_values = {'vehicleType': 'limousine', 'gearbox': 'manuell',
           'model': 'golf', 'fuelType': 'benzin',
           'notRepairedDamage': 'nein'}
base = base.fillna(value = auto_values)

auto_predictors = base.iloc[:, 1:13].values
auto_RealPrice = base.iloc[:, 0].values

from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

onehotencoder = ColumnTransformer(transformers=[("OneHot", OneHotEncoder(), [0,1,3,5,8,9,10])],remainder='passthrough')
auto_predictors = onehotencoder.fit_transform(auto_predictors).toarray()

def auto_CreateNeural(optimizer, loss, kernel_initializer, activation, activ1, activ2, activ3, neurons1, neurons2, neurons3, kernel_ini1, kernel_ini2, kernel_ini3):
    k.clear_session()
    regressor = Sequential([
                   tf.keras.layers.Dense(units = neurons1, activation = activ1, kernel_initializer = kernel_ini1, input_dim=316),
                   tf.keras.layers.Dropout(0.2),
                   tf.keras.layers.Dense(units = neurons2, activation = activ2, kernel_initializer = kernel_ini2),
                   tf.keras.layers.Dropout(0.2),
                   tf.keras.layers.Dense(units = neurons3, activation = activ3, kernel_initializer = kernel_ini3),
                   tf.keras.layers.Dropout(0.2),
                   tf.keras.layers.Dense(units = 1, activation = 'linear')])
    regressor.compile(optimizer = optimizer, loss = loss, kernel_initializer = ['kernel_ini1', 'kernel_ini2', 'kernel_ini3'], activation = ['activ1', 'activ2', 'activ3'],
                      units = ['neurons1', 'neurons2', 'neurons3'], metrics = ['binary_accuracy'])
    return regressor

regressor = KerasRegressor(build_fn = auto_CreateNeural, verbose=0)
auto_parameters = {'batch_size': [350, 500],
              'epochs': [3, 5],
              'optimizer': ['adam', 'Adamax'],
              'loss': ['kullback_leibler_divergence','binary_crossentropy', 'hinge', 'mean_absolute_error'],
              'kernel_ini1': ['random_uniform', 'normal', 'glorot_uniform'],
              'kernel_ini2': ['normal', 'glorot_uniform'],
              'kernel_ini3': ['random_uniform', 'normal'],
              'activ1': ['relu', 'elu', 'tanh', 'softmax'],
              'activ2': ['tanh', 'softmax'],
              'activ3': ['elu', 'softmax'],              
              'neurons1': [100, 158, 200, 316],
              'neurons2': [80, 115, 158, 225, 330],
              'neurons3': [70, 90, 120, 158, 250]}
grid_search = GridSearchCV(estimator = regressor,
                           param_grid = auto_parameters,
                           scoring = 'neg_mean_absolute_error', cv = 2) 
grid_result = grid_search.fit(auto_predictors, auto_RealPrice)

# summarize results
print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_))
means = grid_result.cv_results_['mean_test_score']
stds = grid_result.cv_results_['std_test_score']
params = grid_result.cv_results_['params']

for mean, stdev, param in zip(means, stds, params):
    print("%f (%f) with: %r" % (mean, stdev, param))
best_auto_parameters = grid_search.best_params_
best_precision = grid_search.best_score_

I am new to neural networks, and despite having read several documentations for the past days, I am completely stuck and cannot go further without help. I need to find the best values for each and all of the variables described in "auto_parameters".

What should I exactly do in the code to get it working properly? If not asking much, a brief explanation of the corrections would be really valuable.

Answer 1

The errors in SciKeras could use some work, but what that error is telling you is that every argument here:

def auto_CreateNeural(optimizer, loss, kernel_initializer, activation, activ1, activ2, activ3, neurons1, neurons2, neurons3, kernel_ini1, kernel_ini2, kernel_ini3)

Requires a corresponding default value when constructing KerasRegressor. That is, you need to change:

KerasRegressor(build_fn = auto_CreateNeural, verbose=0)

To:

KerasRegressor(build_fn = auto_CreateNeural, kernel_initializer=...), activ1=..., ...)

Scikit-Learn expects estimators to work both in and out of a pipeline, without those defaults this estimator would behave differently inside and outside of a pipeline (namely, it would not work outside of a pipeline).