Grid search arriving at differents best parameters

Question

I am running a Neural Network (MLP) for a time series analysis. I have defined a grid search, but each time I fit the model with the training set, the "best_estimator_" shows a different setting. The only way I have found to keep the output is by defining the random state inside the MLP. The drawback about this is that this doesn't necessarily arrive to the best solution.

What can I do to keep the best estimators instead of running the model multiple times until I arrive to the setting that perform the best? Below is the code I am using to find this parameters:

np.random.seed(1234)

mlp = MLPRegressor(max_iter=100) #If I define here the random state, the output is always the same but it is not necessarily the optimal

hidden_layer_sizes = [(50,50,50), (50,100,50), (100,)]
activation = ['tanh','relu']
solver = ['sgd', 'adam']
alpha = [0.001, 0.01,0.1]
learning_rate = ['constant','adaptive']

clf = GridSearchCV(estimator=mlp, param_grid=dict(hidden_layer_sizes=hidden_layer_sizes,activation=activation,
                                                  solver=solver,alpha=alpha,learning_rate=learning_rate), n_jobs=-1, cv=5)
MLP_optimal = clf.fit(X.iloc[:,1:],X.iloc[:,:1]).best_estimator_

Here the X.iloc[:,1:] represents my features (the lags of the time series) and X.iloc[:,:1] represents my Y (the sales I am using to train my model).

Here are some examples of my outputs for this code with the same training set:

MLPRegressor(activation='tanh', alpha=0.1, hidden_layer_sizes=(50, 50, 50), learning_rate='adaptive', max_iter=100, solver='sgd')

MLPRegressor(alpha=0.1, hidden_layer_sizes=(50, 50, 50),learning_rate='adaptive', max_iter=100)

MLPRegressor(alpha=0.001, hidden_layer_sizes=(50, 100, 50),learning_rate='adaptive', max_iter=100)

As you can see, each time I run it I arrive to a different setting. Any help would be much appreciated!