How to Get Reproducible Results (Keras, Tensorflow):

Question

To make the results reproducible I've red more than 20 articles and added to my script maximum of the functions ... but failed.

In the official source I red there are 2 kinds of seeds - global and operational. May be, the key to solving my problem is setting the operational seed, but I don't understand where to apply it.

Would you, please, help me to achieve reproducible results with tensorflow (version > 2.0)? Thank you very much.

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import pandas as pd
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split
from keras.optimizers import adam
from sklearn.preprocessing import MinMaxScaler


np.random.seed(7)
import tensorflow as tf
tf.random.set_seed(7) #analogue of set_random_seed(seed_value)
import random
random.seed(7)
tf.random.uniform([1], seed=1)
tf.Graph.as_default #analogue of  tf.get_default_graph().finalize()

rng = tf.random.experimental.Generator.from_seed(1234)
rng.uniform((), 5, 10, tf.int64)  # draw a random scalar (0-D tensor) between 5 and 10

df = pd.read_csv("s54.csv", 
                 delimiter = ';', 
                 decimal=',', 
                 dtype = object).apply(pd.to_numeric).fillna(0)

#data normalization
scaler = MinMaxScaler() 
scaled_values = scaler.fit_transform(df) 
df.loc[:,:] = scaled_values


X_train, X_test, y_train, y_test = train_test_split(df.iloc[:,1:],
                                                    df.iloc[:,:1],
                                                    test_size=0.2,
                                                    random_state=7,
                                                    stratify = df.iloc[:,:1])

model = Sequential()
model.add(Dense(1200, input_dim=len(X_train.columns), activation='relu'))  
model.add(Dense(150, activation='relu'))
model.add(Dense(80, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1, activation='sigmoid')) 

loss="binary_crossentropy"
optimizer=adam(lr=0.01)
metrics=['accuracy']
epochs = 2
batch_size = 32
verbose = 0

model.compile(loss=loss,  
              optimizer=optimizer, 
              metrics=metrics) 
model.fit(X_train, y_train, epochs = epochs, batch_size=batch_size, verbose = verbose)
predictions = model.predict(X_test)
tn, fp, fn, tp = confusion_matrix(y_test, predictions>.5).ravel()

Answer 1

As of TensorFlow 2.8, there is tf.config.experimental.enable_op_determinism().

You can ensure reproducibility, even on GPU, through

import tensorflow as tf

tf.keras.utils.set_random_seed(42)  # sets seeds for base-python, numpy and tf
tf.config.experimental.enable_op_determinism()

Answer 2

When we use layers >= 3 and neurons >= 100, number of the processor' cores is meaningful. My problem was to run the script on the 2 different servers:

-with 32 cores

-with 16 cores

When I ran the script on the servers with 32 cores (and even with 32 and 24 cores), results were identical.