How to predict the accuracy of input data after training a neural network

Question

The essence of the problem: I wanted to understand the simplest neural network using a ready-made example from the Internet. I trained it, and then I don't understand how to test its validity on the input data of a conditional user. I found functions for model on the Internet: predict(), save(), loaded_model(). If save() and loaded_model() turned out and the folder '16_model' was created, then errors are thrown from predict(). Please tell me how to use it or how to test the neural network on input, and not on test data.

import keras
from keras.models import Sequential
from keras.layers import Dense
import numpy

numpy.random.seed(2)

dataset = numpy.loadtxt("diabet.csv", delimiter=",")
X, Y = dataset[:,0:8], dataset[:,8]

model = Sequential()
model.add(Dense(12, input_dim=8, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss="binary_crossentropy", optimizer="adam", metrics=['accuracy'])

model.fit(X, Y, epochs = 100, batch_size=10)

scores = model.evaluate(X, Y)
#model.save('16_model')
#model_loaded = keras.models.load_model('16_model')
print("\n%s: %.2f%%" % (model.metrics_names[1], scores[1]*100))

Answer 1

You can split the dataset into training and testing part as below, and use those training set to train the model and test dataset to evaluate the model:

import pandas as pd
from sklearn.model_selection import train_test_split
dataset =pd.read_csv("/content/../diabetes.csv")#, delimiter=",")
dataset.head()
X = dataset.drop('Outcome', 1)
y = dataset['Outcome']
Y.shape

train_X, test_X, train_Y, test_Y=train_test_split(X,y, test_size=0.2)
train_X.shape

Then after defining and training the model with train dataset, you can evaluate the model with test dataset.

model_loss, model_acc = model.evaluate(test_X, test_Y)

print("Model Accuracy", model_acc*100)
model.save('16_model')
model_loaded = keras.models.load_model('16_model')

Output:

5/5 [==============================] - 0s 2ms/step - loss: 0.5637 - accuracy: 0.7208
Model Accuracy 72.07792401313782
INFO:tensorflow:Assets written to: 16_model/assets

Now evaluating the model again after saving and loading back the same model which shows same accuracy as before of saving the model.

model_loss, model_acc = model_loaded.evaluate(test_X, test_Y)

Output:

5/5 [==============================] - 0s 3ms/step - loss: 0.5637 - accuracy: 0.7208

Here is Prediction Part:

(This will show all the value within the range 0 to 1 due to sigmoid function used for binary classification)

pred=model_loaded.predict(train_X[:2])
print(pred))

Output:

[[0.925119  ]
 [0.45006576]]

All Values from <0.5 will come under class 0 and rest all will be assumed as class 1.

for p in pred:
    if p>=.5:
        pred_class=1
    else:
        pred_class=0
    print(pred_class)

Output:

1
0

You can verify this prediction with actual label:

train_Y[:2]

Output:

369    1
653    0
Name: Outcome, dtype: int64