I am getting "Failed precondition" error message

Question

I am attempting to run a GoogLeNet code, but when I run the following via the terminal:

python.exe googlenet_cifar10.py --model output/minigooglenet_cifar10.hdf5 --output output

I am new to python, so I attempted to research it but having issues finding a solution for it. This is out my output.

PS C:\Users\JoshG\PycharmProjects\GoogLeNet> python.exe googlenet_cifar10.py --model output/minigooglenet_cifar10.hdf5 --output output
2022-05-17 23:35:08.398853: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
[INFO] loading CIFAR-10 data...
[INFO] compiling model...
2022-05-17 23:35:10.617462: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library nvcuda.dll
2022-05-17 23:35:10.629376: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: 
pciBusID: 0000:06:00.0 name: NVIDIA GeForce RTX 3080 computeCapability: 8.6
coreClock: 1.71GHz coreCount: 68 deviceMemorySize: 10.00GiB deviceMemoryBandwidth: 707.88GiB/s
2022-05-17 23:35:10.629433: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2022-05-17 23:35:10.643459: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll
2022-05-17 23:35:10.643494: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll
2022-05-17 23:35:10.646207: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cufft64_10.dll
2022-05-17 23:35:10.647015: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library curand64_10.dll
2022-05-17 23:35:10.653755: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusolver64_11.dll
2022-05-17 23:35:10.655811: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusparse64_11.dll
2022-05-17 23:35:10.656141: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll
2022-05-17 23:35:10.656230: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2022-05-17 23:35:10.656508: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-17 23:35:10.657260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:06:00.0 name: NVIDIA GeForce RTX 3080 computeCapability: 8.6
coreClock: 1.71GHz coreCount: 68 deviceMemorySize: 10.00GiB deviceMemoryBandwidth: 707.88GiB/s
2022-05-17 23:35:10.657440: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2022-05-17 23:35:10.968121: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-05-17 23:35:10.968243: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0 
2022-05-17 23:35:10.968320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N
2022-05-17 23:35:10.968494: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7440 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3080, pci bus id: 0000:06:00.0, compute capability: 8.6)
[INFO] training network...
2022-05-17 23:35:11.692348: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
Epoch 1/5
2022-05-17 23:35:21.386941: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll
2022-05-17 23:35:21.964261: I tensorflow/stream_executor/cuda/cuda_dnn.cc:359] Loaded cuDNN version 8101
2022-05-17 23:35:22.716959: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll
2022-05-17 23:35:23.236591: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll
2022-05-17 23:35:23.711216: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
781/781 [==============================] - 27s 19ms/step - loss: 1.7178 - accuracy: 0.3670 - val_loss: 1.2684 - val_accuracy: 0.5425
Epoch 2/5
781/781 [==============================] - 13s 17ms/step - loss: 1.1309 - accuracy: 0.5967 - val_loss: 1.1878 - val_accuracy: 0.5845
Traceback (most recent call last):
  File "C:\Users\JoshG\PycharmProjects\GoogLeNet\googlenet_cifar10.py", line 84, in <module>
    model.fit(aug.flow(trainX, trainY, batch_size=64),
  File "C:\Users\JoshG\AppData\Local\Programs\Python\Python39\lib\site-packages\keras\engine\training.py", line 1204, in fit
    callbacks.on_epoch_end(epoch, epoch_logs)
  File "C:\Users\JoshG\AppData\Local\Programs\Python\Python39\lib\site-packages\keras\callbacks.py", line 414, in on_epoch_end
    callback.on_epoch_end(epoch, numpy_logs)
  File "C:\Users\JoshG\PycharmProjects\GoogLeNet\pipeline\callbacks\trainingmonitor.py", line 58, in on_epoch_end
    plt.plot(N, self.H["acc"], label = "train_acc")
KeyError: 'acc'
2022-05-17 23:35:52.743176: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
         [[{{node PyFunc}}]]
PS C:\Users\JoshG\PycharmProjects\GoogLeNet> 

I get the following error message:

2022-05-17 17:19:00.803770: W tensorflow/core/kernels/data/generator_dataset_op.cc:107] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
         [[{{node PyFunc}}]]

Below is the entire code and i am using CIFAR-10 Dataset

import matplotlib
matplotlib.use("Agg")

    # import packages
    from sklearn.metrics import classification_report
    from sklearn.preprocessing import LabelBinarizer
    from pipeline.nn.conv import MiniGoogLeNet
    from pipeline.callbacks import TrainingMonitor
    from keras.preprocessing.image import ImageDataGenerator
    from keras.callbacks import LearningRateScheduler
    from keras.optimizers import SGD
    from keras.datasets import cifar10
    import numpy as np
    import argparse
    import os
    import tensorflow as tf
    
    # define the total number of epochs to train for along with initial learning rate
    NUM_EPOCHS =70
    INIT_LR = 5e-3
    
    def poly_decay(epoch):
        # initialize the maximum number of epochs, base learning rate,
        # and power of the polynomial
        maxEpochs = NUM_EPOCHS
        baseLR = INIT_LR
        power = 1.0
    
        # compute the new learning rate based on polynomial decay
        alpha = baseLR * (1 - (epoch / float(maxEpochs))) ** power
    
        # return the new learning rate
        return alpha
    
    # construct the argument parser
    ap = argparse.ArgumentParser()
    ap.add_argument("-m", "--model", required = True, help = "path to output model")
    ap.add_argument("-o", "--output", required = True,
        help = "path to output directory (logs, plots, etc.)")
    args = vars(ap.parse_args())
    
    # load the training and testing data, converting the image from integers to floats
    print("[INFO] loading CIFAR-10 data...")
    ((trainX, trainY), (testX, testY)) = cifar10.load_data()
    trainX = trainX.astype("float")
    testX = testX.astype("float")
    
    # apply mean subtraction to the data
    mean = np.mean(trainX, axis = 0)
    trainX -= mean
    testX -= mean
    
    # convert the labels from integers to vectors
    lb = LabelBinarizer()
    trainY = lb.fit_transform(trainY)
    testY = lb.transform(testY)
    
    # initialize the label name for CIFAR-10 dataset
    labelNames = ["airplane", "automobile", "bird", "cat", "deer",
        "dog", "frog", "horse", "ship", "truck"]
    
    # construct the image generator for data augmentation
    aug = ImageDataGenerator(width_shift_range = 0.1, height_shift_range = 0.1,
        horizontal_flip = True, fill_mode = "nearest")
    
    # construct the set of callbacks
    figPath = os.path.sep.join([args["output"], "{}.png".format(os.getpid())])
    jsonPath = os.path.sep.join([args["output"], "{}.json".format(os.getpid())])
    callbacks = [TrainingMonitor(figPath, jsonPath = jsonPath),
        LearningRateScheduler(poly_decay)]
    
    # initialize the optimizer and model
    print("[INFO] compiling model...")
    opt = SGD(lr = INIT_LR, momentum = 0.9)
    model = MiniGoogLeNet.build(width = 32, height = 32, depth = 3, classes = 10)
    #model.compile(loss = "categorical_crossentropy", optimizer = opt, metrics = ["accuracy"]
    model.compile(optimizer = tf.keras.optimizers.Adam(learning_rate=0.001), loss = 'categorical_crossentropy', metrics = ["accuracy"])
    
    # train the network
    print("[INFO] training network...")
    model.fit(aug.flow(trainX, trainY, batch_size = 10),
        validation_data = (testX, testY), steps_per_epoch = len(trainX) // 64,
        epochs = NUM_EPOCHS, callbacks = callbacks, verbose = 1)
    
    # evaluate network
    print("[INFO] evaluating network...")
    predictions = model.predict(testX, batch_size = 64)
    print(classification_report(testY.argmax(axis = 1),
        predictions.argmax(axis = 1), target_names = labelNames))
    
    # save the network to disk
    print("[INFO] serializing network...")
    model.save(args["model"])

    print('Test1')
    #Run Command: python.exe googlenet_cifar10.py --model output/minigooglenet_cifar10.hdf5 --output output

Answer 1

I fixed the issue. The solution was: replacing ["acc"] with ["accuracy"] everywhere.

In my case, I was unable to plot the parameters of the history of my training. I had to replace

plt.plot(N, self.H["acc"], label = "train_acc")
                plt.plot(N, self.H["val_acc"], label = "val_acc")

to

 plt.plot(N, self.H["accuracy"], label = "train_accuracy")
            plt.plot(N, self.H["val_accuracy"], label = "val_accuracy")