'Value in checkpoint could not be found in the restored object

I'm hitting an error when trying to restore a tensorflow model after the training has completed. In particular, I have a GAN and I separately save generator and discriminator weights to be restored later on. I do like this:

Save model weights for generator and discriminator CNNs (both are classes inheriting from tf.keras.Model):

g_ckp_dir = os.path.join(checkpoint_dir, 'generator')
d_ckp_dir = os.path.join(checkpoint_dir, 'discriminator')
generator.save_weights(os.path.join(g_ckp_dir, 'checkpoint'))
discriminator.save_weights(os.path.join(f"{d_ckp_dir}", 'checkpoint'))

Then, I load as:

g_ckp_dir = os.path.join(checkpoint_dir, 'generator')
d_ckp_dir = os.path.join(checkpoint_dir, 'discriminator')
generator.load_weights(os.path.join(g_ckp_dir, 'checkpoint'))
discriminator.load_weights(os.path.join(f"{d_ckp_dir}", 'checkpoint'))

However, the model does not seem to be restored correctly because predictions are not meaningful, and I get also the warnings below:

WARNING:tensorflow:Detecting that an object or model or tf.train.Checkpoint is being deleted with unrestored values. See the following logs for the specific values in question. To silence these warnings, use status.expect_partial(). See https://www.tensorflow.org/api_docs/python/tf/train/Checkpoint#restorefor details about the status object returned by the restore function.

WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).to_image_ops.4.kernel WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).to_image_ops.4.bias WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).to_image_ops.5.kernel WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).to_image_ops.5.bias WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).to_image_ops.6.kernel WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).to_image_ops.6.bias WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).spade_blocks.0.conv1.kernel WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).spade_blocks.0.conv1.bias WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).spade_blocks.0.conv2.kernel WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).spade_blocks.0.conv2.bias WARNING:tensorflow:Value in checkpoint could not be found in the restored object: (root).spade_blocks.1.conv1.kernel ...

What is wrong with this? Why do I get such an error?

I use Python 3.8.10, and TensorFlow 2.8.0



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source