'run_eagerly=True make the training result different in Tensorflow 2.3.2
Recently I come across a strange question in Running Neural network code on Tensorflow 2.3.2. The question is that when I only changed run_eagerly=True to run_eagerly=False in the config
model.compile(
loss={"label": "binary_crossentropy"},
metrics=tf.keras.metrics.AUC(name="auc"),
run_eagerly=True
)
the model will get quite different results, Mainly about AUC and LOSS changes the AUC and Loss in run_eagerly=True is:
INFO:root:batch (99), speed (67.83 qps/s), loss (7.76), auc (0.50)
INFO:root:batch (199), speed (77.42 qps/s), loss (7.70), auc (0.50)
INFO:root:batch (299), speed (77.81 qps/s), loss (7.69), auc (0.50)
INFO:root:batch (399), speed (75.01 qps/s), loss (7.64), auc (0.50)
INFO:root:batch (499), speed (70.51 qps/s), loss (7.68), auc (0.50)
INFO:root:batch (599), speed (77.87 qps/s), loss (7.70), auc (0.50)
INFO:root:batch (699), speed (75.42 qps/s), loss (7.70), auc (0.50)
INFO:root:batch (199), speed (107.17 qps/s), loss (1.12), auc (0.51)
INFO:root:batch (299), speed (100.84 qps/s), loss (1.00), auc (0.52)
INFO:root:batch (399), speed (98.40 qps/s), loss (0.93), auc (0.53)
INFO:root:batch (499), speed (101.34 qps/s), loss (0.89), auc (0.55)
INFO:root:batch (599), speed (102.09 qps/s), loss (0.86), auc (0.56)
INFO:root:batch (699), speed (94.13 qps/s), loss (0.83), auc (0.57)
My model is defined as follows:
import logging
import tensorflow as tf
from models.core.fully_connected_layers import MultiHeadLayer, get_fc_layers
from .base_net import BaseNet
class SingleTowerNet(BaseNet):
def __init__(self, features, net_conf, **kwargs):
super(SingleTowerNet, self).__init__(**kwargs)
self.features = features
self.net_conf = net_conf
self.user_hidden_num_list = [int(i) for i in self.net_conf["User"].split(",")]
self.train_features = features.train_feature_names
self.user_feature_names = features.user_feature_names
self.label_name = features.label_feature_names[0]
assert len(features.label_feature_names) == 1, "must have only one label name"
self.preprocess_layers = self.get_deep_process_layer_map(self.train_features, features)
self.user_fc_layers = get_fc_layers(self.user_hidden_num_list)
def get_user_embedding(self, user_features):
user_emb_list = [self.preprocess_layers[name](user_features[name]) for name in
self.user_feature_names]
user_concat = tf.concat(user_emb_list, axis=1)
user_embedding = self.user_fc_layers(user_concat)
user_embedding_norm = tf.math.l2_normalize(user_embedding, axis=1)
return user_embedding_norm
def call(self, train_data, training=False):
user_feature = {name: train_data[name] for name in self.user_feature_names}
user_embedding_norm = self.get_user_embedding(user_feature)
d = tf.math.reduce_sum(user_embedding_norm * 10, axis=1)
p = tf.math.sigmoid(d)
if training:
return {self.label_name: p}
else:
label = tf.squeeze(train_data[self.label_name])
ad_group_id = tf.squeeze(train_data["ad_group_id"])
return {"predict": p,
"label": label,
"ad_group_id": ad_group_id}
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|