'Optimizing Sum of TensorFlow Model Predictions
I'm relatively new to TensorFlow and have a question. I've fit a series of models using the following command:
models = []
for i in range(10):
g = Sequential()
g.add(InputLayer(input_shape = (3,)))
g.add(Dense(nodes, activation))
g.add(Dense(1))
g.compile(loss = 'mse')
g.fit(X[i], Y[i])
models.append(g)
I'd now like to optimize the following function
def objective(input):
return np.sum([g.predict(input) for g in models])
I use a standard scipy optimizer to do this optimization,
scipy.optimize.minimize(func = objective, x0 = [0, 0, 0])
but the problem is that while doing so I get the following warnings:
WARNING:tensorflow:5 out of the last 5 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fa5ce80a820> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
WARNING:tensorflow:6 out of the last 6 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7fa5ce396040> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
I'm clearly doing something obviously wrong, but would tremendouly appreciate any suggestions! I'm not sure how to deal with the tracing issues given this problem where I'm calling the predictions from multiple networks.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
