tensorflow seems to recognize my gpu but don't use it

Question

I just followed the tutorial on how to install cuda cudnn etc to use the gpu with tensorflow. When I run:

import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))

It returns Num GPUs Available: 1

from tensorflow.python.client import device_lib 
print(device_lib.list_local_devices())

returns

[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 5352387470003847961
xla_global_id: -1
, name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 6273040384
locality {
  bus_id: 1
  links {
  }
}
incarnation: 1135099878766203926
physical_device_desc: "device: 0, name: NVIDIA GeForce RTX 2070 Super, pci bus id: 
0000:01:00.0, compute capability: 7.5"
xla_global_id: 416903419
]

So I think that my GPU is recognized. However when I launch:

m=32
refmodel = tf.keras.Sequential()
refmodel.add(Flatten(input_shape=data_train[0].shape))
refmodel.add(Dense(m, activation='sigmoid'))
refmodel.add(Dense(len(classes)))

refmodel.compile(optimizer='adam',
          loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
          metrics=['accuracy'])
t=time()
with tf.device('/gpu:0'):
    history1 = refmodel.fit(data_train, target_train, epochs=7, validation_data= 
(data_test, target_test), batch_size=64)
print(time()-t)

It seems to take exactly the same time to run as before. Is this normal (with a RTX 2070S) or I am doing something wrong?