Blas xGEMMBatched launch failed on 3080ti x 2 GPUs Tensorflow 1.14, CUDA 10.0, CudNN 7.6.5

Question

I am working on the extension of the following method, which involves retraining network. https://github.com/brownvc/matryodshka

The PC I am using has the following specifications:

1. Ubuntu 20.04 LTS
2. NVIDIA 3080ti (x2) GPUs
3. 64 GB RAM

I have created the conda environment with the provided yml file, and separately installed cudatoolkit==10.0.130 and cudnn=7.6.5.

Then training with the provided code, returns this error when the GPUs are utilized.

Blas xGEMMBatched launch failed

On the other hand, the training runs without error if I disable GPU support.

I tried available solutions to resolve this problem but to no benefit, such as: I set config.gpu_options.allow_growth to True.

I don't think it is an OOM error, as there is no such text in the log file.