Unimplemented Error Node: 'sequential/conv1d/Conv1D' DNN library is not found running in Jupyter on Windows

Question

I have the Cuda version 11.2.2_461.33 with the Nvidia driver 11.2.109, cudnn version cudnn-11.2-windows-x64-v8.1.1.33 for Windows 10. I am running tensorflow version 2.8.0 in Jupyter notebook with Python 3.9 through anaconda. I have GPU enabled successfully.

so I am getting errors when I try to fit this model

# Model Definition with Conv1D
model_conv = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),
    tf.keras.layers.Conv1D(filters, kernel_size, activation='relu'),
    tf.keras.layers.GlobalMaxPooling1D(),
    tf.keras.layers.Dense(dense_dim, activation='relu'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

the error is

Epoch 1/10
---------------------------------------------------------------------------
UnimplementedError                        Traceback (most recent call last)
Input In [6], in <cell line: 4>()
      1 NUM_EPOCHS = 10
      3 # Train the model
----> 4 history_conv = model_conv.fit(training_padded, training_labels, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels))

File ~\.conda\envs\tf-gpu\lib\site-packages\keras\utils\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File ~\.conda\envs\tf-gpu\lib\site-packages\tensorflow\python\eager\execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     52 try:
     53   ctx.ensure_initialized()
---> 54   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     55                                       inputs, attrs, num_outputs)
     56 except core._NotOkStatusException as e:
     57   if name is not None:

UnimplementedError: Graph execution error:

Detected at node 'sequential/conv1d/Conv1D' defined at (most recent call last):
    File "C:\Users\me\.conda\envs\tf-gpu\lib\runpy.py", line 197, in _run_module_as_main
      return _run_code(code, main_globals, None,

...

Node: 'sequential/conv1d/Conv1D'
DNN library is not found.
     [[{{node sequential/conv1d/Conv1D}}]] [Op:__inference_train_function_842]

I have installed cuda and the drivers following these instructions for my tensorflow version (cuda 11.2, cuDNN 11.2 for tensorflow 2.8.0):

https://www.tensorflow.org/install/source_windows

(I have not installed bazel though as I use anaconda)

and this guide here, step by step:

https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#installcuda-windows

Also I get this error when running a bidirectional layer:

InternalError: Graph execution error:

Failed to call ThenRnnForward with model config: [rnn_mode, rnn_input_mode, rnn_direction_mode]: 2, 0, 0 , [num_layers, input_size, num_units, dir_count, max_seq_length, batch_size, cell_num_units]: [1, 64, 64, 1, 1551, 256, 64] 
     [[{{node CudnnRNN}}]]
     [[sequential/bidirectional/backward_lstm/PartitionedCall]] [Op:__inference_train_function_5897]

So it looks like my installation have not gone smoothly after all. hopefully, someone can advise something.