How can I Get, Edit and Set Gradient Matrix in Training of Keras Model?

Question

I am creating a sparse neural network as described in the image below. Keras only provides a dense layer and we can't choose how many neurons we want to be connected to the previous layer. For implementing this using Keras, I am trying to implement the following approach:

1- Get the Gradient Matrix of each layer in each epoch

2- Multiply Gradient Matrix with a mask matrix to make it sparse

3- Update new weights according to that updated gradient matrix

I could not be able to find a gradient matrix in Keras. How can I get it and update it during epochs?

tf.GradientTape() is only for see gradients.

Thanks in Advance. Please see attach picture below

Sparse Network Image