In tf-agent, what is `reg_loss` in DQN

Question

When I was monitoring my DQN training, I noticed that there's a reg_loss part to my total loss. I don't know what this comes from, or what it means. It decreases during training, but does not seem to be used when calculating gradients.

Code snippet from dqn_agent.py

# line 500, reg_loss is not calculated here, but from q_network
# I didn't find definition of losses in the Network interface
      agg_loss = common.aggregate_losses(
          per_example_loss=td_loss,
          sample_weight=weights,
          regularization_loss=self._q_network.losses)
......
# line 535, notice here only td_loss is returned, which then used to get gradient
      return tf_agent.LossInfo(total_loss, DqnLossInfo(td_loss=td_loss,
                                                       td_error=td_error))

Can anyone tell me what this loss is and what's the use of it? Thanks.