'How to check the understanding of a trained model?

I'm currently training two models (BERT & MPNet) for a semantic textual similarity (STS) task using the SentenceTransformers library.

Now I want to check, if the base models and/or the trained models understand specific words/names which occur within the training dataset. I tried masking or calculating the similarity to specific categories related to the words/names but the results were hardly distinguishable.

Is there any way to check or even prove if a model understands a specific word or sequence?

huggingface-transformersbert-language-modelsentence-similaritysentence-transformers


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions

why do pooler use tanh as a activation func in bert, rather than gelu?

Does Fine-tunning Bert Model in multiple times with different dataset make it more accuracy?

huggingface transformers convert logit scores to probability

Why does BERT Model fail to find an option that matches my input positional arguments?

How to apply max_length to truncate the token sequence from the left in a HuggingFace tokenizer?

Pretraining a language model on a small custom corpus

what's the difference between "self-attention mechanism" and "full-connection" layer?

Target Data Missing from tensorflow fit()

Bert embedding layer raises 'ValueError: A target array with shape ' with BiLSTM in keras tensorflow

ReadError: file could not be opened successfully. But I am not sure where the tar file is stored to resolve this

Continual pre-training vs. Fine-tuning a language model with MLM

Early stopping in Bert Trainer instances

how to train a bert model from scratch with huggingface?

transformers and BERT downloading to your local machine

How to specify a proxy in transformers pipeline