ImportError when from transformers import BertTokenizer

Question

My code is:

import torch
from transformers import BertTokenizer
from IPython.display import clear_output

I got error in line from transformers import BertTokenizer:

ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /mnt/home/wbj/anaconda3/envs/pytorch/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-x86_64-linux-gnu.so)

and I found an answer that the problem is due to the file /lib/x86_64-linux-gnu/libm.so.6, when I use code strings /lib/x86_64-linux-gnu/libm.so.6 | grep GLIBC_ I get the output

GLIBC_2.2.5
GLIBC_2.4
GLIBC_2.15
GLIBC_2.18
GLIBC_2.23
GLIBC_PRIVATE

The file doesn't support GLIBC_2.29. How can I fix the problem?

Answer 1

I had the same issue. This is due to the version of GLIBC that the tokenizers package requires

You have two options

1. You can upgrade the glibc libraries if you have the rights

2. The option that I have used. Installing an older version of tokenizers, for example with anaconda

In this second case, you can just run this command:

conda install -c huggingface tokenizers=0.10.1 transformers=4.6.1

Note: You can choose other versions for transformers, in this case the errors just come when you select newer versions of tokenizers

Answer 2

Yes, this was due to my transformers version running on Ubuntu 18.04 LTS. I followed this path:

conda install -c huggingface tokenizers=0.10.1 transformers=4.6.1

However, this is not ideal if your dependencies rely on some other packages which need a greater version of transformers and tokenizers.

I found this was to due with Ubuntu not being up to scratch for the task, so you are faced:

• The first option is to migrate your application to a system that supports GLIBC higher than or equal to 2.29. This would mean a lot of work though. It seems Ubuntu 19.04 actually uses that version.

• The second option would be to actually build your GLIBC from source using the version you want or need. I’ve researched it a bit and found a website which actually gives you the steps for you to build the package from source :

http://www.linuxfromscratch.org/lfs/view/9.0-systemd/chapter05/glibc.html

I hope this helps any fellow Ubuntu 18.04 users.