Simple transformer by huggingface dump using joblib

Question

I've got a question about dumping a simple transofmer model.

I trained a model, than dump it using joblib :

dump(model, 'Simple_transfomers.joblib') 

and when I load it in the other script using :

files = glob.glob('Simple_transfomers.joblib')

model = ClassificationModel(
    "bert", "models/files"
)

or even tried other way :

model = ClassificationModel(
    "bert", "outputs/pytorch_model.bin"
)

I recieved :

404 Client Error: Not Found for url: https://huggingface.co/*outputs/pytorch_model.bin*/resolve/main/config.json

I think the problem could be using google colab and store the model on google drive.

Is there any possibility to read the gdrive directory using ClassificationModel ?

Read the docs on https://simpletransformers.ai/docs/usage/ but cannot find any solution.

Answer 1

If your dump there is joblib.dump, you can simply use joblib.load

model = joblib.load("Simple_transfomers.joblib")

joblib is like pickle (it even uses pickle I think) that dumps a python object to a file.