'pickling error for a sklearn pipeline with custom transformations

I am trying to pickle a Pipeline with some custom transformation over my features and store it in Google cloud storage but i am getting pcikling error

PicklingError: Can't pickle <function train_xgboost_model.<locals>.email_length at 0x7fd17cc4b680>: it's not found as __main__.train_xgboost_model.<locals>.email_length

Example of how my Pipeline is created :

xgb_model = XGBClassifier(eval_metric='logloss', 
                          use_label_encoder=False)

get_email_length = FunctionTransformer(email_length)
get_domain_length = FunctionTransformer(domain_length)

preproc = FeatureUnion([
    ('email_length', get_email_length),
    ('domain_length', get_domain_length)])

pipe = Pipeline([
    ('preproc', preproc),
    ('classifier', xgb_model)
    ])

and whenever i want to pickle pipe like this

joblib.dump(pipe, "gs://...etc..")

i have the Pickling error from above



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source