'pickling error for a sklearn pipeline with custom transformations
I am trying to pickle a Pipeline with some custom transformation over my features and store it in Google cloud storage but i am getting pcikling error
PicklingError: Can't pickle <function train_xgboost_model.<locals>.email_length at 0x7fd17cc4b680>: it's not found as __main__.train_xgboost_model.<locals>.email_length
Example of how my Pipeline is created :
xgb_model = XGBClassifier(eval_metric='logloss',
use_label_encoder=False)
get_email_length = FunctionTransformer(email_length)
get_domain_length = FunctionTransformer(domain_length)
preproc = FeatureUnion([
('email_length', get_email_length),
('domain_length', get_domain_length)])
pipe = Pipeline([
('preproc', preproc),
('classifier', xgb_model)
])
and whenever i want to pickle pipe like this
joblib.dump(pipe, "gs://...etc..")
i have the Pickling error from above
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
