How to load pickle(.pkl) file from S3 bucket

Question

I have successfully read the csv file from Amazon S3. But I have .pkl file of sentiment model. I want to load this .pkl file to predict sentiment. Here is my code -

import cPickle
import boto3
import pandas as pd
import boto3.session

session = boto3.session.Session(region_name='eu-central-1')
s3client = session.client('s3', config= boto3.session.Config(signature_version='s3v4'),aws_access_key_id='my-ACCESS-KEY-ID',
         aws_secret_access_key='my-ACCESS-KEY')

response = s3client.get_object(Bucket='sentiment-data', Key='positive_model_data.pkl')

nb_detector = cPickle.load(open(response['Body']))
nb_predict = nb_detector.predict('food is very good')[0]
print nb_predict 

Error coercing to Unicode: need string or buffer, StreamingBody found

How to load pickel file from S3???

Answer 1

cPickle.load() method requires a file. You need to use loads method instead of load. loads requires string data, as stated in the error message. However, response['Body'] gives you StreamingBody. StreamingBody has a method named read which can return string content.

...
body_string = response['Body'].read()
positive_model_data = cPickle.loads(body_string)
print positive_model_data
...

Does it work for you?

Answer 2

boto3 client returns a streaming body type when you subscript using ['Body'] you need to first read the byte content in the streaming body before loading it. This is a working implementation using your code subsection.

s3_data = response['Body'].read() #read byte data

nb_detector = pickle.load(s3_data) #load pickle data

nb_predict = nb_detector.predict('food is very good')[0]
print(nb_predict)