'AWS: How to add tensorflow modules from EFS to Lambda function dependencies?
I'm trying to deploy my machine learning model to AWS Lambda function. I trained my model locally with Keras and got .h5 file.
I zipped Tensorflow and Keras modules and added it to EFS mounted on EC2 instance. Then, i successfully added the EFS to the file system of Lambda function.
print("EFS Directory :",os.listdir("/mnt/efs"))
EFS Directory : ['utils.py', 'model.py', 'parallel_model.py', 'm_rcnn.py', 'mask_rcnn_object_0008.h5', 'python2.zip']
(python2.zip is a zip folder of Tensorflow and Keras modules)
The last step is to load my model with keras.models.load_model() to load my .h5 model and predict the result. The issue is that i can't import the dependencies from python2.zip.
Firstly, i tried
sys.path.insert(0, "/mnt/efs/python2.zip")
from python2.keras.models import load_model
I got
Response
{
"errorMessage": "No module named 'python2'",
"errorType": "ModuleNotFoundError",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 49, in lambda_handler\n from python2.keras.models import load_model\n"
]
}
Then, I used the tool zipimport.zipimporter.find_module() to find the path to keras.model at 'python2/tensorflow-1.13.1.data/purelib/tensorflow/python/keras/models' but it return 'None'
Does anyone have ideas how to add keras.models from python2.zip in EFS to the Lambda function path? My goal is just to use load_model and predict function from Keras.
Thank you
EDIT
I used pip3 install tensorflow-cpu in EC2 cli which seems to work since i didn't get an error when open python3 editor and import tensorflow in EC2 cli.
pip3 show tensorflow-cpu
Name: tensorflow-cpu
Version: 2.8.0
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: [email protected]
License: Apache 2.0
Location: /home/ec2-user/.local/lib/python3.7/site-packages
Requires: six, google-pasta, tensorflow-io-gcs-filesystem, protobuf, termcolor, libclang, numpy, flatbuffers, tf-estimator-nightly, gast, typing-extensions, keras-preprocessing, h5py, astunparse, opt-einsum, grpcio, keras, absl-py, tensorboard, setuptools, wrapt
Required-by:
However, i still unable to add that tensorflow dependencies to Lambda function.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
