'Boto3 import not found by SageMaker inference endpoint
I have a PyTorch model deployed on SageMaker, having an inference endpoint. Its configuration is serverless. The structure of the tar.gz model package is as follows (its has been uploaded to S3):
model
|-code
|-shared
|-aws_utils.py
|-utils.py
|-inference.py
|-requirements.txt
|- model.pkl
The basic structure has been described here.
The inference.py script contains:
from model.code.shared import method_from_utils
The method previously referenced is in utils.py, that also has an import:
from model.code.shared.aws_utils import AWSUtils
And, here's the important part, aws_utils.py has:
import boto3
After invoking the SageMaker inference endpoint (version 1.4.0), even though the requirements.txt contains boto3==1.22.3, I get the following error:
File "/opt/ml/model/code/inference.py", line 9, in <module>
from model.code.shared.utils import method_from_utils
File "/.sagemaker/mms/models/model/model/code/shared/utils.py", line 4, in <module>
from model.code.shared.aws_utils import AWSUtils
File "/.sagemaker/mms/models/model/model/code/shared/aws_utils.py", line 1, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
I have also tried with PyTorch version 1.3.1 or 1.5.0, and then I get the error:
response = runtime.invoke_endpoint(
File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 415, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 745, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received server error (500) from model with message "No module named 'fastai'". See https://us-west-1.console.aws.amazon.com/cloudwatch/home?region=us-west-1#logEventViewer:group=/aws/sagemaker/Endpoints/pytorch-inference-2022-04-29-11-09-36-580 in account XXXXXX for more information.
Needless to say, in requirements.txt I have fastai==2.6.0. In the logs all I see is:
2022-04-29 11:24:10,941 [INFO ] W-9000-model ACCESS_LOG - /127.0.0.1:39076 "POST /invocations HTTP/1.1" 500 10588
The PyTorchModel is as follows and I do not have any env variables set:
model = PyTorchModel(model_data=model_data,
framework_version="1.5.0",
py_version="py3",
role=role,
entry_point='./{}/code/inference.py'.format(name.split(".tar")[0]),
source_dir=""
)
What am I missing?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
