How to pass environment variables while deploying sagemaker endpoint?

Question

I'm trying to send a parameter as an environment variable for my deployed model. I ran a hyperparameter tuning run and I want to pass the string for one of the model parameters into the deployed endpoint.

I'm loading my trained PyTorch model with:

inference_model = PyTorchModel(
entry_point="inference.py",
source_dir="serve",
role=role,
model_data=model_data_s3_path,
env={'MODEL_ARCHITECTURE': best_architecture_name}, 
framework_version="1.8.1",
py_version="py36",
)

I'm using "env" to try and grab it during inference, but it doesn't seem to work when I use os.environ['MODEL_ARCHITECTURE']. I get an error that there is no such environment variable? What am I missing? Should I even be passing the string this way?

I'm also using os.environ['MODEL_ARCHITECTURE'] in the model.py script, which is called in inference.py.

How can I pass the string along? I need to do it this way (it needs to be automated, I can't change the model name by hand) since I'll be handing it off to some people to run it straight through and they can't manually change the architecture name.

EDIT: I tried SM_ARCHITECTURE_NAME since SageMaker often adds "SM_" to the environment variables / hyperparameters during training, but that didn't work either.

Answer 1

This is the correct way to pass environment variables to an endpoint. Can you make sure you are using the latest version of the SageMaker SDK and try this again?