'can one run sagemaker notebook code locally in visual studio code
The code below works fine in a sagemaker notebook in the cloud. Locally I also have aws credentials created via the aws cli. Personally, I do not like notebooks (unless I do some EDA or the like). So I wonder if one can also fire this code up from a local machine (e.g. in visual studio code) as it only tells sagemaker what to do anyway? I guess it is only a question of authenticating and getting the session object? Thanks!
import boto3
import os
import sagemaker
from sagemaker import get_execution_role
from sagemaker.inputs import TrainingInput
from sagemaker.serializers import CSVSerializer
from sagemaker import image_uris
region_name = boto3.session.Session().region_name
s3_bucket_name = 'bucket_name'
# this image cannot be used below !!! there must be an issue with sagemaker ?
training_image_name = image_uris.retrieve(framework='xgboost', region=region_name, version='latest')
role = get_execution_role()
s3_prefix = 'my_model'
train_file_name = 'sagemaker_train.csv'
val_file_name = 'sagemaker_val.csv'
sagemaker_session = sagemaker.Session()
s3_input_train = TrainingInput(s3_data='s3://{}/{}/{}'.format(s3_bucket_name, s3_prefix, train_file_name), content_type='csv')
s3_input_val = TrainingInput(s3_data='s3://{}/{}/{}'.format(s3_bucket_name, s3_prefix, val_file_name), content_type='csv')
hyperparameters = {
"max_depth":"5",
"eta":"0.2",
"gamma":"4",
"min_child_weight":"6",
"subsample":"0.7",
"objective":"reg:squarederror",
"num_round":"10"}
output_path = 's3://{}/{}/output'.format(s3_bucket_name, s3_prefix)
estimator = sagemaker.estimator.Estimator(image_uri=sagemaker.image_uris.retrieve("xgboost", region_name, "1.2-2"),
hyperparameters=hyperparameters,
role=role,
instance_count=1,
instance_type='ml.m5.2xlarge',
volume_size=1, # 1 GB
output_path=output_path)
estimator.fit({'train': s3_input_train, 'validation': s3_input_val})
Solution 1:[1]
On a local machine,
- Make sure to install AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
- Create an access key id and secret access key to access Sagemaker services locally: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html
- Set these credentials using
aws configure
command locally. - The code should work fine except getting the execution role. You can either hard code the Sagemaker role in the code (not best practice) or store it in the Parameter store and access it from there.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | Luv |