I have been having a hard time to deploy my locally trained SKlearn model (pipeline with custom code + logistic model) to Sagemaker Endpoint. My Pipeline is as follows: All this custom code (RecodeCategorias) does is normalize and recode some…
With the AWS CLI ec2 describe-instance-types I can get a list of all the EC2 instance types but Sagemaker Instance Types like ml.t3.medium, ml.t3.large, ml.m5.xlarge, etc are not there. aws ec2 describe-instance-types --filters "Name=instance-type,Values=ml.*" --query "InstanceTypes[].{Type: InstanceType, MaxENI: NetworkInfo.MaximumNetworkInterfaces, IPv4addr:…
When trying to run one of the Neptune-ML examples in a notebook instance when running the line endpoints=neptune_ml.setup_pretrained_endpoints(s3_bucket_uri, setup_node_classification, setup_node_regression, setup_link_prediction, setup_edge_classification, setup_edge_regression) it returns None, and I can see the following error in the logs: ERROR:root:Unable to determine the…
I'm migrating our ML notebooks from Azure Databricks to AWS environment using Sagemaker and Step functions. I have separate notebooks for data processing, feature engineering and ML algorithms which I want to run in a sequence after completion of previous…
I've got an XGBoost model trained leveraging Sagemaker Hyperparameter Tuning job. Now, I want to generate predictions for about 182GB of Csv files. I've been testing different combinations of instance_types, counts, MaxPayloadInMB, and MaxConcurrentTransforms but haven't been able to run…
When trying to use Huggingface estimator on sagemaker, Run training on Amazon SageMaker e.g. # create the Estimator huggingface_estimator = HuggingFace( entry_point='train.py', source_dir='./scripts', instance_type='ml.p3.2xlarge', instance_count=1, role=role, transformers_version='4.17', pytorch_version='1.10', py_version='py38', hyperparameters = hyperparameters ) When I tried to increase the version…
I was working on a sagemaker studio for ML work, I attached Lifecycle Configuration with it, which was creating problem. Then I deleted the lifecycle configuration without detaching it, and this problem is happening. Can't start sagemaker studio notebook and…
After I login into a SageMaker Studio with a specific user profile, how can I find out which user profile I am logged in with? I know I can easily figure out which execution role I am using, but I…
Is there a way to use the PyTorch profiler when using Amazon SageMaker? Or can we call PyTorch profiler in the profiler config for SageMaker debugger
For an AI project I want to train a model over a dataset which is about 300 GB. I want to use the AWS SageMaker framework. In SageMaker documentation, they write that SageMaker can import data from AWS S3 bucket.…