I’m looking to combine the Autopilot AutoML training and deployment step into a wider Pipeline using Sagemaker.
I can’t find anything in the documentation around using AutoML with Pipelines.
Does anyone have any experience with trying something like this with Sagemaker?
4
Answers
I don’t believe pipelines support AutoML, you can see a list of supported steps here – https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html
You could potentially add in a Lambda step and trigger an AutoML job from the lambda function. I’d also recommend rethinking the flow, since AutoML runs and trains using multiple algorithms. While it is great for exploration, it might not be the best for reusable pipelines.
Pipelines don’t support an AutoML step today. Check out this blog post which shows how to integrate AutoML into your Pipeline using Lambdas.
https://aws.amazon.com/blogs/machine-learning/move-amazon-sagemaker-autopilot-ml-models-from-experimentation-to-production-using-amazon-sagemaker-pipelines/
Particularly the example of how to launch AutoML in a Lambda function is here:
https://github.com/aws/amazon-sagemaker-examples/blob/main/autopilot/sagemaker-autopilot-pipelines/start_autopilot_job.py
You can now integrate Sagemaker autopilot step into your pipeline as per this example:
Good Luck
Sagemaker pipeline now only supports ensembling training mode of AutoML as a native pipeline step which you can refer to the public doc and example notebook.