'How to get a placeholder/dummy step in SageMaker pipelines?

I'd like to sketch out some steps in a SageMaker pipeline, and only fill them in one at a time, but I don't think there's an EmptyStep option anywhere.

I've considered using some vacuously true ConditionalSteps, or subclassing sagemaker.workflow.steps.Step, but the former can't be chained, and the latter seems likely to break things, given my implementation wouldn't necessarily conform to what the service is looking for.

Is there a good way to go about this? An empty processor step?



Solution 1:[1]

There's no way to create your own empty step in a SageMaker Pipeline. The easiest way for you to achieve this would be to use a LambdaStep and create stub Lambda functions. With a processor you will pay the penalty of cold start for each job.

I work at AWS and my opinions are my own.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Kirit Thadaka