sagemaker: Support multi-model endpoints

[petermeansrock]

Describe the feature

As described in the SageMaker Endpoint L2 construct RFC:

Multi-Model Endpoints: By default (and as described in the technical solution above), SageMaker expects the model data URL on each container to point to an S3 object containing a gzipped tar file of artifacts, which will be automatically extracted upon instance provisioning. To support colocation of multiple logical models into a single container, the Mode attribute was added to the ContainerDefinition CloudFormation structure to either explicit configure SingleModel mode (the default) or MultiModel mode. In multi-model mode, SageMaker now expects the customer configured model data URL to point to an S3 path under which multiple gzipped tar files exist. When invoking a multi-model endpoint, the client invoking the endpoint must specify the target model representing the exact S3 path suffix pointing to a specific gzipped tar file.

Please 👍 this issue to help with the prioritization of this feature.

Use Case

Colocating multiple models behind a single endpoint can help customers save money as described here.

Proposed Solution

As described in the SageMaker Endpoint L2 construct RFC:

To accommodate this feature, the proposed ModelData.fromAsset API should be adjusted to support zip file assets capable of containing one or more gzipped tar files within them. Even though the code need not be aware of .tar.gz files specifically, it might prove a better customer experience to at least put up guard rails to prevent zip file assets from being used in single model mode where as multi-model mode could be more permissive.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.54.0-alpha.0

Environment details (OS name and version, etc.)

macOS Ventura