Spreading the load: How Salesforce met Multi-AZ HA with SageMaker Inference Components
· By Antonio Sedino, CTRO · Published by Reinventy Solutions Corp.
Salesforce leverages Amazon SageMaker AI Inference Component placement to distribute model copies across multiple Availability Zones, meeting Multi-AZ high availability compliance requirements without sacrificing the cost efficiency of multi-model co-hosting.
Salesforce used Amazon SageMaker AI Inference Component placement (the SchedulingConfig parameter) to distribute model copies across multiple Availability Zones. This approach meets Multi-AZ high availability compliance requirements while maintaining the cost efficiency of multi-model co-hosting. The SchedulingConfig parameter enables distribution of model copies across zones, providing resilience against zone-level failures.