Amazon SageMaker AI real-time endpoints now support KV cache aware routing
· AI-assisted summary · reviewed by Reinventy Solutions Corp.

Amazon SageMaker AI real-time endpoints now support KV cache-aware routing, which directs requests to the instance with the longest cached prompt match.
