Enhancing Contact Center Experiences with Generative AI and Amazon SageMaker Inference_SPEEDY AUTOSCALING RELEASE REFERENCE
Cisco’s Webex Collaboration AI team is at the forefront of leveraging AI-driven features to enhance its products and services. With a focus on generative AI and large language models (LLMs), the team has been able to improve productivity and user experiences, particularly in the realm of customer engagement solutions like Webex Contact Center. However, as the models grew in size and complexity, the team faced challenges in efficiently allocating resources and scaling applications.
To address these challenges, Cisco worked with Amazon SageMaker Inference to optimize its AI/ML infrastructure. By migrating LLMs to SageMaker, Cisco was able to improve speed, scalability, and price-performance. This architectural shift allowed for better resource utilization and streamlined development, testing, and deployment of new AI-powered features for the Webex portfolio.
One notable improvement came in the form of faster autoscaling with SageMaker’s new predefined metric types. By utilizing high-resolution metrics like SageMakerVariantConcurrentRequestsPerModelHighResolution, Cisco saw up to a 50% improvement in end-to-end inference latency. This enhancement enabled faster detection of scaling needs and more efficient allocation of resources, ultimately leading to improved performance and efficiency for their critical Generative AI applications.
Looking ahead, Cisco plans to continue working with SageMaker Inference to drive further improvements in variables that impact autoscaling latencies, such as model download and load times. With this new feature, Cisco looks forward to broadening its rollout in multiple regions and delivering even more impactful generative AI features to its customers.
The collaboration between Cisco and Amazon SageMaker highlights the power of AI-driven innovation in enhancing collaboration experiences and customer engagement solutions. With a focus on leveraging advanced technologies like LLMs and generative AI, Cisco is paving the way for more efficient and personalized customer interactions. As the partnership continues to evolve, we can expect to see even more exciting developments in the realm of AI-driven collaboration.