Cocktail: Leveraging Ensemble Learning for Optimized Model Serving in Public Cloud

no code implementations • 9 Jun 2021 • Jashwant Raj Gunasekaran, Cyan Subhra Mishra, Prashanth Thinakaran, Mahmut Taylan Kandemir, Chita R. Das

Towards this, we proposeCocktail, a costeffective ensembling-based model serving framework. Cock-tailcomprises of two key components: (i) a dynamic modelselection framework, which reduces the number of modelsin the ensemble, while satisfying the accuracy and latencyrequirements; (ii) an adaptive resource management (RM)framework that employs a distributed proactive autoscalingpolicy combined with importance sampling, to efficiently allo-cate resources for the models.

Ensemble Learning Management