| dc.description.abstract | Cloud workload orchestration plays a pivotal role in optimizing the performance, resource utilization, and cost effectiveness of applications in data centers. As modern businesses and IT operations are migrating their businesses to the cloud, understanding the dynamics of cloud data centers has become indispensable. Often, two perspectives play a pivotal role in workload orchestration in data centers. One is from the cloud provider side, whose goal is to provision as many applications as possible on the available resources biding to SLA constraints thereby increasing return on investment. Other being from the side of enterprises and individual customers, often referred to as end users, whose primary objective is to ensure application performance with reduction in deployment cost. Containerization has gained popularity for deploying applications on public clouds, where large enterprises manage numerous applications through thousands of containers placed onto Virtual Machines (VMs). While the need for cost efficient placement in cloud data centers is undeniable, the complexities involved in achieving this goal cannot be understated. This problem is usually modelled as a multi-dimensional Vector Bin packing Problem (VBP). Solving VBP optimally is NP-hard and practical solutions requiring real-time decisions use heuristics. This work explores the landscape of cloud data centers, emphasizing the significance of efficient bin packing in achieving optimal cost and resource utilization. Traditional methods, including heuristics and optimal algorithms, face limitations in handling continuous request arrivals and the dynamic nature of cloud workloads. Integer Linear Programming (ILP), which can provide optimal solutions for small problem sizes with tens of requests, may take minutes to hours to complete even at such scales. Moreover, optimal algorithms inherently demand perfect knowledge of all current and future requests to be placed within the bins, rendering them unsuitable for the dynamic and often unpredictable online placement scenarios prevalent in cloud setups. To address these challenges, this work introduces a novel approach to solving VBP through Reinforcement Learning (RL), trained on the historical container workload trace for an enterprise a.k.a. CARL (Cost-optimized container placement using Adversarial Reinforcement Learning). The proposed work evaluates the effectiveness of CARL in comparison to traditional methods. CARL leverages historical container workload traces, learning from a semi-optimal VBP solver while optimizing VM costs. The contributions of this research extend beyond traditional methods, providing insights into the advantages and disadvantages of heuristics, optimal algorithms, and learning approaches. We trained and evaluated CARL on workloads derived from realistic traces from Google Cloud and Alibaba for the placement of 10,000 container requests onto over 8000 VMs. CARL is fast, making placement decisions for request sets with 124 containers per second within 65 ms onto 1000s of potential VMs. It is also efficient, achieving up to 13.98% lower VM costs than baseline heuristics for larger traces. To push the boundaries further, the research uses Mixture of Experts (MoE) strategy in CARL wherein multiple experts are used that helps CARL in learning placement policies of various approaches combined. The inclusion of a MoE strategy enhances CARL’s adaptability to changes in workload distribution, ensuring competitive performance in scenarios with skewed resource needs or inter-arrival times. | en_US |
