Empirical Studies of Load Generation for Data Center Networks

Abstract

Data center networks form the backbone for modern Internet giants like Google, Facebook, Microsoft, and Amazon. Such networks consist of a layer of a large number of storage and compute servers connected through two or three layers of switches. Depending on the target application, data center networks can be designed to have the desired properties. In many cases, the service providers have to meet service level agreements for target applications. These agreements include the provision for redundant paths, the acceptable level of encryption, and the acceptable latency. The design choice available at the onset is architecture selection. Once the architecture is fixed, the general design choices include path selection, admission control, and encryption levels. The performance of a data center network depends critically on the traffic profile in the network. Unfortunately, most traffic data is proprietary and seldom available to academic researchers. The industry benchmark performance tests are performed for periodic and burst packet traffic transmission. These are not representative of real traffic in the data center networks. In this work, we focus on the problem of representative load generation for the empirical studies of data center networks through emulation and simulation. The main characteristic of traffic is the inter-arrival time of flows for each source destination pair, their corresponding lifetimes, and their bandwidths. In this thesis, we study the type of traffic required and how to generate realistic traffic that is rich enough to test the data center networks. We propose an open source simulation framework to understand the type of traffic required and an open source realistic traffic generator which plays real life traffic traces, which would help us understand the response of the data center networks in production. The traffic generator contains a library of traffic patterns that can be used for the load testing by varying lifetime, packet types, data rate and a few other traits obtained from traffic archive groups like MAWI, NLANR, WIDE, etc.