Scaling the performance of web servers using a greedy data buffering and caching strategy

Abstract

Pervasive use of the web has placed extreme performance demands on its key architectural elements of which the web server is the most critical. A web server is a highly I/O intensive application, and I/O data handling accounts for the largest chunk of a web server’s processing time. It has been known that data copy is a key bottleneck in I/O. In this work we study the web server’s data sending path and attempt to optimize the processing by reducing data copies in the path. We propose a new method of data handling for Linux operating system. We call this method Network Aware Buffering and Caching (NABC). NABC attempts to do three things: (1) Lay out the data in memory in a way that protocol processing can be done without data copies, (2) keep a unified cache of data in kernel and ensure safe access to it by various processes and kernel, and (3) pass only the necessary metadata between processes so that the significant data handling time spent during IPC can be reduced. We discuss the implementation of NABC in Linux. We implement a set of system calls and a set of library routines atop the system calls. The end product of the implementation is a set of APIs (NABC - APIs) specifically designed for use by the web servers. For performance evaluation, we design and implement a scalable high performance web server which we call SWEET (Scalable WEb server using Events and Threads). We port SWEET to NABC APIs. We evaluate the performance using a range of workloads both simulated and real, taken from web server’s log files. The results show that server employing NABC APIs outperforms an otherwise similar server (using traditional APIs), by a margin of 12% to 21% in throughput for static file serving. For lightweight dynamic content generation, the throughput gain is an impressive 1.6 to 4 times.