Peng Sun will present his research seminar/general exam on Wednesday May 16 at 2PM in Room 402. The members of his committee are: Jennifer Rexford (advisor), Michael Freedman, and David Walker. Everyone is invited to attend his talk, and those faculty wishing to remain for the oral exam following are welcome to do so. His abstract and reading list follow below. ------------------------------------------------- Abstract: Datacenters run many traffic management tasks to manage the flow of applications' traffic. They are very important in achieving better application performance, and lower server/network workload. However, many traffic management tasks in the datacenters are divided based on physical boundaries between the servers and the network infrastructure. Network-based traffic management doesn’t have access to fine-grained per-connection information due to scalability concerns, and it can’t obtain the information in the application and transport layer. At the same time, the end hosts are able to collect per-connection and higher-layer statistics in a more efficient way, but they don't know the routing configurations and link statuses in the network. The division leads to uncoordinated responses to the same events, poor visibility into the root cause of performance problems, and missed opportunities to jointly optimize application performance and resource usage. So we propose to bring the end hosts into traffic management in datacenters. The end hosts can help datacenters achieve a more application-oriented traffic management solution with more fine-grained data in a more efficient and scalable way. In addition, traffic management tasks vary in what measurement data they need, how they use the data for computation, and what control actions they generate. Designing each one from scratch is hard. A generic platform is needed to offer reusable and expressive interface to make traffic management easier. We find that all datacenter traffic management tasks share a common pattern of a three-stage pipeline: measurement, analysis, and control. We propose a programming framework that abstracts each stage and offers a sufficiently expressive interface. The framework is based on streaming MapReduce-like functional programming, and it offers great freedom in defining diverse traffic management functionalities, while enabling partitioning the programs into distributed and scalable execution. Extending to include end hosts and making traffic management programmable, we present our joint HOst-NEtwork traffic management system for datacenters - HONE. HONE offers these things: 1) a host agent that makes fine-grained data from end hosts available for traffic management, 2) an expressive programming framework for defining management functionality, and 3) automatic partitioning of the program across the controller and the end hosts for efficient and scalable execution. We evaluate the performance of our prototype, and demonstrate HONE's power with typical traffic management tasks in datacenters: distributed rate limiting, network diagnosis, and flow scheduling. Reading list: L. Peterson, and B. Davie, "Computer Networks: A Systems Approach" , Morgan Kaufmann, 5e, 2011 D. Clark, "The Design Philosophy of the DARPA Internet Protocols" , ACM SIGCOMM, August 1988 J.Saltzer, D.Reed and D.Clark, "End-to-End Arguments in System Design" , ACM Transactions on Computer Systems, Nov 1984 S.Kandula, S. Sengupta, A. Greenberg, P. Patel, R. Chaiken, "The Nature of Data Center Traffic: Measurement & Analysis" , ACM SIGCOMM conference on Internet Measurement Conference (IMC), November 2009 J. Dean, S. Ghemawat. "MapReduce: Simplified Data Processing on Large Clusters" , USENIX OSDI, 2004 Y. Ylleyu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P. Gunda, J. Currey, "DryadLINQ: A System for General -Purpose Distributed Data-Parallel Computing Using a High-Level Language" , USENIX OSDI, 2008 N. Foster, R. Harrison, M. Freedman, C. Monsanto, J. Rexford, A. Story, D. Walker, "Frenetic: A Network Programming Language" , ACM SIGPLAN International Conference on Functional Programming (ICFP), September 2011 R. Newton, L. Girod, M. Craig, S. Madden, G. Morrisett, "WaveScript: A Case-Study in Applying a Distributed Stream-Processing Language" , MIT CSAIL Technical Report MIT-CSAIL-TR-2008-005, 2008 M. Yu, A. Greenberg, D. Maltz, J. Rexford, L. Yuan, S. Kandula, C. Kim, "Profiling network performance for multi-tier data center applications" , USENIX NSDI, March 2011 S. Kandula, S Agarwal, R. Mahajan, P. Verkaik, J. Padhye, P. Bahl, "Detailed Diagnosis in Enterprise Networks" , ACM SIGCOMM, August 2009 B. Raghavan, K. Vishwanath, S. Ramabhadran, K. Yocum, A. Snoeren, "Cloud Control with Distributed Rate Limiting" , ACM SIGCOMM, August 2007 M. Al-Fares,S. Radhakrishnan, B. Raghavan, N. Huang, A. Vahdat, "Hedera: Dynamic Flow Scheduling for Data Center Networks" , USENIX NSDI, April 2010