Various extensions of TCP/IP have been proposed to reduce network latency; examples include Explicit Congestion Notification (ECN), Data Center TCP (DCTCP) and several proposals for Active Queue Management (AQM). Combining these techniques requires adjusting various parameters, and recent studies have found that it is difficult to do so while obtaining both high throughput performance and low latency. This is especially true for mixed use data centres that host both latency-sensitive applications and high-throughput workloads with east–west traffic such as Hadoop. This paper studies the difficulty in configuration, and characterises the problem as related to ACK packets. Such packets cannot be set as ECN Capable Transport (ECT), with the consequence that a disproportionate number of them are dropped. The same issue can affect other non-ECT-capable traffic that may co-exist on the network. We explain how this behavior adversely afects throughput, and propose a small change to the way that non-ECT-capable packets are handled in the network switches. Using NS–2 simulation, we demonstrate robust performance for modified AQMs on a Hadoop cluster, maintaining full throughput while reducing latency by 85%. We also demonstrate that commodity switches with shallow buffers are able to reach the same throughput as deeper buffer switches. Finally, we explain how both TCP using ECN and DCTCP can achieve the best performance using a simple marking threshold, in constrast to the current preference for relying on AQMs to mark packets. Overall, we provide recommendations to network equipment manufacturers, cluster administrators and the whole industry on how best to combine high-throughput and latency-sensitive workloads. This article is an extension of our previous work [1], which was published in Proceedings of the 19th IEEE International Conference on Cluster Computing (CLUSTER 2017).

The research leading to these results has received funding from the European Union’s Seventh Framework Programme (FP7/2007–2013) under grant agreement number 610456 (Euroserver). The research was also supported by the Ministry of Economy and Competitiveness of Spain under the contracts TIN2012-34557 and TIN2015-65316-P, Generalitat de Catalunya (contracts 2014-SGR-1051 and 2014-SGR-1272), HiPEAC-3 Network of Excellence (ICT- 287759), and the Severo Ochoa Program (SEV-2011-00067) of the Spanish Government.