needs. For example, collective operations, such as global sum, are commonplace in supercomputing
applications and rare in Internet services. This is largely because Internet applications evolved from
simple hardware primitives (e.g., low-cost ethernet NIC) and common communication models (e.g.,
TCP sockets) that were incapable of such operations.
As processor and memory performance continues to increase, the interconnection network
is becoming increasingly important and largely determines the bandwidth and latency of remote
memory access. Going forward, the emergence of super datacenters will convolve into exa -scale