a relatively small budget. To their benefit, the common benchmark for measuring the performance
of a parallel computer is LINPACK, which is not communication intensive, so it was commonplace
to use inexpensive Ethernet networks to string together commodity nodes. As a result, Ethernet got
a foothold on the list of the TOP500 [ 62 ] civilian supercomputers with almost 50% of the TOP500
systems using Ethernet.
PUTTING IT ALL TOGETHER
The first Cray-1 [ 54 ] supercomputer had expected to ship one system per quarter in 1977. Today,
microprocessor companies have refined their CMOS processes and manufacturing making them
very cost-effective building blocks for large-scale parallel systems capable of 10s of petaflops. This
shift away from “proprietary” processors and trend toward “commodity” processors has fueled the
growth of systems. At the time of this writing, the largest computer on the TOP500 list [ 62 ] has in
excess of 220,000 cores (see Figure 7.5 ) and consumes almost seven megawatts!
A datacenter server has many commonalities as one used in a supercomputer, however, there
are also some very glaring differences. We enumerate several properties of both a warehouse-scale
computer (WSC) and a supercomputer (Cray XE6).
Sockets per server 2 sockets x86 platform
Memory capacity 16 GB DRAM
Disk capacity 5
1TB disk drive, and 1
160GB SSD (FLASH)
Compute density 80 sockets per rack
Network bandwidth per rack 1
48-port GigE switch with 40 down links, and 8 uplinks (5
Network bandwidth per socket 100 Mb/s if 1 GigE rack switch, or 1 Gb/s if 10 GigE rack
Sockets per server 8 sockets x86 platform
Memory capacity 32 or 64 GB DRAM
Disk capacity IO capacity varies. Each XIO blade has four PCIe-Gen2 interfaces, for a total
of 96 PCIe-Gen2
16 IO devices for a peak IO bandwidth of 768 GB/s per direction.
Compute density 192 sockets per rack