InfiniBand is a high-performance, multipurpose network architecture based on a switch design often referred to as the “switched fabric.” InfiniBand (“IB” for short) was designed for use in I/O networks such as storage area networks (SANs) or cluster networks. It has become a leading standard in high-performance computing. More than 200 of the world’s 500 fastest supercomputers use InfiniBand, more than use Gigabit Ethernet.
InfiniBand History
Work on InfiniBand began in the 1990s under different names by two independent industry groups designing technical standards for system interconnects. After the merger of the two groups in 1999, “InfiniBand” finally emerged as the name of the new architecture. Version 1.0 of the InfiniBand architecture standard was released in 2000.
How InfiniBand works
The InfiniBand architecture specifications cover layers 1 through 4 of the OSI model. It covers data link and physical layer hardware requirements and also has connection-oriented and connectionless transport protocols analogous to TCP and UDP. InfiniBand uses IPv6 for addressing at the network layer.
InfinBand implements an application messaging service called Channel I/O that bypasses network operating systems to achieve high performance in specialized environments. Provides the ability for two Infiniband-enabled applications to create a direct communication channel with send and receive queues called Queue Pairs. The queues are assigned to memory spaces accessible to each application to share data (called Remote Direct Memory Access or RDMA).
An InfiniBand network consists of four main components:
- Gateway
- host channel adapters
- Subnet Managers
- switches
Like other network gateways, an InfiniBand gateway connects an IB network with external local networks.
Host Channel Adapters – Connect InfiniBand devices to the IB fabric, like more traditional network adapters.
Subnet Managersoftware manages the flow of traffic on an InfiniBand network. Each IB device runs a Subnet Manager Agent to communicate with the Central Manager.
InfiniBand switches are a necessary element of the network, to allow a set of devices to pair with each other in different combinations. Unlike Ethernet and Wi-Fi, IB networks do not typically use routers.
How fast is InfiniBand?
InfiniBand supports multi-gigabit network speeds, up to 56 Gbps and higher depending on your configuration. The technology roadmap includes support for 100 Gbps and faster speeds in future releases.
InfiniBand Limitations
InfiniBand applications have been largely limited to cluster supercomputers and other specialized network systems. Marketing claims aside, InfiniBand was not designed for general-purpose application data networks in a way that could replace Ethernet or Fiber Channel in Internet data centers. It does not use traditional network protocol stacks such as TCP/IP due to the performance limitations of these protocols, but doing so is not supported by core applications.
It has not yet become a mainstream technology, in part because standard networking software libraries such as WinSock cannot work with InfiniBand without sacrificing the performance benefits of the architecture.