The non-volatile memory express (NVMe) storage controller interface continues extending into the data center, on the back of growing requirements for new levels of application performance acceleration. NVMe is typically deployed as direct-attach storage via a Peripheral Component Interconnect Express (PCIe) interface, which bypasses the latency overhead of storage networking but at the same time does not scale. NVMe-over Fabrics (NVMe-oF) is emerging to enable greater scalability with levels of latency that are closer to those of direct-attached storage. As a result, the NVMe Organization industry consortium is extending support for storage transport protocols. Fibre Channel (FC) and Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) are supported as fabric transports, and the Transmission Control Protocol (TCP) was recently added as another transport option in late 2018.
A precursor to broader adoption of NVMe technology is testing to ensure that new products comply with industry requirements – and also that they are interoperable, to facilitate deployment in production. The achieve this, the University of New Hampshire InterOperability Laboratory (UNH-IOL) works with the NVMe organization to implement an interoperability and compliance program for NVMe protocols.
UNH-IOL
The UNH-IOL provides an industry-funded, independent environment in which networking, storage and other data communications vendors can test the performance, multi-vendor interoperability and standards conformance of their products. The lab is a 32,000-square foot facility located at the University of New Hampshire (UNH) that employs more than 120 graduate and undergraduate students, as well as more than 20 full-time staff members, including veteran engineers.
The lab got its start in 1988 working with Fiber Distributed Data Interface (FDDI) and 10BASE-T Ethernet equipment, and has since worked with a plethora of technologies including token ring local area networks (LANs), IPv6, many Ethernet and DSL varieties, and various storage networking technologies. It is unique in the range of technologies that it tests as well as the granularity to which it can test technologies.
The UNH-IOL hosts approximately 25-30 Plugfests, which are group test events for specific technical standards, per year. The events are typically a week long, and they draw engineers from around the world that are seeking to certify their product for integration on the applicable Systems Integrator list.
In collaboration with industry consortiums such as the NVMe Organization, UNH-IOL engineers design and publish test tools and plans, and they develop applicable test beds. During Plugfest events, the UNH-IOL team administers conformance tests. Because interoperability tests are more technical, engineers from vendors are typically more hands on with these tests, with the UNH staff overseeing the test and collecting results.
UNH-IOL NVMe over TCP Test Services
Plugfests can help to make significant strides in areas such as debugging, and they serve as kickoff points for new technical standards. The UNH-IOL specifically contributes to the development and interpretation of technical specifications across a range of industries, both mature and cutting edge. When it comes to NVMe, UNH-IOL has hosted several Plugfests and provides its testing and validation services for NVMe over FC and RoCE protocols, and it has scheduled its first Plugfest for the TCP protocol for June 2019.
The emergence of NVMe over TCP (NVMe/TCP) is significant. TCP is one of the most commonly-used networking protocols in the world today. It is very scalable, and it can be deployed without the need for specialized Ethernet networking, as is the case with RoCE. In sum, it is a strong match for modern, cloud-oriented workloads, and it lowers the cost and complexity barriers to NVMe-oF while still facilitating very low latency (although not the same degrees of low latency as NVMe/FC and NVMe/RoCE).