iSER – Changing Landscape of Ethernet Storage
Piyush Gupta dwells upon fundamentals of iSER and how is it different from iSCSI and why it is so relevant today.
IBM India System Development Lab
To achieve ultra-low latency, high bandwidth and much higher IO operations per second.
Industrial Communication Technology
Ethernet is known for its low cost commodity components and administrative ease. It is simple to deploy, has a huge list of upper layer protocols and applications that it supports.
Ethernet is well known and accepted since a long time by a vast spectrum of data centre administrators that are well trained on managing Ethernet host connectivity setups. These attributes of Ethernet make it a perfect choice for a data centre, but what about raw storage performance?
Low-latency data storage traffic over Ethernet is not a new concept. Industry work on developing standards has been ongoing for some years. RDMA over Converged Ethernet (RoCE) is a network protocol that enables remote direct memory access (RDMA) over an Ethernet network. RoCE is already on Version 2. Internet WideArea RDMA protocol (iWARP), is a TCP-based standard. It was originally described back in 2007 and then updated as recently as 2014. Neither RoCEv2 nor iWARP have seen wide adoption in the past, but perhaps the advent of iSER and NVMeF protocols is changing that scenario.
iSER (iSCSI Extensions for RDMA) is a protocol that leverages the same administrative model as that of iSCSI but replaces the data path with the fast RDMA protocol to achieve ultra-low latency, high bandwidth and much higher IO operations per second, making it a perfect choice for performance sensitive applications like All Flash block storage.
This is how Wikipedia defines iSER: “iSCSI Extensions for RDMA (iSER) is a network protocol that extends the Internet Small Computer System Interface (iSCSI) protocol to use Remote Direct Memory Access (RDMA). RDMA is provided by either the Transmission Control Protocol (TCP) with RDMA services (iWARP), RoCE (RDMA over Converged Ethernet) that does not need the TCP layer and therefore provides lower latency, or InfiniBand”.
iSER differs from iSCSI with regard to how SCSI read/write operations are performed over RDMA vs traditional TCP/IP, making it very efficient. By doing that, iSER delivers ultra low latency and low CPU utilisation compared to iSCSI while maintaining the iSCSI administrative model.
iSER is protocol that transfers data directly between initiator and target memory. The protocol eliminates the need for memory copies (at each step), doesn’t use the native TCP/IP stack, and bypasses the CPU involvement in data transfers between initiator and target. iSER is a low overhead protocol that significantly betters CPU efficiency by offloading data transfer tasks to the adapter. iSER works on top of standard Ethernet switches and RDMA NICs (that are not very expensive). This means it’s low cost, yet its performance ratio is far beyond contemporary Ethernet protocols like iSCSI and its throughput approaches wire speed, 25/40/50/100/200 Gbps.
These characteristics make iSER an ideal protocol for All Flash block storage connectivity over Ethernet backbones available in almost all data centres. In addition to performance it is a stable and mature protocol that incorporates security and high availability.
Which applications run on top ofiSER?
Any application that runs over traditional iSCSI can use iSER for better performance without any modifications. The administrator would need to purchase RDMA capable Network Interface Cards (NICs) for the servers to run iSER, continue to use the existing 10/40/100 Gb or 25/50/100 Gb Ethernet switching backbone and Storage Controllers that support iSER host attach.
iSER requires investments in RDMA capable NICs and high speed Ethernet switching equipment if that is not already part of the data centre.
On the Storage Controller side, the high speed Ethernet ports that used to serve iSCSI earlier will now support iSER. In future, these same ports will support NVMeF over RDMA with a simple software upgrade to target controllers. Hence, iSER provides All Flash connectivity today and at the same time provides a simple transition to NVMeF over Ethernet when the ecosystem is ready for its adoption.
IBM India System Development Lab
He is Senior Manager Storage Development, IBM India System Development Lab, responsible for development of IBM Spectrum Virtualize line of products with special focus on Ethernet storage protocol development like iSCSI , iSER , NVMeF, iWARP & RoCE, and IP based Replication. Piyush has worked with various organisations in storage and virtualisation industry in past like NetApp and VMWare. He is an engineering graduate in electronics and management certified from IIM Indore. Email: firstname.lastname@example.org