Ultra-Low Latency with User-Mode Kernel-Bypass using WireDirect and TOE Solution Overview

2 min Chelsio T6 Latency Video

2 min Chelsio iWARP RDMA: Competitive Ethernet Alternative to InfiniBand

Terminator Features RDMA – iWARP TCP Offload Engine (TOE) iSCSI Offload FCoE Offload SMB Direct Wire Direct Network Direct Virtualization Packet Filtering Traffic Management Stream Engine DCB GPUDirect RDMA Crypto Offload

Chelsio Acceleration Kernel-Bypass – WireDirect and TOE

WireDirect (WD) and TOE (TCP Offload Engine) are technologies that achieve unprecedented levels of efficiency, thanks to direct system or application memory-to-memory communication, without CPU involvement or data copies. With WireDirect and TOE enabled adapters, all packet and protocol processing required for communication is handled in hardware by the network adapter, for high performance and the queues will be polled for the lowest latency.

WD-UDP & TOE – Require No Application Changes

WD-UDP uses hardware RDMA semantics to expose queues to a user-level UDP stack and TOE uses a hardware TCP/IP stack that runs in the adapter, completely bypassing the host software stack, thus eliminating any inefficiencies due to software processing. WD-UDP and TOE provide CPU bypass and zero-copy without changing the application, while operating over standard Ethernet networks.

WD-QP & WD-TOE – May Need Some Application Changes

WD-QP uses hardware RDMA semantics to expose Raw-Ethernet queues and WD-TOE uses a hardware TCP/IP stack that runs in the adapter with the TOE queues exposed to user-space, completely bypassing the host software stack, thus eliminating any inefficiencies due to software processing. WD-QP and WD-TOE provide CPU bypass and zero-copy, but changes may be needed to access the Raw-Ethernet queues and TOE queues, while operating over standard Ethernet network.

Figure 1 – Chelsio WireDirect and TOE Stack

WireDirect and TOE Benchmarks

Figure 2 – Chelsio WireDirect UDP and TOE TCP Benchmarks

Benchmark results illustrate the benefits of WireDirect and TOE using Chelsio’s 25G T6225-LL-CR and 100G T62100-LP-CR Unified Wire Ethernet adapters. Using WireDirect and TOE results in a drastic reduction in latency and increase in message rate for the test application. Latency reduction at such magnitude explains the benefits of WireDirect and TOE and proves it to be a low latency and high performance solution for FSI HFT markets and other markets that require ultra-low latency.