NVMe over Fabrics with SPDK with iRDMA总结 - 2

4.0 Configure and Test RDMA (All Servers) 配置并测试 RDMA（所有服务器）

4.1          Install Intel LAN Driver, RDMA Driver, and Related Dependencies安装英特尔 LAN 驱动程序、RDMA 驱动程序和相关依赖软件

Download the latest Linux driver package from Intel for the Device Under Test (DUT) and follow the installation procedure outlined in the included RDMA irdma README file to install the LAN driver, dependencies, and RDMA driver. 从英特尔为被测设备 (DUT) 下载最新的 Linux 驱动程序软件包，并按照随附的 RDMA irdma README 文件中概述的安装步骤安装 LAN 驱动程序、依赖软件和 RDMA 驱动程序。

Notes:提示

• Intel? Ethernet 800 Series supports both RoCEv2 and iWARP RDMA technologies. Refer to the irdma README file for instructions on how to select which technology on driver load. 英特尔?以太网 800 系列支持 RoCEv2 和 iWARP RDMA 技术。有关如何在加载驱动程序时选择技术的说明，请参阅 irdma README 文件。
• Intel? Ethernet Connection X722/Intel? Ethernet Network Adapter X722 supports only iWARP RDMA. 英特尔?以太网连接 X722/英特尔?以太网网络适配器 X722 仅支持 iWARP RDMA。

4.2 Upgrade NVM to Latest Image on NIC升级网卡上的NVM

Download the latest NVM upgrade package from Intel for the DUT and follow the included documentation to perform the upgrade. 从英特尔为 DUT 下载最新的 NVM 升级包，并按照随附的文档执行升级。

4.3 Disable SELinux and Firewall禁用 SELinux 和防火墙

When running performance testing, disabling the firewall and Security-Enhanced Linux (SELinux) is recommended for highest performance. 运行性能测试时，建议禁用防火墙和安全增强型 Linux (SELinux)，以获得最高性能。

1. Disable the firewall:禁用防火墙

systemctl stop firewalld systemctl mask firewalld

1. Disable SELinux by editing the following file and changing enforcing to disabled (requires reboot): 编辑以下文件并将 enforcing 更改为 disabled（需要重启），从而禁用 SELinux：

vi /etc/selinux/conf

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

# enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing.

# disabled - No SELinux policy is loaded.

SELINUX=disabled

# SELINUXTYPE= can take one of three two values:

# targeted - Targeted processes are protected,

# minimum - Modification of targeted policy. Only selected processes are protected.

# mls - Multi Level Security protection. SELINUXTYPE=targeted

4.4 Enable Flow Control启用流量控制

It is recommended to enable Link-Level Flow Control (LFC) or Priority Flow Control (PFC) for RDMA performance testing for best performance. LFC is provided here as a reference for simplicity. 建议在 RDMA 性能测试中启用链路级流量控制 (LFC) 或优先级流量控制 (PFC)，以获得最佳性能。为简单起见，此处提供 LFC 作为参考。

1. Enable flow control on the adapter using the ethtool -A command: 使用 ethtool -A 命令在适配器上启用流量控制：

ethtool -A <ethx> rx on tx on

1. Confirm the setting with the ethtool -a command: 使用 ethtool -a 命令确认设置：

ethtool -a <ethx>

You should see the following output: 输出结果如下

Pause parameters for <ethx>:

Autonegotiate: off

RX: on

TX: on

Note:      When connected to a switch, the switch port must also be configured to enable LFC on both Rx and Tx per port. To enable link-level flow control on the switch, refer to your switch vendor's documentation. Depending on the switch vendor, the technology may be called pause, LLFC, or flow control. 连接到交换机时，交换机端口也必须配置为在每个端口的 Rx 和 Tx 上启用 LFC。要在交换机上启用链路级流量控制，请参阅交换机供应商的文档。根据交换机供应商的不同，该技术可能被称为暂停、LLFC 或流量控制。

4.5  Check RDMA检查 RDMA

1. Ensure that the RDMA interfaces listed on each server are shown when running the following command: ibv_devices确保在运行以下命令时显示每个服务器上的 RDMA 接口： ibv_devices
2. Use rping to check for RDMA connectivity between target interface and host interface: 使用 rping 检查目标接口和主机接口之间的 RDMA 连接：
   1. Assign IPs to the RDMA interfaces on Target and Host. 为目标和主机上的 RDMA 接口分配 IP。
   2. On Target, run the following: 在 "目标主机 "上运行以下程序：

rping -sdVa <targetIP>

1. 1. On Host, run the following: 在客户端主机上运行以下程序：

rping -cdVa