Dell OS10 (used on PowerSwitch S-series) supports RDMA over Converged Ethernet (RoCE). The SmartFabric manual has some good examples on how to configure it but here's the summary.
A few remarks first: you can choose which queue to place traffic in. The manual states Q3 which is the required queue if you also use iSCSI (which is by default placed in Q4), but if you don't use iSCSI, you can opt to place the traffic in Q4 instead of Q3.
Also, when entering the commands, be sure to check whether you're in the proper context in CONFIG mode. I did not add the necessary "exit" commands so don't blindly copy/paste this.
First, enable DCB(X), which is required for RoCE (PFC and ETS):
dcbx enable
Create the inbound QoS class and policy:
class-map type network-qos pfcdot1p4 match qos-group 4 policy-map type network-qos policy_pfcdot1p4 class pfcdot1p4 pause pfc-cos 4
Create the necessary queues for outbound traffic:
class-map type queuing Q0 match queue 0 class-map type queuing Q4 match queue 4
If you are using RoCEv2, you can also enable ECN:
wred wred_ecn random-detect ecn random-detect color green minimum-threshold 1000 maximum-threshold 2000 drop-probability 100 random-detect color yellow minimum-threshold 500 maximum-threshold 1000 drop-probability 100 random-detect color red minimum-threshold 100 maximum-threshold 500 drop-probability 100 exit
Create the policy for outbound traffic:
policy-map type queuing policy_2Q class Q0 bandwidth percent 40 exit class Q4 bandwidth percent 60 random-detect wred_ecn
In the code above, leave out the last line if you don't want to use ECN.
Finally, create the QoS map:
qos-map traffic-class 2Q queue 0 qos-group 0-3,5-7 queue 4 qos-group 4
Note that certain items (such as enabling ETS) can also be done globally.
For each interface you wish to enable RoCE on, add these lines:
interface ethernet 1/1/x priority-flow-control mode on service-policy input type network-qos policy_pfcdot1p4 service-policy output type queuing policy_2Q ets mode on qos-map traffic-class 2Q trust-map dot1p default
Note that if you want to use RDMA across switches, you need to configure the policies on the uplink ports as well. For LAGs, this means the member ports and not the LAG (port-channel) itself.
« ‹ | December 2024 | › » | ||||
Sun | Mon | Tue | Wed | Thu | Fri | Sat |
1 | 2 | 3 | 4 | 5 | 6 | 7 |
8 | 9 | 10 | 11 | 12 | 13 | 14 |
15 | 16 | 17 | 18 | 19 | 20 | 21 |
22 | 23 | 24 | 25 | 26 | 27 | 28 |
29 | 30 | 31 |