[dpdk-users] Vcenter DVS Vlan Vmxnet3 DPDK Performance Poor

[dpdk-users] Vcenter DVS Vlan Vmxnet3 DPDK Performance Poor

[Shawn Zheng]

Hi,all:
     We are facing a problem with vmxnet3 poor performance issue.
    
     Our envrionment is below:     Esxi 6.5 and 6.7, using vmxnet3 nic, we are use a l2fwd vm to trunk all the packet, and do the packets retransmit     There are 3 vm in my env,  1 tx-vm , 1 rx-vm  and 1 l2fwd-vm
     1) Tx vm using vlan 4 in portgroup1, send packets;     2) Rx vm using vlan 5 in portgroup2, receivce packets;
     3) Our L2fwd vm using vlan trunk in portgroup3, and change the vlan-4 to vlan-5 so that rx-vm can receive the packets.     4) Use dpdk 18.05 , 18.11 and 19.02 
     Here is my l2fwd code changes when forward the packets :             struct ether_hdr *eth;	void *tmp;	eth = rte_pktmbuf_mtod(m, struct ether_hdr *);	uint16_t ether_type = eth->ether_type;	uint16_t hdr_len = sizeof(struct ether_hdr);	if (ether_type == rte_cpu_to_be_16(ETHER_TYPE_VLAN)) {		struct vlan_hdr * vh = rte_pktmbuf_mtod_offset(m, struct vlan_hdr *, hdr_len);		vh->vlan_tci = 0x0500;                       // change the vlan tag to vlan 5	}     
    And we see l2fwd dropped many packets when tx-vm transimit about 5G (1518bytes) packets, about 420000 pps , and only receive about 350000 pps.    [root@vrouter1 home]# ./l2fwd -c 6 -n 4 --socket-mem  1000,0 -- -p 1 -T1
Port statistics ====================================Statistics for port 0 ------------------------------Packets sent:                  8836653Packets received:             10000022Packets dropped:               1163369Aggregate statistics ===============================Total packets sent:            8836653Total packets received:       10000022Total packets dropped:         1163369====================================================       I know the dropped stats means the vmxnet3 tx ring exhausted , but when  tx-vm and rx-vm in same vlan , there is no drop packets. Or with diffrent vlan , the kernel driver is better than dpdk. 

    Could somebody give some suggestion ? Thanks in advance !!


--------------------------------


Regards  
Shawn Zheng