DPDK usage discussions
 help / color / mirror / Atom feed
* the packet delay is too long when I used dpdk i40e NIC sent single packet
@ 2023-07-05  1:32 jiangheng
  2023-07-06  3:10 ` Xing, Beilei
  0 siblings, 1 reply; 5+ messages in thread
From: jiangheng @ 2023-07-05  1:32 UTC (permalink / raw)
  To: Yuying.Zhang, beilei.xing; +Cc: users

Hi Beilei:
we used dpdk implement a user mode tcp protocol stack, and we are testing the delay of a single packet.
however, on i40e NIC(XL710), the packet delay is higher than that of the kernel. After some tests, we find that the time consuming may be the pahse when i40e sends packet.(period from rte_eth_tx_burst to packet be sent)
The same test is performed on the ixgbe(82599) NIC. the delay is lower than that in kernel.

I used dpdk-21.11 stable version. 
I tried upgrading the i40e's firmware version to 9.10, but there was no improvement.
Any suggestion?
Thank you.



The following is packet delay statistics of tcpdump:
i40e NIC:
410	10.373817	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9642 Win=1048384 Len=0
    10us
411	10.373827	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
    27us
412	10.373854	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
    9us
413	10.373863	192.168.4.5	192.168.4.4	TCP	90	58312→8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
    28us
414	10.373891	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
    11us
415	10.373902	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
    28us
416	10.373930	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
    11us
417	10.373941	192.168.4.5	192.168.4.4	TCP	90	58312→8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
    25us
418	10.373967	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
    11us
419	10.373978	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9714 Ack=6511 Win=1048512 Len=36
    4us
420	10.373982	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9750 Win=1048256 Len=0


ixgbe NIC:
110	13.235208	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7266 Win=1047808 Len=0
    10us
111	13.235218	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
    6us
112	13.235224	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
    10us
113	13.235234	192.168.4.11	192.168.4.12	TCP	94	58312→8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
    6us
114	13.235240	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
    11us
115	13.235251	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
    5us
116	13.235256	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
    11us
117	13.235267	192.168.4.11	192.168.4.12	TCP	94	58312→8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
    6us
118	13.235273	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
    10us
119	13.235283	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7338 Ack=6511 Win=1048512 Len=36
    6us
120	13.235289	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7374 Win=1047680 Len=0

^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: the packet delay is too long when I used dpdk i40e NIC sent single packet
  2023-07-05  1:32 the packet delay is too long when I used dpdk i40e NIC sent single packet jiangheng
@ 2023-07-06  3:10 ` Xing, Beilei
  2023-07-11  7:28   ` jiangheng
  0 siblings, 1 reply; 5+ messages in thread
From: Xing, Beilei @ 2023-07-06  3:10 UTC (permalink / raw)
  To: jiangheng, Zhang, Yuying; +Cc: users

Hi,

Could you try to change interval configuration to check if it helps?
Please refer to i40e_calc_itr_interval function.
In theory, the lower interval, the lower latency.

BR,
Beilei

> -----Original Message-----
> From: jiangheng <jiangheng14@huawei.com>
> Sent: Wednesday, July 5, 2023 9:33 AM
> To: Zhang, Yuying <yuying.zhang@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Cc: users@dpdk.org
> Subject: the packet delay is too long when I used dpdk i40e NIC sent single
> packet
> 
> Hi Beilei:
> we used dpdk implement a user mode tcp protocol stack, and we are testing
> the delay of a single packet.
> however, on i40e NIC(XL710), the packet delay is higher than that of the kernel.
> After some tests, we find that the time consuming may be the pahse when i40e
> sends packet.(period from rte_eth_tx_burst to packet be sent) The same test is
> performed on the ixgbe(82599) NIC. the delay is lower than that in kernel.
> 
> I used dpdk-21.11 stable version.
> I tried upgrading the i40e's firmware version to 9.10, but there was no
> improvement.
> Any suggestion?
> Thank you.
> 
> 
> 
> The following is packet delay statistics of tcpdump:
> i40e NIC:
> 410	10.373817	192.168.4.4	192.168.4.5	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=9642 Win=1048384 Len=0
>     10us
> 411	10.373827	192.168.4.4	192.168.4.5	TCP	94	58312
> →8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
>     27us
> 412	10.373854	192.168.4.5	192.168.4.4	TCP	60	8877
> →58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
>     9us
> 413	10.373863	192.168.4.5	192.168.4.4	TCP	90	58312
> →8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
>     28us
> 414	10.373891	192.168.4.4	192.168.4.5	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
>     11us
> 415	10.373902	192.168.4.4	192.168.4.5	TCP	94	58312
> →8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
>     28us
> 416	10.373930	192.168.4.5	192.168.4.4	TCP	60	8877
> →58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
>     11us
> 417	10.373941	192.168.4.5	192.168.4.4	TCP	90	58312
> →8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
>     25us
> 418	10.373967	192.168.4.4	192.168.4.5	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
>     11us
> 419	10.373978	192.168.4.4	192.168.4.5	TCP	94	58312
> →8877 [PSH, ACK] Seq=9714 Ack=6511 Win=1048512 Len=36
>     4us
> 420	10.373982	192.168.4.5	192.168.4.4	TCP	60	8877
> →58312 [ACK] Seq=6511 Ack=9750 Win=1048256 Len=0
> 
> 
> ixgbe NIC:
> 110	13.235208	192.168.4.12	192.168.4.11	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=7266 Win=1047808 Len=0
>     10us
> 111	13.235218	192.168.4.12	192.168.4.11	TCP	94	58312
> →8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
>     6us
> 112	13.235224	192.168.4.11	192.168.4.12	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
>     10us
> 113	13.235234	192.168.4.11	192.168.4.12	TCP	94	58312
> →8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
>     6us
> 114	13.235240	192.168.4.12	192.168.4.11	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
>     11us
> 115	13.235251	192.168.4.12	192.168.4.11	TCP	94	58312
> →8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
>     5us
> 116	13.235256	192.168.4.11	192.168.4.12	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
>     11us
> 117	13.235267	192.168.4.11	192.168.4.12	TCP	94	58312
> →8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
>     6us
> 118	13.235273	192.168.4.12	192.168.4.11	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
>     10us
> 119	13.235283	192.168.4.12	192.168.4.11	TCP	94	58312
> →8877 [PSH, ACK] Seq=7338 Ack=6511 Win=1048512 Len=36
>     6us
> 120	13.235289	192.168.4.11	192.168.4.12	TCP	64	8877
> →58312 [ACK] Seq=6511 Ack=7374 Win=1047680 Len=0

^ permalink raw reply	[flat|nested] 5+ messages in thread

* the packet delay is too long when I used dpdk i40e NIC sent single packet
  2023-07-06  3:10 ` Xing, Beilei
@ 2023-07-11  7:28   ` jiangheng
  2023-07-12  6:21     ` Xing, Beilei
  0 siblings, 1 reply; 5+ messages in thread
From: jiangheng @ 2023-07-11  7:28 UTC (permalink / raw)
  To: beilei.xing; +Cc: users, jiangheng14, yuying.zhang

Hi,
I changed I40E_QUEUE_ITR_INTERVAL_DEFAULT to 10us, latency is significantly lower.
what is the effect of modifying this parameter?

> Hi,
> 
> Could you try to change interval configuration to check if it helps?
> Please refer to i40e_calc_itr_interval function.
> In theory, the lower interval, the lower latency.
> 
> BR,
> Beilei
> 
>> -----Original Message-----
>> From: jiangheng <jiangheng14@huawei.com>
>> Sent: Wednesday, July 5, 2023 9:33 AM
>> To: Zhang, Yuying <yuying.zhang@intel.com>; Xing, Beilei
>> <beilei.xing@intel.com>
>> Cc: users@dpdk.org
>> Subject: the packet delay is too long when I used dpdk i40e NIC sent single
>> packet
>> 
>> Hi Beilei:
>> we used dpdk implement a user mode tcp protocol stack, and we are testing
>> the delay of a single packet.
>> however, on i40e NIC(XL710), the packet delay is higher than that of the kernel.
>> After some tests, we find that the time consuming may be the pahse when i40e
>> sends packet.(period from rte_eth_tx_burst to packet be sent) The same test is
>> performed on the ixgbe(82599) NIC. the delay is lower than that in kernel.
>> 
>> I used dpdk-21.11 stable version.
>> I tried upgrading the i40e's firmware version to 9.10, but there was no
>> improvement.
>> Any suggestion?
>> Thank you.
>> 
>> 
>> 
>> The following is packet delay statistics of tcpdump:
>> i40e NIC:
>> 410	10.373817	192.168.4.4	192.168.4.5	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=9642 Win=1048384 Len=0
>>     10us
>> 411	10.373827	192.168.4.4	192.168.4.5	TCP	94	58312
>> →8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
>>     27us
>> 412	10.373854	192.168.4.5	192.168.4.4	TCP	60	8877
>> →58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
>>     9us
>> 413	10.373863	192.168.4.5	192.168.4.4	TCP	90	58312
>> →8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
>>     28us
>> 414	10.373891	192.168.4.4	192.168.4.5	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
>>     11us
>> 415	10.373902	192.168.4.4	192.168.4.5	TCP	94	58312
>> →8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
>>     28us
>> 416	10.373930	192.168.4.5	192.168.4.4	TCP	60	8877
>> →58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
>>     11us
>> 417	10.373941	192.168.4.5	192.168.4.4	TCP	90	58312
>> →8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
>>     25us
>> 418	10.373967	192.168.4.4	192.168.4.5	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
>>     11us
>> 419	10.373978	192.168.4.4	192.168.4.5	TCP	94	58312
>> →8877 [PSH, ACK] Seq=9714 Ack=6511 Win=1048512 Len=36
>>     4us
>> 420	10.373982	192.168.4.5	192.168.4.4	TCP	60	8877
>> →58312 [ACK] Seq=6511 Ack=9750 Win=1048256 Len=0
>> 
>> 
>> ixgbe NIC:
>> 110	13.235208	192.168.4.12	192.168.4.11	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=7266 Win=1047808 Len=0
>>     10us
>> 111	13.235218	192.168.4.12	192.168.4.11	TCP	94	58312
>> →8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
>>     6us
>> 112	13.235224	192.168.4.11	192.168.4.12	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
>>     10us
>> 113	13.235234	192.168.4.11	192.168.4.12	TCP	94	58312
>> →8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
>>     6us
>> 114	13.235240	192.168.4.12	192.168.4.11	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
>>     11us
>> 115	13.235251	192.168.4.12	192.168.4.11	TCP	94	58312
>> →8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
>>     5us
>> 116	13.235256	192.168.4.11	192.168.4.12	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
>>     11us
>> 117	13.235267	192.168.4.11	192.168.4.12	TCP	94	58312
>> →8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
>>     6us
>> 118	13.235273	192.168.4.12	192.168.4.11	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
>>     10us
>> 119	13.235283	192.168.4.12	192.168.4.11	TCP	94	58312
>> →8877 [PSH, ACK] Seq=7338 Ack=6511 Win=1048512 Len=36
>>     6us
>> 120	13.235289	192.168.4.11	192.168.4.12	TCP	64	8877
>> →58312 [ACK] Seq=6511 Ack=7374 Win=1047680 Len=0

^ permalink raw reply	[flat|nested] 5+ messages in thread

* RE: the packet delay is too long when I used dpdk i40e NIC sent single packet
  2023-07-11  7:28   ` jiangheng
@ 2023-07-12  6:21     ` Xing, Beilei
  0 siblings, 0 replies; 5+ messages in thread
From: Xing, Beilei @ 2023-07-12  6:21 UTC (permalink / raw)
  To: jiangheng; +Cc: users, Zhang, Yuying

Please refer to the INTERNAL description in 710 datasheet.

Interval
ITR 'n' interval, where 'n' is the register index = 0,1,2 for the three ITRs per interrupt.
It is defined in 2 us units enabling interval range from zero to 8160 us (0xFF0).
Setting the INTERVAL to zero enables immediate interrupt.
This register can also be programmed also by setting the INTERVAL field in the
matched xxINT_DYN_CTLx register.

BR,
Beilei

> -----Original Message-----
> From: jiangheng <jiangheng14@huawei.com>
> Sent: Tuesday, July 11, 2023 3:28 PM
> To: Xing, Beilei <beilei.xing@intel.com>
> Cc: users@dpdk.org; jiangheng14@huawei.com; Zhang, Yuying
> <yuying.zhang@intel.com>
> Subject: the packet delay is too long when I used dpdk i40e NIC sent single
> packet
> 
> Hi,
> I changed I40E_QUEUE_ITR_INTERVAL_DEFAULT to 10us, latency is
> significantly lower.
> what is the effect of modifying this parameter?
> 
> > Hi,
> >
> > Could you try to change interval configuration to check if it helps?
> > Please refer to i40e_calc_itr_interval function.
> > In theory, the lower interval, the lower latency.
> >
> > BR,
> > Beilei
> >
> >> -----Original Message-----
> >> From: jiangheng <jiangheng14@huawei.com>
> >> Sent: Wednesday, July 5, 2023 9:33 AM
> >> To: Zhang, Yuying <yuying.zhang@intel.com>; Xing, Beilei
> >> <beilei.xing@intel.com>
> >> Cc: users@dpdk.org
> >> Subject: the packet delay is too long when I used dpdk i40e NIC sent
> >> single packet
> >>
> >> Hi Beilei:
> >> we used dpdk implement a user mode tcp protocol stack, and we are
> >> testing the delay of a single packet.
> >> however, on i40e NIC(XL710), the packet delay is higher than that of the
> kernel.
> >> After some tests, we find that the time consuming may be the pahse
> >> when i40e sends packet.(period from rte_eth_tx_burst to packet be
> >> sent) The same test is performed on the ixgbe(82599) NIC. the delay is
> lower than that in kernel.
> >>
> >> I used dpdk-21.11 stable version.
> >> I tried upgrading the i40e's firmware version to 9.10, but there was
> >> no improvement.
> >> Any suggestion?
> >> Thank you.
> >>
> >>
> >>
> >> The following is packet delay statistics of tcpdump:
> >> i40e NIC:
> >> 410	10.373817	192.168.4.4	192.168.4.5	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=9642 Win=1048384 Len=0
> >>     10us
> >> 411	10.373827	192.168.4.4	192.168.4.5	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
> >>     27us
> >> 412	10.373854	192.168.4.5	192.168.4.4	TCP	60	8877
> >> →58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
> >>     9us
> >> 413	10.373863	192.168.4.5	192.168.4.4	TCP	90
> 	58312
> >> →8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
> >>     28us
> >> 414	10.373891	192.168.4.4	192.168.4.5	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
> >>     11us
> >> 415	10.373902	192.168.4.4	192.168.4.5	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
> >>     28us
> >> 416	10.373930	192.168.4.5	192.168.4.4	TCP	60	8877
> >> →58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
> >>     11us
> >> 417	10.373941	192.168.4.5	192.168.4.4	TCP	90
> 	58312
> >> →8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
> >>     25us
> >> 418	10.373967	192.168.4.4	192.168.4.5	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
> >>     11us
> >> 419	10.373978	192.168.4.4	192.168.4.5	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=9714 Ack=6511 Win=1048512 Len=36
> >>     4us
> >> 420	10.373982	192.168.4.5	192.168.4.4	TCP	60	8877
> >> →58312 [ACK] Seq=6511 Ack=9750 Win=1048256 Len=0
> >>
> >>
> >> ixgbe NIC:
> >> 110	13.235208	192.168.4.12	192.168.4.11	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=7266 Win=1047808 Len=0
> >>     10us
> >> 111	13.235218	192.168.4.12	192.168.4.11	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
> >>     6us
> >> 112	13.235224	192.168.4.11	192.168.4.12	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
> >>     10us
> >> 113	13.235234	192.168.4.11	192.168.4.12	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
> >>     6us
> >> 114	13.235240	192.168.4.12	192.168.4.11	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
> >>     11us
> >> 115	13.235251	192.168.4.12	192.168.4.11	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
> >>     5us
> >> 116	13.235256	192.168.4.11	192.168.4.12	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
> >>     11us
> >> 117	13.235267	192.168.4.11	192.168.4.12	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
> >>     6us
> >> 118	13.235273	192.168.4.12	192.168.4.11	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
> >>     10us
> >> 119	13.235283	192.168.4.12	192.168.4.11	TCP	94
> 	58312
> >> →8877 [PSH, ACK] Seq=7338 Ack=6511 Win=1048512 Len=36
> >>     6us
> >> 120	13.235289	192.168.4.11	192.168.4.12	TCP	64	8877
> >> →58312 [ACK] Seq=6511 Ack=7374 Win=1047680 Len=0

^ permalink raw reply	[flat|nested] 5+ messages in thread

* the packet delay is too long when I used dpdk i40e NIC sent single packet
@ 2023-07-03  4:00 jiangheng
  0 siblings, 0 replies; 5+ messages in thread
From: jiangheng @ 2023-07-03  4:00 UTC (permalink / raw)
  To: beilei.xing; +Cc: users

Hi Beilei:
we used dpdk implement a user mode tcp protocol stack, and we are testing the delay of a single packet.
however, on i40e NIC(XL710), the packet delay is higher than that of the kernel. After some tests, we find that the time consuming may be the pahse when i40e sends packet.(period from rte_eth_tx_burst to packet be sent)
The same test is performed on the ixgbe(82599) NIC. the delay is lower than that in kernel.

I used dpdk-21.11 stable version. 
I tried upgrading the i40e's firmware version to 9.10, but there was no improvement.
Any suggestion?
Thank you.



The following is packet delay statistics of tcpdump:
i40e NIC:
410	10.373817	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9642 Win=1048384 Len=0
    10us
411	10.373827	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
    27us
412	10.373854	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
    9us
413	10.373863	192.168.4.5	192.168.4.4	TCP	90	58312→8877 [PSH, ACK] Seq=9642 Ack=6511 Win=1048512 Len=36
    28us
414	10.373891	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9678 Win=1048320 Len=0
    11us
415	10.373902	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
    28us
416	10.373930	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
    11us
417	10.373941	192.168.4.5	192.168.4.4	TCP	90	58312→8877 [PSH, ACK] Seq=9678 Ack=6511 Win=1048512 Len=36
    25us
418	10.373967	192.168.4.4	192.168.4.5	TCP	64	8877→58312 [ACK] Seq=6511 Ack=9714 Win=1048320 Len=0
    11us
419	10.373978	192.168.4.4	192.168.4.5	TCP	94	58312→8877 [PSH, ACK] Seq=9714 Ack=6511 Win=1048512 Len=36
    4us
420	10.373982	192.168.4.5	192.168.4.4	TCP	60	8877→58312 [ACK] Seq=6511 Ack=9750 Win=1048256 Len=0


ixgbe NIC:
110	13.235208	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7266 Win=1047808 Len=0
    10us
111	13.235218	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
    6us
112	13.235224	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
    10us
113	13.235234	192.168.4.11	192.168.4.12	TCP	94	58312→8877 [PSH, ACK] Seq=7266 Ack=6511 Win=1048512 Len=36
    6us
114	13.235240	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7302 Win=1047744 Len=0
    11us
115	13.235251	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
    5us
116	13.235256	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
    11us
117	13.235267	192.168.4.11	192.168.4.12	TCP	94	58312→8877 [PSH, ACK] Seq=7302 Ack=6511 Win=1048512 Len=36
    6us
118	13.235273	192.168.4.12	192.168.4.11	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7338 Win=1047744 Len=0
    10us
119	13.235283	192.168.4.12	192.168.4.11	TCP	94	58312→8877 [PSH, ACK] Seq=7338 Ack=6511 Win=1048512 Len=36
    6us
120	13.235289	192.168.4.11	192.168.4.12	TCP	64	8877→58312 [ACK] Seq=6511 Ack=7374 Win=1047680 Len=0


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2023-07-12  6:21 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-05  1:32 the packet delay is too long when I used dpdk i40e NIC sent single packet jiangheng
2023-07-06  3:10 ` Xing, Beilei
2023-07-11  7:28   ` jiangheng
2023-07-12  6:21     ` Xing, Beilei
  -- strict thread matches above, loose matches on Subject: below --
2023-07-03  4:00 jiangheng

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).