DPDK usage discussions
 help / color / Atom feed
* [dpdk-users] i40e vPMD fix out of order Rx read issue
@ 2019-07-25  9:56 Gavin Hu (Arm Technology China)
  2019-07-25 12:11 ` Zhang, Qi Z
  0 siblings, 1 reply; 9+ messages in thread
From: Gavin Hu (Arm Technology China) @ 2019-07-25  9:56 UTC (permalink / raw)
  To: Zhang, Qi Z, bruce.richardson, Ananyev, Konstantin
  Cc: users, Honnappa Nagarahalli, Phil Yang (Arm Technology China)

Hi Qi,

I am working on optimizing the i40e vPMD on aarch64 and I see this patch relevant.
Could you illuminate what issue this patch was fixing?
I understand the PMD works at the driver layer, for delivery of L2 packets.
It does not own the responsibility to keep order(the responsibility lies with the protocol stack, like TCP)?

http://patches.dpdk.org/patch/16665/

Best regards,
Gavin

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-07-25  9:56 [dpdk-users] i40e vPMD fix out of order Rx read issue Gavin Hu (Arm Technology China)
@ 2019-07-25 12:11 ` Zhang, Qi Z
  2019-07-26 10:00   ` Gavin Hu (Arm Technology China)
  0 siblings, 1 reply; 9+ messages in thread
From: Zhang, Qi Z @ 2019-07-25 12:11 UTC (permalink / raw)
  To: Gavin Hu (Arm Technology China), Richardson, Bruce, Ananyev, Konstantin
  Cc: users, Honnappa Nagarahalli, Phil Yang (Arm Technology China)

Hi Gavin:

	in vPMD, we read 4 or 8 packets as batch, we count DD bits for packet received, but not check the if they are continues or not, we assume it should always be 1000, 1100, 1110, 1111 ....(take batch size is 4 as example)
	while the out of order read instruction generated by compiler will cause driver to get un-continues DD bits, like 1011, the descriptor on the hole actually is invalid since when it is read , descriptor is not write back yet, but we still process this as 1110, it cause an corrupted mbuf returned.
	
	hope this is helpful.

Regards
Qi

> -----Original Message-----
> From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> Sent: Thursday, July 25, 2019 5:57 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Cc: users@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>
> Subject: i40e vPMD fix out of order Rx read issue
> 
> Hi Qi,
> 
> I am working on optimizing the i40e vPMD on aarch64 and I see this patch
> relevant.
> Could you illuminate what issue this patch was fixing?
> I understand the PMD works at the driver layer, for delivery of L2 packets.
> It does not own the responsibility to keep order(the responsibility lies with the
> protocol stack, like TCP)?
> 
> http://patches.dpdk.org/patch/16665/
> 
> Best regards,
> Gavin
> 
> IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended recipient,
> please notify the sender immediately and do not disclose the contents to any
> other person, use it for any purpose, or store or copy the information in any
> medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-07-25 12:11 ` Zhang, Qi Z
@ 2019-07-26 10:00   ` Gavin Hu (Arm Technology China)
  2019-07-27  1:32     ` Zhang, Qi Z
  0 siblings, 1 reply; 9+ messages in thread
From: Gavin Hu (Arm Technology China) @ 2019-07-26 10:00 UTC (permalink / raw)
  To: Zhang, Qi Z, Richardson, Bruce, Ananyev, Konstantin
  Cc: users, Honnappa Nagarahalli, Phil Yang (Arm Technology China),
	Ruifeng Wang (Arm Technology China)

Hi Qi,

Thanks for your explanation!
I did some testing and found the barriers caused a big drop in RFC2544 NDR performance on aarch64, how about it on X86?
Is it possible to count DD bits in a way of surviving across the out-of-order descriptors reading?

Best Regards,
Gavin

> -----Original Message-----
> From: Zhang, Qi Z <qi.z.zhang@intel.com>
> Sent: Thursday, July 25, 2019 8:11 PM
> To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Cc: users@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>
> Subject: RE: i40e vPMD fix out of order Rx read issue
>
> Hi Gavin:
>
> in vPMD, we read 4 or 8 packets as batch, we count DD bits for packet
> received, but not check the if they are continues or not, we assume it should
> always be 1000, 1100, 1110, 1111 ....(take batch size is 4 as example)
> while the out of order read instruction generated by compiler will
> cause driver to get un-continues DD bits, like 1011, the descriptor on the hole
> actually is invalid since when it is read , descriptor is not write back yet, but we
> still process this as 1110, it cause an corrupted mbuf returned.
>
> hope this is helpful.
>
> Regards
> Qi
>
> > -----Original Message-----
> > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> > Sent: Thursday, July 25, 2019 5:57 PM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>
> > Cc: users@dpdk.org; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > <Phil.Yang@arm.com>
> > Subject: i40e vPMD fix out of order Rx read issue
> >
> > Hi Qi,
> >
> > I am working on optimizing the i40e vPMD on aarch64 and I see this patch
> > relevant.
> > Could you illuminate what issue this patch was fixing?
> > I understand the PMD works at the driver layer, for delivery of L2 packets.
> > It does not own the responsibility to keep order(the responsibility lies with
> the
> > protocol stack, like TCP)?
> >
> > http://patches.dpdk.org/patch/16665/
> >
> > Best regards,
> > Gavin
> >
> > IMPORTANT NOTICE: The contents of this email and any attachments are
> > confidential and may also be privileged. If you are not the intended recipient,
> > please notify the sender immediately and do not disclose the contents to any
> > other person, use it for any purpose, or store or copy the information in any
> > medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-07-26 10:00   ` Gavin Hu (Arm Technology China)
@ 2019-07-27  1:32     ` Zhang, Qi Z
  2019-08-14  3:19       ` Gavin Hu (Arm Technology China)
  0 siblings, 1 reply; 9+ messages in thread
From: Zhang, Qi Z @ 2019-07-27  1:32 UTC (permalink / raw)
  To: Gavin Hu (Arm Technology China), Richardson, Bruce, Ananyev, Konstantin
  Cc: users, Honnappa Nagarahalli, Phil Yang (Arm Technology China),
	Ruifeng Wang (Arm Technology China)



> -----Original Message-----
> From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> Sent: Friday, July 26, 2019 6:01 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Cc: users@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> <Ruifeng.Wang@arm.com>
> Subject: RE: i40e vPMD fix out of order Rx read issue
> 
> Hi Qi,
> 
> Thanks for your explanation!
> I did some testing and found the barriers caused a big drop in RFC2544 NDR
> performance on aarch64, how about it on X86?
For x86, I think we don't have performance drop, the memory barrier just change the compiler's behavior to avoid generate out of order read instructions, and x86 guarantee no out of read execution, so it does not add new instructions that cost CPU cycles.

> Is it possible to count DD bits in a way of surviving across the out-of-order
> descriptors reading?

I think it is possible, but this will impact performance on x86, 
but for aarch64, you can try out to see if that benefit and do proper optimization on related vPMD implementation.

> 
> Best Regards,
> Gavin
> 
> > -----Original Message-----
> > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> > Sent: Thursday, July 25, 2019 8:11 PM
> > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>
> > Cc: users@dpdk.org; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > <Phil.Yang@arm.com>
> > Subject: RE: i40e vPMD fix out of order Rx read issue
> >
> > Hi Gavin:
> >
> > in vPMD, we read 4 or 8 packets as batch, we count DD bits for packet
> > received, but not check the if they are continues or not, we assume it
> > should always be 1000, 1100, 1110, 1111 ....(take batch size is 4 as
> > example) while the out of order read instruction generated by compiler
> > will cause driver to get un-continues DD bits, like 1011, the
> > descriptor on the hole actually is invalid since when it is read ,
> > descriptor is not write back yet, but we still process this as 1110, it cause an
> corrupted mbuf returned.
> >
> > hope this is helpful.
> >
> > Regards
> > Qi
> >
> > > -----Original Message-----
> > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> > > Sent: Thursday, July 25, 2019 5:57 PM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > > <konstantin.ananyev@intel.com>
> > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > > <Phil.Yang@arm.com>
> > > Subject: i40e vPMD fix out of order Rx read issue
> > >
> > > Hi Qi,
> > >
> > > I am working on optimizing the i40e vPMD on aarch64 and I see this
> > > patch relevant.
> > > Could you illuminate what issue this patch was fixing?
> > > I understand the PMD works at the driver layer, for delivery of L2 packets.
> > > It does not own the responsibility to keep order(the responsibility
> > > lies with
> > the
> > > protocol stack, like TCP)?
> > >
> > > http://patches.dpdk.org/patch/16665/
> > >
> > > Best regards,
> > > Gavin
> > >
> > > IMPORTANT NOTICE: The contents of this email and any attachments are
> > > confidential and may also be privileged. If you are not the intended
> > > recipient, please notify the sender immediately and do not disclose
> > > the contents to any other person, use it for any purpose, or store
> > > or copy the information in any medium. Thank you.
> IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended recipient,
> please notify the sender immediately and do not disclose the contents to any
> other person, use it for any purpose, or store or copy the information in any
> medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-07-27  1:32     ` Zhang, Qi Z
@ 2019-08-14  3:19       ` Gavin Hu (Arm Technology China)
  2019-08-14  3:38         ` Zhang, Qi Z
  0 siblings, 1 reply; 9+ messages in thread
From: Gavin Hu (Arm Technology China) @ 2019-08-14  3:19 UTC (permalink / raw)
  To: Zhang, Qi Z, Richardson, Bruce, Ananyev, Konstantin
  Cc: users, Honnappa Nagarahalli, Phil Yang (Arm Technology China),
	Ruifeng Wang (Arm Technology China)

Hi Qi,

> -----Original Message-----
> From: Zhang, Qi Z <qi.z.zhang@intel.com>
> Sent: Saturday, July 27, 2019 9:33 AM
> To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Cc: users@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> <Ruifeng.Wang@arm.com>
> Subject: RE: i40e vPMD fix out of order Rx read issue
>
>
>
> > -----Original Message-----
> > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> > Sent: Friday, July 26, 2019 6:01 PM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>
> > Cc: users@dpdk.org; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> > <Ruifeng.Wang@arm.com>
> > Subject: RE: i40e vPMD fix out of order Rx read issue
> >
> > Hi Qi,
> >
> > Thanks for your explanation!
> > I did some testing and found the barriers caused a big drop in RFC2544 NDR
> > performance on aarch64, how about it on X86?
> For x86, I think we don't have performance drop, the memory barrier just
> change the compiler's behavior to avoid generate out of order read
> instructions, and x86 guarantee no out of read execution, so it does not add
> new instructions that cost CPU cycles.
>
> > Is it possible to count DD bits in a way of surviving across the out-of-order
> > descriptors reading?
>
> I think it is possible, but this will impact performance on x86,
> but for aarch64, you can try out to see if that benefit and do proper
> optimization on related vPMD implementation.
The patches for aarch64 are in community review, and performances gain were measured
both for ixgbe and i40e vPMD. Very appreciate if you can further review and comment.
Here are the patch links:
IXGBE vPMD:
http://patches.dpdk.org/patch/57649/
http://patches.dpdk.org/patch/57650/
I40e vPMD:
http://patches.dpdk.org/cover/57651/
http://patches.dpdk.org/patch/57652/
> >
> > Best Regards,
> > Gavin
> >
> > > -----Original Message-----
> > > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > Sent: Thursday, July 25, 2019 8:11 PM
> > > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> > > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> > > <konstantin.ananyev@intel.com>
> > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > > <Phil.Yang@arm.com>
> > > Subject: RE: i40e vPMD fix out of order Rx read issue
> > >
> > > Hi Gavin:
> > >
> > > in vPMD, we read 4 or 8 packets as batch, we count DD bits for packet
> > > received, but not check the if they are continues or not, we assume it
> > > should always be 1000, 1100, 1110, 1111 ....(take batch size is 4 as
> > > example) while the out of order read instruction generated by compiler
> > > will cause driver to get un-continues DD bits, like 1011, the
> > > descriptor on the hole actually is invalid since when it is read ,
> > > descriptor is not write back yet, but we still process this as 1110, it cause
> an
> > corrupted mbuf returned.
> > >
> > > hope this is helpful.
> > >
> > > Regards
> > > Qi
> > >
> > > > -----Original Message-----
> > > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> > > > Sent: Thursday, July 25, 2019 5:57 PM
> > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > > > <konstantin.ananyev@intel.com>
> > > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > > > <Phil.Yang@arm.com>
> > > > Subject: i40e vPMD fix out of order Rx read issue
> > > >
> > > > Hi Qi,
> > > >
> > > > I am working on optimizing the i40e vPMD on aarch64 and I see this
> > > > patch relevant.
> > > > Could you illuminate what issue this patch was fixing?
> > > > I understand the PMD works at the driver layer, for delivery of L2
> packets.
> > > > It does not own the responsibility to keep order(the responsibility
> > > > lies with
> > > the
> > > > protocol stack, like TCP)?
> > > >
> > > > http://patches.dpdk.org/patch/16665/
> > > >
> > > > Best regards,
> > > > Gavin
> > > >
> > > > IMPORTANT NOTICE: The contents of this email and any attachments are
> > > > confidential and may also be privileged. If you are not the intended
> > > > recipient, please notify the sender immediately and do not disclose
> > > > the contents to any other person, use it for any purpose, or store
> > > > or copy the information in any medium. Thank you.
> > IMPORTANT NOTICE: The contents of this email and any attachments are
> > confidential and may also be privileged. If you are not the intended
> recipient,
> > please notify the sender immediately and do not disclose the contents to
> any
> > other person, use it for any purpose, or store or copy the information in any
> > medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-08-14  3:19       ` Gavin Hu (Arm Technology China)
@ 2019-08-14  3:38         ` Zhang, Qi Z
  2019-08-16  2:50           ` Ye Xiaolong
  0 siblings, 1 reply; 9+ messages in thread
From: Zhang, Qi Z @ 2019-08-14  3:38 UTC (permalink / raw)
  To: Gavin Hu (Arm Technology China), Richardson, Bruce, Ananyev, Konstantin
  Cc: users, Honnappa Nagarahalli, Phil Yang (Arm Technology China),
	Ruifeng Wang (Arm Technology China),
	Ye, Xiaolong

+ Xiaolong

I think we can merge them to dpdk-next-net-intel early since 19.11 cycle is started.

> -----Original Message-----
> From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> Sent: Wednesday, August 14, 2019 11:20 AM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>
> Cc: users@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> <Ruifeng.Wang@arm.com>
> Subject: RE: i40e vPMD fix out of order Rx read issue
> 
> Hi Qi,
> 
> > -----Original Message-----
> > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> > Sent: Saturday, July 27, 2019 9:33 AM
> > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>
> > Cc: users@dpdk.org; Honnappa Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> > <Ruifeng.Wang@arm.com>
> > Subject: RE: i40e vPMD fix out of order Rx read issue
> >
> >
> >
> > > -----Original Message-----
> > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> > > Sent: Friday, July 26, 2019 6:01 PM
> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > > <konstantin.ananyev@intel.com>
> > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> > > <Ruifeng.Wang@arm.com>
> > > Subject: RE: i40e vPMD fix out of order Rx read issue
> > >
> > > Hi Qi,
> > >
> > > Thanks for your explanation!
> > > I did some testing and found the barriers caused a big drop in
> > > RFC2544 NDR performance on aarch64, how about it on X86?
> > For x86, I think we don't have performance drop, the memory barrier
> > just change the compiler's behavior to avoid generate out of order
> > read instructions, and x86 guarantee no out of read execution, so it
> > does not add new instructions that cost CPU cycles.
> >
> > > Is it possible to count DD bits in a way of surviving across the
> > > out-of-order descriptors reading?
> >
> > I think it is possible, but this will impact performance on x86, but
> > for aarch64, you can try out to see if that benefit and do proper
> > optimization on related vPMD implementation.
> The patches for aarch64 are in community review, and performances gain
> were measured both for ixgbe and i40e vPMD. Very appreciate if you can
> further review and comment.
> Here are the patch links:
> IXGBE vPMD:
> http://patches.dpdk.org/patch/57649/
> http://patches.dpdk.org/patch/57650/
> I40e vPMD:
> http://patches.dpdk.org/cover/57651/
> http://patches.dpdk.org/patch/57652/



> > >
> > > Best Regards,
> > > Gavin
> > >
> > > > -----Original Message-----
> > > > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> > > > Sent: Thursday, July 25, 2019 8:11 PM
> > > > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> > > > Richardson, Bruce <bruce.richardson@intel.com>; Ananyev,
> > > > Konstantin <konstantin.ananyev@intel.com>
> > > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > > > <Phil.Yang@arm.com>
> > > > Subject: RE: i40e vPMD fix out of order Rx read issue
> > > >
> > > > Hi Gavin:
> > > >
> > > > in vPMD, we read 4 or 8 packets as batch, we count DD bits for
> > > > packet received, but not check the if they are continues or not,
> > > > we assume it should always be 1000, 1100, 1110, 1111 ....(take
> > > > batch size is 4 as
> > > > example) while the out of order read instruction generated by
> > > > compiler will cause driver to get un-continues DD bits, like 1011,
> > > > the descriptor on the hole actually is invalid since when it is
> > > > read , descriptor is not write back yet, but we still process this
> > > > as 1110, it cause
> > an
> > > corrupted mbuf returned.
> > > >
> > > > hope this is helpful.
> > > >
> > > > Regards
> > > > Qi
> > > >
> > > > > -----Original Message-----
> > > > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> > > > > Sent: Thursday, July 25, 2019 5:57 PM
> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > > > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > > > > <konstantin.ananyev@intel.com>
> > > > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > > > > <Phil.Yang@arm.com>
> > > > > Subject: i40e vPMD fix out of order Rx read issue
> > > > >
> > > > > Hi Qi,
> > > > >
> > > > > I am working on optimizing the i40e vPMD on aarch64 and I see
> > > > > this patch relevant.
> > > > > Could you illuminate what issue this patch was fixing?
> > > > > I understand the PMD works at the driver layer, for delivery of
> > > > > L2
> > packets.
> > > > > It does not own the responsibility to keep order(the
> > > > > responsibility lies with
> > > > the
> > > > > protocol stack, like TCP)?
> > > > >
> > > > > http://patches.dpdk.org/patch/16665/
> > > > >
> > > > > Best regards,
> > > > > Gavin
> > > > >
> > > > > IMPORTANT NOTICE: The contents of this email and any attachments
> > > > > are confidential and may also be privileged. If you are not the
> > > > > intended recipient, please notify the sender immediately and do
> > > > > not disclose the contents to any other person, use it for any
> > > > > purpose, or store or copy the information in any medium. Thank you.
> > > IMPORTANT NOTICE: The contents of this email and any attachments are
> > > confidential and may also be privileged. If you are not the intended
> > recipient,
> > > please notify the sender immediately and do not disclose the
> > > contents to
> > any
> > > other person, use it for any purpose, or store or copy the
> > > information in any medium. Thank you.
> IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended recipient,
> please notify the sender immediately and do not disclose the contents to any
> other person, use it for any purpose, or store or copy the information in any
> medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-08-14  3:38         ` Zhang, Qi Z
@ 2019-08-16  2:50           ` Ye Xiaolong
  2019-08-16  3:44             ` Zhang, Qi Z
  0 siblings, 1 reply; 9+ messages in thread
From: Ye Xiaolong @ 2019-08-16  2:50 UTC (permalink / raw)
  To: Zhang, Qi Z
  Cc: Gavin Hu (Arm Technology China),
	Richardson, Bruce, Ananyev, Konstantin, users,
	Honnappa Nagarahalli, Phil Yang (Arm Technology China),
	Ruifeng Wang (Arm Technology China)

On 08/14, Zhang, Qi Z wrote:
>+ Xiaolong
>
>I think we can merge them to dpdk-next-net-intel early since 19.11 cycle is started.

Ok, could you help to ack these patches?


Thanks,
Xiaolong
>
>> -----Original Message-----
>> From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
>> Sent: Wednesday, August 14, 2019 11:20 AM
>> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
>> <bruce.richardson@intel.com>; Ananyev, Konstantin
>> <konstantin.ananyev@intel.com>
>> Cc: users@dpdk.org; Honnappa Nagarahalli
>> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
>> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
>> <Ruifeng.Wang@arm.com>
>> Subject: RE: i40e vPMD fix out of order Rx read issue
>> 
>> Hi Qi,
>> 
>> > -----Original Message-----
>> > From: Zhang, Qi Z <qi.z.zhang@intel.com>
>> > Sent: Saturday, July 27, 2019 9:33 AM
>> > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
>> > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
>> > <konstantin.ananyev@intel.com>
>> > Cc: users@dpdk.org; Honnappa Nagarahalli
>> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
>> > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
>> > <Ruifeng.Wang@arm.com>
>> > Subject: RE: i40e vPMD fix out of order Rx read issue
>> >
>> >
>> >
>> > > -----Original Message-----
>> > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
>> > > Sent: Friday, July 26, 2019 6:01 PM
>> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
>> > > <bruce.richardson@intel.com>; Ananyev, Konstantin
>> > > <konstantin.ananyev@intel.com>
>> > > Cc: users@dpdk.org; Honnappa Nagarahalli
>> > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
>> > > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
>> > > <Ruifeng.Wang@arm.com>
>> > > Subject: RE: i40e vPMD fix out of order Rx read issue
>> > >
>> > > Hi Qi,
>> > >
>> > > Thanks for your explanation!
>> > > I did some testing and found the barriers caused a big drop in
>> > > RFC2544 NDR performance on aarch64, how about it on X86?
>> > For x86, I think we don't have performance drop, the memory barrier
>> > just change the compiler's behavior to avoid generate out of order
>> > read instructions, and x86 guarantee no out of read execution, so it
>> > does not add new instructions that cost CPU cycles.
>> >
>> > > Is it possible to count DD bits in a way of surviving across the
>> > > out-of-order descriptors reading?
>> >
>> > I think it is possible, but this will impact performance on x86, but
>> > for aarch64, you can try out to see if that benefit and do proper
>> > optimization on related vPMD implementation.
>> The patches for aarch64 are in community review, and performances gain
>> were measured both for ixgbe and i40e vPMD. Very appreciate if you can
>> further review and comment.
>> Here are the patch links:
>> IXGBE vPMD:
>> http://patches.dpdk.org/patch/57649/
>> http://patches.dpdk.org/patch/57650/
>> I40e vPMD:
>> http://patches.dpdk.org/cover/57651/
>> http://patches.dpdk.org/patch/57652/
>
>
>
>> > >
>> > > Best Regards,
>> > > Gavin
>> > >
>> > > > -----Original Message-----
>> > > > From: Zhang, Qi Z <qi.z.zhang@intel.com>
>> > > > Sent: Thursday, July 25, 2019 8:11 PM
>> > > > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
>> > > > Richardson, Bruce <bruce.richardson@intel.com>; Ananyev,
>> > > > Konstantin <konstantin.ananyev@intel.com>
>> > > > Cc: users@dpdk.org; Honnappa Nagarahalli
>> > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
>> > > > <Phil.Yang@arm.com>
>> > > > Subject: RE: i40e vPMD fix out of order Rx read issue
>> > > >
>> > > > Hi Gavin:
>> > > >
>> > > > in vPMD, we read 4 or 8 packets as batch, we count DD bits for
>> > > > packet received, but not check the if they are continues or not,
>> > > > we assume it should always be 1000, 1100, 1110, 1111 ....(take
>> > > > batch size is 4 as
>> > > > example) while the out of order read instruction generated by
>> > > > compiler will cause driver to get un-continues DD bits, like 1011,
>> > > > the descriptor on the hole actually is invalid since when it is
>> > > > read , descriptor is not write back yet, but we still process this
>> > > > as 1110, it cause
>> > an
>> > > corrupted mbuf returned.
>> > > >
>> > > > hope this is helpful.
>> > > >
>> > > > Regards
>> > > > Qi
>> > > >
>> > > > > -----Original Message-----
>> > > > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
>> > > > > Sent: Thursday, July 25, 2019 5:57 PM
>> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
>> > > > > <bruce.richardson@intel.com>; Ananyev, Konstantin
>> > > > > <konstantin.ananyev@intel.com>
>> > > > > Cc: users@dpdk.org; Honnappa Nagarahalli
>> > > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
>> > > > > <Phil.Yang@arm.com>
>> > > > > Subject: i40e vPMD fix out of order Rx read issue
>> > > > >
>> > > > > Hi Qi,
>> > > > >
>> > > > > I am working on optimizing the i40e vPMD on aarch64 and I see
>> > > > > this patch relevant.
>> > > > > Could you illuminate what issue this patch was fixing?
>> > > > > I understand the PMD works at the driver layer, for delivery of
>> > > > > L2
>> > packets.
>> > > > > It does not own the responsibility to keep order(the
>> > > > > responsibility lies with
>> > > > the
>> > > > > protocol stack, like TCP)?
>> > > > >
>> > > > > http://patches.dpdk.org/patch/16665/
>> > > > >
>> > > > > Best regards,
>> > > > > Gavin
>> > > > >
>> > > > > IMPORTANT NOTICE: The contents of this email and any attachments
>> > > > > are confidential and may also be privileged. If you are not the
>> > > > > intended recipient, please notify the sender immediately and do
>> > > > > not disclose the contents to any other person, use it for any
>> > > > > purpose, or store or copy the information in any medium. Thank you.
>> > > IMPORTANT NOTICE: The contents of this email and any attachments are
>> > > confidential and may also be privileged. If you are not the intended
>> > recipient,
>> > > please notify the sender immediately and do not disclose the
>> > > contents to
>> > any
>> > > other person, use it for any purpose, or store or copy the
>> > > information in any medium. Thank you.
>> IMPORTANT NOTICE: The contents of this email and any attachments are
>> confidential and may also be privileged. If you are not the intended recipient,
>> please notify the sender immediately and do not disclose the contents to any
>> other person, use it for any purpose, or store or copy the information in any
>> medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-08-16  2:50           ` Ye Xiaolong
@ 2019-08-16  3:44             ` Zhang, Qi Z
  2019-08-16  6:24               ` Gavin Hu (Arm Technology China)
  0 siblings, 1 reply; 9+ messages in thread
From: Zhang, Qi Z @ 2019-08-16  3:44 UTC (permalink / raw)
  To: Ye, Xiaolong
  Cc: Gavin Hu (Arm Technology China),
	Richardson, Bruce, Ananyev, Konstantin, users,
	Honnappa Nagarahalli, Phil Yang (Arm Technology China),
	Ruifeng Wang (Arm Technology China)



> -----Original Message-----
> From: Ye, Xiaolong
> Sent: Friday, August 16, 2019 10:51 AM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; users@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> <Ruifeng.Wang@arm.com>
> Subject: Re: i40e vPMD fix out of order Rx read issue
> 
> On 08/14, Zhang, Qi Z wrote:
> >+ Xiaolong
> >
> >I think we can merge them to dpdk-next-net-intel early since 19.11 cycle is
> started.
> 
> Ok, could you help to ack these patches?

I'm not expert on arm, the patch is only impact on the arm platform and it is reviewed by peoples that works on arm already, I think we can just apply it.
> 
> 
> Thanks,
> Xiaolong
> >
> >> -----Original Message-----
> >> From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> >> Sent: Wednesday, August 14, 2019 11:20 AM
> >> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> >> <bruce.richardson@intel.com>; Ananyev, Konstantin
> >> <konstantin.ananyev@intel.com>
> >> Cc: users@dpdk.org; Honnappa Nagarahalli
> >> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> >> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> >> <Ruifeng.Wang@arm.com>
> >> Subject: RE: i40e vPMD fix out of order Rx read issue
> >>
> >> Hi Qi,
> >>
> >> > -----Original Message-----
> >> > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> >> > Sent: Saturday, July 27, 2019 9:33 AM
> >> > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> >> > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> >> > <konstantin.ananyev@intel.com>
> >> > Cc: users@dpdk.org; Honnappa Nagarahalli
> >> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> >> > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> >> > <Ruifeng.Wang@arm.com>
> >> > Subject: RE: i40e vPMD fix out of order Rx read issue
> >> >
> >> >
> >> >
> >> > > -----Original Message-----
> >> > > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> >> > > Sent: Friday, July 26, 2019 6:01 PM
> >> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> >> > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> >> > > <konstantin.ananyev@intel.com>
> >> > > Cc: users@dpdk.org; Honnappa Nagarahalli
> >> > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> >> > > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> >> > > <Ruifeng.Wang@arm.com>
> >> > > Subject: RE: i40e vPMD fix out of order Rx read issue
> >> > >
> >> > > Hi Qi,
> >> > >
> >> > > Thanks for your explanation!
> >> > > I did some testing and found the barriers caused a big drop in
> >> > > RFC2544 NDR performance on aarch64, how about it on X86?
> >> > For x86, I think we don't have performance drop, the memory barrier
> >> > just change the compiler's behavior to avoid generate out of order
> >> > read instructions, and x86 guarantee no out of read execution, so
> >> > it does not add new instructions that cost CPU cycles.
> >> >
> >> > > Is it possible to count DD bits in a way of surviving across the
> >> > > out-of-order descriptors reading?
> >> >
> >> > I think it is possible, but this will impact performance on x86,
> >> > but for aarch64, you can try out to see if that benefit and do
> >> > proper optimization on related vPMD implementation.
> >> The patches for aarch64 are in community review, and performances
> >> gain were measured both for ixgbe and i40e vPMD. Very appreciate if
> >> you can further review and comment.
> >> Here are the patch links:
> >> IXGBE vPMD:
> >> http://patches.dpdk.org/patch/57649/
> >> http://patches.dpdk.org/patch/57650/
> >> I40e vPMD:
> >> http://patches.dpdk.org/cover/57651/
> >> http://patches.dpdk.org/patch/57652/
> >
> >
> >
> >> > >
> >> > > Best Regards,
> >> > > Gavin
> >> > >
> >> > > > -----Original Message-----
> >> > > > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> >> > > > Sent: Thursday, July 25, 2019 8:11 PM
> >> > > > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> >> > > > Richardson, Bruce <bruce.richardson@intel.com>; Ananyev,
> >> > > > Konstantin <konstantin.ananyev@intel.com>
> >> > > > Cc: users@dpdk.org; Honnappa Nagarahalli
> >> > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology
> >> > > > China) <Phil.Yang@arm.com>
> >> > > > Subject: RE: i40e vPMD fix out of order Rx read issue
> >> > > >
> >> > > > Hi Gavin:
> >> > > >
> >> > > > in vPMD, we read 4 or 8 packets as batch, we count DD bits for
> >> > > > packet received, but not check the if they are continues or
> >> > > > not, we assume it should always be 1000, 1100, 1110, 1111
> >> > > > ....(take batch size is 4 as
> >> > > > example) while the out of order read instruction generated by
> >> > > > compiler will cause driver to get un-continues DD bits, like
> >> > > > 1011, the descriptor on the hole actually is invalid since when
> >> > > > it is read , descriptor is not write back yet, but we still
> >> > > > process this as 1110, it cause
> >> > an
> >> > > corrupted mbuf returned.
> >> > > >
> >> > > > hope this is helpful.
> >> > > >
> >> > > > Regards
> >> > > > Qi
> >> > > >
> >> > > > > -----Original Message-----
> >> > > > > From: Gavin Hu (Arm Technology China)
> >> > > > > [mailto:Gavin.Hu@arm.com]
> >> > > > > Sent: Thursday, July 25, 2019 5:57 PM
> >> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> >> > > > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> >> > > > > <konstantin.ananyev@intel.com>
> >> > > > > Cc: users@dpdk.org; Honnappa Nagarahalli
> >> > > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology
> >> > > > > China) <Phil.Yang@arm.com>
> >> > > > > Subject: i40e vPMD fix out of order Rx read issue
> >> > > > >
> >> > > > > Hi Qi,
> >> > > > >
> >> > > > > I am working on optimizing the i40e vPMD on aarch64 and I see
> >> > > > > this patch relevant.
> >> > > > > Could you illuminate what issue this patch was fixing?
> >> > > > > I understand the PMD works at the driver layer, for delivery
> >> > > > > of
> >> > > > > L2
> >> > packets.
> >> > > > > It does not own the responsibility to keep order(the
> >> > > > > responsibility lies with
> >> > > > the
> >> > > > > protocol stack, like TCP)?
> >> > > > >
> >> > > > > http://patches.dpdk.org/patch/16665/
> >> > > > >
> >> > > > > Best regards,
> >> > > > > Gavin
> >> > > > >
> >> > > > > IMPORTANT NOTICE: The contents of this email and any
> >> > > > > attachments are confidential and may also be privileged. If
> >> > > > > you are not the intended recipient, please notify the sender
> >> > > > > immediately and do not disclose the contents to any other
> >> > > > > person, use it for any purpose, or store or copy the information in
> any medium. Thank you.
> >> > > IMPORTANT NOTICE: The contents of this email and any attachments
> >> > > are confidential and may also be privileged. If you are not the
> >> > > intended
> >> > recipient,
> >> > > please notify the sender immediately and do not disclose the
> >> > > contents to
> >> > any
> >> > > other person, use it for any purpose, or store or copy the
> >> > > information in any medium. Thank you.
> >> IMPORTANT NOTICE: The contents of this email and any attachments are
> >> confidential and may also be privileged. If you are not the intended
> >> recipient, please notify the sender immediately and do not disclose
> >> the contents to any other person, use it for any purpose, or store or
> >> copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [dpdk-users] i40e vPMD fix out of order Rx read issue
  2019-08-16  3:44             ` Zhang, Qi Z
@ 2019-08-16  6:24               ` Gavin Hu (Arm Technology China)
  0 siblings, 0 replies; 9+ messages in thread
From: Gavin Hu (Arm Technology China) @ 2019-08-16  6:24 UTC (permalink / raw)
  To: Zhang, Qi Z, Ye, Xiaolong
  Cc: Richardson, Bruce, Ananyev, Konstantin, users,
	Honnappa Nagarahalli, Phil Yang (Arm Technology China),
	Ruifeng Wang (Arm Technology China)

Thanks Qi and Xiaolong,

Yes, it applies to Arm platforms only, we already discussed with Qi about the NIC behavior and we believe the patches are safe, but if Qi can further review, that would be better.

Best regards,
Gavin

> -----Original Message-----
> From: Zhang, Qi Z <qi.z.zhang@intel.com>
> Sent: Friday, August 16, 2019 11:44 AM
> To: Ye, Xiaolong <xiaolong.ye@intel.com>
> Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; users@dpdk.org; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> <Ruifeng.Wang@arm.com>
> Subject: RE: i40e vPMD fix out of order Rx read issue
>
>
>
> > -----Original Message-----
> > From: Ye, Xiaolong
> > Sent: Friday, August 16, 2019 10:51 AM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>
> > Cc: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; Richardson,
> > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>; users@dpdk.org; Honnappa
> Nagarahalli
> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> > <Ruifeng.Wang@arm.com>
> > Subject: Re: i40e vPMD fix out of order Rx read issue
> >
> > On 08/14, Zhang, Qi Z wrote:
> > >+ Xiaolong
> > >
> > >I think we can merge them to dpdk-next-net-intel early since 19.11 cycle
> is
> > started.
> >
> > Ok, could you help to ack these patches?
>
> I'm not expert on arm, the patch is only impact on the arm platform and it is
> reviewed by peoples that works on arm already, I think we can just apply it.
> >
> >
> > Thanks,
> > Xiaolong
> > >
> > >> -----Original Message-----
> > >> From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com]
> > >> Sent: Wednesday, August 14, 2019 11:20 AM
> > >> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > >> <bruce.richardson@intel.com>; Ananyev, Konstantin
> > >> <konstantin.ananyev@intel.com>
> > >> Cc: users@dpdk.org; Honnappa Nagarahalli
> > >> <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology China)
> > >> <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> > >> <Ruifeng.Wang@arm.com>
> > >> Subject: RE: i40e vPMD fix out of order Rx read issue
> > >>
> > >> Hi Qi,
> > >>
> > >> > -----Original Message-----
> > >> > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> > >> > Sent: Saturday, July 27, 2019 9:33 AM
> > >> > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> Richardson,
> > >> > Bruce <bruce.richardson@intel.com>; Ananyev, Konstantin
> > >> > <konstantin.ananyev@intel.com>
> > >> > Cc: users@dpdk.org; Honnappa Nagarahalli
> > >> > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology
> China)
> > >> > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> > >> > <Ruifeng.Wang@arm.com>
> > >> > Subject: RE: i40e vPMD fix out of order Rx read issue
> > >> >
> > >> >
> > >> >
> > >> > > -----Original Message-----
> > >> > > From: Gavin Hu (Arm Technology China)
> [mailto:Gavin.Hu@arm.com]
> > >> > > Sent: Friday, July 26, 2019 6:01 PM
> > >> > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > >> > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > >> > > <konstantin.ananyev@intel.com>
> > >> > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > >> > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology
> China)
> > >> > > <Phil.Yang@arm.com>; Ruifeng Wang (Arm Technology China)
> > >> > > <Ruifeng.Wang@arm.com>
> > >> > > Subject: RE: i40e vPMD fix out of order Rx read issue
> > >> > >
> > >> > > Hi Qi,
> > >> > >
> > >> > > Thanks for your explanation!
> > >> > > I did some testing and found the barriers caused a big drop in
> > >> > > RFC2544 NDR performance on aarch64, how about it on X86?
> > >> > For x86, I think we don't have performance drop, the memory barrier
> > >> > just change the compiler's behavior to avoid generate out of order
> > >> > read instructions, and x86 guarantee no out of read execution, so
> > >> > it does not add new instructions that cost CPU cycles.
> > >> >
> > >> > > Is it possible to count DD bits in a way of surviving across the
> > >> > > out-of-order descriptors reading?
> > >> >
> > >> > I think it is possible, but this will impact performance on x86,
> > >> > but for aarch64, you can try out to see if that benefit and do
> > >> > proper optimization on related vPMD implementation.
> > >> The patches for aarch64 are in community review, and performances
> > >> gain were measured both for ixgbe and i40e vPMD. Very appreciate if
> > >> you can further review and comment.
> > >> Here are the patch links:
> > >> IXGBE vPMD:
> > >> http://patches.dpdk.org/patch/57649/
> > >> http://patches.dpdk.org/patch/57650/
> > >> I40e vPMD:
> > >> http://patches.dpdk.org/cover/57651/
> > >> http://patches.dpdk.org/patch/57652/
> > >
> > >
> > >
> > >> > >
> > >> > > Best Regards,
> > >> > > Gavin
> > >> > >
> > >> > > > -----Original Message-----
> > >> > > > From: Zhang, Qi Z <qi.z.zhang@intel.com>
> > >> > > > Sent: Thursday, July 25, 2019 8:11 PM
> > >> > > > To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> > >> > > > Richardson, Bruce <bruce.richardson@intel.com>; Ananyev,
> > >> > > > Konstantin <konstantin.ananyev@intel.com>
> > >> > > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > >> > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology
> > >> > > > China) <Phil.Yang@arm.com>
> > >> > > > Subject: RE: i40e vPMD fix out of order Rx read issue
> > >> > > >
> > >> > > > Hi Gavin:
> > >> > > >
> > >> > > > in vPMD, we read 4 or 8 packets as batch, we count DD bits for
> > >> > > > packet received, but not check the if they are continues or
> > >> > > > not, we assume it should always be 1000, 1100, 1110, 1111
> > >> > > > ....(take batch size is 4 as
> > >> > > > example) while the out of order read instruction generated by
> > >> > > > compiler will cause driver to get un-continues DD bits, like
> > >> > > > 1011, the descriptor on the hole actually is invalid since when
> > >> > > > it is read , descriptor is not write back yet, but we still
> > >> > > > process this as 1110, it cause
> > >> > an
> > >> > > corrupted mbuf returned.
> > >> > > >
> > >> > > > hope this is helpful.
> > >> > > >
> > >> > > > Regards
> > >> > > > Qi
> > >> > > >
> > >> > > > > -----Original Message-----
> > >> > > > > From: Gavin Hu (Arm Technology China)
> > >> > > > > [mailto:Gavin.Hu@arm.com]
> > >> > > > > Sent: Thursday, July 25, 2019 5:57 PM
> > >> > > > > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Richardson, Bruce
> > >> > > > > <bruce.richardson@intel.com>; Ananyev, Konstantin
> > >> > > > > <konstantin.ananyev@intel.com>
> > >> > > > > Cc: users@dpdk.org; Honnappa Nagarahalli
> > >> > > > > <Honnappa.Nagarahalli@arm.com>; Phil Yang (Arm Technology
> > >> > > > > China) <Phil.Yang@arm.com>
> > >> > > > > Subject: i40e vPMD fix out of order Rx read issue
> > >> > > > >
> > >> > > > > Hi Qi,
> > >> > > > >
> > >> > > > > I am working on optimizing the i40e vPMD on aarch64 and I see
> > >> > > > > this patch relevant.
> > >> > > > > Could you illuminate what issue this patch was fixing?
> > >> > > > > I understand the PMD works at the driver layer, for delivery
> > >> > > > > of
> > >> > > > > L2
> > >> > packets.
> > >> > > > > It does not own the responsibility to keep order(the
> > >> > > > > responsibility lies with
> > >> > > > the
> > >> > > > > protocol stack, like TCP)?
> > >> > > > >
> > >> > > > > http://patches.dpdk.org/patch/16665/
> > >> > > > >
> > >> > > > > Best regards,
> > >> > > > > Gavin
> > >> > > > >
> > >> > > > > IMPORTANT NOTICE: The contents of this email and any
> > >> > > > > attachments are confidential and may also be privileged. If
> > >> > > > > you are not the intended recipient, please notify the sender
> > >> > > > > immediately and do not disclose the contents to any other
> > >> > > > > person, use it for any purpose, or store or copy the information
> in
> > any medium. Thank you.
> > >> > > IMPORTANT NOTICE: The contents of this email and any
> attachments
> > >> > > are confidential and may also be privileged. If you are not the
> > >> > > intended
> > >> > recipient,
> > >> > > please notify the sender immediately and do not disclose the
> > >> > > contents to
> > >> > any
> > >> > > other person, use it for any purpose, or store or copy the
> > >> > > information in any medium. Thank you.
> > >> IMPORTANT NOTICE: The contents of this email and any attachments
> are
> > >> confidential and may also be privileged. If you are not the intended
> > >> recipient, please notify the sender immediately and do not disclose
> > >> the contents to any other person, use it for any purpose, or store or
> > >> copy the information in any medium. Thank you.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, back to index

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-25  9:56 [dpdk-users] i40e vPMD fix out of order Rx read issue Gavin Hu (Arm Technology China)
2019-07-25 12:11 ` Zhang, Qi Z
2019-07-26 10:00   ` Gavin Hu (Arm Technology China)
2019-07-27  1:32     ` Zhang, Qi Z
2019-08-14  3:19       ` Gavin Hu (Arm Technology China)
2019-08-14  3:38         ` Zhang, Qi Z
2019-08-16  2:50           ` Ye Xiaolong
2019-08-16  3:44             ` Zhang, Qi Z
2019-08-16  6:24               ` Gavin Hu (Arm Technology China)

DPDK usage discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/ public-inbox