DPDK patches and discussions
 help / color / mirror / Atom feed
From: Gavin Hu <Gavin.Hu@arm.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: dpdk-dev <dev@dpdk.org>, nd <nd@arm.com>,
	David Marchand <david.marchand@redhat.com>,
	"thomas@monjalon.net" <thomas@monjalon.net>,
	"rasland@mellanox.com" <rasland@mellanox.com>,
	"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
	"tiwei.bie@intel.com" <tiwei.bie@intel.com>,
	"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	Pavan Nikhilesh <pbhagavatula@marvell.com>,
	Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	Ruifeng Wang <Ruifeng.Wang@arm.com>,
	Phil Yang <Phil.Yang@arm.com>, Joyce Kong <Joyce.Kong@arm.com>,
	Steve Capper <Steve.Capper@arm.com>, nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH v2 1/3] eal/arm64: relax the io barrier for aarch64
Date: Fri, 20 Dec 2019 04:19:45 +0000	[thread overview]
Message-ID: <VI1PR08MB53765AD89AF44D92B0C4D2778F2D0@VI1PR08MB5376.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <CALBAE1Pd2KvoU_D_0W0VniLZ95Rg3FH+7Kvfps+uCXticg19mw@mail.gmail.com>

Hi Jerin,

Thanks for review, inline comments, 

> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Friday, December 20, 2019 11:38 AM
> To: Gavin Hu <Gavin.Hu@arm.com>
> Cc: dpdk-dev <dev@dpdk.org>; nd <nd@arm.com>; David Marchand
> <david.marchand@redhat.com>; thomas@monjalon.net;
> rasland@mellanox.com; maxime.coquelin@redhat.com;
> tiwei.bie@intel.com; hemant.agrawal@nxp.com; jerinj@marvell.com;
> Pavan Nikhilesh <pbhagavatula@marvell.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Ruifeng Wang
> <Ruifeng.Wang@arm.com>; Phil Yang <Phil.Yang@arm.com>; Joyce Kong
> <Joyce.Kong@arm.com>; Steve Capper <Steve.Capper@arm.com>
> Subject: Re: [dpdk-dev] [PATCH v2 1/3] eal/arm64: relax the io barrier for
> aarch64
> 
> On Fri, Dec 20, 2019 at 9:03 AM Jerin Jacob <jerinjacobk@gmail.com>
> wrote:
> >
> > On Fri, Dec 20, 2019 at 8:40 AM Gavin Hu <gavin.hu@arm.com> wrote:
> > >
> > > Armv8's peripheral coherence order is a total order on all reads and
> writes
> > > to that peripheral.[1]
> > >
> > > The peripheral coherence order for a memory-mapped peripheral
> signifies the
> > > order in which accesses arrive at the endpoint.  For a read or a write
> RW1
> > > and a read or a write RW2 to the same peripheral, then RW1 will appear
> in
> > > the peripheral coherence order for the peripheral before RW2 if either
> of
> > > the following cases apply:
> > >  1. RW1 and RW2 are accesses using Non-cacheable or Device attributes
> and
> > >     RW1 is Ordered-before RW2.
> > >  2. RW1 and RW2 are accesses using Device-nGnRE or Device-nGnRnE
> attributes
> > >     and RW1 appears in program order before RW2.
> >
> >
> > This is true if RW1 and RW2 addresses are device memory. i.e the
> > registers in the  PCI bar address.
> > If RW1 is DDR address which is been used by the controller(say NIC
> > ring descriptor) then there will be an issue.
> > For example Intel i40e driver, the admin queue update in Host DDR
> > memory and it updates the doorbell.
> > In such a case, this patch will create an issue. Correct? Have you
> > checked this patch with ARM64 + XL710 controllers?

This patch relaxes the rte_io_*mb barriers for pure PCI device memory accesses.

For mixed accesses of DDR and PCI device memory, rte_smp_*mb(DMB ISH) is not sufficient.
But rte_cio_*mb(DMB OSH) is sufficient and can be used.

> >
> > Some of the legacy code is missing such barriers, that's the reason
> > for adding rte_io_* barrier.
> 
> 
> More details:
> 
> https://dev.dpdk.narkive.com/DpIRqDuy/dpdk-dev-patch-v2-i40e-fix-eth-
> i40e-dev-init-sequence-on-thunderx
> 
> >
> > >
> > > On arm platforms, all the PCI resources are mapped to nGnRE device
> memory
> > > [2], the above case 2 holds true, that means the peripheral coherence
> order
> > > applies here and just a compiler barrier is sufficient for rte io barriers.
> > >
> > > [1] Section B2.3.4 of ARMARM,
> https://developer.arm.com/docs/ddi0487/lates
> > > t/arm-architecture-reference-manual-armv8-for-armv8-a-architecture-
> profile
> > > [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
> > > tree/drivers/pci/pci-sysfs.c#n1204
> > >
> > > Signed-off-by: Gavin Hu <gavin.hu@arm.com>
> > > Reviewed-by: Steve Capper <steve.capper@arm.com>
> > > Reviewed-by: Phil Yang <phil.yang@arm.com>
> > > ---
> > >  lib/librte_eal/common/include/arch/arm/rte_atomic_64.h | 6 +++---
> > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> > > index 859ae12..fd63956 100644
> > > --- a/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> > > +++ b/lib/librte_eal/common/include/arch/arm/rte_atomic_64.h
> > > @@ -34,11 +34,11 @@ extern "C" {
> > >
> > >  #define rte_smp_rmb() dmb(ishld)
> > >
> > > -#define rte_io_mb() rte_mb()
> > > +#define rte_io_mb() rte_compiler_barrier()
> > >
> > > -#define rte_io_wmb() rte_wmb()
> > > +#define rte_io_wmb() rte_compiler_barrier()
> > >
> > > -#define rte_io_rmb() rte_rmb()
> > > +#define rte_io_rmb() rte_compiler_barrier()
> > >
> > >  #define rte_cio_wmb() dmb(oshst)
> > >
> > > --
> > > 2.7.4
> > >

  reply	other threads:[~2019-12-20  4:19 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-22 15:27 [dpdk-dev] [PATCH v1 0/3] relax io barrier for aarch64 and use smp barriers for virtual pci memory Gavin Hu
2019-10-22 15:27 ` [dpdk-dev] [PATCH v1 1/3] eal/arm64: relax the io barrier for aarch64 Gavin Hu
2019-10-22 15:27 ` [dpdk-dev] [PATCH v1 2/3] net/virtio: virtual PCI requires smp barriers Gavin Hu
2019-10-22 15:27 ` [dpdk-dev] [PATCH v1 3/3] crypto/virtio: " Gavin Hu
2019-10-23  8:22 ` [dpdk-dev] [PATCH v1 0/3] relax io barrier for aarch64 and use smp barriers for virtual pci memory Maxime Coquelin
2019-11-07  1:13   ` Gavin Hu (Arm Technology China)
2019-12-20  3:09 ` [dpdk-dev] [PATCH v2 " Gavin Hu
2019-12-20  3:09 ` [dpdk-dev] [PATCH v2 1/3] eal/arm64: relax the io barrier for aarch64 Gavin Hu
2019-12-20  3:33   ` Jerin Jacob
2019-12-20  3:38     ` Jerin Jacob
2019-12-20  4:19       ` Gavin Hu [this message]
2019-12-20  4:34         ` Jerin Jacob
2019-12-20  6:32           ` Gavin Hu
2019-12-20  6:55             ` Jerin Jacob
2019-12-23  9:14               ` Gavin Hu
2019-12-23  9:19                 ` Jerin Jacob
2019-12-23 10:16                   ` Gavin Hu
2020-01-02  9:51                     ` Jerin Jacob
2020-01-03  6:30                       ` Gavin Hu
2020-01-03  7:34                         ` Jerin Jacob
2020-01-03  9:12                           ` Gavin Hu
2019-12-20  3:09 ` [dpdk-dev] [PATCH v2 2/3] net/virtio: virtual PCI requires smp barriers Gavin Hu
2019-12-20  8:17   ` Tiwei Bie
2019-12-20 10:19     ` Gavin Hu
2019-12-20  3:09 ` [dpdk-dev] [PATCH v2 3/3] crypto/virtio: " Gavin Hu
2020-02-08 13:48 ` [dpdk-dev] [PATCH v3] net/i40e: relaxed barrier in the tx fastpath Gavin Hu
2020-02-11  2:11   ` Ye Xiaolong
2020-02-12  6:02     ` Gavin Hu
2020-02-15  8:25   ` Jerin Jacob
2020-02-12  5:56 ` [dpdk-dev] [PATCH v4] " Gavin Hu
2020-02-15 15:16   ` Ye Xiaolong
2020-02-16  9:51     ` Thomas Monjalon
2020-02-16 16:38       ` Ye Xiaolong
2020-02-16 17:36         ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=VI1PR08MB53765AD89AF44D92B0C4D2778F2D0@VI1PR08MB5376.eurprd08.prod.outlook.com \
    --to=gavin.hu@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=Joyce.Kong@arm.com \
    --cc=Phil.Yang@arm.com \
    --cc=Ruifeng.Wang@arm.com \
    --cc=Steve.Capper@arm.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=hemant.agrawal@nxp.com \
    --cc=jerinj@marvell.com \
    --cc=jerinjacobk@gmail.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=nd@arm.com \
    --cc=pbhagavatula@marvell.com \
    --cc=rasland@mellanox.com \
    --cc=thomas@monjalon.net \
    --cc=tiwei.bie@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).