DPDK patches and discussions
 help / color / mirror / Atom feed
From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
	"Gavin Hu (Arm Technology China)" <Gavin.Hu@arm.com>,
	Ilya Maximets <i.maximets@samsung.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Cc: nd <nd@arm.com>, "thomas@monjalon.net" <thomas@monjalon.net>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
	"Nipun.gupta@nxp.com" <nipun.gupta@nxp.com>,
	"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"chaozhu@linux.vnet.ibm.com" <chaozhu@linux.vnet.ibm.com>,
	nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH v2] ring: enforce reading the tails before ring operations
Date: Fri, 8 Mar 2019 03:21:17 +0000	[thread overview]
Message-ID: <AM6PR08MB3672277B8FBBF639E5FD7BA7984D0@AM6PR08MB3672.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <2601191342CEEE43887BDE71AB97725801365527AC@irsmsx105.ger.corp.intel.com>

> Hi Gavin,
> 
> > > >>> In weak memory models, like arm64, reading the {prod,cons}.tail
> > > >>> may get reordered after reading or writing the ring slots, which
> > > >>> corrupts the ring and stale data is observed.
> > > >>>
> > > >>> This issue was reported by NXP on 8-A72 DPAA2 board. The problem
> > > >>> is
> > > >> most
> > > >>> likely caused by missing the acquire semantics when reading
> > > >>> cons.tail (in SP enqueue) or prod.tail (in SC dequeue) which
> > > >>> makes it possible to
> > > read
> > > >> a
> > > >>> stale value from the ring slots.
> > > >>>
> > > >>> For MP (and MC) case, rte_atomic32_cmpset() already provides the
> > > >> required
> > > >>> ordering. This patch is to prevent reading and writing the ring
> > > >>> slots get reordered before reading {prod,cons}.tail for SP (and SC)
> case.
> > > >>
> > > >> Read barrier rte_smp_rmb() is OK to prevent reading the ring get
> > > >> reordered before reading the tail. However, to prevent *writing*
> > > >> the ring get reordered *before reading* the tail you need a full
> > > >> memory barrier, i.e.
> > > >> rte_smp_mb().
> > > >
> > > > ISHLD(rte_smp_rmb is DMB(ishld) orders LD/LD and LD/ST, while
> > > > WMB(ST
> > > Option) orders ST/ST.
> > > > For more details, please refer to: Table B2-1 Encoding of the DMB
> > > > and DSB
> > > <option> parameter  in
> > > > https://developer.arm.com/docs/ddi0487/latest/arm-architecture-
> > > reference-manual-armv8-for-armv8-a-architecture-profile
> > >
> > > I see. But you have to change the rte_smp_rmb() function definition
> > > in lib/librte_eal/common/include/generic/rte_atomic.h and assure
> > > that all other architectures follows same rules.
> > > Otherwise, this change is logically wrong, because read barrier in
> > > current definition could not be used to order Load with Store.
> > >
> >
> > Good points, let me re-think how to handle for other architectures.
> > Full MB is required for other architectures(x86? Ppc?), but for arm, read
> barrier(load/store and load/load) is enough.
> 
> For x86, I don't think you need any barrier here, as with IA memory mode:
> -  Reads are not reordered with other reads.
> - Writes are not reordered with older reads.
Agree

> 
> BTW, could you explain a bit more why barrier is necessary even on arm here?
> As I can see, there is a data dependency between the tail value and
> subsequent address calculations for ring writes/reads.
> Isn't that sufficient to prevent re-ordering even for weak memory model?
The tail value affects 'n'. But, the value of 'n' can be speculated because of the following 'if' statement:

if (unlikely(n > *free_entries))
                        n = (behavior == RTE_RING_QUEUE_FIXED) ? 0 : *free_entries;

The address calculations for actual ring writes/reads do not depend on the tail value. Since 'n' can be speculated, the writes/reads can be moved up before the load of the tail value.

> Konstantin
> 
> 
<snip>

  reply	other threads:[~2019-03-08  3:21 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-06  3:07 [dpdk-dev] [PATCH v1] " gavin hu
     [not found] ` <CGME20190306114906eucas1p19c2572b1fe777e1eb0ca96d2e47295bd@eucas1p1.samsung.com>
2019-03-06 11:49   ` [dpdk-dev] [v1] " Ilya Maximets
2019-03-07  6:50     ` Gavin Hu (Arm Technology China)
2019-03-07  6:45 ` [dpdk-dev] [PATCH v2] " gavin hu
2019-03-07  8:52   ` Ilya Maximets
2019-03-07  9:27     ` Gavin Hu (Arm Technology China)
2019-03-07  9:48       ` Ilya Maximets
2019-03-07 10:44         ` Gavin Hu (Arm Technology China)
2019-03-07 11:17           ` Ananyev, Konstantin
2019-03-08  3:21             ` Honnappa Nagarahalli [this message]
2019-03-08  5:27               ` Gavin Hu (Arm Technology China)
2019-03-08 16:33                 ` Ananyev, Konstantin
2019-03-10 20:47                   ` Honnappa Nagarahalli
2019-03-11 13:58                     ` Ananyev, Konstantin
2019-03-08  4:23           ` Gavin Hu (Arm Technology China)
2019-03-08  5:06             ` Honnappa Nagarahalli
2019-03-08 12:13             ` Ananyev, Konstantin
2019-03-08 15:05               ` Gavin Hu (Arm Technology China)
2019-03-08 15:50                 ` Ananyev, Konstantin
2019-03-08 23:18                   ` Thomas Monjalon
2019-03-08 23:48                     ` Honnappa Nagarahalli
2019-03-09 10:28                       ` Gavin Hu (Arm Technology China)
2019-03-12 16:58 ` [dpdk-dev] [PATCH v3 0/1] ring: enforce reading the tail before reading ring slots Gavin Hu
2019-03-12 16:58 ` [dpdk-dev] [PATCH v3 1/1] " Gavin Hu
2019-03-13  8:12   ` Nipun Gupta
2019-03-15 13:26   ` Ananyev, Konstantin
2019-03-15 13:26     ` Ananyev, Konstantin
2019-03-28  0:21     ` Thomas Monjalon
2019-03-28  0:21       ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=AM6PR08MB3672277B8FBBF639E5FD7BA7984D0@AM6PR08MB3672.eurprd08.prod.outlook.com \
    --to=honnappa.nagarahalli@arm.com \
    --cc=Gavin.Hu@arm.com \
    --cc=bruce.richardson@intel.com \
    --cc=chaozhu@linux.vnet.ibm.com \
    --cc=dev@dpdk.org \
    --cc=hemant.agrawal@nxp.com \
    --cc=i.maximets@samsung.com \
    --cc=jerinj@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=nd@arm.com \
    --cc=nipun.gupta@nxp.com \
    --cc=olivier.matz@6wind.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).