DPDK patches and discussions
 help / color / mirror / Atom feed
From: Phil Yang <Phil.Yang@arm.com>
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	"Carrillo, Erik G" <erik.g.carrillo@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>
Cc: "drc@linux.vnet.ibm.com" <drc@linux.vnet.ibm.com>,
	Ruifeng Wang <Ruifeng.Wang@arm.com>,
	Dharmik Thakkar <Dharmik.Thakkar@arm.com>, nd <nd@arm.com>,
	"stable@dpdk.org" <stable@dpdk.org>, nd <nd@arm.com>,
	"jerinj@marvell.com" <jerinj@marvell.com>
Subject: Re: [dpdk-dev] [PATCH 1/3] eventdev: fix race condition on timer list counter
Date: Thu, 2 Jul 2020 03:26:30 +0000	[thread overview]
Message-ID: <VE1PR08MB4640D1AEF9AA9AF3CA312AB9E96D0@VE1PR08MB4640.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <DB6PR0802MB2216B52A7A16EC38FDB082DF989B0@DB6PR0802MB2216.eurprd08.prod.outlook.com>

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Friday, June 19, 2020 2:26 AM
> To: Carrillo, Erik G <erik.g.carrillo@intel.com>; Phil Yang
> <Phil.Yang@arm.com>; dev@dpdk.org
> Cc: drc@linux.vnet.ibm.com; Ruifeng Wang <Ruifeng.Wang@arm.com>;
> Dharmik Thakkar <Dharmik.Thakkar@arm.com>; nd <nd@arm.com>;
> stable@dpdk.org; Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>;
> nd <nd@arm.com>
> Subject: RE: [PATCH 1/3] eventdev: fix race condition on timer list counter
> 
> <snip>
> 
> >
> > Hi Phil,
> >
> > Good catch - thanks for the fix.   I've commented in-line:
> >
> > > -----Original Message-----
> > > From: Phil Yang <phil.yang@arm.com>
> > > Sent: Friday, June 12, 2020 6:20 AM
> > > To: dev@dpdk.org; Carrillo, Erik G <erik.g.carrillo@intel.com>
> > > Cc: drc@linux.vnet.ibm.com; honnappa.nagarahalli@arm.com;
> > > ruifeng.wang@arm.com; dharmik.thakkar@arm.com; nd@arm.com;
> > > stable@dpdk.org
> > > Subject: [PATCH 1/3] eventdev: fix race condition on timer list
> > > counter
> > >
> > > The n_poll_lcores counter and poll_lcore array are shared between
> > > lcores and the update of these variables are out of the protection of
> > > spinlock on each lcore timer list. The read-modify-write operations of
> > > the counter are not atomic, so it has the potential of race condition
> > between lcores.
> > >
> > > Use c11 atomics with RELAXED ordering to prevent confliction.
> > >
> > > Fixes: cc7b73ea9e3b ("eventdev: add new software timer adapter")
> > > Cc: erik.g.carrillo@intel.com
> > > Cc: stable@dpdk.org
> > >
> > > Signed-off-by: Phil Yang <phil.yang@arm.com>
> > > Reviewed-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
> > > Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > > ---
> > >  lib/librte_eventdev/rte_event_timer_adapter.c | 16 ++++++++++++----
> > >  1 file changed, 12 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/lib/librte_eventdev/rte_event_timer_adapter.c
> > > b/lib/librte_eventdev/rte_event_timer_adapter.c
> > > index 005459f..6a0e283 100644
> > > --- a/lib/librte_eventdev/rte_event_timer_adapter.c
> > > +++ b/lib/librte_eventdev/rte_event_timer_adapter.c
> > > @@ -583,6 +583,7 @@ swtim_callback(struct rte_timer *tim)
> > >  uint16_t nb_evs_invalid = 0;
> > >  uint64_t opaque;
> > >  int ret;
> > > +int n_lcores;
> > >
> > >  opaque = evtim->impl_opaque[1];
> > >  adapter = (struct rte_event_timer_adapter *)(uintptr_t)opaque; @@
> > > -605,8 +606,12 @@ swtim_callback(struct rte_timer *tim)
> > >        "with immediate expiry value");
> > >  }
> > >
> > > -if (unlikely(rte_atomic16_test_and_set(&sw-
> > > >in_use[lcore].v)))
> > > -sw->poll_lcores[sw->n_poll_lcores++] = lcore;
> > > +if (unlikely(rte_atomic16_test_and_set(&sw-
> > > >in_use[lcore].v))) {
> > > +n_lcores = __atomic_fetch_add(&sw->n_poll_lcores,
> > > 1,
> > > +__ATOMIC_RELAXED);
> Since this commit will be back ported, we should prefer to use rte_atomic
> APIs for this commit. Otherwise, we will have a mix of rte_atomic and C11
> APIs.
> My suggestion is to fix this bug using rte_atomic so that backported code will
> have only rte_atomic APIs. Add another commit (if required) in this series to
> make the bug fix use C11 APIs (this commit will not be backported).

Hi Honnappa,

It doesn't have an applicable rte_atomic_XXX API to fix this issue.
The rte_atomic32_inc doesn't return the original value of the input parameter and rte_atomic32_add_return can only return the new value.

Meanwhile, the rte_timer_alt_manage & rte_timer_stop_all API not support rte_atomic type parameters. We might need to rewrite these two APIs if we want to use rte_atomic operations for n_pol_lcores and poll_lcores array.

So, a better solution could be to backport the entire c11 solution to stable releases.

Thanks,
Phil

> 
> >
> > Just a nit, but let's align the continued line with the opening parentheses in
> > this location and below.  With these changes:
> >
> > Acked-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
> >
> > > +__atomic_store_n(&sw->poll_lcores[n_lcores],
> > > lcore,
> > > +__ATOMIC_RELAXED);
> > > +}
> > >  } else {
> > >  EVTIM_BUF_LOG_DBG("buffered an event timer expiry event");
> > >
> > > @@ -1011,6 +1016,7 @@ __swtim_arm_burst(const struct
> > > rte_event_timer_adapter *adapter,
> > >  uint32_t lcore_id = rte_lcore_id();
> > >  struct rte_timer *tim, *tims[nb_evtims];
> > >  uint64_t cycles;
> > > +int n_lcores;
> > >
> > >  #ifdef RTE_LIBRTE_EVENTDEV_DEBUG
> > >  /* Check that the service is running. */ @@ -1033,8 +1039,10 @@
> > > __swtim_arm_burst(const struct rte_event_timer_adapter *adapter,
> > >  if (unlikely(rte_atomic16_test_and_set(&sw->in_use[lcore_id].v))) {
> > >  EVTIM_LOG_DBG("Adding lcore id = %u to list of lcores to
> > poll",
> > >        lcore_id);
> > > -sw->poll_lcores[sw->n_poll_lcores] = lcore_id;
> > > -++sw->n_poll_lcores;
> > > +n_lcores = __atomic_fetch_add(&sw->n_poll_lcores, 1,
> > > +__ATOMIC_RELAXED);
> > > +__atomic_store_n(&sw->poll_lcores[n_lcores], lcore_id,
> > > +__ATOMIC_RELAXED);
> > >  }
> > >
> > >  ret = rte_mempool_get_bulk(sw->tim_pool, (void **)tims,
> > > --
> > > 2.7.4
> 


  parent reply	other threads:[~2020-07-02  3:26 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-12 11:19 Phil Yang
2020-06-12 11:19 ` [dpdk-dev] [PATCH 2/3] eventdev: use c11 atomics for lcore timer armed flag Phil Yang
2020-06-23 21:01   ` Carrillo, Erik G
2020-06-28 16:12     ` Phil Yang
2020-06-23 21:20   ` Stephen Hemminger
2020-06-23 21:31     ` Carrillo, Erik G
2020-06-28 16:32       ` Phil Yang
2020-06-12 11:19 ` [dpdk-dev] [PATCH 3/3] eventdev: relax smp barriers with c11 atomics Phil Yang
2020-06-22 10:12   ` Phil Yang
2020-06-23 19:38     ` Carrillo, Erik G
2020-06-28 17:33       ` Phil Yang
2020-06-29 18:07         ` Carrillo, Erik G
2020-06-18 15:17 ` [dpdk-dev] [PATCH 1/3] eventdev: fix race condition on timer list counter Carrillo, Erik G
2020-06-18 18:25   ` Honnappa Nagarahalli
2020-06-22  9:48     ` Phil Yang
2020-07-01 11:22       ` Jerin Jacob
2020-07-02  3:28         ` Phil Yang
2020-07-02  3:26     ` Phil Yang [this message]
2020-07-02  3:56       ` Honnappa Nagarahalli
2020-07-02 21:15         ` Carrillo, Erik G
2020-07-02 21:30           ` Honnappa Nagarahalli
2020-06-22  9:09   ` Phil Yang
2020-07-02  5:26 ` [dpdk-dev] [PATCH v2 1/4] " Phil Yang
2020-07-02  5:26   ` [dpdk-dev] [PATCH v2 2/4] eventdev: use c11 atomics for lcore timer armed flag Phil Yang
2020-07-02 20:21     ` Carrillo, Erik G
2020-07-02  5:26   ` [dpdk-dev] [PATCH v2 3/4] eventdev: remove redundant code Phil Yang
2020-07-03  3:35     ` Dharmik Thakkar
2020-07-02  5:26   ` [dpdk-dev] [PATCH v2 4/4] eventdev: relax smp barriers with c11 atomics Phil Yang
2020-07-02 20:30     ` Carrillo, Erik G
2020-07-03 10:50       ` Jerin Jacob
2020-07-06 10:04     ` Thomas Monjalon
2020-07-06 15:32       ` Phil Yang
2020-07-06 15:40         ` Thomas Monjalon
2020-07-07 11:13   ` [dpdk-dev] [PATCH v3 1/4] eventdev: fix race condition on timer list counter Phil Yang
2020-07-07 11:13     ` [dpdk-dev] [PATCH v3 2/4] eventdev: use C11 atomics for lcore timer armed flag Phil Yang
2020-07-07 11:13     ` [dpdk-dev] [PATCH v3 3/4] eventdev: remove redundant code Phil Yang
2020-07-07 11:13     ` [dpdk-dev] [PATCH v3 4/4] eventdev: relax smp barriers with C11 atomics Phil Yang
2020-07-07 14:29       ` Jerin Jacob
2020-07-07 15:56         ` Phil Yang
2020-07-07 15:54     ` [dpdk-dev] [PATCH v4 1/4] eventdev: fix race condition on timer list counter Phil Yang
2020-07-07 15:54       ` [dpdk-dev] [PATCH v4 2/4] eventdev: use C11 atomics for lcore timer armed flag Phil Yang
2020-07-07 15:54       ` [dpdk-dev] [PATCH v4 3/4] eventdev: remove redundant code Phil Yang
2020-07-07 15:54       ` [dpdk-dev] [PATCH v4 4/4] eventdev: relax smp barriers with C11 atomics Phil Yang
2020-07-08 13:30       ` [dpdk-dev] [PATCH v4 1/4] eventdev: fix race condition on timer list counter Jerin Jacob
2020-07-08 15:01         ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=VE1PR08MB4640D1AEF9AA9AF3CA312AB9E96D0@VE1PR08MB4640.eurprd08.prod.outlook.com \
    --to=phil.yang@arm.com \
    --cc=Dharmik.Thakkar@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=Ruifeng.Wang@arm.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=erik.g.carrillo@intel.com \
    --cc=jerinj@marvell.com \
    --cc=nd@arm.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).