From: Raslan Darawsheh <rasland@nvidia.com>
To: Slava Ovsiienko <viacheslavo@nvidia.com>,
Phil Yang <Phil.Yang@arm.com>, Matan Azrad <matan@nvidia.com>,
Shahaf Shuler <shahafs@nvidia.com>
Cc: nd <nd@arm.com>, Alexander Kozyrev <akozyrev@nvidia.com>,
"Honnappa Nagarahalli" <Honnappa.Nagarahalli@arm.com>,
"dev@dpdk.org" <dev@dpdk.org>, nd <nd@arm.com>
Subject: Re: [dpdk-dev] [PATCH v4] net/mlx5: relaxed ordering for multi-packet RQ buffer refcnt
Date: Wed, 30 Sep 2020 12:52:52 +0000 [thread overview]
Message-ID: <DM6PR12MB2748B7D689345B46A88522D2CF330@DM6PR12MB2748.namprd12.prod.outlook.com> (raw)
In-Reply-To: <MWHPR12MB136031D5CDBC643D9A97BA9CDF330@MWHPR12MB1360.namprd12.prod.outlook.com>
Sure, I'll take the patch since I was waiting for performance impact testing, and since Slava just confirmed it,
Then we can take it yes thank you for your contribution.
Kindest regards,
Raslan Darawsheh
> -----Original Message-----
> From: Slava Ovsiienko <viacheslavo@nvidia.com>
> Sent: Wednesday, September 30, 2020 3:45 PM
> To: Phil Yang <Phil.Yang@arm.com>; Raslan Darawsheh
> <rasland@nvidia.com>; Matan Azrad <matan@nvidia.com>; Shahaf Shuler
> <shahafs@nvidia.com>
> Cc: nd <nd@arm.com>; Alexander Kozyrev <akozyrev@nvidia.com>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org;
> nd <nd@arm.com>
> Subject: RE: [PATCH v4] net/mlx5: relaxed ordering for multi-packet RQ
> buffer refcnt
>
> Looks good to me, and we've checked the performance has no impact.
> Thank you.
>
> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
>
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Phil Yang
> > Sent: Tuesday, September 29, 2020 18:23
> > To: Raslan Darawsheh <rasland@nvidia.com>; Matan Azrad
> > <matan@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>
> > Cc: nd <nd@arm.com>; Alexander Kozyrev <akozyrev@nvidia.com>;
> > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; dev@dpdk.org;
> > nd <nd@arm.com>
> > Subject: Re: [dpdk-dev] [PATCH v4] net/mlx5: relaxed ordering for multi-
> > packet RQ buffer refcnt
> >
> > Hi Raslan,
> >
> > It seems that there are no more comments for this patch.
> > So shall we proceed further?
> >
> > Thanks,
> > Phil Yang
> >
> > > -----Original Message-----
> > > From: Alexander Kozyrev <akozyrev@nvidia.com>
> > > Sent: Thursday, September 10, 2020 9:37 AM
> > > To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Phil Yang
> > > <Phil.Yang@arm.com>; akozyrev@mellanox.com;
> rasland@mellanox.com;
> > > dev@dpdk.org
> > > Cc: Phil Yang <Phil.Yang@arm.com>; matan@mellanox.com; Shahaf
> Shuler
> > > <shahafs@mellanox.com>; viacheslavo@mellanox.com; nd
> > <nd@arm.com>; nd
> > > <nd@arm.com>
> > > Subject: RE: [PATCH v4] net/mlx5: relaxed ordering for multi-packet RQ
> > > buffer refcnt
> > >
> > > > <snip>
> > > >
> > > > >
> > > > > Use c11 atomics with RELAXED ordering instead of the rte_atomic
> > > > > ops which enforce unnecessary barriers on aarch64.
> > > > >
> > > > > Signed-off-by: Phil Yang <phil.yang@arm.com>
> > > > Looks good.
> > > >
> > > > Reviewed-by: Honnappa Nagarahalli
> <honnappa.nagarahalli@arm.com>
> > >
> > > Acked-by: Alexander Kozyrev <akozyrev@nvidia.com>
> > >
> > > >
> > > > > ---
> > > > > v4:
> > > > > Remove the unnecessary ACQUIRE barrier in rx burst path.
> > > > > (Honnappa)
> > > > >
> > > > > v3:
> > > > > Split from the patchset:
> > > > >
> > >
> >
> https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpatch
> > > > >
> > > >
> > >
> >
> work.dpdk.org%2Fcover%2F68159%2F&data=02%7C01%7Cakozyrev%4
> > 0
> > > nv
> > > > idia.
> > > > >
> > > >
> > >
> > com%7Cf16ba4e8cfb145f5d82008d85529348e%7C43083d15727340c1b7db3
> > 9e
> > > f
> > > > d9ccc
> > > > >
> > > >
> > >
> > 17a%7C0%7C0%7C637352982762038088&sdata=0HzTxbzh0Dqk0hZ5PI
> > gE
> > > V
> > > > zieyV%
> > > > > 2BnLTivsVIFFxXFAtI%3D&reserved=0
> > > > >
> > > > > drivers/net/mlx5/mlx5_rxq.c | 2 +-
> > > > > drivers/net/mlx5/mlx5_rxtx.c | 16 +++++++++-------
> > > > > drivers/net/mlx5/mlx5_rxtx.h | 2 +-
> > > > > 3 files changed, 11 insertions(+), 9 deletions(-)
> > > > >
> > > > > diff --git a/drivers/net/mlx5/mlx5_rxq.c
> > > > > b/drivers/net/mlx5/mlx5_rxq.c index
> > > > > 79eb8f8..40e0239 100644
> > > > > --- a/drivers/net/mlx5/mlx5_rxq.c
> > > > > +++ b/drivers/net/mlx5/mlx5_rxq.c
> > > > > @@ -2012,7 +2012,7 @@ mlx5_mprq_buf_init(struct rte_mempool
> > *mp,
> > > void
> > > > > *opaque_arg,
> > > > >
> > > > > memset(_m, 0, sizeof(*buf));
> > > > > buf->mp = mp;
> > > > > - rte_atomic16_set(&buf->refcnt, 1);
> > > > > + __atomic_store_n(&buf->refcnt, 1, __ATOMIC_RELAXED);
> > > > > for (j = 0; j != strd_n; ++j) {
> > > > > shinfo = &buf->shinfos[j];
> > > > > shinfo->free_cb = mlx5_mprq_buf_free_cb; diff --git
> > > > > a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
> > > > > index 1b71e94..549477b 100644
> > > > > --- a/drivers/net/mlx5/mlx5_rxtx.c
> > > > > +++ b/drivers/net/mlx5/mlx5_rxtx.c
> > > > > @@ -1626,10 +1626,11 @@ mlx5_mprq_buf_free_cb(void *addr
> > > > __rte_unused,
> > > > > void *opaque) {
> > > > > struct mlx5_mprq_buf *buf = opaque;
> > > > >
> > > > > - if (rte_atomic16_read(&buf->refcnt) == 1) {
> > > > > + if (__atomic_load_n(&buf->refcnt, __ATOMIC_RELAXED) ==
> 1) {
> > > > > rte_mempool_put(buf->mp, buf);
> > > > > - } else if (rte_atomic16_add_return(&buf->refcnt, -1) == 0) {
> > > > > - rte_atomic16_set(&buf->refcnt, 1);
> > > > > + } else if (unlikely(__atomic_sub_fetch(&buf->refcnt, 1,
> > > > > + __ATOMIC_RELAXED) ==
> 0)) {
> > > > > + __atomic_store_n(&buf->refcnt, 1,
> __ATOMIC_RELAXED);
> > > > > rte_mempool_put(buf->mp, buf);
> > > > > }
> > > > > }
> > > > > @@ -1709,7 +1710,8 @@ mlx5_rx_burst_mprq(void *dpdk_rxq,
> struct
> > > > > rte_mbuf **pkts, uint16_t pkts_n)
> > > > >
> > > > > if (consumed_strd == strd_n) {
> > > > > /* Replace WQE only if the buffer is still in
> use. */
> > > > > - if (rte_atomic16_read(&buf->refcnt) > 1) {
> > > > > + if (__atomic_load_n(&buf->refcnt,
> > > > > + __ATOMIC_RELAXED) > 1) {
> > > > > mprq_buf_replace(rxq, rq_ci &
> wq_mask,
> > > > strd_n);
> > > > > /* Release the old buffer. */
> > > > > mlx5_mprq_buf_free(buf);
> > > > > @@ -1821,9 +1823,9 @@ mlx5_rx_burst_mprq(void *dpdk_rxq,
> struct
> > > > > rte_mbuf **pkts, uint16_t pkts_n)
> > > > > void *buf_addr;
> > > > >
> > > > > /* Increment the refcnt of the whole chunk.
> */
> > > > > - rte_atomic16_add_return(&buf->refcnt, 1);
> > > > > -
> MLX5_ASSERT((uint16_t)rte_atomic16_read(&buf-
> > > > > >refcnt) <=
> > > > > - strd_n + 1);
> > > > > + __atomic_add_fetch(&buf->refcnt, 1,
> > > > > __ATOMIC_RELAXED);
> > > > > + MLX5_ASSERT(__atomic_load_n(&buf-
> >refcnt,
> > > > > + __ATOMIC_RELAXED) <= strd_n +
> 1);
> > > > > buf_addr = RTE_PTR_SUB(addr,
> > > > > RTE_PKTMBUF_HEADROOM);
> > > > > /*
> > > > > * MLX5 device doesn't use iova but it is
> necessary in
> > > a
> > > > diff
> > > > > --git a/drivers/net/mlx5/mlx5_rxtx.h
> > > > > b/drivers/net/mlx5/mlx5_rxtx.h index c02a007..467f31d 100644
> > > > > --- a/drivers/net/mlx5/mlx5_rxtx.h
> > > > > +++ b/drivers/net/mlx5/mlx5_rxtx.h
> > > > > @@ -68,7 +68,7 @@ struct rxq_zip {
> > > > > /* Multi-Packet RQ buffer header. */ struct mlx5_mprq_buf {
> > > > > struct rte_mempool *mp;
> > > > > - rte_atomic16_t refcnt; /* Atomically accessed refcnt. */
> > > > > + uint16_t refcnt; /* Atomically accessed refcnt. */
> > > > > uint8_t pad[RTE_PKTMBUF_HEADROOM]; /* Headroom for
> the first
> > > > packet.
> > > > > */
> > > > > struct rte_mbuf_ext_shared_info shinfos[];
> > > > > /*
> > > > > --
> > > > > 2.7.4
next prev parent reply other threads:[~2020-09-30 12:53 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-13 12:38 [dpdk-dev] [PATCH RFC v1 0/6] barrier fix and optimization for mlx5 on aarch64 Gavin Hu
2020-02-13 12:38 ` [dpdk-dev] [PATCH RFC v1 1/6] net/mlx5: relax the barrier for UAR write Gavin Hu
2020-02-13 12:38 ` [dpdk-dev] [PATCH RFC v1 2/6] net/mlx5: use cio barrier before the BF WQE Gavin Hu
2020-02-13 12:38 ` [dpdk-dev] [PATCH RFC v1 3/6] net/mlx5: add missing barrier Gavin Hu
2020-02-13 12:38 ` [dpdk-dev] [PATCH RFC v1 4/6] net/mlx5: add descriptive comment for a barrier Gavin Hu
2020-02-13 12:38 ` [dpdk-dev] [PATCH RFC v1 5/6] net/mlx5: non-cacheable mapping defaulted for aarch64 Gavin Hu
2020-02-13 12:38 ` [dpdk-dev] [PATCH RFC v1 6/6] net/mlx5: relaxed ordering for multi-packet RQ buffer refcnt Gavin Hu
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 0/7] introduce new barrier class and use it for mlx5 PMD Gavin Hu
2020-04-10 17:20 ` Andrew Rybchenko
2020-04-11 3:46 ` Gavin Hu
2020-04-13 9:51 ` Andrew Rybchenko
2020-04-13 16:46 ` Gavin Hu
2020-05-11 18:06 ` [dpdk-dev] [RFC] eal: adjust barriers for IO on Armv8-a Honnappa Nagarahalli
2020-05-12 6:18 ` Ruifeng Wang
2020-05-12 6:42 ` Jerin Jacob
2020-05-12 8:02 ` Ruifeng Wang
2020-05-12 8:28 ` Jerin Jacob
2020-05-12 21:44 ` Honnappa Nagarahalli
2020-05-13 14:49 ` Jerin Jacob
2020-05-14 1:02 ` Honnappa Nagarahalli
2020-06-27 19:12 ` [dpdk-dev] [PATCH v2] " Honnappa Nagarahalli
2020-06-27 19:25 ` Honnappa Nagarahalli
2020-06-30 5:13 ` Jerin Jacob
2020-07-03 18:57 ` [dpdk-dev] [PATCH v3 1/3] " Honnappa Nagarahalli
2020-07-03 18:57 ` [dpdk-dev] [PATCH v3 2/3] doc: update armv8-a IO barrier changes Honnappa Nagarahalli
2020-07-05 0:57 ` Jerin Jacob
2020-07-03 18:57 ` [dpdk-dev] [PATCH v3 3/3] doc: update deprecation of CIO barrier APIs Honnappa Nagarahalli
2020-07-05 0:57 ` Jerin Jacob
2020-07-07 20:19 ` Ajit Khaparde
2020-07-08 11:05 ` Ananyev, Konstantin
2020-07-06 23:43 ` [dpdk-dev] [PATCH v4 1/3] eal: adjust barriers for IO on Armv8-a Honnappa Nagarahalli
2020-07-06 23:43 ` [dpdk-dev] [PATCH v4 2/3] doc: update armv8-a IO barrier changes Honnappa Nagarahalli
2020-07-07 8:36 ` David Marchand
2020-07-07 18:37 ` Honnappa Nagarahalli
2020-07-06 23:43 ` [dpdk-dev] [PATCH v4 3/3] doc: update deprecation of CIO barrier APIs Honnappa Nagarahalli
2020-07-07 8:39 ` David Marchand
2020-07-07 20:14 ` David Christensen
2020-07-08 11:49 ` David Marchand
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 1/7] eal: introduce new class of barriers for DMA use cases Gavin Hu
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 2/7] net/mlx5: dmb for immediate doorbell ring on aarch64 Gavin Hu
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 3/7] net/mlx5: relax barrier to order UAR writes " Gavin Hu
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 4/7] net/mlx5: relax barrier for aarch64 Gavin Hu
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 5/7] net/mlx5: add descriptive comment for a barrier Gavin Hu
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 6/7] net/mlx5: relax ordering for multi-packet RQ buffer refcnt Gavin Hu
2020-06-23 8:26 ` [dpdk-dev] [PATCH v3] net/mlx5: relaxed " Phil Yang
2020-07-13 3:02 ` Phil Yang
2020-07-20 23:21 ` Alexander Kozyrev
2020-07-21 1:55 ` Phil Yang
2020-07-21 3:58 ` Alexander Kozyrev
2020-07-21 4:03 ` Honnappa Nagarahalli
2020-07-21 4:11 ` Alexander Kozyrev
2020-07-22 12:06 ` Phil Yang
2020-07-23 4:47 ` Honnappa Nagarahalli
2020-07-23 6:11 ` Phil Yang
2020-07-23 16:53 ` Alexander Kozyrev
2020-07-27 14:52 ` Phil Yang
2020-08-06 2:43 ` Alexander Kozyrev
2020-08-11 5:20 ` Honnappa Nagarahalli
2020-09-02 21:52 ` Alexander Kozyrev
2020-09-03 2:55 ` Phil Yang
2020-09-09 13:29 ` Alexander Kozyrev
2020-09-10 1:34 ` Honnappa Nagarahalli
2020-09-03 2:53 ` [dpdk-dev] [PATCH v4] " Phil Yang
2020-09-10 1:30 ` Honnappa Nagarahalli
2020-09-10 1:36 ` Alexander Kozyrev
2020-09-29 15:22 ` Phil Yang
2020-09-30 12:44 ` Slava Ovsiienko
2020-09-30 12:52 ` Raslan Darawsheh [this message]
2020-09-30 13:57 ` Raslan Darawsheh
2020-04-10 16:41 ` [dpdk-dev] [PATCH RFC v2 7/7] doc: clarify one configuration in mlx5 guide Gavin Hu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM6PR12MB2748B7D689345B46A88522D2CF330@DM6PR12MB2748.namprd12.prod.outlook.com \
--to=rasland@nvidia.com \
--cc=Honnappa.Nagarahalli@arm.com \
--cc=Phil.Yang@arm.com \
--cc=akozyrev@nvidia.com \
--cc=dev@dpdk.org \
--cc=matan@nvidia.com \
--cc=nd@arm.com \
--cc=shahafs@nvidia.com \
--cc=viacheslavo@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).