DPDK patches and discussions
 help / color / mirror / Atom feed
From: Olivier MATZ <olivier.matz@6wind.com>
To: "Hanoch Haim (hhaim)" <hhaim@cisco.com>
Cc: Ilya Matveychikov <matvejchikov@gmail.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	Konstantin Ananyev <konstantin.ananyev@intel.com>
Subject: Re: [dpdk-dev] [PATCH v3] mbuf: cleanup rte_pktmbuf_lastseg(), fix atomic usage
Date: Wed, 15 Nov 2017 18:30:59 +0100	[thread overview]
Message-ID: <20171115173058.mrkrv3usbl5sfw3h@platinum> (raw)
In-Reply-To: <ad03f3a3142141afb9dc565500429df9@XCH-RTP-017.cisco.com>

Hi,

On Wed, Nov 15, 2017 at 12:46:15PM +0000, Hanoch Haim (hhaim) wrote:
> +Oliver, 
> Ilia, I assume there is a reason why Oliver did that, I just consolidate the code. 
> He didn't want to *write* the same value to mbuf field. 
> In the common case that pointer was already cleared by the driver, it is better to just read it. From cache synchronization perspective it will run faster.
> 
> Thanks,
> Hanoh
> 
> 
> -----Original Message-----
> From: Ilya Matveychikov [mailto:matvejchikov@gmail.com] 
> Sent: Wednesday, November 15, 2017 1:14 PM
> To: Hanoch Haim (hhaim)
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v3] mbuf: cleanup rte_pktmbuf_lastseg(), fix atomic usage
> 
> 
> > On Nov 15, 2017, at 1:14 PM, Hanoh Haim <hhaim@cisco.com> wrote:
> > 

I think the patch should be renamed in something like:

  mbuf: fix mbuf free performance with non atomic refcnt

A description of the problem in the commit log would also be welcome.

It looks it is a regression introduced by commit 8f094a9ac5d7.
In that case, we should also have:

Fixes: 8f094a9ac5d7 ("mbuf: set mbuf fields while in pool")


> > Signed-off-by: Hanoh Haim <hhaim@cisco.com>
> > ---
> > lib/librte_mbuf/rte_mbuf.h | 27 +++++++++++++--------------
> > 1 file changed, 13 insertions(+), 14 deletions(-)
> > 
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h 
> > index 7e326bb..ab110f8 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -1159,6 +1159,15 @@ static inline void rte_pktmbuf_reset(struct rte_mbuf *m)
> > 	__rte_mbuf_sanity_check(m, 1);
> > }
> > 
> > +
> > +static __rte_always_inline void rte_pktmbuf_reset_before_free(struct 
> > +rte_mbuf *m) {
> > +	if (m->next != NULL) {
> > +		m->next = NULL;
> > +		m->nb_segs = 1;
> > +	}
> > +}
> > +
> 
> Probably it will be more clean to add something __te_pktmbuf_reset_nb_segs() without check for (m->next != NULL) and use it everywhere in mbuf’s the code, not only in
> rte_pktmbuf_prefree_seg() function. And I think it will be better to have separate patch for that.

I'm not convinced that:

    __rte_pktmbuf_reset_nb_segs(m);

is clearer than:

   m->next = NULL;
   m->nb_segs = 1;

Anyway, I agree this should not be part of this patch. We should only keep
the fix.


> 
> > /**
> >  * Allocate a new mbuf from a mempool.
> >  *
> > @@ -1323,8 +1332,7 @@ static inline void rte_pktmbuf_detach(struct rte_mbuf *m)
> > 	m->ol_flags = 0;
> > 
> > 	if (rte_mbuf_refcnt_update(md, -1) == 0) {
> > -		md->next = NULL;
> > -		md->nb_segs = 1;
> 
> Using rte_pktmbuf_reset_before_free() here adds unnecessary check for m->next in that path.

Yes, agree with Ilya.


> 
> > +		rte_pktmbuf_reset_before_free(md);
> > 		rte_mbuf_refcnt_set(md, 1);
> > 		rte_mbuf_raw_free(md);
> > 	}
> > @@ -1354,25 +1362,16 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
> > 		if (RTE_MBUF_INDIRECT(m))
> > 			rte_pktmbuf_detach(m);
> > 
> > -		if (m->next != NULL) {
> > -			m->next = NULL;
> > -			m->nb_segs = 1;
> > -		}
> > -
> > +		rte_pktmbuf_reset_before_free(m);
> > 		return m;
> > 
> > -       } else if (rte_atomic16_add_return(&m->refcnt_atomic, -1) == 0) {
> > -
> > +	} else if (rte_mbuf_refcnt_update(m, -1) == 0) {

I agree with Konstantin's comment done in another thread [1]:

  '''
  That would cause extra read; cmp (and possible slowdown) for atomic refcnt.
  If that really need to be fixed - probably we need to introduce a new function
  that would do update without trying to read refctn first - rte_mbuf_refcnt_update_blind() or so.
  '''

However I'm not sure a new function is really needed: the name is not ideal,
and it would only be used once. What about the patch below?

==============================
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -1361,8 +1361,18 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
 
                return m;
 
-       } else if (rte_atomic16_add_return(&m->refcnt_atomic, -1) == 0) {
+       } else {
 
+               /* We don't use rte_mbuf_refcnt_update() because we already
+                * tested that refcnt != 1.
+                */
+#ifdef RTE_MBUF_REFCNT_ATOMIC
+               ret = rte_atomic16_add_return(&m->refcnt_atomic, -1);
+#else
+               ret = --m->refcnt;
+#endif
+               if (ret != 0)
+                       return NULL;
 
                if (RTE_MBUF_INDIRECT(m))
                        rte_pktmbuf_detach(m);
@@ -1375,7 +1385,6 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m)
 
                return m;
        }
-       return NULL;
 }
 
 /* deprecated, replaced by rte_pktmbuf_prefree_seg() */
==============================

[1] http://dpdk.org/dev/patchwork/patch/31378/


Regards,
Olivier

  reply	other threads:[~2017-11-15 17:31 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-11-15  9:14 Hanoh Haim
2017-11-15 11:13 ` Ilya Matveychikov
2017-11-15 12:46   ` Hanoch Haim (hhaim)
2017-11-15 17:30     ` Olivier MATZ [this message]
2017-11-16  7:16       ` Hanoch Haim (hhaim)
2017-11-16  8:07         ` Ilya Matveychikov
2017-11-16  8:42         ` Olivier MATZ
2017-11-16  9:06           ` Hanoch Haim (hhaim)
2017-11-16  9:32             ` Ilya Matveychikov
2017-11-16  9:37               ` Olivier MATZ
2017-11-16  9:44                 ` Ilya Matveychikov
2017-11-16 10:54       ` Ananyev, Konstantin
2017-12-08 15:46 ` [dpdk-dev] [PATCH v4] mbuf: fix mbuf free performance with non atomic refcnt Olivier Matz
2017-12-08 16:04   ` Ilya Matveychikov
2017-12-08 16:19     ` Olivier MATZ
2017-12-08 16:37   ` Stephen Hemminger
2017-12-10  8:37   ` Hanoch Haim (hhaim)
2017-12-11 10:28   ` Olivier MATZ
2018-01-18 23:23   ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171115173058.mrkrv3usbl5sfw3h@platinum \
    --to=olivier.matz@6wind.com \
    --cc=dev@dpdk.org \
    --cc=hhaim@cisco.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=matvejchikov@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).