DPDK patches and discussions
 help / color / mirror / Atom feed
From: Neil Horman <nhorman@tuxdriver.com>
To: Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>, Hayato Momma <h-momma@ce.jp.nec.com>
Subject: Re: [dpdk-dev] [memnic PATCH v2 6/7] pmd: add branch hint in recv/xmit
Date: Wed, 1 Oct 2014 22:01:26 -0400	[thread overview]
Message-ID: <20141002020126.GA9757@localhost.localdomain> (raw)
In-Reply-To: <7F861DC0615E0C47A872E6F3C5FCDDBD02AE422F@BPXM14GP.gisp.nec.co.jp>

On Wed, Oct 01, 2014 at 11:33:23PM +0000, Hiroshi Shimamoto wrote:
> > Subject: Re: [dpdk-dev] [memnic PATCH v2 6/7] pmd: add branch hint in recv/xmit
> > 
> > On Wed, Oct 01, 2014 at 09:12:44AM +0000, Hiroshi Shimamoto wrote:
> > > > Subject: Re: [dpdk-dev] [memnic PATCH v2 6/7] pmd: add branch hint in recv/xmit
> > > >
> > > > On Tue, Sep 30, 2014 at 11:52:00PM +0000, Hiroshi Shimamoto wrote:
> > > > > Hi,
> > > > >
> > > > > > Subject: Re: [dpdk-dev] [memnic PATCH v2 6/7] pmd: add branch hint in recv/xmit
> > > > > >
> > > > > > On Tue, Sep 30, 2014 at 11:14:40AM +0000, Hiroshi Shimamoto wrote:
> > > > > > > From: Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com>
> > > > > > >
> > > > > > > To reduce instruction cache miss, add branch condition hints into
> > > > > > > recv/xmit functions. This improves a bit performance.
> > > > > > >
> > > > > > > We can see performance improvements with memnic-tester.
> > > > > > > Using Xeon E5-2697 v2 @ 2.70GHz, 4 vCPU.
> > > > > > >  size |  before  |  after
> > > > > > >    64 | 5.54Mpps | 5.55Mpps
> > > > > > >   128 | 5.46Mpps | 5.44Mpps
> > > > > > >   256 | 5.21Mpps | 5.22Mpps
> > > > > > >   512 | 4.50Mpps | 4.52Mpps
> > > > > > >  1024 | 3.71Mpps | 3.73Mpps
> > > > > > >  1280 | 3.21Mpps | 3.22Mpps
> > > > > > >  1518 | 2.92Mpps | 2.93Mpps
> > > > > > >
> > > > > > > Signed-off-by: Hiroshi Shimamoto <h-shimamoto@ct.jp.nec.com>
> > > > > > > Reviewed-by: Hayato Momma <h-momma@ce.jp.nec.com>
> > > > > > > ---
> > > > > > >  pmd/pmd_memnic.c | 18 +++++++++---------
> > > > > > >  1 file changed, 9 insertions(+), 9 deletions(-)
> > > > > > >
> > > > > > > diff --git a/pmd/pmd_memnic.c b/pmd/pmd_memnic.c
> > > > > > > index 7fc3093..875d3ea 100644
> > > > > > > --- a/pmd/pmd_memnic.c
> > > > > > > +++ b/pmd/pmd_memnic.c
> > > > > > > @@ -289,26 +289,26 @@ static uint16_t memnic_recv_pkts(void *rx_queue,
> > > > > > >  	int idx, next;
> > > > > > >  	struct rte_eth_stats *st = &adapter->stats[rte_lcore_id()];
> > > > > > >
> > > > > > > -	if (!adapter->nic->hdr.valid)
> > > > > > > +	if (unlikely(!adapter->nic->hdr.valid))
> > > > > > >  		return 0;
> > > > > > >
> > > > > > >  	pkts = bytes = errs = 0;
> > > > > > >  	idx = adapter->up_idx;
> > > > > > >  	for (nr = 0; nr < nb_pkts; nr++) {
> > > > > > >  		p = &data->packets[idx];
> > > > > > > -		if (p->status != MEMNIC_PKT_ST_FILLED)
> > > > > > > +		if (unlikely(p->status != MEMNIC_PKT_ST_FILLED))
> > > > > > >  			break;
> > > > > > >  		/* prefetch the next area */
> > > > > > >  		next = idx;
> > > > > > > -		if (++next >= MEMNIC_NR_PACKET)
> > > > > > > +		if (unlikely(++next >= MEMNIC_NR_PACKET))
> > > > > > >  			next = 0;
> > > > > > >  		rte_prefetch0(&data->packets[next]);
> > > > > > > -		if (p->len > framesz) {
> > > > > > > +		if (unlikely(p->len > framesz)) {
> > > > > > >  			errs++;
> > > > > > >  			goto drop;
> > > > > > >  		}
> > > > > > >  		mb = rte_pktmbuf_alloc(adapter->mp);
> > > > > > > -		if (!mb)
> > > > > > > +		if (unlikely(!mb))
> > > > > > >  			break;
> > > > > > >
> > > > > > >  		rte_memcpy(rte_pktmbuf_mtod(mb, void *), p->data, p->len);
> > > > > > > @@ -350,7 +350,7 @@ static uint16_t memnic_xmit_pkts(void *tx_queue,
> > > > > > >  	uint64_t pkts, bytes, errs;
> > > > > > >  	uint32_t framesz = adapter->framesz;
> > > > > > >
> > > > > > > -	if (!adapter->nic->hdr.valid)
> > > > > > > +	if (unlikely(!adapter->nic->hdr.valid))
> > > > > > >  		return 0;
> > > > > > >
> > > > > > >  	pkts = bytes = errs = 0;
> > > > > > > @@ -360,7 +360,7 @@ static uint16_t memnic_xmit_pkts(void *tx_queue,
> > > > > > >  		struct rte_mbuf *sg;
> > > > > > >  		void *ptr;
> > > > > > >
> > > > > > > -		if (pkt_len > framesz) {
> > > > > > > +		if (unlikely(pkt_len > framesz)) {
> > > > > > >  			errs++;
> > > > > > >  			break;
> > > > > > >  		}
> > > > > > > @@ -379,7 +379,7 @@ retry:
> > > > > > >  			goto retry;
> > > > > > >  		}
> > > > > > >
> > > > > > > -		if (idx != ACCESS_ONCE(adapter->down_idx)) {
> > > > > > > +		if (unlikely(idx != ACCESS_ONCE(adapter->down_idx))) {
> > > > > > Why are you using ACCESS_ONCE here?  Or for that matter, anywhere else in this
> > > > > > PMD?  The whole idea of the ACCESS_ONCE macro is to assign a value to a variable
> > > > > > once and prevent it from getting reloaded from memory at a later time, this is
> > > > > > exactly contrary to that, both in the sense that you're explicitly reloading the
> > > > > > same variable multiple times, and that you're using it as part of a comparison
> > > > > > operation, rather than an asignment operation
> > > > >
> > > > > ACCESS_ONCE prevents compiler optimization and ensures load from memory.
> > > > > There could be multiple threads which read/write that index.
> > > > > We should compare the value previous and the current value in memory.
> > > > > In that reason, I use ACCESS_ONCE macro to get value in the memory.
> > > >
> > > > Should you not just make the variable volatile? That's the normal way to
> > > > guarantee reads from memory and prevent the compiler caching things in
> > > > registers.
> > >
> > > We don't want always accessing to memory, it could cause performance degradation.
> > > Like linux kernel, I use it in the place only we really load from memory.
> > >
> > Thats not true at all.  Every single read of adapter->down_idx in
> > memnic_xmit_pkts() is wrapped in a ACCESS_ONCE call.  Theres no difference in
> > doing that and just declaring a volitile variable and pointing it to
> > &adapter->down_idx (save for the increased legibility of the code)
> 
> You're right, at this moment there is no reference without ACCESS_ONCE.
> I'm not sure adding code to access that variable in the future, but
> would like to avoid accidentally a code which causes a performance issue,
> I think keeping the declaration in structure without volatile.
> As you mentioned, using local variable which points down_idx will be fine.
So you would like to continue using a macro incorrectly to avoid a possible
performance issue with code that hasn't been written yet?  Thats nonsensical.
What performance issue to see occuring if you created a volatile variable and
then used it in conjunction with ACCESS_ONCE?

Neil

> 
> I will submit a cleanup patch before starting the next development for DPDK v1.8.
> 
> thanks,
> Hiroshi
> 
> > 
> > Neil
> > 
> > > thanks,
> > > Hiroshi
> > >
> > > >
> > > > /Bruce
> > > >
> > > > >
> > > > > thanks,
> > > > > Hiroshi
> > > > >
> > > > > >
> > > > > > Neil
> > > > >
> > >
> 

  reply	other threads:[~2014-10-02  1:54 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-30 11:14 Hiroshi Shimamoto
2014-09-30 13:39 ` Xie, Huawei
2014-10-01  0:01   ` Hiroshi Shimamoto
2014-10-01  6:12     ` Thomas Monjalon
2014-09-30 14:32 ` Neil Horman
2014-09-30 23:52   ` Hiroshi Shimamoto
2014-10-01  8:44     ` Bruce Richardson
2014-10-01  9:12       ` Hiroshi Shimamoto
2014-10-01 11:13         ` Neil Horman
2014-10-01 23:33           ` Hiroshi Shimamoto
2014-10-02  2:01             ` Neil Horman [this message]
2014-10-02  2:07               ` Hiroshi Shimamoto
2014-10-02  2:42                 ` Neil Horman
2014-10-01 11:09       ` Neil Horman
2014-09-30 14:38 ` Xie, Huawei
2014-09-30 14:51   ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20141002020126.GA9757@localhost.localdomain \
    --to=nhorman@tuxdriver.com \
    --cc=dev@dpdk.org \
    --cc=h-momma@ce.jp.nec.com \
    --cc=h-shimamoto@ct.jp.nec.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).