DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Xie, Huawei" <huawei.xie@intel.com>
To: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>, "Michael S. Tsirkin" <mst@redhat.com>
Subject: Re: [dpdk-dev] [PATCH 1/3] vhost: pre update used ring for Tx and Rx
Date: Fri, 3 Jun 2016 08:18:23 +0000	[thread overview]
Message-ID: <C37D651A908B024F974696C65296B57B4C7C3BAD@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <20160601065557.GB10038@yliu-dev.sh.intel.com>

On 6/1/2016 2:53 PM, Yuanhan Liu wrote:
> On Wed, Jun 01, 2016 at 06:40:41AM +0000, Xie, Huawei wrote:
>>>  	/* Retrieve all of the head indexes first to avoid caching issues. */
>>>  	for (i = 0; i < count; i++) {
>>> -		desc_indexes[i] = vq->avail->ring[(vq->last_used_idx + i) &
>>> -					(vq->size - 1)];
>>> +		used_idx = (vq->last_used_idx + i) & (vq->size - 1);
>>> +		desc_indexes[i] = vq->avail->ring[used_idx];
>>> +
>>> +		vq->used->ring[used_idx].id  = desc_indexes[i];
>>> +		vq->used->ring[used_idx].len = 0;
>>> +		vhost_log_used_vring(dev, vq,
>>> +				offsetof(struct vring_used, ring[used_idx]),
>>> +				sizeof(vq->used->ring[used_idx]));
>>>  	}
>>>  
>>>  	/* Prefetch descriptor index. */
>>>  	rte_prefetch0(&vq->desc[desc_indexes[0]]);
>>> -	rte_prefetch0(&vq->used->ring[vq->last_used_idx & (vq->size - 1)]);
>>> -
>>>  	for (i = 0; i < count; i++) {
>>>  		int err;
>>>  
>>> -		if (likely(i + 1 < count)) {
>>> +		if (likely(i + 1 < count))
>>>  			rte_prefetch0(&vq->desc[desc_indexes[i + 1]]);
>>> -			rte_prefetch0(&vq->used->ring[(used_idx + 1) &
>>> -						      (vq->size - 1)]);
>>> -		}
>>>  
>>>  		pkts[i] = rte_pktmbuf_alloc(mbuf_pool);
>>>  		if (unlikely(pkts[i] == NULL)) {
>>> @@ -916,18 +920,12 @@ rte_vhost_dequeue_burst(int vid, uint16_t queue_id,
>>>  			rte_pktmbuf_free(pkts[i]);
>>>  			break;
>>>  		}
>>> -
>>> -		used_idx = vq->last_used_idx++ & (vq->size - 1);
>>> -		vq->used->ring[used_idx].id  = desc_indexes[i];
>>> -		vq->used->ring[used_idx].len = 0;
>>> -		vhost_log_used_vring(dev, vq,
>>> -				offsetof(struct vring_used, ring[used_idx]),
>>> -				sizeof(vq->used->ring[used_idx]));
>>>  	}
>> Had tried post-updating used ring in batch,  but forget the perf change.
> I would assume pre-updating gives better performance gain, as we are
> fiddling with avail and used ring together, which would be more cache
> friendly.
 
The distance between entry for avail ring and used ring are at least 8
cache lines.
The benefit comes from batch updates, if applicable.

>
>> One optimization would be on vhost_log_used_ring.
>> I have two ideas,
>> a) In QEMU side, we always assume use ring will be changed. so that we
>> don't need to log used ring in VHOST.
>>
>> Michael: feasible in QEMU? comments on this?
>>
>> b) We could always mark the total used ring modified rather than entry
>> by entry.
> I doubt it's worthwhile. One fact is that vhost_log_used_ring is
> a non operation in most time: it will take action only in the short
> gap of during live migration.
>
> And FYI, I even tried with all vhost_log_xxx being removed, it showed
> no performance boost at all. Therefore, it's not a factor that will
> impact performance.

I knew this.

> 	--yliu
>


  reply	other threads:[~2016-06-03  8:18 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-05-03  0:46 [dpdk-dev] [PATCH 0/3] [RFC] vhost: micro vhost optimization Yuanhan Liu
2016-05-03  0:46 ` [dpdk-dev] [PATCH 1/3] vhost: pre update used ring for Tx and Rx Yuanhan Liu
2016-06-01  6:40   ` Xie, Huawei
2016-06-01  6:55     ` Yuanhan Liu
2016-06-03  8:18       ` Xie, Huawei [this message]
2016-06-01 13:05     ` Michael S. Tsirkin
2016-05-03  0:46 ` [dpdk-dev] [PATCH 2/3] vhost: optimize dequeue for small packets Yuanhan Liu
2016-06-01  6:24   ` Xie, Huawei
2016-06-01  6:44     ` Yuanhan Liu
2016-06-03  7:42       ` Xie, Huawei
2016-06-03  7:43   ` Xie, Huawei
2016-05-03  0:46 ` [dpdk-dev] [PATCH 3/3] vhost: arrange virtio_net fields for better cache sharing Yuanhan Liu
2016-06-01  6:42   ` Xie, Huawei
2016-05-10 21:49 ` [dpdk-dev] [PATCH 0/3] [RFC] vhost: micro vhost optimization Rich Lane
2016-05-10 22:08   ` Yuanhan Liu
2016-06-14 12:42 ` Yuanhan Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=C37D651A908B024F974696C65296B57B4C7C3BAD@SHSMSX101.ccr.corp.intel.com \
    --to=huawei.xie@intel.com \
    --cc=dev@dpdk.org \
    --cc=mst@redhat.com \
    --cc=yuanhan.liu@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).