DPDK patches and discussions
 help / color / mirror / Atom feed
From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: "Tan, Jianfeng" <jianfeng.tan@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>, "Bie, Tiwei" <tiwei.bie@intel.com>,
	"jfreimann@redhat.com" <jfreimann@redhat.com>
Cc: "stable@dpdk.org" <stable@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH] vhost: avoid concurrency when logging dirty pages
Date: Tue, 27 Mar 2018 13:24:41 +0200	[thread overview]
Message-ID: <905c5e05-7d76-a45b-26c3-39dd5e6a56b1@redhat.com> (raw)
In-Reply-To: <ED26CBA2FAD1BF48A8719AEF02201E36514A5FFD@SHSMSX103.ccr.corp.intel.com>



On 03/27/2018 03:15 AM, Tan, Jianfeng wrote:
> 
> 
>> -----Original Message-----
>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com]
>> Sent: Wednesday, March 21, 2018 11:44 PM
>> To: dev@dpdk.org; Tan, Jianfeng; Bie, Tiwei; jfreimann@redhat.com
>> Cc: stable@dpdk.org; Maxime Coquelin
>> Subject: [PATCH] vhost: avoid concurrency when logging dirty pages
>>
>> This patch aims at fixing a migration performance regression
>> faced since atomic operation is used to log pages as dirty when
>> doing live migration.
>>
>> Instead of setting a single bit by doing an atomic read-modify-write
>> operation to log a page as dirty, this patch write 0xFF to the
>> corresponding byte, and so logs 8 page as dirty.
>>
>> The advantage is that it avoids concurrent atomic operations by
>> multiple PMD threads, the drawback is that some clean pages are
>> marked as dirty and so are transferred twice.
>>
>> Fixes: 897f13a1f726 ("vhost: make page logging atomic")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
> 
> As you mentioned, the concern is if it affects the rate of convergence. Hope we could have some tests on that.

We tested internally with live migration during PVP testing, at a rate
of 3Mpps.
In these conditions, we obtain same numbers as if we revert
897f13a1f726.

> Reviewed-by: Jianfeng Tan <jianfeng.tan@intel.com>

Thanks,
Maxime

> Thanks!
> 
>> ---
>>   lib/librte_vhost/vhost.h | 12 ++++--------
>>   1 file changed, 4 insertions(+), 8 deletions(-)
>>
>> diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
>> index d947bc9e3..aa2606f8a 100644
>> --- a/lib/librte_vhost/vhost.h
>> +++ b/lib/librte_vhost/vhost.h
>> @@ -247,18 +247,14 @@ struct virtio_net {
>>   #define VHOST_LOG_PAGE	4096
>>
>>   /*
>> - * Atomically set a bit in memory.
>> + * Mark all pages belonging to the same dirty log bitmap byte
>> + * as dirty. The goal is to avoid concurrency between different
>> + * threads doing atomic read-modify-writes on the same byte.
>>    */
>> -static __rte_always_inline void
>> -vhost_set_bit(unsigned int nr, volatile uint8_t *addr)
>> -{
>> -	__sync_fetch_and_or_8(addr, (1U << nr));
>> -}
>> -
>>   static __rte_always_inline void
>>   vhost_log_page(uint8_t *log_base, uint64_t page)
>>   {
>> -	vhost_set_bit(page % 8, &log_base[page / 8]);
>> +	log_base[page / 8] = 0xff;
>>   }
>>
>>   static __rte_always_inline void
>> --
>> 2.14.3
> 

  reply	other threads:[~2018-03-27 11:24 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-03-21 15:44 Maxime Coquelin
2018-03-27  1:15 ` Tan, Jianfeng
2018-03-27 11:24   ` Maxime Coquelin [this message]
2018-03-30  7:42 ` Maxime Coquelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=905c5e05-7d76-a45b-26c3-39dd5e6a56b1@redhat.com \
    --to=maxime.coquelin@redhat.com \
    --cc=dev@dpdk.org \
    --cc=jfreimann@redhat.com \
    --cc=jianfeng.tan@intel.com \
    --cc=stable@dpdk.org \
    --cc=tiwei.bie@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).