From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 5CEAD5593; Thu, 17 May 2018 13:40:10 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CE828401EF0E; Thu, 17 May 2018 11:40:09 +0000 (UTC) Received: from [10.36.112.66] (ovpn-112-66.ams2.redhat.com [10.36.112.66]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D36582026E0E; Thu, 17 May 2018 11:40:08 +0000 (UTC) To: Tiwei Bie , "Michael S. Tsirkin" Cc: dev@dpdk.org, stable@dpdk.org References: <20180516165423.5430-1-maxime.coquelin@redhat.com> <20180517060353-mutt-send-email-mst@kernel.org> <20180517045014.GA22736@debian> From: Maxime Coquelin Message-ID: <5315bb19-f902-be74-c438-0ec6aba4e886@redhat.com> Date: Thu, 17 May 2018 13:40:07 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20180517045014.GA22736@debian> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 17 May 2018 11:40:09 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.5]); Thu, 17 May 2018 11:40:09 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH v3] vhost: improve dirty pages logging performance X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 11:40:10 -0000 On 05/17/2018 06:50 AM, Tiwei Bie wrote: > On Thu, May 17, 2018 at 06:06:34AM +0300, Michael S. Tsirkin wrote: >> On Wed, May 16, 2018 at 06:54:23PM +0200, Maxime Coquelin wrote: >>> This patch caches all dirty pages logging until the used ring index >>> is updated. >>> >>> The goal of this optimization is to fix a performance regression >>> introduced when the vhost library started to use atomic operations >>> to set bits in the shared dirty log map. While the fix was valid >>> as previous implementation wasn't safe against concurent accesses, >>> contention was induced. >>> >>> With this patch, during migration, we have: >>> 1. Less atomic operations as only a single atomic OR operation >>> per 32 or 64 (depending on CPU) pages. >>> 2. Less atomic operations as during a burst, the same page will >>> be marked dirty only once. >>> 3. Less write memory barriers. >>> >>> Fixes: 897f13a1f726 ("vhost: make page logging atomic") >>> >>> Cc: stable@dpdk.org >>> >>> Cc: Tiwei Bie >>> Suggested-by: Michael S. Tsirkin >>> Signed-off-by: Maxime Coquelin >> >> >> Deferring updates until GET_BASE would also be possible, >> but would increase the chance that a disconnect causes >> ring to become inconsistent. > > Yeah. The sync of the updates from vhost backend will > be deferred a lot. Another issue is that, it probably > will increase the downtime, as there will be more pages > to sync after the old device is stopped and before the > new device is started. > >> >> I'm not sure whether there is a chance of that with this >> patch (in case of a crash after used idx updated but >> before dirty log update of the used idx), but >> at least it's not bigger than before this patch. > > The used idx update and the corresponding logging > are two operations instead of one atomic operation. > So theoretically, it should be possible. > > I got your point now. Maybe we should add a barrier > between cache sync and the used idx update to ensure > that all dirty pages are logged before they can be > seen by the guest. Good idea, I'll add a carrier at the end of the sync. Thanks, Maxime > Best regards, > Tiwei Bie >