From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by dpdk.org (Postfix) with ESMTP id 472785682 for ; Wed, 28 Oct 2015 10:52:46 +0100 (CET) Received: from zmail25.collab.prod.int.phx2.redhat.com (zmail25.collab.prod.int.phx2.redhat.com [10.5.83.31]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id t9S9qi8C026004; Wed, 28 Oct 2015 05:52:45 -0400 Date: Wed, 28 Oct 2015 05:52:44 -0400 (EDT) From: Amnon Ilan To: Yuanhan Liu Message-ID: <985011766.3510996.1446025964838.JavaMail.zimbra@redhat.com> In-Reply-To: <20151016073738.GK3115@yliu-dev.sh.intel.com> References: <20151014000508-mutt-send-email-mst@redhat.com> <20151016073738.GK3115@yliu-dev.sh.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [10.35.4.114] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF41 (Linux)/8.0.6_GA_5922) Thread-Topic: dpdk/vhost-user and VM migration Thread-Index: ffU3jZlk/MlY4sWNXg1P+ulN3Xxv3A== Cc: dev@dpdk.org, =?utf-8?Q?Marc-Andr=C3=A9?= Lureau , "Michael S. Tsirkin" Subject: Re: [dpdk-dev] dpdk/vhost-user and VM migration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 09:52:46 -0000 ----- Original Message ----- > From: "Yuanhan Liu" > To: "Michael S. Tsirkin" > Cc: dev@dpdk.org > Sent: Friday, October 16, 2015 10:37:38 AM > Subject: Re: [dpdk-dev] dpdk/vhost-user and VM migration > > On Wed, Oct 14, 2015 at 12:16:29AM +0300, Michael S. Tsirkin wrote: > > Hello! > > I am currently looking at how using dpdk on host, accessing VM memory > > using the vhost-user interface, interacts with VM migration. > > > > The issue is that any changes made to VM memory need to be tracked so > > that updates can be sent from migration source to destination. > > > > At the moment, there's a proposal of an interface extension to > > vhost-user which adds ability to do this tracking through shared memory. > > dpdk would then be responsible for tracking these updates using atomic > > operations to set bits (per page written) in a memory bitmap. > > > > This only needs to happen during migration, at other times there could > > be a jump to skip this logging. > > > > Is this a reasonable approach? > > Hi Michael, > > As I stated in another email, adding dpdk/vhost-user vm migration > support is my second TODO. However, I barely know anything about > vm migration so far, that I can't tell now. > > I will re-visit this question when I finished my first item and > after some more investigation. Yuanhan, Live-migration for vhost-user is now available upstream. Do you need some guidance on how to implement it in DPDK? Amnon > > --yliu > > > Would performance degradation during > > migration associated with atomics affect the performance to a level > > where it's no longer useful? Pls note these logs aren't latency > > sensitive, so can be done on a separate core, and can be batched. > > > > > > One alternative I'm considering is extending linux kernel so it can do > > this tracking automatically, by marking pages read-only, detecting a > > pagefault and logging the write, then making the pages writeable. This > > would mean higher worst-case overhead (pagefaults are expensive) but > > lower average one (not extra code after the first fault). Not sure how > > feasible this is yet, this would be harder to implement and it will only > > be apply to newer host kernels. > > > > Any feedback would be appreciated. > > > > -- > > MST >