From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 4D19C8D8C for ; Fri, 16 Oct 2015 09:36:28 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga101.fm.intel.com with ESMTP; 16 Oct 2015 00:36:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.17,687,1437462000"; d="scan'208";a="828321562" Received: from yliu-dev.sh.intel.com (HELO yliu-dev) ([10.239.66.49]) by fmsmga002.fm.intel.com with ESMTP; 16 Oct 2015 00:36:18 -0700 Date: Fri, 16 Oct 2015 15:37:38 +0800 From: Yuanhan Liu To: "Michael S. Tsirkin" Message-ID: <20151016073738.GK3115@yliu-dev.sh.intel.com> References: <20151014000508-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151014000508-mutt-send-email-mst@redhat.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] dpdk/vhost-user and VM migration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Oct 2015 07:36:28 -0000 On Wed, Oct 14, 2015 at 12:16:29AM +0300, Michael S. Tsirkin wrote: > Hello! > I am currently looking at how using dpdk on host, accessing VM memory > using the vhost-user interface, interacts with VM migration. > > The issue is that any changes made to VM memory need to be tracked so > that updates can be sent from migration source to destination. > > At the moment, there's a proposal of an interface extension to > vhost-user which adds ability to do this tracking through shared memory. > dpdk would then be responsible for tracking these updates using atomic > operations to set bits (per page written) in a memory bitmap. > > This only needs to happen during migration, at other times there could > be a jump to skip this logging. > > Is this a reasonable approach? Hi Michael, As I stated in another email, adding dpdk/vhost-user vm migration support is my second TODO. However, I barely know anything about vm migration so far, that I can't tell now. I will re-visit this question when I finished my first item and after some more investigation. --yliu > Would performance degradation during > migration associated with atomics affect the performance to a level > where it's no longer useful? Pls note these logs aren't latency > sensitive, so can be done on a separate core, and can be batched. > > > One alternative I'm considering is extending linux kernel so it can do > this tracking automatically, by marking pages read-only, detecting a > pagefault and logging the write, then making the pages writeable. This > would mean higher worst-case overhead (pagefaults are expensive) but > lower average one (not extra code after the first fault). Not sure how > feasible this is yet, this would be harder to implement and it will only > be apply to newer host kernels. > > Any feedback would be appreciated. > > -- > MST