Re: [dpdk-dev] dpdk/vhost-user and VM migration

[Yuanhan Liu <yuanhan.liu@linux.intel.com>]

On Wed, Oct 28, 2015 at 05:52:44AM -0400, Amnon Ilan wrote:
> 
> 
> ----- Original Message -----
> > From: "Yuanhan Liu" <yuanhan.liu@linux.intel.com>
> > To: "Michael S. Tsirkin" <mst@redhat.com>
> > Cc: dev@dpdk.org
> > Sent: Friday, October 16, 2015 10:37:38 AM
> > Subject: Re: [dpdk-dev] dpdk/vhost-user and VM migration
> > 
> > On Wed, Oct 14, 2015 at 12:16:29AM +0300, Michael S. Tsirkin wrote:
> > > Hello!
> > > I am currently looking at how using dpdk on host, accessing VM memory
> > > using the vhost-user interface, interacts with VM migration.
> > > 
> > > The issue is that any changes made to VM memory need to be tracked so
> > > that updates can be sent from migration source to destination.
> > > 
> > > At the moment, there's a proposal of an interface extension to
> > > vhost-user which adds ability to do this tracking through shared memory.
> > > dpdk would then be responsible for tracking these updates using atomic
> > > operations to set bits (per page written) in a memory bitmap.
> > > 
> > > This only needs to happen during migration, at other times there could
> > > be a jump to skip this logging.
> > > 
> > > Is this a reasonable approach?
> > 
> > Hi Michael,
> > 
> > As I stated in another email, adding dpdk/vhost-user vm migration
> > support is my second TODO. However, I barely know anything about
> > vm migration so far, that I can't tell now.
> > 
> > I will re-visit this question when I finished my first item and
> > after some more investigation.
> 
> Yuanhan, 
> 
> Live-migration for vhost-user is now available upstream.

Hi Ammon,

Yes, I'm aware of that.

> Do you need some guidance on how to implement it in DPDK?

Appreciate a lot for offering the help!  However, I haven't started
it yet, and I should be able to start it in two or three weeks if
everything goes well here.  I may ask your help then if I am in
trouble.

	--yliu

> > > Would performance degradation during
> > > migration associated with atomics affect the performance to a level
> > > where it's no longer useful?  Pls note these logs aren't latency
> > > sensitive, so can be done on a separate core, and can be batched.
> > > 
> > > 
> > > One alternative I'm considering is extending linux kernel so it can do
> > > this tracking automatically, by marking pages read-only, detecting a
> > > pagefault and logging the write, then making the pages writeable.  This
> > > would mean higher worst-case overhead (pagefaults are expensive) but
> > > lower average one (not extra code after the first fault).  Not sure how
> > > feasible this is yet, this would be harder to implement and it will only
> > > be apply to newer host kernels.
> > > 
> > > Any feedback would be appreciated.
> > > 
> > > --
> > > MST
> >