From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <gaetan.rivet@6wind.com>
Received: from mail-wr0-f182.google.com (mail-wr0-f182.google.com
 [209.85.128.182]) by dpdk.org (Postfix) with ESMTP id 3315C20F
 for <users@dpdk.org>; Wed, 21 Jun 2017 11:03:21 +0200 (CEST)
Received: by mail-wr0-f182.google.com with SMTP id 77so125685118wrb.1
 for <users@dpdk.org>; Wed, 21 Jun 2017 02:03:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=6wind-com.20150623.gappssmtp.com; s=20150623;
 h=date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:content-transfer-encoding:in-reply-to
 :user-agent; bh=fVbut2TJlaRk+zbO4OTtCWnaPk5eoT0XRhnZctK/tEo=;
 b=zhucUrJhAjRCtbFlcCr6DHVwWuMIpfKnkOsFxLaJXDbmL+o3n08+jD4/LWNX/nls+h
 7nsYSj8G9S77ZYACCbhDGdZB+9dZHGjp6TPcS8FE7U2Dj8tfxd99gOTp+jbUdveFO0J+
 av+c6Nfjt8Atib9lEdm99tlv1TlW1CLV78OLnK+5lFT2H7VCO3kJUs5t2zu9HnMWdA6K
 8+dH3fAttsvZYkDNB5lHSqPP0Zu46Jj9OH1llgV3XCZT+St66USjpntgZ1NpLcT0kX1/
 XcLL3l2cMjp+mfebjSRtpD+YIUP7Sc8O4t0pHFitDH2kUkJTQLqKMAjBM/vsczwDSHIv
 +Wxw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:references
 :mime-version:content-disposition:content-transfer-encoding
 :in-reply-to:user-agent;
 bh=fVbut2TJlaRk+zbO4OTtCWnaPk5eoT0XRhnZctK/tEo=;
 b=rVN2/6NNCDje9PsKSNpXm4fM0KL0mJDtksZRp6y1bWJai4S3Kq2ykABMRv5xHCH59G
 sKcXoR5ZdFGEe3jYVUelm2aq3boLK3cDdgI+Dc0nMcLk4yflzDgbwI0gPyeeErHuAPKy
 32MbzfBKwjQEUGsLFmealsau6qDkV7+CykwV3FgdjerffoI1HQChVgcfRGhGUeDAawoP
 x1tQDCvLpDUzyN2JvfN4Vu517BU62qsBB4wiFgLGjccYFjy0OqNF8B/RlYYuBEIQ3pkZ
 TP33UiWyv3qPtiljtpYin0oGP+929SkQGj+Ng6NWLLTeHLdgdRLG+x2Kp/aFoIMilXPA
 Zqsw==
X-Gm-Message-State: AKS2vOzZCvvTs/OxmoaQuC+E0zWv3l/1SHReHlIJfyOC1O4IAD6PwMKj
 saGYrjRcHn8HDHCz
X-Received: by 10.223.144.201 with SMTP id i67mr20758739wri.90.1498035800645; 
 Wed, 21 Jun 2017 02:03:20 -0700 (PDT)
Received: from bidouze.vm.6wind.com (host.78.145.23.62.rev.coltfrance.com.
 [62.23.145.78])
 by smtp.gmail.com with ESMTPSA id z99sm19304359wrc.12.2017.06.21.02.03.19
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Wed, 21 Jun 2017 02:03:19 -0700 (PDT)
Date: Wed, 21 Jun 2017 11:03:08 +0200
From: =?iso-8859-1?Q?Ga=EBtan?= Rivet <gaetan.rivet@6wind.com>
To: Sam <batmanustc@gmail.com>
Cc: Pavel Shirshov <pavel.shirshov@gmail.com>, dev@dpdk.org, users@dpdk.org
Message-ID: <20170621090239.GA2344@bidouze.vm.6wind.com>
References: <CAOE=1Z1Bbgb494m6mYRnX1-80eT0tSnF0F6zNM7efEie4rd7DA@mail.gmail.com>
 <CAOE=1Z1LJr9rpaEQ9+b7D5vM3N=Wy0oGunRnzoQ79x3dWA3oaw@mail.gmail.com>
 <CALkUq2gp+x-nkALLTLqrSGmw8nfo5FzNeDpr1Gm3qh8N9mDn0Q@mail.gmail.com>
 <CAOE=1Z0h+Dd=T8QKyVnTHwFjeYwm-qz_Da7HWEDk=pcrMbp9ug@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAOE=1Z0h+Dd=T8QKyVnTHwFjeYwm-qz_Da7HWEDk=pcrMbp9ug@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Subject: Re: [dpdk-users] [dpdk-dev] Will huge page have negative effect on
 guest vm in qemu enviroment?
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/users>,
 <mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/users>,
 <mailto:users-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Wed, 21 Jun 2017 09:03:21 -0000

Hi Sam,

On Wed, Jun 21, 2017 at 03:22:45PM +0800, Sam wrote:
> Thank you~
> 
> 1. We have a compare test on qemu-kvm enviroment with huge page and without
> huge page. Qemu start process is much longer in huge page enviromwnt. And I
> write an email titled with '[DPDK-memory] how qemu waste such long time
> under dpdk huge page envriment?'. I could resend it later.
> 

Are you using 2M hugepages? Do you see any difference with 1G hugepages?
The smaller ones should not incur such delay.

On a side note, if the VM is properly configured the performance loss
should be negligible, and mostly visible in benchmark contexts with
little or no treatment done on the packets.

> 2. Then I have another test on qemu-kvm enviroment with huge page and
> without huge page, which I didn't start ovs-dpdk and vhostuser port in qemu
> start process. And I found Qemu start process is also much longer in huge
> page enviroment.
> 
> So I think huge page enviroment, which grub2.cfg file is specified in
> ‘[DPDK-memory]
> how qemu waste such long time under dpdk huge page envriment?’, will really
> have negative effect on qemu start up process.
> 
> That's why we don't like to use ovs-dpdk. Althrough ovs-dpdk is faster, but
> the start up process of qemu is much longer then normal ovs, and the reason
> is nothing with ovs but huge page. For customers, vm start up time is
> important then network speed.
> 
> BTW, ovs-dpdk start up process is also longer then normal ovs. But I know
> the reason, it's dpdk EAL init process with forking big continous memory
> and zero this memory. For qemu, I don't know why, as there is no log to
> report this.
> 
> 2017-06-21 14:15 GMT+08:00 Pavel Shirshov <pavel.shirshov@gmail.com>:
> 
> > Hi Sam,
> >
> > Below I'm saying about KVM. I don't have experience with vbox and others.
> > 1. I'd suggest don't use dpdk inside of VM if you want to see best
> > perfomance on the box.
> > 2. huge pages enabled globally will not have any bad effect to guest
> > OS. Except you have to enable huge pages inside of VM and provide real
> > huge page for VM's huge pages from the host system. Otherwise dpdk
> > will use "hugepages" inside of VM, but this "huge pages" will not real
> > ones. They will be constructed from normal pages outside. Also when
> > you enable huge pages OS will reserve them from start and your OS will
> > not able use them for other things. Also you can't swap out huge
> > pages, KSM will not work for them and so on.
> > 3. You can enable huge pages just for one numa node. It's impossible
> > to enable them just for one core. Usually you reserve some memory for
> > hugepages when the system starts and you can't use this memory in
> > normal applications unless normal application knows how to use them.
> >
> > Also why it didn't work inside of the docker?
> >
> >
> > On Tue, Jun 20, 2017 at 8:35 PM, Sam <batmanustc@gmail.com> wrote:
> > > BTW, we also think about use ovs-dpdk in docker enviroment, but test
> > result
> > > said it's not good idea, we don't know why.
> > >
> > > 2017-06-21 11:32 GMT+08:00 Sam <batmanustc@gmail.com>:
> > >
> > >> Hi all,
> > >>
> > >> We plan to use DPDK on HP host machine with several core and big memory.
> > >> We plan to use qemu-kvm enviroment. The host will carry 4 or more guest
> > vm
> > >> and 1 ovs.
> > >>
> > >> Ovs-dpdk is much faster then normal ovs, but to use ovs-dpdk, we have to
> > >> enable huge page globally.
> > >>
> > >> My question is, will huge page enabled globally have negative effect on
> > >> guest vm's memory orperate or something? If it is, how to prevent this,
> > or
> > >> could I enable huge page on some core or enable huge page for a part of
> > >> memory?
> > >>
> > >> Thank you~
> > >>
> >

-- 
Gaëtan Rivet
6WIND