From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ua0-f172.google.com (mail-ua0-f172.google.com [209.85.217.172]) by dpdk.org (Postfix) with ESMTP id CD42D4C57 for ; Thu, 22 Jun 2017 11:24:10 +0200 (CEST) Received: by mail-ua0-f172.google.com with SMTP id j53so10757170uaa.2 for ; Thu, 22 Jun 2017 02:24:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=g/fcZ8F86Pfsa5YMLrWCRTMwunjqoSBID0kZVYKckg4=; b=rTyLjnOMVLe5UWEljn5rvjznEmAPck8KdXvuQ3JNK/DAo69z/pzeCAbVxByXCq1S4y 8QBV4GCyQjVdQ4+J05KpgCtzS3wo6EnuGo3GwFg10sqoPGgsnbnLlh1xQ6qOa5pjscJ0 eZbUVKbj7/oa9pnOaCRjXsgtrPZkImGagw2gGoh9UER+TLSFNGaTacvYWQIhXFrosjF1 bQ4OjdzKccCeT+ks+mgt3LTCE2MIhgYA4rVK0M1drFf2bH9TekT/aCOw5efwNcIObyxf ZM2YYjmhkHVDz/ABRX2ekOlTETyFGvN/uNwuX2Ue3kg8Q5GeltzsTSwjB2BBQIsH2+F3 P9IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=g/fcZ8F86Pfsa5YMLrWCRTMwunjqoSBID0kZVYKckg4=; b=isXOUCKSBSaZRuBqmnafs5u83VjhNEAXtu+QPVVY3rOf8hN3sCoUYSWVhF86yKXPtN 4P4DqzNsndbtCPkBOJOi4ZQ+yNV3vBtWQtqbDALtTpTlK1n2gs6pOwrcFAhaAGu4bwaM k4KRMi7V8YIKTUioOo8Hm3TvrrKRcoJlLrNb9e2c+zZ+kNB1q3K8qQIw5LVpgdrP+NY0 +cKoG0DuTfHyUbgj6ezbDNvUdoFB8KH3Bn+pa8Qi4PfpvfP5Shio3oQHIJZddnmlhsi9 99SldjZ8VopKvZDLky74PGBMj+5y2u7uibgyibIIdxjKanWklwccg6uS2aEUeyfr35b7 2CfQ== X-Gm-Message-State: AKS2vOxeY8GivkM0ZX48C6ywl/OwwpnJ3BU5qa3wQW6p1vGiT3gxzK9N 1o2Rb1z0gwQHrj8r1wrNBd6WHxZ67jEc X-Received: by 10.176.18.200 with SMTP id o8mr1574305uac.132.1498123450139; Thu, 22 Jun 2017 02:24:10 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.131.10 with HTTP; Thu, 22 Jun 2017 02:24:09 -0700 (PDT) In-Reply-To: References: From: Alejandro Lucero Date: Thu, 22 Jun 2017 10:24:09 +0100 Message-ID: To: Sam Cc: Pavel Shirshov , dev , users@dpdk.org, qemu-devel@nongnu.org, qemu-discuss@nongnu.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] [dpdk-dev] Will huge page have negative effect on guest vm in qemu enviroment? X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Jun 2017 09:24:11 -0000 On Wed, Jun 21, 2017 at 8:22 AM, Sam wrote: > Thank you~ > > 1. We have a compare test on qemu-kvm enviroment with huge page and witho= ut > huge page. Qemu start process is much longer in huge page enviromwnt. And= I > write an email titled with '[DPDK-memory] how qemu waste such long time > under dpdk huge page envriment?'. I could resend it later. > > DPDK vhost code does some work when configuring the VM virtio port. If dequeue_zero_copy is set, the work to do is heavier, but I do not think this could imply huge longer bootup times. > 2. Then I have another test on qemu-kvm enviroment with huge page and > without huge page, which I didn't start ovs-dpdk and vhostuser port in qe= mu > start process. And I found Qemu start process is also much longer in huge > page enviroment. > If hugepages are available, starting a qemu VM with hugepages should not take too long. So in this case, I would say the hugepages need to be allocated before the VM can boot. The only reason I can think of is transparent hugepages in use, then the system needs to work for undoing those transparent hugepages for allocating space to the VM hugepages. Did you check how many hugepages are available just before starting the QEMU VM? cat /sys/kernel/mm/hugepages/hugepages-*/free_hugepages If this is the problem, easy to solve just disabling transparent hugepages: echo never > /sys/kernel/mm/transparent_hugepage/enabled > So I think huge page enviroment, which grub2.cfg file is specified in > =E2=80=98[DPDK-memory] > how qemu waste such long time under dpdk huge page envriment?=E2=80=99, w= ill really > have negative effect on qemu start up process. > > That's why we don't like to use ovs-dpdk. Althrough ovs-dpdk is faster, b= ut > the start up process of qemu is much longer then normal ovs, and the reas= on > is nothing with ovs but huge page. For customers, vm start up time is > important then network speed. > > BTW, ovs-dpdk start up process is also longer then normal ovs. But I know > the reason, it's dpdk EAL init process with forking big continous memory > and zero this memory. For qemu, I don't know why, as there is no log to > report this. > We had also problems with DPDK apps initialization with a server using 256GB. I guess this is something that maybe could be improved. > > 2017-06-21 14:15 GMT+08:00 Pavel Shirshov : > > > Hi Sam, > > > > Below I'm saying about KVM. I don't have experience with vbox and other= s. > > 1. I'd suggest don't use dpdk inside of VM if you want to see best > > perfomance on the box. > > 2. huge pages enabled globally will not have any bad effect to guest > > OS. Except you have to enable huge pages inside of VM and provide real > > huge page for VM's huge pages from the host system. Otherwise dpdk > > will use "hugepages" inside of VM, but this "huge pages" will not real > > ones. They will be constructed from normal pages outside. Also when > > you enable huge pages OS will reserve them from start and your OS will > > not able use them for other things. Also you can't swap out huge > > pages, KSM will not work for them and so on. > > 3. You can enable huge pages just for one numa node. It's impossible > > to enable them just for one core. Usually you reserve some memory for > > hugepages when the system starts and you can't use this memory in > > normal applications unless normal application knows how to use them. > > > > Also why it didn't work inside of the docker? > > > > > > On Tue, Jun 20, 2017 at 8:35 PM, Sam wrote: > > > BTW, we also think about use ovs-dpdk in docker enviroment, but test > > result > > > said it's not good idea, we don't know why. > > > > > > 2017-06-21 11:32 GMT+08:00 Sam : > > > > > >> Hi all, > > >> > > >> We plan to use DPDK on HP host machine with several core and big > memory. > > >> We plan to use qemu-kvm enviroment. The host will carry 4 or more > guest > > vm > > >> and 1 ovs. > > >> > > >> Ovs-dpdk is much faster then normal ovs, but to use ovs-dpdk, we hav= e > to > > >> enable huge page globally. > > >> > > >> My question is, will huge page enabled globally have negative effect > on > > >> guest vm's memory orperate or something? If it is, how to prevent > this, > > or > > >> could I enable huge page on some core or enable huge page for a part > of > > >> memory? > > >> > > >> Thank you~ > > >> > > >