From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-f193.google.com (mail-qt0-f193.google.com [209.85.216.193]) by dpdk.org (Postfix) with ESMTP id F12579E3; Mon, 19 Jun 2017 05:12:37 +0200 (CEST) Received: by mail-qt0-f193.google.com with SMTP id o21so18001211qtb.1; Sun, 18 Jun 2017 20:12:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=T1bggNjz0t/YAOhPlR7XnDFYAcdASm8Lu2lnskGPtMU=; b=LZJqtOBHYahwiKhUvGl2Gbn6r+sv3YaAsGolGVxzsCd4TOBg9kSyLle07uzLfo3zjl Sx46oKkKG0L9vgR0pTKu5miiJL/v/KtWCvH/7KRZx3yGtncHyX0o77g28DtK7Ta07EMz +iqJEX+OD+sqa2sRY1oYhzFQ74tjd5zUadEqIOEUz06FhmnSFCbZqVJHyVuUZcHTMrJC yxMVGFFcmprLoLTiZdQJhhmgB9pXV5CLufnKKm8p4QKxYVW7grUjV5cIpQdnWLR5S+A3 iclm1MOXNesOSIXAzEVjwpjR/VzF4+zrD3nzn1jInunkXJnX74IlhsBe6QGeWaoIOoQc UFvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=T1bggNjz0t/YAOhPlR7XnDFYAcdASm8Lu2lnskGPtMU=; b=tM/5xFF7N5vQDFTFuyyudf5x+iGmPZTWvdRA1no+MkMmCsHXcRCnsdUQv67jgS09hA 5HacazQKFdkHtdwJddSbB2qv5RUpPGR6Ou1rKzAnqYZzzVYOV9qf0J8de+9zgbXSvziG HpXPXg6ktjOdqrR/3B0WiVx0eiQaxVR0p8mqSMUSqHZ/2he00SFxKybpKhyq+L3dD92S 565bVtIBfFGE7ZlG1R0+fwG7wSl4xuFSTl9QWSKKvXCBa80Lvym+2EIitA0hJU1tBgm5 wgkHRNVnHnpL9Sq1JGrZGiI166KZFV5J4nCjOovlaRlzq9QS8Mm/2srgQS8LGda7Rm88 TbRA== X-Gm-Message-State: AKS2vOwL00F9FUxpQzcNlBscghhIAZi5M0/QdrDQXsx+eqqFSM1CH9QS ZBNmohgLRMBDxPJ2YeTzGsMvw8tJ4EmF X-Received: by 10.237.38.133 with SMTP id q5mr24276587qtd.152.1497841957331; Sun, 18 Jun 2017 20:12:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.90.72 with HTTP; Sun, 18 Jun 2017 20:12:36 -0700 (PDT) In-Reply-To: <20170616085935.GC82628@bricha3-MOBL3.ger.corp.intel.com> References: <20170616085935.GC82628@bricha3-MOBL3.ger.corp.intel.com> From: Sam Date: Mon, 19 Jun 2017 11:12:36 +0800 Message-ID: To: Bruce Richardson Cc: dev@dpdk.org, users@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [DPDK-memory] how qemu waste such long time under dpdk huge page envriment? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2017 03:12:38 -0000 Hi, I print all system call by `strace -f -T -tt -e trace=all -o output.txt $QEMU_CMD`, and I found this: 5900 11:08:11.288701 nanosleep({0, 10000000}, 0x7ff103c13a80) = 0 > <0.010171> > 5900 11:08:11.299052 futex(0x7ff10be24340, FUTEX_WAIT_PRIVATE, 2, NULL > > 5899 11:08:20.138492 rt_sigaction(SIGBUS, {0x7ff10b5b1040, [], > SA_RESTORER|SA_SIGINFO, 0x7ff1085fd100}, NULL, 8) = 0 <0.000012> > 5899 11:08:20.138598 rt_sigprocmask(SIG_SETMASK, [BUS USR1 ALRM IO], > NULL, 8) = 0 <0.000008> > 5899 11:08:20.138646 mmap(NULL, 266240, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff10b380000 <0.000013> > 5899 11:08:20.138793 mmap(NULL, 266240, PROT_READ|PROT_WRITE, > MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff10b33f000 <0.000009> I don't know who call `futex`, and why waste such long time. And for `mmap` system call, I don't found this system call waste too much time. WHY.... 2017-06-16 16:59 GMT+08:00 Bruce Richardson : > On Fri, Jun 16, 2017 at 04:26:40PM +0800, Sam wrote: > > BTW, while running ovs-dpdk, this log is also take long time, does that > > mean dpdk request large memory take long time? > > > > EAL: Setting up physically contiguous memory... > > > > When running with 1G pages, I found that the mmap system call takes a > considerable amount of time to execute. I think this is due to the > kernel zero-ing out the 1G pages. IIRC on one system I measured it as > taking > about 0.4 seconds per 1G page. > > /Bruce > > > > > 2017-06-16 16:11 GMT+08:00 Sam : > > > > > Hi all, > > > > > > I'm running `QEMU_CMD ...` to create a vm under dpdk huge page > envriment > > > (which set huge page 1G). And I enable all events in qemu. > > > > > > For qemu and ovs-dpdk(ovs-2.4.9 with dpdk-2.2.0) environment, detail > log > > > is: > > > > > > > 30012@1497443246.678304:object_dynamic_cast_assert > > > qemu:memory-region->qemu:memory-region (/home/hu > > > > anghuai/cloud/contrib/qemu-2.6.0/memory.c:1076:memory_region_initfn) > > > > 30012@1497443256.274866:object_dynamic_cast_assert > > > qio-channel-socket->qio-channel-socket (io/chann > > > > el-socket.c:389:qio_channel_socket_init) > > > > > > > > > I don't know why qemu doing 'memory_region_initfn' function in this 10 > > > second, does anyone know this? > > > > > > static void memory_region_initfn(Object *obj) > > >> { > > >> MemoryRegion *mr = MEMORY_REGION(obj); > > >> ObjectProperty *op; > > >> mr->ops = &unassigned_mem_ops; > > >> mr->enabled = true; > > >> mr->romd_mode = true; > > >> mr->global_locking = true; > > >> mr->destructor = memory_region_destructor_none; > > >> QTAILQ_INIT(&mr->subregions); > > >> QTAILQ_INIT(&mr->coalesced); > > >> op = object_property_add(OBJECT(mr), "container", > > >> "link<" TYPE_MEMORY_REGION ">", > > >> memory_region_get_container, > > >> NULL, /* memory_region_set_container */ > > >> NULL, NULL, &error_abort); > > >> op->resolve = memory_region_resolve_container; > > >> object_property_add(OBJECT(mr), "addr", "uint64", > > >> memory_region_get_addr, > > >> NULL, /* memory_region_set_addr */ > > >> NULL, NULL, &error_abort); > > >> object_property_add(OBJECT(mr), "priority", "uint32", > > >> memory_region_get_priority, > > >> NULL, /* memory_region_set_priority */ > > >> NULL, NULL, &error_abort); > > >> object_property_add_bool(OBJECT(mr), "may-overlap", > > >> memory_region_get_may_overlap, > > >> NULL, /* memory_region_set_may_overlap */ > > >> &error_abort); > > >> object_property_add(OBJECT(mr), "size", "uint64", > > >> memory_region_get_size, > > >> NULL, /* memory_region_set_size, */ > > >> NULL, NULL, &error_abort); > > >> } > > > > > > > > > Thank you~ > > > >