From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f196.google.com (mail-qk0-f196.google.com [209.85.220.196]) by dpdk.org (Postfix) with ESMTP id D4754378B; Mon, 19 Jun 2017 05:13:04 +0200 (CEST) Received: by mail-qk0-f196.google.com with SMTP id r62so5368366qkf.3; Sun, 18 Jun 2017 20:13:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Q7+d93EhtsjF5USmIK2OzUCXZi/xnKOUq0F+j/CcpR4=; b=MBl/bclXbUFAfMevAZPCbbEpQR7OUxWhZPQ16ek1l0noUVZS+4klT3NG5R5uRrkpCw LOaVyM7/ZpY5d1jtIaV+tugAL51YLMIHTbFKvhxsC3b9RiBam3Eq2zE4QR90aAcxgI19 R64Ueb6TpzlGsysRWVmx5ln4Zl30xjMcZpObdWrjLJ+yRt8R9wjDg56pCqz6D97kJQxy QdrZiP10MCPHXVtcBoltN9Mg3rpC7FJwoyMhcHIKXBrWwqCw1gq2ZRwFqus68vXCgo0z 9vZCYlAhOPb+3UHuy6IxCWmbK/JPPmeDi8p1i+adQVM0Hc/+fR4KAdIqsyA4+tuyvbSY NmSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Q7+d93EhtsjF5USmIK2OzUCXZi/xnKOUq0F+j/CcpR4=; b=s7hbgm+MH/KvjMDdwELba6W14EghYAYYa3na3x4oCa8VYtvMCGyLVoJbGhCI4/ECaD iP3Vvc0a/detPk339yD0yal1bNBd3pN8UBgtQ4kRgbN/lXeAdJ8+PG2J+x8dpycLjAvM LFnVfbve+tTBEwVPVX6j3HzpwY5EuRxLVJ27iiRAe3Psj0YBUyOEDvHINeWgju17Qzd4 f5nBU1eW8f7A1Iyvsq1a7WvvpiSGcOzxbR/S5Eq+HawG6iuDeURmWmlpl1reQjkBvR4z tH+GPbDT7ag6e9Yedk4hSZMjMvLCjswo84/uMLQBWQyFpaaefBPM5uZ36WTYNqqPwUv2 qQ6A== X-Gm-Message-State: AKS2vOzGCuy911bQXSEiy2sEB7+s1m+VljUz5Tb94MrchJgik+GtYbU2 MnFRZW3BU/hm6D9QnLdYrz2T3MXe5g== X-Received: by 10.55.214.78 with SMTP id t75mr23667388qki.239.1497841984310; Sun, 18 Jun 2017 20:13:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.90.72 with HTTP; Sun, 18 Jun 2017 20:13:03 -0700 (PDT) In-Reply-To: References: <20170616085935.GC82628@bricha3-MOBL3.ger.corp.intel.com> From: Sam Date: Mon, 19 Jun 2017 11:13:03 +0800 Message-ID: To: Bruce Richardson Cc: dev@dpdk.org, users@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [DPDK-memory] how qemu waste such long time under dpdk huge page envriment? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jun 2017 03:13:06 -0000 Additional, this is DPDK-QEMU enviroment. 2017-06-19 11:12 GMT+08:00 Sam : > Hi, > > I print all system call by `strace -f -T -tt -e trace=all -o output.txt > $QEMU_CMD`, and I found this: > > 5900 11:08:11.288701 nanosleep({0, 10000000}, 0x7ff103c13a80) = 0 >> <0.010171> >> 5900 11:08:11.299052 futex(0x7ff10be24340, FUTEX_WAIT_PRIVATE, 2, NULL >> >> 5899 11:08:20.138492 rt_sigaction(SIGBUS, {0x7ff10b5b1040, [], >> SA_RESTORER|SA_SIGINFO, 0x7ff1085fd100}, NULL, 8) = 0 <0.000012> >> 5899 11:08:20.138598 rt_sigprocmask(SIG_SETMASK, [BUS USR1 ALRM IO], >> NULL, 8) = 0 <0.000008> >> 5899 11:08:20.138646 mmap(NULL, 266240, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff10b380000 <0.000013> >> 5899 11:08:20.138793 mmap(NULL, 266240, PROT_READ|PROT_WRITE, >> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff10b33f000 <0.000009> > > > I don't know who call `futex`, and why waste such long time. And for > `mmap` system call, I don't found this system call waste too much time. > > WHY.... > > > 2017-06-16 16:59 GMT+08:00 Bruce Richardson : > >> On Fri, Jun 16, 2017 at 04:26:40PM +0800, Sam wrote: >> > BTW, while running ovs-dpdk, this log is also take long time, does that >> > mean dpdk request large memory take long time? >> > >> > EAL: Setting up physically contiguous memory... >> > >> >> When running with 1G pages, I found that the mmap system call takes a >> considerable amount of time to execute. I think this is due to the >> kernel zero-ing out the 1G pages. IIRC on one system I measured it as >> taking >> about 0.4 seconds per 1G page. >> >> /Bruce >> >> > >> > 2017-06-16 16:11 GMT+08:00 Sam : >> > >> > > Hi all, >> > > >> > > I'm running `QEMU_CMD ...` to create a vm under dpdk huge page >> envriment >> > > (which set huge page 1G). And I enable all events in qemu. >> > > >> > > For qemu and ovs-dpdk(ovs-2.4.9 with dpdk-2.2.0) environment, detail >> log >> > > is: >> > > >> > > > 30012@1497443246.678304:object_dynamic_cast_assert >> > > qemu:memory-region->qemu:memory-region (/home/hu >> > > > anghuai/cloud/contrib/qemu-2.6.0/memory.c:1076:memory_region >> _initfn) >> > > > 30012@1497443256.274866:object_dynamic_cast_assert >> > > qio-channel-socket->qio-channel-socket (io/chann >> > > > el-socket.c:389:qio_channel_socket_init) >> > > >> > > >> > > I don't know why qemu doing 'memory_region_initfn' function in this 10 >> > > second, does anyone know this? >> > > >> > > static void memory_region_initfn(Object *obj) >> > >> { >> > >> MemoryRegion *mr = MEMORY_REGION(obj); >> > >> ObjectProperty *op; >> > >> mr->ops = &unassigned_mem_ops; >> > >> mr->enabled = true; >> > >> mr->romd_mode = true; >> > >> mr->global_locking = true; >> > >> mr->destructor = memory_region_destructor_none; >> > >> QTAILQ_INIT(&mr->subregions); >> > >> QTAILQ_INIT(&mr->coalesced); >> > >> op = object_property_add(OBJECT(mr), "container", >> > >> "link<" TYPE_MEMORY_REGION ">", >> > >> memory_region_get_container, >> > >> NULL, /* memory_region_set_container */ >> > >> NULL, NULL, &error_abort); >> > >> op->resolve = memory_region_resolve_container; >> > >> object_property_add(OBJECT(mr), "addr", "uint64", >> > >> memory_region_get_addr, >> > >> NULL, /* memory_region_set_addr */ >> > >> NULL, NULL, &error_abort); >> > >> object_property_add(OBJECT(mr), "priority", "uint32", >> > >> memory_region_get_priority, >> > >> NULL, /* memory_region_set_priority */ >> > >> NULL, NULL, &error_abort); >> > >> object_property_add_bool(OBJECT(mr), "may-overlap", >> > >> memory_region_get_may_overlap, >> > >> NULL, /* memory_region_set_may_overlap >> */ >> > >> &error_abort); >> > >> object_property_add(OBJECT(mr), "size", "uint64", >> > >> memory_region_get_size, >> > >> NULL, /* memory_region_set_size, */ >> > >> NULL, NULL, &error_abort); >> > >> } >> > > >> > > >> > > Thank you~ >> > > >> > >