From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 25CF7A0555; Wed, 19 Feb 2020 17:20:20 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 20AF81B951; Wed, 19 Feb 2020 17:20:19 +0100 (CET) Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by dpdk.org (Postfix) with ESMTP id A0EC223D for ; Wed, 19 Feb 2020 17:20:17 +0100 (CET) Received: by mail-wm1-f46.google.com with SMTP id p17so1312513wma.1 for ; Wed, 19 Feb 2020 08:20:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xn0qvhtY6SEA1gRqvldlp3p/pm64ZzHvzt/ft61vTCg=; b=cosTIkQlOzGJLs1kLhoPihBoL8b2M9CqKyhPcWll79y4jJxBGbFeaIWbP1kWUYhz90 4n3hpTZBQFenXIWO8UEZbJZBPOmSXqDVxnc4qW36cwKWjHhl10itKA79NA+J3v1aVNsu zs6DPj5wiwF1+Kt+lQnm6RNmCJowUv/F41s2TCX0JMKl1Trgar6XMPCD82dq61wABmuQ jpxK+z/KJq4cdu4TbXME6DAZMB7iX1WwxLbis96uLBEnrMdzWgP/Zztqtfs0t04fCVEd NvPonBiYLmgAigR8Pxma3KNr6qnsKBo7YoGm/gDHCOfnQiS9DbCEP1Hs9NfqGV09H/hW j0Lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xn0qvhtY6SEA1gRqvldlp3p/pm64ZzHvzt/ft61vTCg=; b=ja6PUduFdgbN0dVh1DPK087Osexwhz5e058nRjloUdrxwbpDAmkiIrd5jakEYqF9bG ooQgd0Vtt76zBI7IkiTs113CFQF0ZnJOeIv7Yc2qEHned87xT92XBk604P2j3UdIsMEc 701iYjsPEm4hg3AI95yT6s1jKJq7Fg1hj6tRmZ/AP2jDzKpPwK2Q+b2H0wdtsQTvGVXG ryB7/frxLL7TYPA4OIfD82/szXmlGN5W2LL322NRY6Oy9xL2ZVyz7001pA88ZOCQ81JD 95QeUk8CUq+xQ47yGXoERnPHDPPTQqrpzCsyeJL3sfKiVr5QJhS2k0CBBp3boFbois01 6dzg== X-Gm-Message-State: APjAAAW8aAxn7kBb9mgcGU17M+MzRa/4SwBrNyV803LLHxfgLkPvIsPa ZG6Dw92ha/r6Do5Uyh5BH56mISQI23pLzq7gQic= X-Google-Smtp-Source: APXvYqwrDTfLlM7O+TBal0CGH8WwQIJ9n78a+2XR/ucTkHCLcQ4XAUD6kxLEveTqd19wBebsv0URpgKXHCwSip4zHVY= X-Received: by 2002:a1c:c209:: with SMTP id s9mr10044136wmf.156.1582129216993; Wed, 19 Feb 2020 08:20:16 -0800 (PST) MIME-Version: 1.0 References: <5192f94a-e50a-7e61-2e33-a218a4b6b5b4@intel.com> <9c888eb0-2192-137c-da5c-97f30da5204a@intel.com> <56527d9f-b079-a580-4a80-2207c1822260@intel.com> <0792d8b5-b0fd-15c8-f0ce-99a113c4c603@intel.com> In-Reply-To: <0792d8b5-b0fd-15c8-f0ce-99a113c4c603@intel.com> From: Kamaraj P Date: Wed, 19 Feb 2020 21:50:04 +0530 Message-ID: To: "Burakov, Anatoly" Cc: Kevin Traynor , dev@dpdk.org, Nageswara Rao Penumarthy , "Kamaraj P (kamp)" , mtang2@cisco.com Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] CONFIG_RTE_MAX_MEM_MB fails in DPDK18.05 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Anatoly, Yes we are facing an issue with our custom applications. Earlier we have tried with l2fwd DPDK application and does not see any issue with memory initialization. Not sure whether we missed any other options. BTW when we tried with l2fwd application, the application does not seem to hang during the memory initialization where as our application is kind of struck when getting memory. Do we tune any config parameters from DPDK ? Please advise. Please see the screenshot below for reference: ubuntu@client-server:~/dpdk/dpdk-stable/app/test-pmd/build/app$ sudo ./testpmd -l 0-3 -n 4 --log-level eal,8 -- -i --portmask=0x1 --nb-cores=2 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 0 on socket 0 EAL: Detected lcore 2 as core 0 on socket 0 EAL: Detected lcore 3 as core 0 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 4 lcore(s) EAL: Detected 1 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or directory) EAL: VFIO PCI modules not loaded EAL: DPAA Bus not present. Skipping. EAL: Probing VFIO support... EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) EAL: VFIO modules not loaded, skipping VFIO support... EAL: Ask a virtual area of 0x2e000 bytes EAL: Virtual area found at 0x100000000 (size = 0x2e000) EAL: Setting up physically contiguous memory... EAL: Setting maximum number of open files to 1048576 EAL: Detected memory type: socket_id:0 hugepage_sz:2097152 EAL: Creating 4 segment lists: n_segs:8192 socket_id:0 hugepage_sz:2097152 EAL: Ask a virtual area of 0x61000 bytes EAL: Virtual area found at 0x10002e000 (size = 0x61000) EAL: Memseg list allocated: 0x800kB at socket 0 EAL: Ask a virtual area of 0x400000000 bytes EAL: Virtual area found at 0x100200000 (size = 0x400000000) EAL: Ask a virtual area of 0x61000 bytes EAL: Virtual area found at 0x500200000 (size = 0x61000) EAL: Memseg list allocated: 0x800kB at socket 0 EAL: Ask a virtual area of 0x400000000 bytes EAL: Virtual area found at 0x500400000 (size = 0x400000000) EAL: Ask a virtual area of 0x61000 bytes EAL: Virtual area found at 0x900400000 (size = 0x61000) EAL: Memseg list allocated: 0x800kB at socket 0 EAL: Ask a virtual area of 0x400000000 bytes EAL: Virtual area found at 0x900600000 (size = 0x400000000) EAL: Ask a virtual area of 0x61000 bytes EAL: Virtual area found at 0xd00600000 (size = 0x61000) EAL: Memseg list allocated: 0x800kB at socket 0 EAL: Ask a virtual area of 0x400000000 bytes EAL: Virtual area found at 0xd00800000 (size = 0x400000000) EAL: TSC frequency is ~2094950 KHz EAL: Master lcore 0 is ready (tid=7f6507906c00;cpuset=[0]) EAL: lcore 1 is ready (tid=7f6505727700;cpuset=[1]) EAL: lcore 2 is ready (tid=7f6504f26700;cpuset=[2]) EAL: lcore 3 is ready (tid=7f6504725700;cpuset=[3]) EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 2MB EAL: PCI device 0000:03:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:0b:00.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15ad:7b0 net_vmxnet3 EAL: Not managed by a supported kernel driver, skipped EAL: Module /sys/module/vfio not found! error 2 (No such file or directory) testpmd: No probed ethernet devices EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 138MB Interactive-mode selected testpmd: create a new mbuf pool : n=171456, size=2176, socket=0 testpmd: preferred mempool ops selected: ring_mp_mc EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was expanded by 4MB EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: alloc_seg(): mmap() failed: Cannot allocate memory EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x110200000 (size = 0x200000) EAL: attempted to allocate 194 segments, but only 56 were allocated EAL: Restoring previous memory policy: 0 EAL: Trying to obtain current memory policy. EAL: Setting policy MPOL_PREFERRED for socket 0 EAL: alloc_seg(): mmap() failed: Cannot allocate memory EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x110200000 (size = 0x200000) EAL: attempted to allocate 195 segments, but only 56 were allocated EAL: Restoring previous memory policy: 0 EAL: request: mp_malloc_sync EAL: Heap on socket 0 was shrunk by 4MB EAL: Error - exiting with code: 1 Cause: Creation of mbuf pool for socket 0 failed: Cannot allocate memory Thanks, Kamaraj On Wed, Feb 19, 2020 at 9:30 PM Burakov, Anatoly wrote: > On 19-Feb-20 3:42 PM, Kamaraj P wrote: > > Hi Anatoly, > > Thanks for the suggestions. Yeah we have just changed in our application > > to invoke mlockall() before rte_eal_init(). Looks like it does not help > > either. > > > > if (mlockall(MCL_CURRENT | MCL_FUTURE)) { > > printf("Failed mlockall !! ******\n"); > > } > > ret = rte_eal_init(argc, args); > > > > Looks like still observing the struck issue when allocating virtual > pages. > > EAL: Detected lcore 0 as core 0 on socket 0 > > EAL: Support maximum 128 logical core(s) by configuration. > > EAL: Detected 1 lcore(s) > > EAL: Detected 1 NUMA nodes > > EAL: open shared lib /usr/lib64/librte_pmd_ixgbe.so.2.1 > > EAL: open shared lib /usr/lib64/librte_pmd_e1000.so.1.1 > > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > > EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or > > directory) > > EAL: VFIO PCI modules not loaded > > EAL: Probing VFIO support... > > EAL: Module /sys/module/vfio not found! error 2 (No such file or > directory) > > EAL: VFIO modules not loaded, skipping VFIO support... > > EAL: Ask a virtual area of 0x2e000 bytes > > EAL: Virtual area found at 0x100000000 (size = 0x2e000) > > EAL: Setting up physically contiguous memory... > > EAL: Setting maximum number of open files to 4096 > > EAL: Detected memory type: socket_id:0 hugepage_sz:2097152 > > EAL: Creating 32 segment lists: n_segs:1024 socket_id:0 > hugepage_sz:2097152 > > EAL: Ask a virtual area of 0xd000 bytes > > EAL: Virtual area found at 0x10002e000 (size = 0xd000) > > EAL: Memseg list allocated: 0x800kB at socket 0 > > EAL: Ask a virtual area of 0x80000000 bytes > > > > Could you please suggest if there is any other option which we need to > > try it out. > Does this only happen with your application, or does it happen with DPDK > example applications or test/testpmd apps? > > -- > Thanks, > Anatoly >