From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5DC32A052B; Wed, 29 Jul 2020 11:08:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 53EBF1BFF5; Wed, 29 Jul 2020 11:08:04 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 0239510A3 for ; Wed, 29 Jul 2020 11:08:02 +0200 (CEST) IronPort-SDR: 3kNLXj+DbKC4al2b9SRcTF5KOKLLFAOXTBrl3nROwOE1fEg2Pj8xMbc6d26ovQnh8V9PgdRTRE oO+oVe3xwMig== X-IronPort-AV: E=McAfee;i="6000,8403,9696"; a="152619393" X-IronPort-AV: E=Sophos;i="5.75,409,1589266800"; d="scan'208";a="152619393" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2020 02:08:01 -0700 IronPort-SDR: 64OkgTtdoxfptNVAu6plKkrtU+dgqEA1EmdkoPJ3NCHRqTbVxqpLjSl1AgqNsu08pZgUh7/4My nAzbVXEyk/mg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,409,1589266800"; d="scan'208";a="490226557" Received: from jhsinnig-mobl1.amr.corp.intel.com (HELO [10.213.196.51]) ([10.213.196.51]) by fmsmga006.fm.intel.com with ESMTP; 29 Jul 2020 02:07:59 -0700 To: David Marchand , Balaji K N Cc: "dev@dpdk.org" References: From: "Burakov, Anatoly" Message-ID: <27b57cde-a02a-a79c-b115-d173e09f0d79@intel.com> Date: Wed, 29 Jul 2020 10:07:59 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] long initialization time for EAL init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 29-Jul-20 9:52 AM, David Marchand wrote: > On Wed, Jul 29, 2020 at 10:51 AM Balaji K N wrote: >> We are using OVS 2.12.0 with DPDK 18.11.2 release on Ubuntu 16.04 LTS. We do observe long initialization time for EAL init ( ~10 seconds for EAL memory initialization). >> >> Below is our hardware and memory configuration >> Hardware : Dell Server >> Sockets(NUMA) : 2 >> CPUs : Each NUMA with 10 CPUs (Intel(R) Xeon(R) CPU E5-2660 v2 @ 2.20GHz) >> RAM : 128 GB >> Hugepages reserverd : 2048 huge pages of 2MB and 100 Huge pages of 1G (reserved during boot time). We are using only 2MB huge pages for OVS process. >> NIC : Ethernet 10G 2P X520 ( ixgbe driver ) >> >> Passing below EAL arguments with OVS process. >> 2020-07-23T09:46:58.878Z|00014|dpdk|INFO|EAL ARGS: ovs-vswitchd -w 0000:04:00.0,support-multi-driver=1 -w 0000:42:00.0,support-multi-driver=1 -c 0x1 --huge-dir /mnt/huge_ovs_2M --socket-mem 2048,2048 --socket-limit 2048,2048. >> >> We debugged further in DPDK and found out that mmap called in eal_get_virtual_area is taking more time ( rte_eal_init-> rte_eal_memory_init -> rte_eal_memseg_init -> memseg_primary_init -> alloc_va_space -> eal_get_virtual_area ->mmap ). Long initialization time is noticeable difference compared to prior release. Any pointers would be helpful. > > Did you try to start ovs with the --no-mlockall option? > > That's most likely the culprit. I believe the recent mlock/MPROT changes are going to be backported to 18.11? -- Thanks, Anatoly