From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id B18BC374E for ; Mon, 25 Jun 2018 18:16:43 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 25 Jun 2018 09:16:42 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,270,1526367600"; d="scan'208";a="67109818" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.237.220.28]) ([10.237.220.28]) by fmsmga001.fm.intel.com with ESMTP; 25 Jun 2018 09:16:41 -0700 To: "Kumar, Ravi1" , "dev@dpdk.org" References: From: "Burakov, Anatoly" Message-ID: Date: Mon, 25 Jun 2018 17:16:40 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] DPDK 18.05 only works with up to 4 NUMAs systems X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2018 16:16:44 -0000 On 22-Jun-18 5:37 PM, Kumar, Ravi1 wrote: > Hi, > > As the memory subsystem in DPDK 18.05 is reworked, it has introduced a problem for AMD EPYC 2P platforms. > The issue is that DPDK 18.05 only works with up to 4 NUMAs. For AMD EPYC 2P platforms, DPDK now only works with P0 (NUMA 0-3) and does not work with P1 (NUMA 4-7). > > The problem can be fixed by reducing some of the default settings of the memory subsystem. > > To solve this issue: > - We can create our own config file for our integrated 10G NIC, that is for amd_xgbe PMD. This will make amd_xgbe immune to this problem. > - However, when any other NIC (Intel, Mellanox, Cavium or Broadcom etc.) is plugged into NUMA 4-7, the problem will still be exposed. > - If we only fix it in "config/common_base", it will cover all cases. > > Our current workaround is: > Edit config file "./config/common_base" and change the following line > CONFIG_RTE_MAX_MEM_MB_PER_TYPE=131072 > TO > CONFIG_RTE_MAX_MEM_MB_PER_TYPE=65536 > > Any better solution for this issue is welcome. > > We would appreciate if this issue can be fixed in the next release (18.08) so the STOCK version of DPDK works on AMD EPYC 2P platforms. > > Regards, > Ravi > Hi Ravi, What is the reason behind this limitation? Is it too much virtual memory being preallocated? -- Thanks, Anatoly