From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id E158FF72 for ; Wed, 25 Jul 2018 19:43:32 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 0D5BE8550F; Wed, 25 Jul 2018 17:43:32 +0000 (UTC) Received: from ktraynor.remote.csb (ovpn-117-208.ams2.redhat.com [10.36.117.208]) by smtp.corp.redhat.com (Postfix) with ESMTP id 92D7B111AF1B; Wed, 25 Jul 2018 17:43:26 +0000 (UTC) To: Thomas Monjalon , "Burakov, Anatoly" Cc: dev@dpdk.org, Ravi1.Kumar@amd.com, jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, yskoh@mellanox.com, arybchenko@solarflare.com, damarion@cisco.com, stephen@networkplumber.org, olivier.matz@6wind.com, christian.ehrhardt@canonical.com, bluca@debian.org References: <40cf48703f5fae8af8c31dcc8a1a1ecb0b151d27.1532426170.git.anatoly.burakov@intel.com> <7333019.QUcIvQh70r@xps> <6155181.arAJIN2bNs@xps> From: Kevin Traynor Organization: Red Hat Message-ID: <0cde5b6f-558a-08cf-0a03-29eeb7772618@redhat.com> Date: Wed, 25 Jul 2018 18:43:25 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <6155181.arAJIN2bNs@xps> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 25 Jul 2018 17:43:32 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 25 Jul 2018 17:43:32 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'ktraynor@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH] config: reduce memory requirements for DPDK X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2018 17:43:33 -0000 On 07/24/2018 01:03 PM, Thomas Monjalon wrote: > 24/07/2018 13:04, Burakov, Anatoly: >> On 24-Jul-18 11:23 AM, Thomas Monjalon wrote: >>> 24/07/2018 12:03, Anatoly Burakov: >>>> It has been reported that current memory limitations do not work >>>> well on an 8-socket machines in default configuration when big >>>> page sizes are used [1]. >>>> >>>> Fix it by reducing memory amount reserved by DPDK by default to >>>> 32G per page size per NUMA node. This translates to allowing us >>>> to reserve 32G per page size per NUMA node on 8 nodes with 2 >>>> page sizes. >>>> >>>> [1] https://mails.dpdk.org/archives/dev/2018-July/108071.html >>>> >>>> Signed-off-by: Anatoly Burakov >>>> --- >>>> >>>> Notes: >>>> We could have increased CONFIG_RTE_MAX_MEM_MB but this would've >>>> brought other potential problems due to increased memory >>>> preallocation, and secondary process initialization is flaky >>>> enough as it is. I am willing to bet that 32G per page size is >>>> more than enough for the majority of use cases, and any >>>> application with bigger requirements could adjust config options >>>> itself. >>> [...] >>>> -CONFIG_RTE_MAX_MEMSEG_PER_TYPE=32768 >>>> -CONFIG_RTE_MAX_MEM_MB_PER_TYPE=131072 >>>> +CONFIG_RTE_MAX_MEMSEG_PER_TYPE=16384 >>>> +CONFIG_RTE_MAX_MEM_MB_PER_TYPE=32768 >>> >>> Ideally, it should be a run-time option. >>> >> >> It can be, yes, and this can be worked on for next release. However, we >> also need to have good default values that work across all supported >> platforms. > > Yes sure, we can wait the next release for a run-time option. > > How can we be sure these default values are good enough? Why add a new limitation? Why not take the other approach that was suggested of increasing the max possible memory? If there is new limitations or backwards compatibility issues with default settings compared with before the large memory management rework, then it would be good to have that clear in the docs at a high level for the users who want to update. It would also help a lot to add what the implications and limits for changing the most important defines are - will it be slower? will it not work above X? etc. > It would be good to have several acks from various projects or companies. > >