From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by dpdk.org (Postfix) with ESMTP id 20F68312 for ; Tue, 13 May 2014 18:27:03 +0200 (CEST) Received: by mail-wi0-f172.google.com with SMTP id hi2so6567685wib.5 for ; Tue, 13 May 2014 09:27:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :user-agent:in-reply-to:references:mime-version :content-transfer-encoding:content-type; bh=BjW0cmQb+oQI1O2gWRepjBfO+//LKz/4BS/Wx2rRR8c=; b=dp8+oZ9fqiCk6O1HkxvGS4w3sZVFFWbvA9ku5g8RYnLDHKKeYiUX1gcyuUI6GbHtn3 7doQBDkS+cJre4t0gl6gS+IS5Ysy+Z/BIxqOnaL8lSYhuqrkI99eT9XdQvge+ktX6r/L oZ7o09S9xEMMD+FFe1YT2KWjzSH+f0htws5pL4ZYSY62A2wm5yUmx1fG0IJUKs+A8/cO BjzUSxMIOCNqjK78g5K645LU3kNZHKZwD1AppO+5vPshMTGd+ZrywotbPihtRnRIwnBe rkUgYdV6kikrb5J77IcvJTK9F6Y8kmRoX6j1ek9veXewLkXmyJzHmLsC622Ih7kw6GwY j9hg== X-Gm-Message-State: ALoCoQmYgBSdgWltd8y2rOIAMwgaz+G26mjMGVWp3J2GXxqOg7iUm1JloiobETd6UbLcRZs9nhV6 X-Received: by 10.180.9.202 with SMTP id c10mr22035116wib.7.1399998430400; Tue, 13 May 2014 09:27:10 -0700 (PDT) Received: from xps13.localnet (6wind.net2.nerim.net. [213.41.180.237]) by mx.google.com with ESMTPSA id l9sm22723395wic.21.2014.05.13.09.27.08 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 May 2014 09:27:09 -0700 (PDT) From: Thomas Monjalon To: venky.venkatesan@intel.com Date: Tue, 13 May 2014 18:27:07 +0200 Message-ID: <63642020.erThIODFlC@xps13> Organization: 6WIND User-Agent: KMail/4.13 (Linux/3.14.2-1-ARCH; KDE/4.13.0; x86_64; ; ) In-Reply-To: <1399642242-19725-1-git-send-email-david.marchand@6wind.com> References: <1399642242-19725-1-git-send-email-david.marchand@6wind.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH v2] eal: change default per socket memory allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 May 2014 16:27:03 -0000 Hi Venky, There were comments on the first version of this patch and you suggeste= d to=20 try this new implementation. So do you acknowledge this patch? Thanks for your review 2014-05-09 15:30, David Marchand: > From: Didier Pallard >=20 > Currently, if there is more memory in hugepages than the amount > requested by dpdk application, the memory is allocated by taking as m= uch > memory as possible from each socket, starting from first one. > For example if a system is configured with 8 GB in 2 sockets (4 GB pe= r > socket), and dpdk is requesting only 4GB of memory, all memory will b= e > taken in socket 0 (that have exactly 4GB of free hugepages) even if s= ome > cores are configured on socket 1, and there are free hugepages on soc= ket > 1... >=20 > Change this behaviour to allocate memory on all sockets where some co= res > are configured, spreading the memory amongst sockets using following > ratio per socket: > N=B0 of cores configured on the socket / Total number of configured c= ores > * requested memory >=20 > This algorithm is used when memory amount is specified globally using= > -m option. Per socket memory allocation can always be done using > --socket-mem option. >=20 > Changes included in v2: > - only update linux implementation as bsd looks not to be ready for n= uma > - if new algorithm fails, then defaults to previous behaviour >=20 > Signed-off-by: Didier Pallard > Signed-off-by: David Marchand > --- > lib/librte_eal/linuxapp/eal/eal_memory.c | 50 > +++++++++++++++++++++++++++--- 1 file changed, 45 insertions(+), 5 > deletions(-) >=20 > diff --git a/lib/librte_eal/linuxapp/eal/eal_memory.c > b/lib/librte_eal/linuxapp/eal/eal_memory.c index 73a6394..471dcfd 100= 644 > --- a/lib/librte_eal/linuxapp/eal/eal_memory.c > +++ b/lib/librte_eal/linuxapp/eal/eal_memory.c > @@ -881,13 +881,53 @@ calc_num_pages_per_socket(uint64_t * memory, > =09if (num_hp_info =3D=3D 0) > =09=09return -1; >=20 > -=09for (socket =3D 0; socket < RTE_MAX_NUMA_NODES && total_mem !=3D = 0; socket++) > { -=09=09/* if specific memory amounts per socket weren't requested *= / > -=09=09if (internal_config.force_sockets =3D=3D 0) { > +=09/* if specific memory amounts per socket weren't requested */ > +=09if (internal_config.force_sockets =3D=3D 0) { > +=09=09int cpu_per_socket[RTE_MAX_NUMA_NODES]; > +=09=09size_t default_size, total_size; > +=09=09unsigned lcore_id; > + > +=09=09/* Compute number of cores per socket */ > +=09=09memset(cpu_per_socket, 0, sizeof(cpu_per_socket)); > +=09=09RTE_LCORE_FOREACH(lcore_id) { > +=09=09=09cpu_per_socket[rte_lcore_to_socket_id(lcore_id)]++; > +=09=09} > + > +=09=09/* > +=09=09 * Automatically spread requested memory amongst detected sock= ets > according +=09=09 * to number of cores from cpu mask present on each = socket > +=09=09 */ > +=09=09total_size =3D internal_config.memory; > +=09=09for (socket =3D 0; socket < RTE_MAX_NUMA_NODES && total_size != =3D 0; > socket++) { + > +=09=09=09/* Set memory amount per socket */ > +=09=09=09default_size =3D (internal_config.memory * cpu_per_socket[s= ocket]) > +=09=09=09 / rte_lcore_count(); > + > +=09=09=09/* Limit to maximum available memory on socket */ > +=09=09=09default_size =3D RTE_MIN(default_size,=20 get_socket_mem_size(socket)); > + > +=09=09=09/* Update sizes */ > +=09=09=09memory[socket] =3D default_size; > +=09=09=09total_size -=3D default_size; > +=09=09} > + > +=09=09/* > +=09=09 * If some memory is remaining, try to allocate it by getting = all > +=09=09 * available memory from sockets, one after the other > +=09=09 */ > +=09=09for (socket =3D 0; socket < RTE_MAX_NUMA_NODES && total_size != =3D 0; > socket++) { /* take whatever is available */ > -=09=09=09memory[socket] =3D RTE_MIN(get_socket_mem_size(socket), > -=09=09=09=09=09total_mem); > +=09=09=09default_size =3D RTE_MIN(get_socket_mem_size(socket) -=20 memory[socket], > +=09=09=09 total_size); > + > +=09=09=09/* Update sizes */ > +=09=09=09memory[socket] +=3D default_size; > +=09=09=09total_size -=3D default_size; > =09=09} > +=09} > + > +=09for (socket =3D 0; socket < RTE_MAX_NUMA_NODES && total_mem !=3D = 0; socket++) > { /* skips if the memory on specific socket wasn't requested */ > =09=09for (i =3D 0; i < num_hp_info && memory[socket] !=3D 0; i++){ > =09=09=09hp_used[i].hugedir =3D hp_info[i].hugedir;