From: Zhiyong Yang <zhiyong.yang@intel.com>
To: dev@dpdk.org
Cc: thomas@monjalon.net, ferruh.yigit@intel.com,
keith.wiles@intel.com, stephen@networkplumber.org,
Zhiyong Yang <zhiyong.yang@intel.com>
Subject: [dpdk-dev] [PATCH v2 4/4] testpmd: add flexibility to mbuf allocation
Date: Mon, 4 Sep 2017 13:57:34 +0800 [thread overview]
Message-ID: <20170904055734.21354-5-zhiyong.yang@intel.com> (raw)
In-Reply-To: <20170904055734.21354-1-zhiyong.yang@intel.com>
The currnet mechanisim of allocating mbuf depends on RTE_MAX_ETHPORTS
which is hardcoded at the compiling time. Once large numbers of ports
are needed by users, it may easily cause the failure of creating
mempool due to be lack of enough hugepage memory.
The patch introduces the policy to limit the max memory size, and
try to allocate a mempool with nb_mbuf_per_pool * nb_ports firstly,
if this fails due to be lack of enough hugepages, we will keep halving
the size until the allocation succeeds, or reaches the min memory
requirement.
The policy refers to OvS's solution.
Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com>
---
app/test-pmd/testpmd.c | 71 ++++++++++++++++++++++++++++++++++++++------------
app/test-pmd/testpmd.h | 3 +++
2 files changed, 58 insertions(+), 16 deletions(-)
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index ed42a4c08..fb01f28ae 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -482,13 +482,14 @@ set_def_fwd_config(void)
/*
* Configuration initialisation done once at init time.
*/
-static void
+static int
mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf,
unsigned int socket_id)
{
char pool_name[RTE_MEMPOOL_NAMESIZE];
struct rte_mempool *rte_mp = NULL;
uint32_t mb_size;
+ int ret = 0;
mb_size = sizeof(struct rte_mbuf) + mbuf_seg_size;
mbuf_poolname_build(socket_id, pool_name, sizeof(pool_name));
@@ -531,13 +532,12 @@ mbuf_pool_create(uint16_t mbuf_seg_size, unsigned nb_mbuf,
}
err:
- if (rte_mp == NULL) {
- rte_exit(EXIT_FAILURE,
- "Creation of mbuf pool for socket %u failed: %s\n",
- socket_id, rte_strerror(rte_errno));
- } else if (verbose_level > 0) {
+ if (rte_mp == NULL)
+ ret = -1;
+ else if (verbose_level > 0)
rte_mempool_dump(stdout, rte_mp);
- }
+
+ return ret;
}
/*
@@ -570,6 +570,8 @@ init_config(void)
unsigned int nb_mbuf_per_pool;
lcoreid_t lc_id;
uint8_t port_per_socket[RTE_MAX_NUMA_NODES];
+ uint16_t num_ports;
+ int32_t ret = -1;
memset(port_per_socket,0,RTE_MAX_NUMA_NODES);
@@ -632,24 +634,61 @@ init_config(void)
if (param_total_num_mbufs)
nb_mbuf_per_pool = param_total_num_mbufs;
else {
+ num_ports = RTE_MIN(MAX_MULTIPLIER_POOL, RTE_MAX_ETHPORTS);
+ num_ports = RTE_MAX(MIN_MULTIPLIER_POOL, num_ports);
nb_mbuf_per_pool = RTE_TEST_RX_DESC_MAX +
(nb_lcores * mb_mempool_cache) +
RTE_TEST_TX_DESC_MAX + MAX_PKT_BURST;
- nb_mbuf_per_pool *= RTE_MAX_ETHPORTS;
+ nb_mbuf_per_pool *= num_ports;
}
+ /* try to allocate a mempool with nb_mbuf_per_pool * num_ports, if
+ * this fails due to be lack of enough hugepages, we will keep halving
+ * the number until the allocation succeeds. or reaches the min memory
+ * requirement.
+ */
+
if (numa_support) {
uint8_t i;
+ unsigned int nb_mbuf = nb_mbuf_per_pool;
+ uint16_t nb_ports = num_ports;
+
+ for (i = 0; i < num_sockets; i++) {
+ nb_mbuf_per_pool = nb_mbuf;
+ num_ports = nb_ports;
+ do {
+ ret = mbuf_pool_create(mbuf_data_size,
+ nb_mbuf_per_pool,
+ socket_ids[i]);
+ nb_mbuf_per_pool /= 2;
+ num_ports /= 2;
+ } while (ret < 0 && num_ports > MIN_MULTIPLIER_POOL);
+
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE,
+ "Creation of socket %u mbuf pool failed: %s\n",
+ socket_ids[i],
+ rte_strerror(rte_errno));
+ }
- for (i = 0; i < num_sockets; i++)
- mbuf_pool_create(mbuf_data_size, nb_mbuf_per_pool,
- socket_ids[i]);
} else {
- if (socket_num == UMA_NO_CONFIG)
- mbuf_pool_create(mbuf_data_size, nb_mbuf_per_pool, 0);
- else
- mbuf_pool_create(mbuf_data_size, nb_mbuf_per_pool,
- socket_num);
+
+ do {
+ if (socket_num == UMA_NO_CONFIG)
+ ret = mbuf_pool_create(mbuf_data_size,
+ nb_mbuf_per_pool, 0);
+ else
+ ret = mbuf_pool_create(mbuf_data_size,
+ nb_mbuf_per_pool,
+ socket_num);
+ nb_mbuf_per_pool /= 2;
+ num_ports /= 2;
+ } while (ret < 0 && num_ports > MIN_MULTIPLIER_POOL);
+
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE,
+ "Creation of mbuf pool for socket %u failed: %s\n",
+ socket_num, rte_strerror(rte_errno));
}
init_port_config();
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index e00d9eb2d..50cb9c78c 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -90,6 +90,9 @@ enum {
PORT_TOPOLOGY_LOOP,
};
+#define MAX_MULTIPLIER_POOL 64
+#define MIN_MULTIPLIER_POOL 4
+
#ifdef RTE_TEST_PMD_RECORD_BURST_STATS
/**
* The data structure associated with RX and TX packet burst statistics
--
2.13.3
next prev parent reply other threads:[~2017-09-04 5:57 UTC|newest]
Thread overview: 101+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-08-09 8:42 [dpdk-dev] [PATCH 0/2] increase port_id range Zhiyong Yang
2017-08-09 8:42 ` [dpdk-dev] [PATCH 1/2] ethdev: " Zhiyong Yang
2017-08-09 12:52 ` Ferruh Yigit
2017-08-09 12:57 ` Wiles, Keith
2017-08-10 0:53 ` Yang, Zhiyong
2017-08-10 0:51 ` Yang, Zhiyong
2017-08-09 8:42 ` [dpdk-dev] [PATCH 2/2] examples: " Zhiyong Yang
2017-08-09 14:48 ` Stephen Hemminger
2017-08-10 1:03 ` Yang, Zhiyong
2017-08-09 9:00 ` [dpdk-dev] [PATCH 0/2] " De Lara Guarch, Pablo
2017-08-09 9:17 ` Yang, Zhiyong
2017-09-04 5:57 ` [dpdk-dev] [PATCH v2 0/4] " Zhiyong Yang
2017-09-04 5:57 ` [dpdk-dev] [PATCH v2 1/4] ethdev: " Zhiyong Yang
2017-09-04 9:06 ` Bruce Richardson
2017-09-04 9:29 ` Ferruh Yigit
2017-09-04 9:47 ` Yang, Zhiyong
2017-09-04 13:12 ` Adrien Mazarguil
2017-09-04 13:17 ` Richardson, Bruce
2017-09-04 13:36 ` Adrien Mazarguil
2017-09-04 13:59 ` Yang, Zhiyong
2017-09-04 14:41 ` Adrien Mazarguil
2017-09-05 6:51 ` Yang, Zhiyong
2017-09-06 8:32 ` Hemant Agrawal
2017-09-06 8:48 ` Yang, Zhiyong
[not found] ` <CALZ3Guikt9x8sz-oEKCuDCSp_wtKa64bSXTrMhqcWyg7f_dS7g@mail.gmail.com>
2017-09-07 0:45 ` Yang, Zhiyong
2017-09-04 5:57 ` [dpdk-dev] [PATCH v2 2/4] examples: " Zhiyong Yang
2017-09-04 14:15 ` Hunt, David
2017-09-04 15:01 ` Yang, Zhiyong
2017-09-04 5:57 ` [dpdk-dev] [PATCH v2 3/4] common_base: extend RTE_MAX_ETHPORTS from 32 to 1024 Zhiyong Yang
2017-09-04 7:46 ` Yao, Lei A
2017-09-04 7:59 ` Yang, Zhiyong
2017-09-04 9:09 ` Bruce Richardson
2017-09-04 10:05 ` Yang, Zhiyong
2017-09-04 10:27 ` Ananyev, Konstantin
2017-09-04 14:18 ` Yang, Zhiyong
2017-09-06 8:42 ` Hemant Agrawal
2017-09-06 8:52 ` Yang, Zhiyong
2017-09-04 10:29 ` Bruce Richardson
2017-09-04 9:27 ` Ananyev, Konstantin
2017-09-04 5:57 ` Zhiyong Yang [this message]
2017-09-09 14:47 ` [dpdk-dev] [PATCH v3 0/4] increase port_id range Zhiyong Yang
2017-09-09 14:47 ` [dpdk-dev] [PATCH v3 1/4] ethdev: " Zhiyong Yang
2017-09-11 9:37 ` Adrien Mazarguil
2017-09-11 10:51 ` Yang, Zhiyong
2017-09-11 10:21 ` Ferruh Yigit
2017-09-13 2:26 ` Yang, Zhiyong
2017-09-13 11:56 ` Ferruh Yigit
2017-09-13 12:15 ` Yang, Zhiyong
2017-09-13 12:18 ` Thomas Monjalon
2017-09-13 13:33 ` Ferruh Yigit
2017-09-19 6:05 ` Yang, Zhiyong
2017-09-19 12:30 ` Wiles, Keith
2017-09-14 12:49 ` Ferruh Yigit
2017-09-15 5:11 ` Yang, Zhiyong
2017-09-09 14:47 ` [dpdk-dev] [PATCH v3 2/4] test: " Zhiyong Yang
2017-09-09 14:47 ` [dpdk-dev] [PATCH v3 3/4] examples: " Zhiyong Yang
2017-09-14 14:41 ` Ferruh Yigit
2017-09-09 14:47 ` [dpdk-dev] [PATCH v3 4/4] librte_mbuf: modify port initialization value Zhiyong Yang
2017-09-11 10:23 ` [dpdk-dev] [PATCH v3 0/4] increase port_id range Ferruh Yigit
2017-09-11 11:25 ` Yang, Zhiyong
2017-09-13 8:14 ` Matej Vido
2017-09-13 8:21 ` Yang, Zhiyong
2017-09-18 14:54 ` Laatz, Kevin
2017-09-19 1:39 ` Yang, Zhiyong
2017-09-11 10:26 ` Ferruh Yigit
2017-09-11 10:55 ` Yang, Zhiyong
2017-09-11 11:24 ` Ferruh Yigit
2017-09-21 8:32 ` [dpdk-dev] [PATCH v4 0/5] " Zhiyong Yang
2017-09-21 8:32 ` [dpdk-dev] [PATCH v4 1/5] net/bonding: remove bonding APIs using ABI versioning Zhiyong Yang
2017-09-21 10:36 ` Ferruh Yigit
2017-09-22 2:02 ` Yang, Zhiyong
2017-09-21 8:32 ` [dpdk-dev] [PATCH v4 2/5] ethdev: increase port_id range Zhiyong Yang
2017-09-21 11:49 ` Adrien Mazarguil
2017-10-06 14:34 ` Nélio Laranjeiro
2017-09-21 8:32 ` [dpdk-dev] [PATCH v4 3/5] examples: " Zhiyong Yang
2017-09-21 8:32 ` [dpdk-dev] [PATCH v4 4/5] test: " Zhiyong Yang
2017-09-21 8:32 ` [dpdk-dev] [PATCH v4 5/5] librte_mbuf: modify port initialization value Zhiyong Yang
2017-09-25 3:22 ` [dpdk-dev] [PATCH v5 0/5] increase port_id range Zhiyong Yang
2017-09-25 3:22 ` [dpdk-dev] [PATCH v5 1/5] net/bonding: remove bonding APIs using ABI versioning Zhiyong Yang
2017-09-25 11:34 ` Ferruh Yigit
2017-09-25 3:22 ` [dpdk-dev] [PATCH v5 2/5] ethdev: increase port_id range Zhiyong Yang
2017-09-25 11:37 ` Ferruh Yigit
2017-09-25 12:06 ` Ferruh Yigit
2017-09-26 7:01 ` Yang, Zhiyong
2017-09-27 18:44 ` Ferruh Yigit
2017-09-28 2:12 ` Yang, Zhiyong
2017-09-25 3:22 ` [dpdk-dev] [PATCH v5 3/5] examples: " Zhiyong Yang
2017-09-25 3:22 ` [dpdk-dev] [PATCH v5 4/5] test: " Zhiyong Yang
2017-09-25 3:22 ` [dpdk-dev] [PATCH v5 5/5] librte_mbuf: modify port initialization value Zhiyong Yang
2017-09-29 7:17 ` [dpdk-dev] [PATCH v6 0/5] increase port_id range Zhiyong Yang
2017-09-29 7:17 ` [dpdk-dev] [PATCH v6 1/5] net/bonding: remove bonding APIs using ABI versioning Zhiyong Yang
2017-09-29 7:17 ` [dpdk-dev] [PATCH v6 2/5] ethdev: increase port_id range Zhiyong Yang
2017-09-29 7:17 ` [dpdk-dev] [PATCH v6 3/5] examples: " Zhiyong Yang
2017-09-29 7:17 ` [dpdk-dev] [PATCH v6 4/5] test: " Zhiyong Yang
2017-09-29 7:17 ` [dpdk-dev] [PATCH v6 5/5] librte_mbuf: modify port initialization value Zhiyong Yang
2017-10-06 2:15 ` [dpdk-dev] [PATCH v6 0/5] increase port_id range Ferruh Yigit
2017-10-06 13:31 ` Gaëtan Rivet
2017-10-06 14:29 ` Thomas Monjalon
2017-10-06 16:02 ` Thomas Monjalon
2017-10-11 21:21 ` Ferruh Yigit
2017-10-12 1:33 ` Yang, Zhiyong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170904055734.21354-5-zhiyong.yang@intel.com \
--to=zhiyong.yang@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=keith.wiles@intel.com \
--cc=stephen@networkplumber.org \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).