From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id F4185A04C1; Fri, 22 Nov 2019 11:44:12 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9E2141DBC; Fri, 22 Nov 2019 11:44:11 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 745F891 for ; Fri, 22 Nov 2019 11:44:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574419448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=InLpkC0rsLLYKYI8b8ZQLlYaKlBtDNyGaz+VkpN1NdU=; b=UUhNj6WGm8Q7A3wue76Jn+uF2ALCcolotQTaiRLspc2Ny0wxLw7Hg7oYAbbmOFvamLlkYW 19LuAW3UI+oiq/TNwG2X72YsGiKoDzlQkvYYx3YyBhx4mMNwqFr4nRP7YJHu3RA13TVxJ5 EljtlUsDuOkhknIc79WR3FSBYbrjJxY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-368-Y9tbknqmNH2NUslaVZ_ceg-1; Fri, 22 Nov 2019 05:44:05 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 82C19107ACE3; Fri, 22 Nov 2019 10:44:03 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.205.253]) by smtp.corp.redhat.com (Postfix) with ESMTP id B0D904DA71; Fri, 22 Nov 2019 10:43:58 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, ferruh.yigit@intel.com, stephen@networkplumber.org, Wenzhuo Lu , Jingjing Wu , Bernard Iremonger , Andrew Rybchenko Date: Fri, 22 Nov 2019 11:43:23 +0100 Message-Id: <20191122104323.28992-1-david.marchand@redhat.com> In-Reply-To: <20191121151256.20613-1-david.marchand@redhat.com> References: <20191121151256.20613-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: Y9tbknqmNH2NUslaVZ_ceg-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Subject: [dpdk-dev] [PATCH v2] app/testpmd: reduce memory consumption X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Following [1], testpmd memory consumption has skyrocketted. The rte_port structure has gotten quite fat. struct rte_port { [...] struct rte_eth_rxconf rx_conf[65536]; /* 266280 3145728 */ /* --- cacheline 53312 boundary (3411968 bytes) was 40 bytes ago --- */ struct rte_eth_txconf tx_conf[65536]; /* 3412008 3670016 */ /* --- cacheline 110656 boundary (7081984 bytes) was 40 bytes ago --- */ [...] /* size: 8654936, cachelines: 135234, members: 31 */ [...] testpmd handles RTE_MAX_ETHPORTS ports (32 by default) which means that it needs ~256MB just for this internal representation. The reason is that a testpmd rte_port (the name is quite confusing, as it is a local type) maintains configurations for all queues of a port. But where you would expect testpmd to use RTE_MAX_QUEUES_PER_PORT as the maximum queue count, the rte_port uses MAX_QUEUE_ID set to 64k. Prefer the ethdev maximum value. After this patch: struct rte_port { [...] struct rte_eth_rxconf rx_conf[1025]; /* 8240 49200 */ /* --- cacheline 897 boundary (57408 bytes) was 32 bytes ago --- */ struct rte_eth_txconf tx_conf[1025]; /* 57440 57400 */ /* --- cacheline 1794 boundary (114816 bytes) was 24 bytes ago --- */ [...] /* size: 139488, cachelines: 2180, members: 31 */ [...] With this, we can ask for less memory in test-null.sh. [1]: https://git.dpdk.org/dpdk/commit/?id=3D436b3a6b6e62 Signed-off-by: David Marchand Acked-by: Ferruh Yigit Acked-by: Thomas Monjalon --- Changelog since v1: - updated test-null.sh --- app/test-pmd/testpmd.c | 6 +++--- app/test-pmd/testpmd.h | 16 +++++++--------- devtools/test-null.sh | 2 +- 3 files changed, 11 insertions(+), 13 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 73ebf37aae..d28211ba52 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -979,7 +979,7 @@ check_socket_id(const unsigned int socket_id) queueid_t get_allowed_max_nb_rxq(portid_t *pid) { -=09queueid_t allowed_max_rxq =3D MAX_QUEUE_ID; +=09queueid_t allowed_max_rxq =3D RTE_MAX_QUEUES_PER_PORT; =09bool max_rxq_valid =3D false; =09portid_t pi; =09struct rte_eth_dev_info dev_info; @@ -1029,7 +1029,7 @@ check_nb_rxq(queueid_t rxq) queueid_t get_allowed_max_nb_txq(portid_t *pid) { -=09queueid_t allowed_max_txq =3D MAX_QUEUE_ID; +=09queueid_t allowed_max_txq =3D RTE_MAX_QUEUES_PER_PORT; =09bool max_txq_valid =3D false; =09portid_t pi; =09struct rte_eth_dev_info dev_info; @@ -1079,7 +1079,7 @@ check_nb_txq(queueid_t txq) queueid_t get_allowed_max_nb_hairpinq(portid_t *pid) { -=09queueid_t allowed_max_hairpinq =3D MAX_QUEUE_ID; +=09queueid_t allowed_max_hairpinq =3D RTE_MAX_QUEUES_PER_PORT; =09portid_t pi; =09struct rte_eth_hairpin_cap cap; =20 diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 90694a3309..217d577018 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -58,8 +58,6 @@ typedef uint16_t portid_t; typedef uint16_t queueid_t; typedef uint16_t streamid_t; =20 -#define MAX_QUEUE_ID ((1 << (sizeof(queueid_t) * 8)) - 1) - #if defined RTE_LIBRTE_PMD_SOFTNIC #define SOFTNIC=09=09=091 #else @@ -179,22 +177,22 @@ struct rte_port { =09uint8_t need_reconfig_queues; /**< need reconfiguring q= ueues or not */ =09uint8_t rss_flag; /**< enable rss or not */ =09uint8_t dcb_flag; /**< enable dcb */ -=09uint16_t nb_rx_desc[MAX_QUEUE_ID+1]; /**< per queue rx d= esc number */ -=09uint16_t nb_tx_desc[MAX_QUEUE_ID+1]; /**< per queue tx d= esc number */ -=09struct rte_eth_rxconf rx_conf[MAX_QUEUE_ID+1]; /**< per queue rx conf= iguration */ -=09struct rte_eth_txconf tx_conf[MAX_QUEUE_ID+1]; /**< per queue tx conf= iguration */ +=09uint16_t nb_rx_desc[RTE_MAX_QUEUES_PER_PORT+1]; /**< per= queue rx desc number */ +=09uint16_t nb_tx_desc[RTE_MAX_QUEUES_PER_PORT+1]; /**< per= queue tx desc number */ +=09struct rte_eth_rxconf rx_conf[RTE_MAX_QUEUES_PER_PORT+1]; /**< per qu= eue rx configuration */ +=09struct rte_eth_txconf tx_conf[RTE_MAX_QUEUES_PER_PORT+1]; /**< per qu= eue tx configuration */ =09struct rte_ether_addr *mc_addr_pool; /**< pool of multicast addrs */ =09uint32_t mc_addr_nb; /**< nb. of addr. in mc_addr_pool *= / =09uint8_t slave_flag; /**< bonding slave port */ =09struct port_flow *flow_list; /**< Associated flows. */ -=09const struct rte_eth_rxtx_callback *rx_dump_cb[MAX_QUEUE_ID+1]; -=09const struct rte_eth_rxtx_callback *tx_dump_cb[MAX_QUEUE_ID+1]; +=09const struct rte_eth_rxtx_callback *rx_dump_cb[RTE_MAX_QUEUES_PER_PORT+= 1]; +=09const struct rte_eth_rxtx_callback *tx_dump_cb[RTE_MAX_QUEUES_PER_PORT+= 1]; #ifdef SOFTNIC =09struct softnic_port softport; /**< softnic params */ #endif =09/**< metadata value to insert in Tx packets. */ =09uint32_t=09=09tx_metadata; -=09const struct rte_eth_rxtx_callback *tx_set_md_cb[MAX_QUEUE_ID+1]; +=09const struct rte_eth_rxtx_callback *tx_set_md_cb[RTE_MAX_QUEUES_PER_POR= T+1]; }; =20 /** diff --git a/devtools/test-null.sh b/devtools/test-null.sh index 9f9a459f76..6e5b1ad529 100755 --- a/devtools/test-null.sh +++ b/devtools/test-null.sh @@ -25,6 +25,6 @@ else fi =20 (sleep 1 && echo stop) | -$testpmd -c $coremask --no-huge -m 150 \ +$testpmd -c $coremask --no-huge -m 20 \ =09$libs --vdev net_null1 --vdev net_null2 $eal_options -- \ =09--no-mlockall --total-num-mbufs=3D2048 $testpmd_options -ia --=20 2.23.0