From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 51245A04C1; Thu, 21 Nov 2019 16:13:26 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A63BB2BA3; Thu, 21 Nov 2019 16:13:25 +0100 (CET) Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id B8D972BA2 for ; Thu, 21 Nov 2019 16:13:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574349203; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=RdaefnvuVUyFox6RcCyIv74VykRTXrf0MQXD3zAanI0=; b=Oe2CqHvuCCT5S/HBM+Zix1xf7dpUeIhCFPRieOlmgdSRfuZo6/kWEYq/hxvqIcDvBKcsFg bZgnezOmr6rcEzS00T7f24OSt8krMN76UnJHjwV4it47eq+b5SpDzcxNfffQ47RmVYcX7K BAs7KRm0XeM7lhsG8D7K7n9GRIRj+N4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-182-dc-LeyQHOWuJuXVJ_EYz9g-1; Thu, 21 Nov 2019 10:13:19 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7F897800A02; Thu, 21 Nov 2019 15:13:18 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.205.83]) by smtp.corp.redhat.com (Postfix) with ESMTP id 24C3110027B3; Thu, 21 Nov 2019 15:13:15 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, ferruh.yigit@intel.com, Wenzhuo Lu , Jingjing Wu , Bernard Iremonger Date: Thu, 21 Nov 2019 16:12:55 +0100 Message-Id: <20191121151256.20613-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: dc-LeyQHOWuJuXVJ_EYz9g-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Subject: [dpdk-dev] [PATCH] app/testpmd: reduce memory consumption X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Following [1], testpmd memory consumption has skyrocketted. The rte_port structure has gotten quite fat. struct rte_port { [...] struct rte_eth_rxconf rx_conf[65536]; /* 266280 3145728 */ /* --- cacheline 53312 boundary (3411968 bytes) was 40 bytes ago --- */ struct rte_eth_txconf tx_conf[65536]; /* 3412008 3670016 */ /* --- cacheline 110656 boundary (7081984 bytes) was 40 bytes ago --- */ [...] /* size: 8654936, cachelines: 135234, members: 31 */ [...] testpmd handles RTE_MAX_ETHPORTS ports (32 by default) which means that it needs ~256MB just for this internal representation. The reason is that a testpmd rte_port (the name is quite confusing, as it is a local type) maintains configurations for all queues of a port. But where you would expect testpmd to use RTE_MAX_QUEUES_PER_PORT as the maximum queue count, the rte_port uses MAX_QUEUE_ID set to 64k. Prefer the ethdev maximum value. After this patch: struct rte_port { [...] struct rte_eth_rxconf rx_conf[1025]; /* 8240 49200 */ /* --- cacheline 897 boundary (57408 bytes) was 32 bytes ago --- */ struct rte_eth_txconf tx_conf[1025]; /* 57440 57400 */ /* --- cacheline 1794 boundary (114816 bytes) was 24 bytes ago --- */ [...] /* size: 139488, cachelines: 2180, members: 31 */ [...] [1]: https://git.dpdk.org/dpdk/commit/?id=3D436b3a6b6e62 Signed-off-by: David Marchand --- app/test-pmd/testpmd.c | 6 +++--- app/test-pmd/testpmd.h | 16 +++++++--------- 2 files changed, 10 insertions(+), 12 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 73ebf37aae..d28211ba52 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -979,7 +979,7 @@ check_socket_id(const unsigned int socket_id) queueid_t get_allowed_max_nb_rxq(portid_t *pid) { -=09queueid_t allowed_max_rxq =3D MAX_QUEUE_ID; +=09queueid_t allowed_max_rxq =3D RTE_MAX_QUEUES_PER_PORT; =09bool max_rxq_valid =3D false; =09portid_t pi; =09struct rte_eth_dev_info dev_info; @@ -1029,7 +1029,7 @@ check_nb_rxq(queueid_t rxq) queueid_t get_allowed_max_nb_txq(portid_t *pid) { -=09queueid_t allowed_max_txq =3D MAX_QUEUE_ID; +=09queueid_t allowed_max_txq =3D RTE_MAX_QUEUES_PER_PORT; =09bool max_txq_valid =3D false; =09portid_t pi; =09struct rte_eth_dev_info dev_info; @@ -1079,7 +1079,7 @@ check_nb_txq(queueid_t txq) queueid_t get_allowed_max_nb_hairpinq(portid_t *pid) { -=09queueid_t allowed_max_hairpinq =3D MAX_QUEUE_ID; +=09queueid_t allowed_max_hairpinq =3D RTE_MAX_QUEUES_PER_PORT; =09portid_t pi; =09struct rte_eth_hairpin_cap cap; =20 diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 90694a3309..217d577018 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -58,8 +58,6 @@ typedef uint16_t portid_t; typedef uint16_t queueid_t; typedef uint16_t streamid_t; =20 -#define MAX_QUEUE_ID ((1 << (sizeof(queueid_t) * 8)) - 1) - #if defined RTE_LIBRTE_PMD_SOFTNIC #define SOFTNIC=09=09=091 #else @@ -179,22 +177,22 @@ struct rte_port { =09uint8_t need_reconfig_queues; /**< need reconfiguring q= ueues or not */ =09uint8_t rss_flag; /**< enable rss or not */ =09uint8_t dcb_flag; /**< enable dcb */ -=09uint16_t nb_rx_desc[MAX_QUEUE_ID+1]; /**< per queue rx d= esc number */ -=09uint16_t nb_tx_desc[MAX_QUEUE_ID+1]; /**< per queue tx d= esc number */ -=09struct rte_eth_rxconf rx_conf[MAX_QUEUE_ID+1]; /**< per queue rx conf= iguration */ -=09struct rte_eth_txconf tx_conf[MAX_QUEUE_ID+1]; /**< per queue tx conf= iguration */ +=09uint16_t nb_rx_desc[RTE_MAX_QUEUES_PER_PORT+1]; /**< per= queue rx desc number */ +=09uint16_t nb_tx_desc[RTE_MAX_QUEUES_PER_PORT+1]; /**< per= queue tx desc number */ +=09struct rte_eth_rxconf rx_conf[RTE_MAX_QUEUES_PER_PORT+1]; /**< per qu= eue rx configuration */ +=09struct rte_eth_txconf tx_conf[RTE_MAX_QUEUES_PER_PORT+1]; /**< per qu= eue tx configuration */ =09struct rte_ether_addr *mc_addr_pool; /**< pool of multicast addrs */ =09uint32_t mc_addr_nb; /**< nb. of addr. in mc_addr_pool *= / =09uint8_t slave_flag; /**< bonding slave port */ =09struct port_flow *flow_list; /**< Associated flows. */ -=09const struct rte_eth_rxtx_callback *rx_dump_cb[MAX_QUEUE_ID+1]; -=09const struct rte_eth_rxtx_callback *tx_dump_cb[MAX_QUEUE_ID+1]; +=09const struct rte_eth_rxtx_callback *rx_dump_cb[RTE_MAX_QUEUES_PER_PORT+= 1]; +=09const struct rte_eth_rxtx_callback *tx_dump_cb[RTE_MAX_QUEUES_PER_PORT+= 1]; #ifdef SOFTNIC =09struct softnic_port softport; /**< softnic params */ #endif =09/**< metadata value to insert in Tx packets. */ =09uint32_t=09=09tx_metadata; -=09const struct rte_eth_rxtx_callback *tx_set_md_cb[MAX_QUEUE_ID+1]; +=09const struct rte_eth_rxtx_callback *tx_set_md_cb[RTE_MAX_QUEUES_PER_POR= T+1]; }; =20 /** --=20 2.23.0