From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2169DA00C5; Thu, 15 Sep 2022 09:09:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 11A574021D; Thu, 15 Sep 2022 09:09:50 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 54C0440156 for ; Thu, 15 Sep 2022 09:09:49 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 28ENmMJv023140; Thu, 15 Sep 2022 00:07:42 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=IhfQqvO7LBU4pQuxD4YZ53Z6lK+xGIyQlnwvCYUTV5M=; b=c69ai7c22jjVngs2IVRifZfw50WkJaP7RVQCyZnkJhdiBDGUQXsS5d84VtgsutNyan5y DRcE5vEbQpTjiExzwLztPnsq3XdRHR8xEiOoj/RUetz6OsxIPazyVNISDozD3pS106nD Hh7cnVXXXLshhp25IbjoU8v3lHY0O1cySczoqSGK6+x0tvkHA4XTPpp/VXLMIGi6OCbc Og4MsZbz6PNMsjdUkOA49X3L/uUEsT8LHrErMdK0rcYrViMZ9ZVwfRxb2EouTxDZK350 DllMOyl0Izw+mDqBK411zh5pmIp/mCTafzyDPCD1yPahBpKM38wTmcgFYW/3S69MkYAT OA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3jjy0272xe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 15 Sep 2022 00:07:42 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 15 Sep 2022 00:07:40 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 15 Sep 2022 00:07:40 -0700 Received: from localhost.localdomain (unknown [10.28.36.155]) by maili.marvell.com (Postfix) with ESMTP id ECB4F3F70A2; Thu, 15 Sep 2022 00:07:35 -0700 (PDT) From: Hanumanth Pothula To: Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko CC: , , , , , , , , , , , , Hanumanth Pothula Subject: [PATCH v4 1/3] ethdev: Add support for mulitiple mbuf pools per Rx queue Date: Thu, 15 Sep 2022 12:37:30 +0530 Message-ID: <20220915070732.182542-1-hpothula@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220902070047.2812906-1-hpothula@marvell.com> References: <20220902070047.2812906-1-hpothula@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: 8RlTzE4G0B99pR70wgygw7uphDd7-11u X-Proofpoint-ORIG-GUID: 8RlTzE4G0B99pR70wgygw7uphDd7-11u X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-15_03,2022-09-14_04,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds support for multiple mempool capability. Some of the HW has support for choosing memory pools based on the packet's size. Thiscapability allows PMD to choose a memory pool based on the packet's length. This is often useful for saving the memory where the application can create a different pool to steer the specific size of the packet, thus enabling effective use of memory. For example, let's say HW has a capability of three pools, - pool-1 size is 2K - pool-2 size is > 2K and < 4K - pool-3 size is > 4K Here, pool-1 can accommodate packets with sizes < 2K pool-2 can accommodate packets with sizes > 2K and < 4K pool-3 can accommodate packets with sizes > 4K With multiple mempool capability enabled in SW, an application may create three pools of different sizes and send them to PMD. Allowing PMD to program HW based on the packet lengths. So that packets with less than 2K are received on pool-1, packets with lengths between 2K and 4K are received on pool-2 and finally packets greater than 4K are received on pool-3. Signed-off-by: Hanumanth Pothula v4: - Renamed Offload capability name from RTE_ETH_RX_OFFLOAD_BUFFER_SORT to RTE_ETH_RX_OFFLOAD_MUL_MEMPOOL. - In struct rte_eth_rxconf, defined new pointer, which holds array of type struct rte_eth_rx_mempool(memory pools). This array is used by PMD to program multiple mempools. v3: - Implemented Pool Sort capability as new Rx offload capability, RTE_ETH_RX_OFFLOAD_BUFFER_SORT. v2: - Along with spec changes, uploading testpmd and driver changes. --- lib/ethdev/rte_ethdev.c | 78 ++++++++++++++++++++++++++++++++++------- lib/ethdev/rte_ethdev.h | 24 +++++++++++++ 2 files changed, 89 insertions(+), 13 deletions(-) diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 1979dc0850..8618d6b01d 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -1634,6 +1634,45 @@ rte_eth_dev_is_removed(uint16_t port_id) return ret; } +static int +rte_eth_rx_queue_check_mempool(const struct rte_eth_rx_mempool *rx_mempool, + uint16_t n_pool, uint32_t *mbp_buf_size, + const struct rte_eth_dev_info *dev_info) +{ + uint16_t pool_idx; + + if (n_pool > dev_info->max_pools) { + RTE_ETHDEV_LOG(ERR, + "Invalid capabilities, max pools supported %u\n", + dev_info->max_pools); + return -EINVAL; + } + + for (pool_idx = 0; pool_idx < n_pool; pool_idx++) { + struct rte_mempool *mpl = rx_mempool[pool_idx].mp; + + if (mpl == NULL) { + RTE_ETHDEV_LOG(ERR, "null mempool pointer\n"); + return -EINVAL; + } + + *mbp_buf_size = rte_pktmbuf_data_room_size(mpl); + if (*mbp_buf_size < dev_info->min_rx_bufsize + + RTE_PKTMBUF_HEADROOM) { + RTE_ETHDEV_LOG(ERR, + "%s mbuf_data_room_size %u < %u (RTE_PKTMBUF_HEADROOM=%u + min_rx_bufsize(dev)=%u)\n", + mpl->name, *mbp_buf_size, + RTE_PKTMBUF_HEADROOM + dev_info->min_rx_bufsize, + RTE_PKTMBUF_HEADROOM, + dev_info->min_rx_bufsize); + return -EINVAL; + } + + } + + return 0; +} + static int rte_eth_rx_queue_check_split(const struct rte_eth_rxseg_split *rx_seg, uint16_t n_seg, uint32_t *mbp_buf_size, @@ -1733,7 +1772,8 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id, if (mp != NULL) { /* Single pool configuration check. */ - if (rx_conf != NULL && rx_conf->rx_nseg != 0) { + if (rx_conf != NULL && + (rx_conf->rx_nseg != 0 || rx_conf->rx_npool)) { RTE_ETHDEV_LOG(ERR, "Ambiguous segment configuration\n"); return -EINVAL; @@ -1763,30 +1803,42 @@ rte_eth_rx_queue_setup(uint16_t port_id, uint16_t rx_queue_id, dev_info.min_rx_bufsize); return -EINVAL; } - } else { - const struct rte_eth_rxseg_split *rx_seg; - uint16_t n_seg; + } else if (rx_conf->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT || + rx_conf->offloads & RTE_ETH_RX_OFFLOAD_MUL_MEMPOOL) { - /* Extended multi-segment configuration check. */ - if (rx_conf == NULL || rx_conf->rx_seg == NULL || rx_conf->rx_nseg == 0) { + /* Extended multi-segment/pool configuration check. */ + if (rx_conf == NULL || + (rx_conf->rx_seg == NULL && rx_conf->rx_mempool == NULL) || + (rx_conf->rx_nseg == 0 && rx_conf->rx_npool == 0)) { RTE_ETHDEV_LOG(ERR, "Memory pool is null and no extended configuration provided\n"); return -EINVAL; } - rx_seg = (const struct rte_eth_rxseg_split *)rx_conf->rx_seg; - n_seg = rx_conf->rx_nseg; - if (rx_conf->offloads & RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT) { + const struct rte_eth_rxseg_split *rx_seg = + (const struct rte_eth_rxseg_split *)rx_conf->rx_seg; + uint16_t n_seg = rx_conf->rx_nseg; ret = rte_eth_rx_queue_check_split(rx_seg, n_seg, &mbp_buf_size, &dev_info); - if (ret != 0) + if (ret) return ret; - } else { - RTE_ETHDEV_LOG(ERR, "No Rx segmentation offload configured\n"); - return -EINVAL; } + if (rx_conf->offloads & RTE_ETH_RX_OFFLOAD_MUL_MEMPOOL) { + const struct rte_eth_rx_mempool *rx_mempool = + (const struct rte_eth_rx_mempool *)rx_conf->rx_mempool; + ret = rte_eth_rx_queue_check_mempool(rx_mempool, + rx_conf->rx_npool, + &mbp_buf_size, + &dev_info); + if (ret) + return ret; + + } + } else { + RTE_ETHDEV_LOG(ERR, "No Rx offload is configured\n"); + return -EINVAL; } /* Use default specified by driver, if nb_rx_desc is zero */ diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index b62ac5bb6f..17deec2cbd 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -1035,6 +1035,11 @@ union rte_eth_rxseg { /* The other features settings should be added here. */ }; +/* A common structure used to describe mbuf pools per Rx queue */ +struct rte_eth_rx_mempool { + struct rte_mempool *mp; +}; + /** * A structure used to configure an Rx ring of an Ethernet port. */ @@ -1067,6 +1072,23 @@ struct rte_eth_rxconf { */ union rte_eth_rxseg *rx_seg; + /** + * Points to an array of mempools. + * + * This provides support for multiple mbuf pools per Rx queue. + * + * This is often useful for saving the memory where the application can + * create a different pools to steer the specific size of the packet, thus + * enabling effective use of memory. + * + * Note that on Rx scatter enable, a packet may be delivered using a chain + * of mbufs obtained from single mempool or multiple mempools based on + * the NIC implementation. + * + */ + struct rte_eth_rx_mempool *rx_mempool; + uint16_t rx_npool; /** < number of mempools */ + uint64_t reserved_64s[2]; /**< Reserved for future fields */ void *reserved_ptrs[2]; /**< Reserved for future fields */ }; @@ -1395,6 +1417,7 @@ struct rte_eth_conf { #define RTE_ETH_RX_OFFLOAD_OUTER_UDP_CKSUM RTE_BIT64(18) #define RTE_ETH_RX_OFFLOAD_RSS_HASH RTE_BIT64(19) #define RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT RTE_BIT64(20) +#define RTE_ETH_RX_OFFLOAD_MUL_MEMPOOL RTE_BIT64(21) #define RTE_ETH_RX_OFFLOAD_CHECKSUM (RTE_ETH_RX_OFFLOAD_IPV4_CKSUM | \ RTE_ETH_RX_OFFLOAD_UDP_CKSUM | \ @@ -1615,6 +1638,7 @@ struct rte_eth_dev_info { /** Configured number of Rx/Tx queues */ uint16_t nb_rx_queues; /**< Number of Rx queues. */ uint16_t nb_tx_queues; /**< Number of Tx queues. */ + uint16_t max_pools; /** Rx parameter recommendations */ struct rte_eth_dev_portconf default_rxportconf; /** Tx parameter recommendations */ -- 2.25.1