From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2B2A9A0548; Sat, 6 Mar 2021 16:36:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E3E3922A3B5; Sat, 6 Mar 2021 16:34:55 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id CE8B222A3A2 for ; Sat, 6 Mar 2021 16:34:53 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 126FVumJ029368 for ; Sat, 6 Mar 2021 07:34:53 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=3P4knlwMyDhv8TEJpTBrlabHdRMQk64clYkOQcu3QPA=; b=NYX0vh3bN7XL1sPin6MxgeO/1HFpKTN3G8B/1rDyq+6Pmw9w1SvAKNb+R9/EFqRRkSEl TY8JLRYQZEdS9OoeV4F8f869tKycJPUHsSHVFyQBJkR/Pgh5dz9S5GJK8K5kvIse1LA3 0Cp8GoxNdU4Q9krVUI8VXlCMiNy1ANsMabEdd06lnqGE6/Lzp6wtx8Fx2oX3NERAksZn JunYU+N/Ur5SqvptIkHw9qO9beqKuBILI9PtcNMA5wne5AxPt/XGHJ1vvn8kTl5OZdjS Jb0+or0uwzgLDvI35HQ/wq0gBBS0ofit6a4U8Ko69+m3DaEWz+pfZ1HYu/q1mKzwYR8m fA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 3747yurcbn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Sat, 06 Mar 2021 07:34:53 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:51 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sat, 6 Mar 2021 07:34:51 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Sat, 6 Mar 2021 07:34:51 -0800 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 6A01E3F703F; Sat, 6 Mar 2021 07:34:48 -0800 (PST) From: Nithin Dabilpuram To: CC: , , , , , , , Nithin Dabilpuram Date: Sat, 6 Mar 2021 21:03:32 +0530 Message-ID: <20210306153404.10781-13-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20210306153404.10781-1-ndabilpuram@marvell.com> References: <20210306153404.10781-1-ndabilpuram@marvell.com> MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-06_08:2021-03-03, 2021-03-06 signatures=0 Subject: [dpdk-dev] [PATCH 12/44] net/cnxk: add Rx multi-segmented version for cn9k X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Rx burst multi-segmented version for CN9K. Signed-off-by: Nithin Dabilpuram Signed-off-by: Pavan Nikhilesh --- drivers/net/cnxk/cn9k_rx.c | 26 ++++++++++++++++++++ drivers/net/cnxk/cn9k_rx.h | 55 ++++++++++++++++++++++++++++++++++++++++-- drivers/net/cnxk/cnxk_ethdev.h | 3 +++ 3 files changed, 82 insertions(+), 2 deletions(-) diff --git a/drivers/net/cnxk/cn9k_rx.c b/drivers/net/cnxk/cn9k_rx.c index 1c05cf3..5535735 100644 --- a/drivers/net/cnxk/cn9k_rx.c +++ b/drivers/net/cnxk/cn9k_rx.c @@ -88,6 +88,15 @@ nix_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts, void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t pkts) \ { \ return nix_recv_pkts(rx_queue, rx_pkts, pkts, (flags)); \ + } \ + \ + static uint16_t __rte_noinline __rte_hot \ + cn9k_nix_recv_pkts_mseg_##name(void *rx_queue, \ + struct rte_mbuf **rx_pkts, \ + uint16_t pkts) \ + { \ + return nix_recv_pkts(rx_queue, rx_pkts, pkts, \ + (flags) | NIX_RX_MULTI_SEG_F); \ } NIX_RX_FASTPATH_MODES @@ -110,6 +119,8 @@ pick_rx_func(struct rte_eth_dev *eth_dev, void cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) { + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev); + const eth_rx_burst_t nix_eth_rx_burst[2][2][2][2] = { #define R(name, f3, f2, f1, f0, flags) \ [f3][f2][f1][f0] = cn9k_nix_recv_pkts_##name, @@ -118,7 +129,22 @@ cn9k_eth_set_rx_function(struct rte_eth_dev *eth_dev) #undef R }; + const eth_rx_burst_t nix_eth_rx_burst_mseg[2][2][2][2] = { +#define R(name, f3, f2, f1, f0, flags) \ + [f3][f2][f1][f0] = cn9k_nix_recv_pkts_mseg_##name, + + NIX_RX_FASTPATH_MODES +#undef R + }; + pick_rx_func(eth_dev, nix_eth_rx_burst); + if (dev->rx_offloads & DEV_RX_OFFLOAD_SCATTER) + pick_rx_func(eth_dev, nix_eth_rx_burst_mseg); + + /* Copy multi seg version with no offload for tear down sequence */ + if (rte_eal_process_type() == RTE_PROC_PRIMARY) + dev->rx_pkt_burst_no_offload = + nix_eth_rx_burst_mseg[0][0][0][0]; rte_mb(); } diff --git a/drivers/net/cnxk/cn9k_rx.h b/drivers/net/cnxk/cn9k_rx.h index 949fd95..a6b245f 100644 --- a/drivers/net/cnxk/cn9k_rx.h +++ b/drivers/net/cnxk/cn9k_rx.h @@ -99,6 +99,53 @@ nix_update_match_id(const uint16_t match_id, uint64_t ol_flags, } static __rte_always_inline void +nix_cqe_xtract_mseg(const union nix_rx_parse_u *rx, struct rte_mbuf *mbuf, + uint64_t rearm) +{ + const rte_iova_t *iova_list; + struct rte_mbuf *head; + const rte_iova_t *eol; + uint8_t nb_segs; + uint64_t sg; + + sg = *(const uint64_t *)(rx + 1); + nb_segs = (sg >> 48) & 0x3; + mbuf->nb_segs = nb_segs; + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + + eol = ((const rte_iova_t *)(rx + 1) + + ((rx->cn9k.desc_sizem1 + 1) << 1)); + /* Skip SG_S and first IOVA*/ + iova_list = ((const rte_iova_t *)(rx + 1)) + 2; + nb_segs--; + + rearm = rearm & ~0xFFFF; + + head = mbuf; + while (nb_segs) { + mbuf->next = ((struct rte_mbuf *)*iova_list) - 1; + mbuf = mbuf->next; + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + + mbuf->data_len = sg & 0xFFFF; + sg = sg >> 16; + *(uint64_t *)(&mbuf->rearm_data) = rearm; + nb_segs--; + iova_list++; + + if (!nb_segs && (iova_list + 1 < eol)) { + sg = *(const uint64_t *)(iova_list); + nb_segs = (sg >> 48) & 0x3; + head->nb_segs += nb_segs; + iova_list = (const rte_iova_t *)(iova_list + 1); + } + } + mbuf->next = NULL; +} + +static __rte_always_inline void cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, struct rte_mbuf *mbuf, const void *lookup_mem, const uint64_t val, const uint16_t flag) @@ -133,8 +180,12 @@ cn9k_nix_cqe_to_mbuf(const struct nix_cqe_hdr_s *cq, const uint32_t tag, *(uint64_t *)(&mbuf->rearm_data) = val; mbuf->pkt_len = len; - mbuf->data_len = len; - mbuf->next = NULL; + if (flag & NIX_RX_MULTI_SEG_F) { + nix_cqe_xtract_mseg(rx, mbuf, val); + } else { + mbuf->data_len = len; + mbuf->next = NULL; + } } #define RSS_F NIX_RX_OFFLOAD_RSS_F diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h index 58c6710..a94f5eb 100644 --- a/drivers/net/cnxk/cnxk_ethdev.h +++ b/drivers/net/cnxk/cnxk_ethdev.h @@ -163,6 +163,9 @@ struct cnxk_eth_dev { struct cnxk_eth_qconf *tx_qconf; struct cnxk_eth_qconf *rx_qconf; + /* Rx burst for cleanup(Only Primary) */ + eth_rx_burst_t rx_pkt_burst_no_offload; + /* Default mac address */ uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; }; -- 2.8.4