From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D727EA0563; Mon, 16 Mar 2020 10:35:12 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 016201C0B4; Mon, 16 Mar 2020 10:34:57 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 9F4691C07C for ; Mon, 16 Mar 2020 10:34:54 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02G9UN1H015039; Mon, 16 Mar 2020 02:34:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=jktj7WmHyZQFDq5C8rQcUiCizurN5P+ZblzoDYE9rxY=; b=Lh5qUULHuBEZXRXoe4KHYovSVtfS/WClUlaGchdmvdVEizIVd6B/175moO/omL+zxjtN BA6ekRTadMIncMH8F88xxSEfI+/tHB8g9iFNy9CXuh9OwevfsDyiIikcK0Yn6o+7D2rq ZVaZUjIc0Y4/ykKityiLnEaRZPKSj/hgDW+FEMT7V0U60cXJY64zaYDJ5eD48O4F5FvC jPqKCiY+JW4cL74sXN9rSYNN4+VB4ky4jIn1mjleFFN7l8LVuZU9GuOw9VP9148b6MeA Xr8861jE3j5rluNJSlaE+X47aczv4H7z/bVRrTOqYtltDWJR/SCf6HAXCbEs5dfOgoA1 1g== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0b-0016f401.pphosted.com with ESMTP id 2yrxsme52d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 16 Mar 2020 02:34:54 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 02:34:51 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 02:34:51 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.169) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 02:34:50 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZwyfOeTfoF0Tq2p1fmpdKZYoUf9GOJT2uc/KrLoC0NeHhv5vZJK0QLsdhB2ApnSeQH3wwbV7ahD71O3BgOtY89R7kQSzTzGW5cqByXWQGd6Zaqg6kj+Wknoj39rtPHeahXtW69xo2bxtH6zB4PzdxkQEwf6vTeY+7xt+La9lXRy6V/JOzJjicLXN/1u7wAKTIuiVBjM164vqe6cYSSg3ecQpu11MFtq5+FRQV2/XqPlwM6TN0I6qa8/R0T8jp36Cjop67IPPFu6jFvDsQAvY0t9yfv6Sa6a27+j+K1j5zcjgizKLhPqSupmzRaaGacL5SKlM2eIH/t7a6/1EBUse/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jktj7WmHyZQFDq5C8rQcUiCizurN5P+ZblzoDYE9rxY=; b=ghMoIgTJsK7QKEdkgoeqm2ikAKBCvPeSeVKNN7mqUy1el42iKU2pNewubcsStEgLtbLZKdmlioUIEevqAaBlkDoa9xR+45HyGh7G8tXoskn6b+3O74sI0FWz3KnSzFUWlCU5kZtc+FCmJYTFpSRdc1G3cfjvJ6q28AEV7M+rnNorDHQDsF8Dn3atQ+jb9ump2cj5HxyUedL8+ZlEq/LlHBdSjHN1us1chAR0329oYS4+EKyQlbDiZT0GQIHfW70gKw2PY5qBgeZEA/ognjuMPcjBhchBnidsic6l0JX3ixFKtGdyW3jgbz9VvA4UMHf9ngsChrVTPJZD3gzIRA4/RA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jktj7WmHyZQFDq5C8rQcUiCizurN5P+ZblzoDYE9rxY=; b=PvMElnbIWPGtjUCoMW99QWujPtTf6TT1ku8QWP68YHoNA/yozo1eEJD0bmeT/oGNiBLA7UeW+2KNXesYAfP8uLkqrKyzOhPaJiXI/zoegtdnK7FS6CdwjAk6/IzKfJMO9MFE2fla3QISzXiRYDCEhnwZmtilgaawLkf5yLzZjZo= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3117.namprd18.prod.outlook.com (2603:10b6:208:162::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.14; Mon, 16 Mar 2020 09:34:31 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 09:34:31 +0000 From: Harman Kalra To: , , CC: , , Harman Kalra Date: Mon, 16 Mar 2020 15:03:39 +0530 Message-ID: <1584351224-23500-4-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584351224-23500-1-git-send-email-hkalra@marvell.com> References: <1584351224-23500-1-git-send-email-hkalra@marvell.com> Content-Type: text/plain X-ClientProxiedBy: BMXPR01CA0006.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::16) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BMXPR01CA0006.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 09:34:29 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d37e15ee-5d57-41f7-6246-08d7c98d3adc X-MS-TrafficTypeDiagnostic: MN2PR18MB3117: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5236; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(136003)(376002)(39860400002)(346002)(396003)(199004)(52116002)(7696005)(478600001)(4326008)(8676002)(8936002)(107886003)(26005)(66556008)(2906002)(86362001)(5660300002)(36756003)(55236004)(16526019)(2616005)(956004)(66946007)(6486002)(66476007)(81156014)(81166006)(6666004)(186003)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3117; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Haxd/4VTNZFIwuGVa8fklmZnrfUw6l2BOEXB4vROoqnMn7uz4ORgwDWbXEtYlmEkPhGFZ2imNqWt3yR5tmnxOQnzeWSOpj/VbNFQw29A+EzUQp2J79IqeqtCguwVFdVEnqWjmkuBScHv1fclwP1ilKr9KixbejdSAEPjpzZbZMLEXQcVBZB/CG6ZqRqH/AlzLV0PKWga1CCTArnXefa/kp1G5uLKmusVOxO1FyBtdSy17qdNozEQ/MAsEzL/iHzi16wlCXAaz1otP5VzSUaqOs5y1S7rxiYP/uPUZLoeUDq4NumcKsPPGWoilwERB8YyjBaDfCyoQRRD7lYgFAKM4/NoqEWb4xWqJ6XwiJIJC+y2ZHFQ8GhOwFr8fmtGwQDxrVbsrCQxrbxXdDIIQGpxCgiTkyQOo1y+6PCfBu+Y1cokxDhmzVGJu4pF+UfmdI7N X-MS-Exchange-AntiSpam-MessageData: 2yVy8w0EDJtjsd6/Hq/VKBSnPUdDz0pFUlBkHm2lGz9Rld5A01//XHtqBhm3uYkqN1PTLCS67GuOJTLTDSuY6uhsJTJA3RuKVqv2520NKwpTKm+Y5kV9zlbHM7iE+K5mRRb/E7u/kAPT2UA6tjUR5A== X-MS-Exchange-CrossTenant-Network-Message-Id: d37e15ee-5d57-41f7-6246-08d7c98d3adc X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 09:34:31.4735 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VCcP9fozoNvIzdZpBiaTpyo9pSMiOVgnVbhXQ4Qi9rSlCfUNGBYQwrEF8HVzu6d/tCskP5xhPsTgFQifZAtoXw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3117 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-03-16_02:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 3/8] net/octeontx: add fast mbuf free support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds capability to fast release of mbuf following successful transmission. Signed-off-by: Harman Kalra --- drivers/net/octeontx/octeontx_ethdev.c | 4 +- drivers/net/octeontx/octeontx_ethdev.h | 1 + drivers/net/octeontx/octeontx_rxtx.c | 9 +-- drivers/net/octeontx/octeontx_rxtx.h | 93 ++++++++++++++++++++++++-- 4 files changed, 98 insertions(+), 9 deletions(-) diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c index d6adbbc93..97b4298fe 100644 --- a/drivers/net/octeontx/octeontx_ethdev.c +++ b/drivers/net/octeontx/octeontx_ethdev.c @@ -267,7 +267,9 @@ octeontx_tx_offload_flags(struct rte_eth_dev *eth_dev) struct octeontx_nic *nic = octeontx_pmd_priv(eth_dev); uint16_t flags = 0; - /* Created function for supoorting future offloads */ + if (!(nic->tx_offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) + flags |= OCCTX_TX_OFFLOAD_MBUF_NOFF_F; + if (nic->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) flags |= OCCTX_TX_MULTI_SEG_F; diff --git a/drivers/net/octeontx/octeontx_ethdev.h b/drivers/net/octeontx/octeontx_ethdev.h index 06223e6e7..1abe1703b 100644 --- a/drivers/net/octeontx/octeontx_ethdev.h +++ b/drivers/net/octeontx/octeontx_ethdev.h @@ -34,6 +34,7 @@ DEV_RX_OFFLOAD_JUMBO_FRAME) #define OCTEONTX_TX_OFFLOADS (DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_MBUF_FAST_FREE | \ DEV_TX_OFFLOAD_MULTI_SEGS) static inline struct octeontx_nic * diff --git a/drivers/net/octeontx/octeontx_rxtx.c b/drivers/net/octeontx/octeontx_rxtx.c index 3de88e187..c817f7179 100644 --- a/drivers/net/octeontx/octeontx_rxtx.c +++ b/drivers/net/octeontx/octeontx_rxtx.c @@ -41,7 +41,7 @@ octeontx_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) return count; /* return number of pkts received */ } -#define T(name, f1, sz, flags) \ +#define T(name, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ octeontx_xmit_pkts_ ##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -60,14 +60,15 @@ octeontx_set_tx_function(struct rte_eth_dev *dev) { struct octeontx_nic *nic = octeontx_pmd_priv(dev); - const eth_tx_burst_t tx_burst_func[2] = { -#define T(name, f0, sz, flags) \ - [f0] = octeontx_xmit_pkts_ ##name, + const eth_tx_burst_t tx_burst_func[2][2] = { +#define T(name, f1, f0, sz, flags) \ + [f1][f0] = octeontx_xmit_pkts_ ##name, OCCTX_TX_FASTPATH_MODES #undef T }; dev->tx_pkt_burst = tx_burst_func + [!!(nic->tx_offload_flags & OCCTX_TX_OFFLOAD_MBUF_NOFF_F)] [!!(nic->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)]; } diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index 6182cd8ee..2383a8eb6 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -22,6 +22,7 @@ #define OCCTX_RX_MULTI_SEG_F BIT(15) #define OCCTX_TX_OFFLOAD_NONE (0) +#define OCCTX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) #define OCCTX_TX_MULTI_SEG_F BIT(15) /* Packet type table */ @@ -114,9 +115,75 @@ ptype_table[PTYPE_SIZE][PTYPE_SIZE][PTYPE_SIZE] = { }; +static __rte_always_inline uint64_t +octeontx_pktmbuf_detach(struct rte_mbuf *m) +{ + struct rte_mempool *mp = m->pool; + uint32_t mbuf_size, buf_len; + struct rte_mbuf *md; + uint16_t priv_size; + uint16_t refcount; + + /* Update refcount of direct mbuf */ + md = rte_mbuf_from_indirect(m); + refcount = rte_mbuf_refcnt_update(md, -1); + + priv_size = rte_pktmbuf_priv_size(mp); + mbuf_size = (uint32_t)(sizeof(struct rte_mbuf) + priv_size); + buf_len = rte_pktmbuf_data_room_size(mp); + + m->priv_size = priv_size; + m->buf_addr = (char *)m + mbuf_size; + m->buf_iova = rte_mempool_virt2iova(m) + mbuf_size; + m->buf_len = (uint16_t)buf_len; + rte_pktmbuf_reset_headroom(m); + m->data_len = 0; + m->ol_flags = 0; + m->next = NULL; + m->nb_segs = 1; + + /* Now indirect mbuf is safe to free */ + rte_pktmbuf_free(m); + + if (refcount == 0) { + rte_mbuf_refcnt_set(md, 1); + md->data_len = 0; + md->ol_flags = 0; + md->next = NULL; + md->nb_segs = 1; + return 0; + } else { + return 1; + } +} + +static __rte_always_inline uint64_t +octeontx_prefree_seg(struct rte_mbuf *m) +{ + if (likely(rte_mbuf_refcnt_read(m) == 1)) { + if (!RTE_MBUF_DIRECT(m)) + return octeontx_pktmbuf_detach(m); + + m->next = NULL; + m->nb_segs = 1; + return 0; + } else if (rte_mbuf_refcnt_update(m, -1) == 0) { + if (!RTE_MBUF_DIRECT(m)) + return octeontx_pktmbuf_detach(m); + + rte_mbuf_refcnt_set(m, 1); + m->next = NULL; + m->nb_segs = 1; + return 0; + } + + /* Mbuf is having refcount more than 1 so need not to be freed */ + return 1; +} + static __rte_always_inline uint16_t __octeontx_xmit_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, - const uint16_t flag __rte_unused) + const uint16_t flag) { uint16_t gaura_id, nb_desc = 0; @@ -124,6 +191,13 @@ __octeontx_xmit_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, cmd_buf[nb_desc++] = tx_pkt->data_len & 0xffff; cmd_buf[nb_desc++] = 0x0; + /* SEND_HDR[DF] bit controls if buffer is to be freed or + * not, as SG_DESC[I] and SEND_HDR[II] are clear. + */ + if (flag & OCCTX_TX_OFFLOAD_MBUF_NOFF_F) + cmd_buf[0] |= (octeontx_prefree_seg(tx_pkt) << + 58); + /* Mark mempool object as "put" since it is freed by PKO */ if (!(cmd_buf[0] & (1ULL << 58))) __mempool_check_cookies(tx_pkt->pool, (void **)&tx_pkt, @@ -144,7 +218,7 @@ __octeontx_xmit_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, static __rte_always_inline uint16_t __octeontx_xmit_mseg_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, - const uint16_t flag __rte_unused) + const uint16_t flag) { uint16_t nb_segs, nb_desc = 0; uint16_t gaura_id, len = 0; @@ -169,6 +243,14 @@ __octeontx_xmit_mseg_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, PKO_SEND_GATHER_GAUAR((long)gaura_id) | tx_pkt->data_len; + /* SG_DESC[I] bit controls if buffer is to be freed or + * not, as SEND_HDR[DF] and SEND_HDR[II] are clear. + */ + if (flag & OCCTX_TX_OFFLOAD_MBUF_NOFF_F) { + cmd_buf[nb_desc] |= + (octeontx_prefree_seg(tx_pkt) << 57); + } + /* Mark mempool object as "put" since it is freed by * PKO. */ @@ -222,10 +304,13 @@ __octeontx_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t octeontx_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); +#define NOFF_F OCCTX_TX_OFFLOAD_MBUF_NOFF_F #define MULT_F OCCTX_TX_MULTI_SEG_F /* [NOFF] [MULTI_SEG] */ #define OCCTX_TX_FASTPATH_MODES \ -T(no_offload, 0, 4, OCCTX_TX_OFFLOAD_NONE) \ -T(mseg, 1, 14, MULT_F) \ +T(no_offload, 0, 0, 4, OCCTX_TX_OFFLOAD_NONE) \ +T(mseg, 0, 1, 14, MULT_F) \ +T(noff, 1, 0, 4, NOFF_F) \ +T(noff_mseg, 1, 1, 14, NOFF_F | MULT_F) #endif /* __OCTEONTX_RXTX_H__ */ -- 2.18.0