From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 084A2A0559; Mon, 16 Mar 2020 10:35:43 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 66BE71C0C2; Mon, 16 Mar 2020 10:35:00 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 88AAC1C0B2 for ; Mon, 16 Mar 2020 10:34:55 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02G9VOGj024313; Mon, 16 Mar 2020 02:34:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=gV3HB7a7AyAwGdjswMIO8CRBXwaYE69hzMAqCcJyP4o=; b=TOV8dEw4jxLgfTlKVe2uEyivD06KeCuJmz+YxYWj8OpJYBs6T48Izw0hDNNebyKg1vru 1cVNxP0m++pJgm62OGSikDfSPhz16aonjpya41y8DQCnbFklSHHxsLwb1eR9+lQdukQO I+WT2SJw06+KGyPBj9g9+EBip+QOMVLQ+vQcRjk27j0MawWlRlswyOHo430J28bFYd90 Z9nXvi7uiyUv7WR6l1K8CT34+hLIshJ0hU2+lSWMOsBpNPoRu1zCCaVBya+KOt8RwK4y datIah7nPDN/Jsl5caNDeT/zlmczoy+4gIHuEAK+X0ANj9TYbsVkEdpeAi7IVGxeWuMJ 1A== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 2yrvmqppyb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 16 Mar 2020 02:34:54 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 02:34:52 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.170) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 02:34:52 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UTt7wz2unu3dC/EyvP/GQLMc7KfsUEqdD7VEMa1w7ySMpO/h1cQKo8NuYmOZFamq9IMmQR3Qc4vAiAoVIcY1yIpiJIhb88TGg6mmvWUHTxB3USixhJgdURi4xWuIvjX+v/NzTue1SKGlJZStrK97892OuqB+YTSMEGH8YPj5hD5woEXjHozR0ixx9Xrdr4OK7vqYRpvfEX9RRMf+xZRlbdlentAncZ9tfZcF7On26dPqoNpzMBvRMtbM5p+0NJjjNJ8dzQYHu6OBsKk1ySwPgtvK57BRbH/rVNm+e38O+093go7uK+OgG1Y9qoRliA6dSFmqF1JBuM21ANMoMaSCxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gV3HB7a7AyAwGdjswMIO8CRBXwaYE69hzMAqCcJyP4o=; b=gzglFx70ansecqY84peHzyQtXGMaVhS2JZpQOcL8eXTIsrniW5r/RMG/e+vHkPuOjauZX+AV0DgU1HLxu7j+eiN+sDk/8d6KUzFdaLh1qt+a7v6NuX2P0ywDFSIYZ2EIvtawweTsR9Xq6uwY6wp+vKnGSGO7TAP57rEBC2WzkKwt4NsYJSn1qojfbz0xp5LdNsycRNvpZc78Yj2V4X4f5N5zd3x2r98xLhKmTsWGZ+j+PeafxBp5xMk7/vEY3bx05i2+Pf/L8eg6rJa9HuMOXSn2uv0RO91MxJ0mX4X5SS3gZcDcSsW6jOIwz1yDcNoSGHUFn9UE0d3ivKNxGnewbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gV3HB7a7AyAwGdjswMIO8CRBXwaYE69hzMAqCcJyP4o=; b=elBIGMaxp/w75H2j/ZxKEMSGolY49acyUc+bEcGNRhEA0xNcEObYu7BvWt/+Z5Qhyla76m4i3cQjypVawhOsqK2W0DIDVwAJ2Fvybl0lNECc2xSCSel7mxAMOmgRhlq0RKorJz94h7BFdPqqFOkiU3kR+YNKAxHmoagnk+rk9GM= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by BYASPR01MB9.namprd18.prod.outlook.com (2603:10b6:a02:92::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.14; Mon, 16 Mar 2020 09:34:51 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 09:34:51 +0000 From: Harman Kalra To: , , CC: , , Harman Kalra Date: Mon, 16 Mar 2020 15:03:44 +0530 Message-ID: <1584351224-23500-9-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584351224-23500-1-git-send-email-hkalra@marvell.com> References: <1584351224-23500-1-git-send-email-hkalra@marvell.com> Content-Type: text/plain X-ClientProxiedBy: BMXPR01CA0006.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::16) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BMXPR01CA0006.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 09:34:45 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0f198454-e98a-404c-6c23-08d7c98d443e X-MS-TrafficTypeDiagnostic: BYASPR01MB9: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:341; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(396003)(346002)(39860400002)(376002)(136003)(199004)(6486002)(4326008)(36756003)(2906002)(107886003)(30864003)(478600001)(186003)(16526019)(5660300002)(86362001)(2616005)(956004)(26005)(66946007)(81166006)(81156014)(8936002)(8676002)(66556008)(66476007)(55236004)(52116002)(316002)(7696005); DIR:OUT; SFP:1101; SCL:1; SRVR:BYASPR01MB9; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cd5jChQ7Jusnv94C3HZJSmzCFRqv9HRsIEcgcg1j1mfFsX3lSInyPOhlexxAzFEuV4mz91NYNItEgFUZLAKtLpT6ECNgssF4q4PYPx9WcPLuavZcehdGL3nu9TeZkV5Wxozl5oV1USHKq2DVTwrQRFiCp8aCk7I3QkBckT7izJXeKMkbgcbxOqsQa1PkCBsAdtXpS53GxSIL93jtQHgT+JksmrpT4XbMC4Ex3KeJsaaPITWbd9zBDov2jRYgBuAWozakE/y1/pNyITdr9yDx2hWQ1RCEmIxM6zH/Wt/8sbrOdqolQLIO4N++yIQ6UTTa95rzRkCc2l9uGBRfhjGwco+VDTdFKW3UePfoGdySfA3A6qBdSdxlrAud345cbCxv2aMw4JDHbhCJAbf+fqNVuK4QZyO5ibwohINBo2PIrwYVw8hIBb8hK56tL2piM/NR X-MS-Exchange-AntiSpam-MessageData: a8moa4gj8T3kn6mvOVZQFw8uhS88Fed+/3gGyYegEdYpeZDCT/HQLuMJTMcLW3C+Og2/a29Y3VLh19QFo0OiTIE0Y6UMzaIgdg3Q3NY8LVcPn5C6+HwCI2wvYy0miNTlnRbOIomBbl3Ep6j6xwWNYQ== X-MS-Exchange-CrossTenant-Network-Message-Id: 0f198454-e98a-404c-6c23-08d7c98d443e X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 09:34:47.0207 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KunwmFy3Ff61oAsVO6aMQUW31+sP/yuLTJEd98U/9LjhXrQKLZJwJcpK7c39y+P7Dk8mvevJeB48tYhIIX7t5Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYASPR01MB9 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-03-16_02:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 8/8] net/octeontx: support Rx Tx checksum offload X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch implements rx/tx checksum offload. In case of wrong checksum received (inner/outer l3/l4) it reports the corresponding layer which has bad checksum and also corrects it if hw checksum is enabled on tx side. Signed-off-by: Harman Kalra --- doc/guides/nics/features/octeontx.ini | 4 + drivers/net/octeontx/octeontx_ethdev.c | 22 ++- drivers/net/octeontx/octeontx_ethdev.h | 21 ++- drivers/net/octeontx/octeontx_rxtx.c | 10 +- drivers/net/octeontx/octeontx_rxtx.h | 194 +++++++++++++++++++++++-- 5 files changed, 227 insertions(+), 24 deletions(-) diff --git a/doc/guides/nics/features/octeontx.ini b/doc/guides/nics/features/octeontx.ini index 6049c1c43..8a95c216c 100644 --- a/doc/guides/nics/features/octeontx.ini +++ b/doc/guides/nics/features/octeontx.ini @@ -17,6 +17,10 @@ Unicast MAC filter = Y VLAN filter = Y VLAN offload = P CRC offload = Y +L3 checksum offload = Y +L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Flow control = Y Basic stats = Y diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c index 191869683..ea3b278a1 100644 --- a/drivers/net/octeontx/octeontx_ethdev.c +++ b/drivers/net/octeontx/octeontx_ethdev.c @@ -370,6 +370,16 @@ octeontx_tx_offload_flags(struct rte_eth_dev *eth_dev) struct octeontx_nic *nic = octeontx_pmd_priv(eth_dev); uint16_t flags = 0; + if (nic->tx_offloads & DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM || + nic->tx_offloads & DEV_TX_OFFLOAD_OUTER_UDP_CKSUM) + flags |= OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F; + + if (nic->tx_offloads & DEV_TX_OFFLOAD_IPV4_CKSUM || + nic->tx_offloads & DEV_TX_OFFLOAD_TCP_CKSUM || + nic->tx_offloads & DEV_TX_OFFLOAD_UDP_CKSUM || + nic->tx_offloads & DEV_TX_OFFLOAD_SCTP_CKSUM) + flags |= OCCTX_TX_OFFLOAD_L3_L4_CSUM_F; + if (!(nic->tx_offloads & DEV_TX_OFFLOAD_MBUF_FAST_FREE)) flags |= OCCTX_TX_OFFLOAD_MBUF_NOFF_F; @@ -383,13 +393,15 @@ static uint16_t octeontx_rx_offload_flags(struct rte_eth_dev *eth_dev) { struct octeontx_nic *nic = octeontx_pmd_priv(eth_dev); - struct rte_eth_dev_data *data = eth_dev->data; - struct rte_eth_conf *conf = &data->dev_conf; - struct rte_eth_rxmode *rxmode = &conf->rxmode; uint16_t flags = 0; - if (rxmode->mq_mode == ETH_MQ_RX_RSS) - flags |= OCCTX_RX_OFFLOAD_RSS_F; + if (nic->rx_offloads & (DEV_RX_OFFLOAD_TCP_CKSUM | + DEV_RX_OFFLOAD_UDP_CKSUM)) + flags |= OCCTX_RX_OFFLOAD_CSUM_F; + + if (nic->rx_offloads & (DEV_RX_OFFLOAD_IPV4_CKSUM | + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM)) + flags |= OCCTX_RX_OFFLOAD_CSUM_F; if (nic->rx_offloads & DEV_RX_OFFLOAD_SCATTER) { flags |= OCCTX_RX_MULTI_SEG_F; diff --git a/drivers/net/octeontx/octeontx_ethdev.h b/drivers/net/octeontx/octeontx_ethdev.h index dc53b53be..7246fb6d1 100644 --- a/drivers/net/octeontx/octeontx_ethdev.h +++ b/drivers/net/octeontx/octeontx_ethdev.h @@ -53,13 +53,24 @@ #define OCCTX_MAX_MTU (OCCTX_MAX_FRS - OCCTX_L2_OVERHEAD) -#define OCTEONTX_RX_OFFLOADS (DEV_RX_OFFLOAD_CHECKSUM | \ - DEV_RX_OFFLOAD_SCATTER | \ - DEV_RX_OFFLOAD_JUMBO_FRAME | \ +#define OCTEONTX_RX_OFFLOADS ( \ + DEV_RX_OFFLOAD_CHECKSUM | \ + DEV_RX_OFFLOAD_SCTP_CKSUM | \ + DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | \ + DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_JUMBO_FRAME | \ DEV_RX_OFFLOAD_VLAN_FILTER) -#define OCTEONTX_TX_OFFLOADS (DEV_TX_OFFLOAD_MT_LOCKFREE | \ - DEV_TX_OFFLOAD_MBUF_FAST_FREE | \ +#define OCTEONTX_TX_OFFLOADS ( \ + DEV_TX_OFFLOAD_MBUF_FAST_FREE | \ + DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | \ + DEV_TX_OFFLOAD_OUTER_UDP_CKSUM | \ + DEV_TX_OFFLOAD_IPV4_CKSUM | \ + DEV_TX_OFFLOAD_TCP_CKSUM | \ + DEV_TX_OFFLOAD_UDP_CKSUM | \ + DEV_TX_OFFLOAD_SCTP_CKSUM | \ DEV_TX_OFFLOAD_MULTI_SEGS) static inline struct octeontx_nic * diff --git a/drivers/net/octeontx/octeontx_rxtx.c b/drivers/net/octeontx/octeontx_rxtx.c index c817f7179..d2453ba26 100644 --- a/drivers/net/octeontx/octeontx_rxtx.c +++ b/drivers/net/octeontx/octeontx_rxtx.c @@ -41,7 +41,7 @@ octeontx_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) return count; /* return number of pkts received */ } -#define T(name, f1, f0, sz, flags) \ +#define T(name, f3, f2, f1, f0, sz, flags) \ static uint16_t __rte_noinline __hot \ octeontx_xmit_pkts_ ##name(void *tx_queue, \ struct rte_mbuf **tx_pkts, uint16_t pkts) \ @@ -60,9 +60,9 @@ octeontx_set_tx_function(struct rte_eth_dev *dev) { struct octeontx_nic *nic = octeontx_pmd_priv(dev); - const eth_tx_burst_t tx_burst_func[2][2] = { -#define T(name, f1, f0, sz, flags) \ - [f1][f0] = octeontx_xmit_pkts_ ##name, + const eth_tx_burst_t tx_burst_func[2][2][2][2] = { +#define T(name, f3, f2, f1, f0, sz, flags) \ + [f3][f2][f1][f0] = octeontx_xmit_pkts_ ##name, OCCTX_TX_FASTPATH_MODES #undef T @@ -70,5 +70,7 @@ OCCTX_TX_FASTPATH_MODES dev->tx_pkt_burst = tx_burst_func [!!(nic->tx_offload_flags & OCCTX_TX_OFFLOAD_MBUF_NOFF_F)] + [!!(nic->tx_offload_flags & OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F)] + [!!(nic->tx_offload_flags & OCCTX_TX_OFFLOAD_L3_L4_CSUM_F)] [!!(nic->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)]; } diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index cc044dd79..acc1f5cb8 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -18,17 +18,65 @@ #define BIT(nr) (1UL << (nr)) #define OCCTX_RX_OFFLOAD_NONE (0) -#define OCCTX_RX_OFFLOAD_RSS_F BIT(0) -#define OCCTX_RX_VLAN_FLTR_F BIT(1) -#define OCCTX_RX_MULTI_SEG_F BIT(15) +#define OCCTX_RX_MULTI_SEG_F BIT(0) +#define OCCTX_RX_OFFLOAD_CSUM_F BIT(1) +#define OCCTX_RX_VLAN_FLTR_F BIT(2) #define OCCTX_TX_OFFLOAD_NONE (0) +#define OCCTX_TX_MULTI_SEG_F BIT(0) +#define OCCTX_TX_OFFLOAD_L3_L4_CSUM_F BIT(1) +#define OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F BIT(2) #define OCCTX_TX_OFFLOAD_MBUF_NOFF_F BIT(3) -#define OCCTX_TX_MULTI_SEG_F BIT(15) /* Packet type table */ #define PTYPE_SIZE OCCTX_PKI_LTYPE_LAST +/* octeontx send header sub descriptor structure */ +RTE_STD_C11 +union octeontx_send_hdr_w0_u { + uint64_t u; + struct { + uint64_t total : 16; + uint64_t markptr : 8; + uint64_t l3ptr : 8; + uint64_t l4ptr : 8; + uint64_t ii : 1; + uint64_t shp_dis : 1; + uint64_t ckle : 1; + uint64_t cklf : 2; + uint64_t ckl3 : 1; + uint64_t ckl4 : 2; + uint64_t p : 1; + uint64_t format : 7; + uint64_t tstamp : 1; + uint64_t tso_eom : 1; + uint64_t df : 1; + uint64_t tso : 1; + uint64_t n2 : 1; + uint64_t scntn1 : 3; + }; +}; + +RTE_STD_C11 +union octeontx_send_hdr_w1_u { + uint64_t u; + struct { + uint64_t tso_mss : 14; + uint64_t shp_ra : 2; + uint64_t tso_sb : 8; + uint64_t leptr : 8; + uint64_t lfptr : 8; + uint64_t shp_chg : 9; + uint64_t tso_fn : 7; + uint64_t l2len : 8; + }; +}; + +struct octeontx_send_hdr_s { + union octeontx_send_hdr_w0_u w0; + union octeontx_send_hdr_w1_u w1; +}; + static const uint32_t __rte_cache_aligned ptype_table[PTYPE_SIZE][PTYPE_SIZE][PTYPE_SIZE] = { [LC_NONE][LE_NONE][LF_NONE] = RTE_PTYPE_UNKNOWN, @@ -182,6 +230,90 @@ octeontx_prefree_seg(struct rte_mbuf *m) return 1; } +static __rte_always_inline void +octeontx_tx_checksum_offload(uint64_t *cmd_buf, const uint16_t flags, + struct rte_mbuf *m) +{ + struct octeontx_send_hdr_s *send_hdr = + (struct octeontx_send_hdr_s *)cmd_buf; + uint64_t ol_flags = m->ol_flags; + + /* PKO Checksum L4 Algorithm Enumeration + * 0x0 - No checksum + * 0x1 - UDP L4 checksum + * 0x2 - TCP L4 checksum + * 0x3 - SCTP L4 checksum + */ + const uint8_t csum = (!(((ol_flags ^ PKT_TX_UDP_CKSUM) >> 52) & 0x3) + + (!(((ol_flags ^ PKT_TX_TCP_CKSUM) >> 52) & 0x3) * 2) + + (!(((ol_flags ^ PKT_TX_SCTP_CKSUM) >> 52) & 0x3) * 3)); + + const uint8_t is_tunnel_parsed = (!!(ol_flags & PKT_TX_TUNNEL_GTP) || + !!(ol_flags & PKT_TX_TUNNEL_VXLAN_GPE) || + !!(ol_flags & PKT_TX_TUNNEL_VXLAN) || + !!(ol_flags & PKT_TX_TUNNEL_GRE) || + !!(ol_flags & PKT_TX_TUNNEL_GENEVE) || + !!(ol_flags & PKT_TX_TUNNEL_IP) || + !!(ol_flags & PKT_TX_TUNNEL_IPIP)); + + const uint8_t csum_outer = (!!(ol_flags & PKT_TX_OUTER_UDP_CKSUM) || + !!(ol_flags & PKT_TX_TUNNEL_UDP)); + const uint8_t outer_l2_len = m->outer_l2_len; + const uint8_t l2_len = m->l2_len; + + if ((flags & OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F) && + (flags & OCCTX_TX_OFFLOAD_L3_L4_CSUM_F)) { + if (is_tunnel_parsed) { + /* Outer L3 */ + send_hdr->w0.l3ptr = outer_l2_len; + send_hdr->w0.l4ptr = outer_l2_len + m->outer_l3_len; + /* Set clk3 for PKO to calculate IPV4 header checksum */ + send_hdr->w0.ckl3 = !!(ol_flags & PKT_TX_OUTER_IPV4); + + /* Outer L4 */ + send_hdr->w0.ckl4 = csum_outer; + + /* Inner L3 */ + send_hdr->w1.leptr = send_hdr->w0.l4ptr + l2_len; + send_hdr->w1.lfptr = send_hdr->w1.leptr + m->l3_len; + /* Set clke for PKO to calculate inner IPV4 header + * checksum. + */ + send_hdr->w0.ckle = !!(ol_flags & PKT_TX_IPV4); + + /* Inner L4 */ + send_hdr->w0.cklf = csum; + } else { + /* Inner L3 */ + send_hdr->w0.l3ptr = l2_len; + send_hdr->w0.l4ptr = l2_len + m->l3_len; + /* Set clk3 for PKO to calculate IPV4 header checksum */ + send_hdr->w0.ckl3 = !!(ol_flags & PKT_TX_IPV4); + + /* Inner L4 */ + send_hdr->w0.ckl4 = csum; + } + } else if (flags & OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F) { + /* Outer L3 */ + send_hdr->w0.l3ptr = outer_l2_len; + send_hdr->w0.l4ptr = outer_l2_len + m->outer_l3_len; + /* Set clk3 for PKO to calculate IPV4 header checksum */ + send_hdr->w0.ckl3 = !!(ol_flags & PKT_TX_OUTER_IPV4); + + /* Outer L4 */ + send_hdr->w0.ckl4 = csum_outer; + } else if (flags & OCCTX_TX_OFFLOAD_L3_L4_CSUM_F) { + /* Inner L3 */ + send_hdr->w0.l3ptr = l2_len; + send_hdr->w0.l4ptr = l2_len + m->l3_len; + /* Set clk3 for PKO to calculate IPV4 header checksum */ + send_hdr->w0.ckl3 = !!(ol_flags & PKT_TX_IPV4); + + /* Inner L4 */ + send_hdr->w0.ckl4 = csum; + } +} + static __rte_always_inline uint16_t __octeontx_xmit_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, const uint16_t flag) @@ -192,6 +324,11 @@ __octeontx_xmit_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, cmd_buf[nb_desc++] = tx_pkt->data_len & 0xffff; cmd_buf[nb_desc++] = 0x0; + /* Enable tx checksum offload */ + if ((flag & OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F) || + (flag & OCCTX_TX_OFFLOAD_L3_L4_CSUM_F)) + octeontx_tx_checksum_offload(cmd_buf, flag, tx_pkt); + /* SEND_HDR[DF] bit controls if buffer is to be freed or * not, as SG_DESC[I] and SEND_HDR[II] are clear. */ @@ -230,6 +367,11 @@ __octeontx_xmit_mseg_prepare(struct rte_mbuf *tx_pkt, uint64_t *cmd_buf, cmd_buf[nb_desc++] = tx_pkt->pkt_len & 0xffff; cmd_buf[nb_desc++] = 0x0; + /* Enable tx checksum offload */ + if ((flag & OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F) || + (flag & OCCTX_TX_OFFLOAD_L3_L4_CSUM_F)) + octeontx_tx_checksum_offload(cmd_buf, flag, tx_pkt); + do { m_next = tx_pkt->next; /* To handle case where mbufs belong to diff pools, like @@ -305,13 +447,45 @@ __octeontx_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t octeontx_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); +#define L3L4CSUM_F OCCTX_TX_OFFLOAD_L3_L4_CSUM_F +#define OL3OL4CSUM_F OCCTX_TX_OFFLOAD_OL3_OL4_CSUM_F #define NOFF_F OCCTX_TX_OFFLOAD_MBUF_NOFF_F #define MULT_F OCCTX_TX_MULTI_SEG_F -/* [NOFF] [MULTI_SEG] */ -#define OCCTX_TX_FASTPATH_MODES \ -T(no_offload, 0, 0, 4, OCCTX_TX_OFFLOAD_NONE) \ -T(mseg, 0, 1, 14, MULT_F) \ -T(noff, 1, 0, 4, NOFF_F) \ -T(noff_mseg, 1, 1, 14, NOFF_F | MULT_F) + +/* [L3L4CSUM_F] [OL3OL4CSUM_F] [NOFF] [MULTI_SEG] */ +#define OCCTX_TX_FASTPATH_MODES \ +T(no_offload, 0, 0, 0, 0, 4, \ + OCCTX_TX_OFFLOAD_NONE) \ +T(mseg, 0, 0, 0, 1, 14, \ + MULT_F) \ +T(l3l4csum, 0, 0, 1, 0, 4, \ + L3L4CSUM_F) \ +T(l3l4csum_mseg, 0, 0, 1, 1, 14, \ + L3L4CSUM_F | MULT_F) \ +T(ol3ol4csum, 0, 1, 0, 0, 4, \ + OL3OL4CSUM_F) \ +T(ol3l4csum_mseg, 0, 1, 0, 1, 14, \ + OL3OL4CSUM_F | MULT_F) \ +T(ol3l4csum_l3l4csum, 0, 1, 1, 0, 4, \ + OL3OL4CSUM_F | L3L4CSUM_F) \ +T(ol3l4csum_l3l4csum_mseg, 0, 1, 1, 1, 14, \ + OL3OL4CSUM_F | L3L4CSUM_F | MULT_F) \ +T(noff, 1, 0, 0, 0, 4, \ + NOFF_F) \ +T(noff_mseg, 1, 0, 0, 1, 14, \ + NOFF_F | MULT_F) \ +T(noff_l3l4csum, 1, 0, 1, 0, 4, \ + NOFF_F | L3L4CSUM_F) \ +T(noff_l3l4csum_mseg, 1, 0, 1, 1, 14, \ + NOFF_F | L3L4CSUM_F | MULT_F) \ +T(noff_ol3ol4csum, 1, 1, 0, 0, 4, \ + NOFF_F | OL3OL4CSUM_F) \ +T(noff_ol3ol4csum_mseg, 1, 1, 0, 1, 14, \ + NOFF_F | OL3OL4CSUM_F | MULT_F) \ +T(noff_ol3ol4csum_l3l4csum, 1, 1, 1, 0, 4, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F) \ +T(noff_ol3ol4csum_l3l4csum_mseg, 1, 1, 1, 1, 14, \ + NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F | \ + MULT_F) #endif /* __OCTEONTX_RXTX_H__ */ -- 2.18.0