From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1B20BA0559; Mon, 16 Mar 2020 10:34:33 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 107E91C07B; Mon, 16 Mar 2020 10:34:29 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 8B5F91C067 for ; Mon, 16 Mar 2020 10:34:27 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 02G9UQ2k015065; Mon, 16 Mar 2020 02:34:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=f60ZcwX1ztmlecR/rg3D1JpIYPdUsKAB6UjfyIlSurs=; b=j6v11FOYFlGfizzaU6G/gi3AlF0BLRvSuSbquXD/FXSHSThpVPweIRNa7dBFw8PJ+5FX pujA9cjru6v+36gPk1nb68oTtjJtME/QOAMCHSnEEzlLUgtWJ2srdw7Hiu8xxKJD5+o8 faazES4ETdhKeDV933oTrkj/oV5GqyiJfc7Tqgsd8dFYEG9mx4m5tCmx4Fi7wv25loJk sg8Gg8dnJIw/RLboG3l1Arqc7bY+l6fgjByyHp4oAqy9xVAn8w4Cxn1ZuQwDHDiymOyr TdPS8hFaXZRlHW5VyikuQQSr8E5DT2EzTt5jZYF8WBhYnZMHgqXy6qCwXkyvYpekw+lp bA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2yrxsme4xw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 16 Mar 2020 02:34:26 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 02:34:24 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Mar 2020 02:34:23 -0700 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.172) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Mon, 16 Mar 2020 02:34:23 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EAGkK+hhW56Sek9uHFTZsjh5d8CkGUrnun0OsrXA7oHmAcQ/erOgpc/BVZH5/ge05J0grFd359ELEvK05vxpMzTLhjdYDkh6OeQ8xGnxKmVvCarFYAQzXcTDSjCwiaEJmWtPcE3TJOwEiUWeJ4EoawhS6j1VYh1ZLLm7otKjZX9QiFlpMfKCL+y7GGUA1RA44OgFPWWIdwGMU9EDZvicCAQOk+ySA2d+8b+ONZqHqIAgnPYpFrJcx7QnTlcThDo3Jh0zt1UKf5MZxy3YFdUnbW7hlca9twPU47lZw1CXX51WyrBAYgXPmtkHUROacD9YvJhwg6UXecR1A6HNS1WSaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f60ZcwX1ztmlecR/rg3D1JpIYPdUsKAB6UjfyIlSurs=; b=Sr/8q8LnrVhW6rTNKqJXgEuW15W5pkeLD5FjVmiV7UDkvPPuPelHgvsoIOl6M9QIjuojB+wlCp6N/0P5s7XPDDLpujABsxNJOF2NIbggurwEF8yn4aBKraiMv7fshGILI7QR+zYr3ZIibKKFnLXxnAvkgDa5g4nEAkHyaZjp1j0JO8Rt3/OeFSBrN9LvZ7dhIVpatx1LGT6vyQJHptS3l7L9p5M5+Ce3pN1p5HxYeSwcxVKRYb7NGww9W8doxclfAd87g3Ov6qncPEUt5hHPTqAJbigV4TP5oV+hsNJ2T3WcLRWICcjBcyTxssPpPCm0gesKuOT1Fg5AuvorTLTG7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f60ZcwX1ztmlecR/rg3D1JpIYPdUsKAB6UjfyIlSurs=; b=CTO3Ca7aokbFZg1vtd5vSUweM4/PsxtG5+JhIOFa/VJJufDGqIlTz1K8y6d7APUlgv9zqRKEHINyRyVusjCq9eAVvNDsrMTDpF9U5CQU1bWsq/kMC8mQEake+Onmif+NXyGNaDsUhJIbDxls9ddMG5/1ZIMyPVYtPu4DYbwVI7g= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3117.namprd18.prod.outlook.com (2603:10b6:208:162::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.14; Mon, 16 Mar 2020 09:34:22 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::f829:3e55:94f6:4efb%5]) with mapi id 15.20.2814.021; Mon, 16 Mar 2020 09:34:22 +0000 From: Harman Kalra To: , , CC: , , Harman Kalra Date: Mon, 16 Mar 2020 15:03:37 +0530 Message-ID: <1584351224-23500-2-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1584351224-23500-1-git-send-email-hkalra@marvell.com> References: <1584351224-23500-1-git-send-email-hkalra@marvell.com> Content-Type: text/plain X-ClientProxiedBy: BMXPR01CA0006.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::16) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BMXPR01CA0006.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2814.13 via Frontend Transport; Mon, 16 Mar 2020 09:34:20 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8445be8e-a56c-45ff-c8f0-08d7c98d355e X-MS-TrafficTypeDiagnostic: MN2PR18MB3117: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:142; X-Forefront-PRVS: 03449D5DD1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(136003)(376002)(39860400002)(346002)(396003)(199004)(52116002)(7696005)(478600001)(4326008)(8676002)(8936002)(107886003)(26005)(66556008)(30864003)(2906002)(86362001)(5660300002)(36756003)(55236004)(16526019)(2616005)(956004)(66946007)(6486002)(66476007)(81156014)(81166006)(6666004)(186003)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3117; H:MN2PR18MB2848.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; Received-SPF: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OvqVKCMPGrOaIoYCXSwFnHv4PtAPtqtBTmla11Dn8+eW257NdMaLbe3vjQmxoyCXkfZR9iRqQufgXR6fjqyxQFtULOSbWy4knZICzORr4LyheT+rzB6hmc6NhgxuZjmAf98IZ5JSKT4X+hU5xZPXZkVaWQms6fkhqi9bGedcqbIVsUnhRsRwyKnMNafGZBAGvJsD6hdbkYo82I71aZKXTEKdmVHSM8xEx3hqXjnqfSzxnWZTTnC/JNWBdluFZlpZi9xQ9cLfZulJqGSDGEkHE/1bzKpbBI0rdXhODnuG3L36S4aAoo2uiCwu8p1DZIGnywOKV7YshsO89uVhlNRDoag62D4FYn40JjCUot6nZL+PV0M/v3IjWYiVChpJU2TxrImEYDMqBF9yDhRe72pGdU2ZRSMCWteWbflKr3iny0qJif2O1XY3IHb+574jp95F X-MS-Exchange-AntiSpam-MessageData: GG6edHCDMdToVYNvVV1/RwPHWuyAmnzxBdOH2h4n4F/WLr7BWlo7gHbg6y5bh3M5R+U+wUjSxkkMXR4I80s5hK7rfBXzK+keRQ6LdTCfSLHv3DsjTCjtDyT8GVvNcOgWkZkINNGDbiKOGG+YlGfSOA== X-MS-Exchange-CrossTenant-Network-Message-Id: 8445be8e-a56c-45ff-c8f0-08d7c98d355e X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Mar 2020 09:34:22.1059 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pFQEeIL5SMbKRLYJLl/oHp2mnWGPw1DTtvqy61+xj2t2sTFHcyAojf74eLhSSQOXIsarpBvtcfbOiYZlRJou9Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3117 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-03-16_02:2020-03-12, 2020-03-16 signatures=0 Subject: [dpdk-dev] [PATCH 1/8] net/octeontx: add multi segment support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding multi segment support to the octeontx PMD. Also adding the logic to share rx/tx ofloads with the eventdev code. Signed-off-by: Harman Kalra --- doc/guides/nics/features/octeontx.ini | 1 + doc/guides/nics/octeontx.rst | 1 + drivers/event/octeontx/ssovf_worker.c | 2 +- drivers/net/octeontx/base/octeontx_pki_var.h | 32 ++++++ drivers/net/octeontx/base/octeontx_pkovf.h | 5 +- drivers/net/octeontx/octeontx_ethdev.c | 98 +++++++++++++++++- drivers/net/octeontx/octeontx_ethdev.h | 13 ++- drivers/net/octeontx/octeontx_rxtx.c | 30 +++++- drivers/net/octeontx/octeontx_rxtx.h | 102 +++++++++++++++---- 9 files changed, 253 insertions(+), 31 deletions(-) diff --git a/doc/guides/nics/features/octeontx.ini b/doc/guides/nics/features/octeontx.ini index 323befe59..19caee61b 100644 --- a/doc/guides/nics/features/octeontx.ini +++ b/doc/guides/nics/features/octeontx.ini @@ -9,6 +9,7 @@ Link status = Y Lock-free Tx queue = Y Queue start/stop = P Jumbo frame = Y +Scattered Rx = Y Promiscuous mode = Y Unicast MAC filter = Y CRC offload = Y diff --git a/doc/guides/nics/octeontx.rst b/doc/guides/nics/octeontx.rst index 8fc53810b..0c36e10cb 100644 --- a/doc/guides/nics/octeontx.rst +++ b/doc/guides/nics/octeontx.rst @@ -20,6 +20,7 @@ Features of the OCTEON TX Ethdev PMD are: - Promiscuous mode - Port hardware statistics - Jumbo frames +- Scatter-Gather IO support - Link state information - SR-IOV VF - Multiple queues for TX diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index d940b5dd6..f11b9d8c4 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -300,7 +300,7 @@ sso_event_tx_adapter_enqueue(void *port, dq = &txq->dq; if (__octeontx_xmit_pkts(dq->lmtline_va, dq->ioreg_va, dq->fc_status_va, - m) < 0) + m, OCCTX_TX_OFFLOAD_NONE) < 0) return 0; return 1; diff --git a/drivers/net/octeontx/base/octeontx_pki_var.h b/drivers/net/octeontx/base/octeontx_pki_var.h index f4661d24e..4445369ce 100644 --- a/drivers/net/octeontx/base/octeontx_pki_var.h +++ b/drivers/net/octeontx/base/octeontx_pki_var.h @@ -215,4 +215,36 @@ enum lf_type_e { LF_UDP_VXLAN = OCCTX_PKI_LTYPE_UDP_VXLAN, LF_NVGRE = OCCTX_PKI_LTYPE_NVGRE, }; + +/* Word 0 of HW segment buflink structure */ +typedef union octtx_pki_buflink_w0_u { + uint64_t v; + struct { + uint64_t size:16; + uint64_t rsvd1:15; + uint64_t invfree:1; + /** Aura number of the next segment */ + uint64_t aura:16; + uint64_t sw:9; + uint64_t later_invfree:1; + uint64_t rsvd2:5; + /** 1 if aura number is set */ + uint64_t has_aura:1; + } s; +} octtx_pki_buflink_w0_t; + +/* Word 1 of HW segment buflink structure */ +typedef union octtx_pki_buflink_w1_u { + uint64_t v; + struct { + uint64_t addr; + } s; +} octtx_pki_buflink_w1_t; + +/* HW structure linking packet segments into singly linked list */ +typedef struct octtx_pki_buflink_s { + octtx_pki_buflink_w0_t w0; /* Word 0 of the buflink */ + octtx_pki_buflink_w1_t w1; /* Word 1 of the buflink */ +} octtx_pki_buflink_t; + #endif /* __OCTEONTX_PKI_VAR_H__ */ diff --git a/drivers/net/octeontx/base/octeontx_pkovf.h b/drivers/net/octeontx/base/octeontx_pkovf.h index 4208ef880..4e0bb7c2e 100644 --- a/drivers/net/octeontx/base/octeontx_pkovf.h +++ b/drivers/net/octeontx/base/octeontx_pkovf.h @@ -36,7 +36,10 @@ /* pko_send_hdr_s + pko_send_link */ #define PKO_CMD_SZ (2 << 1) -#define PKO_SEND_GATHER_SUBDC (0x0ull << 60) +#define PKO_SEND_BUFLINK_SUBDC (0x0ull << 60) +#define PKO_SEND_BUFLINK_LDTYPE(x) ((x) << 58) +#define PKO_SEND_BUFLINK_GAUAR(x) ((x) << 24) +#define PKO_SEND_GATHER_SUBDC (0x2ull << 60) #define PKO_SEND_GATHER_LDTYPE(x) ((x) << 58) #define PKO_SEND_GATHER_GAUAR(x) ((x) << 24) diff --git a/drivers/net/octeontx/octeontx_ethdev.c b/drivers/net/octeontx/octeontx_ethdev.c index e8aa4ec78..24c4e83a9 100644 --- a/drivers/net/octeontx/octeontx_ethdev.c +++ b/drivers/net/octeontx/octeontx_ethdev.c @@ -24,6 +24,10 @@ #include "octeontx_rxtx.h" #include "octeontx_logs.h" +struct evdev_priv_data { + OFFLOAD_FLAGS; /*Sequence should not be changed */ +} __rte_cache_aligned; + struct octeontx_vdev_init_params { uint8_t nr_port; }; @@ -257,6 +261,43 @@ devconf_set_default_sane_values(struct rte_event_dev_config *dev_conf, info->max_num_events; } +static uint16_t +octeontx_tx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct octeontx_nic *nic = octeontx_pmd_priv(eth_dev); + uint16_t flags = 0; + + /* Created function for supoorting future offloads */ + if (nic->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) + flags |= OCCTX_TX_MULTI_SEG_F; + + return flags; +} + +static uint16_t +octeontx_rx_offload_flags(struct rte_eth_dev *eth_dev) +{ + struct octeontx_nic *nic = octeontx_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_eth_conf *conf = &data->dev_conf; + struct rte_eth_rxmode *rxmode = &conf->rxmode; + uint16_t flags = 0; + + if (rxmode->mq_mode == ETH_MQ_RX_RSS) + flags |= OCCTX_RX_OFFLOAD_RSS_F; + + if (nic->rx_offloads & DEV_RX_OFFLOAD_SCATTER) { + flags |= OCCTX_RX_MULTI_SEG_F; + eth_dev->data->scattered_rx = 1; + /* If scatter mode is enabled, TX should also be in multi + * seg mode, else memory leak will occur + */ + nic->tx_offloads |= DEV_TX_OFFLOAD_MULTI_SEGS; + } + + return flags; +} + static int octeontx_dev_configure(struct rte_eth_dev *dev) { @@ -321,6 +362,11 @@ octeontx_dev_configure(struct rte_eth_dev *dev) nic->pki.hash_enable = true; nic->pki.initialized = false; + nic->rx_offloads |= rxmode->offloads; + nic->tx_offloads |= txmode->offloads; + nic->rx_offload_flags |= octeontx_rx_offload_flags(dev); + nic->tx_offload_flags |= octeontx_tx_offload_flags(dev); + return 0; } @@ -359,6 +405,51 @@ octeontx_dev_close(struct rte_eth_dev *dev) dev->rx_pkt_burst = NULL; } +static int +octeontx_recheck_rx_offloads(struct octeontx_rxq *rxq) +{ + struct rte_eth_dev *eth_dev = rxq->eth_dev; + struct octeontx_nic *nic = octeontx_pmd_priv(eth_dev); + struct rte_eth_dev_data *data = eth_dev->data; + struct rte_pktmbuf_pool_private *mbp_priv; + struct evdev_priv_data *evdev_priv; + struct rte_eventdev *dev; + uint32_t buffsz; + + /* Get rx buffer size */ + mbp_priv = rte_mempool_get_priv(rxq->pool); + buffsz = mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM; + + /* Setup scatter mode if needed by jumbo */ + if (data->dev_conf.rxmode.max_rx_pkt_len > buffsz) { + nic->rx_offloads |= DEV_RX_OFFLOAD_SCATTER; + nic->rx_offload_flags |= octeontx_rx_offload_flags(eth_dev); + nic->tx_offload_flags |= octeontx_tx_offload_flags(eth_dev); + } + + /* Sharing offload flags via eventdev priv region */ + dev = &rte_eventdevs[rxq->evdev]; + evdev_priv = dev->data->dev_private; + evdev_priv->rx_offload_flags = nic->rx_offload_flags; + evdev_priv->tx_offload_flags = nic->tx_offload_flags; + + return 0; +} + +static void +octeontx_set_tx_function(struct rte_eth_dev *dev) +{ + struct octeontx_nic *nic = octeontx_pmd_priv(dev); + + const eth_tx_burst_t tx_burst_func[2] = { + [0] = octeontx_xmit_pkts, + [1] = octeontx_xmit_pkts_mseg, + }; + + dev->tx_pkt_burst = + tx_burst_func[!!(nic->tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS)]; +} + static int octeontx_dev_start(struct rte_eth_dev *dev) { @@ -371,7 +462,7 @@ octeontx_dev_start(struct rte_eth_dev *dev) /* * Tx start */ - dev->tx_pkt_burst = octeontx_xmit_pkts; + octeontx_set_tx_function(dev); ret = octeontx_pko_channel_start(nic->base_ochan); if (ret < 0) { octeontx_log_err("fail to conf VF%d no. txq %d chan %d ret %d", @@ -599,10 +690,8 @@ octeontx_dev_default_mac_addr_set(struct rte_eth_dev *dev, struct rte_ether_addr *addr) { struct octeontx_nic *nic = octeontx_pmd_priv(dev); - uint8_t prom_mode = dev->data->promiscuous; int ret; - dev->data->promiscuous = 0; ret = octeontx_bgx_port_mac_set(nic->port_id, addr->addr_bytes); if (ret == 0) { /* Update same mac address to BGX CAM table */ @@ -610,7 +699,6 @@ octeontx_dev_default_mac_addr_set(struct rte_eth_dev *dev, 0); } if (ret < 0) { - dev->data->promiscuous = prom_mode; octeontx_log_err("failed to set MAC address on port %d", nic->port_id); } @@ -977,7 +1065,9 @@ octeontx_dev_rx_queue_setup(struct rte_eth_dev *dev, uint16_t qidx, rxq->evdev = nic->evdev; rxq->ev_queues = ev_queues; rxq->ev_ports = ev_ports; + rxq->pool = mb_pool; + octeontx_recheck_rx_offloads(rxq); dev->data->rx_queues[qidx] = rxq; dev->data->rx_queue_state[qidx] = RTE_ETH_QUEUE_STATE_STOPPED; return 0; diff --git a/drivers/net/octeontx/octeontx_ethdev.h b/drivers/net/octeontx/octeontx_ethdev.h index 50fae35d9..10da6a2a0 100644 --- a/drivers/net/octeontx/octeontx_ethdev.h +++ b/drivers/net/octeontx/octeontx_ethdev.h @@ -29,8 +29,12 @@ #define OCTEONTX_MAX_BGX_PORTS 4 #define OCTEONTX_MAX_LMAC_PER_BGX 4 -#define OCTEONTX_RX_OFFLOADS DEV_RX_OFFLOAD_CHECKSUM -#define OCTEONTX_TX_OFFLOADS DEV_TX_OFFLOAD_MT_LOCKFREE +#define OCTEONTX_RX_OFFLOADS (DEV_RX_OFFLOAD_CHECKSUM | \ + DEV_RX_OFFLOAD_SCATTER | \ + DEV_RX_OFFLOAD_JUMBO_FRAME) + +#define OCTEONTX_TX_OFFLOADS (DEV_TX_OFFLOAD_MT_LOCKFREE | \ + DEV_TX_OFFLOAD_MULTI_SEGS) static inline struct octeontx_nic * octeontx_pmd_priv(struct rte_eth_dev *dev) @@ -73,6 +77,10 @@ struct octeontx_nic { uint16_t ev_queues; uint16_t ev_ports; + uint64_t rx_offloads; + uint16_t rx_offload_flags; + uint64_t tx_offloads; + uint16_t tx_offload_flags; } __rte_cache_aligned; struct octeontx_txq { @@ -88,6 +96,7 @@ struct octeontx_rxq { struct rte_eth_dev *eth_dev; uint16_t ev_queues; uint16_t ev_ports; + struct rte_mempool *pool; } __rte_cache_aligned; #endif /* __OCTEONTX_ETHDEV_H__ */ diff --git a/drivers/net/octeontx/octeontx_rxtx.c b/drivers/net/octeontx/octeontx_rxtx.c index 1e201f322..8f6d14b5f 100644 --- a/drivers/net/octeontx/octeontx_rxtx.c +++ b/drivers/net/octeontx/octeontx_rxtx.c @@ -32,8 +32,34 @@ octeontx_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) rte_cio_wmb(); while (count < nb_pkts) { res = __octeontx_xmit_pkts(dq->lmtline_va, dq->ioreg_va, - dq->fc_status_va, - tx_pkts[count]); + dq->fc_status_va, tx_pkts[count], + OCCTX_TX_OFFLOAD_NONE); + if (res < 0) + break; + + count++; + } + + return count; /* return number of pkts transmitted */ +} + +uint16_t __hot +octeontx_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts) +{ + int count; + struct octeontx_txq *txq = tx_queue; + octeontx_dq_t *dq = &txq->dq; + int res; + + count = 0; + + rte_cio_wmb(); + while (count < nb_pkts) { + res = __octeontx_xmit_pkts(dq->lmtline_va, dq->ioreg_va, + dq->fc_status_va, tx_pkts[count], + OCCTX_TX_OFFLOAD_NONE | + OCCTX_TX_MULTI_SEG_F); if (res < 0) break; diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index d0d73b304..562268f16 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -11,6 +11,19 @@ #define __hot __attribute__((hot)) #endif +#define OFFLOAD_FLAGS \ + uint16_t rx_offload_flags; \ + uint16_t tx_offload_flags + +#define BIT(nr) (1UL << (nr)) + +#define OCCTX_RX_OFFLOAD_NONE (0) +#define OCCTX_RX_OFFLOAD_RSS_F BIT(0) +#define OCCTX_RX_MULTI_SEG_F BIT(15) + +#define OCCTX_TX_OFFLOAD_NONE (0) + +#define OCCTX_TX_MULTI_SEG_F BIT(15) /* Packet type table */ #define PTYPE_SIZE OCCTX_PKI_LTYPE_LAST @@ -102,33 +115,76 @@ ptype_table[PTYPE_SIZE][PTYPE_SIZE][PTYPE_SIZE] = { static __rte_always_inline int __octeontx_xmit_pkts(void *lmtline_va, void *ioreg_va, int64_t *fc_status_va, - struct rte_mbuf *tx_pkt) + struct rte_mbuf *tx_pkt, const uint16_t flag) { - uint64_t cmd_buf[4] __rte_cache_aligned; - uint16_t gaura_id; + uint8_t sz = (4 + (!!(flag & OCCTX_TX_MULTI_SEG_F) * 10)); + /* Max size of PKO SEND desc is 112 bytes*/ + uint64_t cmd_buf[sz] __rte_cache_aligned; + uint8_t nb_segs, nb_desc = 0; + uint16_t gaura_id, len = 0; + struct rte_mbuf *m_next = NULL; if (unlikely(*((volatile int64_t *)fc_status_va) < 0)) return -ENOSPC; - /* Get the gaura Id */ - gaura_id = octeontx_fpa_bufpool_gpool((uintptr_t)tx_pkt->pool->pool_id); - - /* Setup PKO_SEND_HDR_S */ - cmd_buf[0] = tx_pkt->data_len & 0xffff; - cmd_buf[1] = 0x0; - /* Set don't free bit if reference count > 1 */ - if (rte_mbuf_refcnt_read(tx_pkt) > 1) - cmd_buf[0] |= (1ULL << 58); /* SET DF */ - - /* Setup PKO_SEND_GATHER_S */ - cmd_buf[(1 << 1) | 1] = rte_mbuf_data_iova(tx_pkt); - cmd_buf[(1 << 1) | 0] = PKO_SEND_GATHER_SUBDC | - PKO_SEND_GATHER_LDTYPE(0x1ull) | - PKO_SEND_GATHER_GAUAR((long)gaura_id) | - tx_pkt->data_len; - - octeontx_reg_lmtst(lmtline_va, ioreg_va, cmd_buf, PKO_CMD_SZ); + if (flag & OCCTX_TX_MULTI_SEG_F) { + nb_segs = tx_pkt->nb_segs; + /* Setup PKO_SEND_HDR_S */ + cmd_buf[nb_desc++] = tx_pkt->pkt_len & 0xffff; + cmd_buf[nb_desc++] = 0x0; + + do { + m_next = tx_pkt->next; + /* To handle case where mbufs belong to diff pools, like + * fragmentation + */ + gaura_id = octeontx_fpa_bufpool_gpool((uintptr_t) + tx_pkt->pool->pool_id); + + /* Setup PKO_SEND_GATHER_S */ + cmd_buf[nb_desc] = PKO_SEND_GATHER_SUBDC | + PKO_SEND_GATHER_LDTYPE(0x1ull) | + PKO_SEND_GATHER_GAUAR((long) + gaura_id) | + tx_pkt->data_len; + /* Mark mempool object as "put" since it is freed by + * PKO. + */ + if (!(cmd_buf[nb_desc] & (1ULL << 57))) { + tx_pkt->next = NULL; + __mempool_check_cookies(tx_pkt->pool, + (void **)&tx_pkt, 1, 0); + } + nb_desc++; + + cmd_buf[nb_desc++] = rte_mbuf_data_iova(tx_pkt); + + nb_segs--; + len += tx_pkt->data_len; + tx_pkt = m_next; + } while (nb_segs); + } else { + /* Setup PKO_SEND_HDR_S */ + cmd_buf[nb_desc++] = tx_pkt->data_len & 0xffff; + cmd_buf[nb_desc++] = 0x0; + + /* Mark mempool object as "put" since it is freed by PKO */ + if (!(cmd_buf[0] & (1ULL << 58))) + __mempool_check_cookies(tx_pkt->pool, (void **)&tx_pkt, + 1, 0); + /* Get the gaura Id */ + gaura_id = octeontx_fpa_bufpool_gpool((uintptr_t) + tx_pkt->pool->pool_id); + + /* Setup PKO_SEND_BUFLINK_S */ + cmd_buf[nb_desc++] = PKO_SEND_BUFLINK_SUBDC | + PKO_SEND_BUFLINK_LDTYPE(0x1ull) | + PKO_SEND_BUFLINK_GAUAR((long)gaura_id) | + tx_pkt->data_len; + cmd_buf[nb_desc++] = rte_mbuf_data_iova(tx_pkt); + } + octeontx_reg_lmtst(lmtline_va, ioreg_va, cmd_buf, nb_desc); return 0; } @@ -136,6 +192,10 @@ __octeontx_xmit_pkts(void *lmtline_va, void *ioreg_va, int64_t *fc_status_va, uint16_t octeontx_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts); +uint16_t +octeontx_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, + uint16_t nb_pkts); + uint16_t octeontx_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts); -- 2.18.0