From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18986A00BE; Tue, 28 Apr 2020 14:41:31 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EACF91D5CC; Tue, 28 Apr 2020 14:41:30 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 6BEFB1D5CC for ; Tue, 28 Apr 2020 14:41:28 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 03SCehgP021441 for ; Tue, 28 Apr 2020 05:41:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=6LWAGMT6Zmc9L0tUS27NEjgU3NBOpWauAXa3jGISACY=; b=MW/+MPaDbi5l23oDGjXJ9RY+o7EHxRaBDfLlJUZ1Opr7Rg6pBeXaaU1Ns7t8eFRbSuuT vo7eb6Yb1CdY42YRVLaMUYtp8A5sStCHhMNJWs4DMuv78UZvLLPm3DTPwNeHPg+/XxeI 4yfYKm/Jcl5vjpmWNP5galEtDUFkc8hEL0ayOJqz4qkyKuar7M6pI08TUYhq1bxSu0Na /MLwFQCstKIhXU3K8TvKD/dLhRUsipM/8R4LWzwR9qChhcMkZZYJRX1T0NfIWM1c6pfw 3KTBYTsF3MKAdGyM9fpT8yjqvu9DQv/TvySHsWRisgULu/28G89kacxDwX/w8NbzG9pl Aw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 30mjjqc7jr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 28 Apr 2020 05:41:27 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 28 Apr 2020 05:41:25 -0700 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (104.47.37.56) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Tue, 28 Apr 2020 05:41:25 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oI+tiK/yeUsW5d51aUP2EKzIoqvo/1LaghIFS4zANTipW/XVh9d289rWGTsu42FWyrFnrCvHdrlU1TuZkAoeojslybRDkNRZ8N2p43LkoAVJLodNC7RGbkpT6+cxi/+rw6lfblnASFWGAjIAFhjn8aTfivy1M4GmV0jhclF6Ovu09QsuoGiWw8Wu7gNIct+6vbagoOan4WdY30ip0ttPWQrPOGLp2UmKf1YrSnbXPTQ6azECW0s/WPdnSB/92Gq4KIvP1rs//4yxqu0Ijupe5Q0hJ+n2o6WJ9S/BWoG1aeb4AJ3LOubTg/vPQId/38EgSD9KkVBnzkZNM8ns+Ipiqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6LWAGMT6Zmc9L0tUS27NEjgU3NBOpWauAXa3jGISACY=; b=ZPNxPiEH7cQlf7KaLEs5ItCXodXVG0qr+bvZeRQPCGXnQDKqqi6AZV/6vI8VchZTLGmLIf6T7F/Dx4PzdYhmgbn2rOh9gwDFbck62fCngQnpMCZWUpl+Clajr8FY5bbl0HciwGS3lZpwAwrNa6Ld9K4w+2hUBP0RN0HJljaDVtNr/mRIMCJy4Kbnsc1w9xaaqRrXsZPgZAp4ZyokTOJpSM+rJnCOzZs5OSgkaYHKCJVmMH8pPLGa9eL9mlaLovBXUp9BNXLTyPnoeS+NFwuY31zr3BtLs09KzhxwT2cyLdmzo5ySm29+wxS5wn7eQhrYaduezvk9wly/a+UVKhWDgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6LWAGMT6Zmc9L0tUS27NEjgU3NBOpWauAXa3jGISACY=; b=iYR/7bxO5/3/wDpenJ18yMsle92C4bHaJwGGqCOFcoPoEr5v5Y+RZ7KM/CRUp/Jsx0hEOllL9pYjANCnqDUuc4WFAGf2+orqlLgTaXNZkm9n0Z8oVIk6TS/sI/KATqhN2paUDvvgPXiaFiXysH665N4Wpj7zvAcj59R3f94Ys+Y= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3070.namprd18.prod.outlook.com (2603:10b6:208:ff::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.22; Tue, 28 Apr 2020 12:41:24 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::a8b1:45c4:b84:9c02]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::a8b1:45c4:b84:9c02%7]) with mapi id 15.20.2937.026; Tue, 28 Apr 2020 12:41:24 +0000 From: Harman Kalra To: CC: , Harman Kalra Date: Tue, 28 Apr 2020 18:10:09 +0530 Message-ID: <1588077612-15905-2-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588077612-15905-1-git-send-email-hkalra@marvell.com> References: <1588077612-15905-1-git-send-email-hkalra@marvell.com> Content-Type: text/plain X-ClientProxiedBy: BMXPR01CA0031.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:c::17) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BMXPR01CA0031.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2937.13 via Frontend Transport; Tue, 28 Apr 2020 12:41:23 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 428741d4-75cc-4f07-2a14-08d7eb717646 X-MS-TrafficTypeDiagnostic: MN2PR18MB3070: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:312; X-Forefront-PRVS: 0387D64A71 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR18MB2848.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(136003)(396003)(39860400002)(366004)(376002)(2906002)(52116002)(81156014)(7696005)(36756003)(316002)(2616005)(6636002)(5660300002)(8676002)(55236004)(34206002)(30864003)(86362001)(4326008)(8936002)(6666004)(956004)(37006003)(16526019)(107886003)(6486002)(66476007)(66556008)(66946007)(478600001)(186003)(26005); DIR:OUT; SFP:1101; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kfui5hshnIM2IFqyY2gKFikzYzfa+HMI12zHbMtO0wZ9PO2TuSn3rcp6HjYdBvMezp5XWax0s0Cx8EzWMaulLkpwpncvYlUJYJN0xroBsjyGGa1hZNlPViw2Z7Eej2+/S0nOfaxZZ1d9HWanqxXbZCAdhpdSZ640RhqL2+vKZoeyC1jE+DVcc3NxfwHWWsy5LV5htC5nDiefPsC9T3bAbGqWtpT7MzpsKeREil6N8YXPnb9s4AMQJN0fo+EUDuHfnj52zF33dzewUPQyal6vKE5aiqWxMvXQSmfOBNNeU4I41WrZIraKySXIbt85wFtwpv3flMafWbpZNUx5pk+ihGsrg+E3QI8Fa/uk2mLbpUIaXIZEHPtEg1zzzAPUsfoJlxplJo2B3f0dWk7j0lo4PoXQXijpkfhtwbQWq9QREi/uAlEgpuOwWLqyne60F3Qg X-MS-Exchange-AntiSpam-MessageData: AukVDoItV89wN91S/FsQBGtK8snJXpk+3Y3rP+DZxuTHIOjGnU/HN1aHZdPXkneuUpL2B8bEWk/fBktsRhgHayeOq+Gn6gM13S4G3Ege1SZGqN7rZYB2H8xXLFcZkw44vQ/wguK4Yu08Imp+ItcczhnnS2vPpEhIRVKZxm/0HWTfC4qvIsdDhzlRR9HcQb3dWNpcoDG49s5hQY35pC70NW4kE/h9XbfC7bjJRxaSNBhRGoHFBi7vE/ksNzqTLS5BtyScjzjh3SLRLCpLBPDuT/modHter1v8EhtY4POKyuGSYEAS3IOqmMV160AHbYrtwSpqjUM7woV1iOHki4mdPOJ6Q8h72abCuQpUk/aOHJmWNLARGOY9Iock/4cEypmrZlXfx7durd6IXPmu+4yuiqyZMmvVgt70LNnlDLF+ftTrNtZLUyJ/dQV5Pfb/z7pQbiKaaOD7HU2380+QxyVIgsAXyyi1cM9vB87cR2Gc/x5lA4GRZ2o7kj7rHbk1aqh4HIM3tAv1VJdx8WHMotd56xLKbjtianex00qknuo3ADjYzKnDwUdg62P0nWcdc29oYDGQsb93dT7o4aq2dOfg6yRck36geK/3uJCUQTor7ZjFoOisEnTCucFiVf3FfwM+noJSCm/ZJrSY2aylG5gS43bqD6cyeGUlR9/qcbiMqjwUy0OQfPm3B3D50Le4Eo8oM6XJR+YP60+PBi1ZjuYOzkgur1Af35mxzrVFYnuP2RZYcJbBYNfC2kr1a4Zod57OS3+HVsY0a8qM7YJ7IfS/YzwqDi9w86QFxerncYNr+E0= X-MS-Exchange-CrossTenant-Network-Message-Id: 428741d4-75cc-4f07-2a14-08d7eb717646 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2020 12:41:24.6354 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YKY3StjHrbBjFd5xTM+6Psj4RBOXFYoD2BWMlcBvUR0HVadzbXWZK0TIH2NYL9e7c1iwro0/Q1yDte/G7QRJVw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3070 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-28_09:2020-04-28, 2020-04-28 signatures=0 Subject: [dpdk-dev] [PATCH v2 1/4] event/octeontx: add multi segment support to eventdev X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding support for multi segment to the eventdev PMD. Signed-off-by: Harman Kalra --- drivers/event/octeontx/ssovf_evdev.c | 33 +++++++--- drivers/event/octeontx/ssovf_evdev.h | 13 ++++ drivers/event/octeontx/ssovf_worker.c | 90 ++++++++++++++++++++++++--- drivers/event/octeontx/ssovf_worker.h | 76 +++++++++++++++++++--- 4 files changed, 189 insertions(+), 23 deletions(-) diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index f9e93244f..1024b7284 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -146,15 +146,31 @@ ssovf_fastpath_fns_set(struct rte_eventdev *dev) dev->enqueue_burst = ssows_enq_burst; dev->enqueue_new_burst = ssows_enq_new_burst; dev->enqueue_forward_burst = ssows_enq_fwd_burst; - dev->dequeue = ssows_deq; - dev->dequeue_burst = ssows_deq_burst; - dev->txa_enqueue = sso_event_tx_adapter_enqueue; - dev->txa_enqueue_same_dest = dev->txa_enqueue; - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout; - dev->dequeue_burst = ssows_deq_timeout_burst; + if (!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)) { + dev->dequeue = ssows_deq_mseg; + dev->dequeue_burst = ssows_deq_burst_mseg; + + if (edev->is_timeout_deq) { + dev->dequeue = ssows_deq_timeout_mseg; + dev->dequeue_burst = ssows_deq_timeout_burst_mseg; + } + } else { + dev->dequeue = ssows_deq; + dev->dequeue_burst = ssows_deq_burst; + + if (edev->is_timeout_deq) { + dev->dequeue = ssows_deq_timeout; + dev->dequeue_burst = ssows_deq_timeout_burst; + } } + + if (!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)) + dev->txa_enqueue = sso_event_tx_adapter_enqueue_mseg; + else + dev->txa_enqueue = sso_event_tx_adapter_enqueue; + + dev->txa_enqueue_same_dest = dev->txa_enqueue; } static void @@ -411,6 +427,7 @@ ssovf_eth_rx_adapter_queue_add(const struct rte_eventdev *dev, { int ret = 0; const struct octeontx_nic *nic = eth_dev->data->dev_private; + struct ssovf_evdev *edev = ssovf_pmd_priv(dev); pki_mod_qos_t pki_qos; RTE_SET_USED(dev); @@ -447,6 +464,8 @@ ssovf_eth_rx_adapter_queue_add(const struct rte_eventdev *dev, ssovf_log_err("failed to modify QOS, port=%d, q=%d", nic->port_id, queue_conf->ev.queue_id); + edev->rx_offload_flags = nic->rx_offload_flags; + edev->tx_offload_flags = nic->tx_offload_flags; return ret; } diff --git a/drivers/event/octeontx/ssovf_evdev.h b/drivers/event/octeontx/ssovf_evdev.h index 0e622152c..1c3ae8556 100644 --- a/drivers/event/octeontx/ssovf_evdev.h +++ b/drivers/event/octeontx/ssovf_evdev.h @@ -12,6 +12,8 @@ #include #include +#include "octeontx_rxtx.h" + #define EVENTDEV_NAME_OCTEONTX_PMD event_octeontx #define SSOVF_LOG(level, fmt, args...) \ @@ -132,6 +134,7 @@ enum ssovf_type { }; struct ssovf_evdev { + OFFLOAD_FLAGS; /*Sequence should not be changed */ uint8_t max_event_queues; uint8_t max_event_ports; uint8_t is_timeout_deq; @@ -175,6 +178,14 @@ uint16_t ssows_deq_timeout(void *port, struct rte_event *ev, uint64_t timeout_ticks); uint16_t ssows_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); +uint16_t ssows_deq_mseg(void *port, struct rte_event *ev, + uint64_t timeout_ticks); +uint16_t ssows_deq_burst_mseg(void *port, struct rte_event ev[], + uint16_t nb_events, uint64_t timeout_ticks); +uint16_t ssows_deq_timeout_mseg(void *port, struct rte_event *ev, + uint64_t timeout_ticks); +uint16_t ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], + uint16_t nb_events, uint64_t timeout_ticks); typedef void (*ssows_handle_event_t)(void *arg, struct rte_event ev); void ssows_flush_events(struct ssows *ws, uint8_t queue_id, @@ -182,6 +193,8 @@ void ssows_flush_events(struct ssows *ws, uint8_t queue_id, void ssows_reset(struct ssows *ws); uint16_t sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], uint16_t nb_events); +uint16_t sso_event_tx_adapter_enqueue_mseg(void *port, + struct rte_event ev[], uint16_t nb_events); int ssovf_info(struct ssovf_info *info); void *ssovf_bar(enum ssovf_type, uint8_t id, uint8_t bar); int test_eventdev_octeontx(void); diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index ab34233d2..a811c2252 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -103,7 +103,7 @@ ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks) ssows_swtag_wait(ws); return 1; } else { - return ssows_get_work(ws, ev); + return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); } } @@ -118,9 +118,9 @@ ssows_deq_timeout(void *port, struct rte_event *ev, uint64_t timeout_ticks) ws->swtag_req = 0; ssows_swtag_wait(ws); } else { - ret = ssows_get_work(ws, ev); + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev); + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); } return ret; } @@ -143,6 +143,61 @@ ssows_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, return ssows_deq_timeout(port, ev, timeout_ticks); } +__rte_always_inline uint16_t __rte_hot +ssows_deq_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) +{ + struct ssows *ws = port; + + RTE_SET_USED(timeout_ticks); + + if (ws->swtag_req) { + ws->swtag_req = 0; + ssows_swtag_wait(ws); + return 1; + } else { + return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); + } +} + +__rte_always_inline uint16_t __rte_hot +ssows_deq_timeout_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) +{ + struct ssows *ws = port; + uint64_t iter; + uint16_t ret = 1; + + if (ws->swtag_req) { + ws->swtag_req = 0; + ssows_swtag_wait(ws); + } else { + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) + ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); + } + return ret; +} + +uint16_t __rte_hot +ssows_deq_burst_mseg(void *port, struct rte_event ev[], uint16_t nb_events, + uint64_t timeout_ticks) +{ + RTE_SET_USED(nb_events); + + return ssows_deq_mseg(port, ev, timeout_ticks); +} + +uint16_t __rte_hot +ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], + uint16_t nb_events, uint64_t timeout_ticks) +{ + RTE_SET_USED(nb_events); + + return ssows_deq_timeout_mseg(port, ev, timeout_ticks); +} + __rte_always_inline uint16_t __rte_hot ssows_enq(void *port, const struct rte_event *ev) { @@ -231,7 +286,9 @@ ssows_flush_events(struct ssows *ws, uint8_t queue_id, ev.event = sched_type_queue | (get_work0 & 0xffffffff); if (get_work1 && ev.event_type == RTE_EVENT_TYPE_ETHDEV) ev.mbuf = ssovf_octeontx_wqe_to_pkt(get_work1, - (ev.event >> 20) & 0x7F); + (ev.event >> 20) & 0x7F, + OCCTX_RX_OFFLOAD_NONE | + OCCTX_RX_MULTI_SEG_F); else ev.u64 = get_work1; @@ -262,9 +319,9 @@ ssows_reset(struct ssows *ws) } } -uint16_t -sso_event_tx_adapter_enqueue(void *port, - struct rte_event ev[], uint16_t nb_events) +static __rte_always_inline uint16_t +__sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], + uint16_t nb_events, const uint16_t flag) { uint16_t port_id; uint16_t queue_id; @@ -298,5 +355,22 @@ sso_event_tx_adapter_enqueue(void *port, ethdev = &rte_eth_devices[port_id]; txq = ethdev->data->tx_queues[queue_id]; - return __octeontx_xmit_pkts(txq, &m, 1, cmd, OCCTX_TX_OFFLOAD_NONE); + return __octeontx_xmit_pkts(txq, &m, 1, cmd, flag); +} + +uint16_t +sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], + uint16_t nb_events) +{ + return __sso_event_tx_adapter_enqueue(port, ev, nb_events, + OCCTX_TX_OFFLOAD_NONE); +} + +uint16_t +sso_event_tx_adapter_enqueue_mseg(void *port, struct rte_event ev[], + uint16_t nb_events) +{ + return __sso_event_tx_adapter_enqueue(port, ev, nb_events, + OCCTX_TX_OFFLOAD_NONE | + OCCTX_TX_MULTI_SEG_F); } diff --git a/drivers/event/octeontx/ssovf_worker.h b/drivers/event/octeontx/ssovf_worker.h index 009b9c18a..0eacec69a 100644 --- a/drivers/event/octeontx/ssovf_worker.h +++ b/drivers/event/octeontx/ssovf_worker.h @@ -19,8 +19,45 @@ enum { /* SSO Operations */ +static __rte_always_inline void +ssovf_octeontx_wqe_xtract_mseg(octtx_wqe_t *wqe, + struct rte_mbuf *mbuf) +{ + octtx_pki_buflink_t *buflink; + rte_iova_t *iova_list; + uint8_t nb_segs; + uint64_t bytes_left = wqe->s.w1.len - wqe->s.w5.size; + + nb_segs = wqe->s.w0.bufs; + + buflink = (octtx_pki_buflink_t *)((uintptr_t)wqe->s.w3.addr - + sizeof(octtx_pki_buflink_t)); + + /*TODO: work with rearm data */ + + while (--nb_segs) { + iova_list = (rte_iova_t *)(uintptr_t)(buflink->w1.s.addr); + mbuf->next = (struct rte_mbuf *)(rte_iova_t *)(iova_list - 2) + - (OCTTX_PACKET_LATER_SKIP / 128); + mbuf = mbuf->next; + + mbuf->data_off = sizeof(octtx_pki_buflink_t); + + __mempool_check_cookies(mbuf->pool, (void **)&mbuf, 1, 1); + if (nb_segs == 1) + mbuf->data_len = bytes_left; + else + mbuf->data_len = buflink->w0.s.size; + + bytes_left = bytes_left - buflink->w0.s.size; + buflink = (octtx_pki_buflink_t *)(rte_iova_t *)(iova_list - 2); + + } +} + static __rte_always_inline struct rte_mbuf * -ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info) +ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info, + const uint16_t flag) { struct rte_mbuf *mbuf; octtx_wqe_t *wqe = (octtx_wqe_t *)(uintptr_t)work; @@ -31,10 +68,18 @@ ssovf_octeontx_wqe_to_pkt(uint64_t work, uint16_t port_info) mbuf->packet_type = ptype_table[wqe->s.w2.lcty][wqe->s.w2.lety][wqe->s.w2.lfty]; mbuf->data_off = RTE_PTR_DIFF(wqe->s.w3.addr, mbuf->buf_addr); - mbuf->pkt_len = wqe->s.w1.len; - mbuf->data_len = mbuf->pkt_len; - mbuf->nb_segs = 1; mbuf->ol_flags = 0; + mbuf->pkt_len = wqe->s.w1.len; + + if (!!(flag & OCCTX_RX_MULTI_SEG_F)) { + mbuf->nb_segs = wqe->s.w0.bufs; + mbuf->data_len = wqe->s.w5.size; + ssovf_octeontx_wqe_xtract_mseg(wqe, mbuf); + } else { + mbuf->nb_segs = 1; + mbuf->data_len = mbuf->pkt_len; + } + mbuf->port = rte_octeontx_pchan_map[port_info >> 4][port_info & 0xF]; rte_mbuf_refcnt_set(mbuf, 1); @@ -45,14 +90,29 @@ static __rte_always_inline void ssovf_octeontx_wqe_free(uint64_t work) { octtx_wqe_t *wqe = (octtx_wqe_t *)(uintptr_t)work; - struct rte_mbuf *mbuf; + uint8_t nb_segs = wqe->s.w0.bufs; + octtx_pki_buflink_t *buflink; + struct rte_mbuf *mbuf, *head; + rte_iova_t *iova_list; mbuf = (struct rte_mbuf *)((uintptr_t)wqe - OCTTX_PACKET_WQE_SKIP); - rte_pktmbuf_free(mbuf); + buflink = (octtx_pki_buflink_t *)((uintptr_t)wqe->s.w3.addr - + sizeof(octtx_pki_buflink_t)); + head = mbuf; + while (--nb_segs) { + iova_list = (rte_iova_t *)(uintptr_t)(buflink->w1.s.addr); + mbuf = (struct rte_mbuf *)(rte_iova_t *)(iova_list - 2) + - (OCTTX_PACKET_LATER_SKIP / 128); + + mbuf->next = NULL; + rte_pktmbuf_free(mbuf); + buflink = (octtx_pki_buflink_t *)(rte_iova_t *)(iova_list - 2); + } + rte_pktmbuf_free(head); } static __rte_always_inline uint16_t -ssows_get_work(struct ssows *ws, struct rte_event *ev) +ssows_get_work(struct ssows *ws, struct rte_event *ev, const uint16_t flag) { uint64_t get_work0, get_work1; uint64_t sched_type_queue; @@ -67,7 +127,7 @@ ssows_get_work(struct ssows *ws, struct rte_event *ev) if (get_work1 && ev->event_type == RTE_EVENT_TYPE_ETHDEV) { ev->mbuf = ssovf_octeontx_wqe_to_pkt(get_work1, - (ev->event >> 20) & 0x7F); + (ev->event >> 20) & 0x7F, flag); } else if (unlikely((get_work0 & 0xFFFFFFFF) == 0xFFFFFFFF)) { ssovf_octeontx_wqe_free(get_work1); return 0; -- 2.18.0