From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DE60CA00BE; Tue, 28 Apr 2020 14:41:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CA5491D5DD; Tue, 28 Apr 2020 14:41:36 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 9131A1D5DA for ; Tue, 28 Apr 2020 14:41:34 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 03SCeiSu029070 for ; Tue, 28 Apr 2020 05:41:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0818; bh=tc0v+MWmibw2E0jr8KzumIKIYI0opyMUEfL8m8MINN8=; b=OYwTxpUkI5gsHVxx4nvrkDRHBFANol+G7x+jDyswmMiZVDSGdp0sZevmOPH+rbRqthhZ fnLKzwim02fo5C8v8MWpM6ZCGKNAT8XvQcehfpRYmRZkTK4JfIZ2YNUdlSzGSqS5dkW0 iDLMEEm6Uk5zM11m6DJR7yWTbw8UKmI84XbTe05tQ3VTAwZI+JuHq/C4+9WKZYKSgIhx 8swrUgrFdO4hufyeYVrBwPIIf5NJNxvM2VAvVHhuaIqZoYg5dlxACNy/woVk64NDoKIP zNZdlhke6mTvlsns5oRMnslshvlG5VTi5Lds5MNXVNnyEUaqX6KurVAaqkDTi1kHC2dL 0A== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 30mmqmksv0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 28 Apr 2020 05:41:33 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 28 Apr 2020 05:41:31 -0700 Received: from SC-EXCH02.marvell.com (10.93.176.82) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 28 Apr 2020 05:41:30 -0700 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (104.47.37.54) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Tue, 28 Apr 2020 05:41:30 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IVMNlj4fu62HZoDWwyaVH4l4CFcOv0Ofpj1tMRT2p0NXRI6rDXyYThQPkZCEXgyuvg0aB1Wxn70Hn5mVnZmi40QYDdkHhwNbJiIITRfYUkYcNRH59IOe2wLI8iRe0APR2Nzq8UxzB7kkGDX4s7JbwcNh/TzyZz+53IpXFpQzwopFrE6NytSloBNBz2Tzb8OAEEufkaf9lpYnO+5RWBvhU+ZB9V9iGMRuI+WYskECGx6VpQwP/BC7FvxiO3z7T9/DI3HC9RC9cX+KFRr+OVg0sUTfLepofEWqIhcKexQJQwf4yQv7noYPGo3tW9PN2JWxHuAawQtoO618JrW0BBtYzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tc0v+MWmibw2E0jr8KzumIKIYI0opyMUEfL8m8MINN8=; b=MR1At/eCohBvZWAMnZ03aBgPH+0TNA/aYXXXm4zGbGpfBk2MJmRHQdgLwNKODDczdpcXgESi0Wv+J8BZHN39v/On5dIUMFR+wdrW4DUVJuFKSw2kQxzrr5yFvNVJXXGHFe5ImLDgsbYDhqKweG/VuAApTJHgXzA2kJr6LXUUaKle3FINwjUEcXwnq36+2FPAGpLPhwfBUFyR62olpKF3I0sLqg/JULW5IJdxM+hcbNkRARd7wluQ7Dy6aSW1ycCm2jDsOrBtD9r+pyaWknk8r/zUaEwvaX5h/x2TsxyWpPwcJKj5AYql5rEBJMoyicg6Dplz/uXeHnkILH1ZB47Xfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tc0v+MWmibw2E0jr8KzumIKIYI0opyMUEfL8m8MINN8=; b=PsY6cuLx97WAqCuFQPDFC7UGezrUxdJgfKFAVMHZRB9CsTd6qxsF+1TgnuKU+LfqP0tVT9Dde3RFXmVth3foBbP5mDo+X7hFjZiKJUeCm4XGGIKn4/POwwVRcdS1Tk76dJDoyuweZJR1g4pKOKjGn53rcOyVlzp7JfrXqDT3q0c= Received: from MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) by MN2PR18MB3070.namprd18.prod.outlook.com (2603:10b6:208:ff::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.22; Tue, 28 Apr 2020 12:41:29 +0000 Received: from MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::a8b1:45c4:b84:9c02]) by MN2PR18MB2848.namprd18.prod.outlook.com ([fe80::a8b1:45c4:b84:9c02%7]) with mapi id 15.20.2937.026; Tue, 28 Apr 2020 12:41:29 +0000 From: Harman Kalra To: CC: , Harman Kalra Date: Tue, 28 Apr 2020 18:10:10 +0530 Message-ID: <1588077612-15905-3-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588077612-15905-1-git-send-email-hkalra@marvell.com> References: <1588077612-15905-1-git-send-email-hkalra@marvell.com> Content-Type: text/plain X-ClientProxiedBy: BMXPR01CA0031.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:c::17) To MN2PR18MB2848.namprd18.prod.outlook.com (2603:10b6:208:38::19) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BMXPR01CA0031.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.2937.13 via Frontend Transport; Tue, 28 Apr 2020 12:41:27 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 996537f9-d099-4697-af2b-08d7eb7178d6 X-MS-TrafficTypeDiagnostic: MN2PR18MB3070: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-Forefront-PRVS: 0387D64A71 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR18MB2848.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(346002)(136003)(396003)(39860400002)(366004)(376002)(2906002)(52116002)(81156014)(7696005)(36756003)(316002)(2616005)(6636002)(5660300002)(8676002)(55236004)(34206002)(30864003)(86362001)(4326008)(8936002)(6666004)(956004)(37006003)(16526019)(107886003)(6486002)(66476007)(66556008)(66946007)(478600001)(186003)(26005); DIR:OUT; SFP:1101; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iMa+cmATqNhbBHqEiUTTyWyvkLsWIw8sYajaBGQ1HKe42IiB6Sm9s/DUwRp1wrtAn/1UC0TEBJehUVpaafG9jOQQiVS/VQBuUopyaWvVqhu7o6cJzYI5DwmbtStAxu4UoEniOxyNi5GPFCD1VQpbaTvnq9EFdNRIigtpGj8lCWuE68eXo/yOTI7uMJiAzdAmwcvu3OwOcv4Z0OMjlYvIYfJ3schoAeX6EsAP3oZ5odxP1Zwq3KFIZiYOByUO3SU7MljbLVN68kQ2Ai8M3rKmBzfYK72H/07YPgDOaIrx/RjBRE1rMELLvbFGmpCfk7Yr4vOi+FghJklhkwj7YMDHS9Ie66Pk3hqC9KbrAC2/jWwxfvfkx2OMV9KdrF66U1QjQGkMxBMQ0vuN85poLA7LwVD58VBnm1zEY0gkoaz+Z8yhok1eriyfy67c9AJ2aJQQ X-MS-Exchange-AntiSpam-MessageData: 1ewOYFwxKZ6I+M5h50mVJbt2p1vmJ3v/VrPpkN6SG8HzFh4713A0VvLvHAyVXqNv6a3rBLBJAk7lQYwHe4HeIR5tMx25V9Zq4hT2PsopbH8BJVHEQq8Ttsft7qlhIh+bRXaSoRtOh4AyozH5u9itUFRn4Tk3diV779D0K1nUp0wXKY3wNzc5gjz5SY8GXnKcNc56URBbYOOI9voDXW3XD188Ey/p/MLzoj5Hz2gS/6lelwfa5o3NWGhhkEmCVHd8b0zNJfpEHoGiz037InBDJt9uj99D+Lj6dnOJeTA4EFP53spTFwCFIeEuSw2aeM6p6tJleWWJyZxqx42voVIM4PnSn5tf63ADEyaBsXfQ05PaMAGII4b9umv/GaehCWlEXf5hLsF6/RYEQm9rLJv/3tgLEnu3UDyTiLZFICjNmYIYJGA/xBjZ5xE3IOk8LbQDDS7xmod5H7L0U6OsQNC1g+tzXdkNcv8v304vEDoUkLge/CQQyJVMPUgy2E7O1EJgrfwardcDgnlbIEVA90sd7iWpsDtfWvvBYHL8y722iyHiAGneoRTARFJUmGoHr1HDDXFsbicnTfBZkowUZU+xwUXjEML4RMzzHnlxRXhp+Cxm9vfX5cnnlLJY4tYt6sJIlbGT2dmoBjece3w/KGk6ckBeKj9n8KRCW//sI2xQy54VlSGhvguRZtwVehfS1TGUT+TmKuLf0iJ9W0fRQuD7s/O036W3bigkfar/xh25b/xU1yzXybyEkt5luFNfePRny2A2ZeDxhOzKTpwzFDq7soeIvT07jGTXTc6zm/oPQ38= X-MS-Exchange-CrossTenant-Network-Message-Id: 996537f9-d099-4697-af2b-08d7eb7178d6 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2020 12:41:28.9200 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: H1pHvQzQuzoa6pqbFIMRJj5T4M4iTIhOZY/242hG46OZpD9atLKWhlcE5ClYijFz4JPnevKNF5CsRjVyBFqq3A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3070 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.676 definitions=2020-04-28_09:2020-04-28, 2020-04-28 signatures=0 Subject: [dpdk-dev] [PATCH v2 2/4] event/octeontx: add framework for Rx/Tx offloads X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Adding macro based framework to hook dequeue/enqueue function pointers to the appropriate function based on rx/tx offloads. Signed-off-by: Harman Kalra --- drivers/event/octeontx/ssovf_evdev.c | 36 ---- drivers/event/octeontx/ssovf_evdev.h | 24 +-- drivers/event/octeontx/ssovf_worker.c | 259 ++++++++++++++------------ drivers/net/octeontx/octeontx_rxtx.h | 7 + 4 files changed, 150 insertions(+), 176 deletions(-) diff --git a/drivers/event/octeontx/ssovf_evdev.c b/drivers/event/octeontx/ssovf_evdev.c index 1024b7284..5d074bcbc 100644 --- a/drivers/event/octeontx/ssovf_evdev.c +++ b/drivers/event/octeontx/ssovf_evdev.c @@ -137,42 +137,6 @@ ssovf_mbox_timeout_ticks(uint64_t ns, uint64_t *tmo_ticks) return 0; } -static void -ssovf_fastpath_fns_set(struct rte_eventdev *dev) -{ - struct ssovf_evdev *edev = ssovf_pmd_priv(dev); - - dev->enqueue = ssows_enq; - dev->enqueue_burst = ssows_enq_burst; - dev->enqueue_new_burst = ssows_enq_new_burst; - dev->enqueue_forward_burst = ssows_enq_fwd_burst; - - if (!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)) { - dev->dequeue = ssows_deq_mseg; - dev->dequeue_burst = ssows_deq_burst_mseg; - - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout_mseg; - dev->dequeue_burst = ssows_deq_timeout_burst_mseg; - } - } else { - dev->dequeue = ssows_deq; - dev->dequeue_burst = ssows_deq_burst; - - if (edev->is_timeout_deq) { - dev->dequeue = ssows_deq_timeout; - dev->dequeue_burst = ssows_deq_timeout_burst; - } - } - - if (!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)) - dev->txa_enqueue = sso_event_tx_adapter_enqueue_mseg; - else - dev->txa_enqueue = sso_event_tx_adapter_enqueue; - - dev->txa_enqueue_same_dest = dev->txa_enqueue; -} - static void ssovf_info_get(struct rte_eventdev *dev, struct rte_event_dev_info *dev_info) { diff --git a/drivers/event/octeontx/ssovf_evdev.h b/drivers/event/octeontx/ssovf_evdev.h index 1c3ae8556..1f5066c9a 100644 --- a/drivers/event/octeontx/ssovf_evdev.h +++ b/drivers/event/octeontx/ssovf_evdev.h @@ -14,6 +14,9 @@ #include "octeontx_rxtx.h" +#define SSO_RX_ADPTR_ENQ_FASTPATH_FUNC OCCTX_RX_FASTPATH_MODES +#define SSO_TX_ADPTR_ENQ_FASTPATH_FUNC OCCTX_TX_FASTPATH_MODES + #define EVENTDEV_NAME_OCTEONTX_PMD event_octeontx #define SSOVF_LOG(level, fmt, args...) \ @@ -171,32 +174,13 @@ uint16_t ssows_enq_new_burst(void *port, const struct rte_event ev[], uint16_t nb_events); uint16_t ssows_enq_fwd_burst(void *port, const struct rte_event ev[], uint16_t nb_events); -uint16_t ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks); -uint16_t ssows_deq_burst(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_timeout(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_burst(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_mseg(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_mseg(void *port, struct rte_event *ev, - uint64_t timeout_ticks); -uint16_t ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks); - typedef void (*ssows_handle_event_t)(void *arg, struct rte_event ev); void ssows_flush_events(struct ssows *ws, uint8_t queue_id, ssows_handle_event_t fn, void *arg); void ssows_reset(struct ssows *ws); -uint16_t sso_event_tx_adapter_enqueue(void *port, - struct rte_event ev[], uint16_t nb_events); -uint16_t sso_event_tx_adapter_enqueue_mseg(void *port, - struct rte_event ev[], uint16_t nb_events); int ssovf_info(struct ssovf_info *info); void *ssovf_bar(enum ssovf_type, uint8_t id, uint8_t bar); int test_eventdev_octeontx(void); +void ssovf_fastpath_fns_set(struct rte_eventdev *dev); #endif /* __SSOVF_EVDEV_H__ */ diff --git a/drivers/event/octeontx/ssovf_worker.c b/drivers/event/octeontx/ssovf_worker.c index a811c2252..b5873c3fa 100644 --- a/drivers/event/octeontx/ssovf_worker.c +++ b/drivers/event/octeontx/ssovf_worker.c @@ -91,112 +91,62 @@ ssows_release_event(struct ssows *ws) ssows_swtag_untag(ws); } -__rte_always_inline uint16_t __rte_hot -ssows_deq(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - - RTE_SET_USED(timeout_ticks); - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - return 1; - } else { - return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - } -} - -__rte_always_inline uint16_t __rte_hot -ssows_deq_timeout(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - uint64_t iter; - uint16_t ret = 1; - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - } else { - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE); - } - return ret; +#define R(name, f0, flags) \ +static uint16_t __rte_noinline __rte_hot \ +ssows_deq_ ##name(void *port, struct rte_event *ev, uint64_t timeout_ticks) \ +{ \ + struct ssows *ws = port; \ + \ + RTE_SET_USED(timeout_ticks); \ + \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ssows_swtag_wait(ws); \ + return 1; \ + } else { \ + return ssows_get_work(ws, ev, flags); \ + } \ +} \ + \ +static uint16_t __rte_hot \ +ssows_deq_burst_ ##name(void *port, struct rte_event ev[], \ + uint16_t nb_events, uint64_t timeout_ticks) \ +{ \ + RTE_SET_USED(nb_events); \ + \ + return ssows_deq_ ##name(port, ev, timeout_ticks); \ +} \ + \ +static uint16_t __rte_hot \ +ssows_deq_timeout_ ##name(void *port, struct rte_event *ev, \ + uint64_t timeout_ticks) \ +{ \ + struct ssows *ws = port; \ + uint64_t iter; \ + uint16_t ret = 1; \ + \ + if (ws->swtag_req) { \ + ws->swtag_req = 0; \ + ssows_swtag_wait(ws); \ + } else { \ + ret = ssows_get_work(ws, ev, flags); \ + for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) \ + ret = ssows_get_work(ws, ev, flags); \ + } \ + return ret; \ +} \ + \ +static uint16_t __rte_hot \ +ssows_deq_timeout_burst_ ##name(void *port, struct rte_event ev[], \ + uint16_t nb_events, uint64_t timeout_ticks) \ +{ \ + RTE_SET_USED(nb_events); \ + \ + return ssows_deq_timeout_ ##name(port, ev, timeout_ticks); \ } -uint16_t __rte_hot -ssows_deq_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq(port, ev, timeout_ticks); -} - -uint16_t __rte_hot -ssows_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_timeout(port, ev, timeout_ticks); -} - -__rte_always_inline uint16_t __rte_hot -ssows_deq_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - - RTE_SET_USED(timeout_ticks); - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - return 1; - } else { - return ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - } -} - -__rte_always_inline uint16_t __rte_hot -ssows_deq_timeout_mseg(void *port, struct rte_event *ev, uint64_t timeout_ticks) -{ - struct ssows *ws = port; - uint64_t iter; - uint16_t ret = 1; - - if (ws->swtag_req) { - ws->swtag_req = 0; - ssows_swtag_wait(ws); - } else { - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - for (iter = 1; iter < timeout_ticks && (ret == 0); iter++) - ret = ssows_get_work(ws, ev, OCCTX_RX_OFFLOAD_NONE | - OCCTX_RX_MULTI_SEG_F); - } - return ret; -} - -uint16_t __rte_hot -ssows_deq_burst_mseg(void *port, struct rte_event ev[], uint16_t nb_events, - uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_mseg(port, ev, timeout_ticks); -} - -uint16_t __rte_hot -ssows_deq_timeout_burst_mseg(void *port, struct rte_event ev[], - uint16_t nb_events, uint64_t timeout_ticks) -{ - RTE_SET_USED(nb_events); - - return ssows_deq_timeout_mseg(port, ev, timeout_ticks); -} +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R __rte_always_inline uint16_t __rte_hot ssows_enq(void *port, const struct rte_event *ev) @@ -321,7 +271,8 @@ ssows_reset(struct ssows *ws) static __rte_always_inline uint16_t __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], - uint16_t nb_events, const uint16_t flag) + uint16_t nb_events, uint64_t *cmd, + const uint16_t flag) { uint16_t port_id; uint16_t queue_id; @@ -329,9 +280,7 @@ __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], struct rte_eth_dev *ethdev; struct ssows *ws = port; struct octeontx_txq *txq; - uint64_t cmd[4]; - RTE_SET_USED(nb_events); switch (ev->sched_type) { case SSO_SYNC_ORDERED: ssows_swtag_norm(ws, ev->event, SSO_SYNC_ATOMIC); @@ -355,22 +304,92 @@ __sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], ethdev = &rte_eth_devices[port_id]; txq = ethdev->data->tx_queues[queue_id]; - return __octeontx_xmit_pkts(txq, &m, 1, cmd, flag); + return __octeontx_xmit_pkts(txq, &m, nb_events, cmd, flag); } -uint16_t -sso_event_tx_adapter_enqueue(void *port, struct rte_event ev[], - uint16_t nb_events) -{ - return __sso_event_tx_adapter_enqueue(port, ev, nb_events, - OCCTX_TX_OFFLOAD_NONE); +#define T(name, f3, f2, f1, f0, sz, flags) \ +static uint16_t __rte_noinline __rte_hot \ +sso_event_tx_adapter_enqueue_ ## name(void *port, struct rte_event ev[], \ + uint16_t nb_events) \ +{ \ + uint64_t cmd[sz]; \ + return __sso_event_tx_adapter_enqueue(port, ev, nb_events, cmd, \ + flags); \ } -uint16_t -sso_event_tx_adapter_enqueue_mseg(void *port, struct rte_event ev[], - uint16_t nb_events) +SSO_TX_ADPTR_ENQ_FASTPATH_FUNC +#undef T + +void +ssovf_fastpath_fns_set(struct rte_eventdev *dev) { - return __sso_event_tx_adapter_enqueue(port, ev, nb_events, - OCCTX_TX_OFFLOAD_NONE | - OCCTX_TX_MULTI_SEG_F); + struct ssovf_evdev *edev = ssovf_pmd_priv(dev); + + dev->enqueue = ssows_enq; + dev->enqueue_burst = ssows_enq_burst; + dev->enqueue_new_burst = ssows_enq_new_burst; + dev->enqueue_forward_burst = ssows_enq_fwd_burst; + + const event_tx_adapter_enqueue ssow_txa_enqueue[2][2][2][2] = { +#define T(name, f3, f2, f1, f0, sz, flags) \ + [f3][f2][f1][f0] = sso_event_tx_adapter_enqueue_ ##name, + +SSO_TX_ADPTR_ENQ_FASTPATH_FUNC +#undef T + }; + + dev->txa_enqueue = ssow_txa_enqueue + [!!(edev->tx_offload_flags & OCCTX_TX_OFFLOAD_MBUF_NOFF_F)] + [0] + [0] + [!!(edev->tx_offload_flags & OCCTX_TX_MULTI_SEG_F)]; + + dev->txa_enqueue_same_dest = dev->txa_enqueue; + + /* Assigning dequeue func pointers */ + const event_dequeue_t ssow_deq[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue = ssow_deq + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + const event_dequeue_burst_t ssow_deq_burst[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_burst_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue_burst = ssow_deq_burst + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + if (edev->is_timeout_deq) { + const event_dequeue_t ssow_deq_timeout[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_timeout_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue = ssow_deq_timeout + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + + const event_dequeue_burst_t ssow_deq_timeout_burst[2] = { +#define R(name, f0, flags) \ + [f0] = ssows_deq_timeout_burst_ ##name, + +SSO_RX_ADPTR_ENQ_FASTPATH_FUNC +#undef R + }; + + dev->dequeue_burst = ssow_deq_timeout_burst + [!!(edev->rx_offload_flags & OCCTX_RX_MULTI_SEG_F)]; + } } diff --git a/drivers/net/octeontx/octeontx_rxtx.h b/drivers/net/octeontx/octeontx_rxtx.h index 5e607f170..144ae055b 100644 --- a/drivers/net/octeontx/octeontx_rxtx.h +++ b/drivers/net/octeontx/octeontx_rxtx.h @@ -484,4 +484,11 @@ T(noff_ol3ol4csum_l3l4csum_mseg, 1, 1, 1, 1, 14, \ NOFF_F | OL3OL4CSUM_F | L3L4CSUM_F | \ MULT_F) +/* RX offload macros */ +#define MULT_RX_F OCCTX_RX_MULTI_SEG_F +/* [MULTI_SEG] */ +#define OCCTX_RX_FASTPATH_MODES \ +R(no_offload, 0, OCCTX_RX_OFFLOAD_NONE) \ +R(mseg, 1, MULT_RX_F) \ + #endif /* __OCTEONTX_RXTX_H__ */ -- 2.18.0