From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24F63A046B for ; Fri, 28 Jun 2019 09:52:58 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 981651B9D8; Fri, 28 Jun 2019 09:51:13 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id EBAA61B965 for ; Fri, 28 Jun 2019 09:50:58 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5S7oUA4025385 for ; Fri, 28 Jun 2019 00:50:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=O2uTjBn0LQv+n9Ci9T81fVqWPnsITZQvQ4VfQx9xzLI=; b=MPPvQBUbt1j6vXS5VazWyy88+7suGpXeBzKksDci0b+WxWqwNSZy0YCEima+GX6GzoIT C9eshRKT29h63yQtasHF82Ad4za1pDNV/mZ9D73GvIRM2T+JQq42FTLj13p18FMCqgF/ VYr6VbkYx9PaswUmVa0Sc+EmDe3xCDT5Tw0W9zFYMJR2GhIamYDyyj0Cr4nZBzxz2wmM skgKK0CH4Y/tLu0YfikBvX5epMG/8YrQevAF5OMnyGXsX8wOLbGQcRNj9c2v23OeI8RZ bKZGF9MNOQ/pgjbK1bFGkgoghl53QgteM6PWlA0KdbFRtclAmlBGHy1vjLGGPCkM1XQV 6A== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2tcvnhc6hy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 28 Jun 2019 00:50:58 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 28 Jun 2019 00:50:56 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 28 Jun 2019 00:50:56 -0700 Received: from BG-LT7430.marvell.com (bg-lt7430.marvell.com [10.28.10.255]) by maili.marvell.com (Postfix) with ESMTP id 86EF53F7040; Fri, 28 Jun 2019 00:50:55 -0700 (PDT) From: To: CC: , Pavan Nikhilesh Date: Fri, 28 Jun 2019 13:19:53 +0530 Message-ID: <20190628075024.404-15-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190628075024.404-1-pbhagavatula@marvell.com> References: <20190628075024.404-1-pbhagavatula@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-28_02:, , signatures=0 Subject: [dpdk-dev] [PATCH v2 14/44] event/octeontx2: add SSO HW device operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Add SSO HW device operations used for enqueue/dequeue. Signed-off-by: Pavan Nikhilesh Signed-off-by: Jerin Jacob --- drivers/event/octeontx2/Makefile | 1 + drivers/event/octeontx2/meson.build | 3 +- drivers/event/octeontx2/otx2_evdev.h | 22 +++ drivers/event/octeontx2/otx2_worker.c | 5 + drivers/event/octeontx2/otx2_worker.h | 187 ++++++++++++++++++++++++++ 5 files changed, 217 insertions(+), 1 deletion(-) create mode 100644 drivers/event/octeontx2/otx2_worker.c create mode 100644 drivers/event/octeontx2/otx2_worker.h diff --git a/drivers/event/octeontx2/Makefile b/drivers/event/octeontx2/Makefile index 4f09c1fc8..a3de5ca23 100644 --- a/drivers/event/octeontx2/Makefile +++ b/drivers/event/octeontx2/Makefile @@ -30,6 +30,7 @@ LIBABIVER := 1 # all source are stored in SRCS-y # +SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EVENTDEV) += otx2_worker.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EVENTDEV) += otx2_evdev.c SRCS-$(CONFIG_RTE_LIBRTE_PMD_OCTEONTX2_EVENTDEV) += otx2_evdev_irq.c diff --git a/drivers/event/octeontx2/meson.build b/drivers/event/octeontx2/meson.build index 5aa8113bd..1d2080b6d 100644 --- a/drivers/event/octeontx2/meson.build +++ b/drivers/event/octeontx2/meson.build @@ -2,7 +2,8 @@ # Copyright(C) 2019 Marvell International Ltd. # -sources = files('otx2_evdev.c', +sources = files('otx2_worker.c', + 'otx2_evdev.c', 'otx2_evdev_irq.c', ) diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index e1d2dcc69..cccce1dea 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -82,6 +82,28 @@ enum otx2_sso_lf_type { SSO_LF_GWS }; +union otx2_sso_event { + uint64_t get_work0; + struct { + uint32_t flow_id:20; + uint32_t sub_event_type:8; + uint32_t event_type:4; + uint8_t op:2; + uint8_t rsvd:4; + uint8_t sched_type:2; + uint8_t queue_id; + uint8_t priority; + uint8_t impl_opaque; + }; +} __rte_aligned(64); + +enum { + SSO_SYNC_ORDERED, + SSO_SYNC_ATOMIC, + SSO_SYNC_UNTAGGED, + SSO_SYNC_EMPTY +}; + struct otx2_sso_evdev { OTX2_DEV; /* Base class */ uint8_t max_event_queues; diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c new file mode 100644 index 000000000..83f535d05 --- /dev/null +++ b/drivers/event/octeontx2/otx2_worker.c @@ -0,0 +1,5 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#include "otx2_worker.h" diff --git a/drivers/event/octeontx2/otx2_worker.h b/drivers/event/octeontx2/otx2_worker.h new file mode 100644 index 000000000..f06ff064e --- /dev/null +++ b/drivers/event/octeontx2/otx2_worker.h @@ -0,0 +1,187 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2019 Marvell International Ltd. + */ + +#ifndef __OTX2_WORKER_H__ +#define __OTX2_WORKER_H__ + +#include +#include + +#include +#include "otx2_evdev.h" + +/* SSO Operations */ + +static __rte_always_inline uint16_t +otx2_ssogws_get_work(struct otx2_ssogws *ws, struct rte_event *ev) +{ + union otx2_sso_event event; + uint64_t get_work1; + + otx2_write64(BIT_ULL(16) | /* wait for work. */ + 1, /* Use Mask set 0. */ + ws->getwrk_op); + +#ifdef RTE_ARCH_ARM64 + asm volatile( + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbz %[tag], 63, done%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbnz %[tag], 63, rty%= \n" + "done%=: dmb ld \n" + " prfm pldl1keep, [%[wqp]] \n" + : [tag] "=&r" (event.get_work0), + [wqp] "=&r" (get_work1) + : [tag_loc] "r" (ws->tag_op), + [wqp_loc] "r" (ws->wqp_op) + ); +#else + event.get_work0 = otx2_read64(ws->tag_op); + while ((BIT_ULL(63)) & event.get_work0) + event.get_work0 = otx2_read64(ws->tag_op); + + get_work1 = otx2_read64(ws->wqp_op); + rte_prefetch0((const void *)get_work1); +#endif + + event.get_work0 = (event.get_work0 & (0x3ull << 32)) << 6 | + (event.get_work0 & (0x3FFull << 36)) << 4 | + (event.get_work0 & 0xffffffff); + ws->cur_tt = event.sched_type; + ws->cur_grp = event.queue_id; + + + ev->event = event.get_work0; + ev->u64 = get_work1; + + return !!get_work1; +} + +/* Used in cleaning up workslot. */ +static __rte_always_inline uint16_t +otx2_ssogws_get_work_empty(struct otx2_ssogws *ws, struct rte_event *ev) +{ + union otx2_sso_event event; + uint64_t get_work1; + +#ifdef RTE_ARCH_ARM64 + asm volatile( + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbz %[tag], 63, done%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldr %[tag], [%[tag_loc]] \n" + " ldr %[wqp], [%[wqp_loc]] \n" + " tbnz %[tag], 63, rty%= \n" + "done%=: dmb ld \n" + " prfm pldl1keep, [%[wqp]] \n" + : [tag] "=&r" (event.get_work0), + [wqp] "=&r" (get_work1) + : [tag_loc] "r" (ws->tag_op), + [wqp_loc] "r" (ws->wqp_op) + ); +#else + event.get_work0 = otx2_read64(ws->tag_op); + while ((BIT_ULL(63)) & event.get_work0) + event.get_work0 = otx2_read64(ws->tag_op); + + get_work1 = otx2_read64(ws->wqp_op); + rte_prefetch0((const void *)get_work1); +#endif + + event.get_work0 = (event.get_work0 & (0x3ull << 32)) << 6 | + (event.get_work0 & (0x3FFull << 36)) << 4 | + (event.get_work0 & 0xffffffff); + ws->cur_tt = event.sched_type; + ws->cur_grp = event.queue_id; + + ev->event = event.get_work0; + ev->u64 = get_work1; + + return !!get_work1; +} + +static __rte_always_inline void +otx2_ssogws_add_work(struct otx2_ssogws *ws, const uint64_t event_ptr, + const uint32_t tag, const uint8_t new_tt, + const uint16_t grp) +{ + uint64_t add_work0; + + add_work0 = tag | ((uint64_t)(new_tt) << 32); + otx2_store_pair(add_work0, event_ptr, ws->grps_base[grp]); +} + +static __rte_always_inline void +otx2_ssogws_swtag_desched(struct otx2_ssogws *ws, uint32_t tag, uint8_t new_tt, + uint16_t grp) +{ + uint64_t val; + + val = tag | ((uint64_t)(new_tt & 0x3) << 32) | ((uint64_t)grp << 34); + otx2_write64(val, ws->swtag_desched_op); +} + +static __rte_always_inline void +otx2_ssogws_swtag_norm(struct otx2_ssogws *ws, uint32_t tag, uint8_t new_tt) +{ + uint64_t val; + + val = tag | ((uint64_t)(new_tt & 0x3) << 32); + otx2_write64(val, ws->swtag_norm_op); +} + +static __rte_always_inline void +otx2_ssogws_swtag_untag(struct otx2_ssogws *ws) +{ + otx2_write64(0, OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op) + + SSOW_LF_GWS_OP_SWTAG_UNTAG); + ws->cur_tt = SSO_SYNC_UNTAGGED; +} + +static __rte_always_inline void +otx2_ssogws_swtag_flush(struct otx2_ssogws *ws) +{ + otx2_write64(0, OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op) + + SSOW_LF_GWS_OP_SWTAG_FLUSH); + ws->cur_tt = SSO_SYNC_EMPTY; +} + +static __rte_always_inline void +otx2_ssogws_desched(struct otx2_ssogws *ws) +{ + otx2_write64(0, OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op) + + SSOW_LF_GWS_OP_DESCHED); +} + +static __rte_always_inline void +otx2_ssogws_swtag_wait(struct otx2_ssogws *ws) +{ +#ifdef RTE_ARCH_ARM64 + uint64_t swtp; + + asm volatile ( + " ldr %[swtb], [%[swtp_loc]] \n" + " cbz %[swtb], done%= \n" + " sevl \n" + "rty%=: wfe \n" + " ldr %[swtb], [%[swtp_loc]] \n" + " cbnz %[swtb], rty%= \n" + "done%=: \n" + : [swtb] "=&r" (swtp) + : [swtp_loc] "r" (ws->swtp_op) + ); +#else + /* Wait for the SWTAG/SWTAG_FULL operation */ + while (otx2_read64(ws->swtp_op)) + ; +#endif +} + +#endif -- 2.22.0