From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D672FA046B for ; Fri, 28 Jun 2019 09:54:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3271658C6; Fri, 28 Jun 2019 09:51:45 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id E54B01B9DB for ; Fri, 28 Jun 2019 09:51:14 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5S7njM3000493; Fri, 28 Jun 2019 00:51:14 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0818; bh=V6OvE3J3d9JyDpo9WEfe640OfE+V/xtZkH1QSyzxdXI=; b=UNaTtydEoHHn/MikHHMb7BjRJXeO7yKPsu9OpaAnXImrlHPBRT4W6s5Psy+F8u/14Fec xNx/d4my2RiMJfLD+tMLDLSHnrpT8q0NB1bxd5SwtSyyvfnV2jOk9rOCZjoIga4rsV9C 0boldL/ndvnwevcRJvDMQuXSu3Q41LUXpv1z4xrOarfH1gzcQepvct/dKt06AVawj/Z5 Hi3Du4XLKwG+t1NH01qBsB2al6qU7svrvks1aCdvplJS2fNAHiYnosGiVor5Z6brcq34 FCSzhos2CYhqG9Mz5V5Oyn4XjVQGeSIbS6a0w5HQlQULVPwx6YCo45FUCBYNoE7Y+kBt 9Q== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2tdd778asa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 28 Jun 2019 00:51:14 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 28 Jun 2019 00:51:13 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 28 Jun 2019 00:51:13 -0700 Received: from BG-LT7430.marvell.com (bg-lt7430.marvell.com [10.28.10.255]) by maili.marvell.com (Postfix) with ESMTP id B61713F7040; Fri, 28 Jun 2019 00:51:11 -0700 (PDT) From: To: , Anatoly Burakov CC: , Pavan Nikhilesh Date: Fri, 28 Jun 2019 13:20:01 +0530 Message-ID: <20190628075024.404-23-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190628075024.404-1-pbhagavatula@marvell.com> References: <20190628075024.404-1-pbhagavatula@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-28_02:, , signatures=0 Subject: [dpdk-dev] [PATCH v2 22/44] event/octeontx2: add device start function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Pavan Nikhilesh Add eventdev start function along with few cleanup API's to maintain sanity. Signed-off-by: Pavan Nikhilesh --- drivers/event/octeontx2/otx2_evdev.c | 127 +++++++++++++++++++++++++- drivers/event/octeontx2/otx2_evdev.h | 6 ++ drivers/event/octeontx2/otx2_worker.c | 74 +++++++++++++++ 3 files changed, 206 insertions(+), 1 deletion(-) diff --git a/drivers/event/octeontx2/otx2_evdev.c b/drivers/event/octeontx2/otx2_evdev.c index 5dc39f029..d6ddee1cd 100644 --- a/drivers/event/octeontx2/otx2_evdev.c +++ b/drivers/event/octeontx2/otx2_evdev.c @@ -38,6 +38,41 @@ sso_get_msix_offsets(const struct rte_eventdev *event_dev) return rc; } +void +sso_fastpath_fns_set(struct rte_eventdev *event_dev) +{ + struct otx2_sso_evdev *dev = sso_pmd_priv(event_dev); + + event_dev->enqueue = otx2_ssogws_enq; + event_dev->enqueue_burst = otx2_ssogws_enq_burst; + event_dev->enqueue_new_burst = otx2_ssogws_enq_new_burst; + event_dev->enqueue_forward_burst = otx2_ssogws_enq_fwd_burst; + + event_dev->dequeue = otx2_ssogws_deq; + event_dev->dequeue_burst = otx2_ssogws_deq_burst; + if (dev->is_timeout_deq) { + event_dev->dequeue = otx2_ssogws_deq_timeout; + event_dev->dequeue_burst = otx2_ssogws_deq_timeout_burst; + } + + if (dev->dual_ws) { + event_dev->enqueue = otx2_ssogws_dual_enq; + event_dev->enqueue_burst = otx2_ssogws_dual_enq_burst; + event_dev->enqueue_new_burst = + otx2_ssogws_dual_enq_new_burst; + event_dev->enqueue_forward_burst = + otx2_ssogws_dual_enq_fwd_burst; + event_dev->dequeue = otx2_ssogws_dual_deq; + event_dev->dequeue_burst = otx2_ssogws_dual_deq_burst; + if (dev->is_timeout_deq) { + event_dev->dequeue = otx2_ssogws_dual_deq_timeout; + event_dev->dequeue_burst = + otx2_ssogws_dual_deq_timeout_burst; + } + } + rte_mb(); +} + static void otx2_sso_info_get(struct rte_eventdev *event_dev, struct rte_event_dev_info *dev_info) @@ -889,6 +924,93 @@ otx2_sso_dump(struct rte_eventdev *event_dev, FILE *f) } } +static void +otx2_handle_event(void *arg, struct rte_event event) +{ + struct rte_eventdev *event_dev = arg; + + if (event_dev->dev_ops->dev_stop_flush != NULL) + event_dev->dev_ops->dev_stop_flush(event_dev->data->dev_id, + event, event_dev->data->dev_stop_flush_arg); +} + +static void +sso_cleanup(struct rte_eventdev *event_dev, uint8_t enable) +{ + struct otx2_sso_evdev *dev = sso_pmd_priv(event_dev); + uint16_t i; + + for (i = 0; i < dev->nb_event_ports; i++) { + if (dev->dual_ws) { + struct otx2_ssogws_dual *ws; + + ws = event_dev->data->ports[i]; + ssogws_reset((struct otx2_ssogws *)&ws->ws_state[0]); + ssogws_reset((struct otx2_ssogws *)&ws->ws_state[1]); + ws->swtag_req = 0; + ws->vws = 0; + ws->ws_state[0].cur_grp = 0; + ws->ws_state[0].cur_tt = SSO_SYNC_EMPTY; + ws->ws_state[1].cur_grp = 0; + ws->ws_state[1].cur_tt = SSO_SYNC_EMPTY; + } else { + struct otx2_ssogws *ws; + + ws = event_dev->data->ports[i]; + ssogws_reset(ws); + ws->swtag_req = 0; + ws->cur_grp = 0; + ws->cur_tt = SSO_SYNC_EMPTY; + } + } + + rte_mb(); + if (dev->dual_ws) { + struct otx2_ssogws_dual *ws = event_dev->data->ports[0]; + struct otx2_ssogws temp_ws; + + memcpy(&temp_ws, &ws->ws_state[0], + sizeof(struct otx2_ssogws_state)); + for (i = 0; i < dev->nb_event_queues; i++) { + /* Consume all the events through HWS0 */ + ssogws_flush_events(&temp_ws, i, ws->grps_base[i], + otx2_handle_event, event_dev); + /* Enable/Disable SSO GGRP */ + otx2_write64(enable, ws->grps_base[i] + + SSO_LF_GGRP_QCTL); + } + ws->ws_state[0].cur_grp = 0; + ws->ws_state[0].cur_tt = SSO_SYNC_EMPTY; + } else { + struct otx2_ssogws *ws = event_dev->data->ports[0]; + + for (i = 0; i < dev->nb_event_queues; i++) { + /* Consume all the events through HWS0 */ + ssogws_flush_events(ws, i, ws->grps_base[i], + otx2_handle_event, event_dev); + /* Enable/Disable SSO GGRP */ + otx2_write64(enable, ws->grps_base[i] + + SSO_LF_GGRP_QCTL); + } + ws->cur_grp = 0; + ws->cur_tt = SSO_SYNC_EMPTY; + } + + /* reset SSO GWS cache */ + otx2_mbox_alloc_msg_sso_ws_cache_inv(dev->mbox); + otx2_mbox_process(dev->mbox); +} + +static int +otx2_sso_start(struct rte_eventdev *event_dev) +{ + sso_func_trace(); + sso_cleanup(event_dev, 1); + sso_fastpath_fns_set(event_dev); + + return 0; +} + /* Initialize and register event driver with DPDK Application */ static struct rte_eventdev_ops otx2_sso_ops = { .dev_infos_get = otx2_sso_info_get, @@ -908,6 +1030,7 @@ static struct rte_eventdev_ops otx2_sso_ops = { .xstats_get_names = otx2_sso_xstats_get_names, .dump = otx2_sso_dump, + .dev_start = otx2_sso_start, }; #define OTX2_SSO_XAE_CNT "xae_cnt" @@ -975,8 +1098,10 @@ otx2_sso_init(struct rte_eventdev *event_dev) event_dev->dev_ops = &otx2_sso_ops; /* For secondary processes, the primary has done all the work */ - if (rte_eal_process_type() != RTE_PROC_PRIMARY) + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + sso_fastpath_fns_set(event_dev); return 0; + } dev = sso_pmd_priv(event_dev); diff --git a/drivers/event/octeontx2/otx2_evdev.h b/drivers/event/octeontx2/otx2_evdev.h index 8e614b109..4428abcfa 100644 --- a/drivers/event/octeontx2/otx2_evdev.h +++ b/drivers/event/octeontx2/otx2_evdev.h @@ -232,6 +232,12 @@ uint16_t otx2_ssogws_dual_deq_timeout(void *port, struct rte_event *ev, uint16_t otx2_ssogws_dual_deq_timeout_burst(void *port, struct rte_event ev[], uint16_t nb_events, uint64_t timeout_ticks); +void sso_fastpath_fns_set(struct rte_eventdev *event_dev); +/* Clean up API's */ +typedef void (*otx2_handle_event_t)(void *arg, struct rte_event ev); +void ssogws_flush_events(struct otx2_ssogws *ws, uint8_t queue_id, + uintptr_t base, otx2_handle_event_t fn, void *arg); +void ssogws_reset(struct otx2_ssogws *ws); /* Init and Fini API's */ int otx2_sso_init(struct rte_eventdev *event_dev); int otx2_sso_fini(struct rte_eventdev *event_dev); diff --git a/drivers/event/octeontx2/otx2_worker.c b/drivers/event/octeontx2/otx2_worker.c index edc574673..7a6d4cad2 100644 --- a/drivers/event/octeontx2/otx2_worker.c +++ b/drivers/event/octeontx2/otx2_worker.c @@ -194,3 +194,77 @@ otx2_ssogws_enq_fwd_burst(void *port, const struct rte_event ev[], return 1; } + +void +ssogws_flush_events(struct otx2_ssogws *ws, uint8_t queue_id, uintptr_t base, + otx2_handle_event_t fn, void *arg) +{ + uint64_t cq_ds_cnt = 1; + uint64_t aq_cnt = 1; + uint64_t ds_cnt = 1; + struct rte_event ev; + uint64_t enable; + uint64_t val; + + enable = otx2_read64(base + SSO_LF_GGRP_QCTL); + if (!enable) + return; + + val = queue_id; /* GGRP ID */ + val |= BIT_ULL(18); /* Grouped */ + val |= BIT_ULL(16); /* WAIT */ + + aq_cnt = otx2_read64(base + SSO_LF_GGRP_AQ_CNT); + ds_cnt = otx2_read64(base + SSO_LF_GGRP_MISC_CNT); + cq_ds_cnt = otx2_read64(base + SSO_LF_GGRP_INT_CNT); + cq_ds_cnt &= 0x3FFF3FFF0000; + + while (aq_cnt || cq_ds_cnt || ds_cnt) { + otx2_write64(val, ws->getwrk_op); + otx2_ssogws_get_work_empty(ws, &ev); + if (fn != NULL && ev.u64 != 0) + fn(arg, ev); + if (ev.sched_type != SSO_TT_EMPTY) + otx2_ssogws_swtag_flush(ws); + rte_mb(); + aq_cnt = otx2_read64(base + SSO_LF_GGRP_AQ_CNT); + ds_cnt = otx2_read64(base + SSO_LF_GGRP_MISC_CNT); + cq_ds_cnt = otx2_read64(base + SSO_LF_GGRP_INT_CNT); + /* Extract cq and ds count */ + cq_ds_cnt &= 0x3FFF3FFF0000; + } + + otx2_write64(0, OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op) + + SSOW_LF_GWS_OP_GWC_INVAL); + rte_mb(); +} + +void +ssogws_reset(struct otx2_ssogws *ws) +{ + uintptr_t base = OTX2_SSOW_GET_BASE_ADDR(ws->getwrk_op); + uint64_t pend_state; + uint8_t pend_tt; + uint64_t tag; + + /* Wait till getwork/swtp/waitw/desched completes. */ + do { + pend_state = otx2_read64(base + SSOW_LF_GWS_PENDSTATE); + rte_mb(); + } while (pend_state & (BIT_ULL(63) | BIT_ULL(62) | BIT_ULL(58))); + + tag = otx2_read64(base + SSOW_LF_GWS_TAG); + pend_tt = (tag >> 32) & 0x3; + if (pend_tt != SSO_TT_EMPTY) { /* Work was pending */ + if (pend_tt == SSO_SYNC_ATOMIC || pend_tt == SSO_SYNC_ORDERED) + otx2_ssogws_swtag_untag(ws); + otx2_ssogws_desched(ws); + } + rte_mb(); + + /* Wait for desched to complete. */ + do { + pend_state = otx2_read64(base + SSOW_LF_GWS_PENDSTATE); + rte_mb(); + } while (pend_state & BIT_ULL(58)); +} -- 2.22.0