From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 78906A00C5; Tue, 19 Jul 2022 13:11:56 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D0899427ED; Tue, 19 Jul 2022 13:11:48 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id EFD6A427ED for ; Tue, 19 Jul 2022 13:11:46 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 26J9unS2016406 for ; Tue, 19 Jul 2022 04:11:46 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=UJYFMmmmyxvP/HKprRX1k/m953GqA+I+NTUtSeGRJKo=; b=M85/TFR3oaggKf+03fL7vh7TIillnHemsLtv4MrhJkQ79D5sUOeUPsIBiRQHOAe1f6bx Blofte5Dz/FPtJMK/XgpM4ejH2Mf2ELiayara0brOhGNoz5YeNny/gFCiiqw3IJiZAgd F6oEpP1DvDN223L+donCRA0bwg5+D+q1enEQIvSNQ18bhCT96610cMNpTG7G0qHsi2D9 Th+8cy1RcORyTBfuyg7wa1bYFTehc7+VTW7eBaznFNV5eh5T7gq9WTlBHHqeGCd4ARcj 1Zwxw3bvA6UVfajZMnqaTEpKIS8CWdJWcbMLDFkKfJguLYaufB8YgtW1PE31WRQbOnln DA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3hbvumt759-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 19 Jul 2022 04:11:46 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 19 Jul 2022 04:11:44 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 19 Jul 2022 04:11:44 -0700 Received: from MININT-80QBFE8.corp.innovium.com (unknown [10.28.161.88]) by maili.marvell.com (Postfix) with ESMTP id C42BB3F7050; Tue, 19 Jul 2022 04:11:39 -0700 (PDT) From: To: , Nithin Dabilpuram , "Kiran Kumar K" , Sunil Kumar Kori , Satha Rao , Pavan Nikhilesh , Shijith Thotton CC: Subject: [PATCH 4/4] event/cnxk: disable timer resolution estimation Date: Tue, 19 Jul 2022 16:41:25 +0530 Message-ID: <20220719111125.8276-4-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220719111125.8276-1-pbhagavatula@marvell.com> References: <20220719111125.8276-1-pbhagavatula@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: QV10nS03fTip1oVd9lY9KIUA25m2Eo5S X-Proofpoint-ORIG-GUID: QV10nS03fTip1oVd9lY9KIUA25m2Eo5S X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-18_22,2022-07-19_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Pavan Nikhilesh Disable timer resolution estimation, read TIM LF clock registers to get the current running clock counter as estimating causes time drift. Signed-off-by: Pavan Nikhilesh --- drivers/common/cnxk/hw/tim.h | 6 ++ drivers/event/cnxk/cnxk_tim_evdev.c | 140 +++------------------------ drivers/event/cnxk/cnxk_tim_evdev.h | 49 +++++++--- drivers/event/cnxk/cnxk_tim_worker.c | 18 ---- drivers/event/cnxk/cnxk_tim_worker.h | 3 +- 5 files changed, 62 insertions(+), 154 deletions(-) diff --git a/drivers/common/cnxk/hw/tim.h b/drivers/common/cnxk/hw/tim.h index a0fe29ddcf..61c38ae175 100644 --- a/drivers/common/cnxk/hw/tim.h +++ b/drivers/common/cnxk/hw/tim.h @@ -31,6 +31,12 @@ #define TIM_LF_INT_VEC_NRSPERR_INT (0x0ull) #define TIM_LF_INT_VEC_RAS_INT (0x1ull) #define TIM_LF_RING_AURA (0x0) +#define TIM_LF_FR_RN_GPIOS (0x020) +#define TIM_LF_FR_RN_GTI (0x030) +#define TIM_LF_FR_RN_PTP (0x040) +#define TIM_LF_FR_RN_TENNS (0x050) +#define TIM_LF_FR_RN_SYNCE (0x060) +#define TIM_LF_FR_RN_BTS (0x070) #define TIM_LF_RING_BASE (0x130) #define TIM_LF_NRSPERR_INT (0x200) #define TIM_LF_NRSPERR_INT_W1S (0x208) diff --git a/drivers/event/cnxk/cnxk_tim_evdev.c b/drivers/event/cnxk/cnxk_tim_evdev.c index af67235503..f8a536e71a 100644 --- a/drivers/event/cnxk/cnxk_tim_evdev.c +++ b/drivers/event/cnxk/cnxk_tim_evdev.c @@ -119,80 +119,6 @@ cnxk_tim_ring_info_get(const struct rte_event_timer_adapter *adptr, sizeof(struct rte_event_timer_adapter_conf)); } -static inline void -sort_multi_array(double ref_arr[], uint64_t arr1[], uint64_t arr2[], - uint64_t arr3[], uint8_t sz) -{ - int x; - - for (x = 0; x < sz - 1; x++) { - if (ref_arr[x] > ref_arr[x + 1]) { - PLT_SWAP(ref_arr[x], ref_arr[x + 1]); - PLT_SWAP(arr1[x], arr1[x + 1]); - PLT_SWAP(arr2[x], arr2[x + 1]); - PLT_SWAP(arr3[x], arr3[x + 1]); - x = -1; - } - } -} - -static inline void -populate_sample(uint64_t tck[], uint64_t ns[], double diff[], uint64_t dst[], - uint64_t req_tck, uint64_t clk_freq, double tck_ns, uint8_t sz, - bool mov_fwd) -{ - int i; - - for (i = 0; i < sz; i++) { - tck[i] = i ? tck[i - 1] : req_tck; - do { - mov_fwd ? tck[i]++ : tck[i]--; - ns[i] = round((double)tck[i] * tck_ns); - if (round((double)tck[i] * tck_ns) > - ((double)tck[i] * tck_ns)) - continue; - } while (ns[i] % (uint64_t)cnxk_tim_ns_per_tck(clk_freq)); - diff[i] = PLT_MAX((double)ns[i], (double)tck[i] * tck_ns) - - PLT_MIN((double)ns[i], (double)tck[i] * tck_ns); - dst[i] = mov_fwd ? tck[i] - req_tck : req_tck - tck[i]; - } -} - -static void -tim_adjust_resolution(uint64_t *req_ns, uint64_t *req_tck, double tck_ns, - uint64_t clk_freq, uint64_t max_tmo, uint64_t m_tck) -{ -#define MAX_SAMPLES 5 - double rmax_diff[MAX_SAMPLES], rmin_diff[MAX_SAMPLES]; - uint64_t min_tck[MAX_SAMPLES], max_tck[MAX_SAMPLES]; - uint64_t min_dst[MAX_SAMPLES], max_dst[MAX_SAMPLES]; - uint64_t min_ns[MAX_SAMPLES], max_ns[MAX_SAMPLES]; - int i; - - populate_sample(max_tck, max_ns, rmax_diff, max_dst, *req_tck, clk_freq, - tck_ns, MAX_SAMPLES, true); - sort_multi_array(rmax_diff, max_dst, max_tck, max_ns, MAX_SAMPLES); - - populate_sample(min_tck, min_ns, rmin_diff, min_dst, *req_tck, clk_freq, - tck_ns, MAX_SAMPLES, false); - sort_multi_array(rmin_diff, min_dst, min_tck, min_ns, MAX_SAMPLES); - - for (i = 0; i < MAX_SAMPLES; i++) { - if (min_dst[i] < max_dst[i] && min_tck[i] > m_tck && - (max_tmo / min_ns[i]) <= - (TIM_MAX_BUCKET_SIZE - TIM_MIN_BUCKET_SIZE)) { - *req_tck = min_tck[i]; - *req_ns = min_ns[i]; - break; - } else if ((max_tmo / max_ns[i]) < - (TIM_MAX_BUCKET_SIZE - TIM_MIN_BUCKET_SIZE)) { - *req_tck = max_tck[i]; - *req_ns = max_ns[i]; - break; - } - } -} - static int cnxk_tim_ring_create(struct rte_event_timer_adapter *adptr) { @@ -263,27 +189,7 @@ cnxk_tim_ring_create(struct rte_event_timer_adapter *adptr) goto tim_hw_free; } - tim_ring->tck_nsec = - round(RTE_ALIGN_MUL_NEAR((long double)rcfg->timer_tick_ns, - cnxk_tim_ns_per_tck(clk_freq))); - if (log10(clk_freq) - floor(log10(clk_freq)) != 0.0) { - uint64_t req_ns, req_tck; - double tck_ns; - - req_ns = tim_ring->tck_nsec; - tck_ns = NSECPERSEC / clk_freq; - req_tck = round(rcfg->timer_tick_ns / tck_ns); - tim_adjust_resolution(&req_ns, &req_tck, tck_ns, clk_freq, - rcfg->max_tmo_ns, min_intvl_cyc); - if ((tim_ring->tck_nsec != req_ns) && - !(rcfg->flags & RTE_EVENT_TIMER_ADAPTER_F_ADJUST_RES)) { - rc = -ERANGE; - goto tim_hw_free; - } - tim_ring->tck_nsec = ceil(req_tck * tck_ns); - } - - tim_ring->tck_int = round((long double)tim_ring->tck_nsec / + tim_ring->tck_int = round((double)rcfg->timer_tick_ns / cnxk_tim_ns_per_tck(clk_freq)); tim_ring->tck_nsec = ceil(tim_ring->tck_int * cnxk_tim_ns_per_tck(clk_freq)); @@ -296,6 +202,13 @@ cnxk_tim_ring_create(struct rte_event_timer_adapter *adptr) tim_ring->chunk_sz = dev->chunk_sz; tim_ring->disable_npa = dev->disable_npa; tim_ring->enable_stats = dev->enable_stats; + tim_ring->base = roc_tim_lf_base_get(&dev->tim, tim_ring->ring_id); + tim_ring->tbase = cnxk_tim_get_tick_base(clk_src, tim_ring->base); + + if (roc_model_is_cn9k() && (tim_ring->clk_src == ROC_TIM_CLK_SRC_GTI)) + tim_ring->tick_fn = cnxk_tim_cntvct; + else + tim_ring->tick_fn = cnxk_tim_tick_read; for (i = 0; i < dev->ring_ctl_cnt; i++) { struct cnxk_tim_ctl *ring_ctl = &dev->ring_ctl_data[i]; @@ -342,7 +255,6 @@ cnxk_tim_ring_create(struct rte_event_timer_adapter *adptr) goto tim_chnk_free; } - tim_ring->base = roc_tim_lf_base_get(&dev->tim, tim_ring->ring_id); plt_write64((uint64_t)tim_ring->bkt, tim_ring->base + TIM_LF_RING_BASE); plt_write64(tim_ring->aura, tim_ring->base + TIM_LF_RING_AURA); @@ -391,31 +303,6 @@ cnxk_tim_ring_free(struct rte_event_timer_adapter *adptr) return 0; } -static void -cnxk_tim_calibrate_start_tsc(struct cnxk_tim_ring *tim_ring) -{ -#define CNXK_TIM_CALIB_ITER 1E6 - uint32_t real_bkt, bucket; - int icount, ecount = 0; - uint64_t bkt_cyc; - - for (icount = 0; icount < CNXK_TIM_CALIB_ITER; icount++) { - real_bkt = plt_read64(tim_ring->base + TIM_LF_RING_REL) >> 44; - bkt_cyc = cnxk_tim_cntvct(); - bucket = (bkt_cyc - tim_ring->ring_start_cyc) / - tim_ring->tck_int; - bucket = bucket % (tim_ring->nb_bkts); - tim_ring->ring_start_cyc = - bkt_cyc - (real_bkt * tim_ring->tck_int); - if (bucket != real_bkt) - ecount++; - } - tim_ring->last_updt_cyc = bkt_cyc; - plt_tim_dbg("Bucket mispredict %3.2f distance %d\n", - 100 - (((double)(icount - ecount) / (double)icount) * 100), - bucket - real_bkt); -} - static int cnxk_tim_ring_start(const struct rte_event_timer_adapter *adptr) { @@ -431,12 +318,16 @@ cnxk_tim_ring_start(const struct rte_event_timer_adapter *adptr) if (rc < 0) return rc; - tim_ring->tot_int = tim_ring->tck_int * tim_ring->nb_bkts; tim_ring->fast_div = rte_reciprocal_value_u64(tim_ring->tck_int); tim_ring->fast_bkt = rte_reciprocal_value_u64(tim_ring->nb_bkts); - cnxk_tim_calibrate_start_tsc(tim_ring); + if (roc_model_is_cn9k() && (tim_ring->clk_src == ROC_TIM_CLK_SRC_GTI)) { + uint64_t start_diff; + start_diff = cnxk_tim_cntvct(tim_ring->tbase) - + cnxk_tim_tick_read(tim_ring->tbase); + tim_ring->ring_start_cyc += start_diff; + } return rc; } @@ -462,7 +353,8 @@ cnxk_tim_stats_get(const struct rte_event_timer_adapter *adapter, struct rte_event_timer_adapter_stats *stats) { struct cnxk_tim_ring *tim_ring = adapter->data->adapter_priv; - uint64_t bkt_cyc = cnxk_tim_cntvct() - tim_ring->ring_start_cyc; + uint64_t bkt_cyc = + tim_ring->tick_fn(tim_ring->tbase) - tim_ring->ring_start_cyc; stats->evtim_exp_count = __atomic_load_n(&tim_ring->arm_cnt, __ATOMIC_RELAXED); diff --git a/drivers/event/cnxk/cnxk_tim_evdev.h b/drivers/event/cnxk/cnxk_tim_evdev.h index 91a90ee2ce..0fda9f4f13 100644 --- a/drivers/event/cnxk/cnxk_tim_evdev.h +++ b/drivers/event/cnxk/cnxk_tim_evdev.h @@ -24,14 +24,14 @@ #define CNXK_TIM_EVDEV_NAME cnxk_tim_eventdev #define CNXK_TIM_MAX_BUCKETS (0xFFFFF) -#define CNXK_TIM_RING_DEF_CHUNK_SZ (4096) +#define CNXK_TIM_RING_DEF_CHUNK_SZ (256) #define CNXK_TIM_CHUNK_ALIGNMENT (16) #define CNXK_TIM_MAX_BURST \ (RTE_CACHE_LINE_SIZE / CNXK_TIM_CHUNK_ALIGNMENT) #define CNXK_TIM_NB_CHUNK_SLOTS(sz) (((sz) / CNXK_TIM_CHUNK_ALIGNMENT) - 1) #define CNXK_TIM_MIN_CHUNK_SLOTS (0x1) #define CNXK_TIM_MAX_CHUNK_SLOTS (0x1FFE) -#define CNXK_TIM_MAX_POOL_CACHE_SZ (128) +#define CNXK_TIM_MAX_POOL_CACHE_SZ (16) #define CN9K_TIM_MIN_TMO_TKS (256) @@ -119,18 +119,18 @@ struct cnxk_tim_bkt { }; struct cnxk_tim_ring { - uintptr_t base; uint16_t nb_chunk_slots; uint32_t nb_bkts; - uint64_t last_updt_cyc; + uintptr_t tbase; + uint64_t (*tick_fn)(uint64_t tbase); uint64_t ring_start_cyc; - uint64_t tck_int; - uint64_t tot_int; struct cnxk_tim_bkt *bkt; struct rte_mempool *chunk_pool; struct rte_reciprocal_u64 fast_div; struct rte_reciprocal_u64 fast_bkt; + uint64_t tck_int; uint64_t arm_cnt; + uintptr_t base; uint8_t prod_type_sp; uint8_t enable_stats; uint8_t disable_npa; @@ -163,19 +163,19 @@ cnxk_tim_priv_get(void) return mz->addr; } -static inline long double +static inline double cnxk_tim_ns_per_tck(uint64_t freq) { - return (long double)NSECPERSEC / freq; + return (double)NSECPERSEC / freq; } #ifdef RTE_ARCH_ARM64 static inline uint64_t -cnxk_tim_cntvct(void) +cnxk_tim_cntvct(uint64_t base __rte_unused) { uint64_t tsc; - asm volatile("mrs %0, cntvct_el0" : "=r"(tsc)); + asm volatile("mrs %0, CNTVCT_EL0" : "=r"(tsc)::"memory"); return tsc; } @@ -189,7 +189,7 @@ cnxk_tim_cntfrq(void) } #else static inline uint64_t -cnxk_tim_cntvct(void) +cnxk_tim_cntvct(uint64_t base __rte_unused) { return 0; } @@ -201,6 +201,12 @@ cnxk_tim_cntfrq(void) } #endif +static inline uint64_t +cnxk_tim_tick_read(uint64_t tick_base) +{ + return plt_read64(tick_base); +} + static inline enum roc_tim_clk_src cnxk_tim_convert_clk_src(enum rte_event_timer_adapter_clk_src clk_src) { @@ -221,6 +227,27 @@ cnxk_tim_convert_clk_src(enum rte_event_timer_adapter_clk_src clk_src) } } +static inline uintptr_t +cnxk_tim_get_tick_base(enum roc_tim_clk_src clk_src, uintptr_t base) +{ + switch (clk_src) { + case ROC_TIM_CLK_SRC_GTI: + return base + TIM_LF_FR_RN_GTI; + case ROC_TIM_CLK_SRC_GPIO: + return base + TIM_LF_FR_RN_GPIOS; + case ROC_TIM_CLK_SRC_10NS: + return base + TIM_LF_FR_RN_TENNS; + case ROC_TIM_CLK_SRC_PTP: + return base + TIM_LF_FR_RN_PTP; + case ROC_TIM_CLK_SRC_SYNCE: + return base + TIM_LF_FR_RN_SYNCE; + case ROC_TIM_CLK_SRC_BTS: + return base + TIM_LF_FR_RN_BTS; + default: + return ROC_TIM_CLK_SRC_INVALID; + } +} + static inline int cnxk_tim_get_clk_freq(struct cnxk_tim_evdev *dev, enum roc_tim_clk_src clk_src, uint64_t *freq) diff --git a/drivers/event/cnxk/cnxk_tim_worker.c b/drivers/event/cnxk/cnxk_tim_worker.c index dfcfbdc797..923a72093b 100644 --- a/drivers/event/cnxk/cnxk_tim_worker.c +++ b/drivers/event/cnxk/cnxk_tim_worker.c @@ -39,22 +39,6 @@ cnxk_tim_format_event(const struct rte_event_timer *const tim, entry->wqe = tim->ev.u64; } -static inline void -cnxk_tim_sync_start_cyc(struct cnxk_tim_ring *tim_ring) -{ - uint64_t cur_cyc = cnxk_tim_cntvct(); - uint32_t real_bkt; - - if (cur_cyc - tim_ring->last_updt_cyc > tim_ring->tot_int) { - real_bkt = plt_read64(tim_ring->base + TIM_LF_RING_REL) >> 44; - cur_cyc = cnxk_tim_cntvct(); - - tim_ring->ring_start_cyc = - cur_cyc - (real_bkt * tim_ring->tck_int); - tim_ring->last_updt_cyc = cur_cyc; - } -} - static __rte_always_inline uint16_t cnxk_tim_timer_arm_burst(const struct rte_event_timer_adapter *adptr, struct rte_event_timer **tim, const uint16_t nb_timers, @@ -65,7 +49,6 @@ cnxk_tim_timer_arm_burst(const struct rte_event_timer_adapter *adptr, uint16_t index; int ret = 0; - cnxk_tim_sync_start_cyc(tim_ring); for (index = 0; index < nb_timers; index++) { if (cnxk_tim_arm_checks(tim_ring, tim[index])) break; @@ -127,7 +110,6 @@ cnxk_tim_timer_arm_tmo_brst(const struct rte_event_timer_adapter *adptr, return 0; } - cnxk_tim_sync_start_cyc(tim_ring); while (arr_idx < nb_timers) { for (idx = 0; idx < CNXK_TIM_MAX_BURST && (arr_idx < nb_timers); idx++, arr_idx++) { diff --git a/drivers/event/cnxk/cnxk_tim_worker.h b/drivers/event/cnxk/cnxk_tim_worker.h index 0c9f29cfbe..8d8ed1d3a1 100644 --- a/drivers/event/cnxk/cnxk_tim_worker.h +++ b/drivers/event/cnxk/cnxk_tim_worker.h @@ -131,7 +131,8 @@ cnxk_tim_get_target_bucket(struct cnxk_tim_ring *const tim_ring, const uint32_t rel_bkt, struct cnxk_tim_bkt **bkt, struct cnxk_tim_bkt **mirr_bkt) { - const uint64_t bkt_cyc = cnxk_tim_cntvct() - tim_ring->ring_start_cyc; + const uint64_t bkt_cyc = + tim_ring->tick_fn(tim_ring->tbase) - tim_ring->ring_start_cyc; uint64_t bucket = rte_reciprocal_divide_u64(bkt_cyc, &tim_ring->fast_div) + rel_bkt; -- 2.25.1