From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 10D9AA04B1; Thu, 24 Sep 2020 14:14:05 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CF62C1DE37; Thu, 24 Sep 2020 14:13:06 +0200 (CEST) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [148.163.129.52]) by dpdk.org (Postfix) with ESMTP id CE5761DE05 for ; Thu, 24 Sep 2020 14:12:53 +0200 (CEST) Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.62]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 572DB6008B for ; Thu, 24 Sep 2020 12:12:53 +0000 (UTC) Received: from us4-mdac16-30.ut7.mdlocal (unknown [10.7.66.140]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 560788009E for ; Thu, 24 Sep 2020 12:12:53 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.7.65.175]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id CC29D280050 for ; Thu, 24 Sep 2020 12:12:52 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 83985700080 for ; Thu, 24 Sep 2020 12:12:52 +0000 (UTC) Received: from ukex01.SolarFlarecom.com (10.17.10.4) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 24 Sep 2020 13:12:46 +0100 Received: from opal.uk.solarflarecom.com (10.17.10.1) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 24 Sep 2020 13:12:45 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (ukv-loginhost.uk.solarflarecom.com [10.17.10.39]) by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id 08OCCj1p025856 for ; Thu, 24 Sep 2020 13:12:45 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (localhost [127.0.0.1]) by ukv-loginhost.uk.solarflarecom.com (Postfix) with ESMTP id D2EA11613CB for ; Thu, 24 Sep 2020 13:12:45 +0100 (BST) From: Andrew Rybchenko To: Date: Thu, 24 Sep 2020 13:11:38 +0100 Message-ID: <1600949555-28043-4-git-send-email-arybchenko@solarflare.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1600949555-28043-1-git-send-email-arybchenko@solarflare.com> References: <1600764594-14752-1-git-send-email-arybchenko@solarflare.com> <1600949555-28043-1-git-send-email-arybchenko@solarflare.com> MIME-Version: 1.0 Content-Type: text/plain X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25674.003 X-TM-AS-Result: No-0.593400-8.000000-10 X-TMASE-MatchedRID: ODoBVYkg5ew5s4xK97Sm/y2416nc3bQleouvej40T4iOSVCvVHWJJ3Io zGa69omdrdoLblq9S5ppXb8sy/poSXgz4aOScPlwrSAIWhdbeu/YuVu0X/rOkHQWhLVqLFM1L9s LsKTPVsdLppfJBfUpsB1WgSKhcF9mO5sXKty63LOYHemjphKeG0ewdu9S21aiTuMthH6Q7+axhF PYXnMHTTxKBm/JHgg6PVl7l1/G8qhOP86VSsrMrCyKzJY7d2nb9oVAKMBioqcOkJQR4QWbsIaxU TVLPuXrRGLtePI1ArxUgpn09r97XhmNqsUuotRsKrDHzH6zmUU1byNCHOgAtWA8lgItGhROIhkP ISqGf1zjHA7psXrHJVixjdsX+oPcLHYzuCshKgieAiCmPx4NwBnUJ0Ek6yhjxEHRux+uk8ifEzJ 5hPndGVinvXXlRB70tOUL3OUSzITZOAcPZFIlbS088my88Cn2XG+uKPLjRvJaDmWVQIAYx5Gubb jGCH9piJRo85ODGlwFrlMGI5VhVRySoBCeolQDq8uPJC94SFgR06l6KId4N3r0M0SKNnXWVlxr1 FJij9s= X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--0.593400-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25674.003 X-MDID: 1600949573-bfAcTz3tLfgI Subject: [dpdk-dev] [PATCH v3 03/60] common/sfc_efx/base: add event queue operation to do polling X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Event queue host interface differ on Riverhead and implementation will be different. Signed-off-by: Andrew Rybchenko Reviewed-by: Andy Moreton --- drivers/common/sfc_efx/base/efx_ev.c | 325 ++++++++++++++----------- drivers/common/sfc_efx/base/efx_impl.h | 2 + 2 files changed, 182 insertions(+), 145 deletions(-) diff --git a/drivers/common/sfc_efx/base/efx_ev.c b/drivers/common/sfc_efx/base/efx_ev.c index e6a8d4ca1a..21fddfb64d 100644 --- a/drivers/common/sfc_efx/base/efx_ev.c +++ b/drivers/common/sfc_efx/base/efx_ev.c @@ -66,6 +66,17 @@ siena_ev_qstats_update( #endif /* EFSYS_OPT_SIENA */ +#if EFX_OPTS_EF10() || EFSYS_OPT_SIENA + +static void +siena_ef10_ev_qpoll( + __in efx_evq_t *eep, + __inout unsigned int *countp, + __in const efx_ev_callbacks_t *eecp, + __in_opt void *arg); + +#endif /* EFX_OPTS_EF10() || EFSYS_OPT_SIENA */ + #if EFSYS_OPT_SIENA static const efx_ev_ops_t __efx_ev_siena_ops = { siena_ev_init, /* eevo_init */ @@ -74,6 +85,7 @@ static const efx_ev_ops_t __efx_ev_siena_ops = { siena_ev_qdestroy, /* eevo_qdestroy */ siena_ev_qprime, /* eevo_qprime */ siena_ev_qpost, /* eevo_qpost */ + siena_ef10_ev_qpoll, /* eevo_qpoll */ siena_ev_qmoderate, /* eevo_qmoderate */ #if EFSYS_OPT_QSTATS siena_ev_qstats_update, /* eevo_qstats_update */ @@ -89,6 +101,7 @@ static const efx_ev_ops_t __efx_ev_ef10_ops = { ef10_ev_qdestroy, /* eevo_qdestroy */ ef10_ev_qprime, /* eevo_qprime */ ef10_ev_qpost, /* eevo_qpost */ + siena_ef10_ev_qpoll, /* eevo_qpoll */ ef10_ev_qmoderate, /* eevo_qmoderate */ #if EFSYS_OPT_QSTATS ef10_ev_qstats_update, /* eevo_qstats_update */ @@ -374,8 +387,6 @@ efx_ev_qprefetch( #endif /* EFSYS_OPT_EV_PREFETCH */ -#define EFX_EV_BATCH 8 - void efx_ev_qpoll( __in efx_evq_t *eep, @@ -383,153 +394,15 @@ efx_ev_qpoll( __in const efx_ev_callbacks_t *eecp, __in_opt void *arg) { - efx_qword_t ev[EFX_EV_BATCH]; - unsigned int batch; - unsigned int total; - unsigned int count; - unsigned int index; - size_t offset; - - /* Ensure events codes match for EF10 (Huntington/Medford) and Siena */ - EFX_STATIC_ASSERT(ESF_DZ_EV_CODE_LBN == FSF_AZ_EV_CODE_LBN); - EFX_STATIC_ASSERT(ESF_DZ_EV_CODE_WIDTH == FSF_AZ_EV_CODE_WIDTH); - - EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_RX_EV == FSE_AZ_EV_CODE_RX_EV); - EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_TX_EV == FSE_AZ_EV_CODE_TX_EV); - EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_DRIVER_EV == FSE_AZ_EV_CODE_DRIVER_EV); - EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_DRV_GEN_EV == - FSE_AZ_EV_CODE_DRV_GEN_EV); -#if EFSYS_OPT_MCDI - EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_MCDI_EV == - FSE_AZ_EV_CODE_MCDI_EVRESPONSE); -#endif + efx_nic_t *enp = eep->ee_enp; + const efx_ev_ops_t *eevop = enp->en_eevop; EFSYS_ASSERT3U(eep->ee_magic, ==, EFX_EVQ_MAGIC); - EFSYS_ASSERT(countp != NULL); - EFSYS_ASSERT(eecp != NULL); - - count = *countp; - do { - /* Read up until the end of the batch period */ - batch = EFX_EV_BATCH - (count & (EFX_EV_BATCH - 1)); - offset = (count & eep->ee_mask) * sizeof (efx_qword_t); - for (total = 0; total < batch; ++total) { - EFSYS_MEM_READQ(eep->ee_esmp, offset, &(ev[total])); - - if (!EFX_EV_PRESENT(ev[total])) - break; - - EFSYS_PROBE3(event, unsigned int, eep->ee_index, - uint32_t, EFX_QWORD_FIELD(ev[total], EFX_DWORD_1), - uint32_t, EFX_QWORD_FIELD(ev[total], EFX_DWORD_0)); - - offset += sizeof (efx_qword_t); - } - -#if EFSYS_OPT_EV_PREFETCH && (EFSYS_OPT_EV_PREFETCH_PERIOD > 1) - /* - * Prefetch the next batch when we get within PREFETCH_PERIOD - * of a completed batch. If the batch is smaller, then prefetch - * immediately. - */ - if (total == batch && total < EFSYS_OPT_EV_PREFETCH_PERIOD) - EFSYS_MEM_PREFETCH(eep->ee_esmp, offset); -#endif /* EFSYS_OPT_EV_PREFETCH */ - - /* Process the batch of events */ - for (index = 0; index < total; ++index) { - boolean_t should_abort; - uint32_t code; - -#if EFSYS_OPT_EV_PREFETCH - /* Prefetch if we've now reached the batch period */ - if (total == batch && - index + EFSYS_OPT_EV_PREFETCH_PERIOD == total) { - offset = (count + batch) & eep->ee_mask; - offset *= sizeof (efx_qword_t); - - EFSYS_MEM_PREFETCH(eep->ee_esmp, offset); - } -#endif /* EFSYS_OPT_EV_PREFETCH */ - - EFX_EV_QSTAT_INCR(eep, EV_ALL); - code = EFX_QWORD_FIELD(ev[index], FSF_AZ_EV_CODE); - switch (code) { - case FSE_AZ_EV_CODE_RX_EV: - should_abort = eep->ee_rx(eep, - &(ev[index]), eecp, arg); - break; - case FSE_AZ_EV_CODE_TX_EV: - should_abort = eep->ee_tx(eep, - &(ev[index]), eecp, arg); - break; - case FSE_AZ_EV_CODE_DRIVER_EV: - should_abort = eep->ee_driver(eep, - &(ev[index]), eecp, arg); - break; - case FSE_AZ_EV_CODE_DRV_GEN_EV: - should_abort = eep->ee_drv_gen(eep, - &(ev[index]), eecp, arg); - break; -#if EFSYS_OPT_MCDI - case FSE_AZ_EV_CODE_MCDI_EVRESPONSE: - should_abort = eep->ee_mcdi(eep, - &(ev[index]), eecp, arg); - break; -#endif - case FSE_AZ_EV_CODE_GLOBAL_EV: - if (eep->ee_global) { - should_abort = eep->ee_global(eep, - &(ev[index]), eecp, arg); - break; - } - /* else fallthrough */ - default: - EFSYS_PROBE3(bad_event, - unsigned int, eep->ee_index, - uint32_t, - EFX_QWORD_FIELD(ev[index], EFX_DWORD_1), - uint32_t, - EFX_QWORD_FIELD(ev[index], EFX_DWORD_0)); - - EFSYS_ASSERT(eecp->eec_exception != NULL); - (void) eecp->eec_exception(arg, - EFX_EXCEPTION_EV_ERROR, code); - should_abort = B_TRUE; - } - if (should_abort) { - /* Ignore subsequent events */ - total = index + 1; - - /* - * Poison batch to ensure the outer - * loop is broken out of. - */ - EFSYS_ASSERT(batch <= EFX_EV_BATCH); - batch += (EFX_EV_BATCH << 1); - EFSYS_ASSERT(total != batch); - break; - } - } - - /* - * Now that the hardware has most likely moved onto dma'ing - * into the next cache line, clear the processed events. Take - * care to only clear out events that we've processed - */ - EFX_SET_QWORD(ev[0]); - offset = (count & eep->ee_mask) * sizeof (efx_qword_t); - for (index = 0; index < total; ++index) { - EFSYS_MEM_WRITEQ(eep->ee_esmp, offset, &(ev[0])); - offset += sizeof (efx_qword_t); - } - - count += total; - - } while (total == batch); + EFSYS_ASSERT(eevop != NULL && + eevop->eevo_qpoll != NULL); - *countp = count; + eevop->eevo_qpoll(eep, countp, eecp, arg); } void @@ -1484,3 +1357,165 @@ siena_ev_fini( } #endif /* EFSYS_OPT_SIENA */ + +#if EFX_OPTS_EF10() || EFSYS_OPT_SIENA + +#define EFX_EV_BATCH 8 + +static void +siena_ef10_ev_qpoll( + __in efx_evq_t *eep, + __inout unsigned int *countp, + __in const efx_ev_callbacks_t *eecp, + __in_opt void *arg) +{ + efx_qword_t ev[EFX_EV_BATCH]; + unsigned int batch; + unsigned int total; + unsigned int count; + unsigned int index; + size_t offset; + + /* Ensure events codes match for EF10 (Huntington/Medford) and Siena */ + EFX_STATIC_ASSERT(ESF_DZ_EV_CODE_LBN == FSF_AZ_EV_CODE_LBN); + EFX_STATIC_ASSERT(ESF_DZ_EV_CODE_WIDTH == FSF_AZ_EV_CODE_WIDTH); + + EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_RX_EV == FSE_AZ_EV_CODE_RX_EV); + EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_TX_EV == FSE_AZ_EV_CODE_TX_EV); + EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_DRIVER_EV == FSE_AZ_EV_CODE_DRIVER_EV); + EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_DRV_GEN_EV == + FSE_AZ_EV_CODE_DRV_GEN_EV); +#if EFSYS_OPT_MCDI + EFX_STATIC_ASSERT(ESE_DZ_EV_CODE_MCDI_EV == + FSE_AZ_EV_CODE_MCDI_EVRESPONSE); +#endif + + EFSYS_ASSERT3U(eep->ee_magic, ==, EFX_EVQ_MAGIC); + EFSYS_ASSERT(countp != NULL); + EFSYS_ASSERT(eecp != NULL); + + count = *countp; + do { + /* Read up until the end of the batch period */ + batch = EFX_EV_BATCH - (count & (EFX_EV_BATCH - 1)); + offset = (count & eep->ee_mask) * sizeof (efx_qword_t); + for (total = 0; total < batch; ++total) { + EFSYS_MEM_READQ(eep->ee_esmp, offset, &(ev[total])); + + if (!EFX_EV_PRESENT(ev[total])) + break; + + EFSYS_PROBE3(event, unsigned int, eep->ee_index, + uint32_t, EFX_QWORD_FIELD(ev[total], EFX_DWORD_1), + uint32_t, EFX_QWORD_FIELD(ev[total], EFX_DWORD_0)); + + offset += sizeof (efx_qword_t); + } + +#if EFSYS_OPT_EV_PREFETCH && (EFSYS_OPT_EV_PREFETCH_PERIOD > 1) + /* + * Prefetch the next batch when we get within PREFETCH_PERIOD + * of a completed batch. If the batch is smaller, then prefetch + * immediately. + */ + if (total == batch && total < EFSYS_OPT_EV_PREFETCH_PERIOD) + EFSYS_MEM_PREFETCH(eep->ee_esmp, offset); +#endif /* EFSYS_OPT_EV_PREFETCH */ + + /* Process the batch of events */ + for (index = 0; index < total; ++index) { + boolean_t should_abort; + uint32_t code; + +#if EFSYS_OPT_EV_PREFETCH + /* Prefetch if we've now reached the batch period */ + if (total == batch && + index + EFSYS_OPT_EV_PREFETCH_PERIOD == total) { + offset = (count + batch) & eep->ee_mask; + offset *= sizeof (efx_qword_t); + + EFSYS_MEM_PREFETCH(eep->ee_esmp, offset); + } +#endif /* EFSYS_OPT_EV_PREFETCH */ + + EFX_EV_QSTAT_INCR(eep, EV_ALL); + + code = EFX_QWORD_FIELD(ev[index], FSF_AZ_EV_CODE); + switch (code) { + case FSE_AZ_EV_CODE_RX_EV: + should_abort = eep->ee_rx(eep, + &(ev[index]), eecp, arg); + break; + case FSE_AZ_EV_CODE_TX_EV: + should_abort = eep->ee_tx(eep, + &(ev[index]), eecp, arg); + break; + case FSE_AZ_EV_CODE_DRIVER_EV: + should_abort = eep->ee_driver(eep, + &(ev[index]), eecp, arg); + break; + case FSE_AZ_EV_CODE_DRV_GEN_EV: + should_abort = eep->ee_drv_gen(eep, + &(ev[index]), eecp, arg); + break; +#if EFSYS_OPT_MCDI + case FSE_AZ_EV_CODE_MCDI_EVRESPONSE: + should_abort = eep->ee_mcdi(eep, + &(ev[index]), eecp, arg); + break; +#endif + case FSE_AZ_EV_CODE_GLOBAL_EV: + if (eep->ee_global) { + should_abort = eep->ee_global(eep, + &(ev[index]), eecp, arg); + break; + } + /* else fallthrough */ + default: + EFSYS_PROBE3(bad_event, + unsigned int, eep->ee_index, + uint32_t, + EFX_QWORD_FIELD(ev[index], EFX_DWORD_1), + uint32_t, + EFX_QWORD_FIELD(ev[index], EFX_DWORD_0)); + + EFSYS_ASSERT(eecp->eec_exception != NULL); + (void) eecp->eec_exception(arg, + EFX_EXCEPTION_EV_ERROR, code); + should_abort = B_TRUE; + } + if (should_abort) { + /* Ignore subsequent events */ + total = index + 1; + + /* + * Poison batch to ensure the outer + * loop is broken out of. + */ + EFSYS_ASSERT(batch <= EFX_EV_BATCH); + batch += (EFX_EV_BATCH << 1); + EFSYS_ASSERT(total != batch); + break; + } + } + + /* + * Now that the hardware has most likely moved onto dma'ing + * into the next cache line, clear the processed events. Take + * care to only clear out events that we've processed + */ + EFX_SET_QWORD(ev[0]); + offset = (count & eep->ee_mask) * sizeof (efx_qword_t); + for (index = 0; index < total; ++index) { + EFSYS_MEM_WRITEQ(eep->ee_esmp, offset, &(ev[0])); + offset += sizeof (efx_qword_t); + } + + count += total; + + } while (total == batch); + + *countp = count; +} + +#endif /* EFX_OPTS_EF10() || EFSYS_OPT_SIENA */ diff --git a/drivers/common/sfc_efx/base/efx_impl.h b/drivers/common/sfc_efx/base/efx_impl.h index 87c7d5df52..1ef68e2d3a 100644 --- a/drivers/common/sfc_efx/base/efx_impl.h +++ b/drivers/common/sfc_efx/base/efx_impl.h @@ -85,6 +85,8 @@ typedef struct efx_ev_ops_s { void (*eevo_qdestroy)(efx_evq_t *); efx_rc_t (*eevo_qprime)(efx_evq_t *, unsigned int); void (*eevo_qpost)(efx_evq_t *, uint16_t); + void (*eevo_qpoll)(efx_evq_t *, unsigned int *, + const efx_ev_callbacks_t *, void *); efx_rc_t (*eevo_qmoderate)(efx_evq_t *, unsigned int); #if EFSYS_OPT_QSTATS void (*eevo_qstats_update)(efx_evq_t *, efsys_stat_t *); -- 2.17.1