From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2608CA04E1; Tue, 22 Sep 2020 11:43:26 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 629A31DDBB; Tue, 22 Sep 2020 11:36:24 +0200 (CEST) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [67.231.154.164]) by dpdk.org (Postfix) with ESMTP id 795391DC0B for ; Tue, 22 Sep 2020 11:35:09 +0200 (CEST) Received: from mx1-us1.ppe-hosted.com (unknown [10.110.50.150]) by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 5242C20059 for ; Tue, 22 Sep 2020 09:35:09 +0000 (UTC) Received: from us4-mdac16-15.at1.mdlocal (unknown [10.110.49.197]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id 51E27800A3 for ; Tue, 22 Sep 2020 09:35:09 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx1-us1.ppe-hosted.com (unknown [10.110.49.30]) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id E5A5810004F for ; Tue, 22 Sep 2020 09:35:08 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id AD745140053 for ; Tue, 22 Sep 2020 09:35:08 +0000 (UTC) Received: from ukex01.SolarFlarecom.com (10.17.10.4) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 22 Sep 2020 10:34:58 +0100 Received: from opal.uk.solarflarecom.com (10.17.10.1) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 22 Sep 2020 10:34:58 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (ukv-loginhost.uk.solarflarecom.com [10.17.10.39]) by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id 08M9Yw4J027025; Tue, 22 Sep 2020 10:34:58 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (localhost [127.0.0.1]) by ukv-loginhost.uk.solarflarecom.com (Postfix) with ESMTP id 38F851613CB; Tue, 22 Sep 2020 10:34:58 +0100 (BST) From: Andrew Rybchenko To: CC: Andy Moreton Date: Tue, 22 Sep 2020 10:34:45 +0100 Message-ID: <1600767288-12109-58-git-send-email-arybchenko@solarflare.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1600767288-12109-1-git-send-email-arybchenko@solarflare.com> References: <1600764594-14752-1-git-send-email-arybchenko@solarflare.com> <1600767288-12109-1-git-send-email-arybchenko@solarflare.com> MIME-Version: 1.0 Content-Type: text/plain X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25674.003 X-TM-AS-Result: No-4.197500-8.000000-10 X-TMASE-MatchedRID: zZSWqN8v4xXm8Tq/Qity6m0lh85vSrk7AKbvziCwm7j82ks92f+Gmki5 7L1pIMlU4f0uNTtAwnss/31GzKkTsX8f69f8DWLaPwKTD1v8YV5MkOX0UoduufdG7cmuMnEoeKa jvO6uWnBgS8jnMi9jc4O9WUy4j2mWTj/OlUrKzKxbUzvsaHW6BuqhuTPUDQDthNrreBB7zJkU+5 qF9ri1FdgjUjMlWxbFFFGVHCK0F0mpmFGiu41fIwdc+KHUCr/8BGvINcfHqheuiyJNZGE/if+Mw M4d0pxTD1xtEq6z2nVwdMejP5s0hcMlA9qlu+4CB89GKHo03naZ2scyRQcerx53XUX0iwoUBrP/ oSIO7kFvgkneme+Cc+PaI7IZzuYes3GguTHcJA66iJsmkdGsWX0tCKdnhB589yM15V5aWpj6C0e Ps7A07cKVf7avePdYUXbbVZBHn+4k4ehBSgR8013nTFng8UGpQ1TubdZyzhwjfi8BgS3O2gWf9j qWhAkI+sSq3aUvwfKVw8yL47PZqi3H5IamPfkL2kC7SwvGlKhDwb7Jglhh1lFkOwaJdCKgviPEq cclEtzAvpLE+mvX8g== X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 10--4.197500-8.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25674.003 X-MDID: 1600767309-kNwP_YCGmuhj Subject: [dpdk-dev] [PATCH v2 57/60] common/sfc_efx/base: poll extended width event queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Andy Moreton Extended width queues use a different layout and so require a different polling loop. Signed-off-by: Andy Moreton Signed-off-by: Andrew Rybchenko --- drivers/common/sfc_efx/base/rhead_ev.c | 155 ++++++++++++++++++++++++- 1 file changed, 154 insertions(+), 1 deletion(-) diff --git a/drivers/common/sfc_efx/base/rhead_ev.c b/drivers/common/sfc_efx/base/rhead_ev.c index a83c6150d8..df5e3e26c6 100644 --- a/drivers/common/sfc_efx/base/rhead_ev.c +++ b/drivers/common/sfc_efx/base/rhead_ev.c @@ -30,7 +30,6 @@ rhead_ev_tx_completion( __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); - static __checkReturn boolean_t rhead_ev_mcdi( __in efx_evq_t *eep, @@ -38,6 +37,22 @@ rhead_ev_mcdi( __in const efx_ev_callbacks_t *eecp, __in_opt void *arg); +#if EFSYS_OPT_EV_EXTENDED_WIDTH +static boolean_t +rhead_ev_ew_dispatch( + __in efx_evq_t *eep, + __in efx_xword_t *eventp, + __in const efx_ev_callbacks_t *eecp, + __in_opt void *arg); + +static void +rhead_ev_ew_qpoll( + __in efx_evq_t *eep, + __inout unsigned int *countp, + __in const efx_ev_callbacks_t *eecp, + __in_opt void *arg); +#endif /* EFSYS_OPT_EV_EXTENDED_WIDTH */ + __checkReturn efx_rc_t rhead_ev_init( @@ -200,6 +215,13 @@ rhead_ev_qpoll( unsigned int index; size_t offset; +#if EFSYS_OPT_EV_EXTENDED_WIDTH + if (eep->ee_flags & EFX_EVQ_FLAGS_EXTENDED_WIDTH) { + rhead_ev_ew_qpoll(eep, countp, eecp, arg); + return; + } +#endif /* EFSYS_OPT_EV_EXTENDED_WIDTH */ + EFSYS_ASSERT3U(eep->ee_magic, ==, EFX_EVQ_MAGIC); EFSYS_ASSERT(countp != NULL); EFSYS_ASSERT(eecp != NULL); @@ -285,6 +307,137 @@ rhead_ev_qpoll( *countp = count; } +#if EFSYS_OPT_EV_EXTENDED_WIDTH +static boolean_t +rhead_ev_ew_dispatch( + __in efx_evq_t *eep, + __in efx_xword_t *eventp, + __in const efx_ev_callbacks_t *eecp, + __in_opt void *arg) +{ + boolean_t should_abort; + uint32_t code; + + EFSYS_ASSERT((eep->ee_flags & EFX_EVQ_FLAGS_EXTENDED_WIDTH) != 0); + + code = EFX_XWORD_FIELD(*eventp, ESF_GZ_EV_256_EV32_TYPE); + switch (code) { + default: + /* Omit currently unused reserved bits from the probe. */ + EFSYS_PROBE7(ew_bad_event, unsigned int, eep->ee_index, + uint32_t, EFX_XWORD_FIELD(*eventp, EFX_DWORD_7), + uint32_t, EFX_XWORD_FIELD(*eventp, EFX_DWORD_4), + uint32_t, EFX_XWORD_FIELD(*eventp, EFX_DWORD_3), + uint32_t, EFX_XWORD_FIELD(*eventp, EFX_DWORD_2), + uint32_t, EFX_XWORD_FIELD(*eventp, EFX_DWORD_1), + uint32_t, EFX_XWORD_FIELD(*eventp, EFX_DWORD_0)); + + EFSYS_ASSERT(eecp->eec_exception != NULL); + (void) eecp->eec_exception(arg, EFX_EXCEPTION_EV_ERROR, code); + should_abort = B_TRUE; + } + + return (should_abort); +} + +/* + * Poll extended width event queue. Size of the batch is equal to cache line + * size divided by event size. + */ +#define EF100_EV_EW_BATCH 2 + +/* + * Check if event is present. + * + * Riverhead EvQs use a phase bit to indicate the presence of valid events, + * by flipping the phase bit on each wrap of the write index. + */ +#define EF100_EV_EW_PRESENT(_xword, _phase_bit) \ + (EFX_XWORD_FIELD((_xword), ESF_GZ_EV_256_EV32_PHASE) == (_phase_bit)) + +static void +rhead_ev_ew_qpoll( + __in efx_evq_t *eep, + __inout unsigned int *countp, + __in const efx_ev_callbacks_t *eecp, + __in_opt void *arg) +{ + efx_xword_t ev[EF100_EV_EW_BATCH]; + unsigned int batch; + unsigned int phase_bit; + unsigned int total; + unsigned int count; + unsigned int index; + size_t offset; + + EFSYS_ASSERT3U(eep->ee_magic, ==, EFX_EVQ_MAGIC); + EFSYS_ASSERT((eep->ee_flags & EFX_EVQ_FLAGS_EXTENDED_WIDTH) != 0); + EFSYS_ASSERT(countp != NULL); + EFSYS_ASSERT(eecp != NULL); + + count = *countp; + do { + /* Read up until the end of the batch period */ + batch = EF100_EV_EW_BATCH - (count & (EF100_EV_EW_BATCH - 1)); + phase_bit = (count & (eep->ee_mask + 1)) != 0; + offset = (count & eep->ee_mask) * sizeof (efx_xword_t); + for (total = 0; total < batch; ++total) { + EFSYS_MEM_READX(eep->ee_esmp, offset, &(ev[total])); + + if (!EF100_EV_EW_PRESENT(ev[total], phase_bit)) + break; + + /* Omit unused reserved bits from the probe. */ + EFSYS_PROBE7(ew_event, unsigned int, eep->ee_index, + uint32_t, EFX_XWORD_FIELD(ev[total], EFX_DWORD_7), + uint32_t, EFX_XWORD_FIELD(ev[total], EFX_DWORD_4), + uint32_t, EFX_XWORD_FIELD(ev[total], EFX_DWORD_3), + uint32_t, EFX_XWORD_FIELD(ev[total], EFX_DWORD_2), + uint32_t, EFX_XWORD_FIELD(ev[total], EFX_DWORD_1), + uint32_t, EFX_XWORD_FIELD(ev[total], EFX_DWORD_0)); + + offset += sizeof (efx_xword_t); + } + + /* Process the batch of events */ + for (index = 0; index < total; ++index) { + boolean_t should_abort; + + EFX_EV_QSTAT_INCR(eep, EV_ALL); + + should_abort = + rhead_ev_ew_dispatch(eep, &(ev[index]), eecp, arg); + + if (should_abort) { + /* Ignore subsequent events */ + total = index + 1; + + /* + * Poison batch to ensure the outer + * loop is broken out of. + */ + EFSYS_ASSERT(batch <= EF100_EV_EW_BATCH); + batch += (EF100_EV_EW_BATCH << 1); + EFSYS_ASSERT(total != batch); + break; + } + } + + /* + * There is no necessity to clear processed events since + * phase bit which is flipping on each write index wrap + * is used for event presence indication. + */ + + count += total; + + } while (total == batch); + + *countp = count; +} +#endif /* EFSYS_OPT_EV_EXTENDED_WIDTH */ + + __checkReturn efx_rc_t rhead_ev_qmoderate( __in efx_evq_t *eep, -- 2.17.1