From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 55838A0032 for ; Fri, 24 Jun 2022 17:02:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5138D40E2D; Fri, 24 Jun 2022 17:02:28 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 6B6D640E2D for ; Fri, 24 Jun 2022 17:02:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656082947; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8LDtEZ4aVrZdtknKNMCctuzoTtF2kJj+aAluJR4GD1g=; b=bRWh4BWflFQhx65QuHS1qm/aU3kd/eLjmX58aeH0xGnORhjcPWczY4fVQILTOac3j4Mk7i bBRZly/H+gkZmFNDtPQuH0Q8f7SNLz0wPdalLDbvSq2LSz1E+BNpXaKExnaWls1HHgVGpl eduXLAqxjbErST//fEB0jIn/nX8o/CI= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-606-HD0X0N9YMCaBuoUAP9KPFg-1; Fri, 24 Jun 2022 11:02:23 -0400 X-MC-Unique: HD0X0N9YMCaBuoUAP9KPFg-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 758B13801EC0; Fri, 24 Jun 2022 15:02:23 +0000 (UTC) Received: from rh.redhat.com (unknown [10.39.194.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4C6A9D63; Fri, 24 Jun 2022 15:02:22 +0000 (UTC) From: Kevin Traynor To: Timothy McDaniel Cc: dpdk stable Subject: patch 'event/dlb2: rework queue drain handling' has been queued to stable release 21.11.2 Date: Fri, 24 Jun 2022 16:01:26 +0100 Message-Id: <20220624150148.1855580-18-ktraynor@redhat.com> In-Reply-To: <20220624150148.1855580-1-ktraynor@redhat.com> References: <20220624150148.1855580-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=ktraynor@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 21.11.2 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 06/27/22. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/kevintraynor/dpdk-stable This queued commit can be viewed at: https://github.com/kevintraynor/dpdk-stable/commit/7c0439f319503ced6777f3e0d2da213ead816b79 Thanks. Kevin --- >From 7c0439f319503ced6777f3e0d2da213ead816b79 Mon Sep 17 00:00:00 2001 From: Timothy McDaniel Date: Mon, 6 Jun 2022 10:55:17 -0500 Subject: [PATCH] event/dlb2: rework queue drain handling [ upstream commit 82a710bccb209f7b5b0309f687741102f093234f ] Previously, anything that prevented the CQs from being drained would hang the PMD for an unacceptably long time. This commit updates the drain logic to be resource and time based, thus eliminating the potential for a long hang when draining the queues in preparation for a reset. Fixes: 1857f1922ce2 ("event/dlb2: use new implementation of resource file") Signed-off-by: Timothy McDaniel --- drivers/event/dlb2/pf/base/dlb2_hw_types.h | 2 +- drivers/event/dlb2/pf/base/dlb2_resource.c | 45 +++++++++++++++++----- 2 files changed, 37 insertions(+), 10 deletions(-) diff --git a/drivers/event/dlb2/pf/base/dlb2_hw_types.h b/drivers/event/dlb2/pf/base/dlb2_hw_types.h index 6b8fee3416..9511521e67 100644 --- a/drivers/event/dlb2/pf/base/dlb2_hw_types.h +++ b/drivers/event/dlb2/pf/base/dlb2_hw_types.h @@ -28,5 +28,5 @@ #define DLB2_MAX_NUM_SEQUENCE_NUMBER_MODES 5 #define DLB2_MAX_CQ_COMP_CHECK_LOOPS 409600 -#define DLB2_MAX_QID_EMPTY_CHECK_LOOPS (32 * 64 * 1024 * (800 / 30)) +#define DLB2_MAX_QID_EMPTY_CHECK_LOOPS (4 * DLB2_MAX_NUM_LDB_CREDITS) #define DLB2_FUNC_BAR 0 diff --git a/drivers/event/dlb2/pf/base/dlb2_resource.c b/drivers/event/dlb2/pf/base/dlb2_resource.c index d4c49c2992..8c57bdfff7 100644 --- a/drivers/event/dlb2/pf/base/dlb2_resource.c +++ b/drivers/event/dlb2/pf/base/dlb2_resource.c @@ -1058,5 +1058,5 @@ static u32 dlb2_dir_cq_token_count(struct dlb2_hw *hw, } -static void dlb2_drain_dir_cq(struct dlb2_hw *hw, +static int dlb2_drain_dir_cq(struct dlb2_hw *hw, struct dlb2_dir_pq_pair *port) { @@ -1090,4 +1090,6 @@ static void dlb2_drain_dir_cq(struct dlb2_hw *hw, os_unmap_producer_port(hw, pp_addr); } + + return cnt; } @@ -1108,4 +1110,5 @@ static int dlb2_domain_drain_dir_cqs(struct dlb2_hw *hw, struct dlb2_list_entry *iter; struct dlb2_dir_pq_pair *port; + int drain_cnt = 0; RTE_SET_USED(iter); @@ -1121,5 +1124,5 @@ static int dlb2_domain_drain_dir_cqs(struct dlb2_hw *hw, dlb2_dir_port_cq_disable(hw, port); - dlb2_drain_dir_cq(hw, port); + drain_cnt = dlb2_drain_dir_cq(hw, port); if (toggle_port) @@ -1127,5 +1130,5 @@ static int dlb2_domain_drain_dir_cqs(struct dlb2_hw *hw, } - return 0; + return drain_cnt; } @@ -1171,8 +1174,18 @@ static int dlb2_domain_drain_dir_queues(struct dlb2_hw *hw, for (i = 0; i < DLB2_MAX_QID_EMPTY_CHECK_LOOPS; i++) { - dlb2_domain_drain_dir_cqs(hw, domain, true); + int drain_cnt; + + drain_cnt = dlb2_domain_drain_dir_cqs(hw, domain, false); if (dlb2_domain_dir_queues_empty(hw, domain)) break; + + /* + * Allow time for DLB to schedule QEs before draining + * the CQs again. + */ + if (!drain_cnt) + rte_delay_us(1); + } @@ -1250,5 +1263,5 @@ static u32 dlb2_ldb_cq_token_count(struct dlb2_hw *hw, } -static void dlb2_drain_ldb_cq(struct dlb2_hw *hw, struct dlb2_ldb_port *port) +static int dlb2_drain_ldb_cq(struct dlb2_hw *hw, struct dlb2_ldb_port *port) { u32 infl_cnt, tkn_cnt; @@ -1290,7 +1303,9 @@ static void dlb2_drain_ldb_cq(struct dlb2_hw *hw, struct dlb2_ldb_port *port) os_unmap_producer_port(hw, pp_addr); } + + return tkn_cnt; } -static void dlb2_domain_drain_ldb_cqs(struct dlb2_hw *hw, +static int dlb2_domain_drain_ldb_cqs(struct dlb2_hw *hw, struct dlb2_hw_domain *domain, bool toggle_port) @@ -1298,4 +1313,5 @@ static void dlb2_domain_drain_ldb_cqs(struct dlb2_hw *hw, struct dlb2_list_entry *iter; struct dlb2_ldb_port *port; + int drain_cnt = 0; int i; RTE_SET_USED(iter); @@ -1303,5 +1319,5 @@ static void dlb2_domain_drain_ldb_cqs(struct dlb2_hw *hw, /* If the domain hasn't been started, there's no traffic to drain */ if (!domain->started) - return; + return 0; for (i = 0; i < DLB2_NUM_COS_DOMAINS; i++) { @@ -1310,5 +1326,5 @@ static void dlb2_domain_drain_ldb_cqs(struct dlb2_hw *hw, dlb2_ldb_port_cq_disable(hw, port); - dlb2_drain_ldb_cq(hw, port); + drain_cnt = dlb2_drain_ldb_cq(hw, port); if (toggle_port) @@ -1316,4 +1332,6 @@ static void dlb2_domain_drain_ldb_cqs(struct dlb2_hw *hw, } } + + return drain_cnt; } @@ -1376,8 +1394,17 @@ static int dlb2_domain_drain_mapped_queues(struct dlb2_hw *hw, for (i = 0; i < DLB2_MAX_QID_EMPTY_CHECK_LOOPS; i++) { - dlb2_domain_drain_ldb_cqs(hw, domain, true); + int drain_cnt; + + drain_cnt = dlb2_domain_drain_ldb_cqs(hw, domain, false); if (dlb2_domain_mapped_queues_empty(hw, domain)) break; + + /* + * Allow time for DLB to schedule QEs before draining + * the CQs again. + */ + if (!drain_cnt) + rte_delay_us(1); } -- 2.34.3 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2022-06-24 15:51:09.576609319 +0100 +++ 0023-event-dlb2-rework-queue-drain-handling.patch 2022-06-24 15:51:08.889984130 +0100 @@ -1 +1 @@ -From 82a710bccb209f7b5b0309f687741102f093234f Mon Sep 17 00:00:00 2001 +From 7c0439f319503ced6777f3e0d2da213ead816b79 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 82a710bccb209f7b5b0309f687741102f093234f ] + @@ -13 +14,0 @@ -Cc: stable@dpdk.org