From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2F1A2A051C for ; Tue, 11 Feb 2020 12:31:00 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 266992BE9; Tue, 11 Feb 2020 12:31:00 +0100 (CET) Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by dpdk.org (Postfix) with ESMTP id 0AC592BE9 for ; Tue, 11 Feb 2020 12:30:59 +0100 (CET) Received: by mail-wr1-f43.google.com with SMTP id z7so11847249wrl.13 for ; Tue, 11 Feb 2020 03:30:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=suMEw0oX1UwpzeEfRKWUF5YfmFfaI5vY+zYnX2T/bsk=; b=eV9erak1n9fjdEfoi/PHfXDCxHIsvUqcUeo5TI2KyMK1EP7MEW1q41m+DkiE7HH3b1 ZbLoxT4Qrjzf//7Pss7hB1dL0X+EdMXOa8YY/nChS36GVCnhSywMeay8cUcD6H3Sk+s7 zKugA3xI5bOdvXCvWCvLK9hDYygVqbkAyPSpTCwFTNVK9zoC6WxLSap6RxfHMBeyBR+H DFCeUkbjFFXB2E9K1IgC5JexR0/tkejhUle2EtS4oS1mTqRsTikeXFY+NkN5ysJGDX5w 4AeAYgZFrqp1aZ3zU5RMUmZKLZEdZwGlsjNSIEFbGiR6I0diGNBg3D227giFD6pNWNgL o9eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=suMEw0oX1UwpzeEfRKWUF5YfmFfaI5vY+zYnX2T/bsk=; b=Kt9RUjU5WsOba0ic7GetFdewUBYn5Gs1poH7C78vpix1cJ4Ss+oZI2OvbpfIkU/NKz JE/O5lDDVWDrjZ9krIAtXhQgU9cS5M7BcmWrewG6jOiHiwDZR1doxuT4HpijCq03748Y rg2/z83ajE8JUiSIOSGDwD5gt64nlLj1fT6RjuFnl5M7r6l2Fm3FeCOrEQYlFbzJ/aBq x1TRCHb6iWhPm7QX3EVV9BfQ0u2Z0ZZhTodLi2N/+Y2zsPaXK73L5/B176WhOnuQBQDC ej/jBmHFnhLNzwFRk2716L132/ZLsqNB/TlMPXv0OOnryhp8/9AHCbCqFyiWGcphG1SU 85wg== X-Gm-Message-State: APjAAAXEd+puXyIUVZdnD2IIs+9wVMd5bbRLZB0fdKA+wAxDsGZeQeVB 1UqRcG1fi2WHF0bG/MIq9Fg= X-Google-Smtp-Source: APXvYqy8O2at7iQusC4Ghum9fnKnNKBxg21SjtVqQ9GFz2fbELzO/2Q34uNRLEJ2mAYn4rhoE4Sgwg== X-Received: by 2002:adf:e5c2:: with SMTP id a2mr7807725wrn.85.1581420658762; Tue, 11 Feb 2020 03:30:58 -0800 (PST) Received: from localhost ([88.98.246.218]) by smtp.gmail.com with ESMTPSA id c13sm5126164wrx.9.2020.02.11.03.30.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Feb 2020 03:30:58 -0800 (PST) From: luca.boccassi@gmail.com To: Viacheslav Ovsiienko Cc: Matan Azrad , dpdk stable Date: Tue, 11 Feb 2020 11:21:05 +0000 Message-Id: <20200211112216.3929-119-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200211112216.3929-1-luca.boccassi@gmail.com> References: <20200211112216.3929-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'net/mlx5: engage free on completion queue' has been queued to stable release 19.11.1 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 19.11.1 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 02/13/20. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Thanks. Luca Boccassi --- >From b79505fd8360c91c2fc31a423c7c4b2f0e7b85ed Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Thu, 9 Jan 2020 17:16:07 +0000 Subject: [PATCH] net/mlx5: engage free on completion queue [ upstream commit 89ef76db3b7627d4a7e656c7e51807b5bf8bdedc ] The free on completion queue keeps the indices of elts array, all mbuf stored below this index should be freed on arrival of normal send completion. In debug version it also contains an index of completed transmitting descriptor (WQE) to check queues synchronization. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5_rxtx.c | 33 +++++++++++++++++---------------- drivers/net/mlx5/mlx5_rxtx.h | 4 +--- drivers/net/mlx5/mlx5_txq.c | 2 -- 3 files changed, 18 insertions(+), 21 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index f8de2a035b..67cafd1e2c 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -2043,8 +2043,7 @@ mlx5_tx_comp_flush(struct mlx5_txq_data *restrict txq, uint16_t tail; txq->wqe_pi = rte_be_to_cpu_16(last_cqe->wqe_counter); - tail = ((volatile struct mlx5_wqe_cseg *) - (txq->wqes + (txq->wqe_pi & txq->wqe_m)))->misc; + tail = txq->fcqs[(txq->cq_ci - 1) & txq->cqe_m]; if (likely(tail != txq->elts_tail)) { mlx5_tx_free_elts(txq, tail, olx); assert(tail == txq->elts_tail); @@ -2095,6 +2094,7 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *restrict txq, * here, before we might perform SQ reset. */ rte_wmb(); + txq->cq_ci = ci; ret = mlx5_tx_error_cqe_handle (txq, (volatile struct mlx5_err_cqe *)cqe); if (unlikely(ret < 0)) { @@ -2108,17 +2108,18 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *restrict txq, /* * We are going to fetch all entries with * MLX5_CQE_SYNDROME_WR_FLUSH_ERR status. + * The send queue is supposed to be empty. */ ++ci; + txq->cq_pi = ci; + last_cqe = NULL; continue; } /* Normal transmit completion. */ + assert(ci != txq->cq_pi); + assert((txq->fcqs[ci & txq->cqe_m] >> 16) == cqe->wqe_counter); ++ci; last_cqe = cqe; -#ifndef NDEBUG - if (txq->cq_pi) - --txq->cq_pi; -#endif /* * We have to restrict the amount of processed CQEs * in one tx_burst routine call. The CQ may be large @@ -2127,7 +2128,7 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *restrict txq, * multiple iterations may introduce significant * latency. */ - if (--count == 0) + if (likely(--count == 0)) break; } while (true); if (likely(ci != txq->cq_ci)) { @@ -2177,15 +2178,15 @@ mlx5_tx_request_completion(struct mlx5_txq_data *restrict txq, /* Request unconditional completion on last WQE. */ last->cseg.flags = RTE_BE32(MLX5_COMP_ALWAYS << MLX5_COMP_MODE_OFFSET); - /* Save elts_head in unused "immediate" field of WQE. */ - last->cseg.misc = head; - /* - * A CQE slot must always be available. Count the - * issued CEQ "always" request instead of production - * index due to here can be CQE with errors and - * difference with ci may become inconsistent. - */ - assert(txq->cqe_s > ++txq->cq_pi); + /* Save elts_head in dedicated free on completion queue. */ +#ifdef NDEBUG + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head; +#else + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head | + (last->cseg.opcode >> 8) << 16; +#endif + /* A CQE slot must always be available. */ + assert((txq->cq_pi - txq->cq_ci) <= txq->cqe_s); } } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 7d1b2fa10c..e362b4afe0 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -273,9 +273,7 @@ struct mlx5_txq_data { uint16_t wqe_thres; /* WQE threshold to request completion in CQ. */ /* WQ related fields. */ uint16_t cq_ci; /* Consumer index for completion queue. */ -#ifndef NDEBUG - uint16_t cq_pi; /* Counter of issued CQE "always" requests. */ -#endif + uint16_t cq_pi; /* Production index for completion queue. */ uint16_t cqe_s; /* Number of CQ elements. */ uint16_t cqe_m; /* Mask for CQ indices. */ /* CQ related fields. */ diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 2db11200d3..1a76f6ea83 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -717,9 +717,7 @@ mlx5_txq_obj_new(struct rte_eth_dev *dev, uint16_t idx, txq_data->cq_db = cq_info.dbrec; txq_data->cqes = (volatile struct mlx5_cqe *)cq_info.buf; txq_data->cq_ci = 0; -#ifndef NDEBUG txq_data->cq_pi = 0; -#endif txq_data->wqe_ci = 0; txq_data->wqe_pi = 0; txq_data->wqe_comp = 0; -- 2.20.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2020-02-11 11:17:42.899051638 +0000 +++ 0119-net-mlx5-engage-free-on-completion-queue.patch 2020-02-11 11:17:38.596005012 +0000 @@ -1,8 +1,10 @@ -From 89ef76db3b7627d4a7e656c7e51807b5bf8bdedc Mon Sep 17 00:00:00 2001 +From b79505fd8360c91c2fc31a423c7c4b2f0e7b85ed Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Thu, 9 Jan 2020 17:16:07 +0000 Subject: [PATCH] net/mlx5: engage free on completion queue +[ upstream commit 89ef76db3b7627d4a7e656c7e51807b5bf8bdedc ] + The free on completion queue keeps the indices of elts array, all mbuf stored below this index should be freed on arrival of normal send completion. In debug version it also contains @@ -18,7 +20,7 @@ 3 files changed, 18 insertions(+), 21 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c -index e879e5f186..5e31f01eda 100644 +index f8de2a035b..67cafd1e2c 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -2043,8 +2043,7 @@ mlx5_tx_comp_flush(struct mlx5_txq_data *restrict txq, @@ -97,7 +99,7 @@ } diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h -index 7d99282797..24fa038dc4 100644 +index 7d1b2fa10c..e362b4afe0 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -273,9 +273,7 @@ struct mlx5_txq_data {