From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [148.163.129.52]) by dpdk.org (Postfix) with ESMTP id 465FB1B4D2 for ; Fri, 5 Apr 2019 14:05:18 +0200 (CEST) X-Virus-Scanned: Proofpoint Essentials engine Received: from webmail.solarflare.com (webmail.solarflare.com [12.187.104.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us2.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id DDF551C008A for ; Fri, 5 Apr 2019 12:05:15 +0000 (UTC) Received: from ocex03.SolarFlarecom.com (10.20.40.36) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Fri, 5 Apr 2019 05:05:13 -0700 Received: from opal.uk.solarflarecom.com (10.17.10.1) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Frontend Transport; Fri, 5 Apr 2019 05:05:13 -0700 Received: from ukv-loginhost.uk.solarflarecom.com (ukv-loginhost.uk.solarflarecom.com [10.17.10.39]) by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id x35C5BDS007504 for ; Fri, 5 Apr 2019 13:05:11 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (localhost [127.0.0.1]) by ukv-loginhost.uk.solarflarecom.com (Postfix) with ESMTP id C76E4161610 for ; Fri, 5 Apr 2019 13:05:11 +0100 (BST) From: Andrew Rybchenko To: Date: Fri, 5 Apr 2019 13:05:07 +0100 Message-ID: <1554465907-11491-1-git-send-email-arybchenko@solarflare.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 Content-Type: text/plain X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.5.1010-24532.005 X-TM-AS-Result: Yes-8.887700-4.000000-11 X-TMASE-MatchedRID: n7nN31NWDDoxqDj7pnlMuLMsPmSZxbpk8GRhP/nTHNaiLeTug3wOViH3 fZSwSQ1sl0ZAjwZQuCyhlYplAxUaFLNEjK1dQ4HMFyqkfsPWu1BCX8V1FiRRkufL+zKjW08PpiZ /efbp/Op9/X+mtZWroUGKHXV7DzL/jHWDWVOJ7F2MVQb49Y23I1AI6wCVrE3vtGoysJhmyv0TBr T5FMhW3d+0sW3E5JtDgDLqnrRlXrZ8nn9tnqel2LI7zVffJqTz2CqsDrefNDpN/t1AaVCY0TUfg PDNOg9xT35uOC36ISnTCTUNJslO9Ln0S5o3tRIxSk6nmHJ1X4F5//3OpX/NSMGcfAE9XKk7 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 11-8.887700-4.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.5.1010-24532.005 X-MDID: 1554465917-57OVfAKgNvk2 Subject: [dpdk-dev] [PATCH] net/sfc: improve Rx free threshold default X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 12:05:18 -0000 Rx refill in one bulk (which is just 8 descriptors) by default is too aggressive and makes too many MMIO writes (Rx doorbells) if packet rate is high. Setting default to 1/8 of Rx descriptors number shows good performance results. Anyway it is a default value which may be overridden by Rx configuration provided by application. Signed-off-by: Andrew Rybchenko --- drivers/net/sfc/sfc_rx.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c index 4b1d01e66..5d8c2765c 100644 --- a/drivers/net/sfc/sfc_rx.c +++ b/drivers/net/sfc/sfc_rx.c @@ -974,6 +974,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index, struct sfc_rxq *rxq; struct sfc_dp_rx_qcreate_info info; struct sfc_dp_rx_hw_limits hw_limits; + uint16_t rx_free_thresh; memset(&hw_limits, 0, sizeof(hw_limits)); hw_limits.rxq_max_entries = sa->rxq_max_entries; @@ -1043,8 +1044,22 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index, rxq = &sa->rxq_ctrl[sw_index]; rxq->evq = evq; rxq->hw_index = sw_index; + /* + * If Rx refill threshold is specified (its value is non zero) in + * Rx configuration, use specified value. Otherwise use 1/8 of + * the Rx descriptors number as the default. It allows to keep + * Rx ring full-enough and does not refill too aggressive if + * packet rate is high. + * + * Since PMD refills in bulks waiting for full bulk may be + * refilled (basically round down), it is better to round up + * here to mitigate it a bit. + */ + rx_free_thresh = (rx_conf->rx_free_thresh != 0) ? + rx_conf->rx_free_thresh : EFX_DIV_ROUND_UP(nb_rx_desc, 8); + /* Rx refill threshold cannot be smaller than refill bulk */ rxq_info->refill_threshold = - RTE_MAX(rx_conf->rx_free_thresh, SFC_RX_REFILL_BULK); + RTE_MAX(rx_free_thresh, SFC_RX_REFILL_BULK); rxq_info->refill_mb_pool = mb_pool; rxq->buf_size = buf_size; -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id CA4FDA0679 for ; Fri, 5 Apr 2019 14:05:20 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6BDCC1B4D8; Fri, 5 Apr 2019 14:05:19 +0200 (CEST) Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [148.163.129.52]) by dpdk.org (Postfix) with ESMTP id 465FB1B4D2 for ; Fri, 5 Apr 2019 14:05:18 +0200 (CEST) X-Virus-Scanned: Proofpoint Essentials engine Received: from webmail.solarflare.com (webmail.solarflare.com [12.187.104.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mx1-us2.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id DDF551C008A for ; Fri, 5 Apr 2019 12:05:15 +0000 (UTC) Received: from ocex03.SolarFlarecom.com (10.20.40.36) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Fri, 5 Apr 2019 05:05:13 -0700 Received: from opal.uk.solarflarecom.com (10.17.10.1) by ocex03.SolarFlarecom.com (10.20.40.36) with Microsoft SMTP Server (TLS) id 15.0.1395.4 via Frontend Transport; Fri, 5 Apr 2019 05:05:13 -0700 Received: from ukv-loginhost.uk.solarflarecom.com (ukv-loginhost.uk.solarflarecom.com [10.17.10.39]) by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id x35C5BDS007504 for ; Fri, 5 Apr 2019 13:05:11 +0100 Received: from ukv-loginhost.uk.solarflarecom.com (localhost [127.0.0.1]) by ukv-loginhost.uk.solarflarecom.com (Postfix) with ESMTP id C76E4161610 for ; Fri, 5 Apr 2019 13:05:11 +0100 (BST) From: Andrew Rybchenko To: Date: Fri, 5 Apr 2019 13:05:07 +0100 Message-ID: <1554465907-11491-1-git-send-email-arybchenko@solarflare.com> X-Mailer: git-send-email 1.8.3.1 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.5.1010-24532.005 X-TM-AS-Result: Yes-8.887700-4.000000-11 X-TMASE-MatchedRID: n7nN31NWDDoxqDj7pnlMuLMsPmSZxbpk8GRhP/nTHNaiLeTug3wOViH3 fZSwSQ1sl0ZAjwZQuCyhlYplAxUaFLNEjK1dQ4HMFyqkfsPWu1BCX8V1FiRRkufL+zKjW08PpiZ /efbp/Op9/X+mtZWroUGKHXV7DzL/jHWDWVOJ7F2MVQb49Y23I1AI6wCVrE3vtGoysJhmyv0TBr T5FMhW3d+0sW3E5JtDgDLqnrRlXrZ8nn9tnqel2LI7zVffJqTz2CqsDrefNDpN/t1AaVCY0TUfg PDNOg9xT35uOC36ISnTCTUNJslO9Ln0S5o3tRIxSk6nmHJ1X4F5//3OpX/NSMGcfAE9XKk7 X-TM-AS-User-Approved-Sender: No X-TM-AS-User-Blocked-Sender: No X-TMASE-Result: 11-8.887700-4.000000 X-TMASE-Version: SMEX-12.5.0.1300-8.5.1010-24532.005 X-MDID: 1554465917-57OVfAKgNvk2 Subject: [dpdk-dev] [PATCH] net/sfc: improve Rx free threshold default X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190405120507.U9hPAv_107sI0_gDhg_mI28I_Qs1OWIAlU2szUPM908@z> Rx refill in one bulk (which is just 8 descriptors) by default is too aggressive and makes too many MMIO writes (Rx doorbells) if packet rate is high. Setting default to 1/8 of Rx descriptors number shows good performance results. Anyway it is a default value which may be overridden by Rx configuration provided by application. Signed-off-by: Andrew Rybchenko --- drivers/net/sfc/sfc_rx.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/net/sfc/sfc_rx.c b/drivers/net/sfc/sfc_rx.c index 4b1d01e66..5d8c2765c 100644 --- a/drivers/net/sfc/sfc_rx.c +++ b/drivers/net/sfc/sfc_rx.c @@ -974,6 +974,7 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index, struct sfc_rxq *rxq; struct sfc_dp_rx_qcreate_info info; struct sfc_dp_rx_hw_limits hw_limits; + uint16_t rx_free_thresh; memset(&hw_limits, 0, sizeof(hw_limits)); hw_limits.rxq_max_entries = sa->rxq_max_entries; @@ -1043,8 +1044,22 @@ sfc_rx_qinit(struct sfc_adapter *sa, unsigned int sw_index, rxq = &sa->rxq_ctrl[sw_index]; rxq->evq = evq; rxq->hw_index = sw_index; + /* + * If Rx refill threshold is specified (its value is non zero) in + * Rx configuration, use specified value. Otherwise use 1/8 of + * the Rx descriptors number as the default. It allows to keep + * Rx ring full-enough and does not refill too aggressive if + * packet rate is high. + * + * Since PMD refills in bulks waiting for full bulk may be + * refilled (basically round down), it is better to round up + * here to mitigate it a bit. + */ + rx_free_thresh = (rx_conf->rx_free_thresh != 0) ? + rx_conf->rx_free_thresh : EFX_DIV_ROUND_UP(nb_rx_desc, 8); + /* Rx refill threshold cannot be smaller than refill bulk */ rxq_info->refill_threshold = - RTE_MAX(rx_conf->rx_free_thresh, SFC_RX_REFILL_BULK); + RTE_MAX(rx_free_thresh, SFC_RX_REFILL_BULK); rxq_info->refill_mb_pool = mb_pool; rxq->buf_size = buf_size; -- 2.17.1