From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 46C0FA0565; Tue, 17 Mar 2020 14:10:23 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2F613292D; Tue, 17 Mar 2020 14:10:22 +0100 (CET) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140054.outbound.protection.outlook.com [40.107.14.54]) by dpdk.org (Postfix) with ESMTP id 572DD25D9; Tue, 17 Mar 2020 14:10:20 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DytqgGaCmpIivdE0wcjWIaGHMBT5Tx1R4hwHbf156T3H4OHCYrtcERcgYN1ak245IwYkqqETd+/b5ZVA7TTQN5Ran6Q5g9rnySVfrUWKFT76KR2EuSJ11EVkaTNuJW13n+EaPt2CHgwJsIOSUsjpI0mnZox07aRbIgib9ZkaYYP5iGu+mnAWDtQVg2fuA7ULa0NngWYZQS2P2j9bHPLjMMlAeSJCPLrwIa3krrTgx58LuPBM2s4pwaKRZY7sS+jVyhSIQyTekb3NnaZe+pNFvo6B5iGy0mvQ2bjLB+I3P5ZGb0CffO4oaQaHYpmA3TOSdqHt/6Y3G0P9W8HKq7SPCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e1orwDwcvNpFssW9XOmkUnwCV/YV71GVr9E2btwbjUc=; b=QTJr0jrApa/iD8dkTryZNHS7NsS78MIiGEg0+83r2GL6i/ghoLhoih21gQHhuQ45v5Rfwlqq4LXsAUK8gV3mHTosIuRgM573+e2cQ0otKWvqZ5uoQWqyPfgXAwPcMXYgZus7sOwXYTP7NX+iyANysDn1zbTCFTC8DBvrZKUnmtbKe6mxAnVrZKGRfIiLnK86nYtLvF6PDdKjLY5EtzC4VmJzfEEDKFJcAUGN3t1fSwWwe4bBIkS/tYXe44qrMDG/e930iuJJbLGq4DP8EZG6RHEMqLjYzfFNKKEV3yuNKgCYp2n2CL2GrXIf28X6SOB1GWKZqWExZWaTrXc3S7AhXA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e1orwDwcvNpFssW9XOmkUnwCV/YV71GVr9E2btwbjUc=; b=j5JfezvZuebuGYMnSCdDIFrxR1F2PAknJj/Je/QVcRyc7pZVhNptVe9JJPvO0hgZHfCQ5NBUvmR7LFZF4+TaPb0f0m+U31nL+B2efIjnz4dTZPh9wv72Yl/bH92eQgq5GsjLprlUhcuhExInBzzK4BzR40wDKoJdAYf3mZIVFkQ= Received: from AM0PR05MB6707.eurprd05.prod.outlook.com (10.186.174.145) by AM0PR05MB5924.eurprd05.prod.outlook.com (20.178.202.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.16; Tue, 17 Mar 2020 13:10:18 +0000 Received: from AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9]) by AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9%7]) with mapi id 15.20.2814.021; Tue, 17 Mar 2020 13:10:18 +0000 From: Raslan Darawsheh To: Alexander Kozyrev , "dev@dpdk.org" CC: Matan Azrad , Slava Ovsiienko , "stable@dpdk.org" Thread-Topic: [PATCH] net/mlx5: reduce txq completion index memory loads Thread-Index: AQHV+6h/x7folgW2ykyIsCSfOJnGR6hMw2ag Date: Tue, 17 Mar 2020 13:10:18 +0000 Message-ID: References: <1584372899-26732-1-git-send-email-akozyrev@mellanox.com> In-Reply-To: <1584372899-26732-1-git-send-email-akozyrev@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-originating-ip: [84.242.49.134] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: d02a9186-44d7-40d5-9f11-08d7ca748a60 x-ms-traffictypediagnostic: AM0PR05MB5924:|AM0PR05MB5924: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:462; x-forefront-prvs: 0345CFD558 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(376002)(346002)(39860400002)(136003)(366004)(199004)(55016002)(9686003)(450100002)(4326008)(2906002)(478600001)(76116006)(64756008)(71200400001)(86362001)(66946007)(66476007)(8936002)(66446008)(66556008)(52536014)(33656002)(81156014)(81166006)(8676002)(53546011)(186003)(54906003)(5660300002)(110136005)(316002)(7696005)(26005)(6506007); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR05MB5924; H:AM0PR05MB6707.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qJ5GXgcuKlZDMHI7yRd/oei1qZDTCQfHyqcMVaTxxJEvSC+a+vbVJmwGZ1Z588BBrtw8kc/qbMygS81fxpu/DFwWLahfYIKPr8w2JpeSy5eJdhELk1cqfZWBPZOBebln3hjf3AmITEphT9axhe7gZqsQfLsu80lZaouk63tzbkOqvgAEDNuZwYRnUEkLa9OoAuVzodc56LJRMREU5VUX8gStf3I0zpEReNqAi0roHW3iZifQMjkXvI/iEfc+UWLBun1mVuip4D7Zd3qzGimyQCfIVekKJRJu3AGCXXMhLW8BNTzeoVcoMDU/aLJYG3riMNDblXuF14ZO9lxXtHdJzZFl9oifX9CCf0E3FLPF2JZ698clgF9DQgmXTtW7au762TD+IBSt5hm4miKZFVnt5mbWDYl6eQWhCIGmp117H24iUPFoNkI8WPxt4gEkgAOU x-ms-exchange-antispam-messagedata: r/N+dFQxgJXtVcSTG443mNwx3jdMdwowMfoUlftyGqFrYVnbSLe8NhPfcSf1IgHa7U31FVgjzm9uswpjRY0buR1JZeZZxvi2Sib7dGEaMf2Os4x43x94LEpVZobkVwnLlH9iLLpyR7PYbDGVnpP1iQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: d02a9186-44d7-40d5-9f11-08d7ca748a60 X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Mar 2020 13:10:18.2242 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: p9fgI2jW56pSxLhU3iaNFt6+T7h2SXpqM6L9qCCEhbIMfQI0xM9SLGgfHq/bk/orad/G+bL4kAXUso5XG1ShpA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB5924 Subject: Re: [dpdk-dev] [PATCH] net/mlx5: reduce txq completion index memory loads X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, > -----Original Message----- > From: Alexander Kozyrev > Sent: Monday, March 16, 2020 5:35 PM > To: dev@dpdk.org > Cc: Raslan Darawsheh ; Matan Azrad > ; Slava Ovsiienko ; > stable@dpdk.org > Subject: [PATCH] net/mlx5: reduce txq completion index memory loads >=20 > There is a non-optimal check if doorbel is needed present in the > mlx5_tx_handle_completion() function. Advancing a copy of the txq > consumer index and checking this copy with initial value causes > unnecessary memory loads and hurts the performance. It is better to > have a simple small boolean variable for this purpose. That allows > to eliminate all the excessive memory operations with the txq consumer > index and restore the performance of the tx completions. >=20 > Fixes: 1fd9af0 ("net/mlx5: update Tx error handling routine") > Cc: stable@dpdk.org >=20 > Signed-off-by: Alexander Kozyrev > Acked-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5_rxtx.c | 29 +++++++++++++---------------- > 1 file changed, 13 insertions(+), 16 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c > index 5ac63da..f3bf763 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.c > +++ b/drivers/net/mlx5/mlx5_rxtx.c > @@ -2160,7 +2160,7 @@ enum mlx5_txcmp_code { > { > unsigned int count =3D MLX5_TX_COMP_MAX_CQE; > volatile struct mlx5_cqe *last_cqe =3D NULL; > - uint16_t ci =3D txq->cq_ci; > + bool ring_doorbell =3D false; > int ret; >=20 > static_assert(MLX5_CQE_STATUS_HW_OWN < 0, "Must be negative > value"); > @@ -2168,8 +2168,8 @@ enum mlx5_txcmp_code { > do { > volatile struct mlx5_cqe *cqe; >=20 > - cqe =3D &txq->cqes[ci & txq->cqe_m]; > - ret =3D check_cqe(cqe, txq->cqe_s, ci); > + cqe =3D &txq->cqes[txq->cq_ci & txq->cqe_m]; > + ret =3D check_cqe(cqe, txq->cqe_s, txq->cq_ci); > if (unlikely(ret !=3D MLX5_CQE_STATUS_SW_OWN)) { > if (likely(ret !=3D MLX5_CQE_STATUS_ERR)) { > /* No new CQEs in completion queue. */ > @@ -2183,7 +2183,6 @@ enum mlx5_txcmp_code { > * here, before we might perform SQ reset. > */ > rte_wmb(); > - txq->cq_ci =3D ci; > ret =3D mlx5_tx_error_cqe_handle > (txq, (volatile struct mlx5_err_cqe *)cqe); > if (unlikely(ret < 0)) { > @@ -2199,16 +2198,18 @@ enum mlx5_txcmp_code { > * MLX5_CQE_SYNDROME_WR_FLUSH_ERR status. > * The send queue is supposed to be empty. > */ > - ++ci; > - txq->cq_pi =3D ci; > + ring_doorbell =3D true; > + ++txq->cq_ci; > + txq->cq_pi =3D txq->cq_ci; > last_cqe =3D NULL; > continue; > } > /* Normal transmit completion. */ > - MLX5_ASSERT(ci !=3D txq->cq_pi); > - MLX5_ASSERT((txq->fcqs[ci & txq->cqe_m] >> 16) =3D=3D > + MLX5_ASSERT(txq->cq_ci !=3D txq->cq_pi); > + MLX5_ASSERT((txq->fcqs[txq->cq_ci & txq->cqe_m] >> 16) > =3D=3D > cqe->wqe_counter); > - ++ci; > + ring_doorbell =3D true; > + ++txq->cq_ci; > last_cqe =3D cqe; > /* > * We have to restrict the amount of processed CQEs > @@ -2221,14 +2222,10 @@ enum mlx5_txcmp_code { > if (likely(--count =3D=3D 0)) > break; > } while (true); > - if (likely(ci !=3D txq->cq_ci)) { > - /* > - * Update completion queue consuming index > - * and ring doorbell to notify hardware. > - */ > + if (likely(ring_doorbell)) { > + /* Ring doorbell to notify hardware. */ > rte_compiler_barrier(); > - txq->cq_ci =3D ci; > - *txq->cq_db =3D rte_cpu_to_be_32(ci); > + *txq->cq_db =3D rte_cpu_to_be_32(txq->cq_ci); > mlx5_tx_comp_flush(txq, last_cqe, olx); > } > } > -- > 1.8.3.1 Patch applied to next-net-mlx, Kindest regards, Raslan Darawsheh