From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 14CAFA0555;
Wed, 19 Feb 2020 13:32:14 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id ECB781BF78;
Wed, 19 Feb 2020 13:32:12 +0100 (CET)
Received: from EUR05-AM6-obe.outbound.protection.outlook.com
(mail-am6eur05on2052.outbound.protection.outlook.com [40.107.22.52])
by dpdk.org (Postfix) with ESMTP id 341B91B951;
Wed, 19 Feb 2020 13:32:11 +0100 (CET)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=B1D8uAkJBCAxqXiaDEdu4iuj6srq1AQMGTGWYDaUxEV0arIyZHTzae1kPnU4zNuUEptgTYsdeuP0rZUKLEkgPpBstEfWlRG8YJSo3Jk5c3hupBscOaOvXgY69c8wluZC2fRClF3DnGbMGriTOgrP5FQ2R5D7E0A9ye3dEwlP+OEDWN2vOtFB4puvnAJrNTSvdQ+zCbO79e3L4rhedA0eSXEAB95jp+Hgt3oo0B8mXvgSvEp6p1nFz/Ln5Qr3MZSUSUp6QgqWODmM9pJ+yXDU9LRioVK8Z4NI6b64El7+Fn2Ov+sSHC3W18f43EQ4wXwyHMc4WDpVewz0uV+sYZ9zBA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=rlK9JQWJsH9qnxAicnkyLy1kMPqz0yETM73UDgxWp/g=;
b=Y2AraAdUODQXav3RbJo+pSRDn1SErpJeJ/NUm7moQCVU3+TCSyrOsP/vhwJv/jEIKXzPTVBqsK9+sIplKt5ZPb/Hq1GBKRYMBjnpGYA5uFLSRO7S0HW+gGcUQoTolprYCKP7tLugVpsgfHBKfUrOacsJqaJ7tp7MBBUWRyCau5vAu4bg4qp5xLywuYXvD8+GN+CxBjdDEujnQ2GC2WdbJ6yISYt1MYaoLnRWcNIU2FZK0YivUbQ7odM9pVllWez5OkfNefGPlV1Kn+MdpRzuCVd30tqVvsnKmIo7vVseXETbNqWecEcQCDEuOGaHjIaZqhEEVLG/+O7LL/xbMNze3w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com;
dkim=pass header.d=mellanox.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com;
s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=rlK9JQWJsH9qnxAicnkyLy1kMPqz0yETM73UDgxWp/g=;
b=fSuHGYpeWEjtFrW1PHSCbx7WQCMQRFHkPqpZLHbNQcL+Snb9/fi5mBKS7mU8HAjTKv2p+wvxigZJb2xXSR2f3jzdbOoOvUtzpB7P9nnXO/Dod1uHQDJBJ5ZxjYRxQfniMDdGrrx4sp9U+8bLp4zqrz9AeWvgOH1Se8CcEgA4Z3s=
Received: from AM6PR05MB5176.eurprd05.prod.outlook.com (20.177.196.158) by
AM6PR05MB4358.eurprd05.prod.outlook.com (52.135.160.161) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.2729.24; Wed, 19 Feb 2020 12:32:08 +0000
Received: from AM6PR05MB5176.eurprd05.prod.outlook.com
([fe80::1888:dbf5:a84e:c53b]) by AM6PR05MB5176.eurprd05.prod.outlook.com
([fe80::1888:dbf5:a84e:c53b%6]) with mapi id 15.20.2729.033; Wed, 19 Feb 2020
12:32:08 +0000
From: Ori Kam <orika@mellanox.com>
To: Bing Zhao <bingz@mellanox.com>, Slava Ovsiienko <viacheslavo@mellanox.com>
CC: Raslan Darawsheh <rasland@mellanox.com>, Matan Azrad <matan@mellanox.com>,
"dev@dpdk.org" <dev@dpdk.org>, "stable@dpdk.org" <stable@dpdk.org>
Thread-Topic: [PATCH v2] net/mlx5: fix the hairpin queue capacity
Thread-Index: AQHV5v6gAQYvWh3aTE27+5Rzt86+nKgicx4Q
Date: Wed, 19 Feb 2020 12:32:07 +0000
Message-ID: <AM6PR05MB5176E159C6C977BF0204C8F4DB100@AM6PR05MB5176.eurprd05.prod.outlook.com>
References: <1581946301-457865-1-git-send-email-bingz@mellanox.com>
<1582100919-334723-1-git-send-email-bingz@mellanox.com>
In-Reply-To: <1582100919-334723-1-git-send-email-bingz@mellanox.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is )
smtp.mailfrom=orika@mellanox.com;
x-originating-ip: [185.175.35.255]
x-ms-publictraffictype: Email
x-ms-office365-filtering-ht: Tenant
x-ms-office365-filtering-correlation-id: c040f900-ef0d-46c1-e243-08d7b537bc21
x-ms-traffictypediagnostic: AM6PR05MB4358:|AM6PR05MB4358:
x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtFwd
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <AM6PR05MB4358B59A6C3AFC264969946FDB100@AM6PR05MB4358.eurprd05.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:494;
x-forefront-prvs: 0318501FAE
x-forefront-antispam-report: SFV:NSPM;
SFS:(10009020)(4636009)(376002)(346002)(39860400002)(136003)(396003)(366004)(199004)(189003)(7696005)(6636002)(6506007)(86362001)(2906002)(53546011)(26005)(54906003)(478600001)(110136005)(71200400001)(64756008)(316002)(8676002)(8936002)(81156014)(9686003)(55016002)(5660300002)(52536014)(33656002)(186003)(81166006)(450100002)(4326008)(66946007)(66556008)(76116006)(66476007)(66446008);
DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR05MB4358;
H:AM6PR05MB5176.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en;
PTR:InfoNoRecords; A:1; MX:1;
received-spf: None (protection.outlook.com: mellanox.com does not designate
permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: yZx5b9DcAGAimIGkgjRcMxs4dVDrDHa6U2HP7wx5JtPbTkRudJ97cdjoqXztGBeREhZYdFeOlm6b122ElJksUvlaYYFUqw8WEsje5sIb09MoEk93oHBh82vrfr/nXLA4GjKbXy5CwyCgwigCyXXxvQ3xwGojNmdkk4+UDekiP2JLzIrHurqlg/fKhFDTV7ToonaRNLuLdK20L2Up0sBq0vlS7cxDr5D2QbS+Bv+rNeBe/S73j6krKtJdxJskCnbih2pjQzK1P8V7d09ZNcS9eBzKSSY9GaMAxwJcTk+nsEgQ08EZCo2KUm2WbHXUln5om3NdJkkMUIR4B6yNKMkC1aY8KqgBHNTilUckdzPEdIEMBhKlXCji/FE22wYlnj5BUIHyFMI7Z3uL5UQL+QhQzKLr/rV1Igp32/6FqXNiztErNbzceU1KydES7ZHqpaiD
x-ms-exchange-antispam-messagedata: jq7Sqt5o38+7NvRyAALksrhqN6A88Uq5ntl89YFjAmONM1GTnKrXMkYRQjmd+IY3K5IWsfT4wrbpcdywbZLRMToJnXymE8l58hS+bZ37VbS5rZBMaA2ng25PbYpxBW39+EEXwnudZp/dK1fuYCSgrQ==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c040f900-ef0d-46c1-e243-08d7b537bc21
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Feb 2020 12:32:07.8721 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 97wnM+FnLgVCZDGu2sONKzoM6NBoCtAerYr4BAU81rQvuJd3TvQGFOPIzAFmT6OfzoRuLeqNd+Y1C5Adkkcj9w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR05MB4358
Subject: Re: [dpdk-dev] [PATCH v2] net/mlx5: fix the hairpin queue capacity
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
> -----Original Message-----
> From: Bing Zhao <bingz@mellanox.com>
> Sent: Wednesday, February 19, 2020 10:29 AM
> To: Ori Kam <orika@mellanox.com>; Slava Ovsiienko
> <viacheslavo@mellanox.com>
> Cc: Raslan Darawsheh <rasland@mellanox.com>; Matan Azrad
> <matan@mellanox.com>; dev@dpdk.org; stable@dpdk.org
> Subject: [PATCH v2] net/mlx5: fix the hairpin queue capacity
>=20
> The hairpin TX/RX queue depth and packet size is fixed in the past.
> When the firmware has some fix or improvement, the PMD will not
> make full use of it. And also, 32 packets for a single queue will not
> guarantee a good performance for hairpin flows. It will make the
> stride size larger and for small packets, it is a waste of memory.
> The recommended stride size is 64B now.
>=20
> The parameter of hairpin queue setup needs to be adjusted.
> 1. A proper buffer size should support the standard jumbo frame with
> 9KB, and also more than 1 jumbo frame packet for performance.
> 2. Number of packets of a single queue should be the maximum
> supported value (total buffer size / stride size).
>=20
> There is no need to support the max capacity of total buffer size
> because the memory consumption should also be taken into
> consideration.
>=20
> Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues")
> Cc: orika@mellanox.com
> Cc: stable@dpdk.org
>=20
> Signed-off-by: Bing Zhao <bingz@mellanox.com>
>=20
> ------------
>=20
Acked-by: Ori Kam <orika@mellanox.com>
Thanks,
Ori
> v2: change the capacity parameters and the commit details
>=20
> ---
> drivers/net/mlx5/mlx5_defs.h | 4 ++++
> drivers/net/mlx5/mlx5_rxq.c | 13 +++++++++----
> drivers/net/mlx5/mlx5_txq.c | 13 +++++++++----
> 3 files changed, 22 insertions(+), 8 deletions(-)
>=20
> diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h
> index 9b392ed..83ca367 100644
> --- a/drivers/net/mlx5/mlx5_defs.h
> +++ b/drivers/net/mlx5/mlx5_defs.h
> @@ -173,6 +173,10 @@
> #define MLX5_FLOW_MREG_HNAME "MARK_COPY_TABLE"
> #define MLX5_DEFAULT_COPY_ID UINT32_MAX
>=20
> +/* Hairpin TX/RX queue configuration parameters. */
> +#define MLX5_HAIRPIN_QUEUE_STRIDE 6
> +#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (15 + 2)
> +
> /* Definition of static_assert found in /usr/include/assert.h */
> #ifndef HAVE_STATIC_ASSERT
> #define static_assert _Static_assert
> diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
> index dc0fd82..8a6b410 100644
> --- a/drivers/net/mlx5/mlx5_rxq.c
> +++ b/drivers/net/mlx5/mlx5_rxq.c
> @@ -1268,6 +1268,7 @@
> struct mlx5_devx_create_rq_attr attr =3D { 0 };
> struct mlx5_rxq_obj *tmpl =3D NULL;
> int ret =3D 0;
> + uint32_t max_wq_data;
>=20
> MLX5_ASSERT(rxq_data);
> MLX5_ASSERT(!rxq_ctrl->obj);
> @@ -1283,11 +1284,15 @@
> tmpl->type =3D MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN;
> tmpl->rxq_ctrl =3D rxq_ctrl;
> attr.hairpin =3D 1;
> - /* Workaround for hairpin startup */
> - attr.wq_attr.log_hairpin_num_packets =3D log2above(32);
> - /* Workaround for packets larger than 1KB */
> + max_wq_data =3D priv->config.hca_attr.log_max_hairpin_wq_data_sz;
> + /* Jumbo frames > 9KB should be supported, and more packets. */
> attr.wq_attr.log_hairpin_data_sz =3D
> - priv->config.hca_attr.log_max_hairpin_wq_data_sz;
> + (max_wq_data < MLX5_HAIRPIN_JUMBO_LOG_SIZE) ?
> + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE;
> + /* Set the packets number to the maximum value for performance. */
> + attr.wq_attr.log_hairpin_num_packets =3D
> + attr.wq_attr.log_hairpin_data_sz -
> + MLX5_HAIRPIN_QUEUE_STRIDE;
> tmpl->rq =3D mlx5_devx_cmd_create_rq(priv->sh->ctx, &attr,
> rxq_ctrl->socket);
> if (!tmpl->rq) {
> diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c
> index bc13abf..2ad849a 100644
> --- a/drivers/net/mlx5/mlx5_txq.c
> +++ b/drivers/net/mlx5/mlx5_txq.c
> @@ -493,6 +493,7 @@
> struct mlx5_devx_create_sq_attr attr =3D { 0 };
> struct mlx5_txq_obj *tmpl =3D NULL;
> int ret =3D 0;
> + uint32_t max_wq_data;
>=20
> MLX5_ASSERT(txq_data);
> MLX5_ASSERT(!txq_ctrl->obj);
> @@ -509,11 +510,15 @@
> tmpl->txq_ctrl =3D txq_ctrl;
> attr.hairpin =3D 1;
> attr.tis_lst_sz =3D 1;
> - /* Workaround for hairpin startup */
> - attr.wq_attr.log_hairpin_num_packets =3D log2above(32);
> - /* Workaround for packets larger than 1KB */
> + max_wq_data =3D priv->config.hca_attr.log_max_hairpin_wq_data_sz;
> + /* Jumbo frames > 9KB should be supported, and more packets. */
> attr.wq_attr.log_hairpin_data_sz =3D
> - priv->config.hca_attr.log_max_hairpin_wq_data_sz;
> + (max_wq_data < MLX5_HAIRPIN_JUMBO_LOG_SIZE) ?
> + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE;
> + /* Set the packets number to the maximum value for performance. */
> + attr.wq_attr.log_hairpin_num_packets =3D
> + attr.wq_attr.log_hairpin_data_sz -
> + MLX5_HAIRPIN_QUEUE_STRIDE;
> attr.tis_num =3D priv->sh->tis->id;
> tmpl->sq =3D mlx5_devx_cmd_create_sq(priv->sh->ctx, &attr);
> if (!tmpl->sq) {
> --
> 1.8.3.1