From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 265A4A0555; Wed, 19 Feb 2020 15:54:54 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4C4E01BF78; Wed, 19 Feb 2020 15:54:53 +0100 (CET) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70045.outbound.protection.outlook.com [40.107.7.45]) by dpdk.org (Postfix) with ESMTP id E6F011B13C; Wed, 19 Feb 2020 15:54:51 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lXuvXmAVq/ezE+rAaE8o3AomnsZ05bSKD3b5Wjya2YJ7BEGwgoQbUirjHEsGS8UiQn7TFz3z2WWDIfycFgVKW/Nc+dzmeA+DW5Y1qGx05GpVDoUIS8vPiN2YxQqUIVMjYyGW+K71EHDLg8knh6ssNz5rPcP7vugihoU8oqXGzeYhWae/Vpp0nOA2h0sOGKNt01d+pXKxwyW/jMMsOHYQ6Adsk65EKN8sz06G4iHwCo8+OleAQE/hiOcHSkoCsyPhB0rpSWTi7Qbxnn/s3WVSgqQBMvOU1lImFILqdeSmAmhR+c8gKmAoNZAo6/ZP9V8nfqr969Zo4nbPQjSKR4JR6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9jPQ7xrQBddgoXmq+3tPBdNInyiHOj67xHRD3Zgp1AQ=; b=AlMqa64C9sUDk6Nwx9UOnHo6zAh0Oout2s9Xa0wkfzUk4ryHrkh+WBxH9bK4woXTdeBBvQnD16S7H+DEzkHKUX2hMVZDdpcEa2FyeyadhA3b+zJ8q1Hkz3LCzuy2NU74bY8Bg3aZ4r7+aEdZwSmpiyMQOvRz/oYk5cFuq1TBeReht0lQWSFTgoArA9tEkJgN8VF1EiCEAOLL3vdZVB6zluWKJ4yhz/BLsIdrg2MoGY4kYWks4g1e/WiJAqjSGtOSOXZVtHTVX2lRunyZD2hybvUq+C6kEMQSNbG8T8frUAy0i1YgevPpl0b/lpTTKq6QqZjG1bsdMVNhBZ2IKdTLww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9jPQ7xrQBddgoXmq+3tPBdNInyiHOj67xHRD3Zgp1AQ=; b=t84R4Hz14mYgZXNKeK9VJz2Qnhqa/rcjiSvK1qJOHWS42sdv2ImSl5ND42kEWt+Qp6TlW5Bgc0l7PK3VG4vURhQrEBMqNGbe19az3cwaayXBZJirsUNXoW5ZULEE+E+TycDVPo/85O5WDC3m9HqfYhzNJgSmJ+I/A2DJMKIeido= Received: from AM0PR05MB6707.eurprd05.prod.outlook.com (10.186.174.145) by AM0PR05MB6259.eurprd05.prod.outlook.com (20.177.40.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2729.24; Wed, 19 Feb 2020 14:54:50 +0000 Received: from AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9]) by AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9%7]) with mapi id 15.20.2750.016; Wed, 19 Feb 2020 14:54:50 +0000 From: Raslan Darawsheh To: Bing Zhao , Ori Kam , Slava Ovsiienko CC: Matan Azrad , "dev@dpdk.org" , "stable@dpdk.org" Thread-Topic: [PATCH v2] net/mlx5: fix the hairpin queue capacity Thread-Index: AQHV5v6gVFI4YIsRik68gWevce4OFKgimvpw Date: Wed, 19 Feb 2020 14:54:50 +0000 Message-ID: References: <1581946301-457865-1-git-send-email-bingz@mellanox.com> <1582100919-334723-1-git-send-email-bingz@mellanox.com> In-Reply-To: <1582100919-334723-1-git-send-email-bingz@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-originating-ip: [212.29.221.74] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 32a49faa-ec3e-4050-9abb-08d7b54bac00 x-ms-traffictypediagnostic: AM0PR05MB6259:|AM0PR05MB6259: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:400; x-forefront-prvs: 0318501FAE x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(136003)(396003)(366004)(346002)(376002)(199004)(189003)(52536014)(86362001)(53546011)(4326008)(478600001)(8936002)(6506007)(2906002)(33656002)(6636002)(71200400001)(55016002)(316002)(450100002)(186003)(26005)(110136005)(54906003)(76116006)(9686003)(66446008)(66556008)(64756008)(66946007)(81166006)(81156014)(7696005)(8676002)(66476007)(5660300002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR05MB6259; H:AM0PR05MB6707.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: WEv3dRiB18bWtwcsPKk1d6dfe0VUICvl0rr2peBg2yJCGIQApnnwbK+/cYdtEc/tvJUH4fSJVx4+jho5QMrRKyynGbhePkqvKLYX4Rpev6g7gb6shXZ/A7TFOECLQ2eFZC8uH3yx4h+aV/ip3GZgHxv0ZzwyLBZQo9ftwqNhWy8rnJtIQ4/oa3u/DwGewm1z+52sVDEvKAl0+RvTEx4+pwXPA/gAOWf5t7VtnECvhCN1ezmZuovNxi8zPd1WPZPhNgKPZjQI4MTD8PwQe1S2oE1OgHACO4j3Y0ZF3yFsg6RfkaQ3wsaVcBFeg+LLGgmu5McOqbuQykU1BLtH2A0eHT5b0Xs8EDq1H26jV+gCfoD+7wTV38ZXZddoz5bISDPVznHfMlCQeIjEjVlC+OTG4tF1q+aIcZYi8q4yd1ptBVZOj5KqFTkBZ2+AMrrweyLW x-ms-exchange-antispam-messagedata: WblJeyeCXkG8Ab2NaRmWNLyXrqVz07zB1RMBeI5LmqiU6ElYrB3CC3mOnl7PcqpWONNoCRCn8tjO7EEIfgM1Vc3MbblxhxlC/wGS8e2Wni5BE9+58M2bmOQBumMBKggzWDjVqb4EkeylAS/qxFHPzQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 32a49faa-ec3e-4050-9abb-08d7b54bac00 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Feb 2020 14:54:50.8642 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: MOgbo3+OVNR6FKFs5LDJvsTGAIvhbUOAm+GMYRIroQIZUE9/REpIhOFIs4hGEeTQknGMSn+O3FZXq6tmFTPcXQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB6259 Subject: Re: [dpdk-dev] [PATCH v2] net/mlx5: fix the hairpin queue capacity X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, > -----Original Message----- > From: Bing Zhao > Sent: Wednesday, February 19, 2020 10:29 AM > To: Ori Kam ; Slava Ovsiienko > > Cc: Raslan Darawsheh ; Matan Azrad > ; dev@dpdk.org; stable@dpdk.org > Subject: [PATCH v2] net/mlx5: fix the hairpin queue capacity >=20 > The hairpin TX/RX queue depth and packet size is fixed in the past. > When the firmware has some fix or improvement, the PMD will not > make full use of it. And also, 32 packets for a single queue will not > guarantee a good performance for hairpin flows. It will make the > stride size larger and for small packets, it is a waste of memory. > The recommended stride size is 64B now. >=20 > The parameter of hairpin queue setup needs to be adjusted. > 1. A proper buffer size should support the standard jumbo frame with > 9KB, and also more than 1 jumbo frame packet for performance. > 2. Number of packets of a single queue should be the maximum > supported value (total buffer size / stride size). >=20 > There is no need to support the max capacity of total buffer size > because the memory consumption should also be taken into > consideration. >=20 > Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") > Cc: orika@mellanox.com > Cc: stable@dpdk.org >=20 > Signed-off-by: Bing Zhao >=20 > ------------ >=20 > v2: change the capacity parameters and the commit details >=20 > --- > drivers/net/mlx5/mlx5_defs.h | 4 ++++ > drivers/net/mlx5/mlx5_rxq.c | 13 +++++++++---- > drivers/net/mlx5/mlx5_txq.c | 13 +++++++++---- > 3 files changed, 22 insertions(+), 8 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h > index 9b392ed..83ca367 100644 > --- a/drivers/net/mlx5/mlx5_defs.h > +++ b/drivers/net/mlx5/mlx5_defs.h > @@ -173,6 +173,10 @@ > #define MLX5_FLOW_MREG_HNAME "MARK_COPY_TABLE" > #define MLX5_DEFAULT_COPY_ID UINT32_MAX >=20 > +/* Hairpin TX/RX queue configuration parameters. */ > +#define MLX5_HAIRPIN_QUEUE_STRIDE 6 > +#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (15 + 2) > + > /* Definition of static_assert found in /usr/include/assert.h */ > #ifndef HAVE_STATIC_ASSERT > #define static_assert _Static_assert > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c > index dc0fd82..8a6b410 100644 > --- a/drivers/net/mlx5/mlx5_rxq.c > +++ b/drivers/net/mlx5/mlx5_rxq.c > @@ -1268,6 +1268,7 @@ > struct mlx5_devx_create_rq_attr attr =3D { 0 }; > struct mlx5_rxq_obj *tmpl =3D NULL; > int ret =3D 0; > + uint32_t max_wq_data; >=20 > MLX5_ASSERT(rxq_data); > MLX5_ASSERT(!rxq_ctrl->obj); > @@ -1283,11 +1284,15 @@ > tmpl->type =3D MLX5_RXQ_OBJ_TYPE_DEVX_HAIRPIN; > tmpl->rxq_ctrl =3D rxq_ctrl; > attr.hairpin =3D 1; > - /* Workaround for hairpin startup */ > - attr.wq_attr.log_hairpin_num_packets =3D log2above(32); > - /* Workaround for packets larger than 1KB */ > + max_wq_data =3D priv->config.hca_attr.log_max_hairpin_wq_data_sz; > + /* Jumbo frames > 9KB should be supported, and more packets. */ > attr.wq_attr.log_hairpin_data_sz =3D > - priv->config.hca_attr.log_max_hairpin_wq_data_sz; > + (max_wq_data < > MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? > + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; > + /* Set the packets number to the maximum value for performance. > */ > + attr.wq_attr.log_hairpin_num_packets =3D > + attr.wq_attr.log_hairpin_data_sz - > + MLX5_HAIRPIN_QUEUE_STRIDE; > tmpl->rq =3D mlx5_devx_cmd_create_rq(priv->sh->ctx, &attr, > rxq_ctrl->socket); > if (!tmpl->rq) { > diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c > index bc13abf..2ad849a 100644 > --- a/drivers/net/mlx5/mlx5_txq.c > +++ b/drivers/net/mlx5/mlx5_txq.c > @@ -493,6 +493,7 @@ > struct mlx5_devx_create_sq_attr attr =3D { 0 }; > struct mlx5_txq_obj *tmpl =3D NULL; > int ret =3D 0; > + uint32_t max_wq_data; >=20 > MLX5_ASSERT(txq_data); > MLX5_ASSERT(!txq_ctrl->obj); > @@ -509,11 +510,15 @@ > tmpl->txq_ctrl =3D txq_ctrl; > attr.hairpin =3D 1; > attr.tis_lst_sz =3D 1; > - /* Workaround for hairpin startup */ > - attr.wq_attr.log_hairpin_num_packets =3D log2above(32); > - /* Workaround for packets larger than 1KB */ > + max_wq_data =3D priv->config.hca_attr.log_max_hairpin_wq_data_sz; > + /* Jumbo frames > 9KB should be supported, and more packets. */ > attr.wq_attr.log_hairpin_data_sz =3D > - priv->config.hca_attr.log_max_hairpin_wq_data_sz; > + (max_wq_data < > MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? > + max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; > + /* Set the packets number to the maximum value for performance. > */ > + attr.wq_attr.log_hairpin_num_packets =3D > + attr.wq_attr.log_hairpin_data_sz - > + MLX5_HAIRPIN_QUEUE_STRIDE; > attr.tis_num =3D priv->sh->tis->id; > tmpl->sq =3D mlx5_devx_cmd_create_sq(priv->sh->ctx, &attr); > if (!tmpl->sq) { > -- > 1.8.3.1 Patch applied to next-net-mlx, Kindest regards, Raslan Darawsheh