From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A98DEA0562; Tue, 31 Mar 2020 16:01:29 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4FFCC322C; Tue, 31 Mar 2020 16:01:29 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140070.outbound.protection.outlook.com [40.107.14.70]) by dpdk.org (Postfix) with ESMTP id 225BE2C6D for ; Tue, 31 Mar 2020 16:01:28 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d2ESX5CVRT2MXh15S1zNdTf8T2GYPqgpr50Epes9FvdXGynYGXs/BoBkjaxdVVXBLmroDkkfQU296HWaT3Piwvk/5znxFD4m2NH+itWp26eII9YGjpFVdmqxF6nmG6SI/2q6MbToy4kMCvwsa7envAqCKU/U5zAxb1NesiccHI+cfmzI8vwHSIfaK9hxwOXWbupvBkYX9ukyHJyOz/ezjgs6KVUIjE84aHD0JXDFT6+mpfeISkzjQStBDDka8aIOqZLx5JrouvL5Ky/+knwYPKN5We5lWiIWhqgt6VN9ZOQmIJ7IflXPsP9C/ftyDtvnlqFFZwb20vvI/YqgQcrSzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lvJPhHQdOmeUrjwl3iHGfsWIUNo9ccLKs/3px3D5X0I=; b=C17pyw3vB6dDsgXRn1k7LmPFXadNS6QFY8N1A3LDE3z3iTnnkKDwXFDwnopFFCyj587SQ2DZGnrk6bzmECNt8BgV3LILnXjGKdSnLnUazEQFpyiVI5/J/5y4aJYOqTcZT0JNizgfSn+Ia7LujggrljyHYwkzU/blzZp4Izw/G+8/seQHFJ5984Y6jaWqbP90jET+XiPkyhL6WgUt9thU7GZLWUKwlPbXirxuSeswa5mrzM1OVjEcdqZ4b6LKOXPbDNeUqPpCy+F+2ITmAB4KUvGdMaYXyWlpI391YLVVeCusV0MhCcYA0W2jLHlJZ5haz121MqzsZggbxsrvjhXbJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lvJPhHQdOmeUrjwl3iHGfsWIUNo9ccLKs/3px3D5X0I=; b=kmnwstoqJ58YpPLGlGdoSypx+b63acVqepdfcd4sWcp1x4dulaUmZgOcoNF4ScHhR2YgZug2NNIrHVG0XWHephW1y4RZ4V/OisVGTCWjRJ113c6cjpuTMQSvGQCTEPXmU9XiDPYEuGDnP7bilCMLQHkar6X3gPgbMr3MaksAkgk= Received: from AM0PR05MB6707.eurprd05.prod.outlook.com (10.186.174.145) by AM0PR05MB4546.eurprd05.prod.outlook.com (52.133.55.161) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.19; Tue, 31 Mar 2020 14:01:26 +0000 Received: from AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9]) by AM0PR05MB6707.eurprd05.prod.outlook.com ([fe80::adbd:45ca:54:bba9%7]) with mapi id 15.20.2856.019; Tue, 31 Mar 2020 14:01:26 +0000 From: Raslan Darawsheh To: Bing Zhao , Ori Kam , Matan Azrad CC: Slava Ovsiienko , "dev@dpdk.org" Thread-Topic: [PATCH] net/mlx5: introduce dev parameter for hairpin Thread-Index: AQHWAdwGWXWixX3EhEGaD/9S1drWCKhixeJQ Date: Tue, 31 Mar 2020 14:01:26 +0000 Message-ID: References: <1585054741-233903-1-git-send-email-bingz@mellanox.com> In-Reply-To: <1585054741-233903-1-git-send-email-bingz@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-originating-ip: [188.161.229.130] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: c2d60203-b68b-4da3-01df-08d7d57c00d2 x-ms-traffictypediagnostic: AM0PR05MB4546:|AM0PR05MB4546: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2089; x-forefront-prvs: 0359162B6D x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR05MB6707.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(376002)(39860400002)(396003)(136003)(366004)(346002)(71200400001)(6506007)(7696005)(64756008)(66446008)(66556008)(110136005)(86362001)(478600001)(66946007)(316002)(66476007)(76116006)(5660300002)(54906003)(52536014)(2906002)(33656002)(81166006)(8936002)(81156014)(53546011)(186003)(26005)(55016002)(8676002)(9686003)(6636002)(4326008); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: JaiKjfO7MbDa4Lrfc6f2Gc2nTGOeqgaGtd+6TqZndDI4VNPNpa+ad/mIZkuxNSTsrJ9+JVPhzDcypby5Y+7kk1cUJ5dgVDz9XYKjU764DEQMj9LtT2Y0p+gpLzXDeFiFIXqMstqcJAjHskPL16a9xt/hbXnz6nby9sEtwC3FalLRHME2EWexP7kOdqXrqY+m7reWspXi6Go52Z1SdmH/M7BvEwN+Z2ghi2uGKja5x63YPhJ+RSNsZhhqnkgHV7MIdm0yNzbXd5z14guDvxsorwwbJGqcAafgZiYxTFTDtfzFHiFyAyGSYmQKN9OAMgnYc/UO54y/2WffyaWNxzlXnZGiSOv8rnLT/GvJ0fw6yKVCObJvGHioIfH9pitw+vIL39CAwDul351t9/bRDr4QiYoqhttq+BkUsjiozajCzHIqXZQczhOalxB58lTnkpFU x-ms-exchange-antispam-messagedata: Nsyl5y+2iumoDHixQ/96BJyIgUmONdJcnYb+T+KhrBWk/KuYfAuXu8sVTGxG9jjgOSFRwQsBlIgBRAS+5is3uQistb+yF9lF5je0yQAWsj6FgzAHN6mWhl/s0bvCH0o/n8O12KKR7ASafSGuv6O5Fg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: c2d60203-b68b-4da3-01df-08d7d57c00d2 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Mar 2020 14:01:26.1772 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: UzophMYepi6mffS0iqsEAporXBcBBCd9OTmWnddhiieAgFonQU4IPYO14OdfofTwoJKtU4HDLdTjvYV3rA98VQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB4546 Subject: Re: [dpdk-dev] [PATCH] net/mlx5: introduce dev parameter for hairpin X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, > -----Original Message----- > From: Bing Zhao > Sent: Tuesday, March 24, 2020 2:59 PM > To: Ori Kam ; Matan Azrad > Cc: Raslan Darawsheh ; Slava Ovsiienko > ; dev@dpdk.org > Subject: [PATCH] net/mlx5: introduce dev parameter for hairpin >=20 > When creating a hairpin queue, the total data size and the maximal > number of packets are interrelated. The differ is the stride size. > Larger buffer size means big packet like jumbo could be supported, > but in the meanwhile, it will introduce more cache misses and have a > side effect on the performance. > Now a new device parameter "hp_buf_log_sz" is introduced for > applications to set the total data buffer size (the logarithm value). > Then the maximal number of packets will also be calculated > automaticlly by this value. > Applications could also change this value to a larger one in order > to support larger packets in hairpin case. A smaller value will be > beneficial for memory consumption. > If it is not set, the default value will be used. >=20 > Signed-off-by: Bing Zhao > --- > drivers/net/mlx5/mlx5.c | 10 ++++++++++ > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_defs.h | 2 +- > drivers/net/mlx5/mlx5_rxq.c | 17 ++++++++++++++--- > drivers/net/mlx5/mlx5_txq.c | 17 ++++++++++++++--- > 5 files changed, 40 insertions(+), 7 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c > index 94aaa60..3e6eaa0 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -150,6 +150,12 @@ > /* Configure timeout of LRO session (in microseconds). */ > #define MLX5_LRO_TIMEOUT_USEC "lro_timeout_usec" >=20 > +/* > + * Device parameter to configure the total data buffer size for a single > + * hairpin queue (logarithm value). > + */ > +#define MLX5_HP_BUF_SIZE "hp_buf_log_sz" > + > #ifndef HAVE_IBV_MLX5_MOD_MPW > #define MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED (1 << 2) > #define MLX5DV_CONTEXT_FLAGS_ENHANCED_MPW (1 << 3) > @@ -1580,6 +1586,8 @@ struct mlx5_flow_id_pool * > config->lro.timeout =3D tmp; > } else if (strcmp(MLX5_CLASS_ARG_NAME, key) =3D=3D 0) { > DRV_LOG(DEBUG, "class argument is %s.", val); > + } else if (strcmp(MLX5_HP_BUF_SIZE, key) =3D=3D 0) { > + config->log_hp_size =3D tmp; > } else { > DRV_LOG(WARNING, "%s: unknown parameter", key); > rte_errno =3D EINVAL; > @@ -1632,6 +1640,7 @@ struct mlx5_flow_id_pool * > MLX5_MAX_DUMP_FILES_NUM, > MLX5_LRO_TIMEOUT_USEC, > MLX5_CLASS_ARG_NAME, > + MLX5_HP_BUF_SIZE, > NULL, > }; > struct rte_kvargs *kvlist; > @@ -3342,6 +3351,7 @@ struct mlx5_flow_id_pool * > }, > .dv_esw_en =3D 1, > .dv_flow_en =3D 1, > + .log_hp_size =3D MLX5_ARG_UNSET, > }; > /* Device specific configuration. */ > switch (pci_dev->id.device_id) { > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h > index d7c519b..53f35fd 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -191,6 +191,7 @@ struct mlx5_dev_config { > unsigned int tso_max_payload_sz; /* Maximum TCP payload for TSO. > */ > unsigned int ind_table_max_size; /* Maximum indirection table size. > */ > unsigned int max_dump_files_num; /* Maximum dump files per > queue. */ > + unsigned int log_hp_size; /* Single hairpin queue data size in total. *= / > int txqs_inline; /* Queue number threshold for inlining. */ > int txq_inline_min; /* Minimal amount of data bytes to inline. */ > int txq_inline_max; /* Max packet size for inlining with SEND. */ > diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h > index 83ca367..19e8253 100644 > --- a/drivers/net/mlx5/mlx5_defs.h > +++ b/drivers/net/mlx5/mlx5_defs.h > @@ -175,7 +175,7 @@ >=20 > /* Hairpin TX/RX queue configuration parameters. */ > #define MLX5_HAIRPIN_QUEUE_STRIDE 6 > -#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (15 + 2) > +#define MLX5_HAIRPIN_JUMBO_LOG_SIZE (14 + 2) >=20 > /* Definition of static_assert found in /usr/include/assert.h */ > #ifndef HAVE_STATIC_ASSERT > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c > index 8a6b410..3fdb632 100644 > --- a/drivers/net/mlx5/mlx5_rxq.c > +++ b/drivers/net/mlx5/mlx5_rxq.c > @@ -1286,9 +1286,20 @@ > attr.hairpin =3D 1; > max_wq_data =3D priv->config.hca_attr.log_max_hairpin_wq_data_sz; > /* Jumbo frames > 9KB should be supported, and more packets. */ > - attr.wq_attr.log_hairpin_data_sz =3D > - (max_wq_data < > MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? > - max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; > + if (priv->config.log_hp_size !=3D (uint32_t)MLX5_ARG_UNSET) { > + if (priv->config.log_hp_size > max_wq_data) { > + DRV_LOG(ERR, "total data size %u power of 2 is " > + "too large for hairpin", > + priv->config.log_hp_size); > + rte_errno =3D ERANGE; > + return NULL; > + } > + attr.wq_attr.log_hairpin_data_sz =3D priv->config.log_hp_size; > + } else { > + attr.wq_attr.log_hairpin_data_sz =3D > + (max_wq_data < > MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? > + max_wq_data : > MLX5_HAIRPIN_JUMBO_LOG_SIZE; > + } > /* Set the packets number to the maximum value for performance. > */ > attr.wq_attr.log_hairpin_num_packets =3D > attr.wq_attr.log_hairpin_data_sz - > diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c > index 57bc116..0653f4c 100644 > --- a/drivers/net/mlx5/mlx5_txq.c > +++ b/drivers/net/mlx5/mlx5_txq.c > @@ -512,9 +512,20 @@ > attr.tis_lst_sz =3D 1; > max_wq_data =3D priv->config.hca_attr.log_max_hairpin_wq_data_sz; > /* Jumbo frames > 9KB should be supported, and more packets. */ > - attr.wq_attr.log_hairpin_data_sz =3D > - (max_wq_data < > MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? > - max_wq_data : MLX5_HAIRPIN_JUMBO_LOG_SIZE; > + if (priv->config.log_hp_size !=3D (uint32_t)MLX5_ARG_UNSET) { > + if (priv->config.log_hp_size > max_wq_data) { > + DRV_LOG(ERR, "total data size %u power of 2 is " > + "too large for hairpin", > + priv->config.log_hp_size); > + rte_errno =3D ERANGE; > + return NULL; > + } > + attr.wq_attr.log_hairpin_data_sz =3D priv->config.log_hp_size; > + } else { > + attr.wq_attr.log_hairpin_data_sz =3D > + (max_wq_data < > MLX5_HAIRPIN_JUMBO_LOG_SIZE) ? > + max_wq_data : > MLX5_HAIRPIN_JUMBO_LOG_SIZE; > + } > /* Set the packets number to the maximum value for performance. > */ > attr.wq_attr.log_hairpin_num_packets =3D > attr.wq_attr.log_hairpin_data_sz - > -- > 1.8.3.1 Patch applied to next-net-mlx, Kindest regards Raslan Darawsheh