From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20082.outbound.protection.outlook.com [40.107.2.82]) by dpdk.org (Postfix) with ESMTP id 675DF10A3 for ; Thu, 1 Nov 2018 09:03:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8nMUkKi+pqV1Wtn8avPksjomYwI6bu8AVYf0BT4L+XA=; b=XIiCy1pumQqGfFxBoXkVteuVIVWnluFy2FQ1zXZko60Y0I/tXCgA6wcs+92conoX4vrUmhsCThnArnv0/mWhqkMsiMjQp3gD/UQo+weXoKaSU/IT6P7ZDFj5iymy4jXW+NPBkuwnoDueYumchAvEK3zPsmqrx0EDclPP+QexwoE= Received: from DB7PR05MB4426.eurprd05.prod.outlook.com (52.134.109.15) by DB7PR05MB4297.eurprd05.prod.outlook.com (52.134.108.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1294.21; Thu, 1 Nov 2018 08:03:47 +0000 Received: from DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::80e:e6b:baf2:d973]) by DB7PR05MB4426.eurprd05.prod.outlook.com ([fe80::80e:e6b:baf2:d973%3]) with mapi id 15.20.1273.027; Thu, 1 Nov 2018 08:03:47 +0000 From: Shahaf Shuler To: Yongseok Koh CC: "dev@dpdk.org" Thread-Topic: [PATCH v2 2/2] net/mlx5: make vectorized Tx threshold configurable Thread-Index: AQHUcCUOIipQJI7vMkSZc6KQtL0neqU6j5Fg Date: Thu, 1 Nov 2018 08:03:47 +0000 Message-ID: References: <20181029231509.39886-1-yskoh@mellanox.com> <20181030074901.40342-1-yskoh@mellanox.com> <20181030074901.40342-3-yskoh@mellanox.com> In-Reply-To: <20181030074901.40342-3-yskoh@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB7PR05MB4297; 6:X/j4oVjNMFN8Vk2HzRmSvc9y04pbR2H3WfZ7Ar22jY6G2oR5SYtNqyN7vknQp9TsIvkeOGKL9l8vXPuoSrdxFvmrm6hIFOPA+xPWQR9dtt7kQ/5Gwibx1XAEH/XoA5ljgDsxD3AAi2eHyhbbQErikv9WG2vR4bmQaEITtmAeE5idcYAuMjgHpTCaImBKBTKA5tFY5kwjkxQs5+IYOnDFl+6LdFWsDmeb2xtz9TluJbB4UKZ4sCnE+lrxPjsKYAfblp6672qL4zR+vcPu/gb0jQuLok6p1OEwSN2K6TLcvL2S0L0xtz5/zFeKDjGCZQQdzfYzVRezl2kIXeEBTjFnl1KOaOgTNYMP4QgQKMboTNrAHbRgcc0qd+e6DOc9fPSQMX6S35fuFkSoQhNo4osaXrCak3LBAHGzgh0QpY/X19zya/I8PiY1pK3qLhdzap4YNARGbILmeheN7u0DijK8Ug==; 5:MQp7/2P/OdAHlYVAO3eiVDARVUc8I3aCNhGrw/4cUXD/sidOXOt+E8AGouoSk2F3zY4CvXoDvItLcjcVNUAbie86hNY6vWnhx5FecJEQZjBc9xPG3qNONeh2Qqtlcq9lBEOS1aoBUxluz9XKN78P2ObqSBXduAfKk4xCQFpjfVY=; 7:QjqgT/Zzit9lsUI3FR1KCbuedGC+DED3CyqLj8jwdQWWmG64hxd7ms25N9+m+R/aKxLEQN49r34IKqMDa9293pqF1cJe9VPhDa48jmwyW4euMGd1ypVKJtMPyvyL3wVMwuqTVrgq0AQrSTjKmPgLEw== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 9f4c9c81-5213-4701-b16a-08d63fd08d24 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB7PR05MB4297; x-ms-traffictypediagnostic: DB7PR05MB4297: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231382)(944501410)(52105095)(6055026)(148016)(149066)(150057)(6041310)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:DB7PR05MB4297; BCL:0; PCL:0; RULEID:; SRVR:DB7PR05MB4297; x-forefront-prvs: 0843C17679 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(376002)(346002)(39860400002)(136003)(366004)(189003)(199004)(99286004)(33656002)(26005)(478600001)(68736007)(106356001)(229853002)(5250100002)(6862004)(53936002)(2906002)(6506007)(9686003)(3846002)(8936002)(6116002)(316002)(7696005)(105586002)(4326008)(76176011)(102836004)(55016002)(97736004)(8676002)(6636002)(25786009)(2900100001)(81166006)(6436002)(7736002)(81156014)(186003)(74316002)(86362001)(5660300001)(66066001)(11346002)(446003)(6246003)(476003)(486006)(71190400001)(256004)(14454004)(71200400001)(305945005)(309714004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB7PR05MB4297; H:DB7PR05MB4426.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: SqQTfzlKk4gGdjvFjC1zM3TzqvFMhsoy05ylqBeH6wha7Z9t0RZUI8i4wmu9Shf1dcuOz5O72EY7hFbmRjR6YKbKeHoU+V0yqT+Vu0SWe8sCbCZQm/19vSYkjjKouEdwcHom/XMGIWTlU1HKbwHxlMJeXQ7ScKg4K7M3b762XtXPGANWqx9Jgw+jzXll70TG9sCt4gGSmWQsKEycrcq98TwKG40oJRwUkdK38bEsxpStFkolOb9bavX8gWre0KjHT2O5Uj+JvZng+lRp92zDaN1MOYNgzWZJv3A+RvmMdhy+wBVzUBlnX1AabWPs6IkqddtFss5dsMl2ucquAU8fmV9G7ize8sB6yChGuBrUKTo= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9f4c9c81-5213-4701-b16a-08d63fd08d24 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Nov 2018 08:03:47.1912 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR05MB4297 Subject: Re: [dpdk-dev] [PATCH v2 2/2] net/mlx5: make vectorized Tx threshold configurable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2018 08:03:48 -0000 Tuesday, October 30, 2018 9:49 AM, Yongseok Koh: > Subject: [PATCH v2 2/2] net/mlx5: make vectorized Tx threshold configurab= le >=20 > Add txqs_max_vec parameter to configure the maximum number of Tx > queues to enable vectorized Tx. And its default value is set according to= the > architecture and device type. >=20 > Signed-off-by: Yongseok Koh > --- > doc/guides/nics/mlx5.rst | 16 +++++++++++++++- > drivers/net/mlx5/mlx5.c | 16 ++++++++++++++++ > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_defs.h | 6 ++++-- > drivers/net/mlx5/mlx5_rxtx_vec.c | 2 +- > 5 files changed, 37 insertions(+), 4 deletions(-) >=20 > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index > 1dc32829ff..7379cf39b0 100644 > --- a/doc/guides/nics/mlx5.rst > +++ b/doc/guides/nics/mlx5.rst > @@ -338,6 +338,20 @@ Run-time configuration >=20 > - Set to 8 by default. >=20 > +- ``txqs_max_vec`` parameter [int] > + > + Enable vectorized Tx only when the number of TX queues is less than > + or equal to this value. Effective only when ``tx_vec_en`` is enabled. > + > + On ConnectX-5: > + > + - Set to 8 by default on ARMv8. > + - Set to 4 by default otherwise. > + > + On Bluefield > + > + - Set to 16 by default. > + > - ``txq_mpw_en`` parameter [int] >=20 > A nonzero value enables multi-packet send (MPS) for ConnectX-4 Lx and > @@ -383,7 +397,7 @@ Run-time configuration > - ``tx_vec_en`` parameter [int] >=20 > A nonzero value enables Tx vector on ConnectX-5 and Bluefield NICs if = the > number of > - global Tx queues on the port is lesser than MLX5_VPMD_MIN_TXQS. > + global Tx queues on the port is less than ``txqs_max_vec``. >=20 > This option cannot be used with certain offloads such as > ``DEV_TX_OFFLOAD_TCP_TSO, > DEV_TX_OFFLOAD_VXLAN_TNL_TSO, DEV_TX_OFFLOAD_GRE_TNL_TSO, > DEV_TX_OFFLOAD_VLAN_INSERT``. > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > 6fa50ba1b1..d575469f9b 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -75,6 +75,12 @@ > */ > #define MLX5_TXQS_MIN_INLINE "txqs_min_inline" >=20 > +/* > + * Device parameter to configure the number of TX queues threshold for > + * enabling vectorized Tx. > + */ > +#define MLX5_TXQS_MAX_VEC "txqs_max_vec" > + > /* Device parameter to enable multi-packet send WQEs. */ #define > MLX5_TXQ_MPW_EN "txq_mpw_en" >=20 > @@ -496,6 +502,8 @@ mlx5_args_check(const char *key, const char *val, > void *opaque) > config->txq_inline =3D tmp; > } else if (strcmp(MLX5_TXQS_MIN_INLINE, key) =3D=3D 0) { > config->txqs_inline =3D tmp; > + } else if (strcmp(MLX5_TXQS_MAX_VEC, key) =3D=3D 0) { > + config->txqs_vec =3D tmp; > } else if (strcmp(MLX5_TXQ_MPW_EN, key) =3D=3D 0) { > config->mps =3D !!tmp; > } else if (strcmp(MLX5_TXQ_MPW_HDR_DSEG_EN, key) =3D=3D 0) { @@ - > 543,6 +551,7 @@ mlx5_args(struct mlx5_dev_config *config, struct > rte_devargs *devargs) > MLX5_RXQS_MIN_MPRQ, > MLX5_TXQ_INLINE, > MLX5_TXQS_MIN_INLINE, > + MLX5_TXQS_MAX_VEC, > MLX5_TXQ_MPW_EN, > MLX5_TXQ_MPW_HDR_DSEG_EN, > MLX5_TXQ_MAX_INLINE_LEN, > @@ -1443,6 +1452,8 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv > __rte_unused, > }; > /* Device speicific configuration. */ > switch (pci_dev->id.device_id) { > + case PCI_DEVICE_ID_MELLANOX_CONNECTX5BF: > + dev_config.txqs_vec =3D > MLX5_VPMD_MAX_TXQS_BLUEFIELD; Missing break?=20 > case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF: > case PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF: > case PCI_DEVICE_ID_MELLANOX_CONNECTX5VF: What about all the VFs? They don't have a default value, will it lead the v= ec code not to be used for VFs at all? I think VFs decision should be like the PF. If the Arch is ARMv8 then 8 oth= erwise 4.=20 Pay attention PCI_DEVICE_ID_MELLANOX_CONNECTX5BFVF is missing, and its defa= ult value should be like above phrase, because the VF port is not on the So= C rather on the host side.=20 > @@ -1450,6 +1461,11 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv > __rte_unused, > dev_config.vf =3D 1; > break; > default: > +#if defined(RTE_ARCH_ARM64) > + dev_config.txqs_vec =3D MLX5_VPMD_MAX_TXQS_ARM64; > #else > + dev_config.txqs_vec =3D MLX5_VPMD_MAX_TXQS; #endif > break; > } > for (i =3D 0; i !=3D n; ++i) { > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > 24a3415c8d..0b4418b80b 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -140,6 +140,7 @@ struct mlx5_dev_config { > unsigned int ind_table_max_size; /* Maximum indirection table size. > */ > int txq_inline; /* Maximum packet size for inlining. */ > int txqs_inline; /* Queue number threshold for inlining. */ > + int txqs_vec; /* Queue number threshold for vectorized Tx. */ > int inline_max_packet_sz; /* Max packet size for inlining. */ }; >=20 > diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h > index f2a1679511..221ca188bb 100644 > --- a/drivers/net/mlx5/mlx5_defs.h > +++ b/drivers/net/mlx5/mlx5_defs.h > @@ -60,8 +60,10 @@ > /* Maximum Packet headers size (L2+L3+L4) for TSO. */ #define > MLX5_MAX_TSO_HEADER 192 >=20 > -/* Default minimum number of Tx queues for vectorized Tx. */ -#define > MLX5_VPMD_MIN_TXQS 4 > +/* Default maximum number of Tx queues for vectorized Tx. */ #define > +MLX5_VPMD_MAX_TXQS 4 #define MLX5_VPMD_MAX_TXQS_ARM64 8 > #define > +MLX5_VPMD_MAX_TXQS_BLUEFIELD 16 >=20 > /* Threshold of buffer replenishment for vectorized Rx. */ #define > MLX5_VPMD_RXQ_RPLNSH_THRESH(n) \ diff --git > a/drivers/net/mlx5/mlx5_rxtx_vec.c b/drivers/net/mlx5/mlx5_rxtx_vec.c > index 1453f4ff63..340292addf 100644 > --- a/drivers/net/mlx5/mlx5_rxtx_vec.c > +++ b/drivers/net/mlx5/mlx5_rxtx_vec.c > @@ -277,7 +277,7 @@ mlx5_check_vec_tx_support(struct rte_eth_dev > *dev) > uint64_t offloads =3D dev->data->dev_conf.txmode.offloads; >=20 > if (!priv->config.tx_vec_en || > - priv->txqs_n > MLX5_VPMD_MIN_TXQS || > + priv->txqs_n > (unsigned int)priv->config.txqs_vec || > priv->config.mps !=3D MLX5_MPW_ENHANCED || > offloads & ~MLX5_VEC_TX_OFFLOAD_CAP) > return -ENOTSUP; > -- > 2.11.0