From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50079.outbound.protection.outlook.com [40.107.5.79]) by dpdk.org (Postfix) with ESMTP id 1C90A1B10E for ; Thu, 1 Nov 2018 18:20:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PmwCDV8P/6E9YS3Ngbx+Dn3MtMOTm7Acy4PKkCIqRBw=; b=aSpSAXRjTZnsbtJ+uf+mKJnwpDCXoyhLSExKklKWIvZ1WemNuWRMacncQvImgDicWYMeAExIkKjHNk7frJjYspXLVY0eoJR4Xh5aaAjONX2RV/KfQsyzteQ/PttCof0fxJjSRJ66u2FH7LCtpdRaBofZ1P/Cqr/N4aXmVHK1dso= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB3947.eurprd05.prod.outlook.com (52.134.71.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1273.19; Thu, 1 Nov 2018 17:20:33 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::f8a1:fcab:94f0:97cc]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::f8a1:fcab:94f0:97cc%4]) with mapi id 15.20.1273.030; Thu, 1 Nov 2018 17:20:33 +0000 From: Yongseok Koh To: Shahaf Shuler CC: "dev@dpdk.org" , Yongseok Koh Thread-Topic: [PATCH v3 2/2] net/mlx5: make vectorized Tx threshold configurable Thread-Index: AQHUcgcxvaZD92G8qEWu0qA/83S/Fw== Date: Thu, 1 Nov 2018 17:20:32 +0000 Message-ID: <20181101172019.26195-3-yskoh@mellanox.com> References: <20181029231509.39886-1-yskoh@mellanox.com> <20181101172019.26195-1-yskoh@mellanox.com> In-Reply-To: <20181101172019.26195-1-yskoh@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR05CA0023.namprd05.prod.outlook.com (2603:10b6:a03:c0::36) To DB3PR0502MB3980.eurprd05.prod.outlook.com (2603:10a6:8:10::27) authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [209.116.155.178] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB3PR0502MB3947; 6:swM6F3MMW+c/gCmddTHemOGzRGSGz8GnECQKCzGCfQ30EAssvWHkLmfd+X+nyrro4cTJ4KvWNSDeSwbzo7baGejRF7JthUeQYMYo46i090SJr6ZbcGgJxlMPEQonWCRHwPz3pKiZ27tZtPWyr3xM3op+BW0+kskUx8BlLX2MIWKMA1aNPFE6ThSEh49dQePPSazKCa0WaosggrvOfG4e/pELMzGhZ/8KrNhQAYxJQyvJnbgssb+/pz8JE8lwEw9GiZyOuCqFBOQPlfZtRn90tUAQCTOiDbs8y6tucGnP6jw8RflAPMfYcq1c1wYOlIyz0AbwuWganMzfaxfkqQRVMkcuvGvKhElLj6qkrFHYN++cCah1PwjDCm+yR62PeoTnKiN8SCajakEd8eh9fHJGME3eowCTbQ41CpTbby/vPbm/AJ1Mp+iFgt504KxI0BFetKRXETr+g6OlQL+a+Ph7Xw==; 5:b126/QoAmpirru5hKDyVSaYXsVaBH0unOVyxs8z+Nfzv657yqkWWyzPjhxVuHRd+uMrfwz5ftPglK9NwtS+MERYUyWUoeBw1kSsVjpiBrRc8wgFCOH+zqjpkdvb8rnHeiVxEGaon1hX6wTslIWwW/zQBvVKCtMRlcvLO+wZjYZU=; 7:ZfjNBTls0Hjk4fyy71rZlamnWwUuGwfiNAE083I8CMS2gafBwgsHvIFSg/IxOulOy0VrH5/4t+2F12ElBcA/YXZoA4iB0/LyF9BN6t3o0osVAqd0Ugooe4QvEtDkkxeUifes6g4lCPZEocJWEX8Mxg== x-ms-office365-filtering-correlation-id: c604babc-d5d8-4d11-960a-08d6401e5429 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB3947; x-ms-traffictypediagnostic: DB3PR0502MB3947: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231382)(944501410)(52105095)(93006095)(93001095)(10201501046)(3002001)(6055026)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:DB3PR0502MB3947; BCL:0; PCL:0; RULEID:; SRVR:DB3PR0502MB3947; x-forefront-prvs: 0843C17679 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(376002)(346002)(396003)(136003)(366004)(39860400002)(199004)(189003)(14444005)(81166006)(2900100001)(81156014)(68736007)(4326008)(5250100002)(105586002)(106356001)(86362001)(8936002)(8676002)(71200400001)(305945005)(25786009)(256004)(7736002)(97736004)(14454004)(99286004)(478600001)(6862004)(5660300001)(54906003)(107886003)(6436002)(36756003)(52116002)(71190400001)(486006)(66066001)(6636002)(26005)(2616005)(476003)(2906002)(446003)(53936002)(6116002)(11346002)(6486002)(102836004)(186003)(3846002)(1076002)(6512007)(6506007)(386003)(76176011)(316002)(37006003)(309714004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB3947; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: JdB6RONUMhGGzkkHLHeCAcQRK174Bu3s4l6bVpdI4VSdcZJ1LRXngt6MtnQ1Q9E2Rg8pg31DYB4SLmzAtJDl/la/I+IMMef1ZBv1Ele0LMeCWRJ4RnSAo+UKAUnpzgra1/bePyTypE0ym0yhGpzSeV8/799WxNhbwWrYQw4x30VQn6/VXx9wQnMAjo3m/Gl+W2SsLYtD6sdsReKjNeuLlHLgEK49VXK4HSifQnhPeawgC/YPW64FiwrqsZw6zrXda83r5qiJ2LVSy+jnmRatMsmvz0MJWEQ8ofRQFpXtUsK/7s0CnJB2JafnZMXk4DwnWNG63GVHv/gEvjCAU1ppjLrHGrk8n81n0rLwjgVWXA0= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: c604babc-d5d8-4d11-960a-08d6401e5429 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Nov 2018 17:20:32.8404 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB3947 Subject: [dpdk-dev] [PATCH v3 2/2] net/mlx5: make vectorized Tx threshold configurable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2018 17:20:34 -0000 Add txqs_max_vec parameter to configure the maximum number of Tx queues to enable vectorized Tx. And its default value is set according to the architecture and device type. Signed-off-by: Yongseok Koh --- doc/guides/nics/mlx5.rst | 16 +++++++++++++++- drivers/net/mlx5/mlx5.c | 16 ++++++++++++++++ drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_defs.h | 9 +++++++-- drivers/net/mlx5/mlx5_rxtx_vec.c | 2 +- 5 files changed, 40 insertions(+), 4 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 1dc32829ff..7379cf39b0 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -338,6 +338,20 @@ Run-time configuration =20 - Set to 8 by default. =20 +- ``txqs_max_vec`` parameter [int] + + Enable vectorized Tx only when the number of TX queues is less than or + equal to this value. Effective only when ``tx_vec_en`` is enabled. + + On ConnectX-5: + + - Set to 8 by default on ARMv8. + - Set to 4 by default otherwise. + + On Bluefield + + - Set to 16 by default. + - ``txq_mpw_en`` parameter [int] =20 A nonzero value enables multi-packet send (MPS) for ConnectX-4 Lx and @@ -383,7 +397,7 @@ Run-time configuration - ``tx_vec_en`` parameter [int] =20 A nonzero value enables Tx vector on ConnectX-5 and Bluefield NICs if th= e number of - global Tx queues on the port is lesser than MLX5_VPMD_MIN_TXQS. + global Tx queues on the port is less than ``txqs_max_vec``. =20 This option cannot be used with certain offloads such as ``DEV_TX_OFFLOA= D_TCP_TSO, DEV_TX_OFFLOAD_VXLAN_TNL_TSO, DEV_TX_OFFLOAD_GRE_TNL_TSO, DEV_TX_OFFLOAD= _VLAN_INSERT``. diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 629f03da99..62ac54f933 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -75,6 +75,12 @@ */ #define MLX5_TXQS_MIN_INLINE "txqs_min_inline" =20 +/* + * Device parameter to configure the number of TX queues threshold for + * enabling vectorized Tx. + */ +#define MLX5_TXQS_MAX_VEC "txqs_max_vec" + /* Device parameter to enable multi-packet send WQEs. */ #define MLX5_TXQ_MPW_EN "txq_mpw_en" =20 @@ -496,6 +502,8 @@ mlx5_args_check(const char *key, const char *val, void = *opaque) config->txq_inline =3D tmp; } else if (strcmp(MLX5_TXQS_MIN_INLINE, key) =3D=3D 0) { config->txqs_inline =3D tmp; + } else if (strcmp(MLX5_TXQS_MAX_VEC, key) =3D=3D 0) { + config->txqs_vec =3D tmp; } else if (strcmp(MLX5_TXQ_MPW_EN, key) =3D=3D 0) { config->mps =3D !!tmp; } else if (strcmp(MLX5_TXQ_MPW_HDR_DSEG_EN, key) =3D=3D 0) { @@ -543,6 +551,7 @@ mlx5_args(struct mlx5_dev_config *config, struct rte_de= vargs *devargs) MLX5_RXQS_MIN_MPRQ, MLX5_TXQ_INLINE, MLX5_TXQS_MIN_INLINE, + MLX5_TXQS_MAX_VEC, MLX5_TXQ_MPW_EN, MLX5_TXQ_MPW_HDR_DSEG_EN, MLX5_TXQ_MAX_INLINE_LEN, @@ -1432,6 +1441,7 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_u= nused, .rx_vec_en =3D 1, .txq_inline =3D MLX5_ARG_UNSET, .txqs_inline =3D MLX5_ARG_UNSET, + .txqs_vec =3D MLX5_ARG_UNSET, .inline_max_packet_sz =3D MLX5_ARG_UNSET, .vf_nl_en =3D 1, .mprq =3D { @@ -1443,6 +1453,9 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_u= nused, }; /* Device speicific configuration. */ switch (pci_dev->id.device_id) { + case PCI_DEVICE_ID_MELLANOX_CONNECTX5BF: + dev_config.txqs_vec =3D MLX5_VPMD_MAX_TXQS_BLUEFIELD; + break; case PCI_DEVICE_ID_MELLANOX_CONNECTX4VF: case PCI_DEVICE_ID_MELLANOX_CONNECTX4LXVF: case PCI_DEVICE_ID_MELLANOX_CONNECTX5VF: @@ -1452,6 +1465,9 @@ mlx5_pci_probe(struct rte_pci_driver *pci_drv __rte_u= nused, default: break; } + /* Set architecture-dependent default value if unset. */ + if (dev_config.txqs_vec =3D=3D MLX5_ARG_UNSET) + dev_config.txqs_vec =3D MLX5_VPMD_MAX_TXQS; for (i =3D 0; i !=3D n; ++i) { uint32_t restore; =20 diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 74d11c0904..bc500b2bc3 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -140,6 +140,7 @@ struct mlx5_dev_config { unsigned int ind_table_max_size; /* Maximum indirection table size. */ int txq_inline; /* Maximum packet size for inlining. */ int txqs_inline; /* Queue number threshold for inlining. */ + int txqs_vec; /* Queue number threshold for vectorized Tx. */ int inline_max_packet_sz; /* Max packet size for inlining. */ }; =20 diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index f2a1679511..bfe6655800 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -60,8 +60,13 @@ /* Maximum Packet headers size (L2+L3+L4) for TSO. */ #define MLX5_MAX_TSO_HEADER 192 =20 -/* Default minimum number of Tx queues for vectorized Tx. */ -#define MLX5_VPMD_MIN_TXQS 4 +/* Default maximum number of Tx queues for vectorized Tx. */ +#if defined(RTE_ARCH_ARM64) +#define MLX5_VPMD_MAX_TXQS 8 +#else +#define MLX5_VPMD_MAX_TXQS 4 +#endif +#define MLX5_VPMD_MAX_TXQS_BLUEFIELD 16 =20 /* Threshold of buffer replenishment for vectorized Rx. */ #define MLX5_VPMD_RXQ_RPLNSH_THRESH(n) \ diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.c b/drivers/net/mlx5/mlx5_rxtx_= vec.c index 1453f4ff63..340292addf 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec.c +++ b/drivers/net/mlx5/mlx5_rxtx_vec.c @@ -277,7 +277,7 @@ mlx5_check_vec_tx_support(struct rte_eth_dev *dev) uint64_t offloads =3D dev->data->dev_conf.txmode.offloads; =20 if (!priv->config.tx_vec_en || - priv->txqs_n > MLX5_VPMD_MIN_TXQS || + priv->txqs_n > (unsigned int)priv->config.txqs_vec || priv->config.mps !=3D MLX5_MPW_ENHANCED || offloads & ~MLX5_VEC_TX_OFFLOAD_CAP) return -ENOTSUP; --=20 2.11.0