From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 47C8DA046B for ; Mon, 22 Jul 2019 07:34:08 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0E0B71B464; Mon, 22 Jul 2019 07:34:08 +0200 (CEST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00088.outbound.protection.outlook.com [40.107.0.88]) by dpdk.org (Postfix) with ESMTP id 32BC85B34 for ; Mon, 22 Jul 2019 07:34:07 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JLeCGREDI6utESVoK0UD2bVBTB75Qec7CaC2jLH71QmiaeTVfgiclTWKAqhvKqbfDdJAXPx2zGfTj5UE9TtmovvSAljdcWg1ux4XUX39yCS5eJbF/sxZZkXTKlXTOc89I6zbCkd1k8LwaUbLeiQyr3FRKZg962iW1pht5uGg5CBe12gidy/j4x+X0ZEJrwyJ+flDDftzqphVFUCdOYL+K88rQpUgg+KcRHq2VRl9sM4zOtrRlustlelx1YdeJ/BUbnsn31cdWJr6FWc+CszNWzat/5MTXOVjGUresFNA3kVPfzQb9tPGVWVd3BgTC1ATVYF7RS8gzD7MiVogo4h/ug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ngp17Q/LWqKGDowfMD6yV97clxVpFmJpZ/OdfoU/PR0=; b=V+my3HeN63UDozvN7EXcYHU4b3FAYdyZEKCwS2pUi85EtrMrUQoFrNTRkQR0YGLAMLYo7xHs4CxyPaVkmV7nBDzEkJV0vS6LCZtxUSvrlt/R+rITjthKCS54jCOVtrV3C6XeJmd3Xj7vyubKVdik5YowEXBVSYLV/m7PS4D2ut9/tTq0O9gauws2ymht3OmMyQgqR9w1vYYbCyK+yc9t7+AcDUnxExu0zgQkQTyqV6k3A/kHpzQT2oyvYxNtSnEmq4hzS71B4v32AXcBQXuA6MNXN7Xa7adStWRdM0BKqbQScQiMY3k9k7aM0SvnQ791kWwPs6yzpBzyUFJXZc7q+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=mellanox.com;dmarc=pass action=none header.from=mellanox.com;dkim=pass header.d=mellanox.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ngp17Q/LWqKGDowfMD6yV97clxVpFmJpZ/OdfoU/PR0=; b=bbCaXlfdkl4EU8lgRPeKMLmn3kcGIlrDshfJ1s1CixmUMhb1mP0/j2wAi6w8h1YvX8tgjJFnAzbYhVBh16IKMIuFnz9gT2f+SjgYQmcSpE5SUDHYWcj7DXhSYSQd1VTlal6o/h18FTyWPkvqyOyP9s+azS8ElZzrxFhGOvYkORM= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB3995.eurprd05.prod.outlook.com (52.134.72.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2094.16; Mon, 22 Jul 2019 05:34:05 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::69c1:c0d7:1fa1:f89f]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::69c1:c0d7:1fa1:f89f%6]) with mapi id 15.20.2094.013; Mon, 22 Jul 2019 05:34:05 +0000 From: Yongseok Koh To: Slava Ovsiienko CC: "dev@dpdk.org" Thread-Topic: [PATCH v4 6/8] net/mlx5: introduce Tx burst routine template Thread-Index: AQHVP9A0Uny3yXFvDk2rgbqK+ECIHqbWHoaA Date: Mon, 22 Jul 2019 05:34:05 +0000 Message-ID: <321B0FFF-5687-4005-BDA4-6F6DE46760AF@mellanox.com> References: <1563346400-1762-1-git-send-email-viacheslavo@mellanox.com> <1563719100-368-1-git-send-email-viacheslavo@mellanox.com> <1563719100-368-7-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1563719100-368-7-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-originating-ip: [69.181.245.183] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f89bc294-b591-44a3-f045-08d70e663670 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DB3PR0502MB3995; x-ms-traffictypediagnostic: DB3PR0502MB3995: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 01068D0A20 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(376002)(346002)(366004)(396003)(136003)(199004)(189003)(478600001)(6486002)(33656002)(53936002)(76176011)(53946003)(6436002)(316002)(37006003)(36756003)(6512007)(81156014)(81166006)(186003)(6862004)(25786009)(446003)(8676002)(26005)(30864003)(2616005)(11346002)(476003)(4326008)(229853002)(6636002)(8936002)(71190400001)(71200400001)(486006)(102836004)(7736002)(2906002)(14454004)(6506007)(53546011)(305945005)(6246003)(66946007)(66476007)(66556008)(64756008)(66446008)(66066001)(76116006)(91956017)(14444005)(256004)(6116002)(68736007)(86362001)(99286004)(5660300002)(3846002)(21314003)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB3995; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: AhZPkFv9GDnXSaJhuS9esTuZpMDQn/fFkJUpNcicO+hcgYQQbjNjMiJl7O2l6WlWUjxRMewsw4WNXigs/h3a3gB8H230ijyhXU9Xq7zlaEEUOFfjgYlO8AvUW1iKnBa8/tKO0QzWVYB6TXH+MTNMu9WIZySRuBQ7vGDucd0ASUK/RlpagRQH2mV/fOQFqrx3jFNbcSpJ5P2ddjNmEaODYhlWJJ0ihzTEfmLfhpbKdKNe97GwU8j3J9km4FgnWcHwH7Yw4wuv9n1xZsNg4kMO9+S+SxToCJIql3JIyxWj3n1s9MAfoUBdBZmatXfFqSBaH7rkL0k0VI67bpjrcd3HxpnE0kuFcU/shq0GFzNs/hy4XSHt2IGZob5CgKfNmS83Lnojz0i/9oByEXsInj7F/PB6xUmVEwEV/osMZJFSZW8= Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: f89bc294-b591-44a3-f045-08d70e663670 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2019 05:34:05.6820 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: yskoh@mellanox.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB3995 Subject: Re: [dpdk-dev] [PATCH v4 6/8] net/mlx5: introduce Tx burst routine template X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > On Jul 21, 2019, at 7:24 AM, Viacheslav Ovsiienko wrote: >=20 > Mellanox NICs support the wide set of Tx offloads. The supported > offloads are reported by the mlx5 PMD in rte_eth_dev_info tx_offload_capa > field. An application may choose any combination of supported offloads > and configure the device appropriately. Some of Tx offloads may be > not requested by application, or ever all of them may be omitted. > Most of the Tx offloads require some code branches in tx_burst routine > to support ones. If Tx offload is not requested the tx_burst routine > code may be significantly simplified and consume less CPU cycles. >=20 > For example, if application does not engage TSO offload this code > can be omitted, if multi-segment packet is not supposed the tx_burst > may assume single mbuf packets only, etc. >=20 > Currently, the mlx5 PMD implements multiple tx_burst subroutines > for most common combinations of requested Tx offloads, each branch > has its own dedicated implementation. It is not very easy to update, > support and develop such kind of code - multiple branches impose > the multiple points to process. Also many of frequently requested > offload combinations are not supported yet. That leads to selecting of > not completely matching tx_burst routine and harms the performance. >=20 > This patch introduces the new approach for tx_burst code. It is proposed > to develop the unified template for tx_burst routine, which supports > all the Tx offloads and takes the compile time defined parameter > describing the supposed set of supported offloads. On the base > of this template, the compiler is able to generate multiple tx_burst > routines highly optimized for the statically specified set of Tx offloads= . > Next, in runtime, at Tx queue configuration the best matching optimized > implementation of tx_burst is chosen. >=20 > This patch intentionally omits the template internal implementation, > but just introduces the template itself to emboss the approach of > the multiple specially tuned tx_burst routines. >=20 > Signed-off-by: Viacheslav Ovsiienko > --- Acked-by: Yongseok Koh > drivers/net/mlx5/mlx5_rxtx.c | 520 ++++++++++++++++++++++++++++++++++++++= ++++- > 1 file changed, 516 insertions(+), 4 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c > index 13f9431..16be493 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.c > +++ b/drivers/net/mlx5/mlx5_rxtx.c > @@ -1,6 +1,6 @@ > /* SPDX-License-Identifier: BSD-3-Clause > * Copyright 2015 6WIND S.A. > - * Copyright 2015 Mellanox Technologies, Ltd > + * Copyright 2015-2019 Mellanox Technologies, Ltd > */ >=20 > #include > @@ -34,6 +34,57 @@ > #include "mlx5_defs.h" > #include "mlx5_prm.h" >=20 > +/* TX burst subroutines return codes. */ > +enum mlx5_txcmp_code { > + MLX5_TXCMP_CODE_EXIT =3D 0, > + MLX5_TXCMP_CODE_ERROR, > + MLX5_TXCMP_CODE_SINGLE, > + MLX5_TXCMP_CODE_MULTI, > + MLX5_TXCMP_CODE_TSO, > + MLX5_TXCMP_CODE_EMPW, > +}; > + > +/* > + * These defines are used to configure Tx burst routine option set > + * supported at compile time. The not specified options are optimized ou= t > + * out due to if conditions can be explicitly calculated at compile time= . > + * The offloads with bigger runtime check (require more CPU cycles to > + * skip) overhead should have the bigger index - this is needed to > + * select the better matching routine function if no exact match and > + * some offloads are not actually requested. > + */ > +#define MLX5_TXOFF_CONFIG_MULTI (1u << 0) /* Multi-segment packets.*/ > +#define MLX5_TXOFF_CONFIG_TSO (1u << 1) /* TCP send offload supported.*/ > +#define MLX5_TXOFF_CONFIG_SWP (1u << 2) /* Tunnels/SW Parser offloads.*/ > +#define MLX5_TXOFF_CONFIG_CSUM (1u << 3) /* Check Sums offloaded. */ > +#define MLX5_TXOFF_CONFIG_INLINE (1u << 4) /* Data inlining supported. *= / > +#define MLX5_TXOFF_CONFIG_VLAN (1u << 5) /* VLAN insertion supported.*/ > +#define MLX5_TXOFF_CONFIG_METADATA (1u << 6) /* Flow metadata. */ > +#define MLX5_TXOFF_CONFIG_EMPW (1u << 8) /* Enhanced MPW supported.*/ > + > +/* The most common offloads groups. */ > +#define MLX5_TXOFF_CONFIG_NONE 0 > +#define MLX5_TXOFF_CONFIG_FULL (MLX5_TXOFF_CONFIG_MULTI | \ > + MLX5_TXOFF_CONFIG_TSO | \ > + MLX5_TXOFF_CONFIG_SWP | \ > + MLX5_TXOFF_CONFIG_CSUM | \ > + MLX5_TXOFF_CONFIG_INLINE | \ > + MLX5_TXOFF_CONFIG_VLAN | \ > + MLX5_TXOFF_CONFIG_METADATA) > + > +#define MLX5_TXOFF_CONFIG(mask) (olx & MLX5_TXOFF_CONFIG_##mask) > + > +#define MLX5_TXOFF_DECL(func, olx) \ > +static uint16_t mlx5_tx_burst_##func(void *txq, \ > + struct rte_mbuf **pkts, \ > + uint16_t pkts_n) \ > +{ \ > + return mlx5_tx_burst_tmpl((struct mlx5_txq_data *restrict)txq, \ > + pkts, pkts_n, (olx)); \ > +} > + > +#define MLX5_TXOFF_INFO(func, olx) {mlx5_tx_burst_##func, olx}, > + > static __rte_always_inline uint32_t > rxq_cq_to_pkt_type(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *c= qe); >=20 > @@ -1531,7 +1582,323 @@ > } >=20 > /** > - * Configure the TX function to use. > + * DPDK Tx callback template. This is configured template > + * used to generate routines optimized for specified offload setup. > + * One of this generated functions is chosen at SQ configuration > + * time. > + * > + * @param txq > + * Generic pointer to TX queue structure. > + * @param[in] pkts > + * Packets to transmit. > + * @param pkts_n > + * Number of packets in array. > + * @param olx > + * Configured offloads mask, presents the bits of MLX5_TXOFF_CONFIG_xx= x > + * values. Should be static to take compile time static configuration > + * advantages. > + * > + * @return > + * Number of packets successfully transmitted (<=3D pkts_n). > + */ > +static __rte_always_inline uint16_t > +mlx5_tx_burst_tmpl(struct mlx5_txq_data *restrict txq, > + struct rte_mbuf **restrict pkts, > + uint16_t pkts_n, > + unsigned int olx) > +{ > + (void)txq; > + (void)pkts; > + (void)pkts_n; > + (void)olx; > + return 0; > +} > + > +/* Generate routines with Enhanced Multi-Packet Write support. */ > +MLX5_TXOFF_DECL(full_empw, > + MLX5_TXOFF_CONFIG_FULL | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(none_empw, > + MLX5_TXOFF_CONFIG_NONE | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(md_empw, > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(mt_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(mtsc_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(mti_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(mtv_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(mtiv_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(sc_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(sci_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(scv_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(sciv_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(i_empw, > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(v_empw, > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_DECL(iv_empw, > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +/* Generate routines without Enhanced Multi-Packet Write support. */ > +MLX5_TXOFF_DECL(full, > + MLX5_TXOFF_CONFIG_FULL) > + > +MLX5_TXOFF_DECL(none, > + MLX5_TXOFF_CONFIG_NONE) > + > +MLX5_TXOFF_DECL(md, > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(mt, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(mtsc, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(mti, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA) > + > + > +MLX5_TXOFF_DECL(mtv, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > + > +MLX5_TXOFF_DECL(mtiv, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(sc, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(sci, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA) > + > + > +MLX5_TXOFF_DECL(scv, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > + > +MLX5_TXOFF_DECL(sciv, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(i, > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(v, > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_DECL(iv, > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +/* > + * Array of declared and compiled Tx burst function and corresponding > + * supported offloads set. The array is used to select the Tx burst > + * function for specified offloads set at Tx queue configuration time. > + */ > +const struct { > + eth_tx_burst_t func; > + unsigned int olx; > +} txoff_func[] =3D { > +MLX5_TXOFF_INFO(full_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(none_empw, > + MLX5_TXOFF_CONFIG_NONE | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(md_empw, > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(mt_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(mtsc_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(mti_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(mtv_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(mtiv_empw, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(sc_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(sci_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(scv_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(sciv_empw, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(i_empw, > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(v_empw, > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(iv_empw, > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA | MLX5_TXOFF_CONFIG_EMPW) > + > +MLX5_TXOFF_INFO(full, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(none, > + MLX5_TXOFF_CONFIG_NONE) > + > +MLX5_TXOFF_INFO(md, > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(mt, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(mtsc, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(mti, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA) > + > + > +MLX5_TXOFF_INFO(mtv, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(mtiv, > + MLX5_TXOFF_CONFIG_MULTI | MLX5_TXOFF_CONFIG_TSO | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(sc, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(sci, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(scv, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(sciv, > + MLX5_TXOFF_CONFIG_SWP | MLX5_TXOFF_CONFIG_CSUM | > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(i, > + MLX5_TXOFF_CONFIG_INLINE | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(v, > + MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > + > +MLX5_TXOFF_INFO(iv, > + MLX5_TXOFF_CONFIG_INLINE | MLX5_TXOFF_CONFIG_VLAN | > + MLX5_TXOFF_CONFIG_METADATA) > +}; > + > +/** > + * Configure the Tx function to use. The routine checks configured > + * Tx offloads for the device and selects appropriate Tx burst > + * routine. There are multiple Tx burst routines compiled from > + * the same template in the most optimal way for the dedicated > + * Tx offloads set. > * > * @param dev > * Pointer to private data structure. > @@ -1542,8 +1909,153 @@ > eth_tx_burst_t > mlx5_select_tx_function(struct rte_eth_dev *dev) > { > - (void)dev; > - return removed_tx_burst; > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_dev_config *config =3D &priv->config; > + uint64_t tx_offloads =3D dev->data->dev_conf.txmode.offloads; > + unsigned int diff =3D 0, olx =3D 0, i, m; > + > + static_assert(MLX5_WQE_SIZE_MAX / MLX5_WSEG_SIZE <=3D > + MLX5_DSEG_MAX, "invalid WQE max size"); > + static_assert(MLX5_WQE_CSEG_SIZE =3D=3D MLX5_WSEG_SIZE, > + "invalid WQE Control Segment size"); > + static_assert(MLX5_WQE_ESEG_SIZE =3D=3D MLX5_WSEG_SIZE, > + "invalid WQE Ethernet Segment size"); > + static_assert(MLX5_WQE_DSEG_SIZE =3D=3D MLX5_WSEG_SIZE, > + "invalid WQE Data Segment size"); > + static_assert(MLX5_WQE_SIZE =3D=3D 4 * MLX5_WSEG_SIZE, > + "invalid WQE size"); > + assert(priv); > + if (tx_offloads & DEV_TX_OFFLOAD_MULTI_SEGS) { > + /* We should support Multi-Segment Packets. */ > + olx |=3D MLX5_TXOFF_CONFIG_MULTI; > + } > + if (tx_offloads & (DEV_TX_OFFLOAD_TCP_TSO | > + DEV_TX_OFFLOAD_VXLAN_TNL_TSO | > + DEV_TX_OFFLOAD_GRE_TNL_TSO | > + DEV_TX_OFFLOAD_IP_TNL_TSO | > + DEV_TX_OFFLOAD_UDP_TNL_TSO)) { > + /* We should support TCP Send Offload. */ > + olx |=3D MLX5_TXOFF_CONFIG_TSO; > + } > + if (tx_offloads & (DEV_TX_OFFLOAD_IP_TNL_TSO | > + DEV_TX_OFFLOAD_UDP_TNL_TSO | > + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM)) { > + /* We should support Software Parser for Tunnels. */ > + olx |=3D MLX5_TXOFF_CONFIG_SWP; > + } > + if (tx_offloads & (DEV_TX_OFFLOAD_IPV4_CKSUM | > + DEV_TX_OFFLOAD_UDP_CKSUM | > + DEV_TX_OFFLOAD_TCP_CKSUM | > + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM)) { > + /* We should support IP/TCP/UDP Checksums. */ > + olx |=3D MLX5_TXOFF_CONFIG_CSUM; > + } > + if (tx_offloads & DEV_TX_OFFLOAD_VLAN_INSERT) { > + /* We should support VLAN insertion. */ > + olx |=3D MLX5_TXOFF_CONFIG_VLAN; > + } > + if (priv->txqs_n && (*priv->txqs)[0]) { > + struct mlx5_txq_data *txd =3D (*priv->txqs)[0]; > + > + if (txd->inlen_send) { > + /* > + * Check the data inline requirements. Data inline > + * is enabled on per device basis, we can check > + * the first Tx queue only. > + * > + * If device does not support VLAN insertion in WQE > + * and some queues are requested to perform VLAN > + * insertion offload than inline must be enabled. > + */ > + olx |=3D MLX5_TXOFF_CONFIG_INLINE; > + } > + } > + if (config->mps =3D=3D MLX5_MPW_ENHANCED && > + config->txq_inline_min <=3D 0) { > + /* > + * The NIC supports Enhanced Multi-Packet Write. > + * We do not support legacy MPW due to its > + * hardware related problems, so we just ignore > + * legacy MLX5_MPW settings. There should be no > + * minimal required inline data. > + */ > + olx |=3D MLX5_TXOFF_CONFIG_EMPW; > + } > + if (tx_offloads & DEV_TX_OFFLOAD_MATCH_METADATA) { > + /* We should support Flow metadata. */ > + olx |=3D MLX5_TXOFF_CONFIG_METADATA; > + } > + /* > + * Scan the routines table to find the minimal > + * satisfying routine with requested offloads. > + */ > + m =3D RTE_DIM(txoff_func); > + for (i =3D 0; i < RTE_DIM(txoff_func); i++) { > + unsigned int tmp; > + > + tmp =3D txoff_func[i].olx; > + if (tmp =3D=3D olx) { > + /* Meets requested offloads exactly.*/ > + m =3D i; > + break; > + } > + if ((tmp & olx) !=3D olx) { > + /* Does not meet requested offloads at all. */ > + continue; > + } > + if ((olx ^ tmp) & MLX5_TXOFF_CONFIG_EMPW) > + /* Do not enable eMPW if not configured. */ > + continue; > + if ((olx ^ tmp) & MLX5_TXOFF_CONFIG_INLINE) > + /* Do not enable inlining if not configured. */ > + continue; > + /* > + * Some routine meets the requirements. > + * Check whether it has minimal amount > + * of not requested offloads. > + */ > + tmp =3D __builtin_popcountl(tmp & ~olx); > + if (m >=3D RTE_DIM(txoff_func) || tmp < diff) { > + /* First or better match, save and continue. */ > + m =3D i; > + diff =3D tmp; > + continue; > + } > + if (tmp =3D=3D diff) { > + tmp =3D txoff_func[i].olx ^ txoff_func[m].olx; > + if (__builtin_ffsl(txoff_func[i].olx & ~tmp) < > + __builtin_ffsl(txoff_func[m].olx & ~tmp)) { > + /* Lighter not requested offload. */ > + m =3D i; > + } > + } > + } > + if (m >=3D RTE_DIM(txoff_func)) { > + DRV_LOG(DEBUG, "port %u has no selected Tx function" > + " for requested offloads %04X", > + dev->data->port_id, olx); > + return NULL; > + } > + DRV_LOG(DEBUG, "port %u has selected Tx function" > + " supporting offloads %04X/%04X", > + dev->data->port_id, olx, txoff_func[m].olx); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_MULTI) > + DRV_LOG(DEBUG, "\tMULTI (multi segment)"); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_TSO) > + DRV_LOG(DEBUG, "\tTSO (TCP send offload)"); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_SWP) > + DRV_LOG(DEBUG, "\tSWP (software parser)"); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_CSUM) > + DRV_LOG(DEBUG, "\tCSUM (checksum offload)"); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_INLINE) > + DRV_LOG(DEBUG, "\tINLIN (inline data)"); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_VLAN) > + DRV_LOG(DEBUG, "\tVLANI (VLAN insertion)"); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_METADATA) > + DRV_LOG(DEBUG, "\tMETAD (tx Flow metadata)"); > + if (txoff_func[m].olx & MLX5_TXOFF_CONFIG_EMPW) > + DRV_LOG(DEBUG, "\tEMPW (Enhanced MPW)"); > + return txoff_func[m].func; > } >=20 >=20 > --=20 > 1.8.3.1 >=20