From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B9681A046B for ; Mon, 22 Jul 2019 07:33:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8D18E1B4B6; Mon, 22 Jul 2019 07:33:10 +0200 (CEST) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00076.outbound.protection.outlook.com [40.107.0.76]) by dpdk.org (Postfix) with ESMTP id 7BAD031FC for ; Mon, 22 Jul 2019 07:33:09 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hlcJAW7x0Y9cW9On5difRJf5Q8ekPQGGaAuNKeJ42Wz+oE599xRrM06s4PIBaDkh+0/PMg7rfhgpVSGAcEBxat/VJ1iw/7DyMYljU0+bxAQ+J3JdS3qu65nB4A2QypzacXHef+HNBwM1+r9F41rMyNYOaz5wU5U9JJXz5dEykEAyueWbikqoZWRjsLH9VT9x8rl4lAmAXgFJg7c9JS0A7d/NLmidtuxZ9MpNNG8gXeaBf6IGl0T7fpc1IYezJqoe4Mb7zTm4z6N3y5WPspkLeuRoJPOoPNOKvkLKvQbuHFILy9WR0qjVEjn28muKNkGzSlLs9YH+YvuCgmxIazFMuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wig0IAbyoDYhHgxaJjhTLBwkP5KhTW585Bz3ntIFVWU=; b=YvsXdb9h+tSBVx3WMyt6Xjo7dmyJnxnAgeQR/5LUSV0JGmQU2Yzn2YKP8b9knQ5OfdlsjzQ+78RgbTp/U7kyUr5d5fBETmHFKM9sO9WWhEl903qYrgLPdEu4bzKSr1oDzbAge+kzkz7+ndLLsl6zp9KyqANEtLQIuO6mjR9y2JgJvxGWFIjN/Hqd5zsZVgPkEM0YS0StbqOs+OFCbqCLLLBT3jjWY6Ys8w0jRMtwP7kNLdkWSul6TPy0oRszAqQNhhi/xvCHzSsQf9yfyP+A8vvHONfXFYnwZXFBq/DNgtY6kXmqLOfQBQGN7zNl5Dl64F1LDsnQbedQKMEW/PtOwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=mellanox.com;dmarc=pass action=none header.from=mellanox.com;dkim=pass header.d=mellanox.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wig0IAbyoDYhHgxaJjhTLBwkP5KhTW585Bz3ntIFVWU=; b=BYY1ryjIr1Lp+t0OaoVrKXPBNRPwqb3d+TQ60T1HlCLR2N7RyCN9ooLmOgwVV2K2jVCeUe/m77dBRl6gvAW79GteLaCSmq/4bJRs0rpC8llRdUdBk5P5TS4j5uFwFRVjtvIJPfWuCYjDCKauSsNPSGxG2bhuicugMfzn5IcSbaM= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB3995.eurprd05.prod.outlook.com (52.134.72.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2094.16; Mon, 22 Jul 2019 05:33:08 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::69c1:c0d7:1fa1:f89f]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::69c1:c0d7:1fa1:f89f%6]) with mapi id 15.20.2094.013; Mon, 22 Jul 2019 05:33:08 +0000 From: Yongseok Koh To: Slava Ovsiienko CC: "dev@dpdk.org" Thread-Topic: [PATCH v4 3/8] net/mlx5: update Tx datapath definitions Thread-Index: AQHVP9AuuI3WgdtbMEmsaaQz8g9DaKbWHkGA Date: Mon, 22 Jul 2019 05:33:08 +0000 Message-ID: References: <1563346400-1762-1-git-send-email-viacheslavo@mellanox.com> <1563719100-368-1-git-send-email-viacheslavo@mellanox.com> <1563719100-368-4-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1563719100-368-4-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-originating-ip: [69.181.245.183] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 15468c8a-7986-4879-b11d-08d70e66143f x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DB3PR0502MB3995; x-ms-traffictypediagnostic: DB3PR0502MB3995: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 01068D0A20 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(376002)(346002)(366004)(396003)(136003)(199004)(189003)(478600001)(6486002)(33656002)(53936002)(76176011)(6436002)(316002)(37006003)(15650500001)(36756003)(6512007)(81156014)(81166006)(186003)(6862004)(25786009)(446003)(8676002)(26005)(2616005)(11346002)(476003)(4326008)(229853002)(6636002)(8936002)(71190400001)(71200400001)(486006)(102836004)(7736002)(2906002)(14454004)(6506007)(53546011)(305945005)(6246003)(66946007)(66476007)(66556008)(64756008)(66446008)(66066001)(76116006)(91956017)(14444005)(256004)(6116002)(68736007)(86362001)(99286004)(5660300002)(3846002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB3995; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: o7byUmU9M1Dn+nmI4YlmgAbs2CRLnzW0QpWEFb60n+UtDSpDiwrjUa2QvuUqOd+DCiTSbn8k1p6yuRU3Xr2Smwa7JNuJ7nolkXjjqF7bcqVJxUCsLQhNvMoyCAKCj65Y+Wb8zQxDU7+oLjUMABuPgfhDWstZKPzT+NDu814AMaGhnUW+9WhZNLrdo+/INhnrdHqQbFAdo8Tn/E+FDMjdRaQUn3E+MuD0TPKr1+GoP4+ty9W3tCWW0BLT5CQijOZDsqsE4ammX1KjWWX+8FWKDMFdCoTDQnoENv7W943swIdRk/VsLB0QYayKdMdULg1/xlvsg8tVFc4ehepNDbXreAZHlfUqLd4ePH1TROAo4fqyAk82tRzXpiuB07VF/6cgmJN8e/8iuyv+UGPJ0xNJfUQDRHyu+ereMNwCmQ8gTME= Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 15468c8a-7986-4879-b11d-08d70e66143f X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jul 2019 05:33:08.3856 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: yskoh@mellanox.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB3995 Subject: Re: [dpdk-dev] [PATCH v4 3/8] net/mlx5: update Tx datapath definitions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > On Jul 21, 2019, at 7:24 AM, Viacheslav Ovsiienko wrote: >=20 > This patch updates Tx datapath definitions, mostly hardware related. > The Tx descriptor structures are redefined with required fields, > size definitions are renamed to reflect the meanings in more > appropriate way. This is a preparation step before introducing > the new Tx datapath implementation. >=20 > Signed-off-by: Viacheslav Ovsiienko > --- Acked-by: Yongseok Koh > drivers/net/mlx5/mlx5_defs.h | 2 +- > drivers/net/mlx5/mlx5_prm.h | 164 ++++++++++++++++++++++++++++++++++++++= +---- > 2 files changed, 152 insertions(+), 14 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h > index 6861304..873a595 100644 > --- a/drivers/net/mlx5/mlx5_defs.h > +++ b/drivers/net/mlx5/mlx5_defs.h > @@ -58,7 +58,7 @@ > #define MLX5_MAX_XSTATS 32 >=20 > /* Maximum Packet headers size (L2+L3+L4) for TSO. */ > -#define MLX5_MAX_TSO_HEADER 192 > +#define MLX5_MAX_TSO_HEADER (128u + 34u) >=20 > /* Threshold of buffer replenishment for vectorized Rx. */ > #define MLX5_VPMD_RXQ_RPLNSH_THRESH(n) \ > diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h > index dfd9317..97abdb2 100644 > --- a/drivers/net/mlx5/mlx5_prm.h > +++ b/drivers/net/mlx5/mlx5_prm.h > @@ -39,14 +39,85 @@ > /* Invalidate a CQE. */ > #define MLX5_CQE_INVALIDATE (MLX5_CQE_INVALID << 4) >=20 > -/* WQE DWORD size */ > -#define MLX5_WQE_DWORD_SIZE 16 > - > -/* WQE size */ > -#define MLX5_WQE_SIZE (4 * MLX5_WQE_DWORD_SIZE) > +/* WQE Segment sizes in bytes. */ > +#define MLX5_WSEG_SIZE 16u > +#define MLX5_WQE_CSEG_SIZE sizeof(struct mlx5_wqe_cseg) > +#define MLX5_WQE_DSEG_SIZE sizeof(struct mlx5_wqe_dseg) > +#define MLX5_WQE_ESEG_SIZE sizeof(struct mlx5_wqe_eseg) > + > +/* WQE/WQEBB size in bytes. */ > +#define MLX5_WQE_SIZE sizeof(struct mlx5_wqe) > + > +/* > + * Max size of a WQE session. > + * Absolute maximum size is 63 (MLX5_DSEG_MAX) segments, > + * the WQE size field in Control Segment is 6 bits wide. > + */ > +#define MLX5_WQE_SIZE_MAX (60 * MLX5_WSEG_SIZE) > + > +/* > + * Default minimum number of Tx queues for inlining packets. > + * If there are less queues as specified we assume we have > + * no enough CPU resources (cycles) to perform inlining, > + * the PCIe throughput is not supposed as bottleneck and > + * inlining is disabled. > + */ > +#define MLX5_INLINE_MAX_TXQS 8u > +#define MLX5_INLINE_MAX_TXQS_BLUEFIELD 16u > + > +/* > + * Default packet length threshold to be inlined with > + * enhanced MPW. If packet length exceeds the threshold > + * the data are not inlined. Should be aligned in WQEBB > + * boundary with accounting the title Control and Ethernet > + * segments. > + */ > +#define MLX5_EMPW_DEF_INLINE_LEN (3U * MLX5_WQE_SIZE + \ > + MLX5_DSEG_MIN_INLINE_SIZE - \ > + MLX5_WQE_DSEG_SIZE) > +/* > + * Maximal inline data length sent with enhanced MPW. > + * Is based on maximal WQE size. > + */ > +#define MLX5_EMPW_MAX_INLINE_LEN (MLX5_WQE_SIZE_MAX - \ > + MLX5_WQE_CSEG_SIZE - \ > + MLX5_WQE_ESEG_SIZE - \ > + MLX5_WQE_DSEG_SIZE + \ > + MLX5_DSEG_MIN_INLINE_SIZE) > +/* > + * Minimal amount of packets to be sent with EMPW. > + * This limits the minimal required size of sent EMPW. > + * If there are no enough resources to built minimal > + * EMPW the sending loop exits. > + */ > +#define MLX5_EMPW_MIN_PACKETS (2 + 3 * 4) > +#define MLX5_EMPW_MAX_PACKETS ((MLX5_WQE_SIZE_MAX - \ > + MLX5_WQE_CSEG_SIZE - \ > + MLX5_WQE_ESEG_SIZE) / \ > + MLX5_WSEG_SIZE) > +/* > + * Default packet length threshold to be inlined with > + * ordinary SEND. Inlining saves the MR key search > + * and extra PCIe data fetch transaction, but eats the > + * CPU cycles. > + */ > +#define MLX5_SEND_DEF_INLINE_LEN (5U * MLX5_WQE_SIZE + \ > + MLX5_ESEG_MIN_INLINE_SIZE - \ > + MLX5_WQE_CSEG_SIZE - \ > + MLX5_WQE_ESEG_SIZE - \ > + MLX5_WQE_DSEG_SIZE) > +/* > + * Maximal inline data length sent with ordinary SEND. > + * Is based on maximal WQE size. > + */ > +#define MLX5_SEND_MAX_INLINE_LEN (MLX5_WQE_SIZE_MAX - \ > + MLX5_WQE_CSEG_SIZE - \ > + MLX5_WQE_ESEG_SIZE - \ > + MLX5_WQE_DSEG_SIZE + \ > + MLX5_ESEG_MIN_INLINE_SIZE) >=20 > -#define MLX5_OPC_MOD_ENHANCED_MPSW 0 > -#define MLX5_OPCODE_ENHANCED_MPSW 0x29 > +/* Missed in mlv5dv.h, should define here. */ > +#define MLX5_OPCODE_ENHANCED_MPSW 0x29u >=20 > /* CQE value to inform that VLAN is stripped. */ > #define MLX5_CQE_VLAN_STRIPPED (1u << 0) > @@ -114,6 +185,12 @@ > /* Inner L3 type is IPV6. */ > #define MLX5_ETH_WQE_L3_INNER_IPV6 (1u << 0) >=20 > +/* VLAN insertion flag. */ > +#define MLX5_ETH_WQE_VLAN_INSERT (1u << 31) > + > +/* Data inline segment flag. */ > +#define MLX5_ETH_WQE_DATA_INLINE (1u << 31) > + > /* Is flow mark valid. */ > #if RTE_BYTE_ORDER =3D=3D RTE_LITTLE_ENDIAN > #define MLX5_FLOW_MARK_IS_VALID(val) ((val) & 0xffffff00) > @@ -130,12 +207,21 @@ > /* Default mark value used when none is provided. */ > #define MLX5_FLOW_MARK_DEFAULT 0xffffff >=20 > -/* Maximum number of DS in WQE. */ > +/* Maximum number of DS in WQE. Limited by 6-bit field. */ > #define MLX5_DSEG_MAX 63 >=20 > /* The completion mode offset in the WQE control segment line 2. */ > #define MLX5_COMP_MODE_OFFSET 2 >=20 > +/* Amount of data bytes in minimal inline data segment. */ > +#define MLX5_DSEG_MIN_INLINE_SIZE 12u > + > +/* Amount of data bytes in minimal inline eth segment. */ > +#define MLX5_ESEG_MIN_INLINE_SIZE 18u > + > +/* Amount of data bytes after eth data segment. */ > +#define MLX5_ESEG_EXTRA_DATA_SIZE 32u > + > /* Completion mode. */ > enum mlx5_completion_mode { > MLX5_COMP_ONLY_ERR =3D 0x0, > @@ -144,11 +230,6 @@ enum mlx5_completion_mode { > MLX5_COMP_CQE_AND_EQE =3D 0x3, > }; >=20 > -/* Small common part of the WQE. */ > -struct mlx5_wqe { > - uint32_t ctrl[4]; > -}; > - > /* MPW mode. */ > enum mlx5_mpw_mode { > MLX5_MPW_DISABLED, > @@ -156,6 +237,63 @@ enum mlx5_mpw_mode { > MLX5_MPW_ENHANCED, /* Enhanced Multi-Packet Send WQE, a.k.a MPWv2. */ > }; >=20 > +/* WQE Control segment. */ > +struct mlx5_wqe_cseg { > + uint32_t opcode; > + uint32_t sq_ds; > + uint32_t flags; > + uint32_t misc; > +} __rte_packed __rte_aligned(MLX5_WSEG_SIZE); > + > +/* Header of data segment. Minimal size Data Segment */ > +struct mlx5_wqe_dseg { > + uint32_t bcount; > + union { > + uint8_t inline_data[MLX5_DSEG_MIN_INLINE_SIZE]; > + struct { > + uint32_t lkey; > + uint64_t pbuf; > + } __rte_packed; > + }; > +} __rte_packed; > + > +/* Subset of struct WQE Ethernet Segment. */ > +struct mlx5_wqe_eseg { > + union { > + struct { > + uint32_t swp_offs; > + uint8_t cs_flags; > + uint8_t swp_flags; > + uint16_t mss; > + uint32_t metadata; > + uint16_t inline_hdr_sz; > + union { > + uint16_t inline_data; > + uint16_t vlan_tag; > + }; > + } __rte_packed; > + struct { > + uint32_t offsets; > + uint32_t flags; > + uint32_t flow_metadata; > + uint32_t inline_hdr; > + } __rte_packed; > + }; > +} __rte_packed; > + > +/* The title WQEBB, header of WQE. */ > +struct mlx5_wqe { > + union { > + struct mlx5_wqe_cseg cseg; > + uint32_t ctrl[4]; > + }; > + struct mlx5_wqe_eseg eseg; > + union { > + struct mlx5_wqe_dseg dseg[2]; > + uint8_t data[MLX5_ESEG_EXTRA_DATA_SIZE]; > + }; > +} __rte_packed; > + > /* WQE for Multi-Packet RQ. */ > struct mlx5_wqe_mprq { > struct mlx5_wqe_srq_next_seg next_seg; > --=20 > 1.8.3.1 >=20