From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10054.outbound.protection.outlook.com [40.107.1.54]) by dpdk.org (Postfix) with ESMTP id 25334592B; Mon, 11 Sep 2017 14:51:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=fLnyEMrr9fF82OMrnpMfW0QkCjiEAcY8oUDi7YKiyOY=; b=wVHeDEpPvzKMFL8Ma2w0i7Dqboe7xAvqDgsf3YQZMlWWM+G/RMnutS0KfBL7z4MRN6/QdvMNiKPBirsdnC8BYTMP4nag6JZ8PaEOg89LdZ2ZVs61xMguCSc7m0aHghpfbIM6ArkpRpxOXEvaMw0jVrbXQsawHt0Mk8NkSXwizKo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; Received: from mellanox.com (82.166.227.17) by DB6PR05MB3143.eurprd05.prod.outlook.com (2603:10a6:6:1a::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.35.12; Mon, 11 Sep 2017 12:51:06 +0000 From: Shahaf Shuler To: nelio.laranjeiro@6wind.com Cc: dev@dpdk.org, stable@dpdk.org Date: Mon, 11 Sep 2017 15:50:58 +0300 Message-Id: X-Mailer: git-send-email 2.12.0 In-Reply-To: <1e7320c4ee4162c9e509b7560be215077f43ca87.1504076528.git.shahafs@mellanox.com> References: <1e7320c4ee4162c9e509b7560be215077f43ca87.1504076528.git.shahafs@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [82.166.227.17] X-ClientProxiedBy: AM5PR0602CA0015.eurprd06.prod.outlook.com (2603:10a6:203:a3::25) To DB6PR05MB3143.eurprd05.prod.outlook.com (2603:10a6:6:1a::17) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dc4702bc-9346-49c1-eb4e-08d4f913c537 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(48565401081)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:DB6PR05MB3143; X-Microsoft-Exchange-Diagnostics: 1; DB6PR05MB3143; 3:jjTJVWPNzPW5/KfxvIgjj+0JOeT+RDLtevej1SK4aaiFo+basRShO0aEAhVHpahfkeQyHUrTCvB3t7XkUkwpjHNOKS/8zYKbh2VZiY2t1QeHij0GU6q8bXmdFyo09xIUgKInOvqv2DMRM7Ko0SDJFdaEgx3tI1z27VuSTnvKlvVszGfUd+dNFZI95lvRhv8pvOxnmi/oMPSLbLk24S8IQy7C+q5Y9P8xkz+CMU6mfoF6SFr80TDfrnT26DJoh5a6; 25:MtkEPmUgzHQV3da54UZw50bYUr/SG2FiXdMJLNmcLHAHHaiunesnMJ/OT/ysq1ZrNdeLeA+/GwSRWIRelQ31eQGiiWkjmcNohtoRj6gsoC3/DNDSs+RnKKSmjXIBQa4/GvnJejpprCh4tcTbGeV3RDID70YsFmvQuE+qrDvjpG8i7DJ+94MoUCPQxAYyYn45qNpEEX7Ch8a6xNO91fiFmFB7RL+M2RUBzB8Ej5IIE65g2gn08oFzlM+qieWp1f3xl2qOkUr6ld7JJs1no6yw/zScnmgUYaOxj4p738zQvJlxlOaf0scr8fOXAOrxX6+pp0vjkeG98guxTyPmFbUS0Q==; 31:l+Vi9gFjr9RgV+yEOZIxlz0/RS6JYSf0TGqZfe9TALWmxWoPlsGZGT3/XRNEiTvqVtkmUl1ccPU2/1dnT0ZyfIjSrrpyJYvU15DwP4yjMD8o3sZKzR6pMctXLLt6eD/SD7EWOqVLRTc4bfFuVYGaebe+6n7S26AtgSEG4zxGin7rPCdLXKlNeI2K617a4fxU3clbAL0bz+Hr5c52YedDihY9BDmnd6PBMXY7HeOzVlI= X-MS-TrafficTypeDiagnostic: DB6PR05MB3143: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; DB6PR05MB3143; 20:VsxxkJ5Z6991tKjhufanKdajSj0QZAeLVAIEgn/Q1wpCEDv1VUaEQFD9xKNuFiHjsdjYKqpZACYn9bp90WZUbJj+4YiUB70j7EroWRxFIkCrpnmRUW0UFrXA0m2h8ZnJTBPx5+YdNhT3K+0ehYqB0J5T7w4BrPUy71wRmUq1pSX4mKggrhrsfgqPjNNkXO0GZTEkk5GMjijMbaBt7waWTiuabG384BXrFao42nfFLnKoUxWwzi4u1iSXKKHnIARPxoTvyR5jVfv5/aw3/g/fUaVwGJYK+/w6J9Wwmx116FgdwpkWWv54nzbpDiRjqWl3CuMLgudZCazBIHEO30rzvQucwQTyQgRA+qFFnikZM4wUIoB/evyf1jmpHdZg2ezHOCvYOAO7/9eHYKpuLWFZA3+A50db9fAE535ZCtf7LQh810ztug0P1neXkVKUz4R5gimbC+yG94CYhTGTlBvs4ol7CGLsgInbp54s62ol5AT3icRC1xSDE3tC+A8pd0Wj; 4:DamBNsuqCF+M/PO925ocmcJ4OZTnmM9qtaa8KYnCF48TxiqPMUsK2Myp/jEaV9FBhfInLfTGX+CVNhxZqhLp3zswvhtsmOkpHQ+TsPxqc9UhYIpusN4pAE7/G7XIRtxn8lmui993+GcaW6Rj2DCMwrwU//FBC5vkUmwpd1Hvm4u2ToCh2Tn1/2vnLziBYkYdnJv1D+AyqsmPRAQBpIW3sveZ6FHYdCCcFFM497A6Yicere/PT4RrJuuzFx5V5bIb X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(100000703101)(100105400095)(10201501046)(93006095)(93001095)(6055026)(6041248)(20161123558100)(20161123555025)(20161123560025)(20161123564025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DB6PR05MB3143; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DB6PR05MB3143; X-Forefront-PRVS: 04270EF89C X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(7370300001)(6009001)(39860400002)(189002)(199003)(101416001)(5660300001)(3846002)(6116002)(25786009)(8676002)(110136004)(86362001)(575784001)(4326008)(2906002)(33026002)(36756003)(50226002)(42186005)(81156014)(305945005)(7736002)(68736007)(118296001)(33646002)(69596002)(6916009)(2950100002)(66066001)(21086003)(7350300001)(6666003)(47776003)(2361001)(81166006)(105586002)(106356001)(478600001)(53936002)(2351001)(50466002)(5003940100001)(97736004)(48376002)(189998001)(50986999)(76176999)(55016002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR05MB3143; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DB6PR05MB3143; 23:ADLMTW8N2cgKc0rzOJExW+jQo/zpGIWJjx2BGOaoK?= =?us-ascii?Q?7lpkdf+aXPLe2tgrUoEE3qigFER8aJtCHxex9M+EMUCpcMHfGN39dVpngx/L?= =?us-ascii?Q?A3dOe/tzaK7EO1hWf2GKN8eo0s8OeZrYrR9qXAxC4HjxA8RYccssnHOmIv5+?= =?us-ascii?Q?0utnZdUvDkq82kBbMjtQ+vH+OhOC04MyISCC3dYON7vMNKJkbZSUqvP6yWIx?= =?us-ascii?Q?cszSOB6WjwvwHRCX7MGMlMdmCysns4h+H44DtxqZM659s7/yL052cIdsRD4p?= =?us-ascii?Q?Pq6QnBNXswGCzABokNmvVUI6rn/98dMOQ4DEcqkd89I07KOIqwtbvYW0vSl5?= =?us-ascii?Q?ZsOqqgtJ1maDI9qJxmiw/6XPwjfBOZOCFPb6oarQ1W6cl3FpqkdfeX+n3+uR?= =?us-ascii?Q?aXgcGwQZQnLOAm8dT7SBS45kJoF7DMaWAlj5FQgkzsWmv6JQyzx6vfniBzpr?= =?us-ascii?Q?MhRRi6Yl3Twmve2FyZhnmh5W8GzBeugme5BgOJD/5PQtnc9oWqddHJwjkT4u?= =?us-ascii?Q?s4F3akFVrUUbXcTUl6tQ6h9BCWAKs8dIBhBYBXr1QVTIL1unsssRGQEOtNlF?= =?us-ascii?Q?CehL/inVC84Nite+bJxCy+Ih7o/CduQ0Uaoa1VVX76ebFXsmAj9VDZRWKePf?= =?us-ascii?Q?CU1AwGHehnnvQQgJ0nY3cAHl5MbxVcQjwZSaVVDQNvOd/IsHq1T/J3fm0yRX?= =?us-ascii?Q?CGqo3uVWbqSV2iW8uRzNvs7cVI91nq2dT8p4w2b4KdSK3ZNJ/c2RYVRClqZn?= =?us-ascii?Q?qpNiTdokTzw50mz5XwI0IcUgpwHGQBdx7o2Jg57A0kd8Lm472V5ZlT8b01Xs?= =?us-ascii?Q?+fhupKNspR2+/42E5M+uwWbCD6xZF1Z1/L0aUyfoEFI/Q2dpIQM54wWOU3Pu?= =?us-ascii?Q?3b6/t9pEY/Dh7ALB2MavwpjRNuwbfq0As/yzfnnkIUyLN94x4sNrntgyZN7Q?= =?us-ascii?Q?+XGhUSMxjA1/RRM6c7XUg7F2/ojVSNpURlfgHpjF9SSvsSCeyGn9WTtvMVec?= =?us-ascii?Q?EfAjH/4A4GDckQckhCnrJtrm0oTrexwoaP77agSfCUeTYSKxKJ/pOL/5roub?= =?us-ascii?Q?B/45aKnEWWXGU5fNVLZZs7aupslGfjcmhaMZblDBT2f5Vx6z0cSEcUuhRJTi?= =?us-ascii?Q?39FcV3tDNMfI/AfS+YzDciORIABixBb+6o6Px9FEA0mwjGfVTP5ribffmPOP?= =?us-ascii?Q?kih/nfSYmxN/TI+t/zPuDk4L8kimWBBegWHqiYXeDoy96AMOkKU1o43kg=3D?= =?us-ascii?Q?=3D?= X-Microsoft-Exchange-Diagnostics: 1; DB6PR05MB3143; 6:Ix32gOwf0g03UAjG0QrYqB+t2cR+OBzkxjqWzY2SA24zNgroeo5slnJNLWR1XxM2g0v1dFr9VqvKHw8EPlqnpn35Tb+5I6PbpxuFsduQT2HsT8GPciTB20o/ITMzqH9o5ESm27FsoAuavv3+mdUFXnqZwg8NMsMH3z7sO0MIa1kejpexJuCjSxX0Z7S2Ce0lRjHkxWCzn7EuWg+ozfoRtG04RWF7c3/dYVXdI189jvEWn1/QgyIObUtaQiEZT9ElRt6/8TIOwXFaRZELx3dISPFAVLEw8jprJdqsegHUoehOvaaRQ9XIaWtCu9flFZqokoZ0FSpwfW4DH7ITXiG4xg==; 5:8CH7GP/6m3YHItaIRZMSHURbmMlxplKAydy547JkJZsW9THnVzASFzeVlogDPOt4yo0tRsHZmPcyyA7C1vSa++Zq0rsQ9jTn1owcEWI8fUyO9JphUBM+GbhRKDaw4yRizWYD5WNOUO6i3fcFMPq/YQ==; 24:y7BekRs/5DijZbwt5BRb31byPnUQM0ASWXKjXIhh8574LPl544YTYjZlMkxIQvdJQqfvQVRWI9HZ5d+2pD+ccmLF03uwSvEEgbQfiqrpn48=; 7:EmSip37jExmT3e8H/QpBOvZxDcdLBW5j5C0wzwsX84TpOs935WHouq4Y20qSdJlIhC2WZcCx69O/XF4WwEBvqMiZeEgeta90KzTM6cwKtNfpltA1Km9zsVXm501Mk+0Lgr7emfX/0nrWQUe/uSoXDKEiKfwSmrrNzNOSYia9cG0IuAFcfnN1HxAB179QKBZ+QdCgznnEDIGcV0vuICUMjrdJV9QHMeUU9YKi8MwJCi4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Sep 2017 12:51:06.9866 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR05MB3143 Subject: [dpdk-dev] [PATCH v3 3/3] net/mlx5: enforce Tx num of segments limitation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Sep 2017 12:51:09 -0000 Mellanox NICs has a limitation on the number of mbuf segments a multi segment mbuf can have. The max number depends on the Tx offloads requested. The current code not enforce such limitation, which might cause malformed work requests to be written to the device. This commit adds verification for the number of mbuf segments posted to the device. In case of overflow the packet will not be sent. In addition update the nic documentation with the limitation. Considering device limitation is 63 data segments in a work request, the maximum number of segment in mbuf was calculated taking TSO as the worst case: max_nb_segs = 63 - (control_segment + ethernet segment + TSO headers inline + inline segment + extra inline to align to cacheline) Cc: stable@dpdk.org Signed-off-by: Shahaf Shuler --- on v3: - remove debug prints from datapath. on v2: - remove parenthesis around MLX5_MAX_DS. - add limitation to nic guide. - update commit message. - fix typo. --- doc/guides/nics/mlx5.rst | 2 ++ drivers/net/mlx5/mlx5_defs.h | 3 ++- drivers/net/mlx5/mlx5_prm.h | 3 +++ drivers/net/mlx5/mlx5_rxtx.c | 4 ++++ drivers/net/mlx5/mlx5_rxtx_vec_sse.c | 5 +++++ drivers/net/mlx5/mlx5_txq.c | 27 +++++++++++++++++++++++++++ 6 files changed, 43 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index f4cb18bca..d8244de97 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -124,6 +124,8 @@ Limitations Will match any ipv4 packet (VLAN included). +- A multi segment mbuf must have less than 50 segments. That means mbuf->nb_segs < 50. + Configuration ------------- diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index a76bc6f65..3de0e5d81 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -100,7 +100,8 @@ /* * Maximum size of burst for vectorized Tx. This is related to the maximum size - * of Enhaned MPW (eMPW) WQE as vectorized Tx is supported with eMPW. + * of Enhanced MPW (eMPW) WQE as vectorized Tx is supported with eMPW. + * Careful when changing, large value can cause wqe DS to overlap. */ #define MLX5_VPMD_TX_MAX_BURST 32U diff --git a/drivers/net/mlx5/mlx5_prm.h b/drivers/net/mlx5/mlx5_prm.h index 608072f7e..bc2b72333 100644 --- a/drivers/net/mlx5/mlx5_prm.h +++ b/drivers/net/mlx5/mlx5_prm.h @@ -154,6 +154,9 @@ /* Default mark value used when none is provided. */ #define MLX5_FLOW_MARK_DEFAULT 0xffffff +/* Maximum number of DS in WQE. */ +#define MLX5_MAX_DS 63 + /* Subset of struct mlx5_wqe_eth_seg. */ struct mlx5_wqe_eth_seg_small { uint32_t rsvd0; diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 7567f2329..fdd7067da 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -661,6 +661,10 @@ mlx5_tx_burst(void *dpdk_txq, struct rte_mbuf **pkts, uint16_t pkts_n) else j += sg; next_pkt: + if (ds > MLX5_MAX_DS) { + txq->stats.oerrors++; + break; + } ++elts_head; ++pkts; ++i; diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.c b/drivers/net/mlx5/mlx5_rxtx_vec_sse.c index f89762ff8..3583e6780 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.c +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.c @@ -248,6 +248,10 @@ txq_scatter_v(struct txq *txq, struct rte_mbuf **pkts, uint16_t pkts_n) if (segs_n == 1 || max_elts < segs_n || max_wqe < 2) break; + if (segs_n > MLX5_MPW_DSEG_MAX) { + txq->stats.oerrors++; + break; + } wqe = &((volatile struct mlx5_wqe64 *) txq->wqes)[wqe_ci & wq_mask].hdr; if (buf->ol_flags & @@ -365,6 +369,7 @@ txq_burst_v(struct txq *txq, struct rte_mbuf **pkts, uint16_t pkts_n, max_elts = (elts_n - (elts_head - txq->elts_tail)); max_wqe = (1u << txq->wqe_n) - (txq->wqe_ci - txq->wqe_pi); pkts_n = RTE_MIN((unsigned int)RTE_MIN(pkts_n, max_wqe), max_elts); + assert(pkts_n <= MLX5_MAX_DS - nb_dword_in_hdr); if (unlikely(!pkts_n)) return 0; elts = &(*txq->elts)[elts_head & elts_m]; diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 4b0b532b1..091b1a93d 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -288,6 +288,8 @@ txq_ctrl_setup(struct rte_eth_dev *dev, struct txq_ctrl *txq_ctrl, .comp_mask = IBV_EXP_QP_INIT_ATTR_PD, }; if (priv->txq_inline && (priv->txqs_n >= priv->txqs_inline)) { + unsigned int ds_cnt; + tmpl.txq.max_inline = ((priv->txq_inline + (RTE_CACHE_LINE_SIZE - 1)) / RTE_CACHE_LINE_SIZE); @@ -320,6 +322,31 @@ txq_ctrl_setup(struct rte_eth_dev *dev, struct txq_ctrl *txq_ctrl, attr.init.cap.max_inline_data = tmpl.txq.max_inline * RTE_CACHE_LINE_SIZE; } + /* + * Check if the inline size is too large in a way which + * can make the wqe DS to overflow. + * Considering in calculation: + * WQE CTRL (1 DS) + * WQE ETH (1 DS) + * inline part (N DS) + */ + ds_cnt = 2 + + (attr.init.cap.max_inline_data / MLX5_WQE_DWORD_SIZE); + if (ds_cnt > MLX5_MAX_DS) { + unsigned int max_inline = (MLX5_MAX_DS - 2) * + MLX5_WQE_DWORD_SIZE; + + /* Ceil down*/ + max_inline = max_inline - (max_inline % + RTE_CACHE_LINE_SIZE); + WARN("txq inline is too large (%d) setting it to " + "the maximum possible: %d\n", + priv->txq_inline, max_inline); + tmpl.txq.max_inline = max_inline / RTE_CACHE_LINE_SIZE; + attr.init.cap.max_inline_data = max_inline; + if (priv->mps == MLX5_MPW_ENHANCED) + tmpl.txq.inline_max_packet_sz = max_inline; + } } if (priv->tso) { attr.init.max_tso_header = -- 2.12.0