From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0079.outbound.protection.outlook.com [104.47.1.79]) by dpdk.org (Postfix) with ESMTP id E0C34AAC0 for ; Tue, 5 Jun 2018 02:12:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LjQmGk0Zjd5sZQzh9RnClpptnhcvFk3fVUMi7B53OG8=; b=ZrITp/U7ISO32ZxWh5PBVOm2xV608snCoH3Z5ga7rRL/7c5S5c5ADBX8u9O+MXjjJGW/UQGTigmmg3zVdtdWSNJVXYkTTdYrqnUBNsg//K9HBsgfQ3dswbU4rW4JuHFwU3Nava7O5rciRls4gBz92Twhd1STN8g0nfCKmEhDz/I= Received: from mellanox.com (209.116.155.178) by AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.11; Tue, 5 Jun 2018 00:11:56 +0000 From: Yongseok Koh To: yliu@fridaylinux.org Cc: stable@dpdk.org, shahafs@mellanox.com, adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com, Yongseok Koh Date: Mon, 4 Jun 2018 17:10:25 -0700 Message-Id: <20180605001129.13184-4-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180605001129.13184-1-yskoh@mellanox.com> References: <20180605001129.13184-1-yskoh@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: CO1PR15CA0060.namprd15.prod.outlook.com (2603:10b6:101:1f::28) To AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:AM5PR0501MB2036; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 3:oQPNWtoBWBul9c7HsTAcxp1Z/qVBLn6rj3RHFgsrMZD4ZkcdIHHho8E0Zs17HT9AU7oyLyLgqaYQss53p+I/L4Xkg9kJTCBZbn5FXBt8n0i0raD35gNNUe69xD/SWjVBjDwia/YMScEu7nPiaB/tTxfMCBdK6wb4fp1BmK+/3TcDD1clV+BceRanUdOpka+cAqO+fQTpUij2e736895c5ysrGuRDG+pfE8lHZF+RmR+hydwB9nkxhoNTXXWDfT/U; 25:TVp5pSNym2a0vd77JYi4+Lyqty43lIhUsqpTrLucXVZlkmFABz9NxK5yekRsU0Rxil161mHERHazCvvC9pWBDyLMda7E8VbGbguTJf6hc/PsC6wo5/naZlkLIjecvaBQThuyPBgu8anlyo20zofZyoe63y8CEGYqYgpb6wLqltDkhVj5m65VBy7An8fCL3Qn6Jc29SCt73k5S0UoZFAChd9lXNniGreXZBcvaAzEG8gwysAZOun9wvAtjj8I+3GmnBvcQdIEKN395OzI0jIt979ebeZ9r5KWVNksAqodJ2hs7Xnae8GFSuy52a6jf8krSgpVyC4gzw3XWg3+ZeO/WA==; 31:FpODK4KRKvKT6LfqiXyYoi73BHtZDgzxCKhG8N5U1id4DTfebrymllIGdxt+O8kYvtZhLa+MrFrGiKZNgPwbLX+uUJmIykwRZ9jRNTwS6vbOKwE7lnLVROl9kJuYVi07AXtuL54KaAyqxDNlVd7H4tXN4sEUmyD/gyht0xBdZgpdnwJAxVBD9Du/nn9wKsbLWLi2ZPdWwctDwEc3VqTmimVALzjAJ7uv5gC5QMZPMDI= X-MS-TrafficTypeDiagnostic: AM5PR0501MB2036: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 20:W3DTbwQTk2mWM+fGnQQj3Zjgju2Euu615C6aivAVNr+5IbIEByVXj00cguBB2vE90nxaqBnIlIj1+uDq4TuKk0oiEPhkRZokCZ8WW6PwMLwcuxEuHucRZK8Cb2IdoatR4F1/CIGUstoI0XDO76cCXpTu5yqjr3ubRojjHOc4K8UHez5zF/ni/ks8RONHjsWMArCkR2pWWrTBhlbHd2o04hES18tT7zJFbcD79bMJHCDigAKzbsjLUrebYzzoha3Z5qvhCyDrRCoS9CwAuhd631LSPl0TeyBMHnBe7U3IJRONkrUPt7Ye5YERgAzXe0mfKpXRf8lwLR0TWs8aUBKq0XCFU46k1KiyDApu2U6Yn+A3z1HkE6oX9MMvGnBLUYFLxB30B/p+7HbrBzjcXtrOEO1cO4v7EKgky2Md6nvE8FUC6TwnTDPDIrcMKFN/vEAA1yb3UcSTyOidwr9y+dLEw0O7N9OVOUn3KdDTEgOKFsM2b2UwbftraO0tzYPyn8F6; 4:1veJG3isiQwsbtJKetb7pxDBhg7wwuxkDpQk64x6Nep2nApKtN5LeHmSBSzKWyAtCV4HLqExjlpIYap/+CAfUn6pE229SPehwFR7S7Tr5JHD4vgvpPu/wM4iStvJrd71v4EJ8oab+1fkkpBcazmQK97SiZg271IJEEs1gjDjNPia4GHkWpqP/IGBRRoYpRCBkj1mQKA+whWCx2Aiked5E4wHQl2o1IEEqiqQiTW/QN8LcnOsMteRIA9HKPbh/vtfxIsfFWXQRdHZpaOjKPryLA== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3231254)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(20161123560045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:AM5PR0501MB2036; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0501MB2036; X-Forefront-PRVS: 0694C54398 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39380400002)(366004)(396003)(39860400002)(346002)(376002)(189003)(199004)(478600001)(81156014)(76176011)(51416003)(7696005)(956004)(2361001)(50226002)(305945005)(2616005)(50466002)(47776003)(486006)(97736004)(5660300001)(6666003)(2351001)(106356001)(8936002)(186003)(81166006)(52116002)(6116002)(476003)(6916009)(386003)(446003)(11346002)(59450400001)(66066001)(26005)(3846002)(8676002)(16526019)(1076002)(25786009)(21086003)(86362001)(53936002)(107886003)(4326008)(2906002)(316002)(55016002)(36756003)(48376002)(105586002)(69596002)(68736007)(7736002)(16586007); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2036; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0501MB2036; 23:UxzaEqA8wXq7LeL4OCBm5vT7h80Av1OYN+KZULO?= =?us-ascii?Q?cJ30j/z3Rp7jRrsmeP6z6ECEPkvcJwrSCXdK67QFwzpoLQIoSk963/BBKPmD?= =?us-ascii?Q?X0p8HzWcbJGblqxthFaR4EkITCxoPTPuhTOltgyVC9e/j0WPEbK1gmNP4Ix3?= =?us-ascii?Q?ijZMULlapt/rz4fcLkgGMNH/0FzIwfYHFlIejIdl8tBWQn3f5Thhthk8nRYy?= =?us-ascii?Q?QM6X60sfm2XZK4nEifEqB39mJrIVj8lEUyByRMm/Jy98sx7417QIjyqpqecz?= =?us-ascii?Q?4d+FbimB3Ote1ecKgahwti1yPtfrbmVxwMv+6499wnOgaWFjTpSmqM0HmrZL?= =?us-ascii?Q?vIswhWPZLP6Nvh4njn6aJ7ioOgZRu6aXY8+6bnMKdSlrYEcjQ6PAPOZXKAkJ?= =?us-ascii?Q?Pmk4g4SXBdt3Xbo3h8X7NBNNhB3m32vCpA7saVHrSBoYU9h5mYTYBzvQvjmC?= =?us-ascii?Q?O6LA6HwWexNIpkWjUQamROFX7Xzp6ArjeP0WwZYKlGijPCT28/apQDjoP24+?= =?us-ascii?Q?ppnITuA3000U0GuFj/KYTTTBwjZp8RLBxvFBu2WNkehfzFx+0xw3tN2w0igP?= =?us-ascii?Q?/rdSEeSfoORa6qmU7NrTJDzdDrZWO/NsEaCS/UbEGsjPxrzw1PAOsecQGuKp?= =?us-ascii?Q?GjNCvz9r5TCBAnkFOYXQEmXXgk9Stav5oyJxy1fnDUzsPw8djpALfGwSo36B?= =?us-ascii?Q?sqtJekQCsxNccEWotlGl1pl2lQmYuk2HeWC5TMOzB92QgWMUrz4akey6tH0s?= =?us-ascii?Q?cvs5Upf8KYfYhbXu8BPZFpkEjHWEKrkGFVGSrnICyE+66bKdEP2FDXpME+HJ?= =?us-ascii?Q?aLDrr9piF8nwNPLRTkJ+hN61mJPSky215D6s7axtwD5ixJTxr7BXRxjkEO0N?= =?us-ascii?Q?td4hwueCPgDfmeUXMwH/VrRyOk0RL5S4DyfUv8Zvnfm4AB0Cxu6Z3+F+txQP?= =?us-ascii?Q?vGk0oxPV6tCSBYfEm7WJNOzwk+Gq/TYi4sC9Vo/ybtT7seLen92n9WMtyIEW?= =?us-ascii?Q?l18VUiWGnvVpfCsTsJO5zhE4ZJd4YhnbS1sZbQEwnSr+/i9bqD4pHPtnJfi/?= =?us-ascii?Q?WfAzr/s8q0grXDn4QHxzjFQr4F3cNZOeCzUKDFXe/NFcjNUf6pQ0pi5N+cgQ?= =?us-ascii?Q?yaE6qG6Z0QI2lqtiELb6LEMP2MloEHylqb2w7ypHVdINa07Y5nFJJzhj8Nf8?= =?us-ascii?Q?aA8cT6BVZgluYqlGANiId1NwTRsRs8U1ogz5Y5nU5O+1ZqwGrjIexA4rIiAB?= =?us-ascii?Q?zR8kCluA4N/WxgbJkm7Uhzjct4t0fFlCyRdNiHS5mVlA9fnOjN9Woa1Fwhp3?= =?us-ascii?Q?TOOivtuwTRjT09OLVOUN90iE=3D?= X-Microsoft-Antispam-Message-Info: +QWjGdViKfiOgyCGvCYv/NTXIFGzLrX6ZFN+m9UgSwJVw7OoANcxgPkm2SkQbnzzmXqGMmxypedKh82ujk7mnMhdpbkhwyGN91NifoXj0KIGVSMEQ7WVQpTW51kF+dCDChj4IMFEpS05GRew5bVbvV9/zYSLTsOLuU0wQ/LtIp2KL9EKQiS1QjWCfzT3iZ5z X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 6:WLs78c4scS3qx42IdVHt2oARU2tvy2azjcKBAso1Fx4fIyLw2PD4oTFkI/BQR6VpFSxlC5/06zkLV0hVqIr5nc0cd1V+1tRp4m2w/D5dfM/k9I6hNoY5OdfD6iq81+xQK3+xJyKPuYjsNSQg+mya9V4sRuu4Ro/6Ilz49UalxA93cgJAyLVw0mfsc9PqBgMEHQtJ+Ew5WWwt0qFPSKUMZ1WAj4wbtPNZDSrm0kRaA6c5jw8EWXXcgJJXFbQEy8Z1p94pJexRe6o3l9XCG0UZwA0peg3OuT4n+T2w6QDPz8yf9d5cdMeD5CDb20K3Sl5k+5GFkIKNg0p3rYlbpJ3wyeknf6mogn7Io//0QO305PKE26HZztAp0txpmUsdV7OyMczRWxfVJuDpVmcw1Q/UxMiRBJr1gDekDNJIeeHgZhhe4dHzxDFU4rPS6NhmkQX2hqW0Q5J7S/aeQs8GBCY9iA==; 5:qVk/E+Ul42IZQ+5zGvz7TY6AuFP9JND4GBRRowJ8/SSJZh0ycxCbrdtV9Vks7hDVCOacheB8ljyLOMi2qxpJ+9Q4AqvVu559l8mNDwa+sIx/f8q5LHQk0OzuGpxEIO0+u8RhWam8kxlhzsFMvpaBODVbubdhI6kD+37OH5sh6PM=; 24:Fb1wNx3SSkofMc3jsslay7nA69Jny6PbqNo+qAlwvTmj7AGvJSPOqa0LKSk6fc3fLgPCDFJi8XM0DPNUSfyttiK4/1jSWtTILSC/o0aLny0= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 7:wpUmDfjwrWRRR0nYuhEexwaW+ZoXGJo6/0/mC8wcexVMB/Y2kGbB18qK2LR2cHWyJb7HgktUUoluWSsqz1tJ0JeOTDGwbGW5cpF3CAzBLCysgJEMBuS8RS/7krzqpBCENjBZiomBxqkkotgP+57iA1Hhhj6kmMTMZBpmZh7xaEhm/TMY/H7JR41JWsy+OcQAp1y6/UXM75Z6guExD0Wz7RId76TPHxCl2zzvLOaEfIlhztwi9pkK6wwvmRIUmsUA X-MS-Office365-Filtering-Correlation-Id: f7b27c12-c903-4823-04a0-08d5ca78f3af X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2018 00:11:56.2262 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f7b27c12-c903-4823-04a0-08d5ca78f3af X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2036 Subject: [dpdk-stable] [PATCH 03/67] net/mlx5: fix synchronization on polling Rx completions X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 00:12:02 -0000 [ backported from upstream commit 1742c2d9fab07e66209f2d14e7daa50829fc4423 ] Polling a new packet is basically sensing the generation bit in a completion entry. For some processors not having strongly-ordered memory model, there has to be a memory barrier between reading the generation bit and other fields of the entry in order to guarantee data is not stale. Fixes: 570acdb1da8a ("net/mlx5: add vectorized Rx/Tx burst for ARM") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh Acked-by: Shahaf Shuler Acked-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_rxtx.c | 1 + drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 53 ++++++++++++++++++++--------------- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 2 +- 3 files changed, 32 insertions(+), 24 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 9f1478cdb..c2dab1f4c 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -1717,6 +1717,7 @@ mlx5_rx_poll_len(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe, return 0; ++rxq->cq_ci; op_own = cqe->op_own; + rte_io_rmb(); if (MLX5_CQE_FORMAT(op_own) == MLX5_COMPRESSED) { volatile struct mlx5_mini_cqe8 (*mc)[8] = (volatile struct mlx5_mini_cqe8 (*)[8]) diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 06f83ef14..0b842f9a0 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -806,6 +806,7 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, uint16x4_t mask; uint16x4_t byte_cnt; uint32x4_t ptype_info, flow_tag; + register uint64x2_t c0, c1, c2, c3; uint8_t *p0, *p1, *p2, *p3; uint8_t *e0 = (void *)&elts[pos]->pkt_len; uint8_t *e1 = (void *)&elts[pos + 1]->pkt_len; @@ -822,6 +823,16 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, p1 = p0 + (pkts_n - pos > 1) * sizeof(struct mlx5_cqe); p2 = p1 + (pkts_n - pos > 2) * sizeof(struct mlx5_cqe); p3 = p2 + (pkts_n - pos > 3) * sizeof(struct mlx5_cqe); + /* B.0 (CQE 3) load a block having op_own. */ + c3 = vld1q_u64((uint64_t *)(p3 + 48)); + /* B.0 (CQE 2) load a block having op_own. */ + c2 = vld1q_u64((uint64_t *)(p2 + 48)); + /* B.0 (CQE 1) load a block having op_own. */ + c1 = vld1q_u64((uint64_t *)(p1 + 48)); + /* B.0 (CQE 0) load a block having op_own. */ + c0 = vld1q_u64((uint64_t *)(p0 + 48)); + /* Synchronize for loading the rest of blocks. */ + rte_io_rmb(); /* Prefetch next 4 CQEs. */ if (pkts_n - pos >= 2 * MLX5_VPMD_DESCS_PER_LOOP) { unsigned int next = pos + MLX5_VPMD_DESCS_PER_LOOP; @@ -831,50 +842,46 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, rte_prefetch_non_temporal(&cq[next + 3]); } __asm__ volatile ( - /* B.1 (CQE 3) load a block having op_own. */ - "ld1 {v19.16b}, [%[p3]] \n\t" - "sub %[p3], %[p3], #48 \n\t" - /* B.2 (CQE 3) load the rest blocks. */ + /* B.1 (CQE 3) load the rest of blocks. */ "ld1 {v16.16b - v18.16b}, [%[p3]] \n\t" + /* B.2 (CQE 3) move the block having op_own. */ + "mov v19.16b, %[c3].16b \n\t" /* B.3 (CQE 3) extract 16B fields. */ "tbl v23.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" + /* B.1 (CQE 2) load the rest of blocks. */ + "ld1 {v16.16b - v18.16b}, [%[p2]] \n\t" /* B.4 (CQE 3) adjust CRC length. */ "sub v23.8h, v23.8h, %[crc_adj].8h \n\t" - /* B.1 (CQE 2) load a block having op_own. */ - "ld1 {v19.16b}, [%[p2]] \n\t" - "sub %[p2], %[p2], #48 \n\t" /* C.1 (CQE 3) generate final structure for mbuf. */ "tbl v15.16b, {v23.16b}, %[mb_shuf_m].16b \n\t" - /* B.2 (CQE 2) load the rest blocks. */ - "ld1 {v16.16b - v18.16b}, [%[p2]] \n\t" + /* B.2 (CQE 2) move the block having op_own. */ + "mov v19.16b, %[c2].16b \n\t" /* B.3 (CQE 2) extract 16B fields. */ "tbl v22.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" + /* B.1 (CQE 1) load the rest of blocks. */ + "ld1 {v16.16b - v18.16b}, [%[p1]] \n\t" /* B.4 (CQE 2) adjust CRC length. */ "sub v22.8h, v22.8h, %[crc_adj].8h \n\t" - /* B.1 (CQE 1) load a block having op_own. */ - "ld1 {v19.16b}, [%[p1]] \n\t" - "sub %[p1], %[p1], #48 \n\t" /* C.1 (CQE 2) generate final structure for mbuf. */ "tbl v14.16b, {v22.16b}, %[mb_shuf_m].16b \n\t" - /* B.2 (CQE 1) load the rest blocks. */ - "ld1 {v16.16b - v18.16b}, [%[p1]] \n\t" + /* B.2 (CQE 1) move the block having op_own. */ + "mov v19.16b, %[c1].16b \n\t" /* B.3 (CQE 1) extract 16B fields. */ "tbl v21.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" + /* B.1 (CQE 0) load the rest of blocks. */ + "ld1 {v16.16b - v18.16b}, [%[p0]] \n\t" /* B.4 (CQE 1) adjust CRC length. */ "sub v21.8h, v21.8h, %[crc_adj].8h \n\t" - /* B.1 (CQE 0) load a block having op_own. */ - "ld1 {v19.16b}, [%[p0]] \n\t" - "sub %[p0], %[p0], #48 \n\t" /* C.1 (CQE 1) generate final structure for mbuf. */ "tbl v13.16b, {v21.16b}, %[mb_shuf_m].16b \n\t" - /* B.2 (CQE 0) load the rest blocks. */ - "ld1 {v16.16b - v18.16b}, [%[p0]] \n\t" + /* B.2 (CQE 0) move the block having op_own. */ + "mov v19.16b, %[c0].16b \n\t" + /* A.1 load mbuf pointers. */ + "ld1 {v24.2d - v25.2d}, [%[elts_p]] \n\t" /* B.3 (CQE 0) extract 16B fields. */ "tbl v20.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" /* B.4 (CQE 0) adjust CRC length. */ "sub v20.8h, v20.8h, %[crc_adj].8h \n\t" - /* A.1 load mbuf pointers. */ - "ld1 {v24.2d - v25.2d}, [%[elts_p]] \n\t" /* D.1 extract op_own byte. */ "tbl %[op_own].8b, {v20.16b - v23.16b}, %[owner_shuf_m].8b \n\t" /* C.2 (CQE 3) adjust flow mark. */ @@ -909,9 +916,9 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, [byte_cnt]"=&w"(byte_cnt), [ptype_info]"=&w"(ptype_info), [flow_tag]"=&w"(flow_tag) - :[p3]"r"(p3 + 48), [p2]"r"(p2 + 48), - [p1]"r"(p1 + 48), [p0]"r"(p0 + 48), + :[p3]"r"(p3), [p2]"r"(p2), [p1]"r"(p1), [p0]"r"(p0), [e3]"r"(e3), [e2]"r"(e2), [e1]"r"(e1), [e0]"r"(e0), + [c3]"w"(c3), [c2]"w"(c2), [c1]"w"(c1), [c0]"w"(c0), [elts_p]"r"(elts_p), [pkts_p]"r"(pkts_p), [cqe_shuf_m]"w"(cqe_shuf_m), diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index 7ef2c59e1..793142922 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -825,7 +825,7 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, /* B.2 copy mbuf pointers. */ _mm_storeu_si128((__m128i *)&pkts[pos], mbp1); _mm_storeu_si128((__m128i *)&pkts[pos + 2], mbp2); - rte_compiler_barrier(); + rte_io_rmb(); /* C.1 load remained CQE data and extract necessary fields. */ cqe_tmp2 = _mm_load_si128((__m128i *)&cq[pos + p3]); cqe_tmp1 = _mm_load_si128((__m128i *)&cq[pos + p2]); -- 2.11.0