From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0086.outbound.protection.outlook.com [104.47.2.86]) by dpdk.org (Postfix) with ESMTP id 16170AACC for ; Tue, 5 Jun 2018 02:27:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LjQmGk0Zjd5sZQzh9RnClpptnhcvFk3fVUMi7B53OG8=; b=DkOpPoAyP87/M1IC8G8NmFbXJJduui43YS3QpovX3aGwEA+wO4c0QQv8cmcmgLnqg44nt+ZFGd2yf528gIcpnHgwCipYbAbohgZOmdTfcHqawVatZFPTrcaF9HZ3LSPOzZ5QgGKB/8yKUsCsIjOCECbOqEQLwAXSirroVQVhf8Y= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by VI1PR0501MB2045.eurprd05.prod.outlook.com (2603:10a6:800:36::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.11; Tue, 5 Jun 2018 00:27:55 +0000 From: Yongseok Koh To: yliu@fridaylinux.org Cc: stable@dpdk.org, shahafs@mellanox.com, adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com, Yongseok Koh Date: Mon, 4 Jun 2018 17:27:26 -0700 Message-Id: <20180605002732.13866-4-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180605002732.13866-1-yskoh@mellanox.com> References: <20180605002732.13866-1-yskoh@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: CO1PR15CA0050.namprd15.prod.outlook.com (2603:10b6:101:1f::18) To VI1PR0501MB2045.eurprd05.prod.outlook.com (2603:10a6:800:36::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:VI1PR0501MB2045; X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 3:BWJ7kc6uSxUEeR91JwRrUqFBl4GdczsQ/egPbgrpAmBC47Yg1xAu1LDJODi/i+VHHKG7nKo5127oe9io6FsUUHfVAxNTab5Na/87T59jnyghfuiBpgpMbVxr9ty22Iy8ifjnnqysVR7glYyWEO6E2BYBgMw9mzsO2aIey5E1C28z0X0BlozA0+nbkUhcACNcTOpVjCOV/qhq2GX+tkbX8jMwu2rmBe3decIYdGOIrED8SNPsTB2eDbVm1f43kcQp; 25:kwp02p8hBE3lWKsUSA2ylDZ5d024JRTNZRWfaEMC6xPG5cKNS2WSn0f+V3rb32EAkA3w6ggBfXH1b0E11cOx4ICqgQmJCaafgbwJbG5swLBiCqar2r9aQ9Ehd/rBPC0OXtda7eUV9tI+iCMfV+6FnuImzZAkdpaK8Arslab0a+6yHhbSw7L0PK2Js4j5rTiXtAFkIMEaqkyFDXCZ/l3sCnZjG9vOm1Y3fw83wkZKsGaAa8/wr2dIN53cr4SVt8EDO8yULMXWuF9oOlrtKquDUrwQJ7+ETEiiHzVwqlG4hfaXLhlYCtmxI+hjrj7SHD9GOvmu30/NtriP4bngqqVpZQ==; 31:Y5T7U+A6GJvyCRrshKQmtTR3gUB9SGQisWSGe4j/hTj3NoSzuwKVDhveBhBO6EQmCV0pHwFnSth10exNgS0gei3+raIUHlJ4r2RoT1zXAffuH5vpHFb0H2lVy/BI37raBjakznqbU+1nL2Ixk/e+3C/8bX8r95QSiKvzhP8EfNdb2k1Gm2OtQS7mhuzHfzCDybadCBNyGJSyIFVA1DqMy7QImZa/tchTU5b3GN1gxxM= X-MS-TrafficTypeDiagnostic: VI1PR0501MB2045: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 20:wVhyohlavwX0m5/5q4v31OkhlPpVFdoHsnNuT9Sk6ZSJDjgxs5dR3EQX2u8SR8uGt9DI63Nmp5VreRJA0y3SrGUKqPOO0iIR9sbjVsro4X/z4edpDHL7b20g1mkGuYO6e2WTjp3l7DsZnP6MokkIGHSjHhRxBK1VzWy2IMLwBZdRSubfvq1lioWXkF4LUldp/si8bFXPRdzZ9GJlX6jN0YIgm88itjEN6L+TOVaVDJgAHZzt4QI0D2kTes4Iujzas/Z2RsxLqG7KeH7a1xHoa78niFOEPXJCxVtTQ/6PQ1d+i/9WRP0CXAhs8ajtWxvKOciF6Jhjn7Cm/u7QIz4BQs9q2WdF8pejGOGYK3u5Gh3eD0hykJXSAQM05j89+Y9mQ2t76jrka6j2rgGeq6TG7vlbU6vQABdl0d/kWGiFjXTR2XhMKeSxeDuXoGG2RC74DdlypX7Bl8kE0Ges+onWJLgfLwngznPiEsP89CkNBppveGAllYZ5ezdMEFuUZvTO; 4:EFltwA5qyfI0SuerR8n1tgMT37Y6GhcXE4QLSKMSX7Qo5b9t2RFuGc0mFjlR8jvmSyMkUdclbwnfCIN5E93Ty1YuSYqQnci6mYBsvhj6AxB/xYc4sfmfW1GmxRczrvyGC7N4dDWYxSBRKpcsVHu7xszzloqDNDbivdHjyf0kpxK8HzTSUiv6I3hiVZH9GF+vVo4DlSWP+L282nCXe/YoQjzohO6qi/j/2a3JY5NCqrwClaw9564ORDOTi+BSyvJxO+Vhpn/9yqWo3eAbLeonHg== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(3231254)(944501410)(52105095)(10201501046)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(6072148)(201708071742011)(7699016); SRVR:VI1PR0501MB2045; BCL:0; PCL:0; RULEID:; SRVR:VI1PR0501MB2045; X-Forefront-PRVS: 0694C54398 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(376002)(396003)(39380400002)(366004)(346002)(39860400002)(189003)(199004)(316002)(956004)(2616005)(486006)(7696005)(476003)(386003)(11346002)(51416003)(69596002)(52116002)(36756003)(105586002)(25786009)(4326008)(186003)(21086003)(107886003)(76176011)(16526019)(26005)(106356001)(50226002)(59450400001)(478600001)(66066001)(2351001)(55016002)(7736002)(305945005)(8676002)(97736004)(8936002)(5660300001)(3846002)(6116002)(6666003)(47776003)(446003)(1076002)(53936002)(50466002)(16586007)(48376002)(68736007)(81156014)(81166006)(86362001)(2361001)(2906002)(6916009); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2045; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0501MB2045; 23:mvSxlh/OoUViAETZHmdmxR67Z/SrPKAJASjnEPN?= =?us-ascii?Q?JPUoClNCd9F5OYEiaSrIfxgXHSR0iPAQWYOiFiwOJ/OtGmUP85V5T6DlQ9Cu?= =?us-ascii?Q?XudpE+9VhLsGrLSpfE1Vcj1mUvG5ru4HTCQQ21gUnb0Ad0rX+n/GZDzWKO/L?= =?us-ascii?Q?WikJBtMyYhc1PMXipIO71iZ+nENXDj2AQrtKkY20wOaSP2aIn+KR4NAk4/Gr?= =?us-ascii?Q?cg7LDkv3fUmu1uRTlcohSGfLiu31AS4oM1UidxMg6Jqj4jvPKmkP+etoeb2H?= =?us-ascii?Q?zfuyJay5rb1lpJxpX64dX5NM35KYqWmVqAeaiOIdyIyp9QBARv6GxmL3pF1E?= =?us-ascii?Q?ymeu9bca9EGUSUR2GbIwHOIkNQgrIFzeMQrL2qkR4Tfsimmdu+auk3IE4gIV?= =?us-ascii?Q?kjLMGxbY0fh4G8Vilyc9ea2bwAi7OVbxYMspJpbClTwCTo+NtUUfaLy6mTfb?= =?us-ascii?Q?EgEu5fQDihCnUu5PSgaIgvnieNTdeiRqOz/MEMmI+iOIb/dRaBLj+z2M2+LC?= =?us-ascii?Q?7MVcS+Cqr9pdZYtwIavP8hObMjPrNKtrZNY4rtHLGG+QhdgWIGnvF/Vzpd3e?= =?us-ascii?Q?KISVGOcP8oJ0EqzTpWqAFN6Dyocr8SEqRQB4hVmal8bBZ3jcwHPdZegQwlpV?= =?us-ascii?Q?1C4Grh/00880u2P6NKfRDv4VJ0bJfhR5xTcukBInOdZXxIfXVkV5E0Bj87r2?= =?us-ascii?Q?M6OWQQQzzjKqAKpjKBstNq/eRJCS9b5gsUVrzePzrpbMP9N+/5G5oO8Ya7u6?= =?us-ascii?Q?grAFhvlvXm3UZ2Alpd8pXgZS7hqcHdU/ye/gCNAFiK0e1YXB0L2B9yD9YBux?= =?us-ascii?Q?F4euFBShLbwLjaIdrWYe5W7YuIvVqLAHuHbj1LZoIalgtuVhow+FHWsMAGX6?= =?us-ascii?Q?L6ZY4oGFfd/TahwjBYv2jX5bPPkMUdDbYUodoQmOgMgqU5W5TOYszp1n4TAU?= =?us-ascii?Q?/i4rL5iSoNAIEqWyhxH1ctRMhFP4/yNeITgVtAW+4YrwAoYt5a+MSoig/+oU?= =?us-ascii?Q?uuqG3v73UH0dkTRhEj+0nmRrpBfVPbYDqs85EFsaMVDO1NU3IK9NteOaQ5Yz?= =?us-ascii?Q?HTXSNCSIDzMolId1PExANldSfj/4nOzWDb+Q4g9ByzmjB6B7SYTlL1red50f?= =?us-ascii?Q?BXRG3YWL7qmz+DihSwcGIoCDgUdZJNGlzatmxBqR/b+7RKwhCRs4kOkwJj9N?= =?us-ascii?Q?M2c1qir2NtpAoaAjR6bFjGOIJ9RIaLzehqodsvlsxQp3oTc5S65YcF+5b3oJ?= =?us-ascii?Q?P2ZXdN3cgzA3eUho4HdbibJcJ+j5Nip/8Ch8AoMVyCd1BR/HOCNVQSLrOPWu?= =?us-ascii?Q?LbdZywmRuVTZOTWzOZCm1pfY=3D?= X-Microsoft-Antispam-Message-Info: EfBtP5gfTjBST8qBbcIYvQyzos/pj+YIDNBPA+X10KIQj2gFL9P8GgINwDxtke38nj8p/pvd0JHzJ0DGFtyCNXpikzO+ga9VXQm2vKskgoTCGO4r62tgfXCxelvq2KnRFOAol/eODDZ5+lH3DAUB33k2fFT0bWeEgFRm2p+sTCSy3Y9DlH7u3Ngbk5xK4SGT X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 6:qheDonWIQRBT5aIaB/+Xch1Lo7rEOyarsZZdZS9hqeJ1vxUBqfw1QDjUQMKzKr9WynmywFCngR7eEFEeii4vbUHsiVBTNSKmDVsrFtIyeqeuASuNcn2Vpearn/27ymvNMnM7dFT/D47e4Ltu5en8AwlO2peSQ9G77niXg5aOKeNq3Gu70a07ompMhRLMX8varOww4NaG8h6Unum/8ZOa2p64v7BUqUiVnVidyFZXdWDS6vftRd3ogLoVfw+Dp+pcDpUeJZkC2sPIJCNVIElrIRBE5XjZOrjaea2yr+3pIt2epUSgbHIMpUona4O+Pi7IvAOvfFDSXb+Tb+DvWQiMyw04GuqVTJxiv1NkgwwNHK7LcbUQciEEY3UWP8XVqEMiSvdAG5bIS2tLREAEAEguaf3LJlwFk2QQc+xQKs8a/W1BixO2Oo78sG3/pkVq2sRJcQYDESQcPpTYDohIMdon5w==; 5:5qsida9DMYMfHRLuTI5/3cuTH1LSYh7U4ENBgFKWyJ0xPoL98De2FT2Vj92smhmKK8cue8lJOaWWpVIod0Xt+R6ATpZeaNOGkS7nL9EGQI2dhDkk3BFQatmBtCn2vsWLvo+Skl4xyltfOAgpWJR9FzOgnMl8TUQeCqvn2Vb3ynQ=; 24:PLGoNv7JcWb2+H2aE1RcbLRWU70TLqOutMlzE9WPopcF6vRWoZMmLH6cRDLuiI/lCoc/eU82YrKBjp6tLvkM6lFxKyVqG5Nsud/NCThJpSE= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; VI1PR0501MB2045; 7:mvVUl3ZOeAOaKe0c7weeWbt+/1MI4hAWXABlmGoAOUwkFik+QfoWZNPu71SdiV6tibW08VdfWkFMeEpc5j7wUOSyf1xtwN9Ix1lP8fHUaZLLlAIZf4sh96Hr63qD3b4+SJkXjPcnKfZh2QlAprEhH+kBdlbrGpRNoV3Vm+Dwl/8ctRqrkAbWkScgbzhDkfelth5NsJDC0C6tyk15+H5kUFOm/2mNYyjb/ZslkB0fHSxY4xWlucRQLMsCC+kUVL/i X-MS-Office365-Filtering-Correlation-Id: 2c962ac6-f4ec-4223-b11d-08d5ca7b2fa9 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2018 00:27:55.7543 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2c962ac6-f4ec-4223-b11d-08d5ca7b2fa9 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2045 Subject: [dpdk-stable] [PATCH v2 03/67] net/mlx5: fix synchronization on polling Rx completions X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2018 00:27:58 -0000 [ backported from upstream commit 1742c2d9fab07e66209f2d14e7daa50829fc4423 ] Polling a new packet is basically sensing the generation bit in a completion entry. For some processors not having strongly-ordered memory model, there has to be a memory barrier between reading the generation bit and other fields of the entry in order to guarantee data is not stale. Fixes: 570acdb1da8a ("net/mlx5: add vectorized Rx/Tx burst for ARM") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh Acked-by: Shahaf Shuler Acked-by: Nelio Laranjeiro --- drivers/net/mlx5/mlx5_rxtx.c | 1 + drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 53 ++++++++++++++++++++--------------- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 2 +- 3 files changed, 32 insertions(+), 24 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c index 9f1478cdb..c2dab1f4c 100644 --- a/drivers/net/mlx5/mlx5_rxtx.c +++ b/drivers/net/mlx5/mlx5_rxtx.c @@ -1717,6 +1717,7 @@ mlx5_rx_poll_len(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cqe, return 0; ++rxq->cq_ci; op_own = cqe->op_own; + rte_io_rmb(); if (MLX5_CQE_FORMAT(op_own) == MLX5_COMPRESSED) { volatile struct mlx5_mini_cqe8 (*mc)[8] = (volatile struct mlx5_mini_cqe8 (*)[8]) diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 06f83ef14..0b842f9a0 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -806,6 +806,7 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, uint16x4_t mask; uint16x4_t byte_cnt; uint32x4_t ptype_info, flow_tag; + register uint64x2_t c0, c1, c2, c3; uint8_t *p0, *p1, *p2, *p3; uint8_t *e0 = (void *)&elts[pos]->pkt_len; uint8_t *e1 = (void *)&elts[pos + 1]->pkt_len; @@ -822,6 +823,16 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, p1 = p0 + (pkts_n - pos > 1) * sizeof(struct mlx5_cqe); p2 = p1 + (pkts_n - pos > 2) * sizeof(struct mlx5_cqe); p3 = p2 + (pkts_n - pos > 3) * sizeof(struct mlx5_cqe); + /* B.0 (CQE 3) load a block having op_own. */ + c3 = vld1q_u64((uint64_t *)(p3 + 48)); + /* B.0 (CQE 2) load a block having op_own. */ + c2 = vld1q_u64((uint64_t *)(p2 + 48)); + /* B.0 (CQE 1) load a block having op_own. */ + c1 = vld1q_u64((uint64_t *)(p1 + 48)); + /* B.0 (CQE 0) load a block having op_own. */ + c0 = vld1q_u64((uint64_t *)(p0 + 48)); + /* Synchronize for loading the rest of blocks. */ + rte_io_rmb(); /* Prefetch next 4 CQEs. */ if (pkts_n - pos >= 2 * MLX5_VPMD_DESCS_PER_LOOP) { unsigned int next = pos + MLX5_VPMD_DESCS_PER_LOOP; @@ -831,50 +842,46 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, rte_prefetch_non_temporal(&cq[next + 3]); } __asm__ volatile ( - /* B.1 (CQE 3) load a block having op_own. */ - "ld1 {v19.16b}, [%[p3]] \n\t" - "sub %[p3], %[p3], #48 \n\t" - /* B.2 (CQE 3) load the rest blocks. */ + /* B.1 (CQE 3) load the rest of blocks. */ "ld1 {v16.16b - v18.16b}, [%[p3]] \n\t" + /* B.2 (CQE 3) move the block having op_own. */ + "mov v19.16b, %[c3].16b \n\t" /* B.3 (CQE 3) extract 16B fields. */ "tbl v23.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" + /* B.1 (CQE 2) load the rest of blocks. */ + "ld1 {v16.16b - v18.16b}, [%[p2]] \n\t" /* B.4 (CQE 3) adjust CRC length. */ "sub v23.8h, v23.8h, %[crc_adj].8h \n\t" - /* B.1 (CQE 2) load a block having op_own. */ - "ld1 {v19.16b}, [%[p2]] \n\t" - "sub %[p2], %[p2], #48 \n\t" /* C.1 (CQE 3) generate final structure for mbuf. */ "tbl v15.16b, {v23.16b}, %[mb_shuf_m].16b \n\t" - /* B.2 (CQE 2) load the rest blocks. */ - "ld1 {v16.16b - v18.16b}, [%[p2]] \n\t" + /* B.2 (CQE 2) move the block having op_own. */ + "mov v19.16b, %[c2].16b \n\t" /* B.3 (CQE 2) extract 16B fields. */ "tbl v22.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" + /* B.1 (CQE 1) load the rest of blocks. */ + "ld1 {v16.16b - v18.16b}, [%[p1]] \n\t" /* B.4 (CQE 2) adjust CRC length. */ "sub v22.8h, v22.8h, %[crc_adj].8h \n\t" - /* B.1 (CQE 1) load a block having op_own. */ - "ld1 {v19.16b}, [%[p1]] \n\t" - "sub %[p1], %[p1], #48 \n\t" /* C.1 (CQE 2) generate final structure for mbuf. */ "tbl v14.16b, {v22.16b}, %[mb_shuf_m].16b \n\t" - /* B.2 (CQE 1) load the rest blocks. */ - "ld1 {v16.16b - v18.16b}, [%[p1]] \n\t" + /* B.2 (CQE 1) move the block having op_own. */ + "mov v19.16b, %[c1].16b \n\t" /* B.3 (CQE 1) extract 16B fields. */ "tbl v21.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" + /* B.1 (CQE 0) load the rest of blocks. */ + "ld1 {v16.16b - v18.16b}, [%[p0]] \n\t" /* B.4 (CQE 1) adjust CRC length. */ "sub v21.8h, v21.8h, %[crc_adj].8h \n\t" - /* B.1 (CQE 0) load a block having op_own. */ - "ld1 {v19.16b}, [%[p0]] \n\t" - "sub %[p0], %[p0], #48 \n\t" /* C.1 (CQE 1) generate final structure for mbuf. */ "tbl v13.16b, {v21.16b}, %[mb_shuf_m].16b \n\t" - /* B.2 (CQE 0) load the rest blocks. */ - "ld1 {v16.16b - v18.16b}, [%[p0]] \n\t" + /* B.2 (CQE 0) move the block having op_own. */ + "mov v19.16b, %[c0].16b \n\t" + /* A.1 load mbuf pointers. */ + "ld1 {v24.2d - v25.2d}, [%[elts_p]] \n\t" /* B.3 (CQE 0) extract 16B fields. */ "tbl v20.16b, {v16.16b - v19.16b}, %[cqe_shuf_m].16b \n\t" /* B.4 (CQE 0) adjust CRC length. */ "sub v20.8h, v20.8h, %[crc_adj].8h \n\t" - /* A.1 load mbuf pointers. */ - "ld1 {v24.2d - v25.2d}, [%[elts_p]] \n\t" /* D.1 extract op_own byte. */ "tbl %[op_own].8b, {v20.16b - v23.16b}, %[owner_shuf_m].8b \n\t" /* C.2 (CQE 3) adjust flow mark. */ @@ -909,9 +916,9 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, [byte_cnt]"=&w"(byte_cnt), [ptype_info]"=&w"(ptype_info), [flow_tag]"=&w"(flow_tag) - :[p3]"r"(p3 + 48), [p2]"r"(p2 + 48), - [p1]"r"(p1 + 48), [p0]"r"(p0 + 48), + :[p3]"r"(p3), [p2]"r"(p2), [p1]"r"(p1), [p0]"r"(p0), [e3]"r"(e3), [e2]"r"(e2), [e1]"r"(e1), [e0]"r"(e0), + [c3]"w"(c3), [c2]"w"(c2), [c1]"w"(c1), [c0]"w"(c0), [elts_p]"r"(elts_p), [pkts_p]"r"(pkts_p), [cqe_shuf_m]"w"(cqe_shuf_m), diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index 7ef2c59e1..793142922 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -825,7 +825,7 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, /* B.2 copy mbuf pointers. */ _mm_storeu_si128((__m128i *)&pkts[pos], mbp1); _mm_storeu_si128((__m128i *)&pkts[pos + 2], mbp2); - rte_compiler_barrier(); + rte_io_rmb(); /* C.1 load remained CQE data and extract necessary fields. */ cqe_tmp2 = _mm_load_si128((__m128i *)&cq[pos + p3]); cqe_tmp1 = _mm_load_si128((__m128i *)&cq[pos + p2]); -- 2.11.0