From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2BCD243C03; Tue, 27 Feb 2024 06:42:58 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4AFD342E9F; Tue, 27 Feb 2024 06:42:22 +0100 (CET) Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by mails.dpdk.org (Postfix) with ESMTP id 91AD040295 for ; Tue, 27 Feb 2024 06:41:42 +0100 (CET) Received: by linux.microsoft.com (Postfix, from userid 1086) id 403FB20B74CA; Mon, 26 Feb 2024 21:41:40 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 403FB20B74CA DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1709012501; bh=BeEEZxgfyYklvMVnTPlxRiS5ANFInBF/rgMXXjDyXvY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sssfvs12wbLvIUOktEyG1PVqLfD5K00NrgnB0ZL2ndMacG3vhlZmeAysyPVJ2+Elt hb2Ad/yqQXaEBXICOKeN/+oUsOEZ2zNA89rC5nvHapM2ik0LhNOvTGmqjDP0YvcHiu 1lFoN+D8H3JJQpWuzDUA84unYkeZ28p45B1K0rJI= From: Tyler Retzlaff To: dev@dpdk.org Cc: Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Tyler Retzlaff Subject: [PATCH v6 10/23] net/iavf: use mbuf descriptor accessors Date: Mon, 26 Feb 2024 21:41:26 -0800 Message-Id: <1709012499-12813-11-git-send-email-roretzla@linux.microsoft.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Use new rte_mbuf_rearm_data and rte_mbuf_rx_descriptor_fields1 accessors that provide a compatible type pointer without using the marker fields. Signed-off-by: Tyler Retzlaff --- drivers/net/iavf/iavf_rxtx_vec_avx2.c | 72 +++++++--------------------- drivers/net/iavf/iavf_rxtx_vec_avx512.c | 72 +++++++--------------------- drivers/net/iavf/iavf_rxtx_vec_common.h | 4 +- drivers/net/iavf/iavf_rxtx_vec_neon.c | 16 +++---- drivers/net/iavf/iavf_rxtx_vec_sse.c | 85 +++++++-------------------------- 5 files changed, 58 insertions(+), 191 deletions(-) diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx2.c b/drivers/net/iavf/iavf_rxtx_vec_avx2.c index 510b4d8..33f2850 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx2.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx2.c @@ -98,19 +98,6 @@ 0xFF, 0xFF, /* pkt_type set as unknown */ 0xFF, 0xFF /*pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Status/Error flag masks */ /** @@ -372,13 +359,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -398,13 +378,13 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -427,13 +407,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ @@ -622,19 +602,6 @@ 0xFF, 0xFF, /* pkt_type set as unknown */ 0xFF, 0xFF /*pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Status/Error flag masks */ /** @@ -1279,13 +1246,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -1305,13 +1265,13 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -1334,13 +1294,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx512.c b/drivers/net/iavf/iavf_rxtx_vec_avx512.c index 3bb6f30..38d0669 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_avx512.c +++ b/drivers/net/iavf/iavf_rxtx_vec_avx512.c @@ -135,19 +135,6 @@ /* octet 15~14, low 16 bits pkt_len */ 0xFFFFFFFF /* pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); uint16_t i, received; @@ -412,13 +399,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -450,13 +430,13 @@ rearm0 = _mm256_permute2f128_si256(mbuf_init, mb0_1, 0x20); } /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -486,13 +466,13 @@ rearm1 = _mm256_blend_epi32(mbuf_init, mb0_1, 0xF0); } /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ @@ -703,19 +683,6 @@ /* octet 4~5, 16 bits pkt_len */ 0xFFFFFFFF /* pkt_type set as unknown */ ); - /** - * compile-time check the above crc and shuffle layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi - * calls above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); uint16_t i, received; @@ -1435,13 +1402,6 @@ * add in the previously computed rx_descriptor fields to * make a single 256-bit write per mbuf */ - /* check the structure matches expectations */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, - rearm_data), - 16)); /* build up data and do writes */ __m256i rearm0, rearm1, rearm2, rearm3, rearm4, rearm5, rearm6, rearm7; @@ -1461,13 +1421,13 @@ rearm2 = _mm256_permute2f128_si256(rearm2, mb2_3, 0x20); rearm0 = _mm256_permute2f128_si256(rearm0, mb0_1, 0x20); /* write to mbuf */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 6]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 6]), rearm6); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 4]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 4]), rearm4); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 2]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 2]), rearm2); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 0]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 0]), rearm0); /* repeat for the odd mbufs */ @@ -1490,13 +1450,13 @@ rearm3 = _mm256_blend_epi32(rearm3, mb2_3, 0xF0); rearm1 = _mm256_blend_epi32(rearm1, mb0_1, 0xF0); /* again write to mbufs */ - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 7]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 7]), rearm7); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 5]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 5]), rearm5); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 3]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 3]), rearm3); - _mm256_storeu_si256((__m256i *)&rx_pkts[i + 1]->rearm_data, + _mm256_storeu_si256((__m256i *)rte_mbuf_rearm_data(rx_pkts[i + 1]), rearm1); /* extract and record EOP bit */ diff --git a/drivers/net/iavf/iavf_rxtx_vec_common.h b/drivers/net/iavf/iavf_rxtx_vec_common.h index 5c52200..71e3644 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_common.h +++ b/drivers/net/iavf/iavf_rxtx_vec_common.h @@ -197,7 +197,6 @@ static inline int iavf_rxq_vec_setup_default(struct iavf_rx_queue *rxq) { - uintptr_t p; struct rte_mbuf mb_def = { .buf_addr = 0 }; /* zeroed mbuf */ mb_def.nb_segs = 1; @@ -207,8 +206,7 @@ /* prevent compiler reordering: rearm_data covers previous fields */ rte_compiler_barrier(); - p = (uintptr_t)&mb_def.rearm_data; - rxq->mbuf_initializer = *(uint64_t *)p; + rxq->mbuf_initializer = *rte_mbuf_rearm_data(&mb_def); return 0; } diff --git a/drivers/net/iavf/iavf_rxtx_vec_neon.c b/drivers/net/iavf/iavf_rxtx_vec_neon.c index 83825aa..d7ea940 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_neon.c +++ b/drivers/net/iavf/iavf_rxtx_vec_neon.c @@ -159,10 +159,10 @@ rearm2 = vsetq_lane_u64(vgetq_lane_u32(vlan0, 2), mbuf_init, 1); rearm3 = vsetq_lane_u64(vgetq_lane_u32(vlan0, 3), mbuf_init, 1); - vst1q_u64((uint64_t *)&rx_pkts[0]->rearm_data, rearm0); - vst1q_u64((uint64_t *)&rx_pkts[1]->rearm_data, rearm1); - vst1q_u64((uint64_t *)&rx_pkts[2]->rearm_data, rearm2); - vst1q_u64((uint64_t *)&rx_pkts[3]->rearm_data, rearm3); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + vst1q_u64(rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } #define PKTLEN_SHIFT 10 @@ -332,13 +332,13 @@ pkt_mb1 = vreinterpretq_u8_u16(tmp); /* D.3 copy final data to rx_pkts */ - vst1q_u8((void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); - vst1q_u8((void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); - vst1q_u8((void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - vst1q_u8((void *)&rx_pkts[pos]->rx_descriptor_fields1, + vst1q_u8(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); diff --git a/drivers/net/iavf/iavf_rxtx_vec_sse.c b/drivers/net/iavf/iavf_rxtx_vec_sse.c index 96f187f..9d6a453 100644 --- a/drivers/net/iavf/iavf_rxtx_vec_sse.c +++ b/drivers/net/iavf/iavf_rxtx_vec_sse.c @@ -179,14 +179,10 @@ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(vlan0, 4), 0x10); /* write the rearm data and the olflags in one write */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - _mm_store_si128((__m128i *)&rx_pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&rx_pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&rx_pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } static inline __m128i @@ -412,14 +408,10 @@ rearm3 = _mm_blend_epi16(mbuf_init, _mm_srli_si128(flags, 4), 0x30); /* write the rearm data and the olflags in one write */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != - offsetof(struct rte_mbuf, rearm_data) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, rearm_data) != - RTE_ALIGN(offsetof(struct rte_mbuf, rearm_data), 16)); - _mm_store_si128((__m128i *)&rx_pkts[0]->rearm_data, rearm0); - _mm_store_si128((__m128i *)&rx_pkts[1]->rearm_data, rearm1); - _mm_store_si128((__m128i *)&rx_pkts[2]->rearm_data, rearm2); - _mm_store_si128((__m128i *)&rx_pkts[3]->rearm_data, rearm3); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[0]), rearm0); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[1]), rearm1); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[2]), rearm2); + _mm_store_si128((__m128i *)rte_mbuf_rearm_data(rx_pkts[3]), rearm3); } #define PKTLEN_SHIFT 10 @@ -488,14 +480,7 @@ -rxq->crc_len, /* sub crc on pkt_len */ 0, 0 /* ignore pkt_type field */ ); - /* compile-time check the above crc_adjust layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi16 - * call above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); + __m128i dd_check, eop_check; /* nb_pkts has to be floor-aligned to IAVF_VPMD_DESCS_PER_LOOP */ @@ -536,18 +521,6 @@ 15, 14, /* octet 15~14, low 16 bits pkt_len */ 0xFF, 0xFF, 0xFF, 0xFF /* pkt_type set as unknown */ ); - /* Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* Cache is empty -> need to scan the buffer rings, but first move * the next 'n' mbufs into the cache @@ -651,10 +624,10 @@ /* D.3 copy final 3,4 data to rx_pkts */ _mm_storeu_si128( - (void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb4); _mm_storeu_si128( - (void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb3); /* D.2 pkt 1,2 remove crc */ @@ -689,9 +662,9 @@ /* D.3 copy final 1,2 data to rx_pkts */ _mm_storeu_si128( - (void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb2); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb1); desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); /* C.4 calc available number of desc */ @@ -760,16 +733,6 @@ 0x04, 0x0C, 0x00, 0x08); - /** - * compile-time check the above crc_adjust layout is correct. - * NOTE: the first field (lowest address) is given last in set_epi16 - * call above. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - /* 4 packets DD mask */ const __m128i dd_check = _mm_set_epi64x(0x0000000100000001LL, 0x0000000100000001LL); @@ -818,20 +781,6 @@ #endif - /** - * Compile-time verify the shuffle mask - * NOTE: some field positions already verified above, but duplicated - * here for completeness in case of future modifications. - */ - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 4); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 8); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, vlan_tci) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 10); - RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != - offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); - /* Cache is empty -> need to scan the buffer rings, but first move * the next 'n' mbufs into the cache */ @@ -1089,10 +1038,10 @@ /* D.3 copy final 3,4 data to rx_pkts */ _mm_storeu_si128 - ((void *)&rx_pkts[pos + 3]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 3]), pkt_mb3); _mm_storeu_si128 - ((void *)&rx_pkts[pos + 2]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 2]), pkt_mb2); /* C* extract and record EOP bit */ @@ -1116,9 +1065,9 @@ /* D.3 copy final 1,2 data to rx_pkts */ _mm_storeu_si128 - ((void *)&rx_pkts[pos + 1]->rx_descriptor_fields1, + (rte_mbuf_rx_descriptor_fields1(rx_pkts[pos + 1]), pkt_mb1); - _mm_storeu_si128((void *)&rx_pkts[pos]->rx_descriptor_fields1, + _mm_storeu_si128(rte_mbuf_rx_descriptor_fields1(rx_pkts[pos]), pkt_mb0); flex_desc_to_ptype_v(descs, &rx_pkts[pos], ptype_tbl); /* C.4 calc available number of desc */ -- 1.8.3.1