From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 17190A0547; Fri, 10 Sep 2021 14:30:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C37940E46; Fri, 10 Sep 2021 14:30:21 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id EC192406B4 for ; Fri, 10 Sep 2021 14:30:18 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10102"; a="243386257" X-IronPort-AV: E=Sophos;i="5.85,282,1624345200"; d="scan'208";a="243386257" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2021 05:30:09 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,282,1624345200"; d="scan'208";a="514279710" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by orsmga001.jf.intel.com with ESMTP; 10 Sep 2021 05:30:08 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Fri, 10 Sep 2021 13:29:44 +0100 Message-Id: <20210910123003.85448-5-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210910123003.85448-1-cristian.dumitrescu@intel.com> References: <20210910123003.85448-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH 05/24] pipeline: create inline functions for extract instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Signed-off-by: Cristian Dumitrescu --- lib/pipeline/rte_swx_pipeline.c | 130 ++++------------- lib/pipeline/rte_swx_pipeline_internal.h | 178 +++++++++++++++++++++++ 2 files changed, 203 insertions(+), 105 deletions(-) diff --git a/lib/pipeline/rte_swx_pipeline.c b/lib/pipeline/rte_swx_pipeline.c index bcf796f8c3..fd7e31b709 100644 --- a/lib/pipeline/rte_swx_pipeline.c +++ b/lib/pipeline/rte_swx_pipeline.c @@ -1665,52 +1665,12 @@ instr_hdr_lookahead_translate(struct rte_swx_pipeline *p, } static inline void -__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract); - -static inline void -__instr_hdr_extract_exec(struct rte_swx_pipeline *p, uint32_t n_extract) +instr_hdr_extract_exec(struct rte_swx_pipeline *p) { struct thread *t = &p->threads[p->thread_id]; struct instruction *ip = t->ip; - uint64_t valid_headers = t->valid_headers; - uint8_t *ptr = t->ptr; - uint32_t offset = t->pkt.offset; - uint32_t length = t->pkt.length; - uint32_t i; - for (i = 0; i < n_extract; i++) { - uint32_t header_id = ip->io.hdr.header_id[i]; - uint32_t struct_id = ip->io.hdr.struct_id[i]; - uint32_t n_bytes = ip->io.hdr.n_bytes[i]; - - TRACE("[Thread %2u]: extract header %u (%u bytes)\n", - p->thread_id, - header_id, - n_bytes); - - /* Headers. */ - t->structs[struct_id] = ptr; - valid_headers = MASK64_BIT_SET(valid_headers, header_id); - - /* Packet. */ - offset += n_bytes; - length -= n_bytes; - ptr += n_bytes; - } - - /* Headers. */ - t->valid_headers = valid_headers; - - /* Packet. */ - t->pkt.offset = offset; - t->pkt.length = length; - t->ptr = ptr; -} - -static inline void -instr_hdr_extract_exec(struct rte_swx_pipeline *p) -{ - __instr_hdr_extract_exec(p, 1); + __instr_hdr_extract_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1719,10 +1679,10 @@ instr_hdr_extract_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_extract2_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_extract_exec(p, 2); + __instr_hdr_extract2_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1731,10 +1691,10 @@ instr_hdr_extract2_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_extract3_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_extract_exec(p, 3); + __instr_hdr_extract3_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1743,10 +1703,10 @@ instr_hdr_extract3_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_extract4_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_extract_exec(p, 4); + __instr_hdr_extract4_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1755,10 +1715,10 @@ instr_hdr_extract4_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_extract5_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_extract_exec(p, 5); + __instr_hdr_extract5_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1767,10 +1727,10 @@ instr_hdr_extract5_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_extract6_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_extract_exec(p, 6); + __instr_hdr_extract6_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1779,10 +1739,10 @@ instr_hdr_extract6_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_extract7_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_extract_exec(p, 7); + __instr_hdr_extract7_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1791,10 +1751,10 @@ instr_hdr_extract7_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_extract8_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_extract_exec(p, 8); + __instr_hdr_extract8_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1806,35 +1766,7 @@ instr_hdr_extract_m_exec(struct rte_swx_pipeline *p) struct thread *t = &p->threads[p->thread_id]; struct instruction *ip = t->ip; - uint64_t valid_headers = t->valid_headers; - uint8_t *ptr = t->ptr; - uint32_t offset = t->pkt.offset; - uint32_t length = t->pkt.length; - - uint32_t n_bytes_last = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); - uint32_t header_id = ip->io.hdr.header_id[0]; - uint32_t struct_id = ip->io.hdr.struct_id[0]; - uint32_t n_bytes = ip->io.hdr.n_bytes[0]; - - struct header_runtime *h = &t->headers[header_id]; - - TRACE("[Thread %2u]: extract header %u (%u + %u bytes)\n", - p->thread_id, - header_id, - n_bytes, - n_bytes_last); - - n_bytes += n_bytes_last; - - /* Headers. */ - t->structs[struct_id] = ptr; - t->valid_headers = MASK64_BIT_SET(valid_headers, header_id); - h->n_bytes = n_bytes; - - /* Packet. */ - t->pkt.offset = offset + n_bytes; - t->pkt.length = length - n_bytes; - t->ptr = ptr + n_bytes; + __instr_hdr_extract_m_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1846,19 +1778,7 @@ instr_hdr_lookahead_exec(struct rte_swx_pipeline *p) struct thread *t = &p->threads[p->thread_id]; struct instruction *ip = t->ip; - uint64_t valid_headers = t->valid_headers; - uint8_t *ptr = t->ptr; - - uint32_t header_id = ip->io.hdr.header_id[0]; - uint32_t struct_id = ip->io.hdr.struct_id[0]; - - TRACE("[Thread %2u]: lookahead header %u\n", - p->thread_id, - header_id); - - /* Headers. */ - t->structs[struct_id] = ptr; - t->valid_headers = MASK64_BIT_SET(valid_headers, header_id); + __instr_hdr_lookahead_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); diff --git a/lib/pipeline/rte_swx_pipeline_internal.h b/lib/pipeline/rte_swx_pipeline_internal.h index e9fe6632b6..1519bcc305 100644 --- a/lib/pipeline/rte_swx_pipeline_internal.h +++ b/lib/pipeline/rte_swx_pipeline_internal.h @@ -1580,4 +1580,182 @@ __instr_tx_i_exec(struct rte_swx_pipeline *p, struct thread *t, const struct ins port->pkt_tx(port->obj, pkt); } +/* + * extract. + */ +static inline void +__instr_hdr_extract_many_exec(struct rte_swx_pipeline *p __rte_unused, + struct thread *t, + const struct instruction *ip, + uint32_t n_extract) +{ + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + uint32_t i; + + for (i = 0; i < n_extract; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + uint32_t n_bytes = ip->io.hdr.n_bytes[i]; + + TRACE("[Thread %2u]: extract header %u (%u bytes)\n", + p->thread_id, + header_id, + n_bytes); + + /* Headers. */ + t->structs[struct_id] = ptr; + valid_headers = MASK64_BIT_SET(valid_headers, header_id); + + /* Packet. */ + offset += n_bytes; + length -= n_bytes; + ptr += n_bytes; + } + + /* Headers. */ + t->valid_headers = valid_headers; + + /* Packet. */ + t->pkt.offset = offset; + t->pkt.length = length; + t->ptr = ptr; +} + +static inline void +__instr_hdr_extract_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + __instr_hdr_extract_many_exec(p, t, ip, 1); +} + +static inline void +__instr_hdr_extract2_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_extract_many_exec(p, t, ip, 2); +} + +static inline void +__instr_hdr_extract3_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_extract_many_exec(p, t, ip, 3); +} + +static inline void +__instr_hdr_extract4_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_extract_many_exec(p, t, ip, 4); +} + +static inline void +__instr_hdr_extract5_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_extract_many_exec(p, t, ip, 5); +} + +static inline void +__instr_hdr_extract6_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_extract_many_exec(p, t, ip, 6); +} + +static inline void +__instr_hdr_extract7_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_extract_many_exec(p, t, ip, 7); +} + +static inline void +__instr_hdr_extract8_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_extract_many_exec(p, t, ip, 8); +} + +static inline void +__instr_hdr_extract_m_exec(struct rte_swx_pipeline *p __rte_unused, + struct thread *t, + const struct instruction *ip) +{ + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + uint32_t offset = t->pkt.offset; + uint32_t length = t->pkt.length; + + uint32_t n_bytes_last = METADATA_READ(t, ip->io.io.offset, ip->io.io.n_bits); + uint32_t header_id = ip->io.hdr.header_id[0]; + uint32_t struct_id = ip->io.hdr.struct_id[0]; + uint32_t n_bytes = ip->io.hdr.n_bytes[0]; + + struct header_runtime *h = &t->headers[header_id]; + + TRACE("[Thread %2u]: extract header %u (%u + %u bytes)\n", + p->thread_id, + header_id, + n_bytes, + n_bytes_last); + + n_bytes += n_bytes_last; + + /* Headers. */ + t->structs[struct_id] = ptr; + t->valid_headers = MASK64_BIT_SET(valid_headers, header_id); + h->n_bytes = n_bytes; + + /* Packet. */ + t->pkt.offset = offset + n_bytes; + t->pkt.length = length - n_bytes; + t->ptr = ptr + n_bytes; +} + +static inline void +__instr_hdr_lookahead_exec(struct rte_swx_pipeline *p __rte_unused, + struct thread *t, + const struct instruction *ip) +{ + uint64_t valid_headers = t->valid_headers; + uint8_t *ptr = t->ptr; + + uint32_t header_id = ip->io.hdr.header_id[0]; + uint32_t struct_id = ip->io.hdr.struct_id[0]; + + TRACE("[Thread %2u]: lookahead header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + t->structs[struct_id] = ptr; + t->valid_headers = MASK64_BIT_SET(valid_headers, header_id); +} + #endif -- 2.17.1