From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4D159A0547; Fri, 10 Sep 2021 15:37:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0EA9D41161; Fri, 10 Sep 2021 15:37:25 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 2C55E406B4 for ; Fri, 10 Sep 2021 15:37:20 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10102"; a="221113271" X-IronPort-AV: E=Sophos;i="5.85,283,1624345200"; d="scan'208";a="221113271" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Sep 2021 06:37:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,283,1624345200"; d="scan'208";a="549328527" Received: from silpixa00400573.ir.intel.com (HELO silpixa00400573.ger.corp.intel.com) ([10.237.223.107]) by fmsmga002.fm.intel.com with ESMTP; 10 Sep 2021 06:37:19 -0700 From: Cristian Dumitrescu To: dev@dpdk.org Date: Fri, 10 Sep 2021 14:36:55 +0100 Message-Id: <20210910133713.93103-6-cristian.dumitrescu@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210910133713.93103-1-cristian.dumitrescu@intel.com> References: <20210910123003.85448-1-cristian.dumitrescu@intel.com> <20210910133713.93103-1-cristian.dumitrescu@intel.com> Subject: [dpdk-dev] [PATCH V2 06/24] pipeline: create inline functions for emit instruction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Create inline functions for the emit instruction. Signed-off-by: Cristian Dumitrescu --- lib/pipeline/rte_swx_pipeline.c | 162 ++++++++------------- lib/pipeline/rte_swx_pipeline_internal.h | 170 +++++++++++++++++++++++ 2 files changed, 228 insertions(+), 104 deletions(-) diff --git a/lib/pipeline/rte_swx_pipeline.c b/lib/pipeline/rte_swx_pipeline.c index fd7e31b709..80c5fb94bb 100644 --- a/lib/pipeline/rte_swx_pipeline.c +++ b/lib/pipeline/rte_swx_pipeline.c @@ -1810,82 +1810,12 @@ instr_hdr_emit_translate(struct rte_swx_pipeline *p, } static inline void -__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit); - -static inline void -__instr_hdr_emit_exec(struct rte_swx_pipeline *p, uint32_t n_emit) +instr_hdr_emit_exec(struct rte_swx_pipeline *p) { struct thread *t = &p->threads[p->thread_id]; struct instruction *ip = t->ip; - uint64_t valid_headers = t->valid_headers; - uint32_t n_headers_out = t->n_headers_out; - struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; - uint8_t *ho_ptr = NULL; - uint32_t ho_nbytes = 0, first = 1, i; - - for (i = 0; i < n_emit; i++) { - uint32_t header_id = ip->io.hdr.header_id[i]; - uint32_t struct_id = ip->io.hdr.struct_id[i]; - - struct header_runtime *hi = &t->headers[header_id]; - uint8_t *hi_ptr0 = hi->ptr0; - uint32_t n_bytes = hi->n_bytes; - - uint8_t *hi_ptr = t->structs[struct_id]; - - if (!MASK64_BIT_GET(valid_headers, header_id)) - continue; - - TRACE("[Thread %2u]: emit header %u\n", - p->thread_id, - header_id); - - /* Headers. */ - if (first) { - first = 0; - - if (!t->n_headers_out) { - ho = &t->headers_out[0]; - - ho->ptr0 = hi_ptr0; - ho->ptr = hi_ptr; - - ho_ptr = hi_ptr; - ho_nbytes = n_bytes; - - n_headers_out = 1; - - continue; - } else { - ho_ptr = ho->ptr; - ho_nbytes = ho->n_bytes; - } - } - - if (ho_ptr + ho_nbytes == hi_ptr) { - ho_nbytes += n_bytes; - } else { - ho->n_bytes = ho_nbytes; - - ho++; - ho->ptr0 = hi_ptr0; - ho->ptr = hi_ptr; - ho_ptr = hi_ptr; - ho_nbytes = n_bytes; - - n_headers_out++; - } - } - - ho->n_bytes = ho_nbytes; - t->n_headers_out = n_headers_out; -} - -static inline void -instr_hdr_emit_exec(struct rte_swx_pipeline *p) -{ - __instr_hdr_emit_exec(p, 1); + __instr_hdr_emit_exec(p, t, ip); /* Thread. */ thread_ip_inc(p); @@ -1894,81 +1824,105 @@ instr_hdr_emit_exec(struct rte_swx_pipeline *p) static inline void instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + __instr_hdr_emit_tx_exec(p, t, ip); - __instr_hdr_emit_exec(p, 1); - instr_tx_exec(p); + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } static inline void instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_emit_exec(p, 2); - instr_tx_exec(p); + __instr_hdr_emit2_tx_exec(p, t, ip); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } static inline void instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + __instr_hdr_emit3_tx_exec(p, t, ip); - __instr_hdr_emit_exec(p, 3); - instr_tx_exec(p); + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } static inline void instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + __instr_hdr_emit4_tx_exec(p, t, ip); - __instr_hdr_emit_exec(p, 4); - instr_tx_exec(p); + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } static inline void instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; - __instr_hdr_emit_exec(p, 5); - instr_tx_exec(p); + __instr_hdr_emit5_tx_exec(p, t, ip); + + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } static inline void instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + __instr_hdr_emit6_tx_exec(p, t, ip); - __instr_hdr_emit_exec(p, 6); - instr_tx_exec(p); + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } static inline void instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + __instr_hdr_emit7_tx_exec(p, t, ip); - __instr_hdr_emit_exec(p, 7); - instr_tx_exec(p); + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } static inline void instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p) { - TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", - p->thread_id); + struct thread *t = &p->threads[p->thread_id]; + struct instruction *ip = t->ip; + + __instr_hdr_emit8_tx_exec(p, t, ip); - __instr_hdr_emit_exec(p, 8); - instr_tx_exec(p); + /* Thread. */ + thread_ip_reset(p, t); + instr_rx_exec(p); } /* diff --git a/lib/pipeline/rte_swx_pipeline_internal.h b/lib/pipeline/rte_swx_pipeline_internal.h index 1519bcc305..8b37a9812e 100644 --- a/lib/pipeline/rte_swx_pipeline_internal.h +++ b/lib/pipeline/rte_swx_pipeline_internal.h @@ -1758,4 +1758,174 @@ __instr_hdr_lookahead_exec(struct rte_swx_pipeline *p __rte_unused, t->valid_headers = MASK64_BIT_SET(valid_headers, header_id); } +/* + * emit. + */ +static inline void +__instr_hdr_emit_many_exec(struct rte_swx_pipeline *p __rte_unused, + struct thread *t, + const struct instruction *ip, + uint32_t n_emit) +{ + uint64_t valid_headers = t->valid_headers; + uint32_t n_headers_out = t->n_headers_out; + struct header_out_runtime *ho = &t->headers_out[n_headers_out - 1]; + uint8_t *ho_ptr = NULL; + uint32_t ho_nbytes = 0, first = 1, i; + + for (i = 0; i < n_emit; i++) { + uint32_t header_id = ip->io.hdr.header_id[i]; + uint32_t struct_id = ip->io.hdr.struct_id[i]; + + struct header_runtime *hi = &t->headers[header_id]; + uint8_t *hi_ptr0 = hi->ptr0; + uint32_t n_bytes = hi->n_bytes; + + uint8_t *hi_ptr = t->structs[struct_id]; + + if (!MASK64_BIT_GET(valid_headers, header_id)) + continue; + + TRACE("[Thread %2u]: emit header %u\n", + p->thread_id, + header_id); + + /* Headers. */ + if (first) { + first = 0; + + if (!t->n_headers_out) { + ho = &t->headers_out[0]; + + ho->ptr0 = hi_ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out = 1; + + continue; + } else { + ho_ptr = ho->ptr; + ho_nbytes = ho->n_bytes; + } + } + + if (ho_ptr + ho_nbytes == hi_ptr) { + ho_nbytes += n_bytes; + } else { + ho->n_bytes = ho_nbytes; + + ho++; + ho->ptr0 = hi_ptr0; + ho->ptr = hi_ptr; + + ho_ptr = hi_ptr; + ho_nbytes = n_bytes; + + n_headers_out++; + } + } + + ho->n_bytes = ho_nbytes; + t->n_headers_out = n_headers_out; +} + +static inline void +__instr_hdr_emit_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + __instr_hdr_emit_many_exec(p, t, ip, 1); +} + +static inline void +__instr_hdr_emit_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 2 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 1); + __instr_tx_exec(p, t, ip); +} + +static inline void +__instr_hdr_emit2_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 3 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 2); + __instr_tx_exec(p, t, ip); +} + +static inline void +__instr_hdr_emit3_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 4 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 3); + __instr_tx_exec(p, t, ip); +} + +static inline void +__instr_hdr_emit4_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 5 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 4); + __instr_tx_exec(p, t, ip); +} + +static inline void +__instr_hdr_emit5_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 6 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 5); + __instr_tx_exec(p, t, ip); +} + +static inline void +__instr_hdr_emit6_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 7 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 6); + __instr_tx_exec(p, t, ip); +} + +static inline void +__instr_hdr_emit7_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 8 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 7); + __instr_tx_exec(p, t, ip); +} + +static inline void +__instr_hdr_emit8_tx_exec(struct rte_swx_pipeline *p, + struct thread *t, + const struct instruction *ip) +{ + TRACE("[Thread %2u] *** The next 9 instructions are fused. ***\n", p->thread_id); + + __instr_hdr_emit_many_exec(p, t, ip, 8); + __instr_tx_exec(p, t, ip); +} + #endif -- 2.17.1