From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AE60045B14; Fri, 11 Oct 2024 19:57:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4173A40E19; Fri, 11 Oct 2024 19:57:09 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) by mails.dpdk.org (Postfix) with ESMTP id DCD5740E01 for ; Fri, 11 Oct 2024 19:57:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728669428; x=1760205428; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IjQF5u5wW8jk6WqHYRDwvkO/qnYvM/u3w+xonQOt0Y0=; b=J9rpEPCauR4Jq++uKw6A3h8ZjRoExvmmqOhX7Tw+T5YVdVHnz9+49fMS HroS9VaNskSzzsmH7XyAzQsW6oF3I/9UGh2g3haaLZv0t6Qjs99h2GZVP 7ZscsnGJkN+lwrlFbFRq2vQGO0xn0PDTRIiFAod+sAMuNyK84P9CFt1Ws rZxEVYfDKNQ3/3FuT5ZZtl7nbcGgFfqAB1HlrCDNx89Plbn3CAnJCgZTt NIzkPzUz+Eb89PWW7dpUB/cNX1sAgG4ldhfuVNy0+WGnhzJH97ZTCeLAa 4Q76bDbdAy3/4G8f6vLI8AQZ50lcbBAfeZppzXlH7JbT9W68F4uSQNprE w==; X-CSE-ConnectionGUID: +H4qQyJqT1mVZvsjCpUvPQ== X-CSE-MsgGUID: 7j8rIoK1QRyCJNjGeG4GbQ== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="50622285" X-IronPort-AV: E=Sophos;i="6.11,196,1725346800"; d="scan'208";a="50622285" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Oct 2024 10:57:04 -0700 X-CSE-ConnectionGUID: H0DEyB/TSqukE1yOlxgRmg== X-CSE-MsgGUID: jUijwO9aTUidiIdCGDn7xw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,196,1725346800"; d="scan'208";a="76887859" Received: from unknown (HELO silpixa00401176.ir.intel.com) ([10.243.22.76]) by orviesa010.jf.intel.com with ESMTP; 11 Oct 2024 10:57:02 -0700 From: Vladimir Medvedkin To: dev@dpdk.org Cc: rjarry@redhat.com, mb@smartsharesystems.com, david.marchand@redhat.com Subject: [PATCH v4] fib: network byte order IPv4 lookup Date: Fri, 11 Oct 2024 17:57:01 +0000 Message-ID: <20241011175701.116371-1-vladimir.medvedkin@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241010112621.681773-1-vladimir.medvedkin@intel.com> References: <20241010112621.681773-1-vladimir.medvedkin@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Previously when running rte_fib_lookup IPv4 addresses must have been in host byte order. This patch adds a new flag RTE_FIB_FLAG_LOOKUP_BE that can be passed on fib create, which will allow to have IPv4 in network byte order on lookup. Signed-off-by: Vladimir Medvedkin --- app/test/test_fib.c | 10 ++-- app/test/test_fib_perf.c | 2 +- doc/guides/rel_notes/deprecation.rst | 3 - doc/guides/rel_notes/release_24_11.rst | 5 ++ examples/l3fwd/l3fwd_fib.c | 2 +- lib/fib/dir24_8.c | 63 +++++++++++++------- lib/fib/dir24_8.h | 48 ++++++++++++++- lib/fib/dir24_8_avx512.c | 82 +++++++++++++++----------- lib/fib/dir24_8_avx512.h | 15 +++++ lib/fib/meson.build | 63 +++++--------------- lib/fib/rte_fib.c | 8 ++- lib/fib/rte_fib.h | 5 ++ 12 files changed, 189 insertions(+), 117 deletions(-) diff --git a/app/test/test_fib.c b/app/test/test_fib.c index 45dccca1f6..69c439a3a8 100644 --- a/app/test/test_fib.c +++ b/app/test/test_fib.c @@ -33,7 +33,7 @@ int32_t test_create_invalid(void) { struct rte_fib *fib = NULL; - struct rte_fib_conf config; + struct rte_fib_conf config = { 0 }; config.max_routes = MAX_ROUTES; config.rib_ext_sz = 0; @@ -92,7 +92,7 @@ int32_t test_multiple_create(void) { struct rte_fib *fib = NULL; - struct rte_fib_conf config; + struct rte_fib_conf config = { 0 }; int32_t i; config.rib_ext_sz = 0; @@ -119,7 +119,7 @@ int32_t test_free_null(void) { struct rte_fib *fib = NULL; - struct rte_fib_conf config; + struct rte_fib_conf config = { 0 }; config.max_routes = MAX_ROUTES; config.rib_ext_sz = 0; @@ -142,7 +142,7 @@ int32_t test_add_del_invalid(void) { struct rte_fib *fib = NULL; - struct rte_fib_conf config; + struct rte_fib_conf config = { 0 }; uint64_t nh = 100; uint32_t ip = RTE_IPV4(0, 0, 0, 0); int ret; @@ -319,7 +319,7 @@ int32_t test_lookup(void) { struct rte_fib *fib = NULL; - struct rte_fib_conf config; + struct rte_fib_conf config = { 0 }; uint64_t def_nh = 100; int ret; diff --git a/app/test/test_fib_perf.c b/app/test/test_fib_perf.c index d3cd986d2e..1b2ad04854 100644 --- a/app/test/test_fib_perf.c +++ b/app/test/test_fib_perf.c @@ -320,7 +320,7 @@ static int test_fib_perf(void) { struct rte_fib *fib = NULL; - struct rte_fib_conf config; + struct rte_fib_conf config = { 0 }; config.max_routes = 2000000; config.rib_ext_sz = 0; diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index 1535ea7abf..7bc2310bc4 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -166,9 +166,6 @@ Deprecation Notices The legacy actions should be removed once ``MODIFY_FIELD`` alternative is implemented in drivers. -* fib: A new flag field will be introduced in ``rte_fib_conf`` structure - in DPDK 24.11. This field will be used to pass extra configuration settings. - * cryptodev: The Intel IPsec Multi-Buffer version will be bumped to a minimum version of v1.4. This will effect the KASUMI, SNOW3G, ZUC, AESNI GCM, AESNI MB and CHACHAPOLY diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst index 22c4084d83..bfef2cd99e 100644 --- a/doc/guides/rel_notes/release_24_11.rst +++ b/doc/guides/rel_notes/release_24_11.rst @@ -154,6 +154,11 @@ New Features * Added independent enqueue feature. +* **Updated FIB configuration structure.** + + A new flag field introduced in ``rte_fib_conf`` structure. + This field is used to pass an extra configuration settings such as ability + to lookup IPv4 addresses in network byte order. Removed Items ------------- diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c index f38b19af3f..993e36cec2 100644 --- a/examples/l3fwd/l3fwd_fib.c +++ b/examples/l3fwd/l3fwd_fib.c @@ -644,7 +644,7 @@ setup_fib(const int socketid) { struct rte_eth_dev_info dev_info; struct rte_fib6_conf config; - struct rte_fib_conf config_ipv4; + struct rte_fib_conf config_ipv4 = { 0 }; int i; int ret; char s[64]; diff --git a/lib/fib/dir24_8.c b/lib/fib/dir24_8.c index c739e92304..5c856ff6a6 100644 --- a/lib/fib/dir24_8.c +++ b/lib/fib/dir24_8.c @@ -26,67 +26,84 @@ #define ROUNDUP(x, y) RTE_ALIGN_CEIL(x, (1 << (32 - y))) static inline rte_fib_lookup_fn_t -get_scalar_fn(enum rte_fib_dir24_8_nh_sz nh_sz) +get_scalar_fn(enum rte_fib_dir24_8_nh_sz nh_sz, bool be_addr) { switch (nh_sz) { case RTE_FIB_DIR24_8_1B: - return dir24_8_lookup_bulk_1b; + return (be_addr) ? dir24_8_lookup_bulk_1b_be : + dir24_8_lookup_bulk_1b; case RTE_FIB_DIR24_8_2B: - return dir24_8_lookup_bulk_2b; + return (be_addr) ? dir24_8_lookup_bulk_2b_be : + dir24_8_lookup_bulk_2b; case RTE_FIB_DIR24_8_4B: - return dir24_8_lookup_bulk_4b; + return (be_addr) ? dir24_8_lookup_bulk_4b_be : + dir24_8_lookup_bulk_4b; case RTE_FIB_DIR24_8_8B: - return dir24_8_lookup_bulk_8b; + return (be_addr) ? dir24_8_lookup_bulk_8b_be : + dir24_8_lookup_bulk_8b; default: return NULL; } } static inline rte_fib_lookup_fn_t -get_scalar_fn_inlined(enum rte_fib_dir24_8_nh_sz nh_sz) +get_scalar_fn_inlined(enum rte_fib_dir24_8_nh_sz nh_sz, bool be_addr) { switch (nh_sz) { case RTE_FIB_DIR24_8_1B: - return dir24_8_lookup_bulk_0; + return (be_addr) ? dir24_8_lookup_bulk_0_be : + dir24_8_lookup_bulk_0; case RTE_FIB_DIR24_8_2B: - return dir24_8_lookup_bulk_1; + return (be_addr) ? dir24_8_lookup_bulk_1_be : + dir24_8_lookup_bulk_1; case RTE_FIB_DIR24_8_4B: - return dir24_8_lookup_bulk_2; + return (be_addr) ? dir24_8_lookup_bulk_2_be : + dir24_8_lookup_bulk_2; case RTE_FIB_DIR24_8_8B: - return dir24_8_lookup_bulk_3; + return (be_addr) ? dir24_8_lookup_bulk_3_be : + dir24_8_lookup_bulk_3; default: return NULL; } } static inline rte_fib_lookup_fn_t -get_vector_fn(enum rte_fib_dir24_8_nh_sz nh_sz) +get_vector_fn(enum rte_fib_dir24_8_nh_sz nh_sz, bool be_addr) { #ifdef CC_DIR24_8_AVX512_SUPPORT if ((rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512F) <= 0) || + (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512DQ) <= 0) || (rte_vect_get_max_simd_bitwidth() < RTE_VECT_SIMD_512)) return NULL; + if (be_addr && (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX512BW) <= 0)) + return NULL; + switch (nh_sz) { case RTE_FIB_DIR24_8_1B: - return rte_dir24_8_vec_lookup_bulk_1b; + return (be_addr) ? rte_dir24_8_vec_lookup_bulk_1b_be : + rte_dir24_8_vec_lookup_bulk_1b; case RTE_FIB_DIR24_8_2B: - return rte_dir24_8_vec_lookup_bulk_2b; + return (be_addr) ? rte_dir24_8_vec_lookup_bulk_2b_be : + rte_dir24_8_vec_lookup_bulk_2b; case RTE_FIB_DIR24_8_4B: - return rte_dir24_8_vec_lookup_bulk_4b; + return (be_addr) ? rte_dir24_8_vec_lookup_bulk_4b_be : + rte_dir24_8_vec_lookup_bulk_4b; case RTE_FIB_DIR24_8_8B: - return rte_dir24_8_vec_lookup_bulk_8b; + return (be_addr) ? rte_dir24_8_vec_lookup_bulk_8b_be : + rte_dir24_8_vec_lookup_bulk_8b; default: return NULL; } #else RTE_SET_USED(nh_sz); + RTE_SET_USED(be_addr); #endif return NULL; } rte_fib_lookup_fn_t -dir24_8_get_lookup_fn(void *p, enum rte_fib_lookup_type type) +dir24_8_get_lookup_fn(void *p, enum rte_fib_lookup_type type, bool be_addr) { enum rte_fib_dir24_8_nh_sz nh_sz; rte_fib_lookup_fn_t ret_fn; @@ -99,16 +116,18 @@ dir24_8_get_lookup_fn(void *p, enum rte_fib_lookup_type type) switch (type) { case RTE_FIB_LOOKUP_DIR24_8_SCALAR_MACRO: - return get_scalar_fn(nh_sz); + return get_scalar_fn(nh_sz, be_addr); case RTE_FIB_LOOKUP_DIR24_8_SCALAR_INLINE: - return get_scalar_fn_inlined(nh_sz); + return get_scalar_fn_inlined(nh_sz, be_addr); case RTE_FIB_LOOKUP_DIR24_8_SCALAR_UNI: - return dir24_8_lookup_bulk_uni; + return (be_addr) ? dir24_8_lookup_bulk_uni_be : + dir24_8_lookup_bulk_uni; case RTE_FIB_LOOKUP_DIR24_8_VECTOR_AVX512: - return get_vector_fn(nh_sz); + return get_vector_fn(nh_sz, be_addr); case RTE_FIB_LOOKUP_DEFAULT: - ret_fn = get_vector_fn(nh_sz); - return (ret_fn != NULL) ? ret_fn : get_scalar_fn(nh_sz); + ret_fn = get_vector_fn(nh_sz, be_addr); + return (ret_fn != NULL) ? ret_fn : + get_scalar_fn(nh_sz, be_addr); default: return NULL; } diff --git a/lib/fib/dir24_8.h b/lib/fib/dir24_8.h index 7125049f15..dee5275673 100644 --- a/lib/fib/dir24_8.h +++ b/lib/fib/dir24_8.h @@ -7,7 +7,9 @@ #define _DIR24_8_H_ #include +#include +#include #include #include @@ -237,6 +239,50 @@ dir24_8_lookup_bulk_uni(void *p, const uint32_t *ips, } } +#define BSWAP_MAX_LENGTH 64 + +typedef void (*dir24_8_lookup_bulk_be_cb)(void *p, const uint32_t *ips, + uint64_t *next_hops, const unsigned int n); + +static inline void +dir24_8_lookup_bulk_be(void *p, const uint32_t *ips, + uint64_t *next_hops, const unsigned int n, + dir24_8_lookup_bulk_be_cb cb) +{ + uint32_t le_ips[BSWAP_MAX_LENGTH]; + unsigned int i; + +#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN + cb(p, ips, next_hops, n); +#else + for (i = 0; i < n; i += BSWAP_MAX_LENGTH) { + int j; + for (j = 0; j < BSWAP_MAX_LENGTH && i + j < n; j++) + le_ips[j] = rte_be_to_cpu_32(ips[i + j]); + + cb(p, le_ips, next_hops + i, j); + } +#endif +} + +#define DECLARE_BE_LOOKUP_FN(name) \ +static inline void \ +name##_be(void *p, const uint32_t *ips, \ + uint64_t *next_hops, const unsigned int n) \ +{ \ + dir24_8_lookup_bulk_be(p, ips, next_hops, n, name); \ +} + +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_1b) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_2b) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_4b) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_8b) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_0) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_1) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_2) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_3) +DECLARE_BE_LOOKUP_FN(dir24_8_lookup_bulk_uni) + void * dir24_8_create(const char *name, int socket_id, struct rte_fib_conf *conf); @@ -244,7 +290,7 @@ void dir24_8_free(void *p); rte_fib_lookup_fn_t -dir24_8_get_lookup_fn(void *p, enum rte_fib_lookup_type type); +dir24_8_get_lookup_fn(void *p, enum rte_fib_lookup_type type, bool be_addr); int dir24_8_modify(struct rte_fib *fib, uint32_t ip, uint8_t depth, diff --git a/lib/fib/dir24_8_avx512.c b/lib/fib/dir24_8_avx512.c index 43dba28cfb..e6fe08ecfe 100644 --- a/lib/fib/dir24_8_avx512.c +++ b/lib/fib/dir24_8_avx512.c @@ -10,7 +10,7 @@ static __rte_always_inline void dir24_8_vec_lookup_x16(void *p, const uint32_t *ips, - uint64_t *next_hops, int size) + uint64_t *next_hops, int size, bool be_addr) { struct dir24_8_tbl *dp = (struct dir24_8_tbl *)p; __mmask16 msk_ext; @@ -28,6 +28,16 @@ dir24_8_vec_lookup_x16(void *p, const uint32_t *ips, res_msk = _mm512_set1_epi32(UINT16_MAX); ip_vec = _mm512_loadu_si512(ips); + if (be_addr) { + const __m512i bswap32 = _mm512_set_epi32( + 0x0c0d0e0f, 0x08090a0b, 0x04050607, 0x00010203, + 0x0c0d0e0f, 0x08090a0b, 0x04050607, 0x00010203, + 0x0c0d0e0f, 0x08090a0b, 0x04050607, 0x00010203, + 0x0c0d0e0f, 0x08090a0b, 0x04050607, 0x00010203 + ); + ip_vec = _mm512_shuffle_epi8(ip_vec, bswap32); + } + /* mask 24 most significant bits */ idxes = _mm512_srli_epi32(ip_vec, 8); @@ -78,7 +88,7 @@ dir24_8_vec_lookup_x16(void *p, const uint32_t *ips, static __rte_always_inline void dir24_8_vec_lookup_x8_8b(void *p, const uint32_t *ips, - uint64_t *next_hops) + uint64_t *next_hops, bool be_addr) { struct dir24_8_tbl *dp = (struct dir24_8_tbl *)p; const __m512i zero = _mm512_set1_epi32(0); @@ -89,6 +99,13 @@ dir24_8_vec_lookup_x8_8b(void *p, const uint32_t *ips, __mmask8 msk_ext; ip_vec = _mm256_loadu_si256((const void *)ips); + if (be_addr) { + const __m256i bswap32 = _mm256_set_epi8( + 12, 13, 14, 15, 8, 9, 10, 11, 4, 5, 6, 7, 0, 1, 2, 3, + 12, 13, 14, 15, 8, 9, 10, 11, 4, 5, 6, 7, 0, 1, 2, 3 + ); + ip_vec = _mm256_shuffle_epi8(ip_vec, bswap32); + } /* mask 24 most significant bits */ idxes_256 = _mm256_srli_epi32(ip_vec, 8); @@ -114,52 +131,49 @@ dir24_8_vec_lookup_x8_8b(void *p, const uint32_t *ips, _mm512_storeu_si512(next_hops, res); } -void -rte_dir24_8_vec_lookup_bulk_1b(void *p, const uint32_t *ips, - uint64_t *next_hops, const unsigned int n) -{ - uint32_t i; - for (i = 0; i < (n / 16); i++) - dir24_8_vec_lookup_x16(p, ips + i * 16, next_hops + i * 16, - sizeof(uint8_t)); - - dir24_8_lookup_bulk_1b(p, ips + i * 16, next_hops + i * 16, - n - i * 16); +#define DECLARE_VECTOR_FN(suffix, nh_type, be_addr) \ +void \ +rte_dir24_8_vec_lookup_bulk_##suffix(void *p, const uint32_t *ips, \ + uint64_t *next_hops, const unsigned int n) \ +{ \ + uint32_t i; \ + \ + for (i = 0; i < (n / 16); i++) \ + dir24_8_vec_lookup_x16(p, ips + i * 16, next_hops + i * 16, \ + sizeof(nh_type), be_addr); \ + \ + dir24_8_lookup_bulk_##suffix(p, ips + i * 16, next_hops + i * 16, \ + n - i * 16); \ } -void -rte_dir24_8_vec_lookup_bulk_2b(void *p, const uint32_t *ips, - uint64_t *next_hops, const unsigned int n) -{ - uint32_t i; - for (i = 0; i < (n / 16); i++) - dir24_8_vec_lookup_x16(p, ips + i * 16, next_hops + i * 16, - sizeof(uint16_t)); - - dir24_8_lookup_bulk_2b(p, ips + i * 16, next_hops + i * 16, - n - i * 16); -} +DECLARE_VECTOR_FN(1b, uint8_t, false) +DECLARE_VECTOR_FN(2b, uint16_t, false) +DECLARE_VECTOR_FN(4b, uint32_t, false) +DECLARE_VECTOR_FN(1b_be, uint8_t, true) +DECLARE_VECTOR_FN(2b_be, uint16_t, true) +DECLARE_VECTOR_FN(4b_be, uint32_t, true) void -rte_dir24_8_vec_lookup_bulk_4b(void *p, const uint32_t *ips, +rte_dir24_8_vec_lookup_bulk_8b(void *p, const uint32_t *ips, uint64_t *next_hops, const unsigned int n) { uint32_t i; - for (i = 0; i < (n / 16); i++) - dir24_8_vec_lookup_x16(p, ips + i * 16, next_hops + i * 16, - sizeof(uint32_t)); + for (i = 0; i < (n / 8); i++) + dir24_8_vec_lookup_x8_8b(p, ips + i * 8, + next_hops + i * 8, false); - dir24_8_lookup_bulk_4b(p, ips + i * 16, next_hops + i * 16, - n - i * 16); + dir24_8_lookup_bulk_8b(p, ips + i * 8, next_hops + i * 8, n - i * 8); } void -rte_dir24_8_vec_lookup_bulk_8b(void *p, const uint32_t *ips, +rte_dir24_8_vec_lookup_bulk_8b_be(void *p, const uint32_t *ips, uint64_t *next_hops, const unsigned int n) { uint32_t i; for (i = 0; i < (n / 8); i++) - dir24_8_vec_lookup_x8_8b(p, ips + i * 8, next_hops + i * 8); + dir24_8_vec_lookup_x8_8b(p, ips + i * 8, + next_hops + i * 8, true); - dir24_8_lookup_bulk_8b(p, ips + i * 8, next_hops + i * 8, n - i * 8); + dir24_8_lookup_bulk_8b_be(p, ips + i * 8, + next_hops + i * 8, n - i * 8); } diff --git a/lib/fib/dir24_8_avx512.h b/lib/fib/dir24_8_avx512.h index 1d3c2b9317..e9f7b72519 100644 --- a/lib/fib/dir24_8_avx512.h +++ b/lib/fib/dir24_8_avx512.h @@ -21,4 +21,19 @@ void rte_dir24_8_vec_lookup_bulk_8b(void *p, const uint32_t *ips, uint64_t *next_hops, const unsigned int n); +void +rte_dir24_8_vec_lookup_bulk_1b_be(void *p, const uint32_t *ips, + uint64_t *next_hops, const unsigned int n); + +void +rte_dir24_8_vec_lookup_bulk_2b_be(void *p, const uint32_t *ips, + uint64_t *next_hops, const unsigned int n); + +void +rte_dir24_8_vec_lookup_bulk_4b_be(void *p, const uint32_t *ips, + uint64_t *next_hops, const unsigned int n); + +void +rte_dir24_8_vec_lookup_bulk_8b_be(void *p, const uint32_t *ips, + uint64_t *next_hops, const unsigned int n); #endif /* _DIR248_AVX512_H_ */ diff --git a/lib/fib/meson.build b/lib/fib/meson.build index 6795f41a0a..07f207bd41 100644 --- a/lib/fib/meson.build +++ b/lib/fib/meson.build @@ -12,53 +12,20 @@ sources = files('rte_fib.c', 'rte_fib6.c', 'dir24_8.c', 'trie.c') headers = files('rte_fib.h', 'rte_fib6.h') deps += ['rib'] -# compile AVX512 version if: -# we are building 64-bit binary AND binutils can generate proper code -if dpdk_conf.has('RTE_ARCH_X86_64') and binutils_ok - # compile AVX512 version if either: - # a. we have AVX512F supported in minimum instruction set baseline - # b. it's not minimum instruction set, but supported by compiler - # - # in former case, just add avx512 C file to files list - # in latter case, compile c file to static lib, using correct - # compiler flags, and then have the .o file from static lib - # linked into main lib. +if target_has_avx512 + cflags += ['-DCC_DIR24_8_AVX512_SUPPORT', '-DCC_TRIE_AVX512_SUPPORT'] + sources += files('dir24_8_avx512.c', 'trie_avx512.c') - # check if all required flags already enabled (variant a). - acl_avx512_flags = ['__AVX512F__','__AVX512DQ__'] - acl_avx512_on = true - foreach f:acl_avx512_flags - if cc.get_define(f, args: machine_args) == '' - acl_avx512_on = false - endif - endforeach - - if acl_avx512_on == true - cflags += ['-DCC_DIR24_8_AVX512_SUPPORT'] - sources += files('dir24_8_avx512.c') - # TRIE AVX512 implementation uses avx512bw intrinsics along with - # avx512f and avx512dq - if cc.get_define('__AVX512BW__', args: machine_args) != '' - cflags += ['-DCC_TRIE_AVX512_SUPPORT'] - sources += files('trie_avx512.c') - endif - elif cc.has_multi_arguments('-mavx512f', '-mavx512dq') - dir24_8_avx512_tmp = static_library('dir24_8_avx512_tmp', - 'dir24_8_avx512.c', - dependencies: static_rte_eal, - c_args: cflags + ['-mavx512f', '-mavx512dq']) - objs += dir24_8_avx512_tmp.extract_objects('dir24_8_avx512.c') - cflags += ['-DCC_DIR24_8_AVX512_SUPPORT'] - # TRIE AVX512 implementation uses avx512bw intrinsics along with - # avx512f and avx512dq - if cc.has_argument('-mavx512bw') - trie_avx512_tmp = static_library('trie_avx512_tmp', - 'trie_avx512.c', - dependencies: static_rte_eal, - c_args: cflags + ['-mavx512f', \ - '-mavx512dq', '-mavx512bw']) - objs += trie_avx512_tmp.extract_objects('trie_avx512.c') - cflags += ['-DCC_TRIE_AVX512_SUPPORT'] - endif - endif +elif cc_has_avx512 + dir24_8_avx512_tmp = static_library('dir24_8_avx512_tmp', + 'dir24_8_avx512.c', + dependencies: static_rte_eal, + c_args: cflags + ['-mavx512f', '-mavx512dq', '-mavx512bw']) + objs += dir24_8_avx512_tmp.extract_objects('dir24_8_avx512.c') + trie_avx512_tmp = static_library('trie_avx512_tmp', + 'trie_avx512.c', + dependencies: static_rte_eal, + c_args: cflags + ['-mavx512f', '-mavx512dq', '-mavx512bw']) + objs += trie_avx512_tmp.extract_objects('trie_avx512.c') + cflags += ['-DCC_DIR24_8_AVX512_SUPPORT', '-DCC_TRIE_AVX512_SUPPORT'] endif diff --git a/lib/fib/rte_fib.c b/lib/fib/rte_fib.c index 4f9fba5a4f..30a55489ad 100644 --- a/lib/fib/rte_fib.c +++ b/lib/fib/rte_fib.c @@ -42,6 +42,7 @@ EAL_REGISTER_TAILQ(rte_fib_tailq) struct rte_fib { char name[RTE_FIB_NAMESIZE]; enum rte_fib_type type; /**< Type of FIB struct */ + int flags; /**< Flags */ struct rte_rib *rib; /**< RIB helper datastructure */ void *dp; /**< pointer to the dataplane struct*/ rte_fib_lookup_fn_t lookup; /**< FIB lookup function */ @@ -110,7 +111,7 @@ init_dataplane(struct rte_fib *fib, __rte_unused int socket_id, if (fib->dp == NULL) return -rte_errno; fib->lookup = dir24_8_get_lookup_fn(fib->dp, - RTE_FIB_LOOKUP_DEFAULT); + RTE_FIB_LOOKUP_DEFAULT, !!(fib->flags & RTE_FIB_F_NETWORK_ORDER)); fib->modify = dir24_8_modify; return 0; default: @@ -161,6 +162,7 @@ rte_fib_create(const char *name, int socket_id, struct rte_fib_conf *conf) /* Check user arguments. */ if ((name == NULL) || (conf == NULL) || (conf->max_routes < 0) || + (conf->flags & ~RTE_FIB_ALLOWED_FLAGS) || (conf->type > RTE_FIB_DIR24_8)) { rte_errno = EINVAL; return NULL; @@ -214,6 +216,7 @@ rte_fib_create(const char *name, int socket_id, struct rte_fib_conf *conf) rte_strlcpy(fib->name, name, sizeof(fib->name)); fib->rib = rib; fib->type = conf->type; + fib->flags = conf->flags; fib->def_nh = conf->default_nh; ret = init_dataplane(fib, socket_id, conf); if (ret < 0) { @@ -329,7 +332,8 @@ rte_fib_select_lookup(struct rte_fib *fib, switch (fib->type) { case RTE_FIB_DIR24_8: - fn = dir24_8_get_lookup_fn(fib->dp, type); + fn = dir24_8_get_lookup_fn(fib->dp, type, + !!(fib->flags & RTE_FIB_F_NETWORK_ORDER)); if (fn == NULL) return -EINVAL; fib->lookup = fn; diff --git a/lib/fib/rte_fib.h b/lib/fib/rte_fib.h index d7a5aafe53..0c3e3e574e 100644 --- a/lib/fib/rte_fib.h +++ b/lib/fib/rte_fib.h @@ -28,6 +28,10 @@ struct rte_rib; /** Maximum depth value possible for IPv4 FIB. */ #define RTE_FIB_MAXDEPTH 32 +/** If set fib lookup is expecting ipv4 in network byte order */ +#define RTE_FIB_F_NETWORK_ORDER 1 +#define RTE_FIB_ALLOWED_FLAGS (RTE_FIB_F_NETWORK_ORDER) + /** Type of FIB struct */ enum rte_fib_type { RTE_FIB_DUMMY, /**< RIB tree based FIB */ @@ -87,6 +91,7 @@ struct rte_fib_conf { uint32_t num_tbl8; } dir24_8; }; + unsigned int flags; /**< Optional feature flags from RTE_FIB_F_* **/ }; /** -- 2.43.0