From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 524F745B69; Fri, 18 Oct 2024 16:06:58 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 18BE54069D; Fri, 18 Oct 2024 16:06:46 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 0CA6A40657 for ; Fri, 18 Oct 2024 16:06:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1729260402; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gj/VOJhBW3iCqmFNs/UO+ydSMLCe4qzIsfI+adpwLNw=; b=fIZMqZpJCuZIsk/dHA7FHAbVCjEQoSrCYOFpguUeP5hjKrcceTIvQsicylCp7DHiJcXjNK XHWUk4Wa70ntHad8bwVc+iUgNRKMPJPRB0koiPelt2H8dWCY7BRTs05ggPhxZTmi1gEW/p GHd0aMUP9vY9jMBlXfJ6Jx8UQDRIgP4= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-15-h45LCfniOmGZlWwK46uP8w-1; Fri, 18 Oct 2024 10:06:40 -0400 X-MC-Unique: h45LCfniOmGZlWwK46uP8w-1 Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 51F5F1955D5A; Fri, 18 Oct 2024 14:06:31 +0000 (UTC) Received: from ringo.redhat.com (unknown [10.39.208.23]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B7D8919560A2; Fri, 18 Oct 2024 14:06:29 +0000 (UTC) From: Robin Jarry To: dev@dpdk.org, Vladimir Medvedkin , Bruce Richardson , Konstantin Ananyev Subject: [PATCH dpdk v5 06/17] fib6: use IPv6 address structure and utils Date: Fri, 18 Oct 2024 16:05:42 +0200 Message-ID: <20241018140553.79789-7-rjarry@redhat.com> In-Reply-To: <20241018140553.79789-1-rjarry@redhat.com> References: <20240821162516.610624-17-rjarry@redhat.com> <20241018140553.79789-1-rjarry@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Replace ad-hoc uint8_t[16] array types in the API of rte_fib6 with rte_ipv6_addr structures. Replace duplicate functions and macros with common ones from rte_ip6.h. Update all code accordingly. Signed-off-by: Robin Jarry --- app/test-fib/main.c | 8 +- app/test/test_fib6.c | 92 +++++++++--------- app/test/test_fib6_perf.c | 8 +- doc/guides/rel_notes/deprecation.rst | 4 - doc/guides/rel_notes/release_24_11.rst | 6 ++ examples/l3fwd/l3fwd_fib.c | 39 ++++---- lib/fib/meson.build | 3 +- lib/fib/rte_fib6.c | 25 ++--- lib/fib/rte_fib6.h | 16 ++-- lib/fib/trie.c | 123 +++++++++++-------------- lib/fib/trie.h | 19 ++-- lib/fib/trie_avx512.c | 38 ++++---- lib/fib/trie_avx512.h | 10 +- 13 files changed, 195 insertions(+), 196 deletions(-) diff --git a/app/test-fib/main.c b/app/test-fib/main.c index 9f45d03d81fb..6479f48cdf6c 100644 --- a/app/test-fib/main.c +++ b/app/test-fib/main.c @@ -1074,7 +1074,7 @@ run_v6(void) for (k = config.print_fract, i = 0; k > 0; k--) { start = rte_rdtsc_precise(); for (j = 0; j < (config.nb_routes - i) / k; j++) { - ret = rte_fib6_add(fib, rt[i + j].addr.a, + ret = rte_fib6_add(fib, &rt[i + j].addr, rt[i + j].depth, rt[i + j].nh); if (unlikely(ret != 0)) { printf("Can not add a route to FIB, err %d\n", @@ -1119,7 +1119,7 @@ run_v6(void) acc = 0; for (i = 0; i < config.nb_lookup_ips; i += BURST_SZ) { start = rte_rdtsc_precise(); - ret = rte_fib6_lookup_bulk(fib, &tbl6[i].a, + ret = rte_fib6_lookup_bulk(fib, &tbl6[i], fib_nh, BURST_SZ); acc += rte_rdtsc_precise() - start; if (ret != 0) { @@ -1146,7 +1146,7 @@ run_v6(void) for (i = 0; i < config.nb_lookup_ips; i += BURST_SZ) { rte_fib6_lookup_bulk(fib, - &tbl6[i].a, + &tbl6[i], fib_nh, BURST_SZ); rte_lpm6_lookup_bulk_func(lpm, &tbl6[i], @@ -1166,7 +1166,7 @@ run_v6(void) for (k = config.print_fract, i = 0; k > 0; k--) { start = rte_rdtsc_precise(); for (j = 0; j < (config.nb_routes - i) / k; j++) - rte_fib6_delete(fib, rt[i + j].addr.a, rt[i + j].depth); + rte_fib6_delete(fib, &rt[i + j].addr, rt[i + j].depth); printf("AVG FIB delete %"PRIu64"\n", (rte_rdtsc_precise() - start) / j); diff --git a/app/test/test_fib6.c b/app/test/test_fib6.c index 2f836238fbf7..79220a88b112 100644 --- a/app/test/test_fib6.c +++ b/app/test/test_fib6.c @@ -147,7 +147,7 @@ test_add_del_invalid(void) struct rte_fib6 *fib = NULL; struct rte_fib6_conf config; uint64_t nh = 100; - uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE] = {0}; + struct rte_ipv6_addr ip = RTE_IPV6_ADDR_UNSPEC; int ret; uint8_t depth = 24; @@ -157,12 +157,12 @@ test_add_del_invalid(void) config.type = RTE_FIB6_DUMMY; /* rte_fib6_add: fib == NULL */ - ret = rte_fib6_add(NULL, ip, depth, nh); + ret = rte_fib6_add(NULL, &ip, depth, nh); RTE_TEST_ASSERT(ret < 0, "Call succeeded with invalid parameters\n"); /* rte_fib6_delete: fib == NULL */ - ret = rte_fib6_delete(NULL, ip, depth); + ret = rte_fib6_delete(NULL, &ip, depth); RTE_TEST_ASSERT(ret < 0, "Call succeeded with invalid parameters\n"); @@ -170,13 +170,13 @@ test_add_del_invalid(void) fib = rte_fib6_create(__func__, SOCKET_ID_ANY, &config); RTE_TEST_ASSERT(fib != NULL, "Failed to create FIB\n"); - /* rte_fib6_add: depth > RTE_FIB6_MAXDEPTH */ - ret = rte_fib6_add(fib, ip, RTE_FIB6_MAXDEPTH + 1, nh); + /* rte_fib6_add: depth > RTE_IPV6_MAX_DEPTH */ + ret = rte_fib6_add(fib, &ip, RTE_IPV6_MAX_DEPTH + 1, nh); RTE_TEST_ASSERT(ret < 0, "Call succeeded with invalid parameters\n"); - /* rte_fib6_delete: depth > RTE_FIB6_MAXDEPTH */ - ret = rte_fib6_delete(fib, ip, RTE_FIB6_MAXDEPTH + 1); + /* rte_fib6_delete: depth > RTE_IPV6_MAX_DEPTH */ + ret = rte_fib6_delete(fib, &ip, RTE_IPV6_MAX_DEPTH + 1); RTE_TEST_ASSERT(ret < 0, "Call succeeded with invalid parameters\n"); @@ -212,22 +212,22 @@ test_get_invalid(void) */ static int lookup_and_check_asc(struct rte_fib6 *fib, - uint8_t ip_arr[RTE_FIB6_MAXDEPTH][RTE_FIB6_IPV6_ADDR_SIZE], - uint8_t ip_missing[][RTE_FIB6_IPV6_ADDR_SIZE], uint64_t def_nh, + struct rte_ipv6_addr *ip_arr, + struct rte_ipv6_addr *ip_missing, uint64_t def_nh, uint32_t n) { - uint64_t nh_arr[RTE_FIB6_MAXDEPTH]; + uint64_t nh_arr[RTE_IPV6_MAX_DEPTH]; int ret; uint32_t i = 0; - ret = rte_fib6_lookup_bulk(fib, ip_arr, nh_arr, RTE_FIB6_MAXDEPTH); + ret = rte_fib6_lookup_bulk(fib, ip_arr, nh_arr, RTE_IPV6_MAX_DEPTH); RTE_TEST_ASSERT(ret == 0, "Failed to lookup\n"); - for (; i <= RTE_FIB6_MAXDEPTH - n; i++) + for (; i <= RTE_IPV6_MAX_DEPTH - n; i++) RTE_TEST_ASSERT(nh_arr[i] == n, "Failed to get proper nexthop\n"); - for (; i < RTE_FIB6_MAXDEPTH; i++) + for (; i < RTE_IPV6_MAX_DEPTH; i++) RTE_TEST_ASSERT(nh_arr[i] == --n, "Failed to get proper nexthop\n"); @@ -240,22 +240,22 @@ lookup_and_check_asc(struct rte_fib6 *fib, static int lookup_and_check_desc(struct rte_fib6 *fib, - uint8_t ip_arr[RTE_FIB6_MAXDEPTH][RTE_FIB6_IPV6_ADDR_SIZE], - uint8_t ip_missing[][RTE_FIB6_IPV6_ADDR_SIZE], uint64_t def_nh, + struct rte_ipv6_addr *ip_arr, + struct rte_ipv6_addr *ip_missing, uint64_t def_nh, uint32_t n) { - uint64_t nh_arr[RTE_FIB6_MAXDEPTH]; + uint64_t nh_arr[RTE_IPV6_MAX_DEPTH]; int ret; uint32_t i = 0; - ret = rte_fib6_lookup_bulk(fib, ip_arr, nh_arr, RTE_FIB6_MAXDEPTH); + ret = rte_fib6_lookup_bulk(fib, ip_arr, nh_arr, RTE_IPV6_MAX_DEPTH); RTE_TEST_ASSERT(ret == 0, "Failed to lookup\n"); for (; i < n; i++) - RTE_TEST_ASSERT(nh_arr[i] == RTE_FIB6_MAXDEPTH - i, + RTE_TEST_ASSERT(nh_arr[i] == RTE_IPV6_MAX_DEPTH - i, "Failed to get proper nexthop\n"); - for (; i < RTE_FIB6_MAXDEPTH; i++) + for (; i < RTE_IPV6_MAX_DEPTH; i++) RTE_TEST_ASSERT(nh_arr[i] == def_nh, "Failed to get proper nexthop\n"); @@ -270,62 +270,64 @@ static int check_fib(struct rte_fib6 *fib) { uint64_t def_nh = 100; - uint8_t ip_arr[RTE_FIB6_MAXDEPTH][RTE_FIB6_IPV6_ADDR_SIZE]; - uint8_t ip_add[RTE_FIB6_IPV6_ADDR_SIZE] = {0}; - uint8_t ip_missing[1][RTE_FIB6_IPV6_ADDR_SIZE] = { {255} }; + struct rte_ipv6_addr ip_arr[RTE_IPV6_MAX_DEPTH]; + struct rte_ipv6_addr ip_add = RTE_IPV6(0x8000, 0, 0, 0, 0, 0, 0, 0); + struct rte_ipv6_addr ip_missing = + RTE_IPV6(0x7fff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff); uint32_t i, j; int ret; - ip_add[0] = 128; - ip_missing[0][0] = 127; - for (i = 0; i < RTE_FIB6_MAXDEPTH; i++) { - for (j = 0; j < RTE_FIB6_IPV6_ADDR_SIZE; j++) { - ip_arr[i][j] = ip_add[j] | - ~get_msk_part(RTE_FIB6_MAXDEPTH - i, j); + for (i = 0; i < RTE_IPV6_MAX_DEPTH; i++) { + ip_arr[i] = ip_add; + j = (RTE_IPV6_MAX_DEPTH - i) / CHAR_BIT; + if (j < RTE_IPV6_ADDR_SIZE) { + ip_arr[i].a[j] |= UINT8_MAX >> ((RTE_IPV6_MAX_DEPTH - i) % CHAR_BIT); + for (j++; j < RTE_IPV6_ADDR_SIZE; j++) + ip_arr[i].a[j] = 0xff; } } - ret = lookup_and_check_desc(fib, ip_arr, ip_missing, def_nh, 0); + ret = lookup_and_check_desc(fib, ip_arr, &ip_missing, def_nh, 0); RTE_TEST_ASSERT(ret == TEST_SUCCESS, "Lookup and check fails\n"); - for (i = 1; i <= RTE_FIB6_MAXDEPTH; i++) { - ret = rte_fib6_add(fib, ip_add, i, i); + for (i = 1; i <= RTE_IPV6_MAX_DEPTH; i++) { + ret = rte_fib6_add(fib, &ip_add, i, i); RTE_TEST_ASSERT(ret == 0, "Failed to add a route\n"); - ret = lookup_and_check_asc(fib, ip_arr, ip_missing, def_nh, i); + ret = lookup_and_check_asc(fib, ip_arr, &ip_missing, def_nh, i); RTE_TEST_ASSERT(ret == TEST_SUCCESS, "Lookup and check fails\n"); } - for (i = RTE_FIB6_MAXDEPTH; i > 1; i--) { - ret = rte_fib6_delete(fib, ip_add, i); + for (i = RTE_IPV6_MAX_DEPTH; i > 1; i--) { + ret = rte_fib6_delete(fib, &ip_add, i); RTE_TEST_ASSERT(ret == 0, "Failed to delete a route\n"); - ret = lookup_and_check_asc(fib, ip_arr, ip_missing, + ret = lookup_and_check_asc(fib, ip_arr, &ip_missing, def_nh, i - 1); RTE_TEST_ASSERT(ret == TEST_SUCCESS, "Lookup and check fails\n"); } - ret = rte_fib6_delete(fib, ip_add, i); + ret = rte_fib6_delete(fib, &ip_add, i); RTE_TEST_ASSERT(ret == 0, "Failed to delete a route\n"); - ret = lookup_and_check_desc(fib, ip_arr, ip_missing, def_nh, 0); + ret = lookup_and_check_desc(fib, ip_arr, &ip_missing, def_nh, 0); RTE_TEST_ASSERT(ret == TEST_SUCCESS, "Lookup and check fails\n"); - for (i = 0; i < RTE_FIB6_MAXDEPTH; i++) { - ret = rte_fib6_add(fib, ip_add, RTE_FIB6_MAXDEPTH - i, - RTE_FIB6_MAXDEPTH - i); + for (i = 0; i < RTE_IPV6_MAX_DEPTH; i++) { + ret = rte_fib6_add(fib, &ip_add, RTE_IPV6_MAX_DEPTH - i, + RTE_IPV6_MAX_DEPTH - i); RTE_TEST_ASSERT(ret == 0, "Failed to add a route\n"); - ret = lookup_and_check_desc(fib, ip_arr, ip_missing, + ret = lookup_and_check_desc(fib, ip_arr, &ip_missing, def_nh, i + 1); RTE_TEST_ASSERT(ret == TEST_SUCCESS, "Lookup and check fails\n"); } - for (i = 1; i <= RTE_FIB6_MAXDEPTH; i++) { - ret = rte_fib6_delete(fib, ip_add, i); + for (i = 1; i <= RTE_IPV6_MAX_DEPTH; i++) { + ret = rte_fib6_delete(fib, &ip_add, i); RTE_TEST_ASSERT(ret == 0, "Failed to delete a route\n"); - ret = lookup_and_check_desc(fib, ip_arr, ip_missing, def_nh, - RTE_FIB6_MAXDEPTH - i); + ret = lookup_and_check_desc(fib, ip_arr, &ip_missing, def_nh, + RTE_IPV6_MAX_DEPTH - i); RTE_TEST_ASSERT(ret == TEST_SUCCESS, "Lookup and check fails\n"); } diff --git a/app/test/test_fib6_perf.c b/app/test/test_fib6_perf.c index f03cd084aa64..a96a0d6b2cdc 100644 --- a/app/test/test_fib6_perf.c +++ b/app/test/test_fib6_perf.c @@ -73,7 +73,7 @@ test_fib6_perf(void) uint64_t next_hop_add; int status = 0; int64_t count = 0; - uint8_t ip_batch[NUM_IPS_ENTRIES][16]; + struct rte_ipv6_addr ip_batch[NUM_IPS_ENTRIES]; uint64_t next_hops[NUM_IPS_ENTRIES]; conf.type = RTE_FIB6_TRIE; @@ -101,7 +101,7 @@ test_fib6_perf(void) for (i = 0; i < NUM_ROUTE_ENTRIES; i++) { next_hop_add = (i & ((1 << 14) - 1)) + 1; - if (rte_fib6_add(fib, large_route_table[i].ip.a, + if (rte_fib6_add(fib, &large_route_table[i].ip, large_route_table[i].depth, next_hop_add) == 0) status++; } @@ -117,7 +117,7 @@ test_fib6_perf(void) count = 0; for (i = 0; i < NUM_IPS_ENTRIES; i++) - memcpy(ip_batch[i], &large_ips_table[i].ip, 16); + ip_batch[i] = large_ips_table[i].ip; for (i = 0; i < ITERATIONS; i++) { @@ -140,7 +140,7 @@ test_fib6_perf(void) for (i = 0; i < NUM_ROUTE_ENTRIES; i++) { /* rte_fib_delete(fib, ip, depth) */ - status += rte_fib6_delete(fib, large_route_table[i].ip.a, + status += rte_fib6_delete(fib, &large_route_table[i].ip, large_route_table[i].depth); } diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index 0b658fce37f7..582d54aece2f 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -70,10 +70,6 @@ Deprecation Notices - ``struct rte_flow_item_icmp6_nd_na`` - ``struct rte_flow_action_set_ipv6`` - ``struct rte_flow_tunnel`` - fib - - ``rte_fib6_add()`` - - ``rte_fib6_delete()`` - - ``rte_fib6_lookup_bulk()`` gro - ``struct tcp6_flow_key`` hash diff --git a/doc/guides/rel_notes/release_24_11.rst b/doc/guides/rel_notes/release_24_11.rst index 452fd7518ee3..2efa3410edc6 100644 --- a/doc/guides/rel_notes/release_24_11.rst +++ b/doc/guides/rel_notes/release_24_11.rst @@ -289,6 +289,12 @@ API Changes * net: A new IPv6 address structure was introduced to replace ad-hoc ``uint8_t[16]`` arrays. The following libraries and symbols were modified: + fib + - ``rte_fib6_add()`` + - ``rte_fib6_delete()`` + - ``rte_fib6_lookup_bulk()`` + - ``RTE_FIB6_IPV6_ADDR_SIZE`` (deprecated, replaced with ``RTE_IPV6_ADDR_SIZE``) + - ``RTE_FIB6_MAXDEPTH`` (deprecated, replaced with ``RTE_IPV6_MAX_DEPTH``) lpm - ``rte_lpm6_add()`` - ``rte_lpm6_delete()`` diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c index 339cd58116a4..a0eef05a5dd8 100644 --- a/examples/l3fwd/l3fwd_fib.c +++ b/examples/l3fwd/l3fwd_fib.c @@ -46,7 +46,7 @@ static struct rte_fib6 *ipv6_l3fwd_fib_lookup_struct[NB_SOCKETS]; static inline void fib_parse_packet(struct rte_mbuf *mbuf, uint32_t *ipv4, uint32_t *ipv4_cnt, - uint8_t ipv6[RTE_FIB6_IPV6_ADDR_SIZE], + struct rte_ipv6_addr *ipv6, uint32_t *ipv6_cnt, uint8_t *ip_type) { struct rte_ether_hdr *eth_hdr; @@ -65,7 +65,7 @@ fib_parse_packet(struct rte_mbuf *mbuf, /* IPv6 */ else { ipv6_hdr = (struct rte_ipv6_hdr *)(eth_hdr + 1); - rte_mov16(ipv6, ipv6_hdr->dst_addr.a); + *ipv6 = ipv6_hdr->dst_addr; *ip_type = 0; (*ipv6_cnt)++; } @@ -120,7 +120,7 @@ fib_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, uint16_t portid, struct lcore_conf *qconf) { uint32_t ipv4_arr[nb_rx]; - uint8_t ipv6_arr[nb_rx][RTE_FIB6_IPV6_ADDR_SIZE]; + struct rte_ipv6_addr ipv6_arr[nb_rx]; uint16_t hops[nb_rx]; uint64_t hopsv4[nb_rx], hopsv6[nb_rx]; uint8_t type_arr[nb_rx]; @@ -140,7 +140,7 @@ fib_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, i + FIB_PREFETCH_OFFSET], void *)); fib_parse_packet(pkts_burst[i], &ipv4_arr[ipv4_cnt], &ipv4_cnt, - ipv6_arr[ipv6_cnt], &ipv6_cnt, + &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); } @@ -148,7 +148,7 @@ fib_send_packets(int nb_rx, struct rte_mbuf **pkts_burst, for (; i < nb_rx; i++) fib_parse_packet(pkts_burst[i], &ipv4_arr[ipv4_cnt], &ipv4_cnt, - ipv6_arr[ipv6_cnt], &ipv6_cnt, + &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); /* Lookup IPv4 hops if IPv4 packets are present. */ @@ -270,7 +270,7 @@ fib_event_loop(struct l3fwd_event_resources *evt_rsrc, unsigned int lcore_id; uint32_t ipv4_arr[MAX_PKT_BURST]; - uint8_t ipv6_arr[MAX_PKT_BURST][RTE_FIB6_IPV6_ADDR_SIZE]; + struct rte_ipv6_addr ipv6_arr[MAX_PKT_BURST]; uint64_t hopsv4[MAX_PKT_BURST], hopsv6[MAX_PKT_BURST]; uint16_t nh, hops[MAX_PKT_BURST]; uint8_t type_arr[MAX_PKT_BURST]; @@ -323,7 +323,7 @@ fib_event_loop(struct l3fwd_event_resources *evt_rsrc, fib_parse_packet(events[i].mbuf, &ipv4_arr[ipv4_cnt], &ipv4_cnt, - ipv6_arr[ipv6_cnt], &ipv6_cnt, + &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); } @@ -340,7 +340,7 @@ fib_event_loop(struct l3fwd_event_resources *evt_rsrc, fib_parse_packet(events[i].mbuf, &ipv4_arr[ipv4_cnt], &ipv4_cnt, - ipv6_arr[ipv6_cnt], &ipv6_cnt, + &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); } @@ -436,7 +436,7 @@ fib_event_main_loop_tx_q_burst(__rte_unused void *dummy) static __rte_always_inline void fib_process_event_vector(struct rte_event_vector *vec, uint8_t *type_arr, - uint8_t **ipv6_arr, uint64_t *hopsv4, uint64_t *hopsv6, + struct rte_ipv6_addr *ipv6_arr, uint64_t *hopsv4, uint64_t *hopsv6, uint32_t *ipv4_arr, uint16_t *hops) { uint32_t ipv4_arr_assem, ipv6_arr_assem; @@ -463,13 +463,13 @@ fib_process_event_vector(struct rte_event_vector *vec, uint8_t *type_arr, rte_prefetch0(rte_pktmbuf_mtod(mbufs[i + FIB_PREFETCH_OFFSET], void *)); fib_parse_packet(mbufs[i], &ipv4_arr[ipv4_cnt], &ipv4_cnt, - ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); + &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); } /* Parse remaining packet info. */ for (; i < vec->nb_elem; i++) fib_parse_packet(mbufs[i], &ipv4_arr[ipv4_cnt], &ipv4_cnt, - ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); + &ipv6_arr[ipv6_cnt], &ipv6_cnt, &type_arr[i]); /* Lookup IPv4 hops if IPv4 packets are present. */ if (likely(ipv4_cnt > 0)) @@ -480,7 +480,7 @@ fib_process_event_vector(struct rte_event_vector *vec, uint8_t *type_arr, if (ipv6_cnt > 0) rte_fib6_lookup_bulk( lconf->ipv6_lookup_struct, - (uint8_t(*)[RTE_FIB6_IPV6_ADDR_SIZE])ipv6_arr, hopsv6, + ipv6_arr, hopsv6, ipv6_cnt); /* Assign ports looked up in fib depending on IPv4 or IPv6 */ @@ -522,7 +522,8 @@ fib_event_loop_vector(struct l3fwd_event_resources *evt_rsrc, const uint8_t event_d_id = evt_rsrc->event_d_id; const uint16_t deq_len = evt_rsrc->deq_depth; struct rte_event events[MAX_PKT_BURST]; - uint8_t *type_arr, **ipv6_arr, *ptr; + uint8_t *type_arr; + struct rte_ipv6_addr *ipv6_arr; int nb_enq = 0, nb_deq = 0, i; uint64_t *hopsv4, *hopsv6; uint32_t *ipv4_arr; @@ -533,7 +534,7 @@ fib_event_loop_vector(struct l3fwd_event_resources *evt_rsrc, "vector_fib", (sizeof(uint32_t) + sizeof(uint8_t) + sizeof(uint64_t) + sizeof(uint64_t) + sizeof(uint16_t) + sizeof(uint8_t *) + - (sizeof(uint8_t) * RTE_FIB6_IPV6_ADDR_SIZE)) * + sizeof(struct rte_ipv6_addr)) * evt_rsrc->vector_size, RTE_CACHE_LINE_SIZE); if (mem == 0) @@ -543,11 +544,7 @@ fib_event_loop_vector(struct l3fwd_event_resources *evt_rsrc, hopsv4 = (uint64_t *)&type_arr[evt_rsrc->vector_size]; hopsv6 = (uint64_t *)&hopsv4[evt_rsrc->vector_size]; hops = (uint16_t *)&hopsv6[evt_rsrc->vector_size]; - ipv6_arr = (uint8_t **)&hops[evt_rsrc->vector_size]; - - ptr = (uint8_t *)&ipv6_arr[evt_rsrc->vector_size]; - for (i = 0; i < evt_rsrc->vector_size; i++) - ipv6_arr[i] = &ptr[RTE_FIB6_IPV6_ADDR_SIZE + i]; + ipv6_arr = (struct rte_ipv6_addr *)&hops[evt_rsrc->vector_size]; if (event_p_id < 0) { rte_free((void *)mem); @@ -732,7 +729,7 @@ setup_fib(const int socketid) rte_eth_dev_info_get(route_base_v6[i].if_out, &dev_info); ret = rte_fib6_add(ipv6_l3fwd_fib_lookup_struct[socketid], - route_base_v6[i].ip6.a, + &route_base_v6[i].ip6, route_base_v6[i].depth, route_base_v6[i].if_out); @@ -744,7 +741,7 @@ setup_fib(const int socketid) i, socketid); } - if (inet_ntop(AF_INET6, route_base_v6[i].ip6.a, + if (inet_ntop(AF_INET6, &route_base_v6[i].ip6, abuf, sizeof(abuf)) != NULL) { printf("FIB: Adding route %s / %d (%d) [%s]\n", abuf, route_base_v6[i].depth, diff --git a/lib/fib/meson.build b/lib/fib/meson.build index 0004797dd5ac..0c19cc8201bf 100644 --- a/lib/fib/meson.build +++ b/lib/fib/meson.build @@ -12,6 +12,7 @@ sources = files('rte_fib.c', 'rte_fib6.c', 'dir24_8.c', 'trie.c') headers = files('rte_fib.h', 'rte_fib6.h') deps += ['rib'] deps += ['rcu'] +deps += ['net'] if dpdk_conf.has('RTE_ARCH_X86_64') if target_has_avx512 @@ -27,7 +28,7 @@ if dpdk_conf.has('RTE_ARCH_X86_64') objs += dir24_8_avx512_tmp.extract_objects('dir24_8_avx512.c') trie_avx512_tmp = static_library('trie_avx512_tmp', 'trie_avx512.c', - dependencies: [static_rte_eal, static_rte_rcu], + dependencies: [static_rte_eal, static_rte_rcu, static_rte_net], c_args: cflags + cc_avx512_flags) objs += trie_avx512_tmp.extract_objects('trie_avx512.c') endif diff --git a/lib/fib/rte_fib6.c b/lib/fib/rte_fib6.c index 9ad990724a8b..ef334da67cc4 100644 --- a/lib/fib/rte_fib6.c +++ b/lib/fib/rte_fib6.c @@ -14,6 +14,7 @@ #include #include +#include #include #include @@ -49,7 +50,7 @@ struct rte_fib6 { }; static void -dummy_lookup(void *fib_p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], +dummy_lookup(void *fib_p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n) { unsigned int i; @@ -57,7 +58,7 @@ dummy_lookup(void *fib_p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], struct rte_rib6_node *node; for (i = 0; i < n; i++) { - node = rte_rib6_lookup(fib->rib, ips[i]); + node = rte_rib6_lookup(fib->rib, ips[i].a); if (node != NULL) rte_rib6_get_nh(node, &next_hops[i]); else @@ -66,26 +67,26 @@ dummy_lookup(void *fib_p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], } static int -dummy_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], +dummy_modify(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip, uint8_t depth, uint64_t next_hop, int op) { struct rte_rib6_node *node; - if ((fib == NULL) || (depth > RTE_FIB6_MAXDEPTH)) + if ((fib == NULL) || (depth > RTE_IPV6_MAX_DEPTH)) return -EINVAL; - node = rte_rib6_lookup_exact(fib->rib, ip, depth); + node = rte_rib6_lookup_exact(fib->rib, ip->a, depth); switch (op) { case RTE_FIB6_ADD: if (node == NULL) - node = rte_rib6_insert(fib->rib, ip, depth); + node = rte_rib6_insert(fib->rib, ip->a, depth); if (node == NULL) return -rte_errno; return rte_rib6_set_nh(node, next_hop); case RTE_FIB6_DEL: if (node == NULL) return -ENOENT; - rte_rib6_remove(fib->rib, ip, depth); + rte_rib6_remove(fib->rib, ip->a, depth); return 0; } return -EINVAL; @@ -118,28 +119,28 @@ init_dataplane(struct rte_fib6 *fib, __rte_unused int socket_id, } int -rte_fib6_add(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], +rte_fib6_add(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip, uint8_t depth, uint64_t next_hop) { if ((fib == NULL) || (ip == NULL) || (fib->modify == NULL) || - (depth > RTE_FIB6_MAXDEPTH)) + (depth > RTE_IPV6_MAX_DEPTH)) return -EINVAL; return fib->modify(fib, ip, depth, next_hop, RTE_FIB6_ADD); } int -rte_fib6_delete(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], +rte_fib6_delete(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip, uint8_t depth) { if ((fib == NULL) || (ip == NULL) || (fib->modify == NULL) || - (depth > RTE_FIB6_MAXDEPTH)) + (depth > RTE_IPV6_MAX_DEPTH)) return -EINVAL; return fib->modify(fib, ip, depth, 0, RTE_FIB6_DEL); } int rte_fib6_lookup_bulk(struct rte_fib6 *fib, - uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], + const struct rte_ipv6_addr *ips, uint64_t *next_hops, int n) { FIB6_RETURN_IF_TRUE((fib == NULL) || (ips == NULL) || diff --git a/lib/fib/rte_fib6.h b/lib/fib/rte_fib6.h index 2eb8b8267647..21f0492374d9 100644 --- a/lib/fib/rte_fib6.h +++ b/lib/fib/rte_fib6.h @@ -17,14 +17,16 @@ #include +#include +#include #ifdef __cplusplus extern "C" { #endif -#define RTE_FIB6_IPV6_ADDR_SIZE 16 +#define RTE_FIB6_IPV6_ADDR_SIZE (RTE_DEPRECATED(RTE_FIB6_IPV6_ADDR_SIZE) RTE_IPV6_ADDR_SIZE) /** Maximum depth value possible for IPv6 FIB. */ -#define RTE_FIB6_MAXDEPTH 128 +#define RTE_FIB6_MAXDEPTH (RTE_DEPRECATED(RTE_FIB6_MAXDEPTH) RTE_IPV6_MAX_DEPTH) struct rte_fib6; struct rte_rib6; @@ -37,11 +39,11 @@ enum rte_fib6_type { /** Modify FIB function */ typedef int (*rte_fib6_modify_fn_t)(struct rte_fib6 *fib, - const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], uint8_t depth, + const struct rte_ipv6_addr *ip, uint8_t depth, uint64_t next_hop, int op); /** FIB bulk lookup function */ typedef void (*rte_fib6_lookup_fn_t)(void *fib, - uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], + const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n); enum rte_fib6_op { @@ -134,7 +136,7 @@ rte_fib6_free(struct rte_fib6 *fib); * 0 on success, negative value otherwise */ int -rte_fib6_add(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], +rte_fib6_add(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip, uint8_t depth, uint64_t next_hop); /** @@ -151,7 +153,7 @@ rte_fib6_add(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], */ int rte_fib6_delete(struct rte_fib6 *fib, - const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], uint8_t depth); + const struct rte_ipv6_addr *ip, uint8_t depth); /** * Lookup multiple IP addresses in the FIB. @@ -172,7 +174,7 @@ rte_fib6_delete(struct rte_fib6 *fib, */ int rte_fib6_lookup_bulk(struct rte_fib6 *fib, - uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], + const struct rte_ipv6_addr *ips, uint64_t *next_hops, int n); /** diff --git a/lib/fib/trie.c b/lib/fib/trie.c index ca1c2fe3bce3..6bb46541feee 100644 --- a/lib/fib/trie.c +++ b/lib/fib/trie.c @@ -227,7 +227,7 @@ tbl8_recycle(struct rte_trie_tbl *dp, void *par, uint64_t tbl8_idx) #define BYTE_SIZE 8 static inline uint32_t -get_idx(const uint8_t *ip, uint32_t prev_idx, int bytes, int first_byte) +get_idx(const struct rte_ipv6_addr *ip, uint32_t prev_idx, int bytes, int first_byte) { int i; uint32_t idx = 0; @@ -235,7 +235,7 @@ get_idx(const uint8_t *ip, uint32_t prev_idx, int bytes, int first_byte) for (i = first_byte; i < (first_byte + bytes); i++) { bitshift = (int8_t)(((first_byte + bytes - 1) - i)*BYTE_SIZE); - idx |= ip[i] << bitshift; + idx |= ip->a[i] << bitshift; } return (prev_idx * TRIE_TBL8_GRP_NUM_ENT) + idx; } @@ -282,7 +282,7 @@ recycle_root_path(struct rte_trie_tbl *dp, const uint8_t *ip_part, } static inline int -build_common_root(struct rte_trie_tbl *dp, const uint8_t *ip, +build_common_root(struct rte_trie_tbl *dp, const struct rte_ipv6_addr *ip, int common_bytes, void **tbl) { void *tbl_ptr = NULL; @@ -352,13 +352,13 @@ write_edge(struct rte_trie_tbl *dp, const uint8_t *ip_part, uint64_t next_hop, return ret; } -#define IPV6_MAX_IDX (RTE_FIB6_IPV6_ADDR_SIZE - 1) +#define IPV6_MAX_IDX (RTE_IPV6_ADDR_SIZE - 1) #define TBL24_BYTES 3 -#define TBL8_LEN (RTE_FIB6_IPV6_ADDR_SIZE - TBL24_BYTES) +#define TBL8_LEN (RTE_IPV6_ADDR_SIZE - TBL24_BYTES) static int -install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r, - uint64_t next_hop) +install_to_dp(struct rte_trie_tbl *dp, const struct rte_ipv6_addr *ledge, + const struct rte_ipv6_addr *r, uint64_t next_hop) { void *common_root_tbl; void *ent; @@ -366,18 +366,18 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r, int i; int common_bytes; int llen, rlen; - uint8_t redge[16]; + struct rte_ipv6_addr redge; /* decrement redge by 1*/ - rte_rib6_copy_addr(redge, r); + redge = *r; for (i = 15; i >= 0; i--) { - redge[i]--; - if (redge[i] != 0xff) + redge.a[i]--; + if (redge.a[i] != 0xff) break; } for (common_bytes = 0; common_bytes < 15; common_bytes++) { - if (ledge[common_bytes] != redge[common_bytes]) + if (ledge->a[common_bytes] != redge.a[common_bytes]) break; } @@ -388,14 +388,14 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r, uint8_t first_tbl8_byte = RTE_MAX(common_bytes, TBL24_BYTES); for (i = IPV6_MAX_IDX; i > first_tbl8_byte; i--) { - if (ledge[i] != 0) + if (ledge->a[i] != 0) break; } llen = i - first_tbl8_byte + (common_bytes < 3); for (i = IPV6_MAX_IDX; i > first_tbl8_byte; i--) { - if (redge[i] != UINT8_MAX) + if (redge.a[i] != UINT8_MAX) break; } rlen = i - first_tbl8_byte + (common_bytes < 3); @@ -405,10 +405,10 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r, uint8_t first_idx_len = (common_bytes < 3) ? 3 : 1; uint32_t left_idx = get_idx(ledge, 0, first_idx_len, first_byte_idx); - uint32_t right_idx = get_idx(redge, 0, first_idx_len, first_byte_idx); + uint32_t right_idx = get_idx(&redge, 0, first_idx_len, first_byte_idx); ent = get_tbl_p_by_idx(common_root_tbl, left_idx, dp->nh_sz); - ret = write_edge(dp, &ledge[first_tbl8_byte + !(common_bytes < 3)], + ret = write_edge(dp, &ledge->a[first_tbl8_byte + !(common_bytes < 3)], next_hop, llen, LEDGE, ent); if (ret < 0) return ret; @@ -420,7 +420,7 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r, right_idx - (left_idx + 1)); } ent = get_tbl_p_by_idx(common_root_tbl, right_idx, dp->nh_sz); - ret = write_edge(dp, &redge[first_tbl8_byte + !((common_bytes < 3))], + ret = write_edge(dp, &redge.a[first_tbl8_byte + !((common_bytes < 3))], next_hop, rlen, REDGE, ent); if (ret < 0) return ret; @@ -428,12 +428,12 @@ install_to_dp(struct rte_trie_tbl *dp, const uint8_t *ledge, const uint8_t *r, uint8_t common_tbl8 = (common_bytes < TBL24_BYTES) ? 0 : common_bytes - (TBL24_BYTES - 1); ent = get_tbl24_p(dp, ledge, dp->nh_sz); - recycle_root_path(dp, ledge + TBL24_BYTES, common_tbl8, ent); + recycle_root_path(dp, ledge->a + TBL24_BYTES, common_tbl8, ent); return 0; } static void -get_nxt_net(uint8_t *ip, uint8_t depth) +get_nxt_net(struct rte_ipv6_addr *ip, uint8_t depth) { int i; uint8_t part_depth; @@ -442,73 +442,62 @@ get_nxt_net(uint8_t *ip, uint8_t depth) for (i = 0, part_depth = depth; part_depth > 8; part_depth -= 8, i++) ; - prev_byte = ip[i]; - ip[i] += 1 << (8 - part_depth); - if (ip[i] < prev_byte) { + prev_byte = ip->a[i]; + ip->a[i] += 1 << (8 - part_depth); + if (ip->a[i] < prev_byte) { while (i > 0) { - ip[--i] += 1; - if (ip[i] != 0) + ip->a[--i] += 1; + if (ip->a[i] != 0) break; } } } -static int -v6_addr_is_zero(const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE]) -{ - uint8_t ip_addr[RTE_FIB6_IPV6_ADDR_SIZE] = {0}; - - return rte_rib6_is_equal(ip, ip_addr); -} - static int modify_dp(struct rte_trie_tbl *dp, struct rte_rib6 *rib, - const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], + const struct rte_ipv6_addr *ip, uint8_t depth, uint64_t next_hop) { struct rte_rib6_node *tmp = NULL; - uint8_t ledge[RTE_FIB6_IPV6_ADDR_SIZE]; - uint8_t redge[RTE_FIB6_IPV6_ADDR_SIZE]; + struct rte_ipv6_addr ledge, redge; int ret; uint8_t tmp_depth; if (next_hop > get_max_nh(dp->nh_sz)) return -EINVAL; - rte_rib6_copy_addr(ledge, ip); + ledge = *ip; do { - tmp = rte_rib6_get_nxt(rib, ip, depth, tmp, + tmp = rte_rib6_get_nxt(rib, ip->a, depth, tmp, RTE_RIB6_GET_NXT_COVER); if (tmp != NULL) { rte_rib6_get_depth(tmp, &tmp_depth); if (tmp_depth == depth) continue; - rte_rib6_get_ip(tmp, redge); - if (rte_rib6_is_equal(ledge, redge)) { - get_nxt_net(ledge, tmp_depth); + rte_rib6_get_ip(tmp, redge.a); + if (rte_ipv6_addr_eq(&ledge, &redge)) { + get_nxt_net(&ledge, tmp_depth); continue; } - ret = install_to_dp(dp, ledge, redge, - next_hop); + ret = install_to_dp(dp, &ledge, &redge, next_hop); if (ret != 0) return ret; - get_nxt_net(redge, tmp_depth); - rte_rib6_copy_addr(ledge, redge); + get_nxt_net(&redge, tmp_depth); + ledge = redge; /* * we got to the end of address space * and wrapped around */ - if (v6_addr_is_zero(ledge)) + if (rte_ipv6_addr_is_unspec(&ledge)) break; } else { - rte_rib6_copy_addr(redge, ip); - get_nxt_net(redge, depth); - if (rte_rib6_is_equal(ledge, redge) && - !v6_addr_is_zero(ledge)) + redge = *ip; + get_nxt_net(&redge, depth); + if (rte_ipv6_addr_eq(&ledge, &redge) && + !rte_ipv6_addr_is_unspec(&ledge)) break; - ret = install_to_dp(dp, ledge, redge, - next_hop); + ret = install_to_dp(dp, &ledge, &redge, next_hop); if (ret != 0) return ret; } @@ -518,7 +507,7 @@ modify_dp(struct rte_trie_tbl *dp, struct rte_rib6 *rib, } int -trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], +trie_modify(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip, uint8_t depth, uint64_t next_hop, int op) { struct rte_trie_tbl *dp; @@ -526,12 +515,12 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], struct rte_rib6_node *tmp = NULL; struct rte_rib6_node *node; struct rte_rib6_node *parent; - uint8_t ip_masked[RTE_FIB6_IPV6_ADDR_SIZE]; - int i, ret = 0; + struct rte_ipv6_addr ip_masked; + int ret = 0; uint64_t par_nh, node_nh; uint8_t tmp_depth, depth_diff = 0, parent_depth = 24; - if ((fib == NULL) || (ip == NULL) || (depth > RTE_FIB6_MAXDEPTH)) + if ((fib == NULL) || (ip == NULL) || (depth > RTE_IPV6_MAX_DEPTH)) return -EINVAL; dp = rte_fib6_get_dp(fib); @@ -539,15 +528,15 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], rib = rte_fib6_get_rib(fib); RTE_ASSERT(rib); - for (i = 0; i < RTE_FIB6_IPV6_ADDR_SIZE; i++) - ip_masked[i] = ip[i] & get_msk_part(depth, i); + ip_masked = *ip; + rte_ipv6_addr_mask(&ip_masked, depth); if (depth > 24) { - tmp = rte_rib6_get_nxt(rib, ip_masked, + tmp = rte_rib6_get_nxt(rib, ip_masked.a, RTE_ALIGN_FLOOR(depth, 8), NULL, RTE_RIB6_GET_NXT_COVER); if (tmp == NULL) { - tmp = rte_rib6_lookup(rib, ip); + tmp = rte_rib6_lookup(rib, ip->a); if (tmp != NULL) { rte_rib6_get_depth(tmp, &tmp_depth); parent_depth = RTE_MAX(tmp_depth, 24); @@ -557,14 +546,14 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], depth_diff = depth_diff >> 3; } } - node = rte_rib6_lookup_exact(rib, ip_masked, depth); + node = rte_rib6_lookup_exact(rib, ip_masked.a, depth); switch (op) { case RTE_FIB6_ADD: if (node != NULL) { rte_rib6_get_nh(node, &node_nh); if (node_nh == next_hop) return 0; - ret = modify_dp(dp, rib, ip_masked, depth, next_hop); + ret = modify_dp(dp, rib, &ip_masked, depth, next_hop); if (ret == 0) rte_rib6_set_nh(node, next_hop); return 0; @@ -574,7 +563,7 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], dp->number_tbl8s - depth_diff)) return -ENOSPC; - node = rte_rib6_insert(rib, ip_masked, depth); + node = rte_rib6_insert(rib, ip_masked.a, depth); if (node == NULL) return -rte_errno; rte_rib6_set_nh(node, next_hop); @@ -584,9 +573,9 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], if (par_nh == next_hop) return 0; } - ret = modify_dp(dp, rib, ip_masked, depth, next_hop); + ret = modify_dp(dp, rib, &ip_masked, depth, next_hop); if (ret != 0) { - rte_rib6_remove(rib, ip_masked, depth); + rte_rib6_remove(rib, ip_masked.a, depth); return ret; } @@ -601,14 +590,14 @@ trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], rte_rib6_get_nh(parent, &par_nh); rte_rib6_get_nh(node, &node_nh); if (par_nh != node_nh) - ret = modify_dp(dp, rib, ip_masked, depth, + ret = modify_dp(dp, rib, &ip_masked, depth, par_nh); } else - ret = modify_dp(dp, rib, ip_masked, depth, dp->def_nh); + ret = modify_dp(dp, rib, &ip_masked, depth, dp->def_nh); if (ret != 0) return ret; - rte_rib6_remove(rib, ip, depth); + rte_rib6_remove(rib, ip->a, depth); dp->rsvd_tbl8s -= depth_diff; return 0; diff --git a/lib/fib/trie.h b/lib/fib/trie.h index 2c20184a26a3..f87fc0f6d2a7 100644 --- a/lib/fib/trie.h +++ b/lib/fib/trie.h @@ -8,6 +8,9 @@ #include +#include +#include + /** * @file * RTE IPv6 Longest Prefix Match (LPM) @@ -15,8 +18,6 @@ /* @internal Total number of tbl24 entries. */ #define TRIE_TBL24_NUM_ENT (1 << 24) -/* Maximum depth value possible for IPv6 LPM. */ -#define TRIE_MAX_DEPTH 128 /* @internal Number of entries in a tbl8 group. */ #define TRIE_TBL8_GRP_NUM_ENT 256ULL /* @internal Total number of tbl8 groups in the tbl8. */ @@ -42,13 +43,13 @@ struct rte_trie_tbl { }; static inline uint32_t -get_tbl24_idx(const uint8_t *ip) +get_tbl24_idx(const struct rte_ipv6_addr *ip) { - return ip[0] << 16|ip[1] << 8|ip[2]; + return ip->a[0] << 16|ip->a[1] << 8|ip->a[2]; } static inline void * -get_tbl24_p(struct rte_trie_tbl *dp, const uint8_t *ip, uint8_t nh_sz) +get_tbl24_p(struct rte_trie_tbl *dp, const struct rte_ipv6_addr *ip, uint8_t nh_sz) { uint32_t tbl24_idx; @@ -107,7 +108,7 @@ is_entry_extended(uint64_t ent) #define LOOKUP_FUNC(suffix, type, nh_sz) \ static inline void rte_trie_lookup_bulk_##suffix(void *p, \ - uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], \ + const struct rte_ipv6_addr *ips, \ uint64_t *next_hops, const unsigned int n) \ { \ struct rte_trie_tbl *dp = (struct rte_trie_tbl *)p; \ @@ -115,10 +116,10 @@ static inline void rte_trie_lookup_bulk_##suffix(void *p, \ uint32_t i, j; \ \ for (i = 0; i < n; i++) { \ - tmp = ((type *)dp->tbl24)[get_tbl24_idx(&ips[i][0])]; \ + tmp = ((type *)dp->tbl24)[get_tbl24_idx(&ips[i])]; \ j = 3; \ while (is_entry_extended(tmp)) { \ - tmp = ((type *)dp->tbl8)[ips[i][j++] + \ + tmp = ((type *)dp->tbl8)[ips[i].a[j++] + \ ((tmp >> 1) * TRIE_TBL8_GRP_NUM_ENT)]; \ } \ next_hops[i] = tmp >> 1; \ @@ -138,7 +139,7 @@ rte_fib6_lookup_fn_t trie_get_lookup_fn(void *p, enum rte_fib6_lookup_type type); int -trie_modify(struct rte_fib6 *fib, const uint8_t ip[RTE_FIB6_IPV6_ADDR_SIZE], +trie_modify(struct rte_fib6 *fib, const struct rte_ipv6_addr *ip, uint8_t depth, uint64_t next_hop, int op); #endif /* _TRIE_H_ */ diff --git a/lib/fib/trie_avx512.c b/lib/fib/trie_avx512.c index d4d70d84bf60..f49482a95dfa 100644 --- a/lib/fib/trie_avx512.c +++ b/lib/fib/trie_avx512.c @@ -9,7 +9,7 @@ #include "trie_avx512.h" static __rte_always_inline void -transpose_x16(uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE], +transpose_x16(const struct rte_ipv6_addr *ips, __m512i *first, __m512i *second, __m512i *third, __m512i *fourth) { __m512i tmp1, tmp2, tmp3, tmp4; @@ -21,10 +21,10 @@ transpose_x16(uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE], }; /* load all ip addresses */ - tmp1 = _mm512_loadu_si512(&ips[0][0]); - tmp2 = _mm512_loadu_si512(&ips[4][0]); - tmp3 = _mm512_loadu_si512(&ips[8][0]); - tmp4 = _mm512_loadu_si512(&ips[12][0]); + tmp1 = _mm512_loadu_si512(&ips[0]); + tmp2 = _mm512_loadu_si512(&ips[4]); + tmp3 = _mm512_loadu_si512(&ips[8]); + tmp4 = _mm512_loadu_si512(&ips[12]); /* transpose 4 byte chunks of 16 ips */ tmp5 = _mm512_unpacklo_epi32(tmp1, tmp2); @@ -48,7 +48,7 @@ transpose_x16(uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE], } static __rte_always_inline void -transpose_x8(uint8_t ips[8][RTE_FIB6_IPV6_ADDR_SIZE], +transpose_x8(const struct rte_ipv6_addr *ips, __m512i *first, __m512i *second) { __m512i tmp1, tmp2, tmp3, tmp4; @@ -57,8 +57,8 @@ transpose_x8(uint8_t ips[8][RTE_FIB6_IPV6_ADDR_SIZE], }, }; - tmp1 = _mm512_loadu_si512(&ips[0][0]); - tmp2 = _mm512_loadu_si512(&ips[4][0]); + tmp1 = _mm512_loadu_si512(&ips[0]); + tmp2 = _mm512_loadu_si512(&ips[4]); tmp3 = _mm512_unpacklo_epi64(tmp1, tmp2); *first = _mm512_permutexvar_epi64(perm_idxes.z, tmp3); @@ -67,7 +67,7 @@ transpose_x8(uint8_t ips[8][RTE_FIB6_IPV6_ADDR_SIZE], } static __rte_always_inline void -trie_vec_lookup_x16x2(void *p, uint8_t ips[32][RTE_FIB6_IPV6_ADDR_SIZE], +trie_vec_lookup_x16x2(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, int size) { struct rte_trie_tbl *dp = (struct rte_trie_tbl *)p; @@ -213,7 +213,7 @@ trie_vec_lookup_x16x2(void *p, uint8_t ips[32][RTE_FIB6_IPV6_ADDR_SIZE], } static void -trie_vec_lookup_x8x2_8b(void *p, uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE], +trie_vec_lookup_x8x2_8b(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops) { struct rte_trie_tbl *dp = (struct rte_trie_tbl *)p; @@ -306,40 +306,40 @@ trie_vec_lookup_x8x2_8b(void *p, uint8_t ips[16][RTE_FIB6_IPV6_ADDR_SIZE], } void -rte_trie_vec_lookup_bulk_2b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], +rte_trie_vec_lookup_bulk_2b(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n) { uint32_t i; for (i = 0; i < (n / 32); i++) { - trie_vec_lookup_x16x2(p, (uint8_t (*)[16])&ips[i * 32][0], + trie_vec_lookup_x16x2(p, &ips[i * 32], next_hops + i * 32, sizeof(uint16_t)); } - rte_trie_lookup_bulk_2b(p, (uint8_t (*)[16])&ips[i * 32][0], + rte_trie_lookup_bulk_2b(p, &ips[i * 32], next_hops + i * 32, n - i * 32); } void -rte_trie_vec_lookup_bulk_4b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], +rte_trie_vec_lookup_bulk_4b(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n) { uint32_t i; for (i = 0; i < (n / 32); i++) { - trie_vec_lookup_x16x2(p, (uint8_t (*)[16])&ips[i * 32][0], + trie_vec_lookup_x16x2(p, &ips[i * 32], next_hops + i * 32, sizeof(uint32_t)); } - rte_trie_lookup_bulk_4b(p, (uint8_t (*)[16])&ips[i * 32][0], + rte_trie_lookup_bulk_4b(p, &ips[i * 32], next_hops + i * 32, n - i * 32); } void -rte_trie_vec_lookup_bulk_8b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], +rte_trie_vec_lookup_bulk_8b(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n) { uint32_t i; for (i = 0; i < (n / 16); i++) { - trie_vec_lookup_x8x2_8b(p, (uint8_t (*)[16])&ips[i * 16][0], + trie_vec_lookup_x8x2_8b(p, &ips[i * 16], next_hops + i * 16); } - rte_trie_lookup_bulk_8b(p, (uint8_t (*)[16])&ips[i * 16][0], + rte_trie_lookup_bulk_8b(p, &ips[i * 16], next_hops + i * 16, n - i * 16); } diff --git a/lib/fib/trie_avx512.h b/lib/fib/trie_avx512.h index ef8c7f0e3474..1028a4899fc7 100644 --- a/lib/fib/trie_avx512.h +++ b/lib/fib/trie_avx512.h @@ -5,16 +5,20 @@ #ifndef _TRIE_AVX512_H_ #define _TRIE_AVX512_H_ +#include + +struct rte_ipv6_addr; + void -rte_trie_vec_lookup_bulk_2b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], +rte_trie_vec_lookup_bulk_2b(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n); void -rte_trie_vec_lookup_bulk_4b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], +rte_trie_vec_lookup_bulk_4b(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n); void -rte_trie_vec_lookup_bulk_8b(void *p, uint8_t ips[][RTE_FIB6_IPV6_ADDR_SIZE], +rte_trie_vec_lookup_bulk_8b(void *p, const struct rte_ipv6_addr *ips, uint64_t *next_hops, const unsigned int n); #endif /* _TRIE_AVX512_H_ */ -- 2.47.0