From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7959C42B83; Tue, 23 May 2023 18:23:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 522D640A80; Tue, 23 May 2023 18:23:13 +0200 (CEST) Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2049.outbound.protection.outlook.com [40.107.21.49]) by mails.dpdk.org (Postfix) with ESMTP id D906840689 for ; Tue, 23 May 2023 18:23:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5VVbgsNtZjaUC3sJQ7BPq9QrRwgakCnTvHzDNAbRo+w=; b=G+YEWyowaj7c46zfruk80RjgkJfVJXuVaXgH3yx7z3e6OdwCIG2O5cisXPfjXbFglizWEfGgzasRbXpkttBAtFT23b5id29MhCmDqupwA6FmmHCTaL5F/dwDAkv7husKe/fVUrBCoFCpS5ljtnoIMi3PLNNDnfGfq6G3dzuiBg8= Received: from DBBPR09CA0013.eurprd09.prod.outlook.com (2603:10a6:10:c0::25) by AS8PR08MB9242.eurprd08.prod.outlook.com (2603:10a6:20b:5a1::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.29; Tue, 23 May 2023 16:23:07 +0000 Received: from DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:c0:cafe::dc) by DBBPR09CA0013.outlook.office365.com (2603:10a6:10:c0::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.29 via Frontend Transport; Tue, 23 May 2023 16:23:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DBAEUR03FT012.mail.protection.outlook.com (100.127.142.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6433.14 via Frontend Transport; Tue, 23 May 2023 16:23:06 +0000 Received: ("Tessian outbound 3a01b65b5aad:v136"); Tue, 23 May 2023 16:23:06 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 57e70b0a918f80aa X-CR-MTA-TID: 64aa7808 Received: from 02ee3eae2916.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 887280A8-43E3-492B-ADF0-9A76EF8657A4.1; Tue, 23 May 2023 16:22:59 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 02ee3eae2916.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 23 May 2023 16:22:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FaUkpMIBqNQd7IqxZ3IQrAnl7aP9Ge9FE34LSnfxG0YEQyyxAQq54j6oHtNCZYN87KHeJBoUQcUEwlCoZ6R9MZuJx0kkFm2B2VVQpS9fkMyiQdWIkOFH9GQc+emXhH3l+dIo09rJbpHg3jmwiAfj77qsE93YuFyRyfPKikzDvSmb+GdEBVB5KU+pFv6eoZq0lD4A3TEds6e5h7t34I0QoCni7moU9wANSFTOyF0v+eOhucu96jrICcm/miA9JuH8xcTW8Xgout7FtZoZY1mtGDKN2DGSwF7+LeTJC850eQw07qqCHVW1+F1/70HcAym2xN3cXKYuDvHwK49jfPDYDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5VVbgsNtZjaUC3sJQ7BPq9QrRwgakCnTvHzDNAbRo+w=; b=iDU83b70yvnBk+Jy3kfnzMt4mlMHemdtClyHtYuESKKrBU4MlGD8T2EktKhAKoujQkLNyJy0r/yAZ1FJFSwJ0BQHOYPy/N1XZcFrJo0RNHUZMFsufdkEKJl3BW2T2T+0LseD7BFqlnk2BsJFHjTEPbPGRA5Mj5daJWA449qutv1KdtQQZdVm6/Ck7RgasDfxtzKuYBsvvje3FD/cAMkvLTihDDn6NeXY/T9301i1KIcwLIukXkC4dp0QX2xlSUSkzfMkXBmzYmZ3yf7ImfV8pwQizcFRbYW+5SCnkKoloGJr/fZ/jo9hZuTIdGwkXl9fPWL1ajZoXrB2jk5AogTGnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5VVbgsNtZjaUC3sJQ7BPq9QrRwgakCnTvHzDNAbRo+w=; b=G+YEWyowaj7c46zfruk80RjgkJfVJXuVaXgH3yx7z3e6OdwCIG2O5cisXPfjXbFglizWEfGgzasRbXpkttBAtFT23b5id29MhCmDqupwA6FmmHCTaL5F/dwDAkv7husKe/fVUrBCoFCpS5ljtnoIMi3PLNNDnfGfq6G3dzuiBg8= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by PA4PR08MB6270.eurprd08.prod.outlook.com (2603:10a6:102:f3::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6411.27; Tue, 23 May 2023 16:22:55 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::621c:838a:cb11:19b7]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::621c:838a:cb11:19b7%6]) with mapi id 15.20.6411.028; Tue, 23 May 2023 16:22:55 +0000 From: Honnappa Nagarahalli To: "pbhagavatula@marvell.com" , "jerinj@marvell.com" , nd , Konstantin Ananyev CC: "dev@dpdk.org" , nd , nd Subject: RE: [PATCH v2 2/3] ip_frag: improve reassembly lookup performance Thread-Topic: [PATCH v2 2/3] ip_frag: improve reassembly lookup performance Thread-Index: AQHZjYSHCPZOF6DQS0ekEYwRBOOZg69oCmbA Date: Tue, 23 May 2023 16:22:55 +0000 Message-ID: References: <20230523125413.6324-1-pbhagavatula@marvell.com> <20230523143921.7420-1-pbhagavatula@marvell.com> <20230523143921.7420-2-pbhagavatula@marvell.com> In-Reply-To: <20230523143921.7420-2-pbhagavatula@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 5F95B6951B344048928A085515F379A1.0 Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; x-ms-traffictypediagnostic: DBAPR08MB5814:EE_|PA4PR08MB6270:EE_|DBAEUR03FT012:EE_|AS8PR08MB9242:EE_ X-MS-Office365-Filtering-Correlation-Id: 0968124a-2a7a-4528-af13-08db5ba9fdb8 x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: e+ol3JSer7UBvmwquIOMHUTLpn40gMv2QM9kkp3q6ZfpubSVij6aHe4uZnZx2tJY8Ow27wIv90N/5v4VNLq1jekCaz8NftJRYlH+zLbDOeL22cIK+CWBUvu8WrDNoo+Hsj4XB1HJSYDH5R3vxj+9JO9bulvZP27QY03N+mpsAZzeaP+UMCYe5l+YCt/WHj0WpNI/sg4SVzHg7fhRU6w/y9Yuv4UjCQDFyw1gZv7PhSj54A8URz+Z0h5yZxBMozMqXYjR6S7wQVJIigm8jjK5+e1vQpRgXt1qwwuxDfq86xcxSu0X2t2aQToHiHwfasj4Alk4r31IvTJc56BYnhgeqt/B81wL2l+pF2XcaXGPIuwabJocI+pQpyHW3NHKZszpv3mM4B5nkRIKJzTpiMFXC6eVqo5J+3Su/saeh4CpAM4T8tm9lX1aWYtvXtXuo0yeqQiROvGOiMnvCHIUGgj9zK96QGjGsI9EDVF+l2Y3eDfoR6uZ7ZqKqv387DYgE6/wbQuKSBY4wA4XuZtt0jHj3axXEM8re9hnEUmtRCu0dDYMZvlB6rHEjNkg3lFKLW9SyN71HudWJJl52JxkFXtnzGk7AqCXGjouyJeC5X2U449NQO/R4OLGsWIQAjo0dSmx X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(4636009)(366004)(39860400002)(346002)(376002)(136003)(396003)(451199021)(8676002)(8936002)(5660300002)(52536014)(83380400001)(186003)(9686003)(6506007)(26005)(122000001)(86362001)(38100700002)(53546011)(38070700005)(41300700001)(64756008)(7696005)(33656002)(71200400001)(55016003)(76116006)(66446008)(66476007)(66556008)(66946007)(316002)(4326008)(478600001)(110136005)(54906003)(2906002); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB6270 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 6184ff26-2a35-4181-7698-08db5ba9f716 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +813irbkK2tpFqwYAQ/30wGiK/N+xCs2BNvwOpAZriDQAbOsekbw/tlZjOZXIwSUhqvJ9I3ErbYcUzS5lWQeSLQPAk2+a92juALmSGtWkjU3VBVcKtOEHvIhF4FdlsiuildKDrIAH6zLBb5u2MXDsqPkRkO/+ZAldXSoLY9Nk8W9F0uLhvF2aC2s/MBvPOuM/4JbTaRqaE3ZCiOugSUIv7jMWbH18bqGmjaKpJJpN72wibUejQr7fA0a1Hst9zHvcK6PSD2PKlNs8wE++TmOLpCthm1ScoyFQ4D31uYbD5GGsUT5813sIP2cKtSHgyYKPuCvNblW4xjX7gamqouWmfLR5A06tgmqKLRgTrKmX1mct6++ftXmwnwc7Kfw1WpJDo8QdF9RK8vXLOo0OsJIXZ+7IkJiz/GWPl1jetI004JuxmvHE75THhVQnzh0tR4g5ikpGKnJyjSHMTkcbIn1cUr/l3SJQQ0hWYWnUQVnaKwq+TLUGqNps2DUzdU9woKn5oF+9BQaWaXefVsFta640P82RwRrx35uK2grLiG6sJjg+LImFKOFwkXaqJLSVcSK6x+53PsPrrYdsnN11mj62DvfuxmdzWSAuiFs6opR3RycYyHeu2AtlOGDHl0RZzhynzPiaRMIAzG/OtwxMsBaqVa2Otr73n8rg7t3vYgW3GGyJ+e006HGXiuTwHuc2KvJ3Vlh07vQfnXT2ldZ/55QVxuDKek9eUuArBJiwww/v5thlcKyS0qpOP0vC3z8kgsZ/z59ZzF4TlZKXVtEMxcC8g== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230028)(4636009)(136003)(396003)(39860400002)(376002)(346002)(451199021)(40470700004)(36840700001)(46966006)(316002)(4326008)(70206006)(70586007)(336012)(478600001)(82310400005)(54906003)(110136005)(8676002)(8936002)(7696005)(41300700001)(86362001)(52536014)(5660300002)(26005)(6506007)(33656002)(36860700001)(82740400003)(83380400001)(81166007)(356005)(9686003)(47076005)(53546011)(55016003)(186003)(40460700003)(30864003)(2906002)(40480700001)(23180200003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 May 2023 16:23:06.7240 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0968124a-2a7a-4528-af13-08db5ba9fdb8 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DBAEUR03FT012.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9242 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: pbhagavatula@marvell.com > Sent: Tuesday, May 23, 2023 9:39 AM > To: jerinj@marvell.com; Honnappa Nagarahalli > ; nd ; Konstantin Ananyev > > Cc: dev@dpdk.org; Pavan Nikhilesh > Subject: [PATCH v2 2/3] ip_frag: improve reassembly lookup performance >=20 > From: Pavan Nikhilesh >=20 > Improve reassembly lookup performance by using NEON intrinsics for key > validation. What is the improvement do you see with this? >=20 > Signed-off-by: Pavan Nikhilesh > --- > lib/ip_frag/ip_frag_internal.c | 224 +++++++++++++++++++++++++------ > lib/ip_frag/ip_reassembly.h | 6 + > lib/ip_frag/rte_ip_frag_common.c | 10 ++ > 3 files changed, 196 insertions(+), 44 deletions(-) >=20 > diff --git a/lib/ip_frag/ip_frag_internal.c b/lib/ip_frag/ip_frag_interna= l.c index > 7cbef647df..de78a0ed8f 100644 > --- a/lib/ip_frag/ip_frag_internal.c > +++ b/lib/ip_frag/ip_frag_internal.c > @@ -4,8 +4,9 @@ >=20 > #include >=20 > -#include > #include > +#include > +#include >=20 > #include "ip_frag_common.h" >=20 > @@ -280,10 +281,166 @@ ip_frag_find(struct rte_ip_frag_tbl *tbl, struct > rte_ip_frag_death_row *dr, > return pkt; > } >=20 > -struct ip_frag_pkt * > -ip_frag_lookup(struct rte_ip_frag_tbl *tbl, > - const struct ip_frag_key *key, uint64_t tms, > - struct ip_frag_pkt **free, struct ip_frag_pkt **stale) > +static inline void > +ip_frag_dbg(struct rte_ip_frag_tbl *tbl, struct ip_frag_pkt *p, > + uint32_t list_idx, uint32_t list_cnt) { > + RTE_SET_USED(tbl); > + RTE_SET_USED(list_idx); > + RTE_SET_USED(list_cnt); > + if (p->key.key_len =3D=3D IPV4_KEYLEN) > + IP_FRAG_LOG(DEBUG, > + "%s:%d:\n" > + "tbl: %p, max_entries: %u, use_entries: %u\n" > + "ipv4_frag_pkt line0: %p, index: %u from %u\n" > + "key: <%" PRIx64 ", %#x>, start: %" PRIu64 "\n", > + __func__, __LINE__, tbl, tbl->max_entries, > + tbl->use_entries, p, list_idx, list_cnt, > + p->key.src_dst[0], p->key.id, p->start); > + else > + IP_FRAG_LOG(DEBUG, > + "%s:%d:\n" > + "tbl: %p, max_entries: %u, use_entries: %u\n" > + "ipv6_frag_pkt line0: %p, index: %u from %u\n" > + "key: <" IPv6_KEY_BYTES_FMT > + ", %#x>, start: %" PRIu64 "\n", > + __func__, __LINE__, tbl, tbl->max_entries, > + tbl->use_entries, p, list_idx, list_cnt, > + IPv6_KEY_BYTES(p1[i].key.src_dst), p->key.id, > + p->start); > +} > + > +#if defined(RTE_ARCH_ARM64) > +static inline struct ip_frag_pkt * > +ip_frag_lookup_neon(struct rte_ip_frag_tbl *tbl, const struct ip_frag_ke= y > *key, uint64_t tms, > + struct ip_frag_pkt **free, struct ip_frag_pkt **stale) { > + struct ip_frag_pkt *empty, *old; > + struct ip_frag_pkt *p1, *p2; > + uint32_t assoc, sig1, sig2; > + uint64_t max_cycles; > + > + empty =3D NULL; > + old =3D NULL; > + > + max_cycles =3D tbl->max_cycles; > + assoc =3D tbl->bucket_entries; > + > + if (tbl->last !=3D NULL && ip_frag_key_cmp(key, &tbl->last->key) =3D=3D= 0) > + return tbl->last; > + > + /* different hashing methods for IPv4 and IPv6 */ > + if (key->key_len =3D=3D IPV4_KEYLEN) > + ipv4_frag_hash(key, &sig1, &sig2); > + else > + ipv6_frag_hash(key, &sig1, &sig2); > + > + p1 =3D IP_FRAG_TBL_POS(tbl, sig1); > + p2 =3D IP_FRAG_TBL_POS(tbl, sig2); > + > + uint64x2_t key0, key1, key2, key3; > + uint64_t vmask, zmask, ts_mask; > + uint64x2_t ts0, ts1; > + uint32x4_t nz_key; > + uint8_t idx; > + /* Bucket entries are always power of 2. */ > + rte_prefetch0(&p1[0].key); > + rte_prefetch0(&p1[1].key); > + rte_prefetch0(&p2[0].key); > + rte_prefetch0(&p2[1].key); > + > + while (assoc > 1) { > + if (assoc > 2) { > + rte_prefetch0(&p1[2].key); > + rte_prefetch0(&p1[3].key); > + rte_prefetch0(&p2[2].key); > + rte_prefetch0(&p2[3].key); > + } > + struct ip_frag_pkt *p[] =3D {&p1[0], &p2[0], &p1[1], &p2[1]}; > + key0 =3D vld1q_u64(&p[0]->key.id_key_len); > + key1 =3D vld1q_u64(&p[1]->key.id_key_len); > + key2 =3D vld1q_u64(&p[2]->key.id_key_len); > + key3 =3D vld1q_u64(&p[3]->key.id_key_len); > + > + nz_key =3D > vsetq_lane_u32(vgetq_lane_u32(vreinterpretq_u32_u64(key0), 1), nz_key, 0)= ; > + nz_key =3D > vsetq_lane_u32(vgetq_lane_u32(vreinterpretq_u32_u64(key1), 1), nz_key, 1)= ; > + nz_key =3D > vsetq_lane_u32(vgetq_lane_u32(vreinterpretq_u32_u64(key2), 1), nz_key, 2)= ; > + nz_key =3D > vsetq_lane_u32(vgetq_lane_u32(vreinterpretq_u32_u64(key3), > +1), nz_key, 3); > + > + nz_key =3D vceqzq_u32(nz_key); > + zmask =3D > vget_lane_u64(vreinterpret_u64_u16(vshrn_n_u32(nz_key, 16)), 0); > + vmask =3D ~zmask; > + > + vmask &=3D 0x8000800080008000; > + for (; vmask > 0; vmask &=3D vmask - 1) { > + idx =3D __builtin_ctzll(vmask) >> 4; > + if (ip_frag_key_cmp(key, &p[idx]->key) =3D=3D 0) > + return p[idx]; > + } > + > + vmask =3D ~zmask; > + if (zmask && empty =3D=3D NULL) { > + zmask &=3D 0x8000800080008000; > + idx =3D __builtin_ctzll(zmask) >> 4; > + empty =3D p[idx]; > + } > + > + if (vmask && old =3D=3D NULL) { > + const uint64x2_t max_cyc =3D > vdupq_n_u64(max_cycles); > + const uint64x2_t cur_cyc =3D vdupq_n_u64(tms); > + > + ts0 =3D vsetq_lane_u64(vgetq_lane_u64(key0, 1), ts0, > 0); > + ts0 =3D vsetq_lane_u64(vgetq_lane_u64(key1, 1), ts0, > 1); > + ts1 =3D vsetq_lane_u64(vgetq_lane_u64(key2, 1), ts1, > 0); > + ts1 =3D vsetq_lane_u64(vgetq_lane_u64(key3, 1), ts1, > 1); > + > + ts0 =3D vcgtq_u64(cur_cyc, vaddq_u64(ts0, max_cyc)); > + ts1 =3D vcgtq_u64(cur_cyc, vaddq_u64(ts1, max_cyc)); > + > + ts_mask =3D > vget_lane_u64(vreinterpret_u64_u16(vshrn_n_u32( > + > vuzp1q_u32(vreinterpretq_u32_u64(ts0), > + > vreinterpretq_u32_u64(ts1)), > + 16)), > + 0); > + vmask &=3D 0x8000800080008000; > + ts_mask &=3D vmask; > + if (ts_mask) { > + idx =3D __builtin_ctzll(ts_mask) >> 4; > + old =3D p[idx]; > + } > + } > + p1 +=3D 2; > + p2 +=3D 2; > + assoc -=3D 4; > + } > + while (assoc) { > + if (ip_frag_key_cmp(key, &p1->key) =3D=3D 0) > + return p1; > + else if (ip_frag_key_is_empty(&p1->key)) > + empty =3D (empty =3D=3D NULL) ? p1 : empty; > + else if (max_cycles + p1->start < tms) > + old =3D (old =3D=3D NULL) ? p1 : old; > + > + if (ip_frag_key_cmp(key, &p2->key) =3D=3D 0) > + return p2; > + else if (ip_frag_key_is_empty(&p2->key)) > + empty =3D (empty =3D=3D NULL) ? p2 : empty; > + else if (max_cycles + p2->start < tms) > + old =3D (old =3D=3D NULL) ? p2 : old; > + p1++; > + p2++; > + assoc--; > + } > + > + *free =3D empty; > + *stale =3D old; > + return NULL; > +} > +#endif > + > +static struct ip_frag_pkt * > +ip_frag_lookup_scalar(struct rte_ip_frag_tbl *tbl, const struct ip_frag_= key > *key, uint64_t tms, > + struct ip_frag_pkt **free, struct ip_frag_pkt **stale) > { > struct ip_frag_pkt *p1, *p2; > struct ip_frag_pkt *empty, *old; > @@ -309,25 +466,7 @@ ip_frag_lookup(struct rte_ip_frag_tbl *tbl, > p2 =3D IP_FRAG_TBL_POS(tbl, sig2); >=20 > for (i =3D 0; i !=3D assoc; i++) { > - if (p1->key.key_len =3D=3D IPV4_KEYLEN) > - IP_FRAG_LOG(DEBUG, "%s:%d:\n" > - "tbl: %p, max_entries: %u, > use_entries: %u\n" > - "ipv4_frag_pkt line0: %p, index: %u > from %u\n" > - "key: <%" PRIx64 ", %#x>, start: %" PRIu64 "\n", > - __func__, __LINE__, > - tbl, tbl->max_entries, tbl->use_entries, > - p1, i, assoc, > - p1[i].key.src_dst[0], p1[i].key.id, p1[i].start); > - else > - IP_FRAG_LOG(DEBUG, "%s:%d:\n" > - "tbl: %p, max_entries: %u, > use_entries: %u\n" > - "ipv6_frag_pkt line0: %p, index: %u > from %u\n" > - "key: <" IPv6_KEY_BYTES_FMT ", %#x>, start: %" > PRIu64 "\n", > - __func__, __LINE__, > - tbl, tbl->max_entries, tbl->use_entries, > - p1, i, assoc, > - IPv6_KEY_BYTES(p1[i].key.src_dst), p1[i].key.id, > p1[i].start); > - > + ip_frag_dbg(tbl, &p1[i], i, assoc); > if (ip_frag_key_cmp(key, &p1[i].key) =3D=3D 0) > return p1 + i; > else if (ip_frag_key_is_empty(&p1[i].key)) > @@ -335,29 +474,11 @@ ip_frag_lookup(struct rte_ip_frag_tbl *tbl, > else if (max_cycles + p1[i].start < tms) > old =3D (old =3D=3D NULL) ? (p1 + i) : old; >=20 > - if (p2->key.key_len =3D=3D IPV4_KEYLEN) > - IP_FRAG_LOG(DEBUG, "%s:%d:\n" > - "tbl: %p, max_entries: %u, > use_entries: %u\n" > - "ipv4_frag_pkt line1: %p, index: %u > from %u\n" > - "key: <%" PRIx64 ", %#x>, start: %" PRIu64 "\n", > - __func__, __LINE__, > - tbl, tbl->max_entries, tbl->use_entries, > - p2, i, assoc, > - p2[i].key.src_dst[0], p2[i].key.id, p2[i].start); > - else > - IP_FRAG_LOG(DEBUG, "%s:%d:\n" > - "tbl: %p, max_entries: %u, > use_entries: %u\n" > - "ipv6_frag_pkt line1: %p, index: %u > from %u\n" > - "key: <" IPv6_KEY_BYTES_FMT ", %#x>, start: %" > PRIu64 "\n", > - __func__, __LINE__, > - tbl, tbl->max_entries, tbl->use_entries, > - p2, i, assoc, > - IPv6_KEY_BYTES(p2[i].key.src_dst), p2[i].key.id, > p2[i].start); > - > + ip_frag_dbg(tbl, &p2[i], i, assoc); > if (ip_frag_key_cmp(key, &p2[i].key) =3D=3D 0) > return p2 + i; > else if (ip_frag_key_is_empty(&p2[i].key)) > - empty =3D (empty =3D=3D NULL) ?( p2 + i) : empty; > + empty =3D (empty =3D=3D NULL) ? (p2 + i) : empty; > else if (max_cycles + p2[i].start < tms) > old =3D (old =3D=3D NULL) ? (p2 + i) : old; > } > @@ -366,3 +487,18 @@ ip_frag_lookup(struct rte_ip_frag_tbl *tbl, > *stale =3D old; > return NULL; > } > + > +struct ip_frag_pkt * > +ip_frag_lookup(struct rte_ip_frag_tbl *tbl, const struct ip_frag_key *ke= y, > uint64_t tms, > + struct ip_frag_pkt **free, struct ip_frag_pkt **stale) { > + switch (tbl->lookup_fn) { > +#if defined(RTE_ARCH_ARM64) > + case REASSEMBLY_LOOKUP_NEON: > + return ip_frag_lookup_neon(tbl, key, tms, free, stale); #endif > + case REASSEMBLY_LOOKUP_SCALAR: > + default: > + return ip_frag_lookup_scalar(tbl, key, tms, free, stale); > + } > +} > diff --git a/lib/ip_frag/ip_reassembly.h b/lib/ip_frag/ip_reassembly.h in= dex > ef9d8c0d75..049437ae32 100644 > --- a/lib/ip_frag/ip_reassembly.h > +++ b/lib/ip_frag/ip_reassembly.h > @@ -12,6 +12,11 @@ >=20 > #include >=20 > +enum ip_frag_lookup_func { > + REASSEMBLY_LOOKUP_SCALAR =3D 0, > + REASSEMBLY_LOOKUP_NEON, > +}; > + > enum { > IP_LAST_FRAG_IDX, /* index of last fragment */ > IP_FIRST_FRAG_IDX, /* index of first fragment */ > @@ -83,6 +88,7 @@ struct rte_ip_frag_tbl { > struct ip_frag_pkt *last; /* last used entry. */ > struct ip_pkt_list lru; /* LRU list for table entries. */ > struct ip_frag_tbl_stat stat; /* statistics counters. */ > + enum ip_frag_lookup_func lookup_fn; /* hash table lookup function. > */ > __extension__ struct ip_frag_pkt pkt[]; /* hash table. */ }; >=20 > diff --git a/lib/ip_frag/rte_ip_frag_common.c > b/lib/ip_frag/rte_ip_frag_common.c > index c1de2e81b6..ef3c104e45 100644 > --- a/lib/ip_frag/rte_ip_frag_common.c > +++ b/lib/ip_frag/rte_ip_frag_common.c > @@ -5,7 +5,9 @@ > #include > #include >=20 > +#include > #include > +#include >=20 > #include "ip_frag_common.h" >=20 > @@ -75,6 +77,14 @@ rte_ip_frag_table_create(uint32_t bucket_num, > uint32_t bucket_entries, > tbl->bucket_entries =3D bucket_entries; > tbl->entry_mask =3D (tbl->nb_entries - 1) & ~(tbl->bucket_entries - 1)= ; >=20 > +#if defined(RTE_ARCH_ARM64) > + if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON) && > + rte_vect_get_max_simd_bitwidth() >=3D RTE_VECT_SIMD_128) > + tbl->lookup_fn =3D REASSEMBLY_LOOKUP_NEON; > + else > +#endif > + tbl->lookup_fn =3D REASSEMBLY_LOOKUP_SCALAR; > + > TAILQ_INIT(&(tbl->lru)); > return tbl; > } > -- > 2.25.1