From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BY2-obe.outbound.protection.outlook.com (mail-by2nam01on0084.outbound.protection.outlook.com [104.47.34.84]) by dpdk.org (Postfix) with ESMTP id A4C611077 for ; Thu, 27 Apr 2017 13:43:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=HjtlVJ1pRnp5IptSYGfxL8Pi/siDlci6ah0gGyb8t6g=; b=f1nCxpmijrswkoW/1Mmbp4we9LXop5VB2fGsbrCHSa5pLzgXg0QX2imo9Rqblx5QJqRJ2EeqTY7U9EygxiIwhnGlog8iNLeXTU9Vshx4+5p3m4qlXsopAQNvLMZV+ED1RObiObY+lNnpijc3CXoOePkIIr3R0CYzyuLqzhWfokM= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from 1scrb-1.caveonetworks.com (50.233.148.156) by CY1PR07MB2426.namprd07.prod.outlook.com (10.166.195.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1047.13; Thu, 27 Apr 2017 11:43:03 +0000 From: Ashwin Sekhar T K To: byron.marohn@intel.com, pablo.de.lara.guarch@intel.com, jerin.jacob@caviumnetworks.com, jianbo.liu@linaro.org Cc: dev@dpdk.org, Ashwin Sekhar T K Date: Thu, 27 Apr 2017 04:42:49 -0700 Message-Id: <20170427114249.31863-1-ashwin.sekhar@caviumnetworks.com> X-Mailer: git-send-email 2.13.0.rc1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: MWHPR10CA0061.namprd10.prod.outlook.com (10.169.238.23) To CY1PR07MB2426.namprd07.prod.outlook.com (10.166.195.15) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 26f3f254-7da6-4e59-4c24-08d48d6290a8 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:CY1PR07MB2426; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2426; 3:7SmLFlQPYA0Vj5Y3nXOLj+JoVWqXmtaGjRz3XI94NTaMrFAQbO1ykREYQem3sxlNP2K6R+prI8mXFLE6ZoNYhZ1hl6VCGYifq0tgfIhhA32eKsHnyBHN54Uv+2dkJQuq5qijWITpEG7gMixxSnUySr7ew2NiSYBTHPR/HcPXZ9qjvoAuYpndmwIglRfRRzKBGUz5Lrji83NqiHHN3weCpnvrjxTcjJh9WnTTyem1s/GUFcav88HJ2O3HvAiUjnhRx2NEcL/hNHN6g0dq9++p6mrVXXCs1oX8OlfWBpRGjGt6718wmqKvSnm5cBWnPayFTfR+05Ap8jTEsaaPSvT42g==; 25:VIpwMNV0veGtJr8MMAQ87mRDcLFlBbDe4fYHUWKk2coqX2qt9XAWrrYgR1sOoceaaj9BqsIhYfkIcfS0ZpqSvanPw5NlmKWu6bjJgfg8dUBQbiIZrlqHZKpvDcAHIvAWo9eLXve4C99/76HKVYjZIqOoIWygGfHWeTyJm4wPPMJindUxc7EqK5W4CVaIczC9PRJYKMKAWsG8aqozRvRlDWQd+NfJ/sNC5FZR5t0iHesd7uXPlm4DI0URWpdSHDlAs/XVCKSTzsOeAJ0p1dUC659HLiFT+ZE/MP2dHE3Qb/TN2ryC9ERDu/QYKP8rl66KaIjXMeV8m0Hgc2qoPwW+NYXJe12Hh420p1Dxo5JBFEonXJsOEYxbfo+702gxdZ5EYaECkNLjFO9Jroc9x5Mgcq5i39vTwTV/HIcIsWpXUEm9FTDNYwz5fkUVSAGCvJEVPlw8pqKZfMeumqQBeRsXAg== X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2426; 31:JoN9CtHS+Xyjk4WaOWTichJHiYTWeI0fHOo8lVEdtmDPU+VGlE4wdAyvvmvjEAg4BCg2214ILccVJw9uAJPjPWk0EG0azR5y2U3ozEk+vjbXD5C+dWSf6nMoaeLe8MYA9lWNzM0CDD4d3n3dMbfbEFagu60m+oDQBcdMhHZ+Xywh3VjvWemNThotguKarVSfcG71JcjPx6jB9P2ykYipnqcWTXBa9vYU+PO2tNVI/V87+x8Zlo229MJ1fpxFwznV; 20:uTZcjCdK+av2IN28cryUC8EITbAYZx9YJigGeMVDMlw/bnCci8lT2ePkTk5aHEAm3LVuGvsUO6YwaFZ8XTjTbZ+/PXeNj7Fv0RKkLPPXPqKv+IFVcWFemguNmGPhI1yYqWrC4GZj7+s1O/ii4dKmByxq/l5fQHdiuuQr5d+dEjFlsJgWvru52+Tk0qCCcW1IbiRU3l5yB3kV5bJTOuwgnUca8t7UX7Upt+37+IJIwbkpf9RFamtZeQo9RPq3o+wxyn7/9MS9r/X1JIRL7FXcd5A8FZ4BH3niUQyIoUgkvZqe7QjwjisNzeXbfeMobXdMW4gs9dv10FeGBdQdx2NNg2nyU05Tz07HV2FPnr0k+y6FHTHdVonwP2+fCjFZH4kRNLBlyCAJG/EwGWoe+pFe01tLQ3PcDcrYbadc6sO2O3YjVSJ2fbdDRAeVGZ2YQNzRX94chm3h297CpEJQ1BL4Nwcg7FEw/8wn3UYQ86/fy5axzdF+Ehu6tCiMhKHHTnS0ycwHgjnvyRUZoJ5zl0EEi/GkNv9s54dyz4sRfYC6vwmj95WrJg93TgcQAQZl76fMdLOCSiu5QkEt7t43tPHtNPMMoZWTNdyOxkAcwSPA+IY= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(6041248)(201703131423075)(201702281528075)(201703061421075)(20161123564025)(20161123555025)(20161123562025)(20161123560025)(6072148); SRVR:CY1PR07MB2426; BCL:0; PCL:0; RULEID:; SRVR:CY1PR07MB2426; X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2426; 4:6xcHLg1NUBsLsyDAgLxnZEMs7nxgNOe65ah+5rrLCSlFSsEyRXeW7jHDeq642+UQE4ElAajyBPQkLOG9C7Z/BNSIe4kBQgkf00/gVHUtNRHR57zcM8Aza2Kiq17cXp/xgyX4tZNEMhK+Y1+8HpT/Lxb4c0bGGWUnk22FlTLZTVwEbmy/r9pyen50tzJA1k1e5TwjHfc7ea8aivUixigBC4ZnecVpIKekaa/0Rs5iUNOvghfi3tf7wxPnF1Jnoz23ioD314GZ2OTC1HYQy1iGEGIgwPMt3JY83axstkiDDySIe2HiGXbO/t1i8CzZvakBCq2S0Bggq6RBPp3mt9naX4bhzjX8Gk/+sMxz035bruJRCjlIqvZW0AhJKPTSnt3WAJYM5yNwiNsGJDNbnDaSKs1ZRNwzUS8z0R4UL/9OfowdzJdHi7sXbzTtFs0IrCHs1fa1kJR/9/9u+7/7AaAUUjrXBV6rVVoGAXOm4oYb0kY1TgKa6FWF3xNoofZOCzwaKybXMnDiIITqeyst9v0V/QuGXrnA4epl8excB10NZEUFYkcQ5ooGO0M+plL/u8OfgnA45Fgj8wc8GsGk83NK60U4fqgBPGLZs9qGe5iC4wMRup1E/bW+raOk0WHtaCzLzln51r9FcQ7Mus9l8062Hd6qSWRT7H5TlvUp3LC9ZO7d0YzZnmMxPjtqZEJdqX5j X-Forefront-PRVS: 029097202E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39410400002)(39400400002)(39850400002)(39840400002)(39450400003)(2906002)(6116002)(3846002)(81166006)(305945005)(47776003)(36756003)(1076002)(42882006)(66066001)(25786009)(53936002)(6666003)(8676002)(50986999)(4326008)(48376002)(42186005)(50466002)(6512007)(53416004)(38730400002)(110136004)(107886003)(189998001)(5003940100001)(50226002)(33646002)(5660300001)(6486002)(6506006); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR07MB2426; H:1scrb-1.caveonetworks.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY1PR07MB2426; 23:JrvsEvrB54650kiffjPuHF/2EFrPskwF09poToFpM?= =?us-ascii?Q?GzC0jt+OoqbI0veKe2ZtA/neaVTVO6we5yXKFvulh9VKcYt4VNk6AMk8BMNY?= =?us-ascii?Q?WjB46AzLZwZog9lA1DN/bZwksvBL85CXYDoll9x8NDDq/GjMxLguiGTXyJ5X?= =?us-ascii?Q?OI7oSAdFQmpqNgVmDoxXXAbFagReqpncYCn47sdt67R52eXMRsBs2G/7Elx+?= =?us-ascii?Q?82Gh+KEN9S8AzrSnSDo3tA0jDbkDKLarz9pjHxLiAsT8Sfvil/eRWCpNqlRJ?= =?us-ascii?Q?wzbMbaHmLyTPHP4eMtjMl8S1DkvAEqqNX73+yIVju9pqikALpumasgvNMXqN?= =?us-ascii?Q?s/3VvAXFpI6CCGS3l9fRjHFFPYS8I3FFR3d3XwB2Af6J4YQ/mq3lu5+wSYPI?= =?us-ascii?Q?7tzg3OlBUkfSzBYEAHatymzjyuCu1cobMeTIl9QsbZCqDnFwSO6gWDKZ6ylj?= =?us-ascii?Q?QCLLYLAfIQ6w5r8Y4ML1WxqVr4iSLMCbs5gXGKhVyCuEiLdMXpqE6afMZnnk?= =?us-ascii?Q?naLaxhJ2Uzv22bQDfMsHb9gmAXh5bY3nSJphd717sn8JS+BAP7gIWVYTdztj?= =?us-ascii?Q?6SwCYU1PfX50C2U8ti8+Yu7bgPtJornhwrg7OkgCDOU8hptEIlRDIa/5J9t9?= =?us-ascii?Q?aSpe7XTAaeywRg3UbKZhKcdxDqgmbh2ORRSkUsKN6tNwgndZsQwysmNLZfhY?= =?us-ascii?Q?8uXMv9JGzF5KoI6R4c0pltSvLkhGBWEe0Nb2X3jP+hcC7KQsFgeMc6kvsPwj?= =?us-ascii?Q?ht+xkXc891sD6uKWSoHGxExlAWroM32ewZ0A8TbEneGUneJda0oJIBqc1gF8?= =?us-ascii?Q?vZvGwGTpygia84kQr1vdaXBn8rRY3aZeeM09IYBw4TKWf4ztbzUymaJcHP10?= =?us-ascii?Q?tu1BrF5o3xh4NYIzmwkTVJnki3Ul1aO7EznS0ITG0j9IKJHEotCdj5xaU9vZ?= =?us-ascii?Q?9UfduxvyiIqCL4fUW+MGm5PAoZL/tsWyAUAPBBavpzZtyhgrt03QUU4U6NhX?= =?us-ascii?Q?Q4=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2426; 6:LJoLdkxSwcD+3wOieAjXgoe4FcRCBu+emu6NSC5wP/+OklxYmnqW/s8Rk86+4oJlnufeWrwm1xHYWiIoF7p2IPUTvPVcuT8L83AxcPnzzaFAB7kyeBNa0obwiTrcApHMvqznjO+AURDmWWarvaWkMFH8t/k38y+oDehW4rpUYBlOWZEEV4xLs0BQvAXaU/rFKjEB+/ruGBeB8d2TI7WEEPlY4ken4nUudgeZgkJJEDaZfQA+S7kqQLeZV+XfyVvwZ0V63qHHCXtqmo3E0PXQNpP0eMAHsWu40HCTflRF9mC4dcRo5iqi6Vmaz2cQXWDS2hIKaTfZGOas25fa6csLnqhPNrvGsKkE6DdNaIDDg0AG46gGX+4r9ntR1ExF/R7KRynbrSXSxypjES0QiyfAaeM4es+/NdoiV2vJhY9IDebihHgbxIr68tGTKSAZEmI7GV17+RwRfM1ASuJape2chyKZrwl+o4fkeMjQN7QDhoStrumD8ExXyNwrv5qG2VWCCC/2NTI7Wv0RK8RmRxZrBQ==; 5:jvVR7NR0VCkptWmbyGZieLiRmEHSneWcXTBBug727ihlYRIhGjRvaBZX9s/uOFftPsOB4w/Qb/fU/ai2PMYKtsVQaTBNS1P2ogGsU1Pl1oZvtQjDzslPvAW+IlRhi3RdnstVLR3Zz/i6FCjrct2ImQ==; 24:UGaGqHdpi3I82Bf+ux9Exr3CMcsr62BSHBB3Y5KV8+dTp6J2lgIjqemLnB1/hM9afEGp8AYCXQk/l6B0RaEl1L6upCZdEL+vbdhRRtSohbU= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR07MB2426; 7:VdRdAIK8JX13szqQVROHJJ22VOfUymJi7cFaDIqpxcgk3cTrKTB3K6TUNyTZn/UL4mfGqSELKhMVn78X8UK9RZcX1TEXwjb14Y0cQV0pgCd3L7LMBgwBrETpq0ZHXnuS4kjIdyC+MIWIV/qNOmI+n7+TsbB6c/X2ZcOtddBYyjo1AvKIBmhouUG4MhaIA/M2WpdyvdF5+FEYI0iZwVb4k+tEKg8qAVrngAMFSC4ebEsFm1P9GfschFxv7K7C+x62qrDmdUbpcATPkkxIcNsv8fSvXh/xz2CA5bFE2AgfYJ5hzrmyaf93/oTYnrNwLlllDjKsyhSaR3n6MWR5pOBZYQ== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2017 11:43:03.5599 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR07MB2426 Subject: [dpdk-dev] [PATCH] efd: support lookup using neon intrinsics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Apr 2017 11:43:07 -0000 * Added file lib/librte_efd/rte_efd_arm64.h to hold arm64 specific definitions * Tested using efd_autotest Signed-off-by: Ashwin Sekhar T K --- MAINTAINERS | 1 + lib/librte_efd/rte_efd.c | 22 ++++++++++++ lib/librte_efd/rte_efd_arm64.h | 76 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+) create mode 100644 lib/librte_efd/rte_efd_arm64.h diff --git a/MAINTAINERS b/MAINTAINERS index b6495d2..7d708ae 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -147,6 +147,7 @@ F: lib/librte_eal/common/include/arch/arm/*_64.h F: lib/librte_acl/acl_run_neon.* F: lib/librte_lpm/rte_lpm_neon.h F: lib/librte_hash/rte*_arm64.h +F: lib/librte_efd/rte*_arm64.h F: drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c F: drivers/net/i40e/i40e_rxtx_vec_neon.c F: drivers/net/virtio/virtio_rxtx_simple_neon.c diff --git a/lib/librte_efd/rte_efd.c b/lib/librte_efd/rte_efd.c index f601d62..4d9a088 100644 --- a/lib/librte_efd/rte_efd.c +++ b/lib/librte_efd/rte_efd.c @@ -53,6 +53,8 @@ #include "rte_efd.h" #if defined(RTE_ARCH_X86) #include "rte_efd_x86.h" +#elif defined(RTE_ARCH_ARM64) +#include "rte_efd_arm64.h" #endif #define EFD_KEY(key_idx, table) (table->keys + ((key_idx) * table->key_len)) @@ -103,6 +105,7 @@ allocated memory enum efd_lookup_internal_function { EFD_LOOKUP_SCALAR = 0, EFD_LOOKUP_AVX2, + EFD_LOOKUP_NEON, EFD_LOOKUP_NUM }; @@ -674,6 +677,16 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, table->lookup_fn = EFD_LOOKUP_AVX2; else #endif +#if defined(RTE_ARCH_ARM64) + /* + * For less than or equal to 16 bits, scalar function performs better + * than vectorised version + */ + if (RTE_EFD_VALUE_NUM_BITS > 16 && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON)) + table->lookup_fn = EFD_LOOKUP_NEON; + else +#endif table->lookup_fn = EFD_LOOKUP_SCALAR; /* @@ -1271,6 +1284,15 @@ efd_lookup_internal(const struct efd_online_group_entry * const group, group->lookup_table, hash_val_a, hash_val_b); + break; +#endif +#if defined(RTE_ARCH_ARM64) + case EFD_LOOKUP_NEON: + return efd_lookup_internal_neon(group->hash_idx, + group->lookup_table, + hash_val_a, + hash_val_b); + break; #endif case EFD_LOOKUP_SCALAR: /* Fall-through */ diff --git a/lib/librte_efd/rte_efd_arm64.h b/lib/librte_efd/rte_efd_arm64.h new file mode 100644 index 0000000..cc93411 --- /dev/null +++ b/lib/librte_efd/rte_efd_arm64.h @@ -0,0 +1,76 @@ +/* + * BSD LICENSE + * + * Copyright (C) Cavium networks Ltd. 2017. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Cavium networks nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * rte_efd_arm64.h + * This file holds all arm64 specific EFD functions + */ + +#ifndef __RTE_EFD_ARM64_H__ +#define __RTE_EFD_ARM64_H__ + +#include + +static inline efd_value_t +efd_lookup_internal_neon(const efd_hashfunc_t *group_hash_idx, + const efd_lookuptbl_t *group_lookup_table, + const uint32_t hash_val_a, const uint32_t hash_val_b) +{ + efd_value_t value = 0; + uint32_t i = 0; + uint32x4_t vhash_val_a = vmovq_n_u32(hash_val_a); + uint32x4_t vhash_val_b = vmovq_n_u32(hash_val_b); + int32x4_t vshift = {0, 1, 2, 3}; + uint32x4_t vmask = vdupq_n_u32(0x1); + int32x4_t vincr = vdupq_n_s32(4); + + for (; i < RTE_EFD_VALUE_NUM_BITS; i += 4) { + uint32x4_t vhash_idx = vshll_n_u16( + vld1_u16((uint16_t const *)&group_hash_idx[i]), 0); + uint32x4_t vlookup_table = vshll_n_u16( + vld1_u16((uint16_t const *)&group_lookup_table[i]), 0); + uint32x4_t vhash = vaddq_u32(vhash_val_a, + vmulq_u32(vhash_idx, vhash_val_b)); + int32x4_t vbucket_idx = vnegq_s32(vreinterpretq_s32_u32( + vshrq_n_u32(vhash, EFD_LOOKUPTBL_SHIFT))); + uint32x4_t vresult = vshlq_u32(vlookup_table, vbucket_idx); + + vresult = vandq_u32(vresult, vmask); + vresult = vshlq_u32(vresult, vshift); + value |= vaddvq_u32(vresult); + vshift = vaddq_s32(vshift, vincr); + } + + return value; +} + +#endif /* __RTE_EFD_ARM64_H__ */ -- 2.7.4