From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0081.outbound.protection.outlook.com [104.47.41.81]) by dpdk.org (Postfix) with ESMTP id 26421567F for ; Tue, 2 May 2017 08:00:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hcwWPr8QpGdd6XPSnaOxZqxlTiQEpOsj7M9gy2brb8U=; b=LBffx1Fxhr6Fe+IKmFqOKSvIt7kmC3h70DgeoYGagQFvSfkDQeO/jkaYfV9LHCAkQjnFxPPJCHuXYTZEcIGMj62qhgbja3+c6yjzkx0Txzr5XswBRZcGla1alXf0QS3G51cahXgUXyQtn9SsN++BBdaYUhSw+3VvI4JFtu7F5og= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from 1scrb-1.caveonetworks.com (50.233.148.156) by BL2PR07MB2419.namprd07.prod.outlook.com (10.167.101.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1061.12; Tue, 2 May 2017 06:00:01 +0000 From: Ashwin Sekhar T K To: byron.marohn@intel.com, pablo.de.lara.guarch@intel.com, jerin.jacob@caviumnetworks.com, jianbo.liu@linaro.org Cc: dev@dpdk.org, Ashwin Sekhar T K Date: Mon, 1 May 2017 22:59:53 -0700 Message-Id: <20170502055953.29222-1-ashwin.sekhar@caviumnetworks.com> X-Mailer: git-send-email 2.13.0.rc1 In-Reply-To: <20170427114249.31863-1-ashwin.sekhar@caviumnetworks.com> References: <20170427114249.31863-1-ashwin.sekhar@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: CY4PR1201CA0024.namprd12.prod.outlook.com (10.172.75.34) To BL2PR07MB2419.namprd07.prod.outlook.com (10.167.101.143) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 64fd23fb-7a1f-453e-99a0-08d491207929 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:BL2PR07MB2419; X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 3:4Wj2YI684u8g/QIoo3MlgLjLf2I+z7GjMBWmakZwWAlmB4ysqqrKZYb1VEPUu7hjjmYhKlnzk5JFaNfttPp21gjlmBQmK67buIVJeHQnrRR9AluY0kVInsHqPiK/jlWhuOvPLsaeKvHCiv8RAh1XkJrU8xXRBV9lNczA6CEcBH3uNhvKqJJZ1X6aAinqJTuq6Hgr4Qz0jTJ2J5ZxTVallCBEhcPNzsV047tulu/fDgDnwET7EIcfLslFEU1/zOStiSyTItq4+0BG4j+BN/ltm/y4ivDGsof339mPBgOXkZlcxyalC/aaatEr1JtlGCZXlrTHMleYDU5A6nf6yj9KZA==; 25:1bb1tWOT7of3vjEmq0HiPnlVEWa2+IIDWWHkYAU7dzDOhXiZPCWwnhnz4Jswf2S9IIKrzpDtfJ+saVgzo46GghyAh6sVF5TTixWCujhe8smrccKaSd/Ehr8MTxOQTQKGSC7pRAtVlrICwmpgytbZo5/owJpIO5xhT2p7Op0ROeyJIfTr6hhi+cYHbqmrLFSshh7Jj2zeJZTOJSog2LRfZ1CdvHRYjLy3U0hVnuMYMe7itje0VrryKPX99uI1/qgzSA8kHtfbn1FvomRwlCZOSPz8yUQwBXMejyNARG3kyEKrkTuXzH5lZZHaS0P9ZuillmVVVd1dfa6lfGZS2aPYnam/FjZKcEQvpPN3AiOOhApOXroEz7hSzyVBgrMeNAmzO3g0xo92spLYxsL5knHzg/w1r7MmvgZw9JEekizqyRe4bWjib/9vVFnlHCVa5Bjn6b5ryk6ICjQE+oCeBJA3Zg== X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 31:qjN2iQd1ZMzsdGsvGNjbIzhKV8P2+0HaCKnIoDrD/IKCNu02Ct0plpasaSCgLxfR6A2b9QiMZxFISDlfK0arHcy3WMrsmsu6+e1KuKn9HsvEkQjg7Ari4Ahvvl7ubF4Do9VuQTWmkQ25Bi1IlKu/3O7Als+Jb8CzO976EhqhUcydkmINF8nkgfuOQtWUrxK/NhLoErx922enSThZ5DKiysO6pwpuC5K3C8F8EOy3NUfS7MlOeDrQF6y4IK6br3Jz; 20:6jLCiP/yPiCo2DbxUdAoo3ZlwVrovZPFIDuCaFD3S9jOI9XWzxiM0wWCNiqWZlY99nOoF1vajKYYkmOT7sEMRfLpA6jk0hmFGjlmQyFGEEajDU42e44gZm/Aa4BCbU6v6jrhI7uJBsptUUSfXhHxLb2aCW0mkw0C8n6wxbT3ilyUcMSwkKVdGUg/8SQXGnnt9S2f5CbKgvLuTdPXB0ji4DchDEbi1XPDuwQfAUusW0Kb7FRSGR9eF2LISUdvPuJqojH5fciRafUgK9B41lv6CSOh3aQcDiXzt1cIULvi2VSgiYPKof/go7hLPgUFC5kNw3nwZsFhlUSs08092530JgXVmDBigkL2Ki89FfBjMB/4slI8DG5pf5a3YdPJ1t4YyUsUp5pKXAVadpXZ81wQPzXdKlsorrVUKXp8P5wioZbUJW5rpY7c/jcNP3IP90+GVCW32qhwFhQlrqMuGcl6GHMyaWygWXiqA5ZYvcmciChPbubhW6UjsrQYmXe+hd+rZUK5WILLqiXNjMkLlQWwILMiOKpwizUPuCT8RARXfJhx72goHZvUfhUe1fDixJAKrRmLyzkEOolTbUs/yGY8q+CzpI3UOoyvW252/yQ0wvk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(10201501046)(3002001)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123560025)(20161123564025)(20161123555025)(20161123562025)(20161123558100)(6072148); SRVR:BL2PR07MB2419; BCL:0; PCL:0; RULEID:; SRVR:BL2PR07MB2419; X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 4:nSYogIHuFymixfjHG44fqclxDH+mA5+iV58NDkpZ1HqbyoAymAH17tVSaDl0+oUjxXxRHrysedRFCoqblokNMdI1dChSxs/OeA9kuKQ9sTFhQYdXJywxbRpK3K/NZ4bz8fs5x24RcObmlhGxoXO+V3SUlYCKFkCjXdtlUKo6L9+mW171oYHDs9D63BQ7ewGbUevw92ObXdZIoJvwl7J8hzUDwAzufKTiCfUF3obCHWaGIFkn2aRAz0qbGROZdkKt+J0it1ojdvZk9MXipOWyU9Vz/SwOalQtJq1KK97d/B7ad1YGBMUhtS7G8G/XGJ63GQb48MgpVSXjiej0UcDlEf5nozSeXF16ZfkVMDxIpwCUc3F8zd6DQj/5kJVtyMgBVsgu+tmLxPTfvK5NSBY5SYbDeSQeELycayx6hDFNPcnNCFiSXF6Xf8I2C3/+wzVLaubnePUv10/0ldx4OD8pBu2krp7WSI0Tksrj9dNsVXIUxPbWogVc0jjv7EOvUh3OW9zVMmFMp0X26jhkU5+7DtXxFs79d3raEgbkokx8cT0LC4H5e7YLMUcUK4sx2uT33Od0YQJKydVZ+5OmyOkYIc44l83kqzcgoxrzPjkWSUairqsJMaxLZVOYGaRAV3appyGMgTdeU/3uG5kUDEBD05ym5z+0bbHLvRuwPg7zG6z6+BLyt4V3o2clE/PmbmIItWBguIyheLZJz1Hkh6n/UHnBmIlRznwv5SlLF6O9AJOXCb4VaFottn2UlWqDAnasZr6Wroq54/f79Uu8g38HEA== X-Forefront-PRVS: 02951C14DC X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39410400002)(39850400002)(39400400002)(39840400002)(39450400003)(5003940100001)(7736002)(36756003)(189998001)(107886003)(4326008)(38730400002)(110136004)(47776003)(305945005)(1076002)(478600001)(5660300001)(50226002)(2906002)(8676002)(81166006)(76176999)(50986999)(2950100002)(42882006)(6666003)(50466002)(6116002)(66066001)(3846002)(48376002)(33646002)(6486002)(6506006)(25786009)(53936002)(6512007)(53416004)(42186005); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR07MB2419; H:1scrb-1.caveonetworks.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BL2PR07MB2419; 23:LdeMAik+E+1krIbB/L3Iv4oqbaZAy/5N5Kf9fBDrL?= =?us-ascii?Q?3iCHCzTCxtXZaUfw1384Ne4xZjkRbV/iowh9OIEPs4m3LcIIgSBNN3+ZB/Os?= =?us-ascii?Q?RD/fYMWEiOu4CQPN1MfXnupg9LmFRP4tkQkMqdDhEJ0/5qqyKsxLktJ1JuTC?= =?us-ascii?Q?qStz/FE9P/cW+dZNNfUuM+VtJL32T72r4dKC5Rd7ydMYF24icfvG5PLdTPVf?= =?us-ascii?Q?Vbcf27MzwMgN06gdSyy7CCGy1y9v74B9cjDgZdvo1+wCjAAnc8B4vcXpk5oF?= =?us-ascii?Q?H7WR3VxD2Y6AoolwywUjHhQW2eWF3MJL4R4O3JyT0inmhRw5dEtAuIwXyDRC?= =?us-ascii?Q?UqIgg+QvvcdPnL9FmY6msde+iyOotavfNXUqJus3J5bNDbD+RnZAAxFN+qWX?= =?us-ascii?Q?ba+nLhT2iq2VMFsx+qytRH1+/C3byNs6bSQYuOgQ+5xRr7iDLxDgTxmEb3vN?= =?us-ascii?Q?WVQVIucDqwl1eOm0VKFwPwfbtx+Ez9PVGLK9O5mCDqzQzZmgSMCklILNWiCm?= =?us-ascii?Q?O+2yhR5VPOf07iuK/DC4BocVil+hyPqvlL4F/OSouPkv9b5klFZZ4ToKmLnS?= =?us-ascii?Q?KvfwIinpxjqFGJGLGAoHhoJlURCaAkQAu6F21MjKgALVRQpaAjn/EuKSqn5T?= =?us-ascii?Q?2ibNpWuchnarKlggab+ez58wFPJT2hgZv88j2G7TrEhinffbWTnesRbLHiMv?= =?us-ascii?Q?bZmsSLTtgObfzGPQL/n9TcRsJhdQwmSAzvnUGGG188rFMFAjdyVeb/fYiWf2?= =?us-ascii?Q?9w5qxP0WAtlTyteLdTAte1j3iRAz98G8XiOuEnhEHZ76fkI3cpX68pT64Qhu?= =?us-ascii?Q?y9s5PjQdwhe6c6ubsqZyriK2UipLcPMyJqs88wY+FODf+nQKLENEcJmh4KkC?= =?us-ascii?Q?272xEvKmuP4tG77dvDE8gH0XdEDxRZXxylsL94TbLT6Vu0prpxHb/Jmo1o2O?= =?us-ascii?Q?TAd0G9dTbjus4fTVJlm8wfvudsUBu7VskSIQnGk3EgZwUESuKcZbhmeVDp3C?= =?us-ascii?Q?bmRG2PSMuyX6wbGiu7fPiixAWueSzLnnKHRBO5cwjgMkxFshnbAOiLpffpmx?= =?us-ascii?Q?Vo5c7z8qnUCEeL+XKeOcxwCSQyr?= X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 6:CQKkjo+KK2Ex0FN66d5S65pNRDLe9RXJsA4PtH7Jkf8sBdVbkASi0QaHEqWZlC2rzED69EQt40J1R8oNa2XJxpOejo9wVMFrLtZ1kXbxXIkJFQDEywY7P+04si+qxsyrdTPxjpj7udTFKO2YvohXydRHoBkGfh6Pn28KWeneWjC/X5+uEHLp7BX7vhNlxeWBkQutQrxiJLSeG0k4AoDOT8Rpw9KSwetdCsHoo/dBNfhAmo7hvCE1tTMCerF2B8+mW5bPEQkAWSklbTrXupKTafOOGKiuPISNsgJHf5OZJ4drPOlGOt5wZaKTVOykxxpIWroWnH+VBpOOzG3rMcNrw9Epx5oBeOwZmqsNw+vAXeQghi5EfoXXYePkwoe2AozwuHwKfFVAP4JCdJQ7Lz5SvEp4ZDA77Nr//7icPZtRbsSSxUZwGg6OFUkv9JYmLbiibfQ+1/LGUAHJQ38Xze7eYqEBReA0wrzkrKpVXArCnWO+MPjbfxtADBlhOdNhLOn4vZsQBnrJ/n0+dBrVKC02Tg==; 5:XrbgEzTGDyfnsYFTaMAhavOwPwm1ynQbABFbPMs9Cjjhj0JbN36wMQsKysRrNoeqms863A5kayjOg20KxPgq5lf3KEbEqs7Nb5HIRLym4XWYIuIdiymxvza99vZR74pBQ2Sx+exflABE01GVyiIeBw==; 24:pD3d9MIHXoQLuB3HATgf3ApMWQ+0C0nX/Gkl3az/65ZHJDK+17mXKFpGPrK65mi0VvLEfH2+kCi8Bc+ztnNqHqSEqxAquMGAPL5/4YAl58U= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 7:28TOTct3wVkTKwzWcfDsKUJA31nHTiyUbc/upYBm2PC4YK+IpNTK9E3AkjyGu5mqOVlo4QNnrD2gCJ/Oin+4YUJaYscmmpMWQsn7fiGnrCCfglq/HbZJnRjWy728Rhugo0mR+npBqRHeWFy3LGy/C+THHZ0v6KhhnwCXJJuSj+OpMOaCNnKaqyKUq7hLuTaUL4Ehv5NtwIswnPIQQc0JewAEPrIOYm6teXIgw52W/NSymDTPP7mT0qk+HNBXcxEu/p9AFb/VvPPzUhec84i9xlSUB9NmEuLeg0JxJklKXUL+vGcFmWjSB/yIpFgHjqLxOaBYYNVWELQK4lkobnG41g== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2017 06:00:01.7582 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR07MB2419 Subject: [dpdk-dev] [PATCH v3] efd: support lookup using neon intrinsics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 May 2017 06:00:05 -0000 * Added file lib/librte_efd/rte_efd_arm64.h to hold arm64 specific definitions * Verified the changes with efd_autotest unit test case Signed-off-by: Ashwin Sekhar T K --- v2: * Slightly modified the content of the commit message body * Added prefix [dpdk-dev] to the email subject line v3: * Moved enum 'EFD_LOOKUP_NEON' under '#if defined(RTE_ARCH_ARM64)' MAINTAINERS | 1 + lib/librte_efd/rte_efd.c | 24 +++++++++++++ lib/librte_efd/rte_efd_arm64.h | 76 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 101 insertions(+) create mode 100644 lib/librte_efd/rte_efd_arm64.h diff --git a/MAINTAINERS b/MAINTAINERS index b6495d2..7d708ae 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -147,6 +147,7 @@ F: lib/librte_eal/common/include/arch/arm/*_64.h F: lib/librte_acl/acl_run_neon.* F: lib/librte_lpm/rte_lpm_neon.h F: lib/librte_hash/rte*_arm64.h +F: lib/librte_efd/rte*_arm64.h F: drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c F: drivers/net/i40e/i40e_rxtx_vec_neon.c F: drivers/net/virtio/virtio_rxtx_simple_neon.c diff --git a/lib/librte_efd/rte_efd.c b/lib/librte_efd/rte_efd.c index f601d62..5cc6283 100644 --- a/lib/librte_efd/rte_efd.c +++ b/lib/librte_efd/rte_efd.c @@ -53,6 +53,8 @@ #include "rte_efd.h" #if defined(RTE_ARCH_X86) #include "rte_efd_x86.h" +#elif defined(RTE_ARCH_ARM64) +#include "rte_efd_arm64.h" #endif #define EFD_KEY(key_idx, table) (table->keys + ((key_idx) * table->key_len)) @@ -103,6 +105,9 @@ allocated memory enum efd_lookup_internal_function { EFD_LOOKUP_SCALAR = 0, EFD_LOOKUP_AVX2, +#if defined(RTE_ARCH_ARM64) + EFD_LOOKUP_NEON, +#endif EFD_LOOKUP_NUM }; @@ -674,6 +679,16 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, table->lookup_fn = EFD_LOOKUP_AVX2; else #endif +#if defined(RTE_ARCH_ARM64) + /* + * For less than or equal to 16 bits, scalar function performs better + * than vectorised version + */ + if (RTE_EFD_VALUE_NUM_BITS > 16 && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON)) + table->lookup_fn = EFD_LOOKUP_NEON; + else +#endif table->lookup_fn = EFD_LOOKUP_SCALAR; /* @@ -1271,6 +1286,15 @@ efd_lookup_internal(const struct efd_online_group_entry * const group, group->lookup_table, hash_val_a, hash_val_b); + break; +#endif +#if defined(RTE_ARCH_ARM64) + case EFD_LOOKUP_NEON: + return efd_lookup_internal_neon(group->hash_idx, + group->lookup_table, + hash_val_a, + hash_val_b); + break; #endif case EFD_LOOKUP_SCALAR: /* Fall-through */ diff --git a/lib/librte_efd/rte_efd_arm64.h b/lib/librte_efd/rte_efd_arm64.h new file mode 100644 index 0000000..cc93411 --- /dev/null +++ b/lib/librte_efd/rte_efd_arm64.h @@ -0,0 +1,76 @@ +/* + * BSD LICENSE + * + * Copyright (C) Cavium networks Ltd. 2017. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Cavium networks nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * rte_efd_arm64.h + * This file holds all arm64 specific EFD functions + */ + +#ifndef __RTE_EFD_ARM64_H__ +#define __RTE_EFD_ARM64_H__ + +#include + +static inline efd_value_t +efd_lookup_internal_neon(const efd_hashfunc_t *group_hash_idx, + const efd_lookuptbl_t *group_lookup_table, + const uint32_t hash_val_a, const uint32_t hash_val_b) +{ + efd_value_t value = 0; + uint32_t i = 0; + uint32x4_t vhash_val_a = vmovq_n_u32(hash_val_a); + uint32x4_t vhash_val_b = vmovq_n_u32(hash_val_b); + int32x4_t vshift = {0, 1, 2, 3}; + uint32x4_t vmask = vdupq_n_u32(0x1); + int32x4_t vincr = vdupq_n_s32(4); + + for (; i < RTE_EFD_VALUE_NUM_BITS; i += 4) { + uint32x4_t vhash_idx = vshll_n_u16( + vld1_u16((uint16_t const *)&group_hash_idx[i]), 0); + uint32x4_t vlookup_table = vshll_n_u16( + vld1_u16((uint16_t const *)&group_lookup_table[i]), 0); + uint32x4_t vhash = vaddq_u32(vhash_val_a, + vmulq_u32(vhash_idx, vhash_val_b)); + int32x4_t vbucket_idx = vnegq_s32(vreinterpretq_s32_u32( + vshrq_n_u32(vhash, EFD_LOOKUPTBL_SHIFT))); + uint32x4_t vresult = vshlq_u32(vlookup_table, vbucket_idx); + + vresult = vandq_u32(vresult, vmask); + vresult = vshlq_u32(vresult, vshift); + value |= vaddvq_u32(vresult); + vshift = vaddq_s32(vshift, vincr); + } + + return value; +} + +#endif /* __RTE_EFD_ARM64_H__ */ -- 2.7.4