From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0070.outbound.protection.outlook.com [104.47.41.70]) by dpdk.org (Postfix) with ESMTP id E8AD41077 for ; Thu, 27 Apr 2017 14:44:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=G/EyDJeHybTixBuVlYdREFYcg5jbMfft/H8/UGS5kBU=; b=SLDGcq3n/VPjuNvQ3xoavgArDRm/F6dxtTzfCBFEXWUsGNqQv6lxjVGX2I0t0gqWYOqyZ/W8WKC36Im5t6DaYvCD7rgW2g0tUoMuXqMowoE6kH/a2z0mvCnj9irOcdwwa8p1B8SKuLPGiZX+P0rrTjLG0ICDfXdtBztNUAfFUsI= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from 1scrb-1.caveonetworks.com (50.233.148.156) by BL2PR07MB2419.namprd07.prod.outlook.com (10.167.101.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1047.13; Thu, 27 Apr 2017 12:44:29 +0000 From: Ashwin Sekhar T K To: byron.marohn@intel.com, pablo.de.lara.guarch@intel.com, jerin.jacob@caviumnetworks.com, jianbo.liu@linaro.org Cc: dev@dpdk.org, Ashwin Sekhar T K Date: Thu, 27 Apr 2017 05:44:18 -0700 Message-Id: <20170427124418.32857-1-ashwin.sekhar@caviumnetworks.com> X-Mailer: git-send-email 2.13.0.rc1 In-Reply-To: <20170427114249.31863-1-ashwin.sekhar@caviumnetworks.com> References: <20170427114249.31863-1-ashwin.sekhar@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: BN6PR20CA0013.namprd20.prod.outlook.com (10.173.158.151) To BL2PR07MB2419.namprd07.prod.outlook.com (10.167.101.143) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2950bf87-a399-4de1-c4f5-08d48d6b260f X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:BL2PR07MB2419; X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 3:akkw3tyOQ5k39gzLzqNTtQmR153cRV8oLkj+rc4l74PfIL6kBKzhbm2qH6VNpGQA/j1DbLgSWc3kOLo3PdvzFPR1+Ko0EbYjpaurpQ+ID++Wr1EaF0yAHJjRE5w/WIGWQVHkTR9vzd3IoxtxFD0SqswmiCbl3gUvoTCOP9JpRfNmVA01TsZtjsV3HfAwu/XlLQ69jzuBDypTc9kXP0jOQRyEYJVkT+ArjEtYXWWMBmV0wOHe7HhOje5XT3F1vrOyWX2g6aWQB/FWPo8vZ45jI0k3cWRJl1czXU1ZgnP62ak0FEq7DZ6Jryuc8flzGy4Zhnr9IFqkL36NeC7a95jm5A==; 25:9WurgFEuuzqnDfna7PMYM6D+xKV+OIkSoJzUVEVjj2CBLuZYYGda0gePKQ0+3jhjAb+X0t50oobUk/zqx4ZcO0VI70k3U/rXi/AR5e0upGthc4pNwtuycPoBbyVvow2T4hlyJ97N5DiCBHh7gFps5rf977oX1zVH1uGiRtDXskyqOvU4z6ftVH7XZssL7QzZHstHtcwpMRAxn3pjsXnPsJs9BzIqIlAzqXY3KMJYpSwk8KCEOBnP48VfTrRkIgT+KsXWZBGyvQMLKwaG71ljFyG2Ag//iVyAhVircgeRJTfVK24iQyzL9Bbb19VuKzE15Cs7oH2+o1E2cAW7+NTepl/khBbspDzBAM6oPREyApg73wgHqUfdiLwYYcJZOPDFPnX+8Do/5MvML99VXzeN0hKa6Efr9H4dA/PrR6jQtDwaV60fZ3TPD3ppfL4sdznuAgorweyWmuHCJ4uUWDfC+w== X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 31:K9WcGDxndgIVwNz9xB/hl5z7wSOCEZLuCFDo+JeFkUBZrsyIHynR9uTGsVBTXvw7Ixhz8UscEjSD5qEbm8SeibgIYIfTHtey4Wxm+c+COmiHg0YsfgqHgkLHgudzJcdHq6NmpILDlmRlQfYivmTtJPOwCNgj1MkI2tDiydjUsRb4bY/3co09Hb8UNyXZCXXduWrhCAHmrKkbWQp39fxRmZ0qKWxkOjV4/U81oSP3FXspjMoemt5LO3uiEfteKdmt; 20:cu9zhHHkHAAWA5hMCFfGWpyrY172LoYkqEJRfYoo8fbsGA8py28buGFyixX+/SOrB4Z0DX7VN70c8ODSsEAtUvxFgixeFlfsT/CqjZvoC/OAihy0BjWXiN+XJP1+Fd1VvYpgH871aBsZnWjSspQEPwv7g8pcRbfA3V3NeLMtb5YAk3xNyEipY+6g//h/YYCyBE3uV3jCDhZiJzUB8KZuT/ETdBzvOMeuKYAeRP4Jc6Z1fQasvMRX++OLlQgX3KrNGKhLPfe3CtweS2PbOes76YOveWlSPefJQb1csvquIAOXTwjRQdo0DiephiMwrueLByauw337tPMDGL3xd8Dvxjg8MdnVRsINjcyDc4QuBMJ32SYNnG3+k44L8RMq2XntQs9hqe1tgnXPZjVvbdAOlQ4EctlnrZZjt0m3i17GtPnilQGAQJZUdzHw0oN7z3CWp0cM3jC6l6aipcFsEbb5/rOneUf5MwStZlPZxeaLCMPIw4UEHQO+9GmAJZ7gEoJ1US7ecxY7wrn3rigFU2+jVTV2kuBxjPS/w3PtbuavcuWyBWZ0yk+8e41JBXaQCy7hD5w+49dytQlmJA+Qbxm26ggraFRv6iAj2sTEWmX5UAg= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(93006095)(6041248)(20161123562025)(20161123564025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(20161123555025)(6072148); SRVR:BL2PR07MB2419; BCL:0; PCL:0; RULEID:; SRVR:BL2PR07MB2419; X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 4:8DiUjjArSqC2kD3oROzN8bq7OU6/RjR6DzyIFvoko65D72UxHGlby9ew1koBh7C4AV8nkH1X7tzrCFIPDZQ+t+mTALQAuHZXwHqoaJVIikDY9A0SndfIAFsFfixUEe4UO1qGGZblelPJtAoll9dPyuELC+zyGuSnRFPp4yas0UBWC9HIhiaXuQanJjgJWvdLg6FhdDHvvGIeT4ipzPwuqGmDhC8xXgvTcrChuu+9ZUfNXhjkumGDD3KuZub20N3oT623hhvUhKdoSuB+NyPhIaitTP5I+rMskqIcUe7C+JDPGAB0D/Oi/fmSo6WxMOgOD9fHWspAG99ZNELpUmbH3ftgNBy0fkSk4HZoB5SgCncs01mWTbIXiZIvFnJS+7zGHnZyU/9aV68S76i62xbENXWmoFGesvG02QSRhiwvgco713wgQQG6/YIMniMQ0xjpmpZCAhdTMiGJO0GYcXgI+z+Nprcg3V4yZwkaYEnE7PPeLftFuh0W0LRT8mFjdKqsroBQXmArVzWTRAwQ7xOQJPltOvecHN1rZLkJGdzYSMOmcEL8JOeS9PiIqDY1HwCwtkqhTSVxE/ix6JYTQeL2raHwFaj/cpQMICp+o1Oy8ZGVkDv1yNLUAOprjbFpEv7XloqMkQ8zjc9hAJszZTfQSDhkqOPqy536MzCUeJfAlXeRe3FX6cFCWPhEdSwHmxGw X-Forefront-PRVS: 029097202E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39410400002)(39850400002)(39400400002)(39450400003)(39840400002)(6116002)(3846002)(5003940100001)(53936002)(4326008)(305945005)(110136004)(38730400002)(107886003)(6512007)(53416004)(50986999)(76176999)(189998001)(1076002)(42186005)(66066001)(81166006)(47776003)(6506006)(6486002)(8676002)(50226002)(2906002)(33646002)(25786009)(2950100002)(42882006)(36756003)(50466002)(6666003)(5660300001)(48376002); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR07MB2419; H:1scrb-1.caveonetworks.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BL2PR07MB2419; 23:CZN25km9+iOLLkC0FsPWxUHuprWc1nqmul4dCcuOx?= =?us-ascii?Q?pHQeit2Ygf5QEVdW1ysaYbIMjgUonaSG7/nECW930yUCeT2DOfaVXGX1xQSH?= =?us-ascii?Q?VgXGW7F5mr3JvJC2ydv156Y2B3P7ORjQ6W/iEC+8xwBGEuNQisMGVFchi8Iw?= =?us-ascii?Q?EXb++ntDY7/X/PI3wIToOyGsnm6Q0RhcJm5g+HXtWJ7mT6hdvEUFCo7DzkWz?= =?us-ascii?Q?bUclUb/F3HWjs7egFfJx6ESg9/a533arsiTukqdVsWfk4CInU1Dh/MocegVT?= =?us-ascii?Q?sxYsMkC1NsW9vhkzk+gTBKo2MG9JDyonb8cEcSWlyXKY88fMwfcLiGeEHxXZ?= =?us-ascii?Q?dVsU+Cyzcd8NApP5KjUMa7WQwT2YvJ2HlHeAqjGQ22L6EvVbQmZTwxAht52n?= =?us-ascii?Q?XfzoXr10Q+6yjNvuPoo/Zh/vHBYjPCEMwvTs51exw3VOGVHZyKxY1PzY7MOc?= =?us-ascii?Q?tGV6Gw7Yb4Vwxi5wLJ8QH1DsWJzZlcT5v2w8KtQTEcHkShZcrYRypfnfwbbT?= =?us-ascii?Q?StvHf5wOBr65zKM5U4eFFUTTemqrI6mQmeLFIOXpuyr8nx2BMhoJ2MsrR1te?= =?us-ascii?Q?vUgR3Vb8pfuXCyPDykuO2w5R1+bbYG9934zsk8OiQv178MuVacSkkB7axCrx?= =?us-ascii?Q?ovUuDZQFFZ0Y2Fpg+pZq93ISWBXjnDsRXNlIeU4FSBcbiNOIqCAfHGs0W0P7?= =?us-ascii?Q?aXHgpGueN9/wxa3ViXxoA1W/VIYTdvHujU0ZpCQhqiBeLnFo2AvKIa/x+PF7?= =?us-ascii?Q?cMy3ioMZDPY4MUq8iNIsXBl0BoIi7Lj5CCocG9mUUAuB+jCbKJzZWcPKTfiP?= =?us-ascii?Q?COtDAAjaLmjBAJYwFMb1qBXEMiCkGgtT4qi7eZEhUBxu+dJxyW/LK6S9cNw1?= =?us-ascii?Q?a2A/3S0/hjaMdWsoUkwfUhVku334pUemHI2tkJYeLV/bHw4jLBHv41g7YOHp?= =?us-ascii?Q?VdEGV++zvJqLqMRtg30XIEcJVl98sbcGA6UCYaF3h9G2YnVHWipRfmGaRr7C?= =?us-ascii?Q?YV1QGgJErjFjDAKvbLHC5f4l/rw02cTA9PeGE9+8VG4+Q=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 6:DGDREZSpoGy9qXKGFGiS4GNmlfsH7NUEnzTrh5ziQBuhbhzxt+6v6DBcH0UhfJ+ycdXrQg771tSmb/ANCsxEg9k6l4TNFASZ0BRPQDYfayR73+/bhtxUKKw6NU3Z18XWjxCn7/jiHLErC4yba/lHkgg9POlqKD99L4qI0OSt2UJnB8ABsDUzCGB/v2qvp3sChnk/L0FR6QHPgr+LUZ2Mg/4Nm7G4+BcCWNuHDEChM4bZJK5UBrQyfl27Eci+z9cchXIpdOP2nApfl2r9Ql/6YMS1ARoNDamLWFo4f31/NU0yzInYXeIGwnF2uUqEBB59WZ63By2B3M2tA+6onkRMYLcSrUFt+IhkxUkNOMXAFdAogX/mOgj/0YPH4T34vJ2Wny2n733RojRKanbDHc9K/ot/9yptdEhOzWdjm7zN/AR/sUox2DuHecPFP/mXoaHRsMMYHP14wrvlK//yERzja/wopDxxf3AtaKxDqtzSxNpZsXQaJ2NmHU2XMEBkGi4wigFhc+EzuUvasO7/LSg+tg==; 5:djsAdoV0+VHFBGKNL0l+bur0tI4gQzLmB2Okrf6rsONB/iMFMuvDh0vl6VF+tpi5pMR5KZBD9j2od37qbCutteTMy2GhjjaHwgSt89o2JvXD18iNWLtWmiHYKhrF/uDuzU8/kuuM8U2ax0ZlnpufLQ==; 24:HvP84pxNHKL/e04ScUv1eHTTBRhkSo9j1JO110AJSvmqkaOEijY/RqTBivWy0lSSbLpCPWeQmEd632SgkxE2ceTGLA0MXqdA5wbmAxnuhzA= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2419; 7:ofr5tbQs8IqS4vTI51PsAk/maqoactroMOR2LkY7mfK7S7pV9J2Xl/llBTe5/l0oHp+XAeiA+HuxTK4030P750wQhYrde7eVwu2MAJ1w4/JYbp0WJ3HmV6MjGWWgrqazos8CUbs1mTVNvD+h9nCNXjXa4hmCw4CYMkxXgksIMaGigqrIRGy3uWz+3zHEvZZvGRMF4K9TDktnszbZ9U/LpzQxeod21EpeNh8x62dYefriZeiFrVQNulGkXKOcWyYPlyCxk5JCXByF1VUv2SFmoRSfDZqjLU1EmOsrkaGiHiI3GC05uvVQ19EAexD81JdktrBt1kTUCwsKTn7RwmZNYg== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2017 12:44:29.9642 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR07MB2419 Subject: [dpdk-dev] [PATCH v2] efd: support lookup using neon intrinsics X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Apr 2017 12:44:33 -0000 * Added file lib/librte_efd/rte_efd_arm64.h to hold arm64 specific definitions * Verified the changes with efd_autotest unit test case Signed-off-by: Ashwin Sekhar T K --- v2: * Slightly modified the content of the commit message body * Added prefix [dpdk-dev] to the email subject line MAINTAINERS | 1 + lib/librte_efd/rte_efd.c | 22 ++++++++++++ lib/librte_efd/rte_efd_arm64.h | 76 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+) create mode 100644 lib/librte_efd/rte_efd_arm64.h diff --git a/MAINTAINERS b/MAINTAINERS index b6495d2..7d708ae 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -147,6 +147,7 @@ F: lib/librte_eal/common/include/arch/arm/*_64.h F: lib/librte_acl/acl_run_neon.* F: lib/librte_lpm/rte_lpm_neon.h F: lib/librte_hash/rte*_arm64.h +F: lib/librte_efd/rte*_arm64.h F: drivers/net/ixgbe/ixgbe_rxtx_vec_neon.c F: drivers/net/i40e/i40e_rxtx_vec_neon.c F: drivers/net/virtio/virtio_rxtx_simple_neon.c diff --git a/lib/librte_efd/rte_efd.c b/lib/librte_efd/rte_efd.c index f601d62..4d9a088 100644 --- a/lib/librte_efd/rte_efd.c +++ b/lib/librte_efd/rte_efd.c @@ -53,6 +53,8 @@ #include "rte_efd.h" #if defined(RTE_ARCH_X86) #include "rte_efd_x86.h" +#elif defined(RTE_ARCH_ARM64) +#include "rte_efd_arm64.h" #endif #define EFD_KEY(key_idx, table) (table->keys + ((key_idx) * table->key_len)) @@ -103,6 +105,7 @@ allocated memory enum efd_lookup_internal_function { EFD_LOOKUP_SCALAR = 0, EFD_LOOKUP_AVX2, + EFD_LOOKUP_NEON, EFD_LOOKUP_NUM }; @@ -674,6 +677,16 @@ rte_efd_create(const char *name, uint32_t max_num_rules, uint32_t key_len, table->lookup_fn = EFD_LOOKUP_AVX2; else #endif +#if defined(RTE_ARCH_ARM64) + /* + * For less than or equal to 16 bits, scalar function performs better + * than vectorised version + */ + if (RTE_EFD_VALUE_NUM_BITS > 16 && + rte_cpu_get_flag_enabled(RTE_CPUFLAG_NEON)) + table->lookup_fn = EFD_LOOKUP_NEON; + else +#endif table->lookup_fn = EFD_LOOKUP_SCALAR; /* @@ -1271,6 +1284,15 @@ efd_lookup_internal(const struct efd_online_group_entry * const group, group->lookup_table, hash_val_a, hash_val_b); + break; +#endif +#if defined(RTE_ARCH_ARM64) + case EFD_LOOKUP_NEON: + return efd_lookup_internal_neon(group->hash_idx, + group->lookup_table, + hash_val_a, + hash_val_b); + break; #endif case EFD_LOOKUP_SCALAR: /* Fall-through */ diff --git a/lib/librte_efd/rte_efd_arm64.h b/lib/librte_efd/rte_efd_arm64.h new file mode 100644 index 0000000..cc93411 --- /dev/null +++ b/lib/librte_efd/rte_efd_arm64.h @@ -0,0 +1,76 @@ +/* + * BSD LICENSE + * + * Copyright (C) Cavium networks Ltd. 2017. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Cavium networks nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/* + * rte_efd_arm64.h + * This file holds all arm64 specific EFD functions + */ + +#ifndef __RTE_EFD_ARM64_H__ +#define __RTE_EFD_ARM64_H__ + +#include + +static inline efd_value_t +efd_lookup_internal_neon(const efd_hashfunc_t *group_hash_idx, + const efd_lookuptbl_t *group_lookup_table, + const uint32_t hash_val_a, const uint32_t hash_val_b) +{ + efd_value_t value = 0; + uint32_t i = 0; + uint32x4_t vhash_val_a = vmovq_n_u32(hash_val_a); + uint32x4_t vhash_val_b = vmovq_n_u32(hash_val_b); + int32x4_t vshift = {0, 1, 2, 3}; + uint32x4_t vmask = vdupq_n_u32(0x1); + int32x4_t vincr = vdupq_n_s32(4); + + for (; i < RTE_EFD_VALUE_NUM_BITS; i += 4) { + uint32x4_t vhash_idx = vshll_n_u16( + vld1_u16((uint16_t const *)&group_hash_idx[i]), 0); + uint32x4_t vlookup_table = vshll_n_u16( + vld1_u16((uint16_t const *)&group_lookup_table[i]), 0); + uint32x4_t vhash = vaddq_u32(vhash_val_a, + vmulq_u32(vhash_idx, vhash_val_b)); + int32x4_t vbucket_idx = vnegq_s32(vreinterpretq_s32_u32( + vshrq_n_u32(vhash, EFD_LOOKUPTBL_SHIFT))); + uint32x4_t vresult = vshlq_u32(vlookup_table, vbucket_idx); + + vresult = vandq_u32(vresult, vmask); + vresult = vshlq_u32(vresult, vshift); + value |= vaddvq_u32(vresult); + vshift = vaddq_s32(vshift, vincr); + } + + return value; +} + +#endif /* __RTE_EFD_ARM64_H__ */ -- 2.7.4