From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0052.outbound.protection.outlook.com [104.47.37.52]) by dpdk.org (Postfix) with ESMTP id 931D31077 for ; Thu, 27 Apr 2017 15:47:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=uuuyUCjkBkkTY6m6plBC3OPcasJq/QmidCbU/4fZEdY=; b=g7Qu9jxRLZHIdcAQy5H7FUNh4yuky0r3WAhS94/dOsfEac0X8ztI30Xroyq+6ClEARtnBIXsk1OPWQTrPfja7WTHakjlJaWGxW+NHPTONW/8mbUgAGPM5HjN3BNGAAcN2qswhA7n4RYg1vgypjhAge9j1wkNRDNJr6ak5ssz0hY= Authentication-Results: caviumnetworks.com; dkim=none (message not signed) header.d=none;caviumnetworks.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from 1scrb-1.caveonetworks.com (50.233.148.156) by SN1PR07MB2431.namprd07.prod.outlook.com (10.169.127.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1047.13; Thu, 27 Apr 2017 13:47:35 +0000 From: Ashwin Sekhar T K To: jerin.jacob@caviumnetworks.com, jianbo.liu@linaro.org, cristian.dumitrescu@intel.com Cc: dev@dpdk.org, Ashwin Sekhar T K Date: Thu, 27 Apr 2017 06:47:27 -0700 Message-Id: <20170427134728.35392-1-ashwin.sekhar@caviumnetworks.com> X-Mailer: git-send-email 2.13.0.rc1 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: CY4PR02CA0012.namprd02.prod.outlook.com (10.169.188.22) To SN1PR07MB2431.namprd07.prod.outlook.com (10.169.127.143) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cc3ec607-819e-42b6-36be-08d48d73f68f X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:SN1PR07MB2431; X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2431; 3:aJq5ufExzcgoBUt2DUMRMwSvzKmizJLdxQY0Y6KWh61vC0UQXLvswEFIieW1C9versvhCPJ0ZOrSsv51vBLf5glJhPJgHCXcaR2GxVRQkemYRi/TG1A6XTf/de+hWXwU4nGdvDJL3v7dryNzBmx8mQtbjSuXllwGuAEJmhlBxDxLi5knBMugToeSWbGnbqM+a88C0fgd6miP46OetZ+xVaGM9M6JrU0DVq0d8AHGVPjzdaUjgm5lB2AT1ngKhlmtxBmaLURMWA1vrBANR9A8cX2aGvr/CB4LQadtGS0iWsfI6zmMjrlYVF0L8oEt5U2+RN8eB8Vc9WgYyDtjHeKmcw==; 25:DnqoDuL3lejV4J40yCAFOl8mrsblQ+E9HNPJFT7NVDkd9sBW+UtkFd9O0cOy2MQoZn7N0qiJk5ijN36n/88Q9HgOWn+t7EpZXDAPPITbqz89thPHM+GaWUVGDJqEIJhF2c2ExIKr43n5bjWttkOz+o/daAamE+j0FoHhHNBfHaggnfryFGlnit1vyt/Wd3U3tM35QfeGb3rIgCmwWF+ymquRUlGA+c48mjW87d8MqFV5ePzGj7g3s0vWtJxx5odOfQqP2tPhK7covIvC6qFHwgUIw9lwZyBKNhJSKusGUCAzGYdcwOFgo4VzC939coTZ+LPMiH67fq4IAWmR28n2kMGBxYq7Hb+6feNXLtfAhhVzEk1JWkEUL8OEvrHGXlMrMulH9/qAGBDVmlsw2TjufIkfgYusA3MlXCTOSxFHy/2HZiAyaIZcj5ZlfPC3gdhZCgTwNtlKGaNi5shJXMXINQ== X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2431; 31:WjZvpxuDmGuDkhmhAK7IoylD6QArUzqY3zGCNGbca4J95aScrBFwl6UoI990ErAKsmp1+wDu/F6TsTJpVUIsG2H9nYaNteyAUjCh2c37ZYbSCLwsKeEl4vfKHh7WEpivAM34ThS/VHOK7C7nZ49Kt0OKoeR9SYZhE/8pZaLotf1pcchkxdUAEfskOvgKVQrJi6Oyq4BsnaZqR1Uqir77ZNY/swtleXnhYEfLVoAgcL0=; 20:Lbfp9sRpHLmiyvFtxE80kfXttyTO5TThsq3OoBJBg8ZwN1xGZdGfLo/5BoNEYeY/QuXlRfCbVAOl/mo0k2UvkTKJjHdmXH96ZBI8KK9+MVUnRewNBu14shveqYSEck1UpBnobNDEIs+WeYAso9Lh+2u42+7QWXqOJS3zP1FROiJuWrM5XEcxfCW6EixTvtwfCe31Z1BceESssn93Sg4QRu8lUwHrguWcOxVlqmISyRzcH7FrIEXG6a9J+NpV9men6I0d+ZvgZ8ty38m/Qb/DEEoyIKZc6xfoilXM6hhosi+yv5O2wMqkzBMRI2zzya9rdWVJGFEQol0xHEP+VDcqIcVF4rSJ18ga1QLKByOWgiLewGPPHa6yEK3CUCAcYMqJSGqY8JBwGQtkQ6I8M8oIkwOZkYP8zm9pupVaE1qgDuISG5f84zyXCqyAHn/ki7D9V4NPV5jn4Yfcglq14NBUrYxLLl7qi79qtvPfimI8ykzRU06cjnDAezN4x6vg/VjuiHFili+winbCVZcVbKcqNvEQDMESmVygv9VrBCg9i4QksD5mSVX+FypmjxiiugeXwP6rVmOwRXBn6GuuekleycrMVzIQCTc12/XAB51nP3g= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(93006095)(6041248)(20161123564025)(20161123555025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(20161123562025)(6072148); SRVR:SN1PR07MB2431; BCL:0; PCL:0; RULEID:; SRVR:SN1PR07MB2431; X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2431; 4:kH2It8aPCsJAztTdO0Hg1OwmdcioubazPHSdyp9WE6U9MIIqloRG+63Gkw505cBAfy/dLGlnVfaQUAlNcHnIUwzTkq+VqvYKby5kRBlmDUtUunvtmlABRpw/CSEdsb5eWo9TLMoLVRKEaPdbDdEJzjf0DTTLBKCs9bWowz0fkpMAWsNb5+GYAh60P71yVWsCsVvd//1Sm8bbJNacBopUTEiSFvLp5bdbu6yfk0kC1HMv0fcNKXa/3t71dlcYURiRJPbNE0XDM6khufp7mrGy2v7zNBips+7m+yHv6aJ/E/X7rg8Wb6W+yEY+Dt1HfNNkyRWRhTop1m/SjnDQwX2BlseDC4HKv995v5VGPGGHsEpWMbuuTa/rtXVSyXiuMu4yC7FGrMFUwFsF+A67KEIGGTbA4RQYab0v9o7UBh0WAMCkkB+UomJ4IIpmZKyJlpmw00x23SzKPX+x+N4OG/4OiCZMOxBms+f4yDxYgsL2b1+19O91WbIq/Ok+145xMv9kjxoNt1VUbCtVRmQtEBzDulJAsJvOdTfgVwHk9R+9AHnAql23O+lF32NPZJlWfZiSpO9C6DfAGLzaWd8pvMYrLwGWVAI0JbJkPsgaoTs1o8O28vmaDp/2Ijg8zGgap8fgtVM86r6BcMH5OoIq+kn+aEP/fv+q5++7xJz5/v6I5f5U4ajyVSGnaMIEsOK/zK3F X-Forefront-PRVS: 029097202E X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39840400002)(39410400002)(39850400002)(39450400003)(39400400002)(25786009)(4326008)(33646002)(6506006)(6486002)(5003940100001)(53936002)(42882006)(6666003)(1076002)(189998001)(6116002)(3846002)(36756003)(6512007)(5660300001)(66066001)(48376002)(50226002)(50466002)(81166006)(42186005)(110136004)(47776003)(50986999)(38730400002)(107886003)(305945005)(2906002)(53416004)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR07MB2431; H:1scrb-1.caveonetworks.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN1PR07MB2431; 23:aL3fhoOQllVidyy9m77AYmm2DyKCJEmdZAK2DSnAp?= =?us-ascii?Q?JmF4LXdd/xGHskmwWKbbVW+ZV+UOvo0AJGxkEzD42p/lu2faw39PiXvBeh+i?= =?us-ascii?Q?S7l2fvCcVDgX3d5WHR3oVO2buJK0Td9M7xsKrQy6geZZLk7XqBVaOHOlkqbZ?= =?us-ascii?Q?7J4X8kbTNY9O1x8UmlPWE0F6NyasSQrHHaeSI4ZzAXfHcIO+rOeRM+oJMsGd?= =?us-ascii?Q?m87PphRB8tsAbzCo7iPDhC13eW24pyetQp3ckOgIl1QHzfsdZ5V+IAG8TM33?= =?us-ascii?Q?vKvnq6ND1Cp4X6M7KY2akAuy4rVgAWKg8soK7cX2aZTgJfMPecvBKnfuaZ/m?= =?us-ascii?Q?D6WgI+qo0jMKmVYTTwFC4E01tGhHqgwTySk2L2fWpoaRvyy87uiph0yd1/+z?= =?us-ascii?Q?LLCjgYTzZCkgA0JBuFvNc8zkMUzIyw0EuHBNbgm75AMQXfomNxUSeIzM18m9?= =?us-ascii?Q?Y2GDqMrIYcd7Q1R8w/koJNz68GCPHTUaRiLNlV3HEZz6kAq1eMCH13RCRRA8?= =?us-ascii?Q?iC4JsUndl/ROKWWeQHDn7MCL15qe19vtZeJe3j5c0T+aJ5r0YfZwnmaLfvD1?= =?us-ascii?Q?lmYAVQBqc/tBwt6A0t4NJfZeeY/RhmEKjwZFySfxSH48RzdSBfj+yW/T7aj8?= =?us-ascii?Q?iCUYxG+e7azIpwG7JzIa9iuUw9H1UbWKT6ByEgUD1aMJWsMdddP5KsnhP4fo?= =?us-ascii?Q?1K8rtXxQYyKdmjKsAIKtWe4vQfyIn+PpejsDz27jsW5weAUae+7Rs7Kb/SkR?= =?us-ascii?Q?FPSzuesN5YD2o9UsuWheUk4EaJI3XidmuZlgQFOcoEwUdLG/XkojZaIJUKVY?= =?us-ascii?Q?9UmG9csZsEpzhKcdNCt61OiZhfoOTNgFP9WekLZOHFVJqTigv5rwOnS3y937?= =?us-ascii?Q?eeDVNK9xOrVHqsrKErL+ZPPcF6MkgJp4u7JZX3ZO0pBUSTxYvgt38lWajj05?= =?us-ascii?Q?QAwVv0xUTdTCiHbEX2zohEQ1085MmFsjcbgEAmzA0LprAQ9arhN/gpBdMyvd?= =?us-ascii?Q?C8=3D?= X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2431; 6:RWZUisJGHHP1a2MBlplhAWIykveDn1N3hKTuSDDirHB72z+weguAcCr2yBvSlvk1CMxHPbfj40tTjeNGlBbkNQOlfxLuKULUO+V5tVZgGdYEu9Ii2HjEPcCama2yqnNwjlh8rJ3u6sQAmzVfgslx9z/tHeAb0zS69faHRXpyltseIJBIDAowks1XAwWvhjwuAPuajmgyFfZYtiWxTbovKeMVFHPaI+DKyw8gEL6KjlgVbYVeM/Gzl92YbwTUdjbEHPp70RlGt1dLHyU7uJpTcijMFcGURbzl+9EmFVvhRgKKb7lL2zZ8gTgS0gZ1fZ+/80DGtoXBntoL2M3p4SA0EVANZj0WFjNMK7CnG2MDhn29I6j0Y7g6hbUVplKuSlcwgYeH5E0bqkYJO27yRxNKO3Oe2Y+m57KG2O4cA+cXatoJ/NsbKjNnwBu6c1evqvm3EFpNFt+5/FozW+7z0TihvN1uy/V7j840btEH5Szn/x1Dv2kl40dq1LptJa0IrLUy49Zeh5LGGzRygc81nHJKTg==; 5:TwXUxpal8x59uo3wjJqS4DxKHxJAUYfSwjMWI9Guz/D53ulk4GQl3KG0yKmnKxG+Lb7Bg++9oXxrNo/Cl+V738Jer1rtBKeIDctEgff0zWXvXOXPzq6ljtLf8ArxmfBOSRd7axLAYyZLFuGwGEZ8rw==; 24:cFxzrdNCNEEuCPa0Zyld5NWLNu4N1vDUi+JHVL5GQbNuFSv9j6Novy2eBzyRZnd812Jp39BNCpuwPDHduHJRp+2sqJemSovdrUW0cvpZSkQ= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN1PR07MB2431; 7:+FEfWyB8g4xeRp3AuHlT3X5cvdW5ZsUHFzS+vzf26r8af1D/ivMbu4PMedFv3Zhc6ZAWo9iuVXuX+2YZJHbQ06ioyOMHkHnItYWmAmSkyfXt02Ecg6j0wrytUp1N3rXd4/XXnO/pYvZrTJbVaE/6PESX7t+FIFBe7tMnKDuch2voy3Xa+qKmN/V6ksa7ZzOIFxdPhzf3QFJi+UC0/2BG6BxuUHa8D+fLyC5F1AZXoAeIN/V67EVfsegYuFLXJPKi6T/OPSyN6l4zJJvdPSQnARGTy3U5T0qVafsaLzUQuk50ZRRN/D6ZouFMWUssfI9X9nDXv0lD0XCjHFyHb+KfCg== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2017 13:47:35.9635 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR07MB2431 Subject: [dpdk-dev] [PATCH 1/2] table: separate out arch specific defines from lru header X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Apr 2017 13:47:39 -0000 * Moved all x86 related lru defines to rte_lru_x86.h while retaining all common defines in rte_lru.h * Verified the changes with table_autotest unit test case Signed-off-by: Ashwin Sekhar T K --- lib/librte_table/Makefile | 1 + lib/librte_table/rte_lru.h | 106 ++---------------------------- lib/librte_table/rte_lru_x86.h | 145 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 152 insertions(+), 100 deletions(-) create mode 100644 lib/librte_table/rte_lru_x86.h diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 0d06d36..f40cce7 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -69,6 +69,7 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_acl.h endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_hash.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_lru.h +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_lru_x86.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h diff --git a/lib/librte_table/rte_lru.h b/lib/librte_table/rte_lru.h index e87e062..2fe87c8 100644 --- a/lib/librte_table/rte_lru.h +++ b/lib/librte_table/rte_lru.h @@ -38,31 +38,13 @@ extern "C" { #endif -#include - -#ifdef __INTEL_COMPILER -#define GCC_VERSION (0) +#ifdef RTE_ARCH_X86_64 +#include "rte_lru_x86.h" #else -#define GCC_VERSION (__GNUC__ * 10000+__GNUC_MINOR__*100 + __GNUC_PATCHLEVEL__) -#endif - -#ifndef RTE_TABLE_HASH_LRU_STRATEGY -#ifdef __SSE4_2__ -#define RTE_TABLE_HASH_LRU_STRATEGY 2 -#else /* if no SSE, use simple scalar version */ -#define RTE_TABLE_HASH_LRU_STRATEGY 1 -#endif -#endif - -#ifndef RTE_ARCH_X86_64 #undef RTE_TABLE_HASH_LRU_STRATEGY #define RTE_TABLE_HASH_LRU_STRATEGY 1 #endif -#if (RTE_TABLE_HASH_LRU_STRATEGY < 0) || (RTE_TABLE_HASH_LRU_STRATEGY > 3) -#error Invalid value for RTE_TABLE_HASH_LRU_STRATEGY -#endif - #if RTE_TABLE_HASH_LRU_STRATEGY == 0 #define lru_init(bucket) \ @@ -118,87 +100,11 @@ do { \ bucket->lru_list = x; \ } while (0) -#elif RTE_TABLE_HASH_LRU_STRATEGY == 2 - -#if GCC_VERSION > 40306 -#include -#else -#include -#include -#include -#endif - -#define lru_init(bucket) \ -do \ - bucket->lru_list = 0x0000000100020003LLU; \ -while (0) - -#define lru_pos(bucket) (bucket->lru_list & 0xFFFFLLU) - -#define lru_update(bucket, mru_val) \ -do { \ - /* set up the masks for all possible shuffles, depends on pos */\ - static uint64_t masks[10] = { \ - /* Shuffle order; Make Zero (see _mm_shuffle_epi8 manual) */\ - 0x0100070605040302, 0x8080808080808080, \ - 0x0302070605040100, 0x8080808080808080, \ - 0x0504070603020100, 0x8080808080808080, \ - 0x0706050403020100, 0x8080808080808080, \ - 0x0706050403020100, 0x8080808080808080}; \ - /* load up one register with repeats of mru-val */ \ - uint64_t mru2 = mru_val; \ - uint64_t mru3 = mru2 | (mru2 << 16); \ - uint64_t lru = bucket->lru_list; \ - /* XOR to cause the word we're looking for to go to zero */ \ - uint64_t mru = lru ^ ((mru3 << 32) | mru3); \ - __m128i c = _mm_cvtsi64_si128(mru); \ - __m128i b = _mm_cvtsi64_si128(lru); \ - /* Find the minimum value (first zero word, if it's in there) */\ - __m128i d = _mm_minpos_epu16(c); \ - /* Second word is the index to found word (first word is the value) */\ - unsigned pos = _mm_extract_epi16(d, 1); \ - /* move the recently used location to top of list */ \ - __m128i k = _mm_shuffle_epi8(b, *((__m128i *) &masks[2 * pos]));\ - /* Finally, update the original list with the reordered data */ \ - bucket->lru_list = _mm_extract_epi64(k, 0); \ - /* Phwew! */ \ -} while (0) - -#elif RTE_TABLE_HASH_LRU_STRATEGY == 3 +#elif (RTE_TABLE_HASH_LRU_STRATEGY == 2) || (RTE_TABLE_HASH_LRU_STRATEGY == 3) -#if GCC_VERSION > 40306 -#include -#else -#include -#include -#include -#endif - -#define lru_init(bucket) \ -do \ - bucket->lru_list = ~0LLU; \ -while (0) - - -static inline int -f_lru_pos(uint64_t lru_list) -{ - __m128i lst = _mm_set_epi64x((uint64_t)-1, lru_list); - __m128i min = _mm_minpos_epu16(lst); - return _mm_extract_epi16(min, 1); -} -#define lru_pos(bucket) f_lru_pos(bucket->lru_list) - -#define lru_update(bucket, mru_val) \ -do { \ - const uint64_t orvals[] = {0xFFFFLLU, 0xFFFFLLU << 16, \ - 0xFFFFLLU << 32, 0xFFFFLLU << 48, 0LLU}; \ - const uint64_t decs[] = {0x1000100010001LLU, 0}; \ - __m128i lru = _mm_cvtsi64_si128(bucket->lru_list); \ - __m128i vdec = _mm_cvtsi64_si128(decs[mru_val>>2]); \ - lru = _mm_subs_epu16(lru, vdec); \ - bucket->lru_list = _mm_extract_epi64(lru, 0) | orvals[mru_val]; \ -} while (0) +/** + * These strategies are implemented in architecture specific header files. + */ #else diff --git a/lib/librte_table/rte_lru_x86.h b/lib/librte_table/rte_lru_x86.h new file mode 100644 index 0000000..b55431b --- /dev/null +++ b/lib/librte_table/rte_lru_x86.h @@ -0,0 +1,145 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __INCLUDE_RTE_LRU_X86_H__ +#define __INCLUDE_RTE_LRU_X86_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +#ifdef __INTEL_COMPILER +#define GCC_VERSION (0) +#else +#define GCC_VERSION (__GNUC__ * 10000+__GNUC_MINOR__*100 + __GNUC_PATCHLEVEL__) +#endif + +#ifndef RTE_TABLE_HASH_LRU_STRATEGY +#ifdef __SSE4_2__ +#define RTE_TABLE_HASH_LRU_STRATEGY 2 +#else /* if no SSE, use simple scalar version */ +#define RTE_TABLE_HASH_LRU_STRATEGY 1 +#endif +#endif + +#if RTE_TABLE_HASH_LRU_STRATEGY == 2 + +#if GCC_VERSION > 40306 +#include +#else +#include +#include +#include +#endif + +#define lru_init(bucket) \ +do \ + bucket->lru_list = 0x0000000100020003LLU; \ +while (0) + +#define lru_pos(bucket) (bucket->lru_list & 0xFFFFLLU) + +#define lru_update(bucket, mru_val) \ +do { \ + /* set up the masks for all possible shuffles, depends on pos */\ + static uint64_t masks[10] = { \ + /* Shuffle order; Make Zero (see _mm_shuffle_epi8 manual) */\ + 0x0100070605040302, 0x8080808080808080, \ + 0x0302070605040100, 0x8080808080808080, \ + 0x0504070603020100, 0x8080808080808080, \ + 0x0706050403020100, 0x8080808080808080, \ + 0x0706050403020100, 0x8080808080808080}; \ + /* load up one register with repeats of mru-val */ \ + uint64_t mru2 = mru_val; \ + uint64_t mru3 = mru2 | (mru2 << 16); \ + uint64_t lru = bucket->lru_list; \ + /* XOR to cause the word we're looking for to go to zero */ \ + uint64_t mru = lru ^ ((mru3 << 32) | mru3); \ + __m128i c = _mm_cvtsi64_si128(mru); \ + __m128i b = _mm_cvtsi64_si128(lru); \ + /* Find the minimum value (first zero word, if it's in there) */\ + __m128i d = _mm_minpos_epu16(c); \ + /* Second word is the index to found word (first word is the value) */\ + unsigned pos = _mm_extract_epi16(d, 1); \ + /* move the recently used location to top of list */ \ + __m128i k = _mm_shuffle_epi8(b, *((__m128i *) &masks[2 * pos]));\ + /* Finally, update the original list with the reordered data */ \ + bucket->lru_list = _mm_extract_epi64(k, 0); \ + /* Phwew! */ \ +} while (0) + +#elif RTE_TABLE_HASH_LRU_STRATEGY == 3 + +#if GCC_VERSION > 40306 +#include +#else +#include +#include +#include +#endif + +#define lru_init(bucket) \ +do \ + bucket->lru_list = ~0LLU; \ +while (0) + + +static inline int +f_lru_pos(uint64_t lru_list) +{ + __m128i lst = _mm_set_epi64x((uint64_t)-1, lru_list); + __m128i min = _mm_minpos_epu16(lst); + return _mm_extract_epi16(min, 1); +} +#define lru_pos(bucket) f_lru_pos(bucket->lru_list) + +#define lru_update(bucket, mru_val) \ +do { \ + const uint64_t orvals[] = {0xFFFFLLU, 0xFFFFLLU << 16, \ + 0xFFFFLLU << 32, 0xFFFFLLU << 48, 0LLU}; \ + const uint64_t decs[] = {0x1000100010001LLU, 0}; \ + __m128i lru = _mm_cvtsi64_si128(bucket->lru_list); \ + __m128i vdec = _mm_cvtsi64_si128(decs[mru_val>>2]); \ + lru = _mm_subs_epu16(lru, vdec); \ + bucket->lru_list = _mm_extract_epi64(lru, 0) | orvals[mru_val]; \ +} while (0) + +#endif + +#ifdef __cplusplus +} +#endif + +#endif -- 2.7.4