From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0042.outbound.protection.outlook.com [104.47.33.42]) by dpdk.org (Postfix) with ESMTP id 40D5F2935 for ; Fri, 28 Apr 2017 10:58:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=h/fgDr/GNFbyxClOAYGt6IyZAbvGopwok7duR2c4Yqo=; b=X4yOr397E2x2Vrojfl/cpvw9Y69UOmLt9JljTBDeO296uNe8OKLFnbk+RnWFymbg98xWU4m73MWfMlLVyoBPp7vckzpfjzlNeTgK6z9DFmh5UmRzrNq0gvv+TzmpubnIaDK6lAWj/O8dYLWbFzIPM9rD2X/m1n8030grMMfLC88= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from 1scrb-1.caveonetworks.com (50.233.148.156) by BL2PR07MB2418.namprd07.prod.outlook.com (10.167.101.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1047.13; Fri, 28 Apr 2017 08:58:39 +0000 From: Ashwin Sekhar T K To: cristian.dumitrescu@intel.com, jerin.jacob@caviumnetworks.com, jianbo.liu@linaro.org Cc: dev@dpdk.org, Ashwin Sekhar T K Date: Fri, 28 Apr 2017 01:58:30 -0700 Message-Id: <20170428085831.15945-1-ashwin.sekhar@caviumnetworks.com> X-Mailer: git-send-email 2.13.0.rc1 In-Reply-To: <20170427134728.35392-1-ashwin.sekhar@caviumnetworks.com> References: <20170427134728.35392-1-ashwin.sekhar@caviumnetworks.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: CY4PR02CA0022.namprd02.prod.outlook.com (10.169.188.32) To BL2PR07MB2418.namprd07.prod.outlook.com (10.167.101.142) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 330f8e72-fe5d-40bd-8615-08d48e14c3cc X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:BL2PR07MB2418; X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2418; 3:QBWc4DqgJFQACrGygSjYiZksB6rCr3RmKaRuoiwr68IoC9VFAD8Ks9eJ01rxQjNQALPMg+cdgdjd9QI1CjxYV81XLKPcQinLCileeDZ9CCd+v6tGqkNZJJ5dEw0SbF8Nfy364G5m+L2eAVFIB9sZ16leFiGRUpZ1VW2uM6pQL08hyfhGU/sh2C83Iu7aUsUw3ZevaPatVN6RWAckjh2yw4E3m5VqVeNCXrtDM67AErzC5+0dFdSBgiLjBm0EEQYE3wD7/gv8c25fKHhjnoxZ0EslUmOoCVsgXOcPNYqDdhBrVK3mj9Bd6yOC52NyexJIcInUzEut4PsTCXrczN+3Lw==; 25:/T/pjwfPop7VqJT+0RCyiw6RayJBjLOtT/119mqv7fPM7HIwBRjSkIFA+a/Axouvg0S+5XrJV5Fi7ZioYWImoja1FdrGMgGZ9TgiB7HG3tOCJbS/+lHrUmxVr5rKlVH34wuGfz5uZIJuzK0n/dqqzq8YhNt7wRTwXV0LWyYWZcFKQsKsYsaHtGVdXJvcZpzwPd4GMMvA0rfOTJccsQllKXyMbhGLUd1x93h9J6XLC696VppTBFdPLncma1U04s3sJUagjuK+eoJjx5Vr8CgYC0OZwzCMY+C52rDVNrMJkrGJX3MGj7kxoWCxe9a7ApFaPKNBqjuU2nCGYlsTSMpfX3jpgK5PpfaeUFFHNnrakdCOqORDKuhzQD+8uFeJQ8h9kH0FmmryCEa7VwQtjFgupBkxiarnbIkptmWdG1h8c+oyKZPx80fL+AQZZ4ExwXuYaNO05+uP2tLCJs6sawR3UQ== X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2418; 31:LPzXGYHAHqPMMBrn6Jqymim25hUjQtuww6QPYjVtzOls7jc/vq3kbTZYWVZLs2X1216Z+PoLJYjLyO6PCGE4reJf7ZbteW6WpL/bTWQDF3Klh5s+2i2Fu2+bkzMHY6awzSfAk77sqGh8/H0fg47zoNxlsNxakWVOmuHTeTaycdgTOpKevTOdF+/5GJH2FEbJciXQe1sBCarxMhegz3iBfLMZq4uNZvLP4yqjpFHLggRhzPdpgLVFdSKTi91y16rDBohAgZxT4HkKoYJ40vgI+w==; 20:dOeLwnVrbovwUTUi03W0M2pZrUwxm1rzKgrPJqE814ommiuTQsHzFsnDTq3ytwTT+bbNXdmTy2we+5Eh4cTlUatcCvLUoRN9tCreTp7uDw2zYGKCzH5O9OJVQ8fbqD5Cfb2dXlqqWwqUgXCRE26Obo8k5/vQGZ3A5AHDhu5BXwz5mngSGJ+fCoXSn1BjqN2EgW3+M3aoyucOsXzGKHenqusDH6qGeGgKy0kXaMjHmp3w4dcz9RQs/sqZom/+2y3NoPWnCo26t+YjYJuXU8bBcLpERdtfroEHHG7TN50h1WWzHnwnEyjQr3h7nlq8VWTii1k82T4mNVj9Zf4xGZGe9ImxOyRCEBBX0cCey2J1xDHf6hHFkv7bvhl+kMNGhn/P0ZBYqoxyybo3+T/HIwqrN1/KgD6oNGL/4FH6x9m2hfWzujzKVi5tKAuEQANL0C7ulxIemg9aPJaVhYx2eVbYeiqUii8lWvOT5nz1zXW+NEEa4InP+T2lL5rYTe+vqF5v55dSN3Zc0D/6WeLwx2opqK2ZK9aTpUAxGC0z8xzNur3++AiCQ7wpZx828/e6hF3DPdIligQ/zvy7MNblFeip3m6riImoxoFqrvszWGVXHtQ= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(5005006)(8121501046)(93006095)(10201501046)(3002001)(6041248)(20161123560025)(20161123555025)(20161123564025)(201703131423075)(201703011903075)(201702281528075)(201703061421075)(20161123562025)(6072148); SRVR:BL2PR07MB2418; BCL:0; PCL:0; RULEID:; SRVR:BL2PR07MB2418; X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2418; 4:a1qO6iY/aUyYMrjtmtzWSjVLt49RdIBplK2ggbNZQ+94WUhdPLTGqWSgcK2wUbiGa5S5qQF51KTBVkhTqyosAcovojVc0yynOggYB+/7hI33kGrIN7xGhzZEOgRZj9hcwLcpRAM+/p3/4hvF6QEpu1U1NUIjPUDsv1iK6Fly5wpYTfG869/WYdcXfZWqQTyqZXmrtFmFUP5xQrPxAPM1aIeqnDc1ZvtulCg1msKwXZux8z5U8gG6w2AhdBWSjjh96DKyTzqI1P4fp4saOrdYA9FAl5s7InVSFxVDd3nJggRcjGkOAJuG6RyWSJjymGOq6f8evyLjJOgHLygLav63u8SAnXOt8G60ly8h4WHU7baFbfx7XMlXtYqc2gwfl8YZ2IaGajpps1HAxLy1KdEm4KO3Ap0PaKQYmHaTR644PpuodW8//ZpTx0J0v3vSXQglk6LwmtAG5UxDJaT78tg+SkXiVaskZv+2G4s6CwDyF6iEbuf/U2kmDU7h1xCvNvdGy32rTeH/PETIvs/HGyHavnxAYj1jot1SKTWdFjd8kP+3acpW50/Eup1BOF4tiH4UB8dDDXZM37LE6r+SkHY5GGX/CmA1uaqCbseifXNX5OvQI1W1Sq8tkrt1o6vggCyEOwLtsEkuDw6YO/lm8nqPUIB6tZR0G3Vb2TbeJD45RdzxTfSyREA4W+FITZm2WDevFFPRQIh45Q82M7E1acv2E1LSCKZjsOH+RUokjVuI3f0= X-Forefront-PRVS: 029174C036 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(39450400003)(39840400002)(39400400002)(39850400002)(39410400002)(5660300001)(2906002)(6116002)(3846002)(7736002)(6486002)(305945005)(6512007)(189998001)(6506006)(53936002)(2950100002)(42882006)(36756003)(48376002)(6666003)(50986999)(4326008)(50226002)(76176999)(1076002)(25786009)(50466002)(53416004)(66066001)(42186005)(81166006)(47776003)(8676002)(38730400002)(110136004)(107886003)(33646002); DIR:OUT; SFP:1101; SCL:1; SRVR:BL2PR07MB2418; H:1scrb-1.caveonetworks.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BL2PR07MB2418; 23:WmTAVLDGwNWZtwQKcfxhf4ogJhyyO+6WBf6d89ApS?= =?us-ascii?Q?WZb0sYa87azplke0UT3mdsODxKVV0YeiUIN6w/Ptuu65mj7v/38TPoqJOn4Z?= =?us-ascii?Q?DIkRO0eOIo05KOtzCK8TLb6T8bK1nBAgmow2yklVhUUlzG1tuyhXcR3xxpWR?= =?us-ascii?Q?lOPOf1VzjxwyzhiSlD7QLivLVL7FQt2mYOHXio0qT7NPIrXA/xZgmwM7J+W3?= =?us-ascii?Q?wTs6WTBkSR1RsUWfjrlmUYnuHSCW4rFjiRHmWHiaKjfayoESCPxGqPISH/Kj?= =?us-ascii?Q?UrAgrv13I9pssi/BEIKCM+ae4V/i/cg5IIxbzIEctWkkPW4PfvPvhNs8chhC?= =?us-ascii?Q?QKYbHDAqoXApBvKZA681iK8C7Jb0uC4nskN54hXU//NbBYBvO/+fEjUTdaNx?= =?us-ascii?Q?zMQwsqjhPrpV2PPeh2DC6oKp4nIKZSH9KPOnXoeOFFbARFAMacVF50zzd/o9?= =?us-ascii?Q?TL/mPrbFYjyoYLIxcKNN5NIjHS5jNEdF9AGIqI7pUoe2bQjLDGOuL6QkDgTz?= =?us-ascii?Q?x26UsD1KBf50z/OV2I7ODxxlkOrgTcWHVxQsQlzu4PwmMfCjeWxB1F1eKp/R?= =?us-ascii?Q?UmB4JmBZDFt3VUULYVh2NqdJuNoPnY+IBCX6jhNxIu36EYam53rtTtyLunkA?= =?us-ascii?Q?DQKzAqvJXDKjS8HjJOZnYY9Vzlk9ogniZ1kAJCYHmExnDmiArrSWmAvQ+NwX?= =?us-ascii?Q?dUBr9biIAgsP4wqY07J52zBSA5dXujf4rV/1Neh1GDioGXgz+p3PfZwPE7mp?= =?us-ascii?Q?iXWiXRTWoj8xAMSEMVCZ7c6UgnvFffP0Wkp/8DjUfrJ6WDGnF+JdIW+w2GQP?= =?us-ascii?Q?cy70ITLylajjUuLbAxpmcIl/uCOmpy5xq+FdaBRybOC9DWegPtRbHssBgI2s?= =?us-ascii?Q?MkMkc09kOqmle6+tXSOw1YaWSTQJsAffvtvVJe2+eNKOLNqveDF5c44h3wB6?= =?us-ascii?Q?sQ5tNWmb7Vt0s/0RWKe21v+NvW1+qC9ieI5iqT2G6HOJvPnKsgslXwfG7u5x?= =?us-ascii?Q?MfJ8yPI2COT2M3+OQ17fihz?= X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2418; 6:9uIhNeET0QbfuLkP+dCOJE0LrPsa1mC+uryzA6uapqN75E8QTx6cZu/oESozH8Yl1cObGNWmdYqRuPjb4AR5+ppZuKiXJKtHWaIX4CKM9gSme42OXBBF4tsw3qnsr8dr26eCLpvpSTuc9tKeziVlb8BTe+cvqZxk4KwqEFS5t6je7EuO8bkN8paP+H+VOjdqyyq22fskWgkLNFOSFJppOkAUoNegNf2MJaETnf0mBHHBWuyGM9am2LCuyX50b1qA5HrqjoHdlur4LAmZkdYviPWGXchx19p2pQNKlUiiCjVzhAB/l0K1thQsVflPuWTYZJTpUA3UMAMPgq9VqU9P7ea1KkklkamvoP0g4o3AJ4lQqkJqGHWmJ/Q0vHyaWRhQ7Oq4EY50AIpzAIKIp66xemBm2LGh8+ONRTMyakyXpm8uZsvSPnaQTo0xeHPfHL8mtg5hrzSed9eOgaS18mBGHxs7cY4hEPjHsv+EyM41UJJX6sAesppOqYrIL3nUaGGWlNTchuWGUqvZ21YnuMY3tQ==; 5:E9h1Xw3zBgyKBCZ1t3mFEDmCPTJYbUrFoozz5zzf3ENB9+OSl+th+z0ukWZGDbY8HiA/ZuaUGU7WUAEYJbt2veFS1SKEaaxGfCNqzJ3DaagDgu1dv+TpDSDM//OmAU85Icb571Q2a6EjcMsRiJOTeA==; 24:wcYhV9/8yIyEp1xOtzz5F3Km8q1nZThBYe7D8QhcxM4u8Zb3laPkY3n1lIDOqoI7rXVEu5bx2CSk9QmdPDHLAqkb9TP6h62E9noVYXtbYeo= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BL2PR07MB2418; 7:PadrRqboRrLbTUqdp02VWHdZHuIIDG44sGRXs8neC3XmrwURjoO+nhhhHJwVSTQB1sPyuYubHI3uaYzNCTjVGWzdmG2OGm9cBRgfx6Pf4rdmQc6ctk/qQhoeGSLDUJXoX7GlWX+DdAwyOEDqk5Om/v70ym8ZvbdFJDKi3xqp0065bjrAidHhwi20IanYKmYXaT+wJGfWi6bVaNSJZfrjwEbl00PP0LMUiBXVWnDMn0l9cg1CLwQRpyWZRZuDrThhO6Vdpy3TtTnir1eTVKjFRNof6yqxUKuocOouk4TTafliwW97KQVulVx16K65t83y6LuBCoDg7z6x1xEgnMIK7g== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2017 08:58:39.5988 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR07MB2418 Subject: [dpdk-dev] [PATCH v2 1/2] table: separate out arch specific defines from lru header X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Apr 2017 08:58:43 -0000 * Moved all x86 related lru defines to rte_lru_x86.h while retaining all common defines in rte_lru.h * Verified the changes with table_autotest unit test case Signed-off-by: Ashwin Sekhar T K --- v2: * Fixed checkpatch warnings/errors * Symlink lib/librte_table/rte_lru_x86.h only for x86 architecture lib/librte_table/Makefile | 3 + lib/librte_table/rte_lru.h | 106 ++----------------------------- lib/librte_table/rte_lru_x86.h | 140 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 149 insertions(+), 100 deletions(-) create mode 100644 lib/librte_table/rte_lru_x86.h diff --git a/lib/librte_table/Makefile b/lib/librte_table/Makefile index 0d06d36..042babf 100644 --- a/lib/librte_table/Makefile +++ b/lib/librte_table/Makefile @@ -69,6 +69,9 @@ SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_acl.h endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_hash.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_lru.h +ifeq ($(CONFIG_RTE_ARCH_X86),y) +SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_lru_x86.h +endif SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_array.h SYMLINK-$(CONFIG_RTE_LIBRTE_TABLE)-include += rte_table_stub.h diff --git a/lib/librte_table/rte_lru.h b/lib/librte_table/rte_lru.h index e87e062..2fe87c8 100644 --- a/lib/librte_table/rte_lru.h +++ b/lib/librte_table/rte_lru.h @@ -38,31 +38,13 @@ extern "C" { #endif -#include - -#ifdef __INTEL_COMPILER -#define GCC_VERSION (0) +#ifdef RTE_ARCH_X86_64 +#include "rte_lru_x86.h" #else -#define GCC_VERSION (__GNUC__ * 10000+__GNUC_MINOR__*100 + __GNUC_PATCHLEVEL__) -#endif - -#ifndef RTE_TABLE_HASH_LRU_STRATEGY -#ifdef __SSE4_2__ -#define RTE_TABLE_HASH_LRU_STRATEGY 2 -#else /* if no SSE, use simple scalar version */ -#define RTE_TABLE_HASH_LRU_STRATEGY 1 -#endif -#endif - -#ifndef RTE_ARCH_X86_64 #undef RTE_TABLE_HASH_LRU_STRATEGY #define RTE_TABLE_HASH_LRU_STRATEGY 1 #endif -#if (RTE_TABLE_HASH_LRU_STRATEGY < 0) || (RTE_TABLE_HASH_LRU_STRATEGY > 3) -#error Invalid value for RTE_TABLE_HASH_LRU_STRATEGY -#endif - #if RTE_TABLE_HASH_LRU_STRATEGY == 0 #define lru_init(bucket) \ @@ -118,87 +100,11 @@ do { \ bucket->lru_list = x; \ } while (0) -#elif RTE_TABLE_HASH_LRU_STRATEGY == 2 - -#if GCC_VERSION > 40306 -#include -#else -#include -#include -#include -#endif - -#define lru_init(bucket) \ -do \ - bucket->lru_list = 0x0000000100020003LLU; \ -while (0) - -#define lru_pos(bucket) (bucket->lru_list & 0xFFFFLLU) - -#define lru_update(bucket, mru_val) \ -do { \ - /* set up the masks for all possible shuffles, depends on pos */\ - static uint64_t masks[10] = { \ - /* Shuffle order; Make Zero (see _mm_shuffle_epi8 manual) */\ - 0x0100070605040302, 0x8080808080808080, \ - 0x0302070605040100, 0x8080808080808080, \ - 0x0504070603020100, 0x8080808080808080, \ - 0x0706050403020100, 0x8080808080808080, \ - 0x0706050403020100, 0x8080808080808080}; \ - /* load up one register with repeats of mru-val */ \ - uint64_t mru2 = mru_val; \ - uint64_t mru3 = mru2 | (mru2 << 16); \ - uint64_t lru = bucket->lru_list; \ - /* XOR to cause the word we're looking for to go to zero */ \ - uint64_t mru = lru ^ ((mru3 << 32) | mru3); \ - __m128i c = _mm_cvtsi64_si128(mru); \ - __m128i b = _mm_cvtsi64_si128(lru); \ - /* Find the minimum value (first zero word, if it's in there) */\ - __m128i d = _mm_minpos_epu16(c); \ - /* Second word is the index to found word (first word is the value) */\ - unsigned pos = _mm_extract_epi16(d, 1); \ - /* move the recently used location to top of list */ \ - __m128i k = _mm_shuffle_epi8(b, *((__m128i *) &masks[2 * pos]));\ - /* Finally, update the original list with the reordered data */ \ - bucket->lru_list = _mm_extract_epi64(k, 0); \ - /* Phwew! */ \ -} while (0) - -#elif RTE_TABLE_HASH_LRU_STRATEGY == 3 +#elif (RTE_TABLE_HASH_LRU_STRATEGY == 2) || (RTE_TABLE_HASH_LRU_STRATEGY == 3) -#if GCC_VERSION > 40306 -#include -#else -#include -#include -#include -#endif - -#define lru_init(bucket) \ -do \ - bucket->lru_list = ~0LLU; \ -while (0) - - -static inline int -f_lru_pos(uint64_t lru_list) -{ - __m128i lst = _mm_set_epi64x((uint64_t)-1, lru_list); - __m128i min = _mm_minpos_epu16(lst); - return _mm_extract_epi16(min, 1); -} -#define lru_pos(bucket) f_lru_pos(bucket->lru_list) - -#define lru_update(bucket, mru_val) \ -do { \ - const uint64_t orvals[] = {0xFFFFLLU, 0xFFFFLLU << 16, \ - 0xFFFFLLU << 32, 0xFFFFLLU << 48, 0LLU}; \ - const uint64_t decs[] = {0x1000100010001LLU, 0}; \ - __m128i lru = _mm_cvtsi64_si128(bucket->lru_list); \ - __m128i vdec = _mm_cvtsi64_si128(decs[mru_val>>2]); \ - lru = _mm_subs_epu16(lru, vdec); \ - bucket->lru_list = _mm_extract_epi64(lru, 0) | orvals[mru_val]; \ -} while (0) +/** + * These strategies are implemented in architecture specific header files. + */ #else diff --git a/lib/librte_table/rte_lru_x86.h b/lib/librte_table/rte_lru_x86.h new file mode 100644 index 0000000..041b538 --- /dev/null +++ b/lib/librte_table/rte_lru_x86.h @@ -0,0 +1,140 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef __INCLUDE_RTE_LRU_X86_H__ +#define __INCLUDE_RTE_LRU_X86_H__ + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +#ifdef __INTEL_COMPILER +#define GCC_VERSION (0) +#else +#define GCC_VERSION (__GNUC__ * 10000+__GNUC_MINOR__*100 + __GNUC_PATCHLEVEL__) +#endif + +#ifndef RTE_TABLE_HASH_LRU_STRATEGY +#ifdef __SSE4_2__ +#define RTE_TABLE_HASH_LRU_STRATEGY 2 +#else /* if no SSE, use simple scalar version */ +#define RTE_TABLE_HASH_LRU_STRATEGY 1 +#endif +#endif + +#if RTE_TABLE_HASH_LRU_STRATEGY == 2 + +#if GCC_VERSION > 40306 +#include +#else +#include +#include +#include +#endif + +#define lru_init(bucket) \ + { bucket->lru_list = 0x0000000100020003LLU; } + +#define lru_pos(bucket) (bucket->lru_list & 0xFFFFLLU) + +#define lru_update(bucket, mru_val) \ +do { \ + /* set up the masks for all possible shuffles, depends on pos */\ + static uint64_t masks[10] = { \ + /* Shuffle order; Make Zero (see _mm_shuffle_epi8 manual) */\ + 0x0100070605040302, 0x8080808080808080, \ + 0x0302070605040100, 0x8080808080808080, \ + 0x0504070603020100, 0x8080808080808080, \ + 0x0706050403020100, 0x8080808080808080, \ + 0x0706050403020100, 0x8080808080808080}; \ + /* load up one register with repeats of mru-val */ \ + uint64_t mru2 = mru_val; \ + uint64_t mru3 = mru2 | (mru2 << 16); \ + uint64_t lru = bucket->lru_list; \ + /* XOR to cause the word we're looking for to go to zero */ \ + uint64_t mru = lru ^ ((mru3 << 32) | mru3); \ + __m128i c = _mm_cvtsi64_si128(mru); \ + __m128i b = _mm_cvtsi64_si128(lru); \ + /* Find the minimum value (first zero word, if it's in there) */\ + __m128i d = _mm_minpos_epu16(c); \ + /* Second word is the index to found word (first word is the value) */\ + unsigned int pos = _mm_extract_epi16(d, 1); \ + /* move the recently used location to top of list */ \ + __m128i k = _mm_shuffle_epi8(b, *((__m128i *) &masks[2 * pos]));\ + /* Finally, update the original list with the reordered data */ \ + bucket->lru_list = _mm_extract_epi64(k, 0); \ + /* Phwew! */ \ +} while (0) + +#elif RTE_TABLE_HASH_LRU_STRATEGY == 3 + +#if GCC_VERSION > 40306 +#include +#else +#include +#include +#include +#endif + +#define lru_init(bucket) \ + { bucket->lru_list = ~0LLU; } + +static inline int +f_lru_pos(uint64_t lru_list) +{ + __m128i lst = _mm_set_epi64x((uint64_t)-1, lru_list); + __m128i min = _mm_minpos_epu16(lst); + return _mm_extract_epi16(min, 1); +} +#define lru_pos(bucket) f_lru_pos(bucket->lru_list) + +#define lru_update(bucket, mru_val) \ +do { \ + const uint64_t orvals[] = {0xFFFFLLU, 0xFFFFLLU << 16, \ + 0xFFFFLLU << 32, 0xFFFFLLU << 48, 0LLU}; \ + const uint64_t decs[] = {0x1000100010001LLU, 0}; \ + __m128i lru = _mm_cvtsi64_si128(bucket->lru_list); \ + __m128i vdec = _mm_cvtsi64_si128(decs[mru_val>>2]); \ + lru = _mm_subs_epu16(lru, vdec); \ + bucket->lru_list = _mm_extract_epi64(lru, 0) | orvals[mru_val]; \ +} while (0) + +#endif + +#ifdef __cplusplus +} +#endif + +#endif -- 2.7.4