From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A402EA04B7; Wed, 14 Oct 2020 09:24:53 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 59BEE1DC89; Wed, 14 Oct 2020 09:24:52 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20062.outbound.protection.outlook.com [40.107.2.62]) by dpdk.org (Postfix) with ESMTP id 18A111DC88 for ; Wed, 14 Oct 2020 09:24:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zSCzj1uq3YUtIBKeGYFgYTkdAwof9vZJPlM4x+untwI=; b=QuY+ZS6Kw5BO9gBUQblJrKYSQ6nw9ieZXjLHvzVe867hkqQIUPpkongsPPBKkPmF22TD9LhPIJe+Gg6IyHrxO4ogfkBU49HJTvNWshMxeNPpyOHPHAj7TzXjAhgJrRL7JETdXIICwFJfWexGQ0PlMQnmDFdeN/1t79CIU0V0is0= Received: from AM6P193CA0103.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:88::44) by DB7PR08MB3052.eurprd08.prod.outlook.com (2603:10a6:5:28::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.27; Wed, 14 Oct 2020 07:24:47 +0000 Received: from AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:88:cafe::c) by AM6P193CA0103.outlook.office365.com (2603:10a6:209:88::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.24 via Frontend Transport; Wed, 14 Oct 2020 07:24:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT016.mail.protection.outlook.com (10.152.16.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.21 via Frontend Transport; Wed, 14 Oct 2020 07:24:47 +0000 Received: ("Tessian outbound e6c55a0b9ba9:v64"); Wed, 14 Oct 2020 07:24:47 +0000 X-CR-MTA-TID: 64aa7808 Received: from 4c7385af3394.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 8524F0F8-8B70-4083-A6C5-7AF8D606459E.1; Wed, 14 Oct 2020 07:24:42 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4c7385af3394.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 14 Oct 2020 07:24:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=efbKgEz7iDQ9MtDVdwzU8yQVOLcJDc6UNYaRg2ov3Cv01eJ69g0byLYDJe1GnD5McDoqUZr1szfwrxk9Lg9Int64tEyAROQYtsCAUZq/i8WeIArdJdB86lvCbLmwPkOVbHJBb3cO0+S+Jk6UOZGh3T7oUueGkbie95B9JLzoHhzI7Z6pYQd1ppCrdJOQka2hEbZGBIfHF1VW8bcERRy4EZ40TPAa7yuzr2EX2K1ndTLvuDkwH96m74QgdWAyXAZCjRpdz8lGfWBmcj8i/zfIAPxq+b7pu1mZcKjX6Whcy1sRkm8ssM2L4KJQijwf5GOflNU2Ck9Ya/5+vquA8mGr1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zSCzj1uq3YUtIBKeGYFgYTkdAwof9vZJPlM4x+untwI=; b=jZJjP39uuOntmemkzq4vcvUh35TlN5SBRUsNI41AU849FYvux1Oy/iVpl9tRImCZMzgTWswozkCzokxn52eO56iOJOGvNJ9JEEk29rvEiYUoGS9FxOg8mb101+ve15RRulou5q/L/tWs1pU0RBK1e/NnxQ+uqFDl3mL1VtZga3MpfXelC3Iu06yj7co4bECLM9Yx0RnwOl4y1NehG/WhzWtKQZ4lGL/0Lc8LJ9bpnIEPvCiIzSO8hdcydJs+hTmz8GLH8tKuJBfcjgyek21hJaRWhCeQgQIzGhDXWIMEqL2sSPP0gHSQ0q+cUua+iyIJSIIiddMnKynWXLi0us2/ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zSCzj1uq3YUtIBKeGYFgYTkdAwof9vZJPlM4x+untwI=; b=QuY+ZS6Kw5BO9gBUQblJrKYSQ6nw9ieZXjLHvzVe867hkqQIUPpkongsPPBKkPmF22TD9LhPIJe+Gg6IyHrxO4ogfkBU49HJTvNWshMxeNPpyOHPHAj7TzXjAhgJrRL7JETdXIICwFJfWexGQ0PlMQnmDFdeN/1t79CIU0V0is0= Received: from VI1PR0802MB2351.eurprd08.prod.outlook.com (2603:10a6:800:a0::10) by VI1PR08MB5328.eurprd08.prod.outlook.com (2603:10a6:803:13a::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.24; Wed, 14 Oct 2020 07:24:39 +0000 Received: from VI1PR0802MB2351.eurprd08.prod.outlook.com ([fe80::14d4:6ade:368a:204b]) by VI1PR0802MB2351.eurprd08.prod.outlook.com ([fe80::14d4:6ade:368a:204b%3]) with mapi id 15.20.3455.031; Wed, 14 Oct 2020 07:24:39 +0000 From: Ruifeng Wang To: Omkar Maslekar , "dev@dpdk.org" CC: "bruce.richardson@intel.com" , "ciara.loftus@intel.com" , "drc@linux.vnet.ibm.com" , "jerinj@marvell.com" , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v7] eal: add cache-line demote support Thread-Index: AQHWoYBpanbbQ/DXnkGjBCgTcL4/SKmVR4aAgAFrUyA= Date: Wed, 14 Oct 2020 07:24:39 +0000 Message-ID: References: <1599700614-22809-1-git-send-email-omkar.maslekar@intel.com> <1602582191-23807-1-git-send-email-omkar.maslekar@intel.com> <1602582191-23807-2-git-send-email-omkar.maslekar@intel.com> In-Reply-To: <1602582191-23807-2-git-send-email-omkar.maslekar@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: E317258B894F4E41A2289E3DB4DD395B.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [203.126.0.112] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 7eeb8c57-c239-43ac-14d9-08d870123b27 x-ms-traffictypediagnostic: VI1PR08MB5328:|DB7PR08MB3052: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: +VxNBeB+KTk8XsXh1vHArWLsAmhnI62hmfU1+UPDnVqE3RV42FDZSYztBV3ICKiG4V8AlufIUNWpmegCTO0ZS0a68SiyQ6YB/WRuqqY+pp4yiI9RkqhVnY06ibmE1fHB2cPG1ZU6hL3MpTETdgdHBNoe6kWnl8Sdq1PWpsXJIlrm12Uw9VbExijcP5VYXG5MZ0DW1JffP9q+SJrNlTmmuu75LYgCUT4lAylNvlLBSu/auCu9vK4xjP2U5zaSxcF0R7DU7dBFYS+6jdy4j9/BLM1ZkjwnUc5Z5n0R81LbmDAmwwzf5jGX97Cd8RuFvngSda3i7nXhlgTWqXQIZTsMbw== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR0802MB2351.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(396003)(366004)(136003)(376002)(346002)(66476007)(64756008)(66946007)(66556008)(53546011)(6506007)(2906002)(83380400001)(7696005)(76116006)(5660300002)(86362001)(478600001)(110136005)(33656002)(26005)(186003)(52536014)(9686003)(316002)(71200400001)(8936002)(8676002)(66446008)(4326008)(54906003)(55016002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 0xCqljVEnVAXyP9wCxnZgRYiV3Ny772vB1F4/9fwl+1A2Do+zuRqxf+hBFSRmZpiG+mQ6djtcvwJg4sM1zgGwleDGk9NmzoMbav+l2NnqeyyaouBdobtx1YYRZVxKBCo880Xvan9Vf2v1b3xgg8Y87bAPd26yaphdUMODXoqpxs5my3kKq+nf8dPIeKQwjNKvKftKgqRNtyaGdoW2Vyp9UD+dukCqyxI8WK6WqgGo1p73ASzpaDeLuTMpQUFRPMMPbdPC0IyzTj/FY0MNvIkKZ/8fMmzeEFwCz+Jqu1DUpytXx3U0VgBngEsaOTwLrfHvryEEmwBOhGx6r/IfcjYbTqmwTFgQ4EvTj4ahBoZZAcbKiALtzfJ7CXGQRwGzRQxKidSjuDRU9WaKZ6MW84B2Gt+Q8DgriDgPT/UtvfGAOZ/H8rVd/kMAldKCWX0jcviD+93TtH4nmLtwwxP4XVAnfNsp2Bsmpv3yI3X9u7UTa02jw/u41CS1fBK0QbIf399Bgu0AUxmaDsX+4wntXQ/Ndk2NWdaMqfGFrpouqlO7fC3C8IsIogFAWXw5lsLpbitf7RysQ/GNIF0i1XbrhuERR5v4o5IgrJu41eEaU+yQ0n+/Co694smvEBuLnbl4KZuo1HjODUKJDOCQ4dehgb7EA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5328 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: aae5fb2a-6e3b-4372-fe8c-08d870123653 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tbVCti6YOKoU+97GAkESOZzjAwv6o9qTcgcWQHIxNrEwqXq0w7trKGrODoNZd+oWpLvAGx6lulkH0CuK3rYYdIFVC3glyqioEvfUy5Qo/XaG4HGt7g/Gfm93IFPKgdi/aQvFEtQ+pGezLErkWPWWirnG00brOyCEeaxvcAXeNaR5s4uL7m+jExHre9YvGo+pRPJBllftqy4ocGKCA8Y0kmeWwFzqciC3hfVy5lM4n2fL6TnwsqzPeuN7rDqnFoG0mSLiVYABAy8ZJtvm+ENQh3q6++WewXU3baNrMzCkuLnNZnzL4vdwagh+sMRsh8YiE3zZ1knFuKpDheI+B0JExljZbVkhI2KjFnM8+s0AGvIP9ourvvnAQfy3awx4noRdeEXmRHRl6cWXrii/Z85q/Q== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(39860400002)(376002)(396003)(136003)(346002)(46966005)(5660300002)(55016002)(8676002)(316002)(36906005)(47076004)(478600001)(186003)(70206006)(82740400003)(336012)(26005)(86362001)(70586007)(53546011)(6506007)(82310400003)(9686003)(33656002)(52536014)(2906002)(356005)(4326008)(7696005)(110136005)(54906003)(83380400001)(81166007)(8936002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Oct 2020 07:24:47.6290 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7eeb8c57-c239-43ac-14d9-08d870123b27 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT016.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3052 Subject: Re: [dpdk-dev] [PATCH v7] eal: add cache-line demote support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Omkar Maslekar > Sent: Tuesday, October 13, 2020 5:43 PM > To: dev@dpdk.org > Cc: bruce.richardson@intel.com; ciara.loftus@intel.com; > omkar.maslekar@intel.com; drc@linux.vnet.ibm.com; jerinj@marvell.com; > Ruifeng Wang ; Honnappa Nagarahalli > > Subject: [PATCH v7] eal: add cache-line demote support >=20 > rte_cldemote is similar to a prefetch hint - in reverse. cldemote(addr) > enables software to hint to hardware that line is likely to be shared. > Useful in core-to-core communications where cache-line is likely to be > shared. ARM and PPC implementation is provided with NOP and can be > added if any equivalent instructions could be used for implementation on > those architectures. >=20 > Signed-off-by: Omkar Maslekar > Acked-by: Bruce Richardson > Acked-by: David Christensen > Acked-by: Jerin Jacob >=20 > --- > v7: fixed experimental tag >=20 > v6: marked rte_cldemote as experimental > added rte_cldemote call in existing app/test_prefetch.c >=20 > v5: documentation updated > fixed formatting issue in release notes > added Acked-by: Bruce Richardson > * > v4: updated bold text for title and fixed margin in release notes > * > v3: fixed warning regarding whitespace > * > v2: documentation updated > --- > --- > app/test/test_prefetch.c | 4 ++++ > doc/guides/rel_notes/release_20_11.rst | 7 +++++++ > lib/librte_eal/arm/include/rte_prefetch_32.h | 7 +++++++ > lib/librte_eal/arm/include/rte_prefetch_64.h | 7 +++++++ > lib/librte_eal/include/generic/rte_prefetch.h | 15 +++++++++++++++ > lib/librte_eal/ppc/include/rte_prefetch.h | 7 +++++++ > lib/librte_eal/x86/include/rte_prefetch.h | 11 +++++++++++ > 7 files changed, 58 insertions(+) >=20 > diff --git a/app/test/test_prefetch.c b/app/test/test_prefetch.c index > 41f219a..5c58d0c 100644 > --- a/app/test/test_prefetch.c > +++ b/app/test/test_prefetch.c > @@ -26,7 +26,11 @@ > rte_prefetch1(&a); > rte_prefetch2(&a); >=20 > +/* test for marking a line as shared to test cldemote functionality */ > + rte_cldemote(&a); > + > return 0; > } >=20 > + > REGISTER_TEST_COMMAND(prefetch_autotest, test_prefetch); diff --git > a/doc/guides/rel_notes/release_20_11.rst > b/doc/guides/rel_notes/release_20_11.rst > index b7881f2..8a1ed01 100644 > --- a/doc/guides/rel_notes/release_20_11.rst > +++ b/doc/guides/rel_notes/release_20_11.rst > @@ -171,6 +171,13 @@ New Features > * Extern objects and functions can be plugged into the pipeline. > * Transaction-oriented table updates. >=20 > +* **Added new function rte_cldemote in rte_prefetch.h.** > + > + Added a hardware hint CLDEMOTE, which is similar to prefetch in revers= e. > + CLDEMOTE moves the cache line to the more remote cache, where it > + expects sharing to be efficient. Moving the cache line to a level > + more distant from the processor helps to accelerate core-to-core > communication. > + >=20 > Removed Items > ------------- > diff --git a/lib/librte_eal/arm/include/rte_prefetch_32.h > b/lib/librte_eal/arm/include/rte_prefetch_32.h > index e53420a..28b3d48 100644 > --- a/lib/librte_eal/arm/include/rte_prefetch_32.h > +++ b/lib/librte_eal/arm/include/rte_prefetch_32.h > @@ -10,6 +10,7 @@ > #endif >=20 > #include > +#include > #include "generic/rte_prefetch.h" >=20 > static inline void rte_prefetch0(const volatile void *p) @@ -33,6 +34,12= @@ > static inline void rte_prefetch_non_temporal(const volatile void *p) > rte_prefetch0(p); > } >=20 > +__rte_experimental > +static inline void rte_cldemote(const volatile void *p) { > + RTE_SET_USED(p); > +} > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_eal/arm/include/rte_prefetch_64.h > b/lib/librte_eal/arm/include/rte_prefetch_64.h > index fc2b391..1c722eb 100644 > --- a/lib/librte_eal/arm/include/rte_prefetch_64.h > +++ b/lib/librte_eal/arm/include/rte_prefetch_64.h > @@ -10,6 +10,7 @@ > #endif >=20 > #include > +#include > #include "generic/rte_prefetch.h" >=20 > static inline void rte_prefetch0(const volatile void *p) @@ -32,6 +33,12= @@ > static inline void rte_prefetch_non_temporal(const volatile void *p) > asm volatile ("PRFM PLDL1STRM, [%0]" : : "r" (p)); } >=20 > +__rte_experimental > +static inline void rte_cldemote(const volatile void *p) { > + RTE_SET_USED(p); > +} > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_eal/include/generic/rte_prefetch.h > b/lib/librte_eal/include/generic/rte_prefetch.h > index 6e47bdf..ad9844c 100644 > --- a/lib/librte_eal/include/generic/rte_prefetch.h > +++ b/lib/librte_eal/include/generic/rte_prefetch.h > @@ -51,4 +51,19 @@ > */ > static inline void rte_prefetch_non_temporal(const volatile void *p); >=20 > +/** > + * Demote a cache line to a more distant level of cache from the process= or. > + * > + * CLDEMOTE hints to hardware to move (demote) a cache line from the > +closest to > + * the processor to a level more distant from the processor. It is a > +hint and > + * not guarantee. rte_cldemote is intended to move the cache line to > +the more > + * remote cache, where it expects sharing to be efficient and to > +indicate that a > + * line may be accessed by a different core in the future. > + * > + * @param p > + * Address to demote > + */ > +__rte_experimental > +static inline void rte_cldemote(const volatile void *p); > + > #endif /* _RTE_PREFETCH_H_ */ > diff --git a/lib/librte_eal/ppc/include/rte_prefetch.h > b/lib/librte_eal/ppc/include/rte_prefetch.h > index 9ba07c8..b55cac4 100644 > --- a/lib/librte_eal/ppc/include/rte_prefetch.h > +++ b/lib/librte_eal/ppc/include/rte_prefetch.h > @@ -11,6 +11,7 @@ > #endif >=20 > #include > +#include > #include "generic/rte_prefetch.h" >=20 > static inline void rte_prefetch0(const volatile void *p) @@ -34,6 +35,12= @@ > static inline void rte_prefetch_non_temporal(const volatile void *p) > rte_prefetch0(p); > } >=20 > +__rte_experimental > +static inline void rte_cldemote(const volatile void *p) { > + RTE_SET_USED(p); > +} > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_eal/x86/include/rte_prefetch.h > b/lib/librte_eal/x86/include/rte_prefetch.h > index 384c6b3..92ba05a 100644 > --- a/lib/librte_eal/x86/include/rte_prefetch.h > +++ b/lib/librte_eal/x86/include/rte_prefetch.h > @@ -10,6 +10,7 @@ > #endif >=20 > #include > +#include > #include "generic/rte_prefetch.h" >=20 > static inline void rte_prefetch0(const volatile void *p) @@ -32,6 +33,16= @@ > static inline void rte_prefetch_non_temporal(const volatile void *p) > asm volatile ("prefetchnta %[p]" : : [p] "m" (*(const volatile char > *)p)); } >=20 > +/* > + * we're using raw byte codes for now as only the newest compiler > + * versions support this instruction natively. > + */ > +__rte_experimental > +static inline void rte_cldemote(const volatile void *p) { > + asm volatile(".byte 0x0f, 0x1c, 0x06" :: "S" (p)); } > + > #ifdef __cplusplus > } > #endif > -- > 1.8.3.1 Reviewed-by: Ruifeng Wang