From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10075.outbound.protection.outlook.com [40.107.1.75]) by dpdk.org (Postfix) with ESMTP id C795E1B2BE for ; Wed, 17 Jan 2018 19:39:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=qlsRJY0Av91HP/dt9tgMVCvSC/uxXVd/hmO2DqA+wgQ=; b=JqnLx1yiRqor5KhzFLFNuUpRJzSV7PYWKyLcOu4y9gtcYJp6it+rNXQWNKQ7l4ZLstUSifJkfLRmbrUAymZFOXZJT05p3DdYAy5CLt8MIi/k2bCFktv8vP9vdJ1BhdB9u+wHjK66/INLrbTb6bDY3f2fPrhYoZXh9aPN2N5SyMQ= Received: from VI1PR0501MB2045.eurprd05.prod.outlook.com (10.167.195.147) by VI1PR0501MB2688.eurprd05.prod.outlook.com (10.172.15.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.407.7; Wed, 17 Jan 2018 18:39:25 +0000 Received: from VI1PR0501MB2045.eurprd05.prod.outlook.com ([fe80::2d4b:9768:b21a:e7e9]) by VI1PR0501MB2045.eurprd05.prod.outlook.com ([fe80::2d4b:9768:b21a:e7e9%14]) with mapi id 15.20.0407.012; Wed, 17 Jan 2018 18:39:25 +0000 From: Yongseok Koh To: Thomas Monjalon CC: Jianbo Liu , Andrew Rybchenko , "dev@dpdk.org" , Adrien Mazarguil , =?iso-8859-1?Q?N=E9lio_Laranjeiro?= , "jerin.jacob@caviumnetworks.com" , "konstantin.ananyev@intel.com" , "bruce.richardson@intel.com" , Chao Zhu Thread-Topic: [dpdk-dev] [PATCH v2 1/8] eal: introduce DMA memory barriers Thread-Index: AQHTjmcQ3r4dfdovVEiBLSv+cotiIqN2ICkAgAAWzICAAd9agIAAUeAA Date: Wed, 17 Jan 2018 18:39:25 +0000 Message-ID: References: <20171227042824.33373-1-yskoh@mellanox.com> <20180116091040.GA15629@arm.com> <3720864.2redlIt54T@xps> In-Reply-To: <3720864.2redlIt54T@xps> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [209.116.155.178] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR0501MB2688; 6:+51G0ACgUY5pFgN5XPCxd1L8gAsSh7VszcGDBc3M2ZNUtxBINGvnviKJg7hYL1nVVbLxMBk5M6JBfs2hnWPPMCig2a6DvOKtGpm/DHGKl8jg25A6UpeS2/2fjNMqeP+XL97K6lxYK5HDjZ1BmtrxsIXH/yEkbhdkZmCmSiRLbIqWStowORRgpUOHxFIvV2QEdR07KZRGQH5Su3DpCi91e0oQ1lH9NEqC7sRHu4HDxuJ8gLwaonhfoaWz6HeAYsoTmUEHPqJaxFvkF8lTcciw1Mlv4Ootith7fNAxWIodEPfsxvgtmIZ1wYJjHUsvfqY9gSfCukzQsVdIudMb82jGdZ7uIQYFQNCxO2ogoz0y76dbStJK93X74Id9PA1XjHEB; 5:0edwNrqfHJEFfpcnG5SpoKOPrnLZ7Cr4Sf3NvHvMjNxf9flPzur7GuPThysB7HOPv8t6OQlZ5qTwuM1MWKTb3Xnl7UiHTm4EJxFrI1uM54qirKWp5w9iU1TdHJSq+GsJ7VNBhDUC2mAaucOC3Ovf/hIagWCRhcF6zMTDsSPXyVI=; 24:TCYDL+F/AyrPlWuBIy5yHjO0ewuOF82nS8NPB3L604nCaLFdiSeMm7wlXA+ouXqbS5bAK/mixa7hhpMJOHYsUffxyXIUqefdVExGI7lSLLg=; 7:YisJFBWPtLG8UUSmZoDngjzE1BQDzugVeWHXVmow+yQ1/l0fCoEOXO82JYe44wHODpLkasGCn76YreHZZFL7T7Tp4165HCk6PKjqFQLRBY/YzAo+5DZAMkCq5mX4XOgeYmL/PWsyLRHnZPeS5WouWX+NQUEfFiGDzce06qvD1Uo9pM8HNBvyhqLHDFfYcrryBeLVdVIzX51kh+66YMg2OOGfX0bSqOIcBGBlXq1hkY/sUhl4W2zPLe3X4dPmnbek x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 60a2ae10-d4dd-4460-fc9c-08d55dd9a27b x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(2017052603307)(7153060)(7193020); SRVR:VI1PR0501MB2688; x-ms-traffictypediagnostic: VI1PR0501MB2688: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(189930954265078)(45079756050767); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(5005006)(8121501046)(3231023)(2400048)(944501161)(93006095)(93001095)(10201501046)(3002001)(6055026)(6041268)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123560045)(20161123564045)(20161123562045)(6072148)(201708071742011); SRVR:VI1PR0501MB2688; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:VI1PR0501MB2688; x-forefront-prvs: 0555EC8317 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(376002)(346002)(39380400002)(39860400002)(366004)(24454002)(199004)(189003)(99286004)(966005)(3280700002)(229853002)(2950100002)(54906003)(53936002)(575784001)(86362001)(6306002)(6436002)(6486002)(6512007)(36756003)(5250100002)(3660700001)(5660300001)(105586002)(82746002)(66066001)(68736007)(106356001)(7736002)(305945005)(8936002)(81166006)(7416002)(8676002)(33656002)(81156014)(316002)(6116002)(83716003)(2900100001)(3846002)(93886005)(25786009)(45080400002)(76176011)(59450400001)(6506007)(4326008)(102836004)(478600001)(6916009)(53546011)(97736004)(14454004)(2906002)(6246003)(26005); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2688; H:VI1PR0501MB2045.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: UNp2PBgCTQNjIapmc8HvAh+rS6P9gHKgEIBNGD9ozHKL1KHhpRajwJSP/skFfRbzQ/l+EyJTZXP9CBHYoeaazA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-ID: <81AB3C7500F71040AA44076BC9754397@eurprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 60a2ae10-d4dd-4460-fc9c-08d55dd9a27b X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jan 2018 18:39:25.7020 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2688 Subject: Re: [dpdk-dev] [PATCH v2 1/8] eal: introduce DMA memory barriers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 18:39:28 -0000 > On Jan 17, 2018, at 5:46 AM, Thomas Monjalon wrote: >=20 > 16/01/2018 10:10, Jianbo Liu: >> The 01/16/2018 10:49, Andrew Rybchenko wrote: >>> On 01/16/2018 04:10 AM, Yongseok Koh wrote: >>>> This commit introduces rte_dma_wmb() and rte_dma_rmb(), in order to >>>> guarantee the ordering of coherent shared memory between the CPU and a= DMA >>>> capable device. >>>>=20 >>>> Signed-off-by: Yongseok Koh >>>> --- >>>> lib/librte_eal/common/include/generic/rte_atomic.h | 18 ++++++++++++++= ++++ >>>> 1 file changed, 18 insertions(+) >>>>=20 >>>> diff --git a/lib/librte_eal/common/include/generic/rte_atomic.h b/lib/= librte_eal/common/include/generic/rte_atomic.h >>>> index 16af5ca57..2e0503ce6 100644 >>>> --- a/lib/librte_eal/common/include/generic/rte_atomic.h >>>> +++ b/lib/librte_eal/common/include/generic/rte_atomic.h >>>> @@ -98,6 +98,24 @@ static inline void rte_io_wmb(void); >>>> */ >>>> static inline void rte_io_rmb(void); >>>> +/** >>>> + * Write memory barrier for coherent memory between lcore and IO devi= ce >>>> + * >>>> + * Guarantees that the STORE operations on coherent memory that >>>> + * precede the rte_dma_wmb() call are visible to I/O device before th= e >>>> + * STORE operations that follow it. >>>> + */ >>>> +static inline void rte_dma_wmb(void); >>>> + >>>> +/** >>>> + * Read memory barrier for coherent memory between lcore and IO devic= e >>>> + * >>>> + * Guarantees that the LOAD operations on coherent memory updated by >>>> + * IO device that precede the rte_dma_rmb() call are visible to CPU >>>> + * before the LOAD operations that follow it. >>>> + */ >>>> +static inline void rte_dma_rmb(void); >>>> + >>>> #endif /* __DOXYGEN__ */ >>>> /** >>>=20 >>> I'm not an ARMv8 expert so, my comments could be a bit ignorant. >>> I'd like to understand the difference between io and added here dma >>> barriers. >>> The difference should be clearly explained. Otherwise we'll constantly = hit >>> on incorrect choice of barrier type. >>> Also I don't understand why "dma" name is chosen taking into account >>> that definition is bound to coherent memory between lcore and IO device= . >>=20 >> A good explanation can be found here. >>=20 >> https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fgit= .kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Fcom= mit%2F%3Fid%3D1077fa36f23e259858caf6f269a47393a5aff523&data=3D02%7C01%7Cysk= oh%40mellanox.com%7C7b526265cbf1449f3db208d55db0c55d%7Ca652971c7d2e4d9ba6a4= d149256f461b%7C0%7C0%7C636517936183877836&sdata=3D2%2Fi8Gs2n%2Fnbe9%2FJ3GWr= 22ndPmQVmvM2Xh12r3j1ZWlg%3D&reserved=3D0 >=20 > I agree that something more is needed to explain when to use rte_io_*. > The only difference between rte_io_* and rte_dma_* is "on coherent memory= ". Okay will add more explanation and send out v3 soon. But, please note that there's no concrete theory when to use which barrier. Actually, it is mostl= y for ARMv8 because it provides more options for barriers. For other archs, a= s you can see in the patches, there's no difference from IO barriers. Thanks, Yongseok