From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2C5F0A0561; Tue, 21 Apr 2020 09:45:56 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 182F61D8F0; Tue, 21 Apr 2020 09:45:56 +0200 (CEST) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10063.outbound.protection.outlook.com [40.107.1.63]) by dpdk.org (Postfix) with ESMTP id 29D881D8EE for ; Tue, 21 Apr 2020 09:45:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mZbRV/aG1/1Q+7s7EtbW8J0FwXJ0XjZV5ucdPjyimcI=; b=4+ekyysyjvNFc5bEutd9OW4QqbpBHXGXYPxU0ay1UOZuN8SL3CY+RFg0CSe5kOs3QC+E1YnFlVTfRoMd4g+ALeZOw2GlPvfM50EThUZOMdqapemgeboe0I2RWkc+93v84DNPoWOezBEfFsRS16wG6mEMO2s68l+q5CC3Rg03sKc= Received: from DB6PR07CA0162.eurprd07.prod.outlook.com (2603:10a6:6:43::16) by DB6PR0801MB1831.eurprd08.prod.outlook.com (2603:10a6:4:38::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.28; Tue, 21 Apr 2020 07:45:54 +0000 Received: from DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com (2603:10a6:6:43:cafe::85) by DB6PR07CA0162.outlook.office365.com (2603:10a6:6:43::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.6 via Frontend Transport; Tue, 21 Apr 2020 07:45:54 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT044.mail.protection.outlook.com (10.152.21.167) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.15 via Frontend Transport; Tue, 21 Apr 2020 07:45:54 +0000 Received: ("Tessian outbound 76e9a4cf1540:v53"); Tue, 21 Apr 2020 07:45:54 +0000 X-CR-MTA-TID: 64aa7808 Received: from 4be1566c9403.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 122FF6BC-189D-4784-B8F8-8374B8A14AE3.1; Tue, 21 Apr 2020 07:45:48 +0000 Received: from EUR02-AM5-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 4be1566c9403.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 21 Apr 2020 07:45:48 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cHA5vur84zphzGtbaJDUw5nB/yNneF6h5//qQQX/5Cp9Nrf40mZ5Y0wZxmnSDIp+yT7G6B0MmKqaSiF61zGtDbvyps2x+D+0zrAhr2/z1jeMkPm86QT5xMoDR+qjKpB+RhZjxU5uBzMnp0bGqq7fx0oJg6pwytzjdYJOxA33BzSOTR++jV0j80+7rqpulM697hMMBmO4xT+/+akNEX5dJQxRy7BvbGGvOiAbx5nCqapN2holm03H8RSYYOVXkMpmifeYeltbOak4eYjGIcfTfdD8mFj4vZTibY7jbzl2+a6ETJggdXcifD3XF0K68LPI55Ce++zDIQqrewMcsuVfig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mZbRV/aG1/1Q+7s7EtbW8J0FwXJ0XjZV5ucdPjyimcI=; b=a2DonFppob1Ml7PDbZzQM0oMgQ0S/fu9S3MbFyvBjvItFKElmsCZ6vwFBzGfjfeWsbGdtZSc7z0kTUGLCZ+CwfcqsrbcasCJz64JBP1VI3phxK2fhrOrRTJjVNuHFNNzKzRxtYlt2JjNrWweQxWGziw5/h1urSE3DEuZBMtAObzO4wO8KzL6GxHIQbK4HGTmqY9jRQCBR9Z82kfuDHgCsvwlp/THp0t2PY/2Ojt02rI4PXk46wrOERqoXmoVYKDWeXEwJ6A4VlMIkc7jg4Szxvm39Hkv3tTIwVSE44JHm0eykbdgeh6jW7ee6nzRQij49YnJsJmolKwvHwwGgmSh3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mZbRV/aG1/1Q+7s7EtbW8J0FwXJ0XjZV5ucdPjyimcI=; b=4+ekyysyjvNFc5bEutd9OW4QqbpBHXGXYPxU0ay1UOZuN8SL3CY+RFg0CSe5kOs3QC+E1YnFlVTfRoMd4g+ALeZOw2GlPvfM50EThUZOMdqapemgeboe0I2RWkc+93v84DNPoWOezBEfFsRS16wG6mEMO2s68l+q5CC3Rg03sKc= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (2603:10a6:803:13e::15) by VI1PR08MB3485.eurprd08.prod.outlook.com (2603:10a6:803:7c::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.29; Tue, 21 Apr 2020 07:45:45 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::a0e2:2a9f:be7b:4b15]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::a0e2:2a9f:be7b:4b15%3]) with mapi id 15.20.2921.030; Tue, 21 Apr 2020 07:45:45 +0000 From: Gavin Hu To: "Chauskin, Igor" , "dev@dpdk.org" CC: nd , "david.marchand@redhat.com" , "thomas@monjalon.net" , "mk@semihalf.com" , "Tzalik, Guy" , "Schmeilin, Evgeny" , "mw@semihalf.com" , Honnappa Nagarahalli , Ruifeng Wang , Phil Yang , Joyce Kong , "Bshara, Saeed" , "Matushevsky, Alexander" , Bruce Richardson , nd , nd Thread-Topic: [PATCH RFC v1 0/7] relax barriers for ENA PMD and small fixes Thread-Index: AQHV+3YoBygG9l9U6EWVq6l5PsWv0ahMa7gwgC+FxgCAB3arAA== Date: Tue, 21 Apr 2020 07:45:45 +0000 Message-ID: References: <20200313091835.58039-1-gavin.hu@arm.com> <10e90cbc7bbe4d84979dabcb0fbe0c41@EX13D12EUA003.ant.amazon.com> <73990a21da324ef5ac759227017682fd@EX13D12EUA003.ant.amazon.com> In-Reply-To: <73990a21da324ef5ac759227017682fd@EX13D12EUA003.ant.amazon.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 571264dc-7080-4a18-94f0-665af20d0548.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 8b478785-c1fd-481e-7477-08d7e5c8054c x-ms-traffictypediagnostic: VI1PR08MB3485:|VI1PR08MB3485:|DB6PR0801MB1831: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; x-forefront-prvs: 038002787A X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR08MB5376.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(39860400002)(346002)(366004)(376002)(396003)(136003)(2906002)(55016002)(86362001)(64756008)(4326008)(52536014)(33656002)(66476007)(478600001)(186003)(66946007)(9686003)(66556008)(76116006)(8936002)(66446008)(81156014)(8676002)(54906003)(110136005)(316002)(26005)(7416002)(55236004)(53546011)(6506007)(5660300002)(71200400001)(7696005); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: XPY6RUjBa7R1Wdd2PrAjISb6hCAl+nuxVwsYeoEZpN8YMCLOBXuREzG/5fzCJUHfYceHe0FBFNIe4tiXzHqDyfTI25IqI9YUgNFTUYUlSrnoqaUYe+pWmVSBCaFj/klXCj9k699+RlArmcboVQOIvfsL3+COJibEDQYIKHnM0HsTE5jf0Za6Vl7DXFLwtE2D2wQ9Vwite59lLfBdjG2D8VkYzivuRT/hC9gumeB3qZYPMJJqywPyMzLixr0cAFYBMFYdY5TemkbL/7cuaESa+WSoe9kUmENFBIlVZipyGABjcACH8YEgKep8KjWnjoO0N6gl694alwqdDIfvoUenLEHtb+FWJok3Gw29120x5ZYky1gJ56LSSZCkLZRGgsa+INN8QLeGBgaLj4Kg3V+ALv5igC+UqwVafr21CGpHyMS4KSRCbUaHS7+rATtt7GlT x-ms-exchange-antispam-messagedata: ilhFddt3viJnxrExZINpGE5HF059rstuB7xQjoHrmiN0WqMmSFrogfc0ui79jx+zVzkDfGSnrZyYJdVOPFQwXsPJyp8gzzVdJpgOeXhqZDoLCaTI997VlaX9FWVCXxPvUK1YXmzuOFnz6SHembnfRQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3485 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT044.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(39860400002)(346002)(376002)(396003)(136003)(46966005)(86362001)(110136005)(5660300002)(33656002)(4326008)(478600001)(2906002)(186003)(81156014)(54906003)(70586007)(8676002)(7696005)(70206006)(316002)(26005)(8936002)(53546011)(6506007)(82740400003)(55016002)(9686003)(81166007)(47076004)(356005)(336012)(52536014); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: b814b0ee-1609-4c49-4856-08d7e5c8005c X-Forefront-PRVS: 038002787A X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ktYieVqymuA9TfsKbNalbc9kZWWLV3u73jZSbzIEsoApSyUMhoreuhj+bqC/hPF9k3bgrx8HecY23GUKU75HnB0DFAfhtc4ttE3a8p6itFgMsvLfqm53pxh3xxPSnzY3CUu7QeerV6iY0zVbuU3Emz6WKmwpSqLcmKI47ZRHsmWyET0SS3WQ3iJrkwDhELF3lbLKM1GOMLLXOFSVh/sf4Vhr9BZelO3FldARvhqbIJ8Buyt86QLFE0JaV5XvYM4z7PoQMwR8j56tCvWF+DHNzyjP/xpXaT6QfNfq/jai3nvbPn7Xy9zGPUh35HOjtEv0vIUVGTrAhJqfhlvGDg2L5BJz1Jo6r7jH5EAr8OzTpJ7dGpNO3TGErYEnb16ylwDA/L3oOOQvT+voS6m9OK1cIM4fs2Wi/8qrTdeUm38fh4pxTi3BTOVpijlvkFk4t/V6Z8XYg+C2IuyiKWZ9JfiZSBdClK1WjBTv5tV9Arp3ys8PcQ1kb1TMcjlbyvXEx6X5tUFOKxbdPHCkD9wjVcr9mQ== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Apr 2020 07:45:54.1250 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8b478785-c1fd-481e-7477-08d7e5c8054c X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0801MB1831 Subject: Re: [dpdk-dev] [PATCH RFC v1 0/7] relax barriers for ENA PMD and small fixes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Chauskin, Igor > Sent: Thursday, April 16, 2020 9:38 PM > To: Gavin Hu ; dev@dpdk.org > Cc: nd ; david.marchand@redhat.com; > thomas@monjalon.net; mk@semihalf.com; Tzalik, Guy > ; Schmeilin, Evgeny ; > mw@semihalf.com; Honnappa Nagarahalli > ; Ruifeng Wang > ; Phil Yang ; Joyce Kong > ; Bshara, Saeed ; > Matushevsky, Alexander ; Bruce Richardson > ; nd > Subject: RE: [PATCH RFC v1 0/7] relax barriers for ENA PMD and small fixe= s >=20 > Hi all, >=20 > Please see the first batch of comments related to these patches: >=20 > 1. Relaxing the register read/write isn't always good enough. Specificall= y, > when barriers are required between different memory types, the reordering > can occur even on x86. Yet in DPDK the io/cio/smp barrier flavors for x86 > are defined as compiler-only barriers, which is not enough in cases invol= ving > different memory types. In ENA driver, when LLQ is on, there is a regular > register memory access before the barrier and write-combined memory > access after the barrier. That's makes sense, we realized that also, we don't mean to change x86 beha= viors. >=20 > We're working on a more extensive change that will include the > optimizations proposed for barriers relaxing while making them applicable > to all platforms. We are working also on an extensive change, helpful to arm64 while not impa= cting x86. More testing is ongoing.=20 >=20 > 2. Regarding the changes for statistics logging - the patch relies on c1= 1 > features. I'm not sure it's acceptable for all situations since we've alr= eady > encountered a reports when even c99-compliant changes caused > compilation issues. C11 is already widely used in other components, even in other projects like= vpp and ovs.=20 Maybe it comes to time to drop C99 as a stringent requirement.=20 >=20 > 3. Removing redundant zeroing of sub-struct - we're currently working on > some extensive changes to the Tx flow, which will include this change > among other. Ok, thanks.=20 >=20 > Thanks, > Igor >=20 > -----Original Message----- > From: Gavin Hu > Sent: Tuesday, March 17, 2020 9:59 AM > To: Chauskin, Igor ; dev@dpdk.org > Cc: nd ; david.marchand@redhat.com; > thomas@monjalon.net; mk@semihalf.com; Tzalik, Guy > ; Schmeilin, Evgeny ; > mw@semihalf.com; Honnappa Nagarahalli > ; Ruifeng Wang > ; Phil Yang ; Joyce Kong > ; Bshara, Saeed ; > Matushevsky, Alexander ; Bruce Richardson > ; nd > Subject: RE: [EXTERNAL] [PATCH RFC v1 0/7] relax barriers for ENA PMD and > small fixes >=20 > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and kno= w > the content is safe. >=20 >=20 >=20 > Hi Igor, >=20 > > -----Original Message----- > > From: Chauskin, Igor > > Sent: Monday, March 16, 2020 5:35 PM > > To: Gavin Hu ; dev@dpdk.org > > Cc: nd ; david.marchand@redhat.com; > thomas@monjalon.net; > > mk@semihalf.com; Tzalik, Guy ; Schmeilin, Evgeny > > ; mw@semihalf.com; Honnappa Nagarahalli > > ; Ruifeng Wang > ; > > Phil Yang ; Joyce Kong ; > > Bshara, Saeed ; Matushevsky, Alexander > > > > Subject: RE: [PATCH RFC v1 0/7] relax barriers for ENA PMD and small > > fixes > > > > Hi Gavin, > > > > Thank you for the contribution. > > Please do not merge these changes (patches 0..7) till we (the ENA > > team) properly review and ack/nack. > > These changes can potentially provide performance improvement, yet we > > need to ensure they are applicable for all possible scenarios. > > Specifically, the behavior on x86 platforms is likely to be different. > > What testing have you done for these patches? Was x86 tested? > As noted in the cover letter, these patches were not tested as we don't h= ave > ENA NICs. > We rely on you to do that, any concerns and comments welcome. > Yes, the behavior on x86 platforms is also different, Intel people are > welcome to comment. > /Gavin > > > > Thanks, > > Igor > > > > -----Original Message----- > > From: Gavin Hu > > Sent: Friday, March 13, 2020 11:18 AM > > To: dev@dpdk.org > > Cc: nd@arm.com; david.marchand@redhat.com; thomas@monjalon.net; > > mk@semihalf.com; Tzalik, Guy ; Schmeilin, Evgeny > > ; Chauskin, Igor ; > > mw@semihalf.com; Honnappa.Nagarahalli@arm.com; > ruifeng.wang@arm.com; > > phil.yang@arm.com; joyce.kong@arm.com > > Subject: [EXTERNAL][PATCH RFC v1 0/7] relax barriers for ENA PMD and > > small fixes > > > > CAUTION: This email originated from outside of the organization. Do > > not click links or open attachments unless you can confirm the sender > > and know the content is safe. > > > > > > > > To ensure the stores to the host memory are observed by NIC HW before > > a door bell ring to the NIC HW and the HW starts actions, for example, > > doing DMA, a barrier is required on weak memory ordering platforms, lik= e > aarch64. > > > > However, unnecessarily too strong barriers like 'dsb' on aarch64 will > > dampen performance. > > > > In a typical doorbell use case, as NIC and CPU are in the outer > > sharable domain, a lighter weight 'dmb osh' barrier is sufficient. > > > > The patch set relaxes the barriers in similar places and include one > > more patch for statistics logging with relaxed ordering and the other > > patch removing duplicate memset. > > > > Note this set is submitted for RFC as we don't have physical ENA NICs > > in the lab and the patch set was not verified nor benchmarked. > > > > Gavin Hu (7): > > net/ena: remove duplicate barrier > > net/ena: relax the barrier for doorbell ring > > net/ena: relax the rmb for DMA > > net/ena: relax barrier for completion queue update > > net/ena: relax the barrier for bounce buffer > > net/ena: use c11 atomic for statistics > > net/ena: remove duplicate memset > > > > drivers/net/ena/base/ena_eth_com.c | 2 +- > > drivers/net/ena/base/ena_eth_com.h | 6 ++-- > > drivers/net/ena/base/ena_plat_dpdk.h | 2 +- > > drivers/net/ena/ena_ethdev.c | 46 +++++++++++++++++----------- > > drivers/net/ena/ena_ethdev.h | 8 ++--- > > 5 files changed, 38 insertions(+), 26 deletions(-) > > > > -- > > 2.17.1