From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B763A0588; Thu, 16 Apr 2020 22:03:38 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4CF811DA87; Thu, 16 Apr 2020 22:03:31 +0200 (CEST) Received: from smtp-fw-33001.amazon.com (smtp-fw-33001.amazon.com [207.171.190.10]) by dpdk.org (Postfix) with ESMTP id 9BE271DCC4 for ; Thu, 16 Apr 2020 15:37:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1587044270; x=1618580270; h=from:to:cc:date:message-id:references:in-reply-to: content-transfer-encoding:mime-version:subject; bh=3DZptiAexJHhPGhZzK+jlAPwk0v+yzvX3yabRpZSyw8=; b=pEea168rXBKI+/U4maiUDO6QXDA3uKd88bRUPEcckt8h2SmyjdQmhUL9 GE7/636bztj9zE9WYbbL34XcLLmDuMN8e8KHSWpwLHgx39TvtVIv//TPn FQEk6vB1O0ILwQ29oELbzwmcLYi5/Vv1ZlLm3wwYQTL3YOr9oHMA+eoVJ o=; IronPort-SDR: qejh+mSUdzalU+kRhNFquwdLaGYLWu+E9hska7OBMhhkmk1ou75ugGd1cjE9OsYWrQ9e/xYTIW 56nsSEJi946A== X-IronPort-AV: E=Sophos;i="5.72,391,1580774400"; d="scan'208";a="38838935" Thread-Topic: [PATCH RFC v1 0/7] relax barriers for ENA PMD and small fixes Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2c-168cbb73.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 16 Apr 2020 13:37:47 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2c-168cbb73.us-west-2.amazon.com (Postfix) with ESMTPS id 424ABA2412; Thu, 16 Apr 2020 13:37:46 +0000 (UTC) Received: from EX13D06EUA001.ant.amazon.com (10.43.165.229) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 16 Apr 2020 13:37:45 +0000 Received: from EX13D12EUA003.ant.amazon.com (10.43.165.147) by EX13D06EUA001.ant.amazon.com (10.43.165.229) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 16 Apr 2020 13:37:44 +0000 Received: from EX13D12EUA003.ant.amazon.com ([10.43.165.147]) by EX13D12EUA003.ant.amazon.com ([10.43.165.147]) with mapi id 15.00.1497.006; Thu, 16 Apr 2020 13:37:44 +0000 From: "Chauskin, Igor" To: Gavin Hu , "dev@dpdk.org" CC: nd , "david.marchand@redhat.com" , "thomas@monjalon.net" , "mk@semihalf.com" , "Tzalik, Guy" , "Schmeilin, Evgeny" , "mw@semihalf.com" , "Honnappa Nagarahalli" , Ruifeng Wang , Phil Yang , Joyce Kong , "Bshara, Saeed" , "Matushevsky, Alexander" , Bruce Richardson , nd Thread-Index: AQHV/DIVvhCOm2Xh00q9xrfG3A7dKqh76lLg Date: Thu, 16 Apr 2020 13:37:44 +0000 Message-ID: <73990a21da324ef5ac759227017682fd@EX13D12EUA003.ant.amazon.com> References: <20200313091835.58039-1-gavin.hu@arm.com> <10e90cbc7bbe4d84979dabcb0fbe0c41@EX13D12EUA003.ant.amazon.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.43.166.83] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailman-Approved-At: Thu, 16 Apr 2020 22:03:27 +0200 Subject: Re: [dpdk-dev] [PATCH RFC v1 0/7] relax barriers for ENA PMD and small fixes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi all, Please see the first batch of comments related to these patches: 1. Relaxing the register read/write isn't always good enough. Specifically,= when barriers are required between different memory types, the reordering = can occur even on x86. Yet in DPDK the io/cio/smp barrier flavors for x86 a= re defined as compiler-only barriers, which is not enough in cases involvin= g different memory types. In ENA driver, when LLQ is on, there is a regular= register memory access before the barrier and write-combined memory access= after the barrier. We're working on a more extensive change that will include the optimization= s proposed for barriers relaxing while making them applicable to all platfo= rms. 2. Regarding the changes for statistics logging - the patch relies on c11 = features. I'm not sure it's acceptable for all situations since we've alrea= dy encountered a reports when even c99-compliant changes caused compilation= issues. 3. Removing redundant zeroing of sub-struct - we're currently working on so= me extensive changes to the Tx flow, which will include this change among o= ther. Thanks, Igor -----Original Message----- From: Gavin Hu =20 Sent: Tuesday, March 17, 2020 9:59 AM To: Chauskin, Igor ; dev@dpdk.org Cc: nd ; david.marchand@redhat.com; thomas@monjalon.net; mk@sem= ihalf.com; Tzalik, Guy ; Schmeilin, Evgeny ; mw@semihalf.com; Honnappa Nagarahalli ; Ruifeng Wang ; Phil Yang ; Jo= yce Kong ; Bshara, Saeed ; Matushevs= ky, Alexander ; Bruce Richardson ; nd Subject: RE: [EXTERNAL] [PATCH RFC v1 0/7] relax barriers for ENA PMD and s= mall fixes CAUTION: This email originated from outside of the organization. Do not cli= ck links or open attachments unless you can confirm the sender and know the= content is safe. Hi Igor, > -----Original Message----- > From: Chauskin, Igor > Sent: Monday, March 16, 2020 5:35 PM > To: Gavin Hu ; dev@dpdk.org > Cc: nd ; david.marchand@redhat.com; thomas@monjalon.net;=20 > mk@semihalf.com; Tzalik, Guy ; Schmeilin, Evgeny=20 > ; mw@semihalf.com; Honnappa Nagarahalli=20 > ; Ruifeng Wang ;=20 > Phil Yang ; Joyce Kong ;=20 > Bshara, Saeed ; Matushevsky, Alexander=20 > > Subject: RE: [PATCH RFC v1 0/7] relax barriers for ENA PMD and small=20 > fixes > > Hi Gavin, > > Thank you for the contribution. > Please do not merge these changes (patches 0..7) till we (the ENA=20 > team) properly review and ack/nack. > These changes can potentially provide performance improvement, yet we=20 > need to ensure they are applicable for all possible scenarios.=20 > Specifically, the behavior on x86 platforms is likely to be different. > What testing have you done for these patches? Was x86 tested? As noted in the cover letter, these patches were not tested as we don't hav= e ENA NICs. We rely on you to do that, any concerns and comments welcome. Yes, the behavior on x86 platforms is also different, Intel people are welc= ome to comment. /Gavin > > Thanks, > Igor > > -----Original Message----- > From: Gavin Hu > Sent: Friday, March 13, 2020 11:18 AM > To: dev@dpdk.org > Cc: nd@arm.com; david.marchand@redhat.com; thomas@monjalon.net;=20 > mk@semihalf.com; Tzalik, Guy ; Schmeilin, Evgeny=20 > ; Chauskin, Igor ;=20 > mw@semihalf.com; Honnappa.Nagarahalli@arm.com; ruifeng.wang@arm.com;=20 > phil.yang@arm.com; joyce.kong@arm.com > Subject: [EXTERNAL][PATCH RFC v1 0/7] relax barriers for ENA PMD and=20 > small fixes > > CAUTION: This email originated from outside of the organization. Do=20 > not click links or open attachments unless you can confirm the sender=20 > and know the content is safe. > > > > To ensure the stores to the host memory are observed by NIC HW before=20 > a door bell ring to the NIC HW and the HW starts actions, for example,=20 > doing DMA, a barrier is required on weak memory ordering platforms, like = aarch64. > > However, unnecessarily too strong barriers like 'dsb' on aarch64 will=20 > dampen performance. > > In a typical doorbell use case, as NIC and CPU are in the outer=20 > sharable domain, a lighter weight 'dmb osh' barrier is sufficient. > > The patch set relaxes the barriers in similar places and include one=20 > more patch for statistics logging with relaxed ordering and the other=20 > patch removing duplicate memset. > > Note this set is submitted for RFC as we don't have physical ENA NICs=20 > in the lab and the patch set was not verified nor benchmarked. > > Gavin Hu (7): > net/ena: remove duplicate barrier > net/ena: relax the barrier for doorbell ring > net/ena: relax the rmb for DMA > net/ena: relax barrier for completion queue update > net/ena: relax the barrier for bounce buffer > net/ena: use c11 atomic for statistics > net/ena: remove duplicate memset > > drivers/net/ena/base/ena_eth_com.c | 2 +- > drivers/net/ena/base/ena_eth_com.h | 6 ++-- > drivers/net/ena/base/ena_plat_dpdk.h | 2 +- > drivers/net/ena/ena_ethdev.c | 46 +++++++++++++++++----------- > drivers/net/ena/ena_ethdev.h | 8 ++--- > 5 files changed, 38 insertions(+), 26 deletions(-) > > -- > 2.17.1