From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B6A72A04A2; Tue, 12 May 2020 08:42:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B58641C06A; Tue, 12 May 2020 08:42:47 +0200 (CEST) Received: from mail-il1-f196.google.com (mail-il1-f196.google.com [209.85.166.196]) by dpdk.org (Postfix) with ESMTP id 293061BFF5 for ; Tue, 12 May 2020 08:42:46 +0200 (CEST) Received: by mail-il1-f196.google.com with SMTP id s10so11113533iln.11 for ; Mon, 11 May 2020 23:42:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=aDjuLek21/YEn3Pppg2Ei5+PKqUcZakf79YzcjBeUwI=; b=rUFLEZcLRXenMovV19Ioccqierv1f0ZAZwYD2V987+/0FmR4CfCh5+qPgOLx9wweTO nSHFNqvaV+NUFYCh8hnjfzojJPha5gq0UoCre5YAKXgifCmjbpenfN6kGgQzE++djrX3 u5hHFvsyR3ncUN49PA1p3y/t4bwWaLmQJIZq4JuzJA7uZtP7toomBa8DYnQIr0LuLI0A 7hYDkexhWJpl+66nCRCjcwuy6T+8+yxIoIHDhJ4fzdsu9xjN3WiXKh9ABoLuOjQb+y1S 8RWERG4RSRze4C8WA6WqP385EPLeLSUqy+AuD+t5q6Q+EZHDo6HPvqftHa4xjgunKuVV ulUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=aDjuLek21/YEn3Pppg2Ei5+PKqUcZakf79YzcjBeUwI=; b=jgM0eebQNA0cuDgicErHtlbEIrLJSFkGh25Kr6zDRqefCVLqiFGoT1x7fl5x4pQdCr D7Zo/kKz9mx0NFlh7o1C/RRA1ICHu1S/8vW9NZZUbG/ak/7sVEODzdO7HP0iOJPrJGlX 8k4E+WtWZhVba72415tSrK3gSNPONJuieD46C38wCXmcadxF4Q0Px/n+wUgWfXoCY+Dj 3+deqdYJapsw8cff5OVeyK2jG5fe4bbY/o9YEHbdRePVq4QSQqvpRIL5xVjiMAKbGUff mSdNhKjJ1UW8vWFUyeongLrtVWG0Rk25UZWpzZEGRkXWtWVOyNJy8gUE50NSCtuyOi3K bhmw== X-Gm-Message-State: AGi0Pua0egyydrWWnjo+EiRjRbfJUf5aQAAN99EWi7IeFEwfFsXSwmio HBTuRILXvnTjeZLQzzvRBJXCUF4StRNef/I+7u4= X-Google-Smtp-Source: APiQypIUYoEuIFU6bfFLapSLKdc6s9kXdwM6LkTMcMYKbkoWMuMffwGbSU2ZsnBPX+5LrSiXGpijSsf5F1iTqGyiv/M= X-Received: by 2002:a92:d188:: with SMTP id z8mr723564ilz.60.1589265765217; Mon, 11 May 2020 23:42:45 -0700 (PDT) MIME-Version: 1.0 References: <20200410164127.54229-1-gavin.hu@arm.com> <20200511180637.22200-1-honnappa.nagarahalli@arm.com> In-Reply-To: From: Jerin Jacob Date: Tue, 12 May 2020 12:12:29 +0530 Message-ID: To: Ruifeng Wang Cc: Honnappa Nagarahalli , "dev@dpdk.org" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , "Ajit Khaparde (ajit.khaparde@broadcom.com)" , "igorch@amazon.com" , "thomas@monjalon.net" , "viacheslavo@mellanox.com" , "arybchenko@solarflare.com" , nd Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [RFC] eal: adjust barriers for IO on Armv8-a X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Tue, May 12, 2020 at 11:48 AM Ruifeng Wang wrote: > > > > -----Original Message----- > > From: Honnappa Nagarahalli > > Sent: Tuesday, May 12, 2020 2:07 AM > > To: dev@dpdk.org; jerinj@marvell.com; hemant.agrawal@nxp.com; Ajit > > Khaparde (ajit.khaparde@broadcom.com) ; > > igorch@amazon.com; thomas@monjalon.net; viacheslavo@mellanox.com; > > arybchenko@solarflare.com; Honnappa Nagarahalli > > > > Cc: Ruifeng Wang ; nd > > Subject: [RFC] eal: adjust barriers for IO on Armv8-a > > > > Change the barrier APIs for IO to reflect that Armv8-a is other-multi-copy > > atomicity memory model. > > > > Armv8-a memory model has been strengthened to require other-multi-copy > > atomicity. This property requires memory accesses from an observer to > > become visible to all other observers simultaneously [3]. This means > > > > a) A write arriving at an endpoint shared between multiple CPUs is > > visible to all CPUs > > b) A write that is visible to all CPUs is also visible to all other > > observers in the shareability domain > > > > This allows for using cheaper DMB instructions in the place of DSB for devices > > that are visible to all CPUs (i.e. devices that DPDK caters to). > > > > Please refer to [1], [2] and [3] for more information. > > > > [1] > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i > > d=22ec71615d824f4f11d38d0e55a88d8956b7e45f > > [2] https://www.youtube.com/watch?v=i6DayghhA8Q > > [3] https://www.cl.cam.ac.uk/~pes20/armv8-mca/ > > > > Signed-off-by: Honnappa Nagarahalli > > --- > > lib/librte_eal/arm/include/rte_atomic_64.h | 10 +++++----- > > 1 file changed, 5 insertions(+), 5 deletions(-) > > > > diff --git a/lib/librte_eal/arm/include/rte_atomic_64.h > > b/lib/librte_eal/arm/include/rte_atomic_64.h > > index 7b7099cdc..e406411bb 100644 > > --- a/lib/librte_eal/arm/include/rte_atomic_64.h > > +++ b/lib/librte_eal/arm/include/rte_atomic_64.h > > @@ -19,11 +19,11 @@ extern "C" { > > #include > > #include > > > > -#define rte_mb() asm volatile("dsb sy" : : : "memory") > > +#define rte_mb() asm volatile("dmb osh" : : : "memory") > > > > -#define rte_wmb() asm volatile("dsb st" : : : "memory") > > +#define rte_wmb() asm volatile("dmb oshst" : : : "memory") > > > > -#define rte_rmb() asm volatile("dsb ld" : : : "memory") > > +#define rte_rmb() asm volatile("dmb oshld" : : : "memory") > > > > #define rte_smp_mb() asm volatile("dmb ish" : : : "memory") > > > > @@ -37,9 +37,9 @@ extern "C" { > > > > #define rte_io_rmb() rte_rmb() > > > > -#define rte_cio_wmb() asm volatile("dmb oshst" : : : "memory") > > +#define rte_cio_wmb() rte_wmb() > > > > -#define rte_cio_rmb() asm volatile("dmb oshld" : : : "memory") > > +#define rte_cio_rmb() rte_rmb() > > > > /*------------------------ 128 bit atomic operations -------------------------*/ > > > > -- > > 2.17.1 > > This change showed about 7% performance gain in testpmd single core NDR test. I am trying to understand this patch wrt DPDK current usage model? 1) Is performance improvement due to the fact that the PMD that you are using it for testing suppose to use existing rte_cio_* but it was using rte_[rw]mb? 2) In my understanding : a) CPU to CPU barrier requirements are addressed by rte_smp_* b) CPU to DMA/Device barrier requirements are addressed by rte_cio_* c) CPU to ANY(CPU or Device) are addressed by rte_[rw]mb If (c) is true then we are violating the DPDK spec with change. Right? This change will not be required if fastpath (CPU to Device) is using rte_cio_*. Right? > Tested-by: Ruifeng Wang >