From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E84A4A0352; Mon, 23 Dec 2019 08:52:11 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 296811C01; Mon, 23 Dec 2019 08:52:11 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 177C6F72; Mon, 23 Dec 2019 08:52:09 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Dec 2019 23:52:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,346,1571727600"; d="scan'208";a="229249831" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga002.jf.intel.com with ESMTP; 22 Dec 2019 23:52:08 -0800 Received: from fmsmsx156.amr.corp.intel.com (10.18.116.74) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sun, 22 Dec 2019 23:52:08 -0800 Received: from shsmsx106.ccr.corp.intel.com (10.239.4.159) by fmsmsx156.amr.corp.intel.com (10.18.116.74) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sun, 22 Dec 2019 23:52:08 -0800 Received: from shsmsx101.ccr.corp.intel.com ([169.254.1.19]) by SHSMSX106.ccr.corp.intel.com ([169.254.10.236]) with mapi id 14.03.0439.000; Mon, 23 Dec 2019 15:52:06 +0800 From: "Li, Xiaoyun" To: "Gavin Hu (Arm Technology China)" , "Wu, Jingjing" CC: "dev@dpdk.org" , "Maslekar, Omkar" , "stable@dpdk.org" , nd Thread-Topic: [dpdk-dev] [PATCH v2] raw/ntb: fix write memory barrier issue Thread-Index: AQHVs7Tq7x2YYQcLw0mJCFIUj/Hkaqe8D3eAgAtTD0A= Date: Mon, 23 Dec 2019 07:52:05 +0000 Message-ID: References: <20191204151916.12607-1-xiaoyun.li@intel.com> <20191216015854.28725-1-xiaoyun.li@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2] raw/ntb: fix write memory barrier issue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi I reconsidered and retested about this issue. I still need to use rte_wmb instead of using rte_io_wmb. Because to achieve high performance, ntb needs to turn on WC(write combinin= g) feature. The perf difference with and without WC enabled is more than 20= X. And when WC enabled, rte_io_wmb cannot make sure the instructions are in or= der only rte_wmb can make sure that. And in my retest, when sending 64 bytes packets, using rte_io_wmb will caus= e out-of-order issue and cause memory corruption on rx side. And using rte_wmb is fine. So I can only use v1 patch and suspend v2 patch in patchwork. Best Regards Xiaoyun Li > -----Original Message----- > From: Gavin Hu (Arm Technology China) [mailto:Gavin.Hu@arm.com] > Sent: Monday, December 16, 2019 18:50 > To: Li, Xiaoyun ; Wu, Jingjing > Cc: dev@dpdk.org; Maslekar, Omkar ; > stable@dpdk.org; nd > Subject: RE: [dpdk-dev] [PATCH v2] raw/ntb: fix write memory barrier issu= e >=20 >=20 >=20 > > -----Original Message----- > > From: dev On Behalf Of Xiaoyun Li > > Sent: Monday, December 16, 2019 9:59 AM > > To: jingjing.wu@intel.com > > Cc: dev@dpdk.org; omkar.maslekar@intel.com; Xiaoyun Li > > ; stable@dpdk.org > > Subject: [dpdk-dev] [PATCH v2] raw/ntb: fix write memory barrier issue > > > > All buffers and ring info should be written before tail register update= . > > This patch relocates the write memory barrier before updating tail > > register to avoid potential issues. > > > > Fixes: 11b5c7daf019 ("raw/ntb: add enqueue and dequeue functions") > > Cc: stable@dpdk.org > > > > Signed-off-by: Xiaoyun Li > > --- > > v2: > > * Replaced rte_wmb with rte_io_wmb since rte_io_wmb is enough. > > --- > > drivers/raw/ntb/ntb.c | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/raw/ntb/ntb.c b/drivers/raw/ntb/ntb.c index > > ad7f6abfd..c7de86f36 100644 > > --- a/drivers/raw/ntb/ntb.c > > +++ b/drivers/raw/ntb/ntb.c > > @@ -683,8 +683,8 @@ ntb_enqueue_bufs(struct rte_rawdev *dev, > > sizeof(struct ntb_used) * nb1); > > rte_memcpy(txq->tx_used_ring, tx_used + nb1, > > sizeof(struct ntb_used) * nb2); > > + rte_io_wmb(); > As both txq->tx_used_ring and *txq->used_cnt are physically reside in the= PCI > device side, rte_io_wmb is correct to ensure the ordering. >=20 > > *txq->used_cnt =3D txq->last_used; > > - rte_wmb(); > > > > /* update queue stats */ > > hw->ntb_xstats[NTB_TX_BYTES_ID + off] +=3D bytes; @@ -789,8 > +789,8 @@ > > ntb_dequeue_bufs(struct rte_rawdev *dev, > > sizeof(struct ntb_desc) * nb1); > > rte_memcpy(rxq->rx_desc_ring, rx_desc + nb1, > > sizeof(struct ntb_desc) * nb2); > > + rte_io_wmb(); > > *rxq->avail_cnt =3D rxq->last_avail; > > - rte_wmb(); > > > > /* update queue stats */ > > off =3D NTB_XSTATS_NUM * ((size_t)context + 1); > > -- > > 2.17.1 >=20 > Reviewed-by: Gavin Hu