From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-SN1-obe.outbound.protection.outlook.com (mail-sn1nam01on0067.outbound.protection.outlook.com [104.47.32.67]) by dpdk.org (Postfix) with ESMTP id 874B55F28 for ; Thu, 20 Sep 2018 10:29:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uevAUr8Oeh8sgEx0zzP0eBNUNux+NVSz+F/rkKMCfm4=; b=N527tdi/REwn01B28Z1w5oCT6lt2ESiva0VQO8usJp3BKAJHBVo/jzzQP4PwGcmTgsY/Rb10jkwlWvpfWrzBfRcIwKNmEC2uXo0mKRnNepJhTTy7qPOyFaU4BbGs0h6C+B0WbTq7ebzKGkNnXdEgrJRFSIqFKZBA1T6U7fJSD1k= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.JacobKollanukkaran@cavium.com; Received: from jerin (111.93.218.67) by BYAPR07MB4997.namprd07.prod.outlook.com (2603:10b6:a03:5b::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.17; Thu, 20 Sep 2018 08:29:07 +0000 Date: Thu, 20 Sep 2018 13:58:47 +0530 From: Jerin Jacob To: Phil Yang Cc: dev@dpdk.org, nd@arm.com, kkokkilagadda@caviumnetworks.com, Honnappa.Nagarahalli@arm.com, Gavin.Hu@arm.com, ferruh.yigit@intel.com Message-ID: <20180920082846.GB19425@jerin> References: <1537363820-3827-1-git-send-email-phil.yang@arm.com> <1537364560-4124-1-git-send-email-phil.yang@arm.com> <1537364560-4124-2-git-send-email-phil.yang@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1537364560-4124-2-git-send-email-phil.yang@arm.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: BM1PR0101CA0064.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:19::26) To BYAPR07MB4997.namprd07.prod.outlook.com (2603:10b6:a03:5b::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 39727e28-cb53-49e0-a521-08d61ed32329 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534165)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BYAPR07MB4997; X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4997; 3:Fx6WBEEKPcWEDLQJ32f03RO6ueXxUTdI9kqUZ3GP4xBqU2RgcKUwZjxPcyK9UvMt+aCPn//6umBcMUylkhucaODYlLwpiRQL7btCgQtPrji+3xsHOYK8RwWXwfFFiqhPFM+Rjhc6KXWlec6OlHfXdIEUDtzEj/EMMBbfVq+w4aRrYnq/wx4CmRLW5RvZuQl/89UWxzvSuAiRm9gsE8m7v6GdEhO5h4QdaeVhEBzzb1TYDumaqkEEgJKoIjlZuFC4; 25:r03LCJ0vhQ+GXKjl3vD7v+nVXiAP75gLL/rwMjfHrj2Trbkyo8T8GH+2j3fJU3Z2MEaAn2Wk7j9a1uH5AD7D/IoDvgN5eXF3R3GqFydFyRzZCGOCS6WIHarQK3wRfA/8iul/NYoaZzjiGD6lbcqa4Z3qcJidkKkf7i+QXKJTIrbDwZtYQqDKtxwYmsxsm3knvXgZmFmLO1ztD++FVuQZcsLObry/xOmCnvJjdZqE8Bf49fe2mXWxdJ3baJrS0rLF3e9CC8+G53RtulOgVVmozSTXaeSnSMoBK1EZFMiE1/7NRqH5EMwDgSmSd9Cu8nhzl7XfT7Y643vcT/MWK734qg==; 31:NF8CWw6PoWpunnyh85HEO4+4ZDj0Bm9ow2NDWJ0y085wxXSKtHB713SZJ1Z6RpSDHobHXo6PL3t5jnO7792fJdCujCLySyFN8bwJr7d5WshN0P9ohj6/NKUlYByfwDEv1l0rFPpg7hxmS5F13IC9M5n6Mv7YoC3KhOk9qRpZXmO58EQFybxhcxIhYZstJvLCFnMjunH4k1IIL6yo5SKkVgd2SVpHQScHzJ+5od3D3KI= X-MS-TrafficTypeDiagnostic: BYAPR07MB4997: X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4997; 20:QedceENK9Um4mhCW2lVLrhwepGhU7KgETMRboz19Lz3ZJNOiU3Sb55wFuKsSRSEXWROnABdfzaMVS/4jP5XtZArcqaww8r69rIPSVD+56ntrfgHSi/sahlzrTgmnArA9xddjTgNZLs5GDh88z7OUpcBtGiW4yA4987DFoBHBdWVcFOEPwoM6uXrnuPUxNvh+9fU3VdeO+9lxT0vdW1ofvTledZFiia165vGuoCO3y7sbFxzHRCgBsZhemN1ysd6cCJbISC4qWvKcwX/sYIOMm1MASQ6VU1EjgKf8oF2zMTZq64XcAtX0RtDAl6A18lbPYfmmiOzww5dVLWa5a9cGFXRB/sMVzLfBwIPfbTL25SOVmFfYIJhbaBDygWkEPt+rwGl2SkBiOgh6iGSTDz/ujTc1ts0Sk5XbhvqBmwahuKKMn09tJFAxCN2y57xuTgEZAHOKT6OGNZdGnZwxpwO87jCwcN7ia/1mhqMnelWM2ni/X1nQnQk61aLgf2831L4yiT1e7/lpXJz6ccZHKxNVaGv1b/YelpEggnvNMmczQ7Zmj3ZNxtzkA+eACkEtgD1D9iomPSnZrJkMqopeELpwIrQWG+TPfmyS38ReEIXMCdY= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(180628864354917)(228905959029699); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(10201501046)(3231355)(944501410)(52105095)(93006095)(149027)(150027)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123562045)(20161123560045)(201708071742011)(7699051); SRVR:BYAPR07MB4997; BCL:0; PCL:0; RULEID:; SRVR:BYAPR07MB4997; X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4997; 4:z5XXapHtEvWNtVodkE5owAFTexLAGiP4BisTrUJUGwAvN0B1gSqKJHltvRmAJkTQUUzu0u3EdaPbPLUCoCVoyw7vLk8zn49DsVRXjfyQnUnRcvpm6eBqrheCso6PlGLtG9sIloLiq4/wH/naSjJeiR8xBMpaKO1diZNZ6FG6qNWGakGd/X9pFm/RwViJU5uyEEKN6lw8N3tc32tAIWOZodMbmaGGJzVpQU5NQhVRzkP1nMO07rZLY4Dfk/nMDvmQt/1gdko3JI6krkmzKyi8ukVxTTEbsyNxh27rcHa5cpHhXxVDYpW/nJEcccfHnVIdUAktX3yfGngG5jgUE+ZTvg== X-Forefront-PRVS: 0801F2E62B X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39860400002)(396003)(376002)(346002)(136003)(366004)(189003)(199004)(13464003)(57704003)(23726003)(476003)(7736002)(478600001)(72206003)(14444005)(76176011)(68736007)(316002)(55016002)(4326008)(2906002)(47776003)(66066001)(33896004)(5009440100003)(81156014)(6246003)(81166006)(6496006)(8676002)(6666003)(52116002)(6916009)(97736004)(8936002)(386003)(186003)(53936002)(1076002)(956004)(33656002)(6116002)(16526019)(486006)(3846002)(33716001)(58126008)(5660300001)(16586007)(229853002)(305945005)(50466002)(44832011)(11346002)(106356001)(9686003)(446003)(42882007)(25786009)(105586002)(26005)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:BYAPR07MB4997; H:jerin; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BYAPR07MB4997; 23:2g3xwBwBaY0Gye5qSxwXTeiH/sg45bjFWNlkuQiXq?= =?us-ascii?Q?Lu6kEzjfmggXofbkzo9o3tqkSlgoesdvDAVblcJF7TFxuyoYHf3jmT5WtsAH?= =?us-ascii?Q?3nrx4iooEF8w748zwHq8RjEDSI0kQwbd9zsNkNHhzgPziomwagRTN87z14bC?= =?us-ascii?Q?tVniFSysX6migUN4qPibpcncs5sNYoYXyifGydbekYV6Gaa1omBBh1Ct8XGK?= =?us-ascii?Q?YhJITqEjo/wnQhN+QOKAfEYlOo9mwu5yf9dluknBB6tPfmhQ5wkEjz2IQ+1K?= =?us-ascii?Q?jc1q699e+ZCDEuDod+GZYMXGZZsql+TVd33xe5kFyfiFlgOqE8rGoShQRdDV?= =?us-ascii?Q?GWKydNi6y+5rRKf5q+8ToOZedH9eJNSxD2pdSV5qryVWlK2V9CJr+pPqVWjo?= =?us-ascii?Q?TPzBJncWXOUzCsEEEDwEYRrOpjndSbGjoFQEo6sTMDpWWOfHh90zrkS+ZiKP?= =?us-ascii?Q?6wpeawBbbLDxJcN//fs6r4FmAGgg6SM1VfOiuzhNmxjYDvNOP5y2W64fHOgT?= =?us-ascii?Q?V5T0/6A1Ao4nXTNPIPiv5ej2Ff08uMtzUXwfubq++LaUbRIm863V6d3rFn/B?= =?us-ascii?Q?I/ibx4It/JMHU8cCijeQbH+8L1b9gZI53S98DGIqTYerFiaNZ9sq1ZSA8lKQ?= =?us-ascii?Q?Y08XTNIca61EH0+jvolFdN3aXfapGByhFkKBFa2r1iOT8MJDcL0yMfPu7ooL?= =?us-ascii?Q?KKLcT4T5fLlVQ6MQe8ntoQJqeF+/hi1cnhZMz3rej8jMk2RZGQZ1wEpH7WfL?= =?us-ascii?Q?NhubNcayOjyKXvk8ijM9PT0uX3EfLZuMr4PWodjCQ3L1KD3oEGXvfQ3IHhPJ?= =?us-ascii?Q?T712+Ic7Kqt7smN3JazreIxXGfelJfPTuzJNRjIBZw8k3zhj32SBzqS0YRzu?= =?us-ascii?Q?PFnVLY5St74kV2VZs5JC+NJhWGZ66SA9VzyK1W5L/QGQXdYyZFFZZwQO1Sv2?= =?us-ascii?Q?zHL8fs+jP3ZfEFOKtnFzavRG7i8BwDOa8zoE4NHysLg8yyfTMH8y3u9HGhHp?= =?us-ascii?Q?GAGjFu90Z0zo4X3l/xlGdVbrgP9vzX3Bz8TMzRIxqAPL+u7gjBdM7UAt1HPB?= =?us-ascii?Q?1aW6B7Fhl56FLR5yCfzu1kcvJ/OgsE5ejbcStDVTaA5V8fUEn18hdRf6pHBR?= =?us-ascii?Q?YjzZWx3dbRPwt3T1Hify7t/y4WERPtqSGH4S3h5mQLDFR7HxGOAhVm78lVug?= =?us-ascii?Q?4Md9pQP5zR9QrxEB53FZ17Y/09eFOwD01s+IL9+m5FzZy9tc37TyuGds7HrH?= =?us-ascii?Q?c/fUYQC75zPS5/BAXcndeShEqkqGTda0kK35P1Ke4Mmp9krCVTee65TL+tPw?= =?us-ascii?Q?RoSa3cuxooJeh2zUgpNzPF/JpOcP/P1JOT/Cqeyb1Q5KfBLLba7CIwyKk3EU?= =?us-ascii?Q?BOFB0txriYj0vbNPjqJ54xOo3P0SNZssdGJrB3eN8FW07Wj?= X-Microsoft-Antispam-Message-Info: YSMRcX0ELJkX5/sL1SbYLBanyBR69zVmrZUkid7C23ibsYjwvP/06bVZViV2R8jBvDVq2GKdpyYvygPfbQUKUFpO1W46KW8vsK1ZWoVxwvt1tX+CRQPuu7JQiVlQG7XEH5bCf8uB0ufM7ubEULmflIsB/b0FRL+JwkxiqQ+Kg9uQPB9TnKjZYu12WqytyQ4zarM+J8WMnDOKQa5GdXqjOgj/n8aNnuVyroJ9rdzbUB+M+LcfLnGR8TA9yNAfxMGcLlEUmhc0cXz3OTFKEOfr9H+gPznuPdJuHn23fbPg63R8l14KP8ESPtFr7QZpX1UTroZn/MRjUwulr/O8rjr3CnWHhCh4uIg0CtPjH1X4ar8= X-Microsoft-Exchange-Diagnostics: 1; BYAPR07MB4997; 6:R0uOKGs/NgzxNs+oIYKYxuKHyiRTTO9AkWaEJHL2kzRmO6GGqwrpNdeNeLWmrcoUZKaV2755IiLPTrAw9m/tl7Ee9M36f5wOatnI7uwQ0jSwIo9BT99D4nIMBYbGxVIuTSJITvVtMXcfAC4/Jz3FAaIkyxVafMos9W/rXIIAluGnTffNgAkw4PMV6W6jlaD6ovTHIifoC82vzbLjsHrPSbQJN1z2RbZXrKKQcc6zeJvk6wx0zm/43uI2o75uwA2yagsyKqFR887MJdTP4GbCdOurSR9BM2tyVI6btvUdZD5O8RYJl5KOsFoOMrDQVpuSHSQIS3m1hcaEyvWwE5z2FsnuXoUv52Bb7kc4mUKPe1TGRzqLbtBblH0RsOtHLBzPNQsDHns972BGjqoQzKZMI2LjWOY94DZQmjqn3nJVSzvYx8zDXqDI3vGShyEW4MAbWCpJEObpMRf/FEEfgfBe6Q==; 5:Q6pMUTHUWEFbJDl8AJzbYj/fA9yXZgTPDbOvje+N8Ti0DvyLwHAMKog/ynEn4oCEsKEfAkUPBCDltsTyg5ccz0HeFf0F3INP8PAptMoiBfyz0km94i2DPbW5OZX0JYZKSixJIOvkpY5rpVtz6HfU/ubICehNsTJiIG1jHwqCpn8=; 7:P7XB7mwr/76vy/+29UNrL03CZyYUX+JEGmqA7RD9XaD7LMxK7I5weN5cPqUUog9uEL7p0mg4nkkCvBto7+ZfuJgtGkDIAtzEkh7TJsNgBnFLtc+E1KDXiO+U1D3mGt6ObS0cg2yK7r1LIUQU/yiePzx7MhiTcvlKCg52SVRIx3QKu+j9E5DywZeb/wKOCNE5QYiDlLlYatkzFhICaPpVb90sULwqvhlv8fBPwP3VaduE5daCNHJMgopylWn3yQOI SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2018 08:29:07.1191 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 39727e28-cb53-49e0-a521-08d61ed32329 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR07MB4997 Subject: Re: [dpdk-dev] [PATCH v2 2/3] kni: fix kni fifo synchronization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 08:29:12 -0000 -----Original Message----- > Date: Wed, 19 Sep 2018 21:42:39 +0800 > From: Phil Yang > To: dev@dpdk.org > CC: nd@arm.com, jerin.jacob@caviumnetworks.com, > kkokkilagadda@caviumnetworks.com, Honnappa.Nagarahalli@arm.com, > Gavin.Hu@arm.com > Subject: [PATCH v2 2/3] kni: fix kni fifo synchronization > X-Mailer: git-send-email 2.7.4 > + Ferruh Yigit > > With existing code in kni_fifo_put, rx_q values are not being updated > before updating fifo_write. While reading rx_q in kni_net_rx_normal, > This is causing the sync issue on other core. The same situation happens > in kni_fifo_get as well. > > So syncing the values by adding C11 atomic memory barriers to make sure > the values being synced before updating fifo_write and fifo_read. > > Fixes: 3fc5ca2 ("kni: initial import") > Signed-off-by: Phil Yang > Reviewed-by: Honnappa Nagarahalli > Reviewed-by: Gavin Hu > --- > .../linuxapp/eal/include/exec-env/rte_kni_common.h | 5 ++++ > lib/librte_kni/rte_kni_fifo.h | 30 +++++++++++++++++++++- > 2 files changed, 34 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > index cfa9448..1fd713b 100644 > --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > @@ -54,8 +54,13 @@ struct rte_kni_request { > * Writing should never overwrite the read position > */ > struct rte_kni_fifo { > +#ifndef RTE_USE_C11_MEM_MODEL > volatile unsigned write; /**< Next position to be written*/ > volatile unsigned read; /**< Next position to be read */ > +#else > + unsigned write; /**< Next position to be written*/ > + unsigned read; /**< Next position to be read */ > +#endif > unsigned len; /**< Circular buffer length */ > unsigned elem_size; /**< Pointer size - for 32/64 bit OS */ > void *volatile buffer[]; /**< The buffer contains mbuf pointers */ > diff --git a/lib/librte_kni/rte_kni_fifo.h b/lib/librte_kni/rte_kni_fifo.h > index ac26a8c..f4171a1 100644 > --- a/lib/librte_kni/rte_kni_fifo.h > +++ b/lib/librte_kni/rte_kni_fifo.h > @@ -28,8 +28,13 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, unsigned num) > { > unsigned i = 0; > unsigned fifo_write = fifo->write; > - unsigned fifo_read = fifo->read; > unsigned new_write = fifo_write; > +#ifdef RTE_USE_C11_MEM_MODEL > + unsigned fifo_read = __atomic_load_n(&fifo->read, > + __ATOMIC_ACQUIRE); > +#else > + unsigned fifo_read = fifo->read; > +#endif Correct. > > for (i = 0; i < num; i++) { > new_write = (new_write + 1) & (fifo->len - 1); > @@ -39,7 +44,12 @@ kni_fifo_put(struct rte_kni_fifo *fifo, void **data, unsigned num) > fifo->buffer[fifo_write] = data[i]; > fifo_write = new_write; > } > +#ifdef RTE_USE_C11_MEM_MODEL > + __atomic_store_n(&fifo->write, fifo_write, __ATOMIC_RELEASE); > +#else > + rte_smp_wmb(); > fifo->write = fifo_write; > +#endif Correct. > return i; > } > > @@ -51,7 +61,12 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) > { > unsigned i = 0; > unsigned new_read = fifo->read; > +#ifdef RTE_USE_C11_MEM_MODEL > + unsigned fifo_write = __atomic_load_n(&fifo->write, __ATOMIC_ACQUIRE); > +#else > unsigned fifo_write = fifo->write; > +#endif Correct. > + > for (i = 0; i < num; i++) { > if (new_read == fifo_write) > break; > @@ -59,7 +74,12 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) > data[i] = fifo->buffer[new_read]; > new_read = (new_read + 1) & (fifo->len - 1); > } > +#ifdef RTE_USE_C11_MEM_MODEL > + __atomic_store_n(&fifo->read, new_read, __ATOMIC_RELEASE); > +#else > + rte_smp_wmb(); > fifo->read = new_read; > +#endif Correct. > return i; > } > > @@ -69,5 +89,13 @@ kni_fifo_get(struct rte_kni_fifo *fifo, void **data, unsigned num) > static inline uint32_t > kni_fifo_count(struct rte_kni_fifo *fifo) > { > +#ifdef RTE_USE_C11_MEM_MODEL > + unsigned fifo_write = __atomic_load_n(&fifo->write, > + __ATOMIC_ACQUIRE); > + unsigned fifo_read = __atomic_load_n(&fifo->read, > + __ATOMIC_ACQUIRE); Isn't too heavy to have two __ATOMIC_ACQUIREs? a simple rte_smp_rmb() would be enough here. Right? or Do we need __ATOMIC_ACQUIRE for fifo_write case? Other than that, I prefer to avoid ifdef clutter by introducing two separate file just like ring C11 implementation. I don't have strong opinion on this this part, I let KNI MAINTAINER to decide on how to accommodate this change. > + return (fifo->len + fifo_write - fifo_read) & (fifo->len - 1); > +#else > return (fifo->len + fifo->write - fifo->read) & (fifo->len - 1); > +#endif > } > -- > 2.7.4 >