From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 558B3A0598; Tue, 21 Apr 2020 19:43:48 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 94AE01D414; Tue, 21 Apr 2020 19:43:47 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 7B0D11D146; Tue, 21 Apr 2020 19:43:44 +0200 (CEST) IronPort-SDR: g0dQi5MRBY14fGkV0kcxdNoGpmDuXjO0N/yFxHHizmHxjHUqggx/fFvRWfNcjh5kAwLRS6282U 49FTin9p1J1A== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Apr 2020 10:43:42 -0700 IronPort-SDR: yKuMXcomSIaEePmX0pXQgNBA12zj1bA/uxPPU1xb3D0LNITH5ZiT0euzWY9C5qKkPMYnPIjTP6 A8couhbghKyg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.72,411,1580803200"; d="scan'208";a="291668700" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga008.jf.intel.com with ESMTP; 21 Apr 2020 10:43:42 -0700 Received: from orsmsx602.amr.corp.intel.com (10.22.229.15) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 21 Apr 2020 10:43:42 -0700 Received: from orsmsx602.amr.corp.intel.com (10.22.229.15) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 21 Apr 2020 10:43:41 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by orsmsx602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 21 Apr 2020 10:43:41 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.100) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 21 Apr 2020 10:43:41 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DvYoc0sZs4wmVc3G23udj0b0GYWAKNNXX7SSQ1GXNrgd56rHMCLHAqg4w8PqO5/chas9HduLBtNJZ5Hq5hZjOv2DCtSXV08Ivl8gqxhWAiMeV94r7XXz4dhfb6eYlmOeKJjk/0H0zsijQLMslSU61hjywlsHvO5aPKdccqU+UFYFQ1LA/QVkb00n+++sfkaMRaAaZj4YR3KxAvE7v53pM5TLjJr5xA0o3PWqer9qvoIzzBHbfQVhYC2OUqI74FYK5U46+rAOU74pKmemKrV9WcTwWHDV7vocQz+OYjFXNtxXDwBRlpwd/GfwnSaG91PhwC0dBrCdg9/CSPi+Kh2Ctg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QpNJMFItdNW7oMJAB5WYJQ1XtCz15VUGphPM9mtaWc0=; b=YKnGlsrUfliVsxTc9q7r539v58eoZia3BAD3bzsVZ1KBq4MGdDcOerdwlJlT41gYP+Us//yH81irJbdQX8q9A60FF7Met0wevI31cVHBLqs87UEZ/01QCkx0ihcwsmM6jzzbOKCVdjx3/zshGcjdzSOL+wAnTQJli0spW3Toj1LfPJ08pcRauc+aRBIMG1WrNRJUSxjeBWUdSAcu+cWMw2W/IZwZicn0gLfWPjWDXiP0t1xY63x7SU12FQ/GcGvtZP7gBxzzP9zLul2qgTwAkdxR8xsGvJhaSBgD1KZ7d4Spjy88c4dvwUxifrCcomITGEj/3TXbkMMWCqy4NY9fEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QpNJMFItdNW7oMJAB5WYJQ1XtCz15VUGphPM9mtaWc0=; b=XNfMGC4nXKpmBpCwkbAdpdzpu3NN80gKDbT1vsrkpRve1D3Z98DjCIsrCRMV7ndQUCB7Zxh3dobiSWoaekTdkQn4rTp8wXGf10ido+A147I5kTwBVeiKuboQpD4LPL3rAZnE2Z6Ji8V7aSxk04Xb/rczogqnfoOGPjhudb5xzg8= Received: from BYAPR11MB3143.namprd11.prod.outlook.com (2603:10b6:a03:92::32) by BYAPR11MB2535.namprd11.prod.outlook.com (2603:10b6:a02:be::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.25; Tue, 21 Apr 2020 17:43:40 +0000 Received: from BYAPR11MB3143.namprd11.prod.outlook.com ([fe80::5917:9757:49b0:3c49]) by BYAPR11MB3143.namprd11.prod.outlook.com ([fe80::5917:9757:49b0:3c49%5]) with mapi id 15.20.2921.030; Tue, 21 Apr 2020 17:43:40 +0000 From: "Van Haaren, Harry" To: Honnappa Nagarahalli , Phil Yang , "thomas@monjalon.net" , "Ananyev, Konstantin" , "stephen@networkplumber.org" , "maxime.coquelin@redhat.com" , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "jerinj@marvell.com" , "hemant.agrawal@nxp.com" , Gavin Hu , Ruifeng Wang , Joyce Kong , nd , "stable@dpdk.org" , nd , nd Thread-Topic: [PATCH v3 09/12] service: avoid race condition for MT unsafe service Thread-Index: AQHV+/one+9ki+cHakWt/XJq7A5CNahnSwOQgAITmoCABkCLIIAAhjMAgAD74QCADXoNAIAFckdg Date: Tue, 21 Apr 2020 17:43:40 +0000 Message-ID: References: <1583999071-22872-1-git-send-email-phil.yang@arm.com> <1584407863-774-1-git-send-email-phil.yang@arm.com> <1584407863-774-10-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: spf=none (sender IP is ) smtp.mailfrom=harry.van.haaren@intel.com; x-originating-ip: [192.198.151.163] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bd1fd2e6-c15f-474f-e723-08d7e61b874b x-ms-traffictypediagnostic: BYAPR11MB2535: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 038002787A x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3143.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10019020)(366004)(136003)(376002)(39860400002)(346002)(396003)(7416002)(54906003)(110136005)(6506007)(86362001)(71200400001)(478600001)(53546011)(4326008)(66946007)(76116006)(66446008)(64756008)(66556008)(66476007)(26005)(7696005)(2906002)(9686003)(52536014)(186003)(8936002)(81156014)(316002)(33656002)(8676002)(5660300002)(55016002); DIR:OUT; SFP:1102; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qTx+XFuvgt3TtB37bcG3PdU+EtNFSaQ0xcTraoJyGTbvgSMlaylAL4GE4n4XaLbxll/Q6Zw/fgRgDKvXgIr9/MFZWTk6uBDWMseNeTDM1yKMYeg4y08qbCg3jjvs3SlMQG0Yp0vPu3SkX6BlAiaU+XMw1F92uxcQ/f5B4PUpmDmwGRCrWScfvmJQo6lXVHTgW2kT6uz1SMFDX67U9z+iiE1M5t0zQNZg+WFzuLKWJs+nPSLzTn8rR1K47FZut4FDJ1VrgAxCQKKhYkSq75PC2UhQQoP9RHQBVZvkdHGovyWPpztOqOcpO76Xdhwt9TYuiTK+/1MhTGNpGjGreaiMvogN4616nYPyvcykvYINZ5TVuAyYrJ+RB5AhV4hbW4KUVMSlv3BEvsrOuH3LyngAOPGqaETOy+O2y8SFKY7wtYtuAETmH/JgGNvdmfMab5FQ x-ms-exchange-antispam-messagedata: KnmP6KJ1gyJPjK3COQJpySc557BCUNR+BzxzPUMuWANkv+BGqiQtUmAsl0b6b2FmbbZNSB+6NMVZLL+GkG45sni8trwJac5uI2kBwv7bkrP52EywTe6PDlfm/NKbIlYZ7afbg1XU5iO3RMLn9Ejjlg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: bd1fd2e6-c15f-474f-e723-08d7e61b874b X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Apr 2020 17:43:40.3209 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Qltkbu4JfUk/k3lIeYeCUnWLlodomrMJ4ey1WCJDuM1qJ0w71gqTjkKm1exfU3m0I2MkjX5NXOqeb7TEhWmKD6u+YBqDAlWqONwOtKBX8zw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2535 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 09/12] service: avoid race condition for MT unsafe service X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Honnappa Nagarahalli > Sent: Saturday, April 18, 2020 7:22 AM > To: Van Haaren, Harry ; Phil Yang > ; thomas@monjalon.net; Ananyev, Konstantin > ; stephen@networkplumber.org; > maxime.coquelin@redhat.com; dev@dpdk.org > Cc: david.marchand@redhat.com; jerinj@marvell.com; > hemant.agrawal@nxp.com; Gavin Hu ; Ruifeng Wang > ; Joyce Kong ; nd > ; stable@dpdk.org; nd ; Honnappa Nagarahalli > ; nd > Subject: RE: [PATCH v3 09/12] service: avoid race condition for MT unsafe= service >=20 > > > > > To achieve our desired goal; > > > > - control thread doing mapping performs the following operations > > > > -- write service->num_mapped_lcores++ (atomic not required, onl= y > > > > single- writer allowed by APIs) > > > This has to be atomic because of rte_service_run_iter_on_app_lcore AP= I. > > > Performance should be fine as this API is not called frequently. But > > > need to consider the implications of more than one thread updating > > num_mapped_cores. > > > > > > > -- MFENCE (full st-ld barrier) to flush write, and force later > > > > loads to > > > issue > > > > after > > > I am not exactly sure what MFENCE on x86 does. On Arm platforms, the > > > full barrier (DMB ISH) just makes sure that memory operations are not > > > re-ordered around it. It does not say anything about when that store > > > is visible to other cores. It will be visible at some point in time t= o cores. > > > But, I do not think we need to be worried about flushing to memory. > > > > > > > -- read the "calls_per_service" counter for each lcores, add th= em up. > > > This can be trimmed down to the single core the service is mapped to > > > currently, no need to add all the counters. > > > > Correct - however that requires figuring out which lcore is running the= service. > > Anyway, agree - it's an implementation detail as to exactly how we dete= ct it. > > > > > > > > > ---- Wait :) > > > > -- re-read the "calls_per_service", and ensure the count has ch= anged. > Here, there is an assumption that the service core function is running on= the > service core. If the service core is not running, the code will be stuck = in this > polling loop. Right - we could add a timeout ehre, but that just moves the problem somewh= ere else (the application) which now needs to handle error rets, and possibly r= etries. It could be a possible solution.. I'm not in favour of it at the moment, bu= t it needs some more time to think. > I could not come up with a good way to check if the service core is runni= ng. > Checking the app_runstate and comp_runstate is not enough as they just > indicate that the service is ready to run. Using the counter 'calls_per_s= ervice' > introduces race conditions. >=20 > Only way I can think of is asking the user to follow a specific sequence = of APIs to > ensure the service core is running before calling rte_service_map_lcore_s= et. Good point - I'm thinking about this - but haven't come to an obvious concl= usion yet. I'm considering other ways to detect the core is/isn't running, and also co= nsidering just "high-jacking" the service function pointer temporarily with a CAS, wh= ich gives some new options on avoiding threads entering the critical section. As above, I don't have a good solution yet.