From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B1757A0350; Tue, 30 Jun 2020 14:07:42 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 28F0B1BF46; Tue, 30 Jun 2020 14:07:42 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id 82EFA1B75C for ; Tue, 30 Jun 2020 14:07:40 +0200 (CEST) IronPort-SDR: skluj+NiNxkyQTx9FYAyutwps3vFYSOl2vsI3IXpVMdx8ybOicEyF2mr/6J9MRh0+UWncTuR5O OrmYtkjjZ59Q== X-IronPort-AV: E=McAfee;i="6000,8403,9666"; a="211278427" X-IronPort-AV: E=Sophos;i="5.75,297,1589266800"; d="scan'208";a="211278427" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2020 05:07:39 -0700 IronPort-SDR: VKzATPusgb0nMglXJaXzK7Nu8Cdg7sOKHvrb1Zf4x+HXWVMxftPnOnl0NSU2LnJeCQAd0vWqnz v8gGvSB84cQA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,297,1589266800"; d="scan'208";a="312345923" Received: from orsmsx108.amr.corp.intel.com ([10.22.240.6]) by orsmga008.jf.intel.com with ESMTP; 30 Jun 2020 05:07:39 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX108.amr.corp.intel.com (10.22.240.6) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 30 Jun 2020 05:07:38 -0700 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 30 Jun 2020 05:07:38 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 30 Jun 2020 05:07:38 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.109) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 30 Jun 2020 05:07:37 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AtDGePgThKi7/Tom6pJk7aC0MYHwgTjh4pRe0P1NbxtgPThtymGwgAaCQ3ekYwWHAXbEs0TNWFIaw45gD6n1UMUZ/EZM/S/AO8GNPVNA11QrA4mvUuFUqxEVn1hFiponD4bH+Yq1uMORAhdKG3ZRGa0crPWqAPJ/KQFRrziWTTJYnYuPleXHPzh3vD1OxOrCICmOxFAn/IEGrMIM3Imz1dEnx4w7+3qBxAX42SW7qfc18Z+jZwU/AdUlW3Dag0KIopabbU3oH0DRooekUqjRgc2+lyjzOqvZrhIPANp6zaNwlQJseeUzh7zdcdKIHhi5qUBdixXaNqk+iLRtWRgRJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zNvBUa6PjfkXmiD5fv5RqzJ6kZalno/BS7oKsYIgv9g=; b=ISq6urWfC7mb3IT5riRLcI78rAtezTkA1R7vxpEZyjFor2dbIsLosoLNRodIRB0l91zAv8eVJOEOCV6P++7DnUR3aeYUEvj8srZbq11MdhLj1Q0qtwi02ujc50hwHmgwtES+JlR35ZN932u0QnVGeoSJd/VywO9AZRRMBd+WXFBa0lSvOjHWf2JCa84+bIxZzOe52R7vrpluc7joTT2d+ez9bvotmKKDKozlFmWLsC8thvDQlDQLfoStuDNzAL8moNxhCg8wK6aNbA1GHWhdYF0FOsbcenCJl6qr8USQOZd3Scq86obn6TP5CVI6Pl/crpdOHBqWG3KXJdj+fH1ZuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zNvBUa6PjfkXmiD5fv5RqzJ6kZalno/BS7oKsYIgv9g=; b=SPlBkqjOiaZUD0FLrtqPcZXv84tdrsBcMg5JJO+NEWf7XRPBcoYZCH2a3sg9XdRPJqVaNGlmS4O14Tu2MeimNNUxM0LNikk8Fvrmy8W2ndmoe76vvGcR1cAlvmtBAWz1ySk9u3Df8MUJy/iVqti2k91Xrktcgqmy1hKqfknftBw= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BYAPR11MB2902.namprd11.prod.outlook.com (2603:10b6:a03:83::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.24; Tue, 30 Jun 2020 12:07:32 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f160:29ab:b8f9:4189]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f160:29ab:b8f9:4189%6]) with mapi id 15.20.3131.027; Tue, 30 Jun 2020 12:07:32 +0000 From: "Ananyev, Konstantin" To: Thomas Monjalon , David Marchand CC: "dev@dpdk.org" , "jerinjacobk@gmail.com" , "Richardson, Bruce" , "mdr@ashroe.eu" , "ktraynor@redhat.com" , "Stokes, Ian" , "i.maximets@ovn.org" , "Mcnamara, John" , "Kovacevic, Marko" , "Burakov, Anatoly" , Olivier Matz , "Andrew Rybchenko" , Neil Horman Thread-Topic: [dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores Thread-Index: AQHWSJjc02miHDhvs0aJZlTx+B7uGKjkxSPggAEPRACAAFFvIIABW1WAgAAPV/CAAAhKgIAAEmgQgANT3wCABgPxAIAADX0Q Date: Tue, 30 Jun 2020 12:07:32 +0000 Message-ID: References: <20200610144506.30505-1-david.marchand@redhat.com> <2939263.AvGHZF5Fiy@thomas> In-Reply-To: <2939263.AvGHZF5Fiy@thomas> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.151.182] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2470510f-8fa9-4350-07d5-08d81cee2b49 x-ms-traffictypediagnostic: BYAPR11MB2902: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0450A714CB x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3ogyJSylgOnn1VDx4MCCkkA8odDwUKJt7PfT3jENmG27uvUg9nbdod1ALpLCaYvQKRC1vnAooSHxuM3Ub7trOehtgJjUNR1KuZRj0JcqNklELNpkGhTVQ+XnZPiiPG11vGIwY1mkjfhYvZxCacE3Jbg6V0K6uiOaTqEsbujZ/P0s1oOSatmZrxwk4cEz6gC7xInqm/jnKj7gTfCDhRaIeCmQeBkjBufrNIoov5u4nE/f+Nj8bTwwiRrKnvezQ7fvyVAtiutREZkNcq9qeh2imBtY+9M0bPzwbo/cf8x+NkfyqseiH1zCuVmE4EJlmeMzyG0BQDY6KfFr72nvr5ehTTwfk8x7oycxOaX6LdmK7d/RIw+/SKgXPBmp4H0+7f2yI3ibfdqNxeo0ZvMvEkURPQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(346002)(366004)(376002)(396003)(2906002)(71200400001)(64756008)(66556008)(66476007)(76116006)(66946007)(66446008)(7416002)(8936002)(8676002)(4326008)(7696005)(966005)(186003)(478600001)(26005)(6506007)(53546011)(316002)(54906003)(110136005)(52536014)(83380400001)(33656002)(86362001)(9686003)(5660300002)(55016002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: OuZz+dWc/uPISI8NWS9QPgN1K/unsIVDU1oGaTlwg5ENng5uZ37furivy3u6qPhLuFePzGH4En20yfscAoVkfkY4HIc8fTs4xD7HlJ8DIVLyMNz9360lEcPB+YhhHI4tBG+G6eQSX7hHFaVvgwdCvlmhlB1k7lF5h7vLohAIJhc5P99FXSuZzhMk9/Nke76MNNcJgqeg5XBfhV3pgN7WwBIBtygE+ujAPqxXEjveUzA4dVWdmybFjhS6LYePvNiBX5nxfDY52DBzna1q0tDMDv8jPDX4/NzZJF07ii/BlcbmvuxVHEXo1Chu5fSObvHEijOAiWdAgm0CqlExVyvB+xT4CN6c4Z0d3ZnVCn1anJM1gGuxxGYIOKQesR1BbzYjPyuZazcDp9lWSZVq9ZHX1XmFRn2D2Qv1rA4RBKtU6aAstHPIlIOVfEPenQm/5qrUc6QjcXBMfEwNjRwCUhpAbS0zt/V7OZUQTCgDs/wSaZtmz0oBsAyaudHE2MZ6EFkG Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3301.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2470510f-8fa9-4350-07d5-08d81cee2b49 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jun 2020 12:07:32.0469 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DRQKspUDJHQHUpqhEgdM6NQXhkAFjxp5XxnPTBAti0jgfLknGJTMrkohpyosmt583nQ25HaMPfvZPhGzCTUMpelrJ6bH7jY63rNcq7aFBok= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2902 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > 26/06/2020 16:43, David Marchand: > > On Wed, Jun 24, 2020 at 1:59 PM Ananyev, Konstantin > > wrote: > > > > > Do you mean - make this new dynamic-lcore API return an error if = callied > > > > > from secondary process? > > > > > > > > Yes, and prohibiting from attaching a secondary process if dynamic > > > > lcore API has been used in primary. > > > > I intend to squash in patch 6: > > > > https://github.com/david-marchand/dpdk/commit/e5861ee734bfe2e4dc23d= 9b919b0db2a32a58aee > > > > > > But secondary process can attach before lcore_register, so we'll have= some sort of inconsistency in behaviour. > > > > If the developer tries to use both features, he gets an ERROR log in > > the two init path. > > So whatever the order at runtime, we inform the developer (who did not > > read/understand the rte_thread_register() documentation) that what he > > is doing is unsupported. >=20 > I agree. > Before this patch, pinning a thread on a random core can > trigger some issues. > After this patch, register an external thread will > take care of logging errors in case of inconsistencies. > So the user will know he is doing something not supported > by the app. I understand that, and return a meaningful error is definitely better the silent crash or memory corruption. The problem with that approach, as I said before, MP group behaviour becomes non-deterministic.=20 >=20 > It is an nice improvement. >=20 > > > If we really want to go ahead with such workaround - >=20 > It is not a workaround. > It is fixing some old issues and making clear what is really impossible. The root cause of the problem is in our MP model design decisions: from one side we treat lcore_id as process local data, from other side in some shared data-structures we use lcore_id as an index. I think to fix it properly we need either:=20 make lcore_id data shared or stop using lcore_id as an index for shared dat= a.=20 So from my perspective this approach is just one of possible workarounds. BTW, there is nothing wrong to have a workaround for the problem we are not ready to fix right now. =20 > > > probably better to introduce explicit EAL flag ( --single-process or = so). > > > As Thomas and Bruce suggested, if I understood them properly. >=20 > No I was thinking to maintain the tri-state information: > - secondary is possible > - secondary is attached > - secondary is forbidden Ok, then I misunderstood you. =20 > Asking the user to use an option to forbid attaching a secondary process > is the same as telling him it is forbidden. I don't think it is the same. On a live and complex system user can't always predict will the primary pro= c=20 use dynamic lcore and if it will at what particular moment. Same for secondary process launching - user might never start it, might start it straight after the primary one, or might be after several hours.=20 > The error log is enough in my opinion. I think it is better than nothing, but probably not the best one. Apart from possible non-consistent behaviour, it is quite restrictive: dynamic lcore_id wouldn't be available on any DPDK MP deployment. Which is a pity - I think it is a cool and useful feature. =20 What do you guys think about different approach: introduce new optional EAL parameter to restrict lcore_id values available for the process. #let say to start primary proc that can use lcore_id=3D[0-99] only: dpdk_primary --lcore-allow=3D0-99 ... --file-prefix=3Dxz1 #to start secondary one for it with allowed lcore_id=3D[100-109]: dpdk_secondary --lcore-allow=3D100-109 ... --file-prefix=3Dxz1 --proc-type= =3Dsecondary =20 =20 It is still a workaround, but that way we don't need to add any new limitations for dynamic lcores and secondary process usage.=20 Now it is up to user to decide would multiple-process use the same shared d= ata and if so - split lcore_id space properly among them (same as he has to do now with static lcores). > > A EAL flag is a stable API from the start, as there is nothing > > describing how we can remove one. > > So a new EAL flag for an experimental API/feature seems contradictory. > > > > Going with a new features status API... I think it is beyond this serie= s. > > > > Thomas seems to suggest an automatic resolution when features conflict > > happens.. ? >=20 > I suggest allowing the maximum and raise an error when usage conflicts. > It seems this is what you did in v4. >=20 > > I'll send the v4, let's discuss it there if you want. >=20