From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6B321A0350; Tue, 30 Jun 2020 21:03:21 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0EAE81BEB3; Tue, 30 Jun 2020 21:03:21 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id C57581BEA9 for ; Tue, 30 Jun 2020 21:03:18 +0200 (CEST) IronPort-SDR: NTvFsMuWPvFRQdxzEtd8xtR+JMyXhO60XVuieLUwucmAcwSM/jTpJvSmsaM+RrYXJt4b9EHRoo QfSRCPQFHnrQ== X-IronPort-AV: E=McAfee;i="6000,8403,9668"; a="125987062" X-IronPort-AV: E=Sophos;i="5.75,298,1589266800"; d="scan'208";a="125987062" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2020 12:02:57 -0700 IronPort-SDR: IwG0uZ2riTfiYNkrrslr0Hk3db9jpr5i1663fuUW1lvOq2Jfq+R7yPfXdv7drpbgK/Ht/+U6cb MYoHLYAQSv/Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,298,1589266800"; d="scan'208";a="295307621" Received: from orsmsx104.amr.corp.intel.com ([10.22.225.131]) by orsmga002.jf.intel.com with ESMTP; 30 Jun 2020 12:02:57 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by ORSMSX104.amr.corp.intel.com (10.22.225.131) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 30 Jun 2020 12:02:57 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.170) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 30 Jun 2020 12:02:57 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l9ZW+A1/UlcXsTfS8MNLkE3IlzqwDmzcnsx1fmhmWeKeLhe9EhiXRm6xDq7+PvolCpdxJVQ2HSUUqsRpH/aCH668s1dIarzLmUiLIrnXlosQYcJSzhTkzQ3zf45lJdddKyrI1o+/RAT756gVUsZlHzwMxxQ1kMlv1DHRucDOtK50oscM3A7k4sJwtamACceRD3BpEu423CNyOIi6bmZNQV8s6zMUsYn573CXv2BR66GEre7j9y76xjRHpTWt5xehYTCQ1te9IC4cFGRp9MFBwGhBHVaZ9Bz1FTtrEcOuALPXd+lDxshOddAJ9/5UA+70m7s28Wde/5LG+HbzzIciLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ohKWH+xSwD8RjJdMvO/Zqo4YNYAz0GAvgmHn28p70Jw=; b=DH780rustsxB++bDuaoza5iKI6PLTtHuGCQjsNrCMTYJ38Ol0feMJwxi2iC+Z0e2r7pHw3MwpUZrhtWhxCUF7/GoqteFhcbya/+jYoXs132cbZhUvB9g7SEJHH1BnMSM1jPx/cgvsxaP2vQybbosRbmcKOmO0N4uH/uluL7dM4N4dxzudS4bWmOCxjiBhdzwL7dxHejDrch23jvm1lF5K8YoOOkZSiX/S4Xs0Yh5VUDa4xHHhK7fLDCUB9rzCZi6XnXhflh9PoAP/nTAq8JeoyXY7gvkKq14+yvLUpHkmtNlTAAE2bhB69NtznvcOG+zU4Or7GbKPZUJkKoGjjDEGQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ohKWH+xSwD8RjJdMvO/Zqo4YNYAz0GAvgmHn28p70Jw=; b=vgNqFOnQ3pYpuD37EUJuebQAX1K8O6EpOk8mpNYmfzSnqGOqN7XkFNw9iP2Ri38WLom/WchbNbjZ3AQn6XpCjuaiZ/YKttQX/osMQrqqxVECefpF5HO3isyxnq014NQ/ayOpaAslDaZ3gh7Gbv+DpubwlIqkTsf8C6m3sq34Vkg= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BYAPR11MB3223.namprd11.prod.outlook.com (2603:10b6:a03:1b::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.26; Tue, 30 Jun 2020 19:02:55 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f160:29ab:b8f9:4189]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f160:29ab:b8f9:4189%6]) with mapi id 15.20.3131.027; Tue, 30 Jun 2020 19:02:55 +0000 From: "Ananyev, Konstantin" To: Olivier Matz CC: Thomas Monjalon , David Marchand , "dev@dpdk.org" , "jerinjacobk@gmail.com" , "Richardson, Bruce" , "mdr@ashroe.eu" , "ktraynor@redhat.com" , "Stokes, Ian" , "i.maximets@ovn.org" , "Mcnamara, John" , "Kovacevic, Marko" , "Burakov, Anatoly" , Andrew Rybchenko , Neil Horman Thread-Topic: [dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores Thread-Index: AQHWSJjc02miHDhvs0aJZlTx+B7uGKjkxSPggAEPRACAAFFvIIABW1WAgAAPV/CAAAhKgIAAEmgQgANT3wCABgPxAIAADX0QgAAWmYCAAA5woA== Date: Tue, 30 Jun 2020 19:02:55 +0000 Message-ID: References: <20200610144506.30505-1-david.marchand@redhat.com> <2939263.AvGHZF5Fiy@thomas> <20200630124409.GL5869@platinum> In-Reply-To: <20200630124409.GL5869@platinum> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: 6wind.com; dkim=none (message not signed) header.d=none;6wind.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.151.182] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 14730a56-7978-4a7f-447e-08d81d2832a6 x-ms-traffictypediagnostic: BYAPR11MB3223: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0450A714CB x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: w8v+0OY8JI5czYa9Vfinl62Mknja5IyXlpgam8pTy8/CsOWfd07f78Lu5bDiqcLSAf/+1f0n1QgcOdaOvlBBH5FCM9grG42l9wvr41zlGsEgmn0RLsR+mIBaKv5azWGER3pE9FjXNp+iP738kJqvuqLi64k+a9sBdLrykJKoCMs46gHwbNcMAttN0rl8si67nGZIH600nWjGAeUUMa7SkrYdNxBfUmHfoUrqStf+2nUyEELWsVtbVdvS663RSzb7Xba9KG7Jgxl+wqq58XxoIgycWQ8k5MHTM5ISQgMj6NJy9SzmsgMGWeHesMDq5sHWjKStnv/4kPK+/GcN6AMAYBPWRuDjbbrEezrd+j1LXWoaAQchEL01o1i9ldzcYTxICMmaYU86HiGnQlifcYR6Yw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(376002)(346002)(366004)(396003)(136003)(4326008)(6916009)(478600001)(7696005)(54906003)(186003)(316002)(52536014)(966005)(26005)(6506007)(5660300002)(33656002)(9686003)(7416002)(83380400001)(8936002)(55016002)(86362001)(8676002)(2906002)(76116006)(66556008)(66476007)(71200400001)(53546011)(66446008)(66946007)(64756008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: 0fghNLUWVNlG+S97lwSvzqkbfim1/JuZkQgthjlT96vuPQHd0mY7o8BWJGaUrQDuCQ3/7hU/6n7VZkg6xEpANzalwzXsfK8r9XRCBjsLnVvRwCOx4hJKl95QCWjOOS+wCAC/yC+Uybp5kvZGCET36gz3d3aM8uAoYKvev7DHXdjMa0G4Z+/ybHRjfIvUz+9wt/BCO29LYTiZhlS86jMut+DcD4gERCkYrt73QTuA7oENVDlIv6DyOTHVye1pfESuAlYoVHhUdBSqJe51L8RudorOpEOp3vGZIgOssJRDq8sdfmSBGLaYsxfElC7f1RuMQfQvHNBllHHfa8MPAZXZzLIOhQkJafTnHezDurwwXoLL1udYbxINZLvcXUxTyfyJyHUycZSKMAHGcOzRl5wHZjIXULK/HVhQHGPjK+7f+smRGQj85VITeaY7cd4+7JVLgIWe3a0sVaoyZ1QerVGToSXPz9Vo0wDTxQUy8LEFnBd6V8mWT6QTq7+0qkn065ns Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3301.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14730a56-7978-4a7f-447e-08d81d2832a6 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jun 2020 19:02:55.7370 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: SK1Sqw7Om8dnnWuVzJORMyGt2zcKwN9ey6igDac+E2gCsAWsg8jhSYvyMLC33eBXUR7GKCsX2PzxU734wgaHWinCyOe0kdJtGropBkRr2fk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3223 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 6/9] eal: register non-EAL threads as lcores X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > On Tue, Jun 30, 2020 at 12:07:32PM +0000, Ananyev, Konstantin wrote: > > > > > > 26/06/2020 16:43, David Marchand: > > > > On Wed, Jun 24, 2020 at 1:59 PM Ananyev, Konstantin > > > > wrote: > > > > > > > Do you mean - make this new dynamic-lcore API return an error= if callied > > > > > > > from secondary process? > > > > > > > > > > > > Yes, and prohibiting from attaching a secondary process if dyna= mic > > > > > > lcore API has been used in primary. > > > > > > I intend to squash in patch 6: > > > > > > https://github.com/david-marchand/dpdk/commit/e5861ee734bfe2e4d= c23d9b919b0db2a32a58aee > > > > > > > > > > But secondary process can attach before lcore_register, so we'll = have some sort of inconsistency in behaviour. > > > > > > > > If the developer tries to use both features, he gets an ERROR log i= n > > > > the two init path. > > > > So whatever the order at runtime, we inform the developer (who did = not > > > > read/understand the rte_thread_register() documentation) that what = he > > > > is doing is unsupported. > > > > > > I agree. > > > Before this patch, pinning a thread on a random core can > > > trigger some issues. > > > After this patch, register an external thread will > > > take care of logging errors in case of inconsistencies. > > > So the user will know he is doing something not supported > > > by the app. > > > > I understand that, and return a meaningful error is definitely > > better the silent crash or memory corruption. > > The problem with that approach, as I said before, MP group > > behaviour becomes non-deterministic. > > > > > > > > It is an nice improvement. > > > > > > > > If we really want to go ahead with such workaround - > > > > > > It is not a workaround. > > > It is fixing some old issues and making clear what is really impossib= le. > > > > The root cause of the problem is in our MP model design decisions: > > from one side we treat lcore_id as process local data, from other side > > in some shared data-structures we use lcore_id as an index. > > I think to fix it properly we need either: > > make lcore_id data shared or stop using lcore_id as an index for shared= data. > > So from my perspective this approach is just one of possible workaround= s. > > BTW, there is nothing wrong to have a workaround for the problem > > we are not ready to fix right now. > > > > > > > probably better to introduce explicit EAL flag ( --single-process= or so). > > > > > As Thomas and Bruce suggested, if I understood them properly. > > > > > > No I was thinking to maintain the tri-state information: > > > - secondary is possible > > > - secondary is attached > > > - secondary is forbidden > > > > Ok, then I misunderstood you. > > > > > Asking the user to use an option to forbid attaching a secondary proc= ess > > > is the same as telling him it is forbidden. > > > > I don't think it is the same. > > On a live and complex system user can't always predict will the primary= proc > > use dynamic lcore and if it will at what particular moment. > > Same for secondary process launching - user might never start it, > > might start it straight after the primary one, > > or might be after several hours. > > > > > The error log is enough in my opinion. > > > > I think it is better than nothing, but probably not the best one. > > Apart from possible non-consistent behaviour, it is quite restrictive: > > dynamic lcore_id wouldn't be available on any DPDK MP deployment. > > Which is a pity - I think it is a cool and useful feature. > > > > What do you guys think about different approach: > > introduce new optional EAL parameter to restrict lcore_id > > values available for the process. > > > > #let say to start primary proc that can use lcore_id=3D[0-99] only: > > dpdk_primary --lcore-allow=3D0-99 ... --file-prefix=3Dxz1 > > > > #to start secondary one for it with allowed lcore_id=3D[100-109]: > > dpdk_secondary --lcore-allow=3D100-109 ... --file-prefix=3Dxz1 --proc-t= ype=3Dsecondary > > > > It is still a workaround, but that way we don't need to > > add any new limitations for dynamic lcores and secondary process usage. > > Now it is up to user to decide would multiple-process use the same shar= ed data > > and if so - split lcore_id space properly among them > > (same as he has to do now with static lcores). >=20 > A variant (more simple) of your approach could be to add > "--proc-type=3Dstandalone" to explicitly disable MP and enable dynamic th= read > registration. >=20 For me it is a bit too restrictive, but yes it is a possible option, and from my perspective - a better one then disabling secondary proc suppor= t on the fly.=20 I tried to summarize different options under discussion in another mail of = this thread. Please have a look.=20 >=20 > > > > A EAL flag is a stable API from the start, as there is nothing > > > > describing how we can remove one. > > > > So a new EAL flag for an experimental API/feature seems contradicto= ry. > > > > > > > > Going with a new features status API... I think it is beyond this s= eries. > > > > > > > > Thomas seems to suggest an automatic resolution when features confl= ict > > > > happens.. ? > > > > > > I suggest allowing the maximum and raise an error when usage conflict= s. > > > It seems this is what you did in v4. > > > > > > > I'll send the v4, let's discuss it there if you want. > > > > >