From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 84D32A04B1; Wed, 9 Sep 2020 04:26:37 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CC9DD1BE0C; Wed, 9 Sep 2020 04:26:36 +0200 (CEST) Received: from nat-hk.nvidia.com (nat-hk.nvidia.com [203.18.50.4]) by dpdk.org (Postfix) with ESMTP id 55CFA1B9B7 for ; Wed, 9 Sep 2020 04:26:35 +0200 (CEST) Received: from hkpgpgate101.nvidia.com (Not Verified[10.18.92.100]) by nat-hk.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Wed, 09 Sep 2020 10:26:33 +0800 Received: from HKMAIL102.nvidia.com ([10.18.16.11]) by hkpgpgate101.nvidia.com (PGP Universal service); Tue, 08 Sep 2020 19:26:33 -0700 X-PGP-Universal: processed; by hkpgpgate101.nvidia.com on Tue, 08 Sep 2020 19:26:33 -0700 Received: from HKMAIL104.nvidia.com (10.18.16.13) by HKMAIL102.nvidia.com (10.18.16.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 9 Sep 2020 02:26:25 +0000 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.171) by HKMAIL104.nvidia.com (10.18.16.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Wed, 9 Sep 2020 02:26:25 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=flWIHedAtY06xS3D5KLqEiCfYuq2GXxZY1E4YnBpEypgcii4nYBe/IrbY8DkXXC93NGCD2wenP7rMlmgeuUY+o5TVHumGJQmis0VCNMz5j5eQV6HBwifdQyh67cjUmfqLB83H9yX5aP6k2A46WIPPOj/nLw2Bakq8HSrHVae9dtJBBkp+J7zvdKYblvsS+UXT9YXTqfpkFYpawY212cdAuwAGfN96Z9hwgHQqz+xahNv4Q6MDkz6wtHkvwIVoy6oMmzAp0LxdYmGvNiqyII2u7BAQwZfFEbzrEF50CeaEX2DDZH6hdRz6eqC1wKbbp4MZEE18y+TPJ6gAWDrUrNY3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ayUf+y8oKnm1IUbwT743eSWj6i+0pIoCWtCEVu4vD4c=; b=OfTtjvYcUCjpTOIopMXvfHHlqf1pCRfLxgXVnfey7nvrBG+DuBHaa0WIBUPm+tFDOAxtWseFfXYScYjZKl3aq99lXtXn2FmgY4R48dEaZQPmI88ptX4NussDTXt+bPHRVaYIhZFdY6o/sOFvxBkJCVmijKq+uqLVx129XkFcnLlDTXqhvuXJ9ziUuJq7oEFzrPF/nrw9eFa6oPjidj9oM5RhQx4f+noaBFUvYxJ1BmwJQ1kKsLb09pH6A+rDndGRBjg0i1ksKj0zSMQkBrTR9cVfPXPKgFVx+utZekC9wbzugHOSAWVDTphZNEVlr1csVlnpgfLmcPj/8sr0zu05+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MN2PR12MB3550.namprd12.prod.outlook.com (2603:10b6:208:108::22) by MN2PR12MB3040.namprd12.prod.outlook.com (2603:10b6:208:ce::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3370.16; Wed, 9 Sep 2020 02:26:22 +0000 Received: from MN2PR12MB3550.namprd12.prod.outlook.com ([fe80::b514:9492:7a05:75bf]) by MN2PR12MB3550.namprd12.prod.outlook.com ([fe80::b514:9492:7a05:75bf%4]) with mapi id 15.20.3348.019; Wed, 9 Sep 2020 02:26:22 +0000 From: Suanming Mou To: Stephen Hemminger , "NBU-Contact-Thomas Monjalon" CC: Ori Kam , Matan Azrad , "Shahaf Shuler" , Viacheslav Ovsiienko , Ferruh Yigit , "Andrew Rybchenko" , "dev@dpdk.org" , "joyce.kong@arm.com" , "phil.yang@arm.com" , "steve.capper@arm.com" , "honnappa.nagarahalli@arm.com" Thread-Topic: [dpdk-dev] [RFC] ethdev: make rte flow API thread safe Thread-Index: AQHWghj/PKRqPjtRmE+dlUFVZ4B+AalcdyOQgAJj6gCAAANJgIAAEHYAgACeeNA= Date: Wed, 9 Sep 2020 02:26:22 +0000 Message-ID: References: <1599108782-230624-1-git-send-email-suanmingm@nvidia.com> <20200908075208.048bfa02@hermes.lan> <3402903.UWDKPdykxe@thomas> <20200908090248.7a78888a@hermes.lan> In-Reply-To: <20200908090248.7a78888a@hermes.lan> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: networkplumber.org; dkim=none (message not signed) header.d=none;networkplumber.org; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [60.176.163.47] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ed7d48b0-1eeb-4f21-848d-08d85467be68 x-ms-traffictypediagnostic: MN2PR12MB3040: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-transport-forked: True x-ms-oob-tlc-oobclassifiers: OLM:7691; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: d5CLklycFYjw+wkz7+jaH/QOfXDakC0MbBBQDiXVv333i85Ak3l9m5ruet55Gz2i/BvSHBbJ7P7nnjcT4fG56GegunVbQ2HWl8sfsPJmFabOUs9gUPw5sLDLO4SrZiWNKK6qVH2YmV5U9zHd2nnUhoAEigK/z3ueM9YDFaTn/khmkq1tvqUwURMVPWck+swXd2dr6yqNCjcJhTn4DF92VP4FMjfkr0ZJ95739Nau3Tf7MWcUa/vXbYV8uYBFRd/0AgrTX4VRMD/OVG1aCy5w3ZXSqIm8lFTu73RxlY+yoQTASt/kjyiYtPRjbUbv61zyn6+qSXaw0I3KNZKKGTg3GA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB3550.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(366004)(39860400002)(136003)(396003)(346002)(55016002)(8676002)(5660300002)(9686003)(66476007)(66556008)(64756008)(66446008)(66946007)(2906002)(186003)(8936002)(76116006)(54906003)(86362001)(52536014)(478600001)(110136005)(316002)(53546011)(6506007)(26005)(83380400001)(71200400001)(7696005)(4326008)(33656002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: otQ4ZH+QIsVGwvsQm1WwYHIIOUd/N42zba26zVHZssWghH1AlNTYzHEImJYQDFqoMVH6wji1eeQgqyyLc0wsEaQmkc4dEVpa0kEz3hEPdiJwNKGX58YXximsFTdX4aGvYGispuahpyKbUa7OVSJqLjaDXpXNZ282OG2buyEl/K42fpMRjWuthHlKIcYytw7XP+ItnMdMqaMRj5Z3ie5GUHRdXZI1ZKM6UsUYuYr3mejKIN9EQeCK1k3dIIFDzbkUOJ1jmb5eAllGYsVhbUHB2JhnekJqG/OMBZnh9stUH8qNQvJPnSY1iIpT+wHS4JcNu/f2aBGMCRv3RReglyYPDXJqezC11Pn8Fe+yXx3OayiUuiW4ism0zJNSj1/Tj//V2CuAv3kzK71W5SEQ6KBrZnRsMC1EnFu7/AkWG4LvpivEPDBRKykUMchcVJdGHZhVPXBfEFKv8YX59a0q/p/tMLuW/7pUiXrmz669FI7BpiGD3g5T4WOK5T8DlkqALfJn44J9tZyYFh5qiGpjJVQTQKk9UuoMwaBLJ9qAl0MCpDflQuOgTRKgliMDElVNVgH4IftGw8A3BIc2tWOvM5+xcDxKzn9BIeRCnn50MxTf//qQTd/iqHsUFIYU66WStjZGngvgUI7VLInjqkVHlkriag== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB3550.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: ed7d48b0-1eeb-4f21-848d-08d85467be68 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2020 02:26:22.3761 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: AepY2EFhVqJykNlvDLKOKWRZlBe5ssXaCHsoiPhGKkraukf/XtFAI5VvtAKX35noR1eTlYsDI5EbkfDf8DINCQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3040 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1599618393; bh=ayUf+y8oKnm1IUbwT743eSWj6i+0pIoCWtCEVu4vD4c=; h=X-PGP-Universal:ARC-Seal:ARC-Message-Signature: ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic: Thread-Index:Date:Message-ID:References:In-Reply-To: Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-microsoft-antispam-prvs:x-ms-exchange-transport-forked: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=Gd7M/sGJe/GBvdth7S3vw0wjgK0Y1QgVsdQnhGOe9VGv5NvAwqXmYIPOjXON7NroU RZVnB0Z3J4Jb9Gk+J72J2XbR3WuIa9eABMej9WDx3kz7rQ9Ol25f4LGluNQuiM5Dxz 8IWku54YEiP9Bm0lHsfdZUa9ZBNqtdO5PAFNrnqjXlDh5pSOoaaNL/IGHrUJlLDYgC rXpQomHwUuOumfxu0bYtOtVuZyeIfTZZBEzWka0EpbHqF60XXbMAlNdqmiNn34Y9gw kgIESU1LWIY+mr5pos3x6mTVS+9LsVQBbuGu5nK/G+PMtt5jh69wtiP46qb++ZYNaU N72XvDuUyg2ug== Subject: Re: [dpdk-dev] [RFC] ethdev: make rte flow API thread safe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Stephen Hemminger > Sent: Wednesday, September 9, 2020 12:03 AM > To: NBU-Contact-Thomas Monjalon > Cc: Suanming Mou ; Ori Kam ; > Matan Azrad ; Shahaf Shuler ; > Viacheslav Ovsiienko ; Ferruh Yigit > ; Andrew Rybchenko ; > dev@dpdk.org; joyce.kong@arm.com; phil.yang@arm.com; > steve.capper@arm.com; honnappa.nagarahalli@arm.com > Subject: Re: [dpdk-dev] [RFC] ethdev: make rte flow API thread safe >=20 > On Tue, 08 Sep 2020 17:03:53 +0200 > Thomas Monjalon wrote: >=20 > > 08/09/2020 16:52, Stephen Hemminger: > > > On Mon, 7 Sep 2020 02:36:48 +0000 > > > Suanming Mou wrote: > > > > > What is the performance impact of this for currently working > > > > > applications that use a single thread to program flow rules. You= are > adding a couple of system > > > > > calls to what was formerly a totally usermode operation. > > > > > > Read the source for glibc and see what pthread_mutex does > > > > What would be the best lock for rte_flow? > > We have spin lock, ticket lock, MCS lock (and rwlock) in DPDK. >=20 > The tradeoff is between speed, correctness, and simplicity. > The flow case is performance sensitive (for connection per second tests);= but > not super critical (ie every packet). > Fastest would be RCU but probably not necessary here. >=20 > There would rarely be contention on this (thread safety is new), and ther= e is no > case where reader makes sense. For hardware, programming flow rules would > basic interaction with TCAM (ie fast). For software drivers, typically ma= king flow > rule requires system call to set classifier etc. Holding spin lock across= system > calls leads to preemption and other issues. Very good explanation, thank you Stephen. Totally agree with that. >=20 > Would it be possible to push the choice of mutual exclusion down to the d= evice > driver? For fast HW devices they could use spinlock and for slow SW devic= es it > would be pthread. That's a good hint. But that will also introduce the vendor PMD to update. = I tried to list some points here. 1. For single thread case, pthread mutex acts similar as spinlock, spinlock= or pthread mutex will not make difference here.=20 2. For multiple threads lock contention, spinlock will introduce more CPU u= sage. DPDK applications currently use mutex later get rid of the outer mute= x will suffer higher CPU usage with the spinlock(fast hardware PMD). And one more general question, can we find some DPDK applications now use s= pinlock with the rte flow APIs?