From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E7227A0540; Wed, 6 Jul 2022 22:21:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF53B4069D; Wed, 6 Jul 2022 22:21:31 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 844EC40691 for ; Wed, 6 Jul 2022 22:21:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657138889; x=1688674889; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=m0HZb33+Szc3oTeGc3rWY5F1zCD1S/qayl+QQi/XTto=; b=dzXvTVSi1E2atypelmz9WlApXXps8nc1pP6giIxYT0sjJSEAtFJLjH+T zl1s3LDJsYM7W3Neczbe6bBW5II+ZNkP/pvCMMIXBOEw38nrQhLyUprVe 0xj1+eUP7XThteGZA7hjny753Olsie4auRhqEjKq0+tz3Eylb4DcdyHlU uEbzoNSs8YDAnvVnakqWzkPjz+/zbfBotOhdOmug+Qt1qp54A+Xsqku2/ ju85+3Q4SeW7M5108xyCICNRJiYq8n/SZ4ab337k6MM4TkS1iFxvbT6tD KnU84Td0H+gJAAiZ5YgF2sALSiW1ebICC1iLKKYvs5KOI4rMik4z6KrPM g==; X-IronPort-AV: E=McAfee;i="6400,9594,10400"; a="370178888" X-IronPort-AV: E=Sophos;i="5.92,250,1650956400"; d="scan'208";a="370178888" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jul 2022 13:21:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,250,1650956400"; d="scan'208";a="543546211" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by orsmga003.jf.intel.com with ESMTP; 06 Jul 2022 13:21:27 -0700 Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Wed, 6 Jul 2022 13:21:27 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Wed, 6 Jul 2022 13:21:27 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.173) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Wed, 6 Jul 2022 13:21:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F8NK7EzV/X55kbYKrC83Yerm7Gg3sUv8j7gGIhPAaGTLUKluOM5h5LE7+t0R/vXbroWtzDbnLK4dGEnCeve4a4Nf7B13TGkdnO/ToTdP62d6teMNrclf1/sEKEXolTk+h5LKPl/Lr76xT3nmZXlu5UPx/44QLC74MRq2SZNTvacYAhT4vhqHfJNyUdPQ4iOJh0nslkpsj1MSntDfnLhs74S7cs4DKdhGKCJc/jxGc4xleLv525OSTGitzYWFt3G8c3zVnoE9XtWMXhJZ/wi3puofAcEYZ6WioDqrtcspsbkPIIAI9GslEi3zJqAfp2i/onDn3vr8DjqQTlx/i5vMxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WH5S4zBz5GH4TPFMSwRRYRxMmvDyvY01xf2C2dz57jY=; b=hiIpmopmk4DJLoFLs1JzKr9AWyw8QrAs6DNV9Sryrzx6sV4vHnXhbTjyrs3C30tfk/pvQqPT1l7wboKPODwptM8JEhBVH/tXyNHO52mTqzqRRc7xGNp8V0XbH6Ab38tCV5sESJ+2pjxHK2/PsH/eYE5IJfRniYG5LXBWTSp/PaeG05ePCr3RieHr2ElQOuixkv+TkaLykt+B6smI0pCP67U9CyH54NymDnUFy01rjbbFXh0bpoF+gKcU5ZsROxOkD0an7xkoU929XORf8vEIyH/KYjq+Dg05Du7Ns9MM1tw78UynC3Wciii4F1wyOqRomIkVPut6ozyJVn0fBirLTQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BY5PR11MB4451.namprd11.prod.outlook.com (2603:10b6:a03:1cb::30) by MN2PR11MB4647.namprd11.prod.outlook.com (2603:10b6:208:262::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.14; Wed, 6 Jul 2022 20:21:25 +0000 Received: from BY5PR11MB4451.namprd11.prod.outlook.com ([fe80::1df7:bcbb:7825:37fe]) by BY5PR11MB4451.namprd11.prod.outlook.com ([fe80::1df7:bcbb:7825:37fe%7]) with mapi id 15.20.5417.016; Wed, 6 Jul 2022 20:21:25 +0000 From: "Chautru, Nicolas" To: Stephen Hemminger , Tom Rix CC: "dev@dpdk.org" , "thomas@monjalon.net" , "gakhil@marvell.com" , "hemant.agrawal@nxp.com" , "maxime.coquelin@redhat.com" , "mdr@ashroe.eu" , "Richardson, Bruce" , "david.marchand@redhat.com" Subject: RE: [PATCH v4 7/7] bbdev: add a lock option for enqueue/dequeue operation Thread-Topic: [PATCH v4 7/7] bbdev: add a lock option for enqueue/dequeue operation Thread-Index: AQHYkNCjt8RQXYQl+kyo51wC8zHMBq1xs+uAgAAFcgCAAAkOUA== Date: Wed, 6 Jul 2022 20:21:25 +0000 Message-ID: References: <1655491040-183649-6-git-send-email-nicolas.chautru@intel.com> <1657067022-54373-1-git-send-email-nicolas.chautru@intel.com> <1657067022-54373-8-git-send-email-nicolas.chautru@intel.com> <25b4ece1-8f67-f119-7a0e-5b133f4e571c@redhat.com> <20220706122048.46555c19@hermes.local> In-Reply-To: <20220706122048.46555c19@hermes.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.500.17 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8b47543a-97c0-4178-a2f3-08da5f8d19b0 x-ms-traffictypediagnostic: MN2PR11MB4647:EE_ x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: RpcVBgBajU1sqToB80gjujhCT2uDo101pI3VApN/Zc/Kxj+zppOaEsz9kBNBywPf+qcyEqqcCcKyKMbvfnY5TaVe5s/QIzrR3c8UWiDSCs20AVFO/xgKIwejwr+GmvsN5MoUNHXolq8kgjQs607+P2fgSFon2n/uHH34PnyS2+FeLzDry1Qg4xTE4K79revRa/tFgp7+o/Y3XuCXsHCW6B0osjarLWbWrC8Gd97Fms4RTKy8VmZYr15enoPPkbA/XNrl91xgZK3L5R0F5jpFSnlVIDJ/BdMVx9buxhXUw5f+vt5WQxP8Twjpyh8xST32hVqz4E67LFN7S+q2a6nawHm7bSrMpWampeX6vEv2BuBM+3IJ5+mfjtunEvw7/9GLf+k8DcWTyWTTKjQJ4QXVI9HDQ23lFnjVZqc8tR/gDXTaIZ/tsP244UhvXmIaRGYb77w9UFyUJPW2xVAYL86AoFe49ZEPNI0KeuiH1LczRjLqKhIBEBiQ5gLPxb0EjQQwNZrGZ3/+o+VSqOVlACtAbn0yfcaS+dKjFmzkX7eo5pRB+EKAuOiLyEK5f6RJ0PxVEGqen8YOe6xH7laTsEFvzeokRyXI4Dx1TEK2mjMInMAi+EniiaF9c/bRbEwl+4gv2k0TKPo9HrEs+vjllCRw/AFR8bfeQrXQbJQoQrE7FTd+5+6rRdXeBaDWD/oMVPJnCu14LCx4nSd43nuckaRTquVtAs3Jnz66OOG3ZbKeSdvl1oc1qtEdc0EbTohngw/XL26QNEHDVZn0u1095ROk3AFPDhwoUnbmT8rm+zrQDB5QxDcn/AI1TPuXdpK+1K8P x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BY5PR11MB4451.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(366004)(39860400002)(396003)(346002)(376002)(136003)(7696005)(33656002)(66946007)(66476007)(66446008)(64756008)(8676002)(66556008)(9686003)(76116006)(26005)(55016003)(83380400001)(316002)(86362001)(6506007)(54906003)(71200400001)(53546011)(41300700001)(110136005)(4326008)(82960400001)(38100700002)(2906002)(5660300002)(122000001)(478600001)(38070700005)(186003)(8936002)(52536014); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?+j+8FRgnMW2UVSSzGHyneF7nMdzF/HWY9gyo73qTRW7hPzQfVmwHreSUs+lj?= =?us-ascii?Q?6Qhk2kmzpD5IRy5/tPWZG+Pd9DJPcF8CsauMZGTwgVBJ8r2rqO0AifpkbDRI?= =?us-ascii?Q?n7Aoed3n6MLYJ4zifnSA7CAklHVp7F/91s3fIGk/WN+uWyg9XiiCXvmVHLgJ?= =?us-ascii?Q?VTsWk7lBcmzLv5tFV0Sz2g+ujuB45fkDluuN0fXFX33YyhQDxAKmMoiAEkdr?= =?us-ascii?Q?Uj9Eo3tRBNK6GF/r8bKogf5dKEK34ux0EwWRnGFctW1T38s9TYUkfZ9d+DBi?= =?us-ascii?Q?1BS12ZkScQi6yV11iAwwearo1euPWu2uWXK47sqsukqu9CxjyCmac7wg55M8?= =?us-ascii?Q?I4SEfUyE/Xm1Wp6ULItdWeD48O693eQmCDl000vx7b+2AcPuZdZ4kH1iZQix?= =?us-ascii?Q?JulQJhxTDvU2M3G48QQ8qmtsfyOSVC+Y9pCYfL41BDt6kksOPA9Bk0jdxkmN?= =?us-ascii?Q?cRE2oU6YnPlVfCEW/1PMSZbVELyln2h8oSUrGXf0/DRlJ7R5sDJ9e3s/hLzV?= =?us-ascii?Q?BbMX4Y2v18on+uAYaKQ6naCsQ2Ny+rFgpPWdHSUuM3+nT1Ocs8oH5GJonU0U?= =?us-ascii?Q?1h5QsP2tYw2sUwvsAaZ9o5G6la0MhUmc7NBSYVER8UQFh41I7xSHfSMb+pZE?= =?us-ascii?Q?V5yCQrPeOyDyBXoH/xZBqUuJIXCxtYnchjGo19ORuK+PDN+kAop/UhBZhq3c?= =?us-ascii?Q?/aamSpllVW1E5frgrhN3Kx3t3pP7VB/BPszPEMseOgXTRdn89WTsofnjtM3P?= =?us-ascii?Q?UxWsZIaPAmdw6Tw1w7XvmTgORMthmH6QgUdNFZHXNn6dKZueq/X861z3npIO?= =?us-ascii?Q?swP5ZjttJKhLYHx8HeZd0JeAIBBNoM+oR6FytgTZUp0sKk3YciiqrGHnWFLi?= =?us-ascii?Q?e9AcoRLPmFGt+ZCN1ihKUwkWy8DlbxxFPUTGDnL6kuMfkpaLh37UO/dL+aF3?= =?us-ascii?Q?aZmfFj0GsJCgIRo7WGqRdtteUgDzeui6GqmUrNDMQyx/R4CpR+QAAhvF3Qdl?= =?us-ascii?Q?vriTnXtVWT7HxPY3sQ/s9UVvFGY8AA2gpKuwGpmL04STFuc2s/Tu9js9r1k/?= =?us-ascii?Q?B0aXrJdEb0gJZP+H3+fGIOYFP1X54bX50ZJ/ziHFGFJrRFck52z36ncJACTQ?= =?us-ascii?Q?G67TEedyJz+EdT6I1z9kkva2wB8+EHbvzoTf9ijGNHSgbwutIj/ex61uzYJ0?= =?us-ascii?Q?c1JtWFc9SKoQgBNigf9DTA0gs9LFeP2M0slEFqN8UtrDT34m1VReEMzLsK7+?= =?us-ascii?Q?cHLkFDRYg4ayoALS2VfuDZwyERjA0spxJpuAMY8RjhQSM2e7EQNSAubjIyVu?= =?us-ascii?Q?A9NsqwarlR0jnO94rqNuvNMu8dvvXma3+qM8gAt4RJczhzit/y9c10WY6rK1?= =?us-ascii?Q?Fv1A2fdg2knrgI3sj2iwQLSQKNaNI/Ly09phuKmOgaCAMrEFjK8C1MvyLy7Z?= =?us-ascii?Q?zv3cppxydfgaAWG2DvNA54l9pRPl1XRSjbT/HkddpcwpM8LCIQQ28HWsFL6z?= =?us-ascii?Q?Q9fwaNiiwqE9YecoCWW1rR53Psnh99nSXHL5t6AHK4JAK22ntxF9Z5qDWk9x?= =?us-ascii?Q?jvBeNq7VxvKrn5fMmY89rpBb5GFbHmibsZd8R710?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BY5PR11MB4451.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8b47543a-97c0-4178-a2f3-08da5f8d19b0 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Jul 2022 20:21:25.2224 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: yhgTMudTJ5b9JgpvK/4e9dm69XtvF/sEgJfM5RU+QSKy7NC17cI8m2dHEVXkChg+masyO/suKqZHt4AXWCFxj/obxgSQBcnhZN4YrwrAs88= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4647 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Stephen, Tom.,=20 > -----Original Message----- > From: Stephen Hemminger >=20 > On Wed, 6 Jul 2022 12:01:19 -0700 > Tom Rix wrote: >=20 > > On 7/5/22 5:23 PM, Nicolas Chautru wrote: > > > Locking is not explicitly required but can be valuable in case the > > > application cannot guarantee to be thread-safe, or specifically is > > > at risk of using the same queue from multiple threads. > > > This is an option for PMD to use this. > > > > > > Signed-off-by: Nicolas Chautru > > > --- > > > lib/bbdev/rte_bbdev.h | 2 ++ > > > 1 file changed, 2 insertions(+) > > > > > > diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h index > > > b7ecf94..8e7ca86 100644 > > > --- a/lib/bbdev/rte_bbdev.h > > > +++ b/lib/bbdev/rte_bbdev.h > > > @@ -407,6 +407,8 @@ struct rte_bbdev_queue_data { > > > struct rte_bbdev_stats queue_stats; /**< Queue statistics */ > > > enum rte_bbdev_enqueue_status enqueue_status; /**< Enqueue > status when op is rejected */ > > > bool started; /**< Queue state */ > > > + rte_rwlock_t lock_enq; /**< lock protection for the Enqueue */ > > > + rte_rwlock_t lock_deq; /**< lock protection for the Dequeue */ > > > > No. > > > > This is a good idea but needs a use before introducing another > > element, particularly a complicated one like locking > > > > Tom The actual usage would be implemented within the PMD. Basically this to pre= vent the corner case when a queue is being accessed from multiple thread fo= r which there is no protection in DPDK (but application does not necessaril= y behaves well).=20 In normal operation there would never be a case when there is a conflict on= the lock. This is not something which was considered for any other PMD? >From DPDK doc : "If multiple threads are to use the same hardware queue on = the same NIC port, then locking, or some other form of mutual exclusion, is= necessary." Basically for AC100 we would purely enforce the lock for any enqueue/dequeu= e operation for a given queue (distinct lock for enqueue and dequeue, since= these would run on different threads). =20 > Having two locks on same cacheline will create lots of ping/pong false sh= aring. You would recommend to purely spread them within the structure? Or somethin= g else?=20 =20 > Also, unless the reader is holding the lock for a significant fraction of= the time a > regular spin lock will be faster. OK Thanks. It should in principle never have to wait for the lock for the u= sage above, only to catch misbehaving application risk.=20 Nic