From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id E7227A0540;
Wed, 6 Jul 2022 22:21:31 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id CF53B4069D;
Wed, 6 Jul 2022 22:21:31 +0200 (CEST)
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
by mails.dpdk.org (Postfix) with ESMTP id 844EC40691
for <dev@dpdk.org>; Wed, 6 Jul 2022 22:21:29 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
d=intel.com; i=@intel.com; q=dns/txt; s=Intel;
t=1657138889; x=1688674889;
h=from:to:cc:subject:date:message-id:references:
in-reply-to:content-transfer-encoding:mime-version;
bh=m0HZb33+Szc3oTeGc3rWY5F1zCD1S/qayl+QQi/XTto=;
b=dzXvTVSi1E2atypelmz9WlApXXps8nc1pP6giIxYT0sjJSEAtFJLjH+T
zl1s3LDJsYM7W3Neczbe6bBW5II+ZNkP/pvCMMIXBOEw38nrQhLyUprVe
0xj1+eUP7XThteGZA7hjny753Olsie4auRhqEjKq0+tz3Eylb4DcdyHlU
uEbzoNSs8YDAnvVnakqWzkPjz+/zbfBotOhdOmug+Qt1qp54A+Xsqku2/
ju85+3Q4SeW7M5108xyCICNRJiYq8n/SZ4ab337k6MM4TkS1iFxvbT6tD
KnU84Td0H+gJAAiZ5YgF2sALSiW1ebICC1iLKKYvs5KOI4rMik4z6KrPM g==;
X-IronPort-AV: E=McAfee;i="6400,9594,10400"; a="370178888"
X-IronPort-AV: E=Sophos;i="5.92,250,1650956400"; d="scan'208";a="370178888"
Received: from orsmga003.jf.intel.com ([10.7.209.27])
by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
06 Jul 2022 13:21:28 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.92,250,1650956400"; d="scan'208";a="543546211"
Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84])
by orsmga003.jf.intel.com with ESMTP; 06 Jul 2022 13:21:27 -0700
Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by
fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2308.27; Wed, 6 Jul 2022 13:21:27 -0700
Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by
fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2308.27 via Frontend Transport; Wed, 6 Jul 2022 13:21:27 -0700
Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.173)
by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.1.2308.27; Wed, 6 Jul 2022 13:21:27 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=F8NK7EzV/X55kbYKrC83Yerm7Gg3sUv8j7gGIhPAaGTLUKluOM5h5LE7+t0R/vXbroWtzDbnLK4dGEnCeve4a4Nf7B13TGkdnO/ToTdP62d6teMNrclf1/sEKEXolTk+h5LKPl/Lr76xT3nmZXlu5UPx/44QLC74MRq2SZNTvacYAhT4vhqHfJNyUdPQ4iOJh0nslkpsj1MSntDfnLhs74S7cs4DKdhGKCJc/jxGc4xleLv525OSTGitzYWFt3G8c3zVnoE9XtWMXhJZ/wi3puofAcEYZ6WioDqrtcspsbkPIIAI9GslEi3zJqAfp2i/onDn3vr8DjqQTlx/i5vMxg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=WH5S4zBz5GH4TPFMSwRRYRxMmvDyvY01xf2C2dz57jY=;
b=hiIpmopmk4DJLoFLs1JzKr9AWyw8QrAs6DNV9Sryrzx6sV4vHnXhbTjyrs3C30tfk/pvQqPT1l7wboKPODwptM8JEhBVH/tXyNHO52mTqzqRRc7xGNp8V0XbH6Ab38tCV5sESJ+2pjxHK2/PsH/eYE5IJfRniYG5LXBWTSp/PaeG05ePCr3RieHr2ElQOuixkv+TkaLykt+B6smI0pCP67U9CyH54NymDnUFy01rjbbFXh0bpoF+gKcU5ZsROxOkD0an7xkoU929XORf8vEIyH/KYjq+Dg05Du7Ns9MM1tw78UynC3Wciii4F1wyOqRomIkVPut6ozyJVn0fBirLTQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
dkim=pass header.d=intel.com; arc=none
Received: from BY5PR11MB4451.namprd11.prod.outlook.com (2603:10b6:a03:1cb::30)
by MN2PR11MB4647.namprd11.prod.outlook.com (2603:10b6:208:262::10)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.14; Wed, 6 Jul
2022 20:21:25 +0000
Received: from BY5PR11MB4451.namprd11.prod.outlook.com
([fe80::1df7:bcbb:7825:37fe]) by BY5PR11MB4451.namprd11.prod.outlook.com
([fe80::1df7:bcbb:7825:37fe%7]) with mapi id 15.20.5417.016; Wed, 6 Jul 2022
20:21:25 +0000
From: "Chautru, Nicolas" <nicolas.chautru@intel.com>
To: Stephen Hemminger <stephen@networkplumber.org>, Tom Rix <trix@redhat.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, "thomas@monjalon.net"
<thomas@monjalon.net>, "gakhil@marvell.com" <gakhil@marvell.com>,
"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>, "mdr@ashroe.eu"
<mdr@ashroe.eu>, "Richardson, Bruce" <bruce.richardson@intel.com>,
"david.marchand@redhat.com" <david.marchand@redhat.com>
Subject: RE: [PATCH v4 7/7] bbdev: add a lock option for enqueue/dequeue
operation
Thread-Topic: [PATCH v4 7/7] bbdev: add a lock option for enqueue/dequeue
operation
Thread-Index: AQHYkNCjt8RQXYQl+kyo51wC8zHMBq1xs+uAgAAFcgCAAAkOUA==
Date: Wed, 6 Jul 2022 20:21:25 +0000
Message-ID: <BY5PR11MB4451D10A42F00D2DA6771403F8809@BY5PR11MB4451.namprd11.prod.outlook.com>
References: <1655491040-183649-6-git-send-email-nicolas.chautru@intel.com>
<1657067022-54373-1-git-send-email-nicolas.chautru@intel.com>
<1657067022-54373-8-git-send-email-nicolas.chautru@intel.com>
<25b4ece1-8f67-f119-7a0e-5b133f4e571c@redhat.com>
<20220706122048.46555c19@hermes.local>
In-Reply-To: <20220706122048.46555c19@hermes.local>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.6.500.17
authentication-results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=intel.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 8b47543a-97c0-4178-a2f3-08da5f8d19b0
x-ms-traffictypediagnostic: MN2PR11MB4647:EE_
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: RpcVBgBajU1sqToB80gjujhCT2uDo101pI3VApN/Zc/Kxj+zppOaEsz9kBNBywPf+qcyEqqcCcKyKMbvfnY5TaVe5s/QIzrR3c8UWiDSCs20AVFO/xgKIwejwr+GmvsN5MoUNHXolq8kgjQs607+P2fgSFon2n/uHH34PnyS2+FeLzDry1Qg4xTE4K79revRa/tFgp7+o/Y3XuCXsHCW6B0osjarLWbWrC8Gd97Fms4RTKy8VmZYr15enoPPkbA/XNrl91xgZK3L5R0F5jpFSnlVIDJ/BdMVx9buxhXUw5f+vt5WQxP8Twjpyh8xST32hVqz4E67LFN7S+q2a6nawHm7bSrMpWampeX6vEv2BuBM+3IJ5+mfjtunEvw7/9GLf+k8DcWTyWTTKjQJ4QXVI9HDQ23lFnjVZqc8tR/gDXTaIZ/tsP244UhvXmIaRGYb77w9UFyUJPW2xVAYL86AoFe49ZEPNI0KeuiH1LczRjLqKhIBEBiQ5gLPxb0EjQQwNZrGZ3/+o+VSqOVlACtAbn0yfcaS+dKjFmzkX7eo5pRB+EKAuOiLyEK5f6RJ0PxVEGqen8YOe6xH7laTsEFvzeokRyXI4Dx1TEK2mjMInMAi+EniiaF9c/bRbEwl+4gv2k0TKPo9HrEs+vjllCRw/AFR8bfeQrXQbJQoQrE7FTd+5+6rRdXeBaDWD/oMVPJnCu14LCx4nSd43nuckaRTquVtAs3Jnz66OOG3ZbKeSdvl1oc1qtEdc0EbTohngw/XL26QNEHDVZn0u1095ROk3AFPDhwoUnbmT8rm+zrQDB5QxDcn/AI1TPuXdpK+1K8P
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:BY5PR11MB4451.namprd11.prod.outlook.com; PTR:; CAT:NONE;
SFS:(13230016)(366004)(39860400002)(396003)(346002)(376002)(136003)(7696005)(33656002)(66946007)(66476007)(66446008)(64756008)(8676002)(66556008)(9686003)(76116006)(26005)(55016003)(83380400001)(316002)(86362001)(6506007)(54906003)(71200400001)(53546011)(41300700001)(110136005)(4326008)(82960400001)(38100700002)(2906002)(5660300002)(122000001)(478600001)(38070700005)(186003)(8936002)(52536014);
DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?+j+8FRgnMW2UVSSzGHyneF7nMdzF/HWY9gyo73qTRW7hPzQfVmwHreSUs+lj?=
=?us-ascii?Q?6Qhk2kmzpD5IRy5/tPWZG+Pd9DJPcF8CsauMZGTwgVBJ8r2rqO0AifpkbDRI?=
=?us-ascii?Q?n7Aoed3n6MLYJ4zifnSA7CAklHVp7F/91s3fIGk/WN+uWyg9XiiCXvmVHLgJ?=
=?us-ascii?Q?VTsWk7lBcmzLv5tFV0Sz2g+ujuB45fkDluuN0fXFX33YyhQDxAKmMoiAEkdr?=
=?us-ascii?Q?Uj9Eo3tRBNK6GF/r8bKogf5dKEK34ux0EwWRnGFctW1T38s9TYUkfZ9d+DBi?=
=?us-ascii?Q?1BS12ZkScQi6yV11iAwwearo1euPWu2uWXK47sqsukqu9CxjyCmac7wg55M8?=
=?us-ascii?Q?I4SEfUyE/Xm1Wp6ULItdWeD48O693eQmCDl000vx7b+2AcPuZdZ4kH1iZQix?=
=?us-ascii?Q?JulQJhxTDvU2M3G48QQ8qmtsfyOSVC+Y9pCYfL41BDt6kksOPA9Bk0jdxkmN?=
=?us-ascii?Q?cRE2oU6YnPlVfCEW/1PMSZbVELyln2h8oSUrGXf0/DRlJ7R5sDJ9e3s/hLzV?=
=?us-ascii?Q?BbMX4Y2v18on+uAYaKQ6naCsQ2Ny+rFgpPWdHSUuM3+nT1Ocs8oH5GJonU0U?=
=?us-ascii?Q?1h5QsP2tYw2sUwvsAaZ9o5G6la0MhUmc7NBSYVER8UQFh41I7xSHfSMb+pZE?=
=?us-ascii?Q?V5yCQrPeOyDyBXoH/xZBqUuJIXCxtYnchjGo19ORuK+PDN+kAop/UhBZhq3c?=
=?us-ascii?Q?/aamSpllVW1E5frgrhN3Kx3t3pP7VB/BPszPEMseOgXTRdn89WTsofnjtM3P?=
=?us-ascii?Q?UxWsZIaPAmdw6Tw1w7XvmTgORMthmH6QgUdNFZHXNn6dKZueq/X861z3npIO?=
=?us-ascii?Q?swP5ZjttJKhLYHx8HeZd0JeAIBBNoM+oR6FytgTZUp0sKk3YciiqrGHnWFLi?=
=?us-ascii?Q?e9AcoRLPmFGt+ZCN1ihKUwkWy8DlbxxFPUTGDnL6kuMfkpaLh37UO/dL+aF3?=
=?us-ascii?Q?aZmfFj0GsJCgIRo7WGqRdtteUgDzeui6GqmUrNDMQyx/R4CpR+QAAhvF3Qdl?=
=?us-ascii?Q?vriTnXtVWT7HxPY3sQ/s9UVvFGY8AA2gpKuwGpmL04STFuc2s/Tu9js9r1k/?=
=?us-ascii?Q?B0aXrJdEb0gJZP+H3+fGIOYFP1X54bX50ZJ/ziHFGFJrRFck52z36ncJACTQ?=
=?us-ascii?Q?G67TEedyJz+EdT6I1z9kkva2wB8+EHbvzoTf9ijGNHSgbwutIj/ex61uzYJ0?=
=?us-ascii?Q?c1JtWFc9SKoQgBNigf9DTA0gs9LFeP2M0slEFqN8UtrDT34m1VReEMzLsK7+?=
=?us-ascii?Q?cHLkFDRYg4ayoALS2VfuDZwyERjA0spxJpuAMY8RjhQSM2e7EQNSAubjIyVu?=
=?us-ascii?Q?A9NsqwarlR0jnO94rqNuvNMu8dvvXma3+qM8gAt4RJczhzit/y9c10WY6rK1?=
=?us-ascii?Q?Fv1A2fdg2knrgI3sj2iwQLSQKNaNI/Ly09phuKmOgaCAMrEFjK8C1MvyLy7Z?=
=?us-ascii?Q?zv3cppxydfgaAWG2DvNA54l9pRPl1XRSjbT/HkddpcwpM8LCIQQ28HWsFL6z?=
=?us-ascii?Q?Q9fwaNiiwqE9YecoCWW1rR53Psnh99nSXHL5t6AHK4JAK22ntxF9Z5qDWk9x?=
=?us-ascii?Q?jvBeNq7VxvKrn5fMmY89rpBb5GFbHmibsZd8R710?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BY5PR11MB4451.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8b47543a-97c0-4178-a2f3-08da5f8d19b0
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Jul 2022 20:21:25.2224 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: yhgTMudTJ5b9JgpvK/4e9dm69XtvF/sEgJfM5RU+QSKy7NC17cI8m2dHEVXkChg+masyO/suKqZHt4AXWCFxj/obxgSQBcnhZN4YrwrAs88=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4647
X-OriginatorOrg: intel.com
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Hi Stephen, Tom.,=20
> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
>=20
> On Wed, 6 Jul 2022 12:01:19 -0700
> Tom Rix <trix@redhat.com> wrote:
>=20
> > On 7/5/22 5:23 PM, Nicolas Chautru wrote:
> > > Locking is not explicitly required but can be valuable in case the
> > > application cannot guarantee to be thread-safe, or specifically is
> > > at risk of using the same queue from multiple threads.
> > > This is an option for PMD to use this.
> > >
> > > Signed-off-by: Nicolas Chautru <nicolas.chautru@intel.com>
> > > ---
> > > lib/bbdev/rte_bbdev.h | 2 ++
> > > 1 file changed, 2 insertions(+)
> > >
> > > diff --git a/lib/bbdev/rte_bbdev.h b/lib/bbdev/rte_bbdev.h index
> > > b7ecf94..8e7ca86 100644
> > > --- a/lib/bbdev/rte_bbdev.h
> > > +++ b/lib/bbdev/rte_bbdev.h
> > > @@ -407,6 +407,8 @@ struct rte_bbdev_queue_data {
> > > struct rte_bbdev_stats queue_stats; /**< Queue statistics */
> > > enum rte_bbdev_enqueue_status enqueue_status; /**< Enqueue
> status when op is rejected */
> > > bool started; /**< Queue state */
> > > + rte_rwlock_t lock_enq; /**< lock protection for the Enqueue */
> > > + rte_rwlock_t lock_deq; /**< lock protection for the Dequeue */
> >
> > No.
> >
> > This is a good idea but needs a use before introducing another
> > element, particularly a complicated one like locking
> >
> > Tom
The actual usage would be implemented within the PMD. Basically this to pre=
vent the corner case when a queue is being accessed from multiple thread fo=
r which there is no protection in DPDK (but application does not necessaril=
y behaves well).=20
In normal operation there would never be a case when there is a conflict on=
the lock.
This is not something which was considered for any other PMD?
>From DPDK doc : "If multiple threads are to use the same hardware queue on =
the same NIC port, then locking, or some other form of mutual exclusion, is=
necessary."
Basically for AC100 we would purely enforce the lock for any enqueue/dequeu=
e operation for a given queue (distinct lock for enqueue and dequeue, since=
these would run on different threads). =20
> Having two locks on same cacheline will create lots of ping/pong false sh=
aring.
You would recommend to purely spread them within the structure? Or somethin=
g else?=20
=20
> Also, unless the reader is holding the lock for a significant fraction of=
the time a
> regular spin lock will be faster.
OK Thanks. It should in principle never have to wait for the lock for the u=
sage above, only to catch misbehaving application risk.=20
Nic