From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id BEEBFA04B6;
Thu, 24 Sep 2020 19:08:29 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 5685E1DEE3;
Thu, 24 Sep 2020 19:08:28 +0200 (CEST)
Received: from mga11.intel.com (mga11.intel.com [192.55.52.93])
by dpdk.org (Postfix) with ESMTP id 6A1FC1DEE2
for <dev@dpdk.org>; Thu, 24 Sep 2020 19:08:26 +0200 (CEST)
IronPort-SDR: 88IMYAqc0/0R6eWLtyZNCmAVxRm3QqKtRcyd3tUkTHh3XU9KBwOovghgPVSF8uqMR6muaUWN55
3b/PLMXtLVvQ==
X-IronPort-AV: E=McAfee;i="6000,8403,9754"; a="158626727"
X-IronPort-AV: E=Sophos;i="5.77,298,1596524400"; d="scan'208";a="158626727"
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga001.jf.intel.com ([10.7.209.18])
by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
24 Sep 2020 10:06:58 -0700
IronPort-SDR: quBSoQ01MmNZD/WgFOoofH2OiX0uzqzv0O8jxcFVPoRH6Z83yxId15ba8QAiXGUTED+zkhCa5p
GPchuy7/9h4g==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.77,298,1596524400"; d="scan'208";a="383133232"
Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16])
by orsmga001.jf.intel.com with ESMTP; 24 Sep 2020 10:06:58 -0700
Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by
ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.1713.5; Thu, 24 Sep 2020 10:06:58 -0700
Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by
ORSMSX610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.1713.5; Thu, 24 Sep 2020 10:06:57 -0700
Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by
orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5
via Frontend Transport; Thu, 24 Sep 2020 10:06:57 -0700
Received: from NAM04-CO1-obe.outbound.protection.outlook.com (104.47.45.51) by
edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.1.1713.5; Thu, 24 Sep 2020 10:06:54 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=LganO7XuFC2fuPBHzty5ETvmKq2u0NgYU4WKPaXbed5oM1PQUJ9JNMEkq25ACMeDYyAiX2AHr9NEtJNu3KWWKIyOBLczUNemwy2WJtMN+LI3lSaC4YYfPc6c7QsU0/TEzo8nzSqmvVbPw4GqtJNocY2I+uJHOvJZPJynsH5EXTm1vsY7/6H9f3xb/HFo8lNQ5OlGoRlzQxN27Qf1WkNr18U3pM3WeSRSB1J2mBwYAGbDYe9cv2PoAP44s078svV4TEjfM3QHiFJpUojBSRXOaRAmkxtGx7hqp+w1PTZq5eGq2aFLhXsmggq5RgwzH27wHBUP4kjcLgnjOoBTdHyHUQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=q0L9uOSTcjlGevJ67M1MVEHIMk861PeeIu+iUpKN2ow=;
b=ilK5fyJJ5oIKVU9FF+R+b7q6IYwVNLDBQQcEKsvklCV/EpucGrJ/HWgV3hPQ+cmME2eyEz+wajM9fKWUZM7yk+8My+49JN18jyZSPaI3t6WWIOI7VLv6xku17dZv/yyrStCYqzqbIce+51mC2m19cL8ijq3PNY25stn0B33W7q2bi2tQKat1UfpR2XDyAtVk3kPhrw6Omeh4a8yXF+MuXQ4ImhHati999VaXgWIXYQt/DbOEwVip+OiHpz5EKZKQyQ1zlNLO9FCpqvZDjsib+VTvfwlSCujg1sRoZzQEEnm5YxV/K6uE/W7KTB9HCdeGOI6fLDnxwlPZvHegUV+dlw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
s=selector2-intel-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=q0L9uOSTcjlGevJ67M1MVEHIMk861PeeIu+iUpKN2ow=;
b=aonsZcLRSARlPG0f2xZzI67dPMPScyB/YFyPkLWUtLBsJ1WuDY2nCzDDoNKHSptPBrGIrA258zBYx9I3ftIU+NFFedu1btHVqanIGxYW8ShPlLuenp01ULdWWIBOzd+dlTcVBVRKx43M1slFGChRLuYvFGDGLLxMbCp+kZA/6Nk=
Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26)
by BYAPR11MB3558.namprd11.prod.outlook.com (2603:10b6:a03:b3::11)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3391.19; Thu, 24 Sep
2020 17:06:52 +0000
Received: from BYAPR11MB3301.namprd11.prod.outlook.com
([fe80::f5a4:3f6b:ade3:296b]) by BYAPR11MB3301.namprd11.prod.outlook.com
([fe80::f5a4:3f6b:ade3:296b%3]) with mapi id 15.20.3412.022; Thu, 24 Sep 2020
17:06:52 +0000
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, "Gujjar, Abhinandan
S" <abhinandan.gujjar@intel.com>, "dev@dpdk.org" <dev@dpdk.org>, "Doherty,
Declan" <declan.doherty@intel.com>
CC: "jerinj@marvell.com" <jerinj@marvell.com>, "Akhil.goyal@nxp.com"
<akhil.goyal@nxp.com>, "Vangati, Narender" <narender.vangati@intel.com>, nd
<nd@arm.com>, nd <nd@arm.com>
Thread-Topic: [dpdk-dev] [v2 1/2] cryptodev: support enqueue callback functions
Thread-Index: AQHWhsTbY+pAzzKww0euN2orABmUValrRWzAgAAl9gCAAW0pMIAGJnUAgAAH/lCAAp3uAIAAgn/ggADASgCAASzNQA==
Date: Thu, 24 Sep 2020 17:06:52 +0000
Message-ID: <BYAPR11MB3301FFC21A26E5A37A9921219A390@BYAPR11MB3301.namprd11.prod.outlook.com>
References: <1599549024-195051-1-git-send-email-abhinandan.gujjar@intel.com>
<BYAPR11MB3301233DDE876BF3D19463909A210@BYAPR11MB3301.namprd11.prod.outlook.com>
<MWHPR11MB183809AE6FF88ED2F2A6C252E8210@MWHPR11MB1838.namprd11.prod.outlook.com>
<BYAPR11MB3301D3074A678BB10B1FE7AF9A3E0@BYAPR11MB3301.namprd11.prod.outlook.com>
<MWHPR11MB1838DAEE401C006C913238C9E83A0@MWHPR11MB1838.namprd11.prod.outlook.com>
<BYAPR11MB3301D8480B010A152FB134159A3A0@BYAPR11MB3301.namprd11.prod.outlook.com>
<DBAPR08MB581470913C8FEE2C30FFB28698380@DBAPR08MB5814.eurprd08.prod.outlook.com>
<DM6PR11MB330896B2863AEA0F2D24B4B39A380@DM6PR11MB3308.namprd11.prod.outlook.com>
<DBAPR08MB581492EEC7314057462689C298380@DBAPR08MB5814.eurprd08.prod.outlook.com>
In-Reply-To: <DBAPR08MB581492EEC7314057462689C298380@DBAPR08MB5814.eurprd08.prod.outlook.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.5.1.3
authentication-results: arm.com; dkim=none (message not signed)
header.d=none;arm.com; dmarc=none action=none header.from=intel.com;
x-originating-ip: [46.7.39.127]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: b5172ab1-a217-48f2-6196-08d860ac3bbe
x-ms-traffictypediagnostic: BYAPR11MB3558:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <BYAPR11MB3558B2D7EF1C5BFC583822C89A390@BYAPR11MB3558.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 6yBydjf+FtXirjE2RvVIw/SuulaJxjMOYRq9phE9i+mf/60bD+voeUIgMb3/nCrjRNB7R7kNn1f8bKkJRyExKqmUrwHl3fLnLm4c3skkkCrJCYONIrcD9fFtbNQlAjE8d/n7gZSaexjMgyQshAlnuLikujS5/znJJ3i+iduhG04qQDoBAvf5kHavXB3wrEayRVzwzNll+1s8L4bMHjGA6c36mzO/vyl83mlaonUN83Bpbf+QBsdW/06sXwnCbcORg4Ks6FPNHm1bkfWVU/hUSjsQxOYWvaV1PobxQHDCdej9Y5bXWpoHGpJCZqYaRwFiErn6cKw+gFWkDsAbgnWIitSeiohCBH1/zB3MuX/OC+1yZm/CordobC5ADHz3+5g01SpP3Sc5G3OsP6t4egx+ag==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE;
SFS:(4636009)(136003)(39860400002)(366004)(346002)(376002)(396003)(86362001)(55016002)(2906002)(110136005)(64756008)(76116006)(6636002)(83380400001)(478600001)(71200400001)(966005)(9686003)(316002)(66446008)(66476007)(66946007)(66556008)(26005)(186003)(30864003)(52536014)(54906003)(8936002)(5660300002)(7696005)(4326008)(8676002)(6506007)(33656002);
DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: vfeqZizd8kKkewLC64NZ/Wgcq3xNk1lSTR9ktxUrsD2diVJ9tnq8nAhqST0VgKnxKYyzjmzq+gQOgNqKl5hUxSyOtggfydrwcOPhojTe1Yy32kqwP4i1KRpBD8ZwnJk3f4klZdxTljCzSiBLtYdPkNxxXCnnw1JQ4Kq2K3midnKI4oPJfnLmIO7UQTjbEzBXfcUqWMNxfjwpffuAOegDNcGWuNQmOgs88vvDgUFScpg4hi6AP9ix+XuHnG6MhQu+w+jZuskmXCsE9R+4VEXCe4vj6apHy+u9Gt2dfky2a16CXKPuha+78xxHbDvVIwYmYlWE4hq71UMPnJCu63e/anvR0UzEoOtWPZoroqNax8kheoCfrB5UNfFfAiawhCl8VIFiON40sSZxnD94SdBorNBwKbgRq+wjN3rDhbkqp/CqfWHzRw6KK1btpGvz2h+UavBwwPqcpqBE3tWi0s0YndgxNCuthdeWjH1l4LwopZI5L3/UozypFcZ/yA4LlSUwRGOAqlGKmP6iF0x0IqXBSsS+7DSCGBydn6FBzJde/mXt5/Zq86XgaKX89cdhYm9PkkvNcpbYyWHcml9DtpUW96vifa8knY4vnJsD4+34jXX7r7v/BxCSVky/AZsGOQYA5wqrgq8ZbTA9EEQw+sKuWA==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3301.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b5172ab1-a217-48f2-6196-08d860ac3bbe
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Sep 2020 17:06:52.3890 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sw7GDUA/UApMsbivG4uLWzlkfPdS+IoTbw34rOTqAjIGl7ogekS4B96B4z+4an2NAX/RgK0gKpE0siUaBXUjA5ccp+MEq4iMfoIMHn07kRM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3558
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [v2 1/2] cryptodev: support enqueue callback
functions
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
>=20
> > > > > > > > >
> > > > > > > > > +#ifdef RTE_CRYPTODEV_CALLBACKS int
> > > > > > > > > +rte_cryptodev_rcu_qsbr_add(uint8_t dev_id, struct
> > > > > > > > > +rte_rcu_qsbr
> > > > > > > > > +*qsbr) {
> > > > > > > > > +
> > > > > > > > > + struct rte_cryptodev *dev;
> > > > > > > > > +
> > > > > > > > > + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> > > > > > > > > + CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8, dev_id);
> > > > > > > > > + return -EINVAL;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + dev =3D &rte_crypto_devices[dev_id];
> > > > > > > > > + dev->qsbr =3D qsbr;
> > > > > > > > > + return 0;
> > > > > > > > > +}
> > > > > > > >
> > > > > > > > So if I understand your patch correctly you propose a new
> > > > > > > > working model for
> > > > > > > > crypto-devs:
> > > > > > > > 1. Control-plane has to allocate/setup rcu_qsbr and do
> > > > > > > > rte_cryptodev_rcu_qsbr_add().
> > > > > > > > 2. Data-plane has somehow to obtain pointer to that rcu_qsb=
r
> > > > > > > > and wrap
> > > > > > > > cryptodev_enqueue()
> > > > > > > > with rcu_qsbr_quiescent() or
> > rcu_qsbr_online()/rcu_qsbr_offline().
> > > > > > > Yes. I think, it is not a new model. It is same as RCU
> > > > > > > integration with
> > > > LPM.
> > > > > > > Please refer: https://patches.dpdk.org/cover/73673/
> > > > > >
> > > > > > I am talking about new working model for crypto-dev
> > enqueue/dequeue.
> > > > > > As I said above now it becomes data-plane thread responsibility=
to:
> > > > > > -somehow to obtain pointer to that rcu_qsbr for each cryptodev
> > > > > > it is
> > > > using.
> > > > > > -call rcu sync functions (quiescent/online/offline) on a regul=
ar basis.
> > > > > It is not on regular basis. When data plane comes up, they report=
online.
> > > > > They report quiescent when they are done with critical section or
> > > > > shared
> > > > structure.
> > > >
> > > > I understand that, but it means all existing apps have to be change=
d that
> > way.
> > > >
> > > > > All though, there is some dataplane changes involved here, I don'=
t
> > > > > think, it
> > > > is major.
> > > >
> > > > I still think our goal here should be to make no visible changes to
> > > > the dataplane.
> > > > I.E. all necessary data-plane changes need to be hidden inside CB
> > > > invocation part.
> > > Please note that this is being implemented using the memory
> > > reclamation framework documented at
> > > https://doc.dpdk.org/guides/prog_guide/rcu_lib.html#resource-reclamat=
i
> > > on-framework-for-dpdk
> > >
> > > While using RCU there are couple of trade-offs that applications have=
to
> > consider:
> > > 1) Performance - reporting the quiescent state too often results in
> > > performance impact on data plane
> > > 2) Amount of outstanding memory to reclaim - reporting less often
> > > results in more outstanding memory to reclaim
> > >
> > > Hence, the quiescent state reporting is left to the application. The
> > > application decides how often it reports the quiescent state and has =
control
> > over the data plane performance and the outstanding memory to reclaim.
> > >
> > > When you say "new working model for crypto-dev enqueue/dequeue",
> > >
> > > 1) are you comparing these with existing crypto-dev enqueue/dequeue
> > APIs? If yes, these are new APIs, it is not breaking anything.
> > > 2) are you comparing these with existing call back functions in ethde=
v
> > > enqueue/dequeue APIs? If yes, agree that this is a new model. But, it=
is
> > possible to support what ethdev supports along with the RCU method used
> > in this patch.
> >
> > What I am talking about:
> >
> > Existing cryptodev enqueue/dequeue model doesn't require for the user t=
o
> > manage any RCU QSBR state manually.
> > I believe that addition of ability to add/remove enqueue/dequeue callba=
cks
> > shouldn't change existing working model.
> > I think that adding/removing such callbacks has to be opaque to the use=
r DP
> > code and shouldn't require user to change it. Same as we have now for
> > ethdev callback implementation.
> The ethdev callback implementation conveniently leaves the problem of fre=
eing memory to the user to resolve, it does not handle the issue.
> Hence, it "looks" to be opaque to the DP code. However, if the applicatio=
n has to implement a safe way to free the call back memory, its DP
> is affected based on call backs are being used or not.
Yes, I think that's big drawback in initial ethdev callback implementation =
-
it simply ignores DP/CP sync problem completely.
Though I think it is possible to have both here:
keep callback "opaque" to DP code and provide some sync mechanism between =
DP/CP.
Hopefully one day we can fix ethdev callbacks too.=20
> > I think that forcing DP code to be aware that callbacks are present or =
not and
> > to modify its behaviour depending on that nearly voids the purpose of h=
aving
> > callbacks at all.
> > In that case DP can just invoke callback function directly from it's co=
depath .
> >
> > > >
> > > > >
> > > > > > Note that now data-plane thread would have to do that always -
> > > > > > even if there are now callbacks installed for that cryptodev qu=
eue
> > right now.
> > > > > > All that changes behaviour of existing apps and I presume would
> > > > > > reduce adoption of that fature.
> > > If I understand this correct, you are talking about a case where in
> > > the application might be registering/unregistering multiple times
> > > during its lifetime. In this case, yes, the application might be repo=
rting the
> > quiescent state even when it has not registered the call backs. But, it=
has the
> > flexibility to not report it if it implements additional logic.
> > > Note that we are assuming that the application has to report quiescen=
t
> > > state only for using callback functions. Most probably the applicatio=
n has
> > other requirements to use RCU.
> > > Why not support what is done for ethdev call back functions along wit=
h
> > providing RCU method?
> > >
> > > > > There is always trade off involved!
> > > > > In the previous patch, you suggested that some lazy app may not
> > > > > free up the memory allocated by add cb. For such apps, this patch
> > > > > has sync mechanism with some additional cost of CP & DP changes.
> > > >
> > > > Sigh, it is not about laziness of the app.
> > > > The problem with current ethedev cb mechanism and yours V1 (which
> > > > was just a clone of it) - CP doesn't know when it is safe after CB
> > > > removal to free related memory.
> > > >
> > > > > > I still think all this callback mechanism should be totally
> > > > > > opaque to data-plane threads - user shouldn't change his app
> > > > > > code depending on would some enqueue/dequeue callbacks be
> > installed or not.
> > > > > I am not sure, how that can be implemented with existing RCU desi=
gn.
> > > >
> > > > As I said below the simplest way - with calling rcu onine/offline
> > > > inside CB invocation block.
> > > > That's why I asked you - did you try that approach and what is the
> > > > perf numbers?
> > > > I presume with no callbacks installed the perf change should be nea=
rly
> > zero.
> > > >
> > > > > @Honnappa Nagarahalli, Do you have any suggestions?
> > > Reporting quiescent state in the call back functions has several
> > disadvantages:
> > > 1) it will have performance impacts and the impacts will increase as =
the
> > number of data plane threads increase.
> > > 2) It will require additional configuration parameters to control how
> > > often the quiescent state is reported to control the performance impa=
ct.
> > > 3) Does not take advantage of the fact that most probably the
> > > application is using RCU already
> > > 4) There are few difficulties as well, please see below.
> >
> > I suggested Abhinandan to use RCU library because it is already there, =
and I
> > thought it would be good not to re-implement the wheel.
> > Though if you feel librte_rcu doesn't match that task - fine, let's do =
it without
> > librte_rcu.
> > After all, what we need here - just an atomic ref count per queue that =
we are
> > going to increment at entering and leaving list of callbacks inside
> > enqueue/dequeue.
> Ok, looks like I missed the point that a queue is used by a single data p=
lane thread.
> Along with ref count increment you need the memory orderings to avoid rac=
e conditions. These will be the same ones used in RCU.
> On the control plane, you need to read this counter and poll for the coun=
ter updates. All this is same cost as in RCU.
Agree.
> To control the cost, you
> will have to control the rate of quiescent state reporting and might have=
to expose this as a configuration parameter.
>=20
> The other important information you have to consider is if the thread is =
making any blocking calls, which may be in some other library. The
> thread is supposed to call rcu_qsbr_thread_offline API before calling a b=
locking call. This allows the RCU to know that this particular thread
> will not report quiescent state. The cryptodev library might not have tha=
t information.
>=20
> If you want to go ahead with this design, you can still use RCU with sing=
le thread configuration (like you have mentioned below) and hide
> the details from the application.
Yes, same thought here - use rcu_qsbr online/offline for DP part and hide=
actual sync details inside callback code.
>=20
> >
> > >
> > > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > That seems quite a big change and I don't think it is
> > > > > > > > acceptable for most users.
> > > > > > > > From my perspective adding/installing call-backs to the dev
> > > > > > > > has to be opaque to the data-plane code.
> > > > > > > > Also note that different callbacks can be installed by
> > > > > > > > different entities (libs) and might have no idea about each=
other.
> > > > > > > > That's why I thought it would be better to make all this RC=
U
> > > > > > > > stuff internal inside cryptodev:
> > > > > > > > hide all this rcu_qsbr allocation/setup inside cryptod
> > > > > > > > somehow to
> > > > > > obtain pointer to that rcu_qsbr ev init/queue setup
> > > > > > > > invoke rcu_qsbr_online()/rcu_qsbr_offline() inside
> > > > > > cryptodev_enqueue().
> > > This will bring in the application related information such as the th=
read ID
> > into the library.
> >
> > I don't think it would.
> > Cryptodev enqueue/dequeue functions are not supposed to be thread safe
> > (same as rx/tx burst).
> > So we can always use RCU with just one thread(thread_id =3D 0).
> Agree, the memory that needs to be freed is accessed by a single thread o=
n the data plane. RCU with one thread would suffice.
>=20
> > But as I said above - if you feel RCU lib is an overhead here, that's f=
ine - I
> > think it would be easy enough to do without librte_rcu.
> >
> > > If the same API calls are being made from multiple data plane threads=
,
> > > you need a way to configure that information to the library. So, it i=
s
> > > better to leave those details for the application to handle.
> > >
> > > > > > > I have already tried exploring above stuffs. There are too
> > > > > > > many
> > > > constraints.
> > > > > > > The changes don't fit in, as per RCU design.
> > > > > >
> > > > > > Hmm could you be more specific here - what constraints are you
> > > > > > referring to?
> > > > > >
> > > > > > > Moreover, having rcu api under enqueue_burst() will affect th=
e
> > > > > > performance.
> > > > > >
> > > > > > It most likely will. Though my expectation it will affect
> > > > > > performance only when some callbacks are installed. My thought
> > here:
> > > > > > callback function by itself will affect cryptdev_enqueue
> > > > > > performance anyway,
> > > > > With existing callback design, I have measured the
> > > > > performance(with
> > > > crypto perf test) on xeon.
> > > > > It was almost negligible and same was shared with Declan.
> > > >
> > > > I am asking about different thing: did you try alternate approach I
> > > > described, that wouldn't require changes in the user data-plane cod=
e.
> > > >
> > > > > That is one of the reasons, I didn't want to add to many stuffs i=
n
> > > > > to the
> > > > callback.
> > > > > The best part of existing design is crypto lib is not much modifi=
ed.
> > > > > The changes are either pushed to CP or DP.
> > > > >
> > > > > so adding extra overhead for sync is probably ok here.
> > > >
> > > > I think that extra overhead when callbacks are present is expected
> > > > and probably acceptable.
> > > > Changes in the upper-layer data-plane code - probably not.
> > > >
> > > > > > Though for situation when no callbacks are installed -
> > > > > > perfomance should be left unaffected (or impact should be as sm=
all
> > as possible).
> > > > > >
> > > > > > > The changes are more on control plane side, which is one time=
.
> > > > > > > The data plane changes are minimal.
> > > > > >
> > > > > > I still think upper layer data-plane code should stay unaffecte=
d
> > > > > > (zero changes).
> > > > > >
> > > > > > > >
>=20
> <snip>