From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 94BADA04B1; Thu, 24 Sep 2020 00:41:19 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 2BC2D1DB10; Thu, 24 Sep 2020 00:41:18 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70080.outbound.protection.outlook.com [40.107.7.80]) by dpdk.org (Postfix) with ESMTP id 150091D970 for ; Thu, 24 Sep 2020 00:41:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5CUcBI1dBospbL5mEPc6RJShHBUhlEG/BXHkloB5IU4=; b=orV8y1p+u4abRJVzv/poHPiwdGUF1vRRGfVIrNq40ebZNZXMPF0IlLdettnVZX5RcZO7tsJyqH5twUOvuR14LEAGI6uoO4v2d1ib9IXka7MtIYvftMtcT6vkdRHHgxvbpQeDvBNRcLWG4wngR0Jl2oH3Ww9a2FFajADPWFIeKiM= Received: from AM5PR0202CA0017.eurprd02.prod.outlook.com (2603:10a6:203:69::27) by DBAPR08MB5733.eurprd08.prod.outlook.com (2603:10a6:10:1b3::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3391.22; Wed, 23 Sep 2020 22:41:13 +0000 Received: from AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:69:cafe::ea) by AM5PR0202CA0017.outlook.office365.com (2603:10a6:203:69::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.21 via Frontend Transport; Wed, 23 Sep 2020 22:41:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT021.mail.protection.outlook.com (10.152.16.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.21 via Frontend Transport; Wed, 23 Sep 2020 22:41:13 +0000 Received: ("Tessian outbound bac899b43a54:v64"); Wed, 23 Sep 2020 22:41:13 +0000 X-CR-MTA-TID: 64aa7808 Received: from 72081b36c3e2.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 42A94233-F396-4117-9C01-C39B77B53393.1; Wed, 23 Sep 2020 22:41:08 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 72081b36c3e2.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 23 Sep 2020 22:41:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JjYTDLcxvAlYhkkgU9C5aX8nvgCt4EVKQF8c6tgZh2gb7Q5LRPfUrTpXiFs6SI2RVYizGc0zkSa3DQMaaR5Mra3spo+j3OcnYXwAAQvfblxNOmZrfTl+BF+gkOq5Gag/4UOh6f5Ihi/vQdwim2FPY1r8VtHVGjzcA//oI7OYEP5wd7XMLJHMtXujSYSmaa1uNngv2PQJU36aaEQanke8krtakSoTi/I5HbZICKqR1ircun6YaEI6AqNtXA3/CR7sp9RUetxJWBvPzkCc5oVkbDxk7bX2AxM7LsFSQPbLajrh3FP0+7sw6F/VIda//JWtGRJWQGiE/KBpsX7pxjkTbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5CUcBI1dBospbL5mEPc6RJShHBUhlEG/BXHkloB5IU4=; b=F+6Jq7IpLZkcX9zMkMj+6s+oZdbxggaXdxXOs8978bjR3u52Tuabl89q2335K8o/MnDkdxVg0Kj+4njnkxPZBmj/ap9s2TZEZVqtS24payEnM9N3TCgbKJgjF3sX6tO+4jJW7BlsQ3r5qssh2t4E+vrZSE1QfvNculDALNYnnuWg/x46nGhCvFcjmmg1wQhXlPCqbcbpqo7E8ZvhNnorF7QD2m2FilnDGOTgJqBZ9nP0BYp1f7RTQXMUte2xVR5SGXZTox3UqdlWDy3QVNPHJ3tKKS6PQpovmiVMoubq18RvieuyFDV6RdfY1GZDdfgk03hxiVCvdmMl08v/abOXDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5CUcBI1dBospbL5mEPc6RJShHBUhlEG/BXHkloB5IU4=; b=orV8y1p+u4abRJVzv/poHPiwdGUF1vRRGfVIrNq40ebZNZXMPF0IlLdettnVZX5RcZO7tsJyqH5twUOvuR14LEAGI6uoO4v2d1ib9IXka7MtIYvftMtcT6vkdRHHgxvbpQeDvBNRcLWG4wngR0Jl2oH3Ww9a2FFajADPWFIeKiM= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by DB6PR0802MB2552.eurprd08.prod.outlook.com (2603:10a6:4:a2::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3391.17; Wed, 23 Sep 2020 22:41:06 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::7814:9c1:781f:475d]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::7814:9c1:781f:475d%4]) with mapi id 15.20.3412.020; Wed, 23 Sep 2020 22:41:06 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "Gujjar, Abhinandan S" , "dev@dpdk.org" , "Doherty, Declan" CC: "jerinj@marvell.com" , "Akhil.goyal@nxp.com" , "Vangati, Narender" , nd , Honnappa Nagarahalli , nd Thread-Topic: [dpdk-dev] [v2 1/2] cryptodev: support enqueue callback functions Thread-Index: AQHWhsTNpueHJn2Z1E+jIJevuNB9zqlrUBQAgAARrrCAAXx3gIAGGrIAgAAS1QCAAk5BgIAA2iGAgACTrsA= Date: Wed, 23 Sep 2020 22:41:06 +0000 Message-ID: References: <1599549024-195051-1-git-send-email-abhinandan.gujjar@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 411D7F3B9F8BCC4EA50F5B7E350BBE36.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [217.140.110.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 8b30faf0-e008-4848-b42f-08d86011c6a2 x-ms-traffictypediagnostic: DB6PR0802MB2552:|DBAPR08MB5733: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: bbvy9whBDuKEEUpWkfmkZ/MBcXUeVyjIehSqVvyvSChld2Gi5ZhBKoVd1Wiy1VW95VQ3GEb8aOnSe//tJnvbnukgpgfcucqBVP4HxGienLggpsq9eCnj8620RPEdzrgcK5/bCpGrFnV4Utd8GjIeAVoJuQzZ5w8yQzXHFHhAxYZFVtpWU82UtlhZAPeIrSUk+tP47NNZiOm08ILeGDxBCuhIoqgWxVkn3OhJa6pJ7Ihruwo1vgQ5OgmZSHqrSOXcNrKI8v0fom1lYomw93v3efGF4Pho03vRJicO4v9/uZIOMFisFt+8IpY9e4gH79aC2IPr6YYP+hKHJPsoYlU/kDXI4S6qxJp5FsK/CEyO1z/4pvn9qOpF2S4Q+kEXd5OyTjKleVkADtvqzzwOsVBmBQ== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(376002)(366004)(136003)(39860400002)(396003)(33656002)(26005)(71200400001)(5660300002)(110136005)(8676002)(316002)(54906003)(7696005)(52536014)(6506007)(83380400001)(4326008)(66946007)(66446008)(2906002)(66556008)(64756008)(86362001)(186003)(9686003)(8936002)(478600001)(30864003)(76116006)(66476007)(55016002)(966005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: Wx3k5JGBnPsTCPYSjnSw7fIRUgQqm3Tf57WPYgr5ea+HqUlKJ+E1KJB8axg4y5rahtACWR7SVB+3Up/nkGiMSz9bdxqbqkp7klmLkFIhSL/LvvIkipOR3eU5R+r+7wm6t2uxH/PXYXmUjkr59WzTZQ8t77TN8ZgFzU27D+yinbKpCcnaJ/z6N96iz40nKd2Xt4PJVPVkKDBRiMNWC7YdMnfHVB15U/Yb11TqFB1Y6JaJ3il6JY1zTY0h8dHm3feoOv7TxYV47XWI8mjgondq2Ef3PtzTWIZpn7Ey8B9Ib4Lb6V6rZge3+4tZLsAp8jpRKgdL/k34EV60KOs/+x0zxKWsOhuK1hWqBi778UjepsyaF6qtdzbMB5W6v02s+W2VGm1w8QpzhIwmQx6hCzxOX5EDHwKGMQrKWEN4MjZZT+5ie+dxG3LGMF+XVbtWpfSeUWDsMbf5OwcpTXl0si7Ed3rbPKwJ4IlU2M9jqM+9gPsumes/TT/tex+k4E7aY9KMlj+I0Ayne4+qnWoKXCCPKWj7rIc3EfRuS3EbiJNmOZe5iIYcXp7nS3QZkNPQ26iQh9DZ+EFP4YJMuP94GgEIIoA3dnO+m/FJe8NbRjtsd0i7wAy5JdHwSkCnl+p+B2vpjRPqCfhqzvA1xvG5MsPAig== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2552 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 1bc9c50f-f10d-418a-3446-08d86011c278 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JEnmSLfduZxkDUMAUn7enhSN0+4QUp4iET2GIr4TbN+u73pspGMPyX9Wbr7xM3A0J/2F87XmY3Woq8RKjEFOcSbB9EQkX2rrsxOls++/1N/lYLEKgKwTFVU6wc6/otZuc7bE2EJb3cGHulZQhANzs9R002IeK91AG02lt8PFbVoAo6dgfBb7A/8nslmKAw4qjScs7gikNu4wbRIVMwQx1VlOr8muSR7FhoJQGCjYZxs3MYXVWBwAEK9Ohzde4YuS0PQGj7qtoOr4dz/HDeM6EhIDK2Z7VliDYkb6eEx15/qxD4bcjTP/IJ9oFwJBjzxdjSKhJguER87nvjjg1Fc6kLPmYqLJM1FoglXrew3xg0S6df4r3TlKEUIGq5+bQHxCc9bV0UVI/eOy3bHK69gz2vAhXNjyIGmaMDbNlaSG3qCST7E4PGtyUl5QYqJh6xh1ud+XjjmqcTjeg5WdJ3vuWN8s+VPlp3K/SRNL9hSV+Vm2Mjy/NhGyC1gVmQYE0GTPPFr0sNjSgQB7rq/rFKwZDg== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(136003)(376002)(396003)(346002)(39860400002)(46966005)(6506007)(82310400003)(81166007)(30864003)(86362001)(33656002)(83380400001)(5660300002)(26005)(336012)(7696005)(356005)(54906003)(52536014)(2906002)(110136005)(186003)(9686003)(36906005)(8936002)(70586007)(47076004)(8676002)(478600001)(55016002)(70206006)(966005)(4326008)(82740400003)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2020 22:41:13.5567 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8b30faf0-e008-4848-b42f-08d86011c6a2 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: AM5EUR03FT021.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5733 Subject: Re: [dpdk-dev] [v2 1/2] cryptodev: support enqueue callback functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > > > > > > > > > +#ifdef RTE_CRYPTODEV_CALLBACKS int > > > > > > > > +rte_cryptodev_rcu_qsbr_add(uint8_t dev_id, struct > > > > > > > > +rte_rcu_qsbr > > > > > > > > +*qsbr) { > > > > > > > > + > > > > > > > > + struct rte_cryptodev *dev; > > > > > > > > + > > > > > > > > + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) { > > > > > > > > + CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8, dev_id); > > > > > > > > + return -EINVAL; > > > > > > > > + } > > > > > > > > + > > > > > > > > + dev =3D &rte_crypto_devices[dev_id]; > > > > > > > > + dev->qsbr =3D qsbr; > > > > > > > > + return 0; > > > > > > > > +} > > > > > > > > > > > > > > So if I understand your patch correctly you propose a new > > > > > > > working model for > > > > > > > crypto-devs: > > > > > > > 1. Control-plane has to allocate/setup rcu_qsbr and do > > > > > > > rte_cryptodev_rcu_qsbr_add(). > > > > > > > 2. Data-plane has somehow to obtain pointer to that rcu_qsbr > > > > > > > and wrap > > > > > > > cryptodev_enqueue() > > > > > > > with rcu_qsbr_quiescent() or > rcu_qsbr_online()/rcu_qsbr_offline(). > > > > > > Yes. I think, it is not a new model. It is same as RCU > > > > > > integration with > > > LPM. > > > > > > Please refer: https://patches.dpdk.org/cover/73673/ > > > > > > > > > > I am talking about new working model for crypto-dev > enqueue/dequeue. > > > > > As I said above now it becomes data-plane thread responsibility t= o: > > > > > -somehow to obtain pointer to that rcu_qsbr for each cryptodev > > > > > it is > > > using. > > > > > -call rcu sync functions (quiescent/online/offline) on a regular= basis. > > > > It is not on regular basis. When data plane comes up, they report o= nline. > > > > They report quiescent when they are done with critical section or > > > > shared > > > structure. > > > > > > I understand that, but it means all existing apps have to be changed = that > way. > > > > > > > All though, there is some dataplane changes involved here, I don't > > > > think, it > > > is major. > > > > > > I still think our goal here should be to make no visible changes to > > > the dataplane. > > > I.E. all necessary data-plane changes need to be hidden inside CB > > > invocation part. > > Please note that this is being implemented using the memory > > reclamation framework documented at > > https://doc.dpdk.org/guides/prog_guide/rcu_lib.html#resource-reclamati > > on-framework-for-dpdk > > > > While using RCU there are couple of trade-offs that applications have t= o > consider: > > 1) Performance - reporting the quiescent state too often results in > > performance impact on data plane > > 2) Amount of outstanding memory to reclaim - reporting less often > > results in more outstanding memory to reclaim > > > > Hence, the quiescent state reporting is left to the application. The > > application decides how often it reports the quiescent state and has co= ntrol > over the data plane performance and the outstanding memory to reclaim. > > > > When you say "new working model for crypto-dev enqueue/dequeue", > > > > 1) are you comparing these with existing crypto-dev enqueue/dequeue > APIs? If yes, these are new APIs, it is not breaking anything. > > 2) are you comparing these with existing call back functions in ethdev > > enqueue/dequeue APIs? If yes, agree that this is a new model. But, it i= s > possible to support what ethdev supports along with the RCU method used > in this patch. >=20 > What I am talking about: >=20 > Existing cryptodev enqueue/dequeue model doesn't require for the user to > manage any RCU QSBR state manually. > I believe that addition of ability to add/remove enqueue/dequeue callback= s > shouldn't change existing working model. > I think that adding/removing such callbacks has to be opaque to the user = DP > code and shouldn't require user to change it. Same as we have now for > ethdev callback implementation. The ethdev callback implementation conveniently leaves the problem of freei= ng memory to the user to resolve, it does not handle the issue. Hence, it "= looks" to be opaque to the DP code. However, if the application has to impl= ement a safe way to free the call back memory, its DP is affected based on = call backs are being used or not. > I think that forcing DP code to be aware that callbacks are present or no= t and > to modify its behaviour depending on that nearly voids the purpose of hav= ing > callbacks at all. > In that case DP can just invoke callback function directly from it's code= path . >=20 > > > > > > > > > > > > Note that now data-plane thread would have to do that always - > > > > > even if there are now callbacks installed for that cryptodev queu= e > right now. > > > > > All that changes behaviour of existing apps and I presume would > > > > > reduce adoption of that fature. > > If I understand this correct, you are talking about a case where in > > the application might be registering/unregistering multiple times > > during its lifetime. In this case, yes, the application might be report= ing the > quiescent state even when it has not registered the call backs. But, it h= as the > flexibility to not report it if it implements additional logic. > > Note that we are assuming that the application has to report quiescent > > state only for using callback functions. Most probably the application = has > other requirements to use RCU. > > Why not support what is done for ethdev call back functions along with > providing RCU method? > > > > > > There is always trade off involved! > > > > In the previous patch, you suggested that some lazy app may not > > > > free up the memory allocated by add cb. For such apps, this patch > > > > has sync mechanism with some additional cost of CP & DP changes. > > > > > > Sigh, it is not about laziness of the app. > > > The problem with current ethedev cb mechanism and yours V1 (which > > > was just a clone of it) - CP doesn't know when it is safe after CB > > > removal to free related memory. > > > > > > > > I still think all this callback mechanism should be totally > > > > > opaque to data-plane threads - user shouldn't change his app > > > > > code depending on would some enqueue/dequeue callbacks be > installed or not. > > > > I am not sure, how that can be implemented with existing RCU design= . > > > > > > As I said below the simplest way - with calling rcu onine/offline > > > inside CB invocation block. > > > That's why I asked you - did you try that approach and what is the > > > perf numbers? > > > I presume with no callbacks installed the perf change should be nearl= y > zero. > > > > > > > @Honnappa Nagarahalli, Do you have any suggestions? > > Reporting quiescent state in the call back functions has several > disadvantages: > > 1) it will have performance impacts and the impacts will increase as th= e > number of data plane threads increase. > > 2) It will require additional configuration parameters to control how > > often the quiescent state is reported to control the performance impact= . > > 3) Does not take advantage of the fact that most probably the > > application is using RCU already > > 4) There are few difficulties as well, please see below. >=20 > I suggested Abhinandan to use RCU library because it is already there, an= d I > thought it would be good not to re-implement the wheel. > Though if you feel librte_rcu doesn't match that task - fine, let's do it= without > librte_rcu. > After all, what we need here - just an atomic ref count per queue that we= are > going to increment at entering and leaving list of callbacks inside > enqueue/dequeue. Ok, looks like I missed the point that a queue is used by a single data pla= ne thread. Along with ref count increment you need the memory orderings to avoid race = conditions. These will be the same ones used in RCU. On the control plane, you need to read this counter and poll for the counte= r updates. All this is same cost as in RCU. To control the cost, you will h= ave to control the rate of quiescent state reporting and might have to expo= se this as a configuration parameter. The other important information you have to consider is if the thread is ma= king any blocking calls, which may be in some other library. The thread is = supposed to call rcu_qsbr_thread_offline API before calling a blocking call= . This allows the RCU to know that this particular thread will not report q= uiescent state. The cryptodev library might not have that information. If you want to go ahead with this design, you can still use RCU with single= thread configuration (like you have mentioned below) and hide the details = from the application. >=20 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > That seems quite a big change and I don't think it is > > > > > > > acceptable for most users. > > > > > > > From my perspective adding/installing call-backs to the dev > > > > > > > has to be opaque to the data-plane code. > > > > > > > Also note that different callbacks can be installed by > > > > > > > different entities (libs) and might have no idea about each o= ther. > > > > > > > That's why I thought it would be better to make all this RCU > > > > > > > stuff internal inside cryptodev: > > > > > > > hide all this rcu_qsbr allocation/setup inside cryptod > > > > > > > somehow to > > > > > obtain pointer to that rcu_qsbr ev init/queue setup > > > > > > > invoke rcu_qsbr_online()/rcu_qsbr_offline() inside > > > > > cryptodev_enqueue(). > > This will bring in the application related information such as the thre= ad ID > into the library. >=20 > I don't think it would. > Cryptodev enqueue/dequeue functions are not supposed to be thread safe > (same as rx/tx burst). > So we can always use RCU with just one thread(thread_id =3D 0). Agree, the memory that needs to be freed is accessed by a single thread on = the data plane. RCU with one thread would suffice. > But as I said above - if you feel RCU lib is an overhead here, that's fin= e - I > think it would be easy enough to do without librte_rcu. >=20 > > If the same API calls are being made from multiple data plane threads, > > you need a way to configure that information to the library. So, it is > > better to leave those details for the application to handle. > > > > > > > > I have already tried exploring above stuffs. There are too > > > > > > many > > > constraints. > > > > > > The changes don't fit in, as per RCU design. > > > > > > > > > > Hmm could you be more specific here - what constraints are you > > > > > referring to? > > > > > > > > > > > Moreover, having rcu api under enqueue_burst() will affect the > > > > > performance. > > > > > > > > > > It most likely will. Though my expectation it will affect > > > > > performance only when some callbacks are installed. My thought > here: > > > > > callback function by itself will affect cryptdev_enqueue > > > > > performance anyway, > > > > With existing callback design, I have measured the > > > > performance(with > > > crypto perf test) on xeon. > > > > It was almost negligible and same was shared with Declan. > > > > > > I am asking about different thing: did you try alternate approach I > > > described, that wouldn't require changes in the user data-plane code. > > > > > > > That is one of the reasons, I didn't want to add to many stuffs in > > > > to the > > > callback. > > > > The best part of existing design is crypto lib is not much modified= . > > > > The changes are either pushed to CP or DP. > > > > > > > > so adding extra overhead for sync is probably ok here. > > > > > > I think that extra overhead when callbacks are present is expected > > > and probably acceptable. > > > Changes in the upper-layer data-plane code - probably not. > > > > > > > > Though for situation when no callbacks are installed - > > > > > perfomance should be left unaffected (or impact should be as smal= l > as possible). > > > > > > > > > > > The changes are more on control plane side, which is one time. > > > > > > The data plane changes are minimal. > > > > > > > > > > I still think upper layer data-plane code should stay unaffected > > > > > (zero changes). > > > > > > > > > > > >