From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 9E4F9A04C0;
Tue, 29 Sep 2020 07:14:45 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id E69701D6D6;
Tue, 29 Sep 2020 07:14:43 +0200 (CEST)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
(mail-db8eur05on2046.outbound.protection.outlook.com [40.107.20.46])
by dpdk.org (Postfix) with ESMTP id 5203E1D6CF
for <dev@dpdk.org>; Tue, 29 Sep 2020 07:14:41 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=HyKCIoSZ82N1NJR6k0QfOdwz66LBLTBBS4STxm6JooM=;
b=tsMR7DOxICePosxLFTihTyTx+HyyXjbtbFITl3OxsDxSAvnajdM0r68KfHF+nko/LvtnpXrDGSd2Y4J6szMFqpQLHUgOHpPtMzrbA3nPUszkF6Ssz0syhaXREVJlz/6P0g0OITPuAxsKQE/F/fPKpMWQ8fHaUFZsyEN5LCEBogE=
Received: from AM5PR04CA0034.eurprd04.prod.outlook.com (2603:10a6:206:1::47)
by AM7PR08MB5304.eurprd08.prod.outlook.com (2603:10a6:20b:10e::21) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.20; Tue, 29 Sep
2020 05:14:38 +0000
Received: from VE1EUR03FT013.eop-EUR03.prod.protection.outlook.com
(2603:10a6:206:1:cafe::7b) by AM5PR04CA0034.outlook.office365.com
(2603:10a6:206:1::47) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.20 via Frontend
Transport; Tue, 29 Sep 2020 05:14:38 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified)
header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none
header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
63.35.35.123 as permitted sender) receiver=protection.outlook.com;
client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
VE1EUR03FT013.mail.protection.outlook.com (10.152.19.37) with
Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.3412.21 via Frontend Transport; Tue, 29 Sep 2020 05:14:38 +0000
Received: ("Tessian outbound 195a290eb161:v64");
Tue, 29 Sep 2020 05:14:37 +0000
X-CR-MTA-TID: 64aa7808
Received: from 04cd94e47f19.1
by 64aa7808-outbound-1.mta.getcheckrecipient.com id
0FBC4DA9-8BFB-433F-B1AE-B63431A36F49.1;
Tue, 29 Sep 2020 05:14:32 +0000
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 04cd94e47f19.1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
Tue, 29 Sep 2020 05:14:32 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=jGfnVI8GCMlqVMAKk2X3aLH2gEJOZyYcbPcIW0eDi9TVQh9iN6KoW8HTspvagiK0pfmFSMYhA5cFi/9fgyF+sCW00x1CUIB6RuzXLg0cMW9uMpbVLAyb0i58FIdKi74WtYR9tfNnyo3Eq3WDdX3dDSPPAacvfr/UyM1b8mJjaepcY5cinJzSpWzTuO1jYhnZ8bglfHwl7znULXA7g6YFqrrrON4xfhde0hrDIjeqIGQDYtpCrSNcHWGvOrwWD//srvg5XhFp7oeKawiL3AJ6RM5Jc9J1ax7ZJMJeRr0sFFgCkHI3e+uzg7s7C8hbWQMF12BQp1Dl9BCdeZOUVL407Q==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=HyKCIoSZ82N1NJR6k0QfOdwz66LBLTBBS4STxm6JooM=;
b=W/eGNFfxfunS3juplFUj5RmAj5MmhTnjCcCpSxSJH4IT0oXVCuVUMaieFVWJplMfW2RLflmhkCRsCgE+4qXjuKpTq5Wr48wtA8Vt9WHJAzIk8BO0KtVhMzbDEqrfv+pJpAdGEcvbKtuL81nfWYsFmXBHdJN62JgJ9QyWkkjNfI96zf8MphCgvYUxkg2mggNfh0LseeiMqAKpbdmeZ4Q4LIhaENosNuZz363B88LxMihwHzIcl5OmCHpU+jAo3PaB165GeUDqOlNqqQusJYUrJqCtBMBJOz8op8IariHT2qwKZEmTMT1KH6tvNb4FgKHu7eGW+3WbVsedrMV8cunkpw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=HyKCIoSZ82N1NJR6k0QfOdwz66LBLTBBS4STxm6JooM=;
b=tsMR7DOxICePosxLFTihTyTx+HyyXjbtbFITl3OxsDxSAvnajdM0r68KfHF+nko/LvtnpXrDGSd2Y4J6szMFqpQLHUgOHpPtMzrbA3nPUszkF6Ssz0syhaXREVJlz/6P0g0OITPuAxsKQE/F/fPKpMWQ8fHaUFZsyEN5LCEBogE=
Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6)
by DB8PR08MB5482.eurprd08.prod.outlook.com (2603:10a6:10:116::9) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.20; Tue, 29 Sep
2020 05:14:30 +0000
Received: from DBAPR08MB5814.eurprd08.prod.outlook.com
([fe80::7814:9c1:781f:475d]) by DBAPR08MB5814.eurprd08.prod.outlook.com
([fe80::7814:9c1:781f:475d%4]) with mapi id 15.20.3412.029; Tue, 29 Sep 2020
05:14:30 +0000
From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>, "Gujjar, Abhinandan
S" <abhinandan.gujjar@intel.com>, "dev@dpdk.org" <dev@dpdk.org>, "Doherty,
Declan" <declan.doherty@intel.com>
CC: "jerinj@marvell.com" <jerinj@marvell.com>, "Akhil.goyal@nxp.com"
<akhil.goyal@nxp.com>, "Vangati, Narender" <narender.vangati@intel.com>, nd
<nd@arm.com>, Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>, nd
<nd@arm.com>
Thread-Topic: [dpdk-dev] [v2 1/2] cryptodev: support enqueue callback functions
Thread-Index: AQHWhsTNpueHJn2Z1E+jIJevuNB9zqlrUBQAgAARrrCAAXx3gIAGGrIAgAAS1QCAAk5BgIAA2iGAgACTrsCAAVTYAIAHE6IA
Date: Tue, 29 Sep 2020 05:14:30 +0000
Message-ID: <DBAPR08MB581477EA5ADBB7D16BCCAC9498320@DBAPR08MB5814.eurprd08.prod.outlook.com>
References: <1599549024-195051-1-git-send-email-abhinandan.gujjar@intel.com>
<BYAPR11MB3301233DDE876BF3D19463909A210@BYAPR11MB3301.namprd11.prod.outlook.com>
<MWHPR11MB183809AE6FF88ED2F2A6C252E8210@MWHPR11MB1838.namprd11.prod.outlook.com>
<BYAPR11MB3301D3074A678BB10B1FE7AF9A3E0@BYAPR11MB3301.namprd11.prod.outlook.com>
<MWHPR11MB1838DAEE401C006C913238C9E83A0@MWHPR11MB1838.namprd11.prod.outlook.com>
<BYAPR11MB3301D8480B010A152FB134159A3A0@BYAPR11MB3301.namprd11.prod.outlook.com>
<DBAPR08MB581470913C8FEE2C30FFB28698380@DBAPR08MB5814.eurprd08.prod.outlook.com>
<DM6PR11MB330896B2863AEA0F2D24B4B39A380@DM6PR11MB3308.namprd11.prod.outlook.com>
<DBAPR08MB581492EEC7314057462689C298380@DBAPR08MB5814.eurprd08.prod.outlook.com>
<BYAPR11MB3301FFC21A26E5A37A9921219A390@BYAPR11MB3301.namprd11.prod.outlook.com>
In-Reply-To: <BYAPR11MB3301FFC21A26E5A37A9921219A390@BYAPR11MB3301.namprd11.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 6EB929F590847E47AB3F1B0B7587EF2F.0
x-checkrecipientchecked: true
Authentication-Results-Original: intel.com; dkim=none (message not signed)
header.d=none;intel.com; dmarc=none action=none header.from=arm.com;
x-originating-ip: [217.140.110.7]
x-ms-publictraffictype: Email
X-MS-Office365-Filtering-HT: Tenant
X-MS-Office365-Filtering-Correlation-Id: e5f1521d-1956-45c1-56df-08d864369055
x-ms-traffictypediagnostic: DB8PR08MB5482:|AM7PR08MB5304:
x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr
x-ms-exchange-transport-forked: True
X-Microsoft-Antispam-PRVS: <AM7PR08MB5304F6351DFEFB5C79AB670A98320@AM7PR08MB5304.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original: 3dWFwr46z18fKImfbLXgdfUXUjTL2LXrZpZN//Rgx374pnY6oIKraGmqY0ISSlI1oZbpoonYXkZd/P6hNZpptjt5Kr8rgH1YDD8mMYO1C2ANybbtbBbogV3VJRtJL9RCiTR1VZVUEaV8oBazs0uHdheD7T07QBoj8GetirJ13H2bqZq2ko3/4lFGYmeYOcI+knBleJLyQ9LuM3EgrGHD2NS2bO4U0DJm3JUrsu/SCyR/2+VrnaUS/XXZTUXKAzWGe3vclW8rtT0FPqRirU+m8tE7uOsEnjI55uY20LB6ynqMMVEwUql3ME8ZIyJ5xK2BFTFDY5S03fknRdJ9e0/RpHP/A39sF3WM/iHmfRiWPXFETwpfbqfWpqUEyxHIJ77tNSitczoTSsHO4EYSGuzDEfN39/LuecTvkd9xSHsEC4ZrDJc5s/fY/0puK04syGZeLrwuCuD4v+GTMOsItnEM3A==
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en;
SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com;
PTR:; CAT:NONE;
SFS:(4636009)(136003)(366004)(396003)(376002)(39860400002)(346002)(55016002)(9686003)(2906002)(7696005)(4326008)(71200400001)(6506007)(26005)(110136005)(186003)(316002)(33656002)(30864003)(8676002)(8936002)(5660300002)(54906003)(478600001)(966005)(76116006)(66946007)(86362001)(66476007)(64756008)(66446008)(66556008)(52536014)(83380400001);
DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: v5yhr7z8DC5Jwhswu47aSRK7By4AX17gQZJiCZKeyuQ9te17mLgwTBkT6KsF9QW+rahJRG1Q+TbHAvOPzZxzeUZHL24mklAnV+giqbtm/9Ajz+q7kXQiBJ3McYD9PDuyvbqIQltkViOTkFQ7Q4ncFZ/NBmeZq3w0McCfb5Z9KU7C6d1BDJ/9RBQ2ZTQW67ICBMJclAV6OaeVAEN7fSKjOa+xRyN5kVF4rCEzIIy/tsu3OVON88VHbPfjq68EbCLjLhduHbFV49na5nduDLQqMLxOpDs6W+RDO/hYtLadokCV+GpEyITsvzUB/rtuBliXacyBxz2l31SgqlQh/5Pxpa+q+/rddp3YxHC6jyB+w2gQDEnESMJJHW7YajO6OaDqL/b9lxOH9QmOWxVxWKihsVGFA8D1r+lNLf/TYF0TgyHes80hBLDuEP168TEaiwrSQ7aw3FSNWCMtjQAzbD6dk6Fqkfq0ngpU+M6u9dD6U9iLtczxKTjb/izxbvaQ2ggf+xUkErUBlQlUKJu7TpqW31bMdwIQHmxSqWfVxD2Or0sYeXer9vtS0/viMwhfL70alpt6MlZYVaBdrklOsbCSx/81N0CERW3YqOxM7/7R/SipwDK43Awc4A5FPhnbl8pMvuc2JBsQPIbAUF/8m0yKSA==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR08MB5482
Original-Authentication-Results: intel.com; dkim=none (message not signed)
header.d=none;intel.com; dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT013.eop-EUR03.prod.protection.outlook.com
X-MS-Office365-Filtering-Correlation-Id-Prvs: d1537cc3-3c45-44ec-7fa9-08d864368bc9
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: em10u4tVtAs2f37hyRmDA94alGUIvM2nUIJ6aOS4zfpLZEfWBvW9fUbVXQF3x+cvRVaIb1H3MqClRG9GlovzWUmq9odczQSA3lIaQEQ0coHzgUtz+t6WnMkwyQ1k2L/ECEGbnMiEtsyAve7Y/3vHKsBpQMvD0AkrtxNcJNbiEVV8P2//Kkv4Mf72NjACL0EvB2wQAl3ElZbVPCAX/KENQffPIeLvsKmF9YTeuEf9WqC2Q3CVJ6rST5Wb4aYaZJYrEC6vQrWSrdar6RX2IU11LnlMOrBYVA7uD+WPJd/AX/HX1TZD8RObN9LpfPfSm88RmgPyfokcTLaE2sCKqM4HzHrHMMaomr13H8xecbtJPA7IEmjlvZ6XhL7tDyh8qOrtmuzAV6f5BiQWw14fYjfzuK+MdnBIDF5di6tLI+Uf9yiU93AI4xIicDwqtsumy2m384VbTAgWZGQSaojar8IYdhIJxj2rfwHKM/oYOU73V3HqLqCAhYd0wdYRq5sxh9ktYx7zcYvbdni7F4cJv1aisQ==
X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:;
IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com;
PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE;
SFS:(4636009)(136003)(396003)(39860400002)(346002)(376002)(46966005)(70586007)(82310400003)(70206006)(30864003)(5660300002)(8936002)(86362001)(8676002)(33656002)(81166007)(356005)(83380400001)(2906002)(966005)(26005)(336012)(6506007)(7696005)(54906003)(110136005)(36906005)(316002)(52536014)(186003)(47076004)(478600001)(55016002)(9686003)(4326008)(82740400003);
DIR:OUT; SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2020 05:14:38.3969 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e5f1521d-1956-45c1-56df-08d864369055
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123];
Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT013.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5304
Subject: Re: [dpdk-dev] [v2 1/2] cryptodev: support enqueue callback
functions
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
<snip>
> >
> > > > > > > > > >
> > > > > > > > > > +#ifdef RTE_CRYPTODEV_CALLBACKS int
> > > > > > > > > > +rte_cryptodev_rcu_qsbr_add(uint8_t dev_id, struct
> > > > > > > > > > +rte_rcu_qsbr
> > > > > > > > > > +*qsbr) {
> > > > > > > > > > +
> > > > > > > > > > + struct rte_cryptodev *dev;
> > > > > > > > > > +
> > > > > > > > > > + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> > > > > > > > > > + CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8,
> dev_id);
> > > > > > > > > > + return -EINVAL;
> > > > > > > > > > + }
> > > > > > > > > > +
> > > > > > > > > > + dev =3D &rte_crypto_devices[dev_id];
> > > > > > > > > > + dev->qsbr =3D qsbr;
> > > > > > > > > > + return 0;
> > > > > > > > > > +}
> > > > > > > > >
> > > > > > > > > So if I understand your patch correctly you propose a
> > > > > > > > > new working model for
> > > > > > > > > crypto-devs:
> > > > > > > > > 1. Control-plane has to allocate/setup rcu_qsbr and do
> > > > > > > > > rte_cryptodev_rcu_qsbr_add().
> > > > > > > > > 2. Data-plane has somehow to obtain pointer to that
> > > > > > > > > rcu_qsbr and wrap
> > > > > > > > > cryptodev_enqueue()
> > > > > > > > > with rcu_qsbr_quiescent() or
> > > rcu_qsbr_online()/rcu_qsbr_offline().
> > > > > > > > Yes. I think, it is not a new model. It is same as RCU
> > > > > > > > integration with
> > > > > LPM.
> > > > > > > > Please refer: https://patches.dpdk.org/cover/73673/
> > > > > > >
> > > > > > > I am talking about new working model for crypto-dev
> > > enqueue/dequeue.
> > > > > > > As I said above now it becomes data-plane thread responsibili=
ty to:
> > > > > > > -somehow to obtain pointer to that rcu_qsbr for each
> > > > > > > cryptodev it is
> > > > > using.
> > > > > > > -call rcu sync functions (quiescent/online/offline) on a reg=
ular
> basis.
> > > > > > It is not on regular basis. When data plane comes up, they repo=
rt
> online.
> > > > > > They report quiescent when they are done with critical section
> > > > > > or shared
> > > > > structure.
> > > > >
> > > > > I understand that, but it means all existing apps have to be
> > > > > changed that
> > > way.
> > > > >
> > > > > > All though, there is some dataplane changes involved here, I
> > > > > > don't think, it
> > > > > is major.
> > > > >
> > > > > I still think our goal here should be to make no visible changes
> > > > > to the dataplane.
> > > > > I.E. all necessary data-plane changes need to be hidden inside
> > > > > CB invocation part.
> > > > Please note that this is being implemented using the memory
> > > > reclamation framework documented at
> > > > https://doc.dpdk.org/guides/prog_guide/rcu_lib.html#resource-recla
> > > > mati
> > > > on-framework-for-dpdk
> > > >
> > > > While using RCU there are couple of trade-offs that applications
> > > > have to
> > > consider:
> > > > 1) Performance - reporting the quiescent state too often results
> > > > in performance impact on data plane
> > > > 2) Amount of outstanding memory to reclaim - reporting less often
> > > > results in more outstanding memory to reclaim
> > > >
> > > > Hence, the quiescent state reporting is left to the application.
> > > > The application decides how often it reports the quiescent state
> > > > and has control
> > > over the data plane performance and the outstanding memory to reclaim=
.
> > > >
> > > > When you say "new working model for crypto-dev enqueue/dequeue",
> > > >
> > > > 1) are you comparing these with existing crypto-dev
> > > > enqueue/dequeue
> > > APIs? If yes, these are new APIs, it is not breaking anything.
> > > > 2) are you comparing these with existing call back functions in
> > > > ethdev enqueue/dequeue APIs? If yes, agree that this is a new
> > > > model. But, it is
> > > possible to support what ethdev supports along with the RCU method
> > > used in this patch.
> > >
> > > What I am talking about:
> > >
> > > Existing cryptodev enqueue/dequeue model doesn't require for the
> > > user to manage any RCU QSBR state manually.
> > > I believe that addition of ability to add/remove enqueue/dequeue
> > > callbacks shouldn't change existing working model.
> > > I think that adding/removing such callbacks has to be opaque to the
> > > user DP code and shouldn't require user to change it. Same as we
> > > have now for ethdev callback implementation.
> > The ethdev callback implementation conveniently leaves the problem of
> freeing memory to the user to resolve, it does not handle the issue.
> > Hence, it "looks" to be opaque to the DP code. However, if the
> > application has to implement a safe way to free the call back memory, i=
ts
> DP is affected based on call backs are being used or not.
>=20
> Yes, I think that's big drawback in initial ethdev callback implementatio=
n - it
> simply ignores DP/CP sync problem completely.
> Though I think it is possible to have both here:
> keep callback "opaque" to DP code and provide some sync mechanism
> between DP/CP.
> Hopefully one day we can fix ethdev callbacks too.
The solution we develop can be used in ethdev too.
>=20
> > > I think that forcing DP code to be aware that callbacks are present
> > > or not and to modify its behaviour depending on that nearly voids
> > > the purpose of having callbacks at all.
> > > In that case DP can just invoke callback function directly from it's
> codepath .
> > >
> > > > >
> > > > > >
> > > > > > > Note that now data-plane thread would have to do that always
> > > > > > > - even if there are now callbacks installed for that
> > > > > > > cryptodev queue
> > > right now.
> > > > > > > All that changes behaviour of existing apps and I presume
> > > > > > > would reduce adoption of that fature.
> > > > If I understand this correct, you are talking about a case where
> > > > in the application might be registering/unregistering multiple
> > > > times during its lifetime. In this case, yes, the application
> > > > might be reporting the
> > > quiescent state even when it has not registered the call backs. But,
> > > it has the flexibility to not report it if it implements additional l=
ogic.
> > > > Note that we are assuming that the application has to report
> > > > quiescent state only for using callback functions. Most probably
> > > > the application has
> > > other requirements to use RCU.
> > > > Why not support what is done for ethdev call back functions along
> > > > with
> > > providing RCU method?
> > > >
> > > > > > There is always trade off involved!
> > > > > > In the previous patch, you suggested that some lazy app may
> > > > > > not free up the memory allocated by add cb. For such apps,
> > > > > > this patch has sync mechanism with some additional cost of CP &=
DP
> changes.
> > > > >
> > > > > Sigh, it is not about laziness of the app.
> > > > > The problem with current ethedev cb mechanism and yours V1
> > > > > (which was just a clone of it) - CP doesn't know when it is safe
> > > > > after CB removal to free related memory.
> > > > >
> > > > > > > I still think all this callback mechanism should be totally
> > > > > > > opaque to data-plane threads - user shouldn't change his app
> > > > > > > code depending on would some enqueue/dequeue callbacks be
> > > installed or not.
> > > > > > I am not sure, how that can be implemented with existing RCU
> design.
> > > > >
> > > > > As I said below the simplest way - with calling rcu
> > > > > onine/offline inside CB invocation block.
> > > > > That's why I asked you - did you try that approach and what is
> > > > > the perf numbers?
> > > > > I presume with no callbacks installed the perf change should be
> > > > > nearly
> > > zero.
> > > > >
> > > > > > @Honnappa Nagarahalli, Do you have any suggestions?
> > > > Reporting quiescent state in the call back functions has several
> > > disadvantages:
> > > > 1) it will have performance impacts and the impacts will increase
> > > > as the
> > > number of data plane threads increase.
> > > > 2) It will require additional configuration parameters to control
> > > > how often the quiescent state is reported to control the performanc=
e
> impact.
> > > > 3) Does not take advantage of the fact that most probably the
> > > > application is using RCU already
> > > > 4) There are few difficulties as well, please see below.
> > >
> > > I suggested Abhinandan to use RCU library because it is already
> > > there, and I thought it would be good not to re-implement the wheel.
> > > Though if you feel librte_rcu doesn't match that task - fine, let's
> > > do it without librte_rcu.
> > > After all, what we need here - just an atomic ref count per queue
> > > that we are going to increment at entering and leaving list of
> > > callbacks inside enqueue/dequeue.
> > Ok, looks like I missed the point that a queue is used by a single data=
plane
> thread.
> > Along with ref count increment you need the memory orderings to avoid
> race conditions. These will be the same ones used in RCU.
> > On the control plane, you need to read this counter and poll for the
> counter updates. All this is same cost as in RCU.
>=20
> Agree.
>=20
> > To control the cost, you
> > will have to control the rate of quiescent state reporting and might ha=
ve to
> expose this as a configuration parameter.
> >
> > The other important information you have to consider is if the thread
> > is making any blocking calls, which may be in some other library. The
> > thread is supposed to call rcu_qsbr_thread_offline API before calling a
> blocking call. This allows the RCU to know that this particular thread wi=
ll not
> report quiescent state. The cryptodev library might not have that informa=
tion.
> >
> > If you want to go ahead with this design, you can still use RCU with
> > single thread configuration (like you have mentioned below) and hide th=
e
> details from the application.
>=20
> Yes, same thought here - use rcu_qsbr online/offline for DP part and hi=
de
> actual sync details inside callback code.
We can give it a try. If we can have the performance numbers, we can decide=
about how to control how often these APIs are called on the data plane.
>=20
> >
> > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > That seems quite a big change and I don't think it is
> > > > > > > > > acceptable for most users.
> > > > > > > > > From my perspective adding/installing call-backs to the
> > > > > > > > > dev has to be opaque to the data-plane code.
> > > > > > > > > Also note that different callbacks can be installed by
> > > > > > > > > different entities (libs) and might have no idea about ea=
ch
> other.
> > > > > > > > > That's why I thought it would be better to make all this
> > > > > > > > > RCU stuff internal inside cryptodev:
> > > > > > > > > hide all this rcu_qsbr allocation/setup inside
> > > > > > > > > cryptod somehow to
> > > > > > > obtain pointer to that rcu_qsbr ev init/queue setup
> > > > > > > > > invoke rcu_qsbr_online()/rcu_qsbr_offline() inside
> > > > > > > cryptodev_enqueue().
> > > > This will bring in the application related information such as the
> > > > thread ID
> > > into the library.
> > >
> > > I don't think it would.
> > > Cryptodev enqueue/dequeue functions are not supposed to be thread
> > > safe (same as rx/tx burst).
> > > So we can always use RCU with just one thread(thread_id =3D 0).
> > Agree, the memory that needs to be freed is accessed by a single thread
> on the data plane. RCU with one thread would suffice.
> >
> > > But as I said above - if you feel RCU lib is an overhead here,
> > > that's fine - I think it would be easy enough to do without librte_rc=
u.
> > >
> > > > If the same API calls are being made from multiple data plane
> > > > threads, you need a way to configure that information to the
> > > > library. So, it is better to leave those details for the applicatio=
n to handle.
> > > >
> > > > > > > > I have already tried exploring above stuffs. There are too
> > > > > > > > many
> > > > > constraints.
> > > > > > > > The changes don't fit in, as per RCU design.
> > > > > > >
> > > > > > > Hmm could you be more specific here - what constraints are
> > > > > > > you referring to?
> > > > > > >
> > > > > > > > Moreover, having rcu api under enqueue_burst() will affect
> > > > > > > > the
> > > > > > > performance.
> > > > > > >
> > > > > > > It most likely will. Though my expectation it will affect
> > > > > > > performance only when some callbacks are installed. My
> > > > > > > thought
> > > here:
> > > > > > > callback function by itself will affect cryptdev_enqueue
> > > > > > > performance anyway,
> > > > > > With existing callback design, I have measured the
> > > > > > performance(with
> > > > > crypto perf test) on xeon.
> > > > > > It was almost negligible and same was shared with Declan.
> > > > >
> > > > > I am asking about different thing: did you try alternate
> > > > > approach I described, that wouldn't require changes in the user d=
ata-
> plane code.
> > > > >
> > > > > > That is one of the reasons, I didn't want to add to many
> > > > > > stuffs in to the
> > > > > callback.
> > > > > > The best part of existing design is crypto lib is not much modi=
fied.
> > > > > > The changes are either pushed to CP or DP.
> > > > > >
> > > > > > so adding extra overhead for sync is probably ok here.
> > > > >
> > > > > I think that extra overhead when callbacks are present is
> > > > > expected and probably acceptable.
> > > > > Changes in the upper-layer data-plane code - probably not.
> > > > >
> > > > > > > Though for situation when no callbacks are installed -
> > > > > > > perfomance should be left unaffected (or impact should be as
> > > > > > > small
> > > as possible).
> > > > > > >
> > > > > > > > The changes are more on control plane side, which is one ti=
me.
> > > > > > > > The data plane changes are minimal.
> > > > > > >
> > > > > > > I still think upper layer data-plane code should stay
> > > > > > > unaffected (zero changes).
> > > > > > >
> > > > > > > > >
> >
> > <snip>