From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 22B16A058B; Wed, 25 Mar 2020 21:44:00 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 00F682C6D; Wed, 25 Mar 2020 21:43:59 +0100 (CET) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00042.outbound.protection.outlook.com [40.107.0.42]) by dpdk.org (Postfix) with ESMTP id DEE9D3B5 for ; Wed, 25 Mar 2020 21:43:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lIOYlMVQJQyAfIICUl2uh+ajgyE54dLLnu+WKVbMrqg=; b=ghjcCXUxXHxppgL2icOzbRvhrPgdV0s11VATAIJje5nAJYbT+lUBBGog5NDeK03kVEvAoEnV1tWGKVx+v3Ky1IFHBziLGiEM1KziwlNnYtP1FGiMJ51UDK1ndtkMEmfqHc4N7OrXTsFD6gXYV0eGkqWHJVGdmi2Vz7rJpp255K4= Received: from AM6P193CA0092.EURP193.PROD.OUTLOOK.COM (2603:10a6:209:88::33) by DB6PR0802MB2357.eurprd08.prod.outlook.com (2603:10a6:4:87::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.20; Wed, 25 Mar 2020 20:43:56 +0000 Received: from VE1EUR03FT017.eop-EUR03.prod.protection.outlook.com (2603:10a6:209:88:cafe::36) by AM6P193CA0092.outlook.office365.com (2603:10a6:209:88::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2835.18 via Frontend Transport; Wed, 25 Mar 2020 20:43:56 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT017.mail.protection.outlook.com (10.152.18.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.17 via Frontend Transport; Wed, 25 Mar 2020 20:43:55 +0000 Received: ("Tessian outbound 19f8d550f75c:v48"); Wed, 25 Mar 2020 20:43:55 +0000 X-CR-MTA-TID: 64aa7808 Received: from f108b58fa803.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 2E534D70-3D48-4E01-9F48-E09F9A0D7C74.1; Wed, 25 Mar 2020 20:43:50 +0000 Received: from EUR02-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id f108b58fa803.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 25 Mar 2020 20:43:50 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jI9Ce1GRI1zThxwW/c4oJRkEB9FyT42tmdTnSHKlZpJKDMOmIRCcuZctWaMaLOzIJOgwhuPz+W2kdcXsUdULu4qxTyX/0u9LthHn8fUg2xRaLKib3ZDFtwydsRnnf3Uw/mejC2iMyMjY3wMBIc1d+0Xm6/OBMqGqbTRLzylDaO10qTStfHODdA+kxYmWzgKUZYIL0ofCue5vmKKcMgF02COT7PJglBhEDLdV+x47zSf9sxUwks6eHSr497TqULgo5g0+5/eqBr/oAwwVKRJn3XszrXPZL+icDNPEJUOZYjyxzw0EEGuOqQDNQLHSE7mT2JH9oEoxslIerJVs4nwyNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lIOYlMVQJQyAfIICUl2uh+ajgyE54dLLnu+WKVbMrqg=; b=l+bz3lbUQOkp+Jd6JMdOf8KGujvliuQWLFhnLG+6AKKgAauW7TzOIoNVr9hFS+gfK4WY8MwlJoMW5frw3WyAaWU6k6YCwEIvXZ3Kd3BHLn4HS4s4PvDY6Vzo7YE5kaBZgxBiO+Ik+gmoqoe/qxnwQ3lJFvZ3xUItdsqLaSo+cxOxW/ZwA9p5UHu42iG3lUIRM/J0Z0DCtOiHSFzWFlMGi09pAxSKKgAYwxv3DxVBoFyVg91tHY3iBuUwQgdVCoabrYAab27q6IbjxBqCWmb3gGcfq/ECdwua9KFr0yBOwvftP5M5YeeKktMrSwVQ8c3YaOK1TMzxonl+kb0JVEEeKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lIOYlMVQJQyAfIICUl2uh+ajgyE54dLLnu+WKVbMrqg=; b=ghjcCXUxXHxppgL2icOzbRvhrPgdV0s11VATAIJje5nAJYbT+lUBBGog5NDeK03kVEvAoEnV1tWGKVx+v3Ky1IFHBziLGiEM1KziwlNnYtP1FGiMJ51UDK1ndtkMEmfqHc4N7OrXTsFD6gXYV0eGkqWHJVGdmi2Vz7rJpp255K4= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB5133.eurprd08.prod.outlook.com (20.179.30.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2856.18; Wed, 25 Mar 2020 20:43:48 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::2573:103b:ed96:90bd]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::2573:103b:ed96:90bd%6]) with mapi id 15.20.2835.023; Wed, 25 Mar 2020 20:43:48 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "olivier.matz@6wind.com" CC: Gavin Hu , "dev@dpdk.org" , nd , Honnappa Nagarahalli , nd Thread-Topic: [RFC 1/1] lib/ring: add scatter gather and serial dequeue APIs Thread-Index: AQHV7OTEiOImxZdrNE6vzPRTj2NEk6gvbsUwgAY2BYCAA0L00IABdaYAgB97RYA= Date: Wed, 25 Mar 2020 20:43:47 +0000 Message-ID: References: <20200224203931.21256-1-honnappa.nagarahalli@arm.com> <20200224203931.21256-2-honnappa.nagarahalli@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 8e912734-9303-4046-b0fa-11d63fdc810c.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 7327cff9-c0fe-4e2b-f3ba-08d7d0fd3cc3 x-ms-traffictypediagnostic: VE1PR08MB5133:|VE1PR08MB5133:|DB6PR0802MB2357: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 0353563E2B X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(366004)(39860400002)(136003)(376002)(346002)(478600001)(66476007)(7696005)(66556008)(64756008)(66446008)(66946007)(5660300002)(966005)(76116006)(8676002)(81156014)(81166006)(52536014)(6506007)(4326008)(26005)(9686003)(55016002)(2906002)(316002)(110136005)(54906003)(186003)(8936002)(71200400001)(33656002)(86362001); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5133; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: a6dZzZbO87AQK4t3huNOoyh0sS2p6gRQVQd6n9ZEp49Q8M77nkU1Vp0h8ISkWhO0muJuXSyDh/ggFZ0Jd92lNkxiH9ls18cNKsGkAcDhQY8pEshFdw7E0qNfnPKlX5EVmaBbzef4R29InBMM2L0gDyH13T11PdXa0Va9IanNQk60mr0Cs5h9vnfldfFXHwmP4iE1v7/JFrVzLO7VcHpEqpYLDzSXOHdUhiqqMWNJQDXc1X59UzvSmlImgv+mP8ZRj4Rew5NlfOII8wZGf9Zd55pOeU/NeYpoXvL7M1t79WIwR3UTWCJuSjYzKZrkUr/YvnPjqqBnB/qaSdv5nwcPiiNSuSxfgSRhg5iIcxwC7FL1yRfjHonlN3gvPbBRPH1g0Lydv1RJezZRyw7QoEMjJ4CoZz01w5u7qdhBDoN8ovFsoco2r1kNMaJWjuDRSpHNIzzBOSV8hGjLazbbWMdNDZ9YUD/mfbJzTWMd5w1ahF10Qpk04eHmgnhKLjzm5VrJWSKgGNyejOAafZsYcXQ8qg== x-ms-exchange-antispam-messagedata: oj7CvzhwelXV5OioBbgIuEMgau5Vz/KF//6R/hwRd3heA6sipuSrn8xb+Ny7ntVGTD7IDAYx3R/61WP0E2P8qCeUI1GThFn1raFtXPO9sRqIjp8URmvVBedmCbepy+4HLDLXX8P0Exl9n83ni62GJQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5133 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT017.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(346002)(136003)(376002)(396003)(39860400002)(46966005)(47076004)(82740400003)(356004)(186003)(55016002)(9686003)(33656002)(70586007)(86362001)(6506007)(5660300002)(4326008)(70206006)(7696005)(26826003)(966005)(81156014)(336012)(2906002)(478600001)(8936002)(110136005)(36906005)(52536014)(54906003)(26005)(316002)(8676002)(81166006); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0802MB2357; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; X-MS-Office365-Filtering-Correlation-Id-Prvs: 0af11f9e-19fd-4f09-3b6d-08d7d0fd380b X-Forefront-PRVS: 0353563E2B X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nixAQBUL53NqqXjQtwA/i7vFM15HwVKjTfIb3ypwmjDmnVOwe3GcBsPlhDa59Ai1kau8HNTkML66zMLkgoFarpuTg7GsJp7sJEMwm4UjyCfZFBRYxOumS4SeX1y67iLJ6+JLLbTBLVo2S4R21XKLoLyROrPtvYBT+fdGTWNl7h9dz9172MYwZMXA/MpRsiLt6PxKoyWtjKbh4tAMnht2wm9zAm4Tp4IAAWt2ReSz0nRrUg8aVS+dMrCXe6lzTy8HzJ4Z6KtKL9llw7v8xMjbGiqyO5uXUS/GAgu/YmVsHvDxcXAg5R+MxGeKpdaIL3/DAqCsziZ3ytK030a3X6dGZe2Ooo4TflsqDn8Lq9k6AvZbibtEyC477LzD+HJFyiPzS+GqaUSy6R/KyNHOTTcYtKtc90mfsfbSjF5zSDSYVCFCBPbEOKDdb5Wq6C+byewU1jIM+HjQg9B2ce4z2KtG44vOFC4re3LFwMNycmOVuKqSVfTgpzHGbjei/VICBVTwrvbabgYE4fqZq6RRHiOUGk96kSBhlO/nH3shYV3vAoBUkH8AAZgzetcVd4PQSbIrNjsOwwiD8V18v860jvf/tfGc8ABfkdMLw/IAS/f+bD4= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2020 20:43:55.9301 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7327cff9-c0fe-4e2b-f3ba-08d7d0fd3cc3 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0802MB2357 Subject: Re: [dpdk-dev] [RFC 1/1] lib/ring: add scatter gather and serial dequeue APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Archived-At: List-Archive: >=20 > > > > > > > > > +/** > > > > > > + * @internal Reserve ring elements to enqueue several objects > > > > > > +on the ring > > > > > > + * > > > > > > + * @param r > > > > > > + * A pointer to the ring structure. > > > > > > + * @param esize > > > > > > + * The size of ring element, in bytes. It must be a multiple= of 4. > > > > > > + * This must be the same value used while creating the ring. > > > Otherwise > > > > > > + * the results are undefined. > > > > > > + * @param n > > > > > > + * The number of elements to reserve in the ring. > > > > > > + * @param behavior > > > > > > + * RTE_RING_QUEUE_FIXED: Reserve a fixed number of > elements > > > from a > > > > > ring > > > > > > + * RTE_RING_QUEUE_VARIABLE: Reserve as many elements as > > > possible > > > > > from ring > > > > > > + * @param is_sp > > > > > > + * Indicates whether to use single producer or multi-produce= r > reserve > > > > > > + * @param old_head > > > > > > + * Producer's head index before reservation. > > > > > > + * @param new_head > > > > > > + * Producer's head index after reservation. > > > > > > + * @param free_space > > > > > > + * returns the amount of space after the reserve operation h= as > > > finished. > > > > > > + * It is not updated if the number of reserved elements is z= ero. > > > > > > + * @param dst1 > > > > > > + * Pointer to location in the ring to copy the data. > > > > > > + * @param n1 > > > > > > + * Number of elements to copy at dst1 > > > > > > + * @param dst2 > > > > > > + * In case of ring wrap around, this pointer provides the lo= cation > to > > > > > > + * copy the remaining elements. The number of elements to co= py > at > > > this > > > > > > + * location is equal to (number of elements reserved - n1) > > > > > > + * @return > > > > > > + * Actual number of elements reserved. > > > > > > + * If behavior =3D=3D RTE_RING_QUEUE_FIXED, this will be 0 o= r n only. > > > > > > + */ > > > > > > +static __rte_always_inline unsigned int > > > > > > +__rte_ring_do_enqueue_elem_reserve(struct rte_ring *r, > > > > > > +unsigned int esize, > > > > > > > > > > > > > > > I do understand the purpose of reserve, then either commit/abort > > > > > for serial sync mode, but what is the purpose of non-serial > > > > > version of > > > reserve/commit? > > > > In RCU, I have the need for scatter-gather feature. i.e. the data > > > > in the ring element is coming from multiple sources ('token' is > > > > generated by the RCU library and the application provides > > > > additional data). If I do not > > > provide the reserve/commit, I need to introduce an intermediate > > > memcpy to get these two data contiguously to copy to the ring > > > element. The sequence is 'reserve(1), memcpy1, mempcy2, commit(1)'. > > > > Hence, you do not see the abort API for the enqueue. > > > > > > > > > In serial MP/MC case, after _reserve_(n) you always have to do > > > > > _commit_(n) - you can't reduce number of elements, or do _abort_. > > > > Agree, the intention here is to provide the scatter/gather feature. > > > > > > > > > Again you cannot avoid memcpy(n) here anyhow. > > > > > So what is the point of these functions for non-serial case? > > > > It avoids an intermediate memcpy when the data is coming from > > > > multiple > > > sources. > > > > > > Ok, I think I understand what was my confusion: > > Yes, the following understanding is correct. > > > > > Your intention: > > > 1) reserve/commit for both serial and non-serial mode - > > > to allow user get/set contents of the ring manually and avoid > > > intermediate load/stores. > > > 2) abort only for serial mode. > > > > > > My intention: > > > 1) commit/reserve/abort only for serial case > > > (as that's the only mode where we can commit less > > > then was reserved or do abort). > > I do not know if there is a requirement on committing less than reserve= d. >=20 > From my perspective, that's a necessary part of peek functionality. > revert/abort function you introduced below is just one special case of it= . > Having just abort is enough when you processing elements in the ring one = by > one, but not sufficient if someone would try to operate in bulks. > Let say you read (reserved) N objects from the ring, inspected them and > found that first M ( remain. Agree, it makes sense from a dequeue perspective. Does it make sense from e= nqueue perspective? >=20 > > I think, if the size of commit is not known during reservation, may be > > the reservation can be delayed till it is known. >=20 > In some cases, you do know how much you'd like to commit, but you can't > guarantee that you can commit that much, till you inspect contents of > reserved elems. The above comment was from enqueue perspective. >=20 > > If there is no requirement to commit less than reserved, then I do not = see a > need for serial APIs for enqueue operation. >=20 > > > > > 2) get/set of ring contents are done as part of either > > > reserve(for dequeue) or commit(for enqueue) API calls > > > (no scatter-gather ability). > > > > > > I still think that this new API you suggest creates too big exposure > > > of ring internals, and makes it less 'safe-to-use': > > > - it provides direct access to contents of the ring. > > > - user has to specify head/tail values directly. > > It is some what complex. But, with the support of user defined element > > size, I think it becomes necessary to support scatter gather feature (s= ince it > is not a single pointer that will be stored). >=20 > I suppose to see the real benefit from scatter-gather, we need a scenario > where there are relatively big elems in the ring (32B+ or so), plus > enqueue/dequeue done in bulks. > If you really envision such use case - I am ok to consider scatter-gathe= r API > too, but I think it shouldn't be the only available API for serial mode. > Might be we can have 'normal' enqueue/dequeue API for serial mode (actual > copy done internally in ring functions, head/tail values are not exposed > directly), plus SG API as addon for some special cases. I will try to run some benchmarks and take a decision on if SG makes an imp= act on RCU defer APIs. >=20 > > > > > > So in case of some programmatic error in related user code, there > > > are less chances it could be catch-up by API, and we can easily > > > end-up with silent memory corruption and other nasty things that > > > would be hard to catch/reproduce. > > > > > > That makes me wonder how critical is this scatter-gather ability in > > > terms of overall RCU performance? > > > Is the gain provided really that significant, especially if you'll > > > update the ring by one element at a time? > > For RCU, it is 64b token and the size of the user data. Not sure how mu= ch > difference it will make. > > I can drop the scatter gather requirement for now. > > > > > > > > > > > > > > > > > > > BTW, I think it would be good to have serial version of _enqueue_= too. > > > > If there is a good use case, they should be provided. I did not > > > > come across a > > > good use case. > > > > > > > > > > > > > > > + unsigned int n, enum rte_ring_queue_behavior > behavior, > > > > > > + unsigned int is_sp, unsigned int *old_head, > > > > > > + unsigned int *new_head, unsigned int *free_space, > > > > > > + void **dst1, unsigned int *n1, void **dst2) > > > > > > > > > > I do understand the intention to avoid memcpy(), but proposed > > > > > API seems overcomplicated, error prone, and not very convenient f= or > the user. > > > > The issue is the need to handle the wrap around in ring storage arr= ay. > > > > i.e. when the space is reserved for more than 1 ring element, the > > > > wrap > > > around might happen. > > > > > > > > > I don't think that avoiding memcpy() will save us that many > > > > > cycles here, so > > > > This depends on the amount of data being copied. > > > > > > > > > probably better to keep API model a bit more regular: > > > > > > > > > > n =3D rte_ring_mp_serial_enqueue_bulk_reserve(ring, num, > &free_space); ... > > > > > /* performs actual memcpy(), m<=3Dn */ > > > > > rte_ring_mp_serial_enqueue_bulk_commit(ring, obj, m); > > > > These do not take care of the wrap-around case or I am not able to > > > understand your comment. > > > > > > I meant that serial_enqueue_commit() will do both: > > > actual copy of elements to the ring and tail update (no > > > Scatter-Gather), see above. > > RCU does not require the serial enqueue APIs, do you have any use case? >=20 > I agree that serial dequeue seems to have more usages then enqueue. > Though I still can name at least two cases for enqueue, from top of my he= ad: > 1. serial mode (both enqueue/dequeue) helps to mitigate ring slowdown > overcommitted scenarios, see RFC I submitted: > http://patches.dpdk.org/cover/66001/ > 2. any intermediate node when you have pop/push from/to some external > queue, and enqueue/dequeue to/from the ring, would like to avoid any > elem drops in between, and by some reason don't want your own > intermediate bufferization. > Let say: > dequeue_from_ring -> tx_burst/cryptodev_enqueue > rx_burst/cryptodev_dequeue -> enqueue_to_ring >=20 > Plus as enqueue/dequeue are sort of mirror, I think it is good to have bo= th > identical. Ok, agreed. I think we need to allow for combination of APIs to be used. i.= e. MP enqueue and serialization on dequeue. >=20 >=20