From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3146DA04BC; Sat, 10 Oct 2020 00:55:00 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6D9491D654; Sat, 10 Oct 2020 00:54:58 +0200 (CEST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50065.outbound.protection.outlook.com [40.107.5.65]) by dpdk.org (Postfix) with ESMTP id 6425C1D583 for ; Sat, 10 Oct 2020 00:54:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qCntOB97Fh2UFG0LxnNvYc0/RM96jwbvRj5WpzPxEhI=; b=EQWA29H2neeKKVdBqGgNXmQGzT6BDWRpFO5B/Gm6sNBMhRzINTJk1EdUzQPCd3GJ1pSMhL3hBZG8Vt5tGOtsZ9IqOjunyMqy6D2emviXi7b8jDVn+wm09nhLyWDGysLz4bitWqBPCLw/65ykQy8+CU8WJcih3nMItikaMZxwnPg= Received: from MR2P264CA0123.FRAP264.PROD.OUTLOOK.COM (2603:10a6:500:30::15) by AS8PR08MB6181.eurprd08.prod.outlook.com (2603:10a6:20b:29a::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.22; Fri, 9 Oct 2020 22:54:53 +0000 Received: from VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com (2603:10a6:500:30:cafe::de) by MR2P264CA0123.outlook.office365.com (2603:10a6:500:30::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.21 via Frontend Transport; Fri, 9 Oct 2020 22:54:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT035.mail.protection.outlook.com (10.152.18.110) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.23 via Frontend Transport; Fri, 9 Oct 2020 22:54:52 +0000 Received: ("Tessian outbound 7a6fb63c1e64:v64"); Fri, 09 Oct 2020 22:54:52 +0000 X-CR-MTA-TID: 64aa7808 Received: from 2e0b18db351d.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 00081D7F-0332-4EF4-9BA5-580800B58DFB.1; Fri, 09 Oct 2020 22:54:46 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 2e0b18db351d.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 09 Oct 2020 22:54:46 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L/2HXWvvQ09kpQYOHm4EyVBk6W41FxlCCm/00DIZojf8Pyp/0mBzAI+i2xDKjlaHJ+rhd8bORtIaTMBJSm3EJ4OZjaXbgAm98TSxH4FPhpeS45pNdyfl9DCyelKW40a0scAFhGWyOlfg0LH9DY7gUfA9Ht8+CspppQ/i7p01tszkC0k0JfQBKZjhlj4cMSJTf4UF0/MRjnly9XjOVuqgijCuNDUTomErojOP0ZVXPuIQgnATicsyoD1JSpwi7vrB7aVnxrURqquoz5pycH9SdNilY0WMovKNbPIlSFFkJh0s6fqQXBaRA7qnjjkODQ6YEd5zc1yAnWw26VutEJrA+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qCntOB97Fh2UFG0LxnNvYc0/RM96jwbvRj5WpzPxEhI=; b=NpHtk3No1bxRUw7+5GjW7AaCrrq32WKB+UnU95KX+7ea3Hi1BDpPyqGJaWW5IXnmF/74eas/l1OZdDmcaF5yXkRVjx+SjQSfztgOsxvS2lltccxvsKdHH/FOSAuaHIyZWm/nwPJLEnhMNA5A6aj3V1TRAtnPJQ/pBlaa1Jj1c1F7iNdlKJEx1SlNCqjjIBMtqFDOoyktUat+3kA3ZrqBfmOJMK/FNXUWB/9M5CT5EZLHVJl5Z5L2zAeeT2xGLbU6n2G21cQfuSDiEC6nu9eikY+XuF3qRo8j2DTonkyY0q5mHVNDGBhNUjEg2Nc42k/i24s0e9o/lhy3HsSE1HWqFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qCntOB97Fh2UFG0LxnNvYc0/RM96jwbvRj5WpzPxEhI=; b=EQWA29H2neeKKVdBqGgNXmQGzT6BDWRpFO5B/Gm6sNBMhRzINTJk1EdUzQPCd3GJ1pSMhL3hBZG8Vt5tGOtsZ9IqOjunyMqy6D2emviXi7b8jDVn+wm09nhLyWDGysLz4bitWqBPCLw/65ykQy8+CU8WJcih3nMItikaMZxwnPg= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by DBBPR08MB4632.eurprd08.prod.outlook.com (2603:10a6:10:db::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3455.24; Fri, 9 Oct 2020 22:54:45 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::7814:9c1:781f:475d]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::7814:9c1:781f:475d%4]) with mapi id 15.20.3455.027; Fri, 9 Oct 2020 22:54:45 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , Olivier Matz CC: "dev@dpdk.org" , "david.marchand@redhat.com" , nd , Honnappa Nagarahalli , nd Thread-Topic: [RFC v2 1/1] lib/ring: add scatter gather APIs Thread-Index: AQHWnIPUNOw1POr43kiYFPhsEItTcamOCuOggADZGwCAAAjhgIAA1nNg Date: Fri, 9 Oct 2020 22:54:45 +0000 Message-ID: References: <20200224203931.21256-1-honnappa.nagarahalli@arm.com> <20201006132905.46205-1-honnappa.nagarahalli@arm.com> <20201006132905.46205-2-honnappa.nagarahalli@arm.com> <20201007082739.GL21395@platinum> <20201009073342.GZ21395@platinum> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 92D0548E9EB08941B6CC95F158E82A0F.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [107.77.222.77] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 14e84e11-b02c-445b-90ad-08d86ca65592 x-ms-traffictypediagnostic: DBBPR08MB4632:|AS8PR08MB6181: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: JCGu6fPWoS4EZiMSUUnBYwzv+kJ54QWUIMfKh95rf+lWWZN0MbBC4DPYu6oN6BVDZN948LY3DitKttTlHWKQDR1zPZi6QV+wLJ1WAt59gw+hXr69kkTWXgNmYSLr/BBCH2DYvJmGSJzTwyEi6qfw0OgSGfoxj3Ch5F7Co1WLDlp1JGagwmioeA2H/GLqfwZCV/Dj+4aN602DXO8wYq64jywqqIp0pnMt1G47v49sCfQp1ekfzQwQtdsCZZzH7RpT8jOI3wUKSR71TvBtcuYC9ZpfLQlYTnOwdNFV/3JhGMsEAbTd4/aEpY2Ay5CmjaNbDdPuu5Jr5slzVoxJq71OeM2u0wl+ieV2x2ALvd2GnpPMYulRM84m5SC10boaMsLQSna7L7N2rUVB5Zt+9pwIrw== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(366004)(136003)(376002)(346002)(396003)(9686003)(66476007)(66556008)(66446008)(66946007)(64756008)(316002)(4326008)(76116006)(54906003)(30864003)(33656002)(110136005)(2906002)(5660300002)(52536014)(86362001)(7696005)(6506007)(26005)(186003)(55016002)(83080400001)(71200400001)(83380400001)(478600001)(8936002)(966005)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: AfPOZS+K3EcTxMdeDAulh2x6xfdEyNXsCDKu0YmUJmqLAn8zKoxJp9b1xS6WBobblGH360INLJR79tYvn3cNXcANZ/DpBlu/ut0iI13opOE36vbT1rRp2CrzlicAhh6yPzyRmSiUubzW3q+e6e8zX6hDF6Q/M4BvFp6EcekQaeM5Nr9C4FgbmRjdyWUmKip6TMZaHje05n8ZyFVUmpwhGP3s1OQhYb6ULh0sMv3KeS5hfAM4lsS/F/dafk4XHn1EJfAbjNs+LT2f++3kKL7DT4l9CfWDZlIsOzO11CdT2/MTAtjYcBvc1ggx4hsGjEmXTyvVo2J2T9b5SGnqI8rFTqrnvAp9wnYLDOHS94nN02PQTp29si8LFPKAznW+PtITcYQVUgWFyt0AY0Spj2HjBB9wdpVg/5BIXHPos4OGjUfWJ5jk3c5O+ZpbyAu++h0tXuz69tNTIsLwEsMMS2W4ZYb2RZ1Vy55UJokR+mEgwJCm6BT2xsSW1/KvxIjEDLtqvb4tJ4lk6hxDWzLByYSDh8297VcA7hqt9VNTIe/EQvpjxkva7zfkc9R7olsVgkalZbHxx6RzFqqZU6c0xXakEV/rIqdaGBejRcL8cyJHERpt97VY4Ln7ldON3w47MuLgR8Ja3SlQQCVRDnDhad5Jhw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4632 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com X-MS-Office365-Filtering-Correlation-Id-Prvs: 901aba39-9db8-431b-cf16-08d86ca65128 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IVN4cWy7y+IYN6yVHpVOxtsIi2UY2PD6Bfcc/purLYN8wkN7dT16PS9lZ4cqm1C8wr6pJWBp2kdoo5M+FfHY/zhwkJgxfR/rZxJn2VTulKyLRQ5/wo56EgiLnBAKBUC7bfTeyC3cGWmTDrS2eu2igic8aP+/48BLjY9F7O1TZK3BmrfwTft0B5o0JEbFdoEvi2GiHPOpM0QIru/BwtZ90EtDMPxMrtfVymvSOmaj5kQf+utgT0SBveZ6fMVj90PgcuuWejzvhZ57TTca84GnM0nfqmz2pquKT4TB0i1jVKHBgs/2Zz6tFkhEtpHR8DRvENU/wdTwQv/2MwOB85zPESCN88McFVvGlSpUencKky3b5VjTqV6PYS0vpkafzkXYJvY1KL0KNXCyJzAdqVR6eF9+L0YLZSY8ZrIBWncWZ5qJ05eDtMJ0y7SfV1geB/HeClVrp0tdthO1txBn2LmkHmYGmusSR7kxyc5uWYwNVpg= X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(4636009)(346002)(376002)(136003)(39860400002)(396003)(46966005)(5660300002)(47076004)(82310400003)(70586007)(70206006)(52536014)(2906002)(83380400001)(82740400003)(186003)(9686003)(336012)(8936002)(8676002)(55016002)(81166007)(110136005)(83080400001)(30864003)(4326008)(26005)(356005)(7696005)(33656002)(86362001)(54906003)(6506007)(316002)(966005)(478600001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2020 22:54:52.7507 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 14e84e11-b02c-445b-90ad-08d86ca65592 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT035.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6181 Subject: Re: [dpdk-dev] [RFC v2 1/1] lib/ring: add scatter gather APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > Hi Honnappa, > > > > > > > > From a quick walkthrough, I have some questions/comments, please > > > > see below. > > > Hi Olivier, appreciate your input. > > > > > > > > > > > On Tue, Oct 06, 2020 at 08:29:05AM -0500, Honnappa Nagarahalli wrot= e: > > > > > Add scatter gather APIs to avoid intermediate memcpy. Use cases > > > > > that involve copying large amount of data to/from the ring can > > > > > benefit from these APIs. > > > > > > > > > > Signed-off-by: Honnappa Nagarahalli > > > > > > > > > > --- > > > > > lib/librte_ring/meson.build | 3 +- > > > > > lib/librte_ring/rte_ring_elem.h | 1 + > > > > > lib/librte_ring/rte_ring_peek_sg.h | 552 > > > > > +++++++++++++++++++++++++++++ > > > > > 3 files changed, 555 insertions(+), 1 deletion(-) create mode > > > > > 100644 lib/librte_ring/rte_ring_peek_sg.h > > > > > > > > > > diff --git a/lib/librte_ring/meson.build > > > > > b/lib/librte_ring/meson.build index 31c0b4649..377694713 100644 > > > > > --- a/lib/librte_ring/meson.build > > > > > +++ b/lib/librte_ring/meson.build > > > > > @@ -12,4 +12,5 @@ headers =3D files('rte_ring.h', > > > > > 'rte_ring_peek.h', > > > > > 'rte_ring_peek_c11_mem.h', > > > > > 'rte_ring_rts.h', > > > > > - 'rte_ring_rts_c11_mem.h') > > > > > + 'rte_ring_rts_c11_mem.h', > > > > > + 'rte_ring_peek_sg.h') > > > > > diff --git a/lib/librte_ring/rte_ring_elem.h > > > > > b/lib/librte_ring/rte_ring_elem.h index 938b398fc..7d3933f15 > > > > > 100644 > > > > > --- a/lib/librte_ring/rte_ring_elem.h > > > > > +++ b/lib/librte_ring/rte_ring_elem.h > > > > > @@ -1079,6 +1079,7 @@ rte_ring_dequeue_burst_elem(struct > > > > > rte_ring *r, void *obj_table, > > > > > > > > > > #ifdef ALLOW_EXPERIMENTAL_API > > > > > #include > > > > > +#include > > > > > #endif > > > > > > > > > > #include > > > > > diff --git a/lib/librte_ring/rte_ring_peek_sg.h > > > > > b/lib/librte_ring/rte_ring_peek_sg.h > > > > > new file mode 100644 > > > > > index 000000000..97d5764a6 > > > > > --- /dev/null > > > > > +++ b/lib/librte_ring/rte_ring_peek_sg.h > > > > > @@ -0,0 +1,552 @@ > > > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > > > + * > > > > > + * Copyright (c) 2020 Arm > > > > > + * Copyright (c) 2007-2009 Kip Macy kmacy@freebsd.org > > > > > + * All rights reserved. > > > > > + * Derived from FreeBSD's bufring.h > > > > > + * Used as BSD-3 Licensed with permission from Kip Macy. > > > > > + */ > > > > > + > > > > > +#ifndef _RTE_RING_PEEK_SG_H_ > > > > > +#define _RTE_RING_PEEK_SG_H_ > > > > > + > > > > > +/** > > > > > + * @file > > > > > + * @b EXPERIMENTAL: this API may change without prior notice > > > > > + * It is not recommended to include this file directly. > > > > > + * Please include instead. > > > > > + * > > > > > + * Ring Peek Scatter Gather APIs > > > > > > > > I am not fully convinced by the API name. To me, "scatter/gather" > > > > is associated to iovecs, like for instance in [1]. The wikipedia > > > > definition [2] may be closer though. > > > > > > > > [1] > > > > > https://www.gnu.org/software/libc/manual/html_node/Scatter_002dGat > > > > he > > > > r.html > > > > [2] > > > > https://en.wikipedia.org/wiki/Gather-scatter_(vector_addressing) > > > The way I understand scatter-gather is, the data to be sent to > > > something (like a device) is coming from multiple sources. It would > > > require > > copying to put the data together to be contiguous. If the device > > supports scatter-gather, such copying is not required. The device can > collect data from multiple locations and make it contiguous. > > > > > > In the case I was looking at, one part of the data was coming from > > > the user of the API and another was generated by the API itself. If > > these two pieces of information have to be enqueued as a single object > > on the ring, I had to create an intermediate copy. But by exposing the = ring > memory to the user, the intermediate copy is avoided. Hence I called it > scatter-gather. > > > > > > > > > > > What about "zero-copy"? > > > I think no-copy (nc for short) or user-copy (uc for short) would > > > convey the meaning better. These would indicate that the rte_ring > > > APIs are > > not copying the objects and it is left to the user to do the actual cop= y. > > > > > > > > > > > Also, the "peek" term looks also a bit confusing to me, but it > > > > existed before your patch. I understand it for dequeue, but not for > enqueue. > > > I kept 'peek' there because the API still offers the 'peek' API > > > capabilities. I am also not sure on what 'peek' means for enqueue > > > API. The > > enqueue 'peek' API was provided to be symmetric with dequeue peek API. > > > > > > > > > > > Or, what about replacing the existing experimental peek API by this= one? > > > > They look quite similar to me. > > > I agree, scatter gather APIs provide the peek capability and the no-c= opy > benefits. > > > Konstantin, any opinions here? >=20 > Sorry, didn't have time yet, to look at this RFC properly. > Will try to do it next week, as I understand that's for 21.02 anyway? This is committed for 20.11. We should be able to get into RC2. >=20 > > > > > > > > > + * Introduction of rte_ring with scatter gather serialized > > > > > + producer/consumer > > > > > + * (HTS sync mode) makes it possible to split public > > > > > + enqueue/dequeue API > > > > > + * into 3 phases: > > > > > + * - enqueue/dequeue start > > > > > + * - copy data to/from the ring > > > > > + * - enqueue/dequeue finish > > > > > + * Along with the advantages of the peek APIs, these APIs > > > > > + provide the ability > > > > > + * to avoid copying of the data to temporary area. > > > > > + * > > > > > + * Note that right now this new API is available only for two sy= nc > modes: > > > > > + * 1) Single Producer/Single Consumer (RTE_RING_SYNC_ST) > > > > > + * 2) Serialized Producer/Serialized Consumer > (RTE_RING_SYNC_MT_HTS). > > > > > + * It is a user responsibility to create/init ring with > > > > > + appropriate sync > > > > > + * modes selected. > > > > > + * > > > > > + * Example usage: > > > > > + * // read 1 elem from the ring: > > > > > > > > Comment should be "prepare enqueuing 32 objects" > > > > > > > > > + * n =3D rte_ring_enqueue_sg_bulk_start(ring, 32, &sgd, NULL); > > > > > + * if (n !=3D 0) { > > > > > + * //Copy objects in the ring > > > > > + * memcpy (sgd->ptr1, obj, sgd->n1 * sizeof(uintptr_t)); > > > > > + * if (n !=3D sgd->n1) > > > > > + * //Second memcpy because of wrapround > > > > > + * n2 =3D n - sgd->n1; > > > > > + * memcpy (sgd->ptr2, obj[n2], n2 * sizeof(uintptr_t)); > > > > > > > > Missing { } > > > > > > > > > + * rte_ring_dequeue_sg_finish(ring, n); > > > > > > > > Should be enqueue > > > > > > > Thanks, will correct them. > > > > > > > > + * } > > > > > + * > > > > > + * Note that between _start_ and _finish_ none other thread can > > > > > + proceed > > > > > + * with enqueue(/dequeue) operation till _finish_ completes. > > > > > + */ > > > > > + > > > > > +#ifdef __cplusplus > > > > > +extern "C" { > > > > > +#endif > > > > > + > > > > > +#include > > > > > + > > > > > +/* Rock that needs to be passed between reserve and commit APIs > > > > > +*/ struct rte_ring_sg_data { > > > > > + /* Pointer to the first space in the ring */ > > > > > + void **ptr1; > > > > > + /* Pointer to the second space in the ring if there is wrap- > around */ > > > > > + void **ptr2; > > > > > + /* Number of elements in the first pointer. If this is equal to > > > > > + * the number of elements requested, then ptr2 is NULL. > > > > > + * Otherwise, subtracting n1 from number of elements > requested > > > > > + * will give the number of elements available at ptr2. > > > > > + */ > > > > > + unsigned int n1; > > > > > +}; > > > > > > > > Would it be possible to simply return the offset instead of this st= ructure? > > > > The wrap could be managed by a __rte_ring_enqueue_elems() > function. > > > Trying to use __rte_ring_enqueue_elems() will force temporary copy. > See below. > > > > > > > > > > > I mean something like this: > > > > > > > > uint32_t start; > > > > n =3D rte_ring_enqueue_sg_bulk_start(ring, 32, &start, NULL); > > > > if (n !=3D 0) { > > > > /* Copy objects in the ring. */ > > > > __rte_ring_enqueue_elems(ring, start, obj, sizeof(uintptr_t), > > > > n); > > > For ex: 'obj' here is temporary copy. > > > > > > > rte_ring_enqueue_sg_finish(ring, n); > > > > } > > > > > > > > It would require to slightly change __rte_ring_enqueue_elems() to > > > > support to be called with prod_head >=3D size, and wrap in that cas= e. > > > > > > > The alternate solution I can think of requires 3 things 1) the base > > > address of the ring 2) Index to where to copy 3) the mask. With > > > these 3 > > things one could write the code like below: > > > for (i =3D 0; i < n; i++) { > > > ring_addr[(index + i) & mask] =3D obj[i]; // ANDing with mask will t= ake > care of wrap-around. > > > } > > > > > > However, I think this does not allow for passing the address in the > > > ring to another function/API to copy the data (It is possible, but > > > the user > > has to calculate the actual address, worry about the wrap-around, secon= d > pointer etc). > > > > > > The current approach hides some details and provides flexibility to t= he > application to use the pointer the way it wants. > > > > I agree that doing the access + masking manually looks too complex. > > > > However I'm not sure to get why using __rte_ring_enqueue_elems() > would > > result in an additional copy. I suppose the object that you want to > > enqueue is already stored somewhere? I think this is the key. The object is not stored any where (yet), it is ge= tting generated. When it is generated, it should get stored directly into t= he ring. I have provided some examples below. > > > > For instance, let's say you have 10 objects to enqueue, located at > > different places: > > > > void *obj_0_to_3 =3D ; > > void *obj_4_to_7 =3D ...; > > void *obj_8_to_9 =3D ...; > > uint32_t start; > > n =3D rte_ring_enqueue_sg_bulk_start(ring, 10, &start, NULL); > > if (n !=3D 0) { > > __rte_ring_enqueue_elems(ring, start, obj_0_to_3, > > sizeof(uintptr_t), 4); > > __rte_ring_enqueue_elems(ring, start + 4, obj_4_to_7, > > sizeof(uintptr_t), 4); > > __rte_ring_enqueue_elems(ring, start + 8, obj_8_to_9, > > sizeof(uintptr_t), 2); > > rte_ring_enqueue_sg_finish(ring, 10); > > } > > >=20 >=20 > As I understand, It is not about different objects stored in different pl= aces, it > is about: > a) object is relatively big (16B+ ?) > b) You compose objects from values stored in few different places. >=20 > Let say you have: > struct elem_obj {uint64_t a; uint32_t b, c;}; >=20 > And then you'd like to copy 'a' value from one location, 'b' from second,= and > 'c' from third one. >=20 > Konstantin >=20 I think there are multiple use cases. Some I have in mind are: 1) Code without this patch: struct rte_mbuf *pkts_burst[32]; /* Create ring with sync type RTE_RING_SYNC_ST or RTE_RING_SYNC_MT_HTS */ /* Pkt I/O core polls packets from the NIC, pkts_burst is the temporary sto= re */ nb_rx =3D rte_eth_rx_burst(portid, queueid, pkts_burst, 32); /* Provide packets to the packet processing cores */ rte_ring_enqueue_burst(ring, pkts_burst, 32, &free_space); Code with the patch: /* Create ring with sync type RTE_RING_SYNC_ST or RTE_RING_SYNC_MT_HTS */ /* Reserve space on the ring */ n =3D rte_ring_enqueue_sg_burst_start(ring, 32, &sgd, NULL); /* Pkt I/O core polls packets from the NIC */ if (n =3D=3D 32) nb_rx =3D rte_eth_rx_burst(portid, queueid, sgd->ptr1, 32); else nb_rx =3D rte_eth_rx_burst(portid, queueid, sgd->ptr1, sgd->n1); /* Provide packets to the packet processing cores */ /* Temporary storage 'pkts_burst' is not required */ rte_ring_enqueue_sg_finish(ring, nb_rx); 2) This is same/similar to what Konstantin mentioned Code without this patch: struct elem_obj {uint64_t a; uint32_t b, c;}; struct elem_obj obj; /* Create ring with sync type RTE_RING_SYNC_ST or RTE_RING_SYNC_MT_HTS */ obj.a =3D rte_get_a(); obj.b =3D rte_get_b(); obj.c =3D rte_get_c(); /* obj is the temporary storage and results in memcpy in the following call= */ rte_ring_enqueue_elem(ring, sizeof(struct elem_obj), 1, &obj, NULL); Code with the patch: struct elem_obj *obj; /* Reserve space on the ring */ n =3D rte_ring_enqueue_sg_bulk_elem_start(ring, sizeof(elem_obj), 1, &sgd, = NULL); obj =3D (struct elem_obj *)sgd->ptr1; obj.a =3D rte_get_a(); obj.b =3D rte_get_b(); obj.c =3D rte_get_c(); /* obj is not a temporary storage */ rte_ring_enqueue_sg_elem_finish(ring, n);