From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4521BA056A; Mon, 2 Mar 2020 19:20:54 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8CD2C2C02; Mon, 2 Mar 2020 19:20:53 +0100 (CET) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id A06683B5 for ; Mon, 2 Mar 2020 19:20:51 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Mar 2020 10:20:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,507,1574150400"; d="scan'208";a="438375231" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by fmsmga005.fm.intel.com with ESMTP; 02 Mar 2020 10:20:49 -0800 Received: from orsmsx603.amr.corp.intel.com (10.22.229.16) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 2 Mar 2020 10:20:49 -0800 Received: from orsmsx603.amr.corp.intel.com (10.22.229.16) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 2 Mar 2020 10:20:48 -0800 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by orsmsx603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Mon, 2 Mar 2020 10:20:48 -0800 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.177) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 2 Mar 2020 10:20:48 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GRf5YNHHIpa2GAjRH7k2QfMt5PYlYjQBgPHDGa4/Xkagnz9bW36fdjlCSVK2Z0NYUWz6SFuVGafwi/cBD9z4lcH4E1PFWgr/MvusmtRZFXVndY+1FZUIgdKv21W5DtuwHzvDmzzzPkKw5u3hSY1oDtBYFxkAUYrS2aFDHvGpCLnVinBoJpOK3yzBOxtKwvnj76R3A+5+WRIFIBPaZhcwPwn/Yaj5gKOQX7JdwCmBIvb3ML3geh+nAQVAqM9IqSZPx0aCx8+JpdS+4dflfdy1LZlgX9CQ4csphYruUsCbtujkfqNXxXrPW2a7CZkUspjwbUtB0Uz/esGpW3z4cX8zDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YHKzSWG0nRJbRR5GnilwV/nEmqo0WFBFf2Q1L23uF+Y=; b=mctBbrZvnUEojJRoe0RB+fxgmzB1/pyogVuo0KU706T4ABoiAGiQ8PGXVQnZXtenrk1XczWzHN4wj78gu0LgnlbKpk5OmkMofSpoPkSTlS+eXPjXcVy/iKyvAInUAmgK89YpT5ya7iIz7zSEl+SaIrTZTXVOCKuW5/2Mxp+nfWBxA97QhdUk5c3Ek13p9EB3ycbPa7GIBzwPCARFvSG/tDVBxL74ovPJ4uw/t8u9t6ex7rnzBy+4SKZqD2duYGTwfFfMxhguYqPZdNc4yyxHkZtWPDuGmFUH5pDfQKO0hPqp+hHzXFLsJyLzr1rY0snd5qbM9zIDURWVI1TUZGzr4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YHKzSWG0nRJbRR5GnilwV/nEmqo0WFBFf2Q1L23uF+Y=; b=wxDE2W4IsExR0Xmy97pEJbik1q20FxWjOsZdKVpJcOt0aRh0l93OoKlUIPNPLRpDGLRrdG3JjZlL0Z2xhCz9xudpyVDoXxAVyeytOPw0H1bz35NDoPMblEbW/fLesICPQ34Nhge3R0bP+epoPRz6Plb0n2ELoxlAU3dC4mMdlG4= Received: from SN6PR11MB2558.namprd11.prod.outlook.com (2603:10b6:805:5d::19) by SN6PR11MB2701.namprd11.prod.outlook.com (2603:10b6:805:54::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2772.18; Mon, 2 Mar 2020 18:20:47 +0000 Received: from SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::395e:eb75:6ab7:2ba5]) by SN6PR11MB2558.namprd11.prod.outlook.com ([fe80::395e:eb75:6ab7:2ba5%3]) with mapi id 15.20.2772.018; Mon, 2 Mar 2020 18:20:47 +0000 From: "Ananyev, Konstantin" To: Honnappa Nagarahalli , "olivier.matz@6wind.com" CC: Gavin Hu , "dev@dpdk.org" , nd , nd Thread-Topic: [RFC 1/1] lib/ring: add scatter gather and serial dequeue APIs Thread-Index: AQHV61KoLY8qxNN4FEGi1nt9XMNavqgt1y+wgAHraACABJNf0A== Date: Mon, 2 Mar 2020 18:20:45 +0000 Message-ID: References: <20200224203931.21256-1-honnappa.nagarahalli@arm.com> <20200224203931.21256-2-honnappa.nagarahalli@arm.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZWVlY2M5ZjYtMjJkMC00NzU2LTg3ODAtNzU5NTQyZDcwZjFmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiRUQxMThseXNlY01RZm1cL1J1ays5WDJHR1RPcjV1VDlrTUI2Z0YwMys4MStuaUVkb3FTZms0UkFMdnNJQW5Ja2YifQ== dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 x-ctpclassification: CTP_NT authentication-results: spf=none (sender IP is ) smtp.mailfrom=konstantin.ananyev@intel.com; x-originating-ip: [192.198.151.191] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ad4969e1-5f8e-4d80-a95e-08d7bed66dd3 x-ms-traffictypediagnostic: SN6PR11MB2701: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 033054F29A x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(136003)(376002)(346002)(39860400002)(396003)(366004)(189003)(199004)(55016002)(8936002)(9686003)(5660300002)(81156014)(81166006)(66446008)(66476007)(66556008)(64756008)(66946007)(86362001)(76116006)(4326008)(33656002)(8676002)(71200400001)(6506007)(7696005)(2906002)(316002)(54906003)(478600001)(52536014)(26005)(110136005)(186003); DIR:OUT; SFP:1102; SCL:1; SRVR:SN6PR11MB2701; H:SN6PR11MB2558.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: IFVMAJlqmfaLPIugkGQvXo8P1hr6WYjPOzpxHcQnTXN9OQohQQ16e0Lobi7A8MrBpJNWBc+S0RRjohZmNxtsbxMfqrcNTZxhb6tnULAlWiNyeHpcbx8AGzFylpvl8seCl3FR3RQFTFfie2t/lRx2Ulo490hNOixLkvm7bqmrYsYweV9JRFu5rYgTH6x8KOGK2GNpqqWVAqkru2z9Dfw4ScUUqDXMrRQIKgQ00sJF+nIO3uLLWz0BgpL/tRhyRMtxzIqKE4mRmmi3oGsx9a8RjXu/X7kNEv58WDbvjMtnDFYLjT0NMcwoJ094I5+hXM0AiEpjqyCa/uItRVM9AHjjj4KS/1/5TVJudRs8+SVYi2vCpTU3c7SBEhJaKm4HnGQDhjP8PIbZ2C7DFbP64nSEr2xsLlgKVxILM/V/iaHHP82qU9GkDnd1SbN0SBnG/XV0 x-ms-exchange-antispam-messagedata: QnLenks+n00PZsYEbBt9wgHdUiFCSJHu8z5G+95hi9azCAA/jWRScC3m00+kUCttRDCYbQ4d1anOddtMxOFZEnCTCsI2Vk9FqQJA/7jD6JKnp2p+Z0oMjxDhnOlm+PwigLCrwyrKULwnt9q6cHBjnA== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: ad4969e1-5f8e-4d80-a95e-08d7bed66dd3 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Mar 2020 18:20:45.6499 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: L7Qp0hCJPi7eIbM2iGh8cyU8p3wX87Lo0C2xX5H4XXSpXgGn19ls9DaDBJvwLlfBkt1Hqt3DdYvtoQV/szUo8hntcp43EKf+FcvpzaRYN0o= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB2701 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [RFC 1/1] lib/ring: add scatter gather and serial dequeue APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > +/** > > > + * @internal Reserve ring elements to enqueue several objects on the > > > +ring > > > + * > > > + * @param r > > > + * A pointer to the ring structure. > > > + * @param esize > > > + * The size of ring element, in bytes. It must be a multiple of 4. > > > + * This must be the same value used while creating the ring. Other= wise > > > + * the results are undefined. > > > + * @param n > > > + * The number of elements to reserve in the ring. > > > + * @param behavior > > > + * RTE_RING_QUEUE_FIXED: Reserve a fixed number of elements fro= m a > > ring > > > + * RTE_RING_QUEUE_VARIABLE: Reserve as many elements as possible > > from ring > > > + * @param is_sp > > > + * Indicates whether to use single producer or multi-producer rese= rve > > > + * @param old_head > > > + * Producer's head index before reservation. > > > + * @param new_head > > > + * Producer's head index after reservation. > > > + * @param free_space > > > + * returns the amount of space after the reserve operation has fin= ished. > > > + * It is not updated if the number of reserved elements is zero. > > > + * @param dst1 > > > + * Pointer to location in the ring to copy the data. > > > + * @param n1 > > > + * Number of elements to copy at dst1 > > > + * @param dst2 > > > + * In case of ring wrap around, this pointer provides the location= to > > > + * copy the remaining elements. The number of elements to copy at = this > > > + * location is equal to (number of elements reserved - n1) > > > + * @return > > > + * Actual number of elements reserved. > > > + * If behavior =3D=3D RTE_RING_QUEUE_FIXED, this will be 0 or n on= ly. > > > + */ > > > +static __rte_always_inline unsigned int > > > +__rte_ring_do_enqueue_elem_reserve(struct rte_ring *r, unsigned int > > > +esize, > > > > > > I do understand the purpose of reserve, then either commit/abort for se= rial > > sync mode, but what is the purpose of non-serial version of reserve/com= mit? > In RCU, I have the need for scatter-gather feature. i.e. the data in the = ring element is coming from multiple sources ('token' is generated by > the RCU library and the application provides additional data). If I do no= t provide the reserve/commit, I need to introduce an intermediate > memcpy to get these two data contiguously to copy to the ring element. Th= e sequence is 'reserve(1), memcpy1, mempcy2, commit(1)'. > Hence, you do not see the abort API for the enqueue. >=20 > > In serial MP/MC case, after _reserve_(n) you always have to do > > _commit_(n) - you can't reduce number of elements, or do _abort_. > Agree, the intention here is to provide the scatter/gather feature. >=20 > > Again you cannot avoid memcpy(n) here anyhow. > > So what is the point of these functions for non-serial case? > It avoids an intermediate memcpy when the data is coming from multiple so= urces. Ok, I think I understand what was my confusion: Your intention: 1) reserve/commit for both serial and non-serial mode - to allow user get/set contents of the ring manually and avoid intermediate load/stores. 2) abort only for serial mode. =20 My intention: 1) commit/reserve/abort only for serial case (as that's the only mode where we can commit less then was reserved or do abort). 2) get/set of ring contents are done as part of either reserve(for dequeue) or commit(for enqueue) API calls (no scatter-gather ability). I still think that this new API you suggest creates too big exposure of ring internals, and makes it less 'safe-to-use': - it provides direct access to contents of the ring. - user has to specify head/tail values directly. So in case of some programmatic error in related user code,=20 there are less chances it could be catch-up by API, and we can easily end-up with silent memory corruption and other nasty things that would be hard to catch/reproduce. That makes me wonder how critical is this scatter-gather ability in terms of overall RCU performance?=20 Is the gain provided really that significant, especially if you'll update t= he ring by one element at a time?=20 =20 >=20 > > > > BTW, I think it would be good to have serial version of _enqueue_ too. > If there is a good use case, they should be provided. I did not come acro= ss a good use case. >=20 > > > > > + unsigned int n, enum rte_ring_queue_behavior behavior, > > > + unsigned int is_sp, unsigned int *old_head, > > > + unsigned int *new_head, unsigned int *free_space, > > > + void **dst1, unsigned int *n1, void **dst2) > > > > I do understand the intention to avoid memcpy(), but proposed API seems > > overcomplicated, error prone, and not very convenient for the user. > The issue is the need to handle the wrap around in ring storage array. i.= e. when the space is reserved for more than 1 ring element, the wrap > around might happen. >=20 > > I don't think that avoiding memcpy() will save us that many cycles here= , so > This depends on the amount of data being copied. >=20 > > probably better to keep API model a bit more regular: > > > > n =3D rte_ring_mp_serial_enqueue_bulk_reserve(ring, num, &free_space); = ... > > /* performs actual memcpy(), m<=3Dn */ > > rte_ring_mp_serial_enqueue_bulk_commit(ring, obj, m); > These do not take care of the wrap-around case or I am not able to unders= tand your comment. I meant that serial_enqueue_commit() will do both: actual copy of elements to the ring and tail update (no Scatter-Gather), se= e above.=20