From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 20DC0A0577; Tue, 14 Apr 2020 19:07:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CB6131D15D; Tue, 14 Apr 2020 19:07:09 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20067.outbound.protection.outlook.com [40.107.2.67]) by dpdk.org (Postfix) with ESMTP id 118861D15C for ; Tue, 14 Apr 2020 19:07:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mBtRYjSQShQiXS0vzEBWF8LvNoZ6T67MomdJqMn97o4=; b=KQfkbWmDTDidqxw4zsJIvLkmdQw+FPtJMOm8f5yCUSgDfO92J7usGkbUvNLlFJxDFkJBTjWQjhcrma80b24s7NjAml+BKQ2e5ZZroJHrLB9pLGu0RYARIFVi8DPnmKC/PS/5Pkmx7jDHVB/HEHsCTjnbeczlBcF9woVkBl10Xb0= Received: from AM5PR0701CA0068.eurprd07.prod.outlook.com (2603:10a6:203:2::30) by VI1PR08MB4077.eurprd08.prod.outlook.com (2603:10a6:803:e0::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.24; Tue, 14 Apr 2020 17:07:05 +0000 Received: from VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com (2603:10a6:203:2:cafe::bf) by AM5PR0701CA0068.outlook.office365.com (2603:10a6:203:2::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2921.12 via Frontend Transport; Tue, 14 Apr 2020 17:07:05 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT011.mail.protection.outlook.com (10.152.18.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.15 via Frontend Transport; Tue, 14 Apr 2020 17:07:04 +0000 Received: ("Tessian outbound 5345ff401cf8:v50"); Tue, 14 Apr 2020 17:07:04 +0000 X-CR-MTA-TID: 64aa7808 Received: from b49563c7069e.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id A5E584BB-3D6E-4D33-B0F5-A0D14F21679A.1; Tue, 14 Apr 2020 17:06:59 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id b49563c7069e.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 14 Apr 2020 17:06:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m+4prQ0vmH4NtxZh6hfnV7fN6gqrVinvVfgrcy+uDN/tHgOPrWjQvhAjK/haOhjg2oVseKnGptUPI/bFW9A7VlvbkypNrGyil2oZ8nRGKz975c1v8FCYQZ2I+RnAOvXgtUX7Qgiv6s5sI3DFbd+Zl5ickRKtH5fLIrSHu+Zx2YKmAszrYmWKu9lK5doL28UZWo6Bxx0CcSr0HPiraOwstUcr4yBCZqrMetupvCyxVJAjbOQBLI8w6YVpGJIxESTVQsTEO2T/8WwTsonRtVoG6YUWAMl/Jm/Ry5OJIIyBE95RSMpv0zeyh5OIu9zCcqmBLTplTY1aA1DfqGZOLgsdPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mBtRYjSQShQiXS0vzEBWF8LvNoZ6T67MomdJqMn97o4=; b=TR+oasezCiUoWCLg85t7pHD5tOAKU+NVXWEkUZ4RoKwju6ZFYne6qSNJN3sThWaejxAU7Rd4DATRheWGvr23MghcHbDrxbBS5E3FntMfzY8dVUpCkGagXRCTp1pFUwV5a3nMw+934GlgLTdSmXbirrmJCvGSPB0Tcm7iLxVhqrHh0WMr8jw+RU60nMMblZkgIYZ3k6FmbK87qWeI/7dSsrc6zQdPGwpx2ZTxMXuZ1zTBXPbZn8wEIDnGTSE7VH7VJzdT6HB0hJ3MCM6C3rhcQSA53gVy7u39aHlGo/a5C7nN3jA/PycbMuFK6tSTgvHS7UWZhYUDefJZBmR/2331ug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mBtRYjSQShQiXS0vzEBWF8LvNoZ6T67MomdJqMn97o4=; b=KQfkbWmDTDidqxw4zsJIvLkmdQw+FPtJMOm8f5yCUSgDfO92J7usGkbUvNLlFJxDFkJBTjWQjhcrma80b24s7NjAml+BKQ2e5ZZroJHrLB9pLGu0RYARIFVi8DPnmKC/PS/5Pkmx7jDHVB/HEHsCTjnbeczlBcF9woVkBl10Xb0= Received: from DBBPR08MB4646.eurprd08.prod.outlook.com (2603:10a6:10:f5::16) by DBBPR08MB4377.eurprd08.prod.outlook.com (2603:10a6:10:c6::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2900.28; Tue, 14 Apr 2020 17:06:57 +0000 Received: from DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d]) by DBBPR08MB4646.eurprd08.prod.outlook.com ([fe80::1870:afc4:b90f:609d%5]) with mapi id 15.20.2900.028; Tue, 14 Apr 2020 17:06:57 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "dev@dpdk.org" CC: "david.marchand@redhat.com" , "jielong.zjl@antfin.com" , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v3 5/9] ring: introduce HTS ring mode Thread-Index: AQHWCd9b4KJ7IMj2O0Wj5LTta1bJU6h0jcCQgARNaYCAAAoogA== Date: Tue, 14 Apr 2020 17:06:57 +0000 Message-ID: References: <20200402220959.29885-1-konstantin.ananyev@intel.com> <20200403174235.23308-1-konstantin.ananyev@intel.com> <20200403174235.23308-6-konstantin.ananyev@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: ba0acbfa-bde0-47b1-996f-fc7c8ce99e21.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [70.113.25.165] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 329cfbdd-add8-4491-43f2-08d7e09641c9 x-ms-traffictypediagnostic: DBBPR08MB4377:|DBBPR08MB4377:|VI1PR08MB4077: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; x-forefront-prvs: 0373D94D15 X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBBPR08MB4646.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(376002)(396003)(366004)(39850400004)(346002)(136003)(55016002)(54906003)(110136005)(81156014)(71200400001)(8936002)(5660300002)(8676002)(52536014)(2906002)(6506007)(316002)(186003)(7696005)(66946007)(33656002)(478600001)(66476007)(4326008)(66556008)(9686003)(86362001)(76116006)(26005)(66446008)(64756008); DIR:OUT; SFP:1101; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: VuVDOQsbVJmSH402RADPupOtL0rjdYipM8Wl6StCkk5ez878L9ZAcMCDUlJNezqEGxtK9IGGVzIbpUSdtIDREmcXKm4kBGaJqwORMJWTK44MZNDjsGBvrYwbKu2B4tCIMyjVWPXdsolVphb74w3lW0SUkq4hc4qI1Kbnzz5xLPcm4JoUszmk/lsHS+LOs9JdV+DEay+o6gjrtAmHUvj9ePxrIgDXFbvv+6ISebAKkXFgWMFpSofqWPAP7l69EuZE0zi+SRW5E6p2G39amtEwQKwIJjDyKd8QRQx92xQA6o1xkJwTdSbEtJKpQTxiFBxECHKP/DtjMaZVtoJtonmhTW9KeVNW9AtRoxe5bDFfZ4OhoIz6wj88VLhVDUCZq7thow9sWLARa2JXqhBjMLEYQa13z7p/Sd94OV0Ra4pTD5lYTFGJ8yc+eB7QXG2iU8R0 x-ms-exchange-antispam-messagedata: aB1huWmP7EU8xcdiXSub7wuAdwdiGoVG7e6GMXzra74eA82pynUUT6687EXipoEbl7WLyh556PoyxeBxYS8Zu0YHYgfxOrT78JckJka3m1kUBN3cix+JPVbwhn9IkpX+zy0zUfUoxeI6z1m2XNHIHw== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4377 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT011.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(10009020)(4636009)(376002)(136003)(39850400004)(396003)(346002)(46966005)(336012)(81166007)(356005)(26005)(110136005)(8676002)(54906003)(478600001)(5660300002)(81156014)(4326008)(52536014)(2906002)(26826003)(82740400003)(7696005)(86362001)(70586007)(55016002)(8936002)(6506007)(316002)(47076004)(186003)(9686003)(33656002)(70206006)(36906005); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 6b5427c1-6368-48d7-b36f-08d7e0963d77 X-Forefront-PRVS: 0373D94D15 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xjUz5GB4t3cnGNyBgQMnEfzGt+NkMhgWkFzruY047NSGjkDhZbYpEPe49VhILCuGZGIPUq63+8GM+NukOH140p5Drx9EXefdjh161A3bkkS5ReXcDhHcfNhKEJfbvHSftv7M229B0VXh9DBgbFgHASz8Qu5bZIt+zP+1ODd3b41kSbfGTxn4j4GIZFmHkUn3MJkdfBUAF3twZMH1RFoEhmSBKZgD/eQX5APkXuulROjrPi33h8R0TVY6eqJ+EgUKD78ROfAK/cXIW9TaxNlmPUQWWk9QsMSaBJaXbCIcvTgWlE87EfJ35QKlScG52cuxvkz0rpUGnSOAtYsg4udGtO1gqtKMSe1tUhW2llNiNhi1eQbo52+Xk4Kpk9w5zqiFxYVuujCYmhFpU0x6u9lBbNw/5UyR/yUrFshDp97Jk5SM/X6Zd/4B2C25SBDMDl3+345Gfr6vqlq5jEyBt72cUvv19Mg+1DFQsc6vHHP9a2JbOLyq3H13IQ5iPeraKtQt X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2020 17:07:04.8609 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 329cfbdd-add8-4491-43f2-08d7e09641c9 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB4077 Subject: Re: [dpdk-dev] [PATCH v3 5/9] ring: introduce HTS ring mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > diff --git a/lib/librte_ring/rte_ring_hts_generic.h > > > b/lib/librte_ring/rte_ring_hts_generic.h > > > new file mode 100644 > > > index 000000000..da08f1d94 > > > --- /dev/null > > > +++ b/lib/librte_ring/rte_ring_hts_generic.h > > > @@ -0,0 +1,198 @@ > > > + > > > +/** > > > + * @internal This function updates the producer head for enqueue > > > + * > > > + * @param r > > > + * A pointer to the ring structure > > > + * @param is_sp > > > + * Indicates whether multi-producer path is needed or not > > > + * @param n > > > + * The number of elements we will want to enqueue, i.e. how far sh= ould > the > > > + * head be moved > > > + * @param behavior > > > + * RTE_RING_QUEUE_FIXED: Enqueue a fixed number of items from a > ring > > > + * RTE_RING_QUEUE_VARIABLE: Enqueue as many items as possible > from > > > ring > > > + * @param old_head > > > + * Returns head value as it was before the move, i.e. where enqueu= e > starts > > > + * @param new_head > > > + * Returns the current/new head value i.e. where enqueue finishes >=20 > Ups, copy/paste thing - will remove. >=20 > > Would be good to return the new_head from this function and use it in > '__rte_ring_hts_update_tail'. >=20 > I think old_head + num should be enough here (see above). >=20 > > > > > + * @param free_entries > > > + * Returns the amount of free space in the ring BEFORE head was > moved > > > + * @return > > > + * Actual number of objects enqueued. > > > + * If behavior =3D=3D RTE_RING_QUEUE_FIXED, this will be 0 or n on= ly. > > > + */ > > Minor, suggest removing the elaborate comments, it is not required and > difficult to maintain. > > I think we should do the same thing for other files too. >=20 > Sorry, didn't get you here: what exactly do you suggest to remove? Following function is an internal function, we can skip the elaborate comme= nts. I see that you have done this in other places. >=20 > > > > > +static __rte_always_inline unsigned int > > > +__rte_ring_hts_move_prod_head(struct rte_ring *r, unsigned int num, > > > + enum rte_ring_queue_behavior behavior, uint32_t *old_head, > > > + uint32_t *free_entries) > > > +{ > > > + uint32_t n; > > > + union rte_ring_ht_pos np, op; > > > + > > > + const uint32_t capacity =3D r->capacity; > > > + > > > + do { > > > + /* Reset n to the initial burst count */ > > > + n =3D num; > > > + > > > + /* wait for tail to be equal to head */ > > > + __rte_ring_hts_head_wait(&r->hts_prod, &op); > > > + > > > + /* add rmb barrier to avoid load/load reorder in weak > > > + * memory model. It is noop on x86 > > > + */ > > > + rte_smp_rmb(); > > > + > > > + /* > > > + * The subtraction is done between two unsigned 32bits > > > value > > > + * (the result is always modulo 32 bits even if we have > > > + * *old_head > cons_tail). So 'free_entries' is always between > > > 0 > > > + * and capacity (which is < size). > > > + */ > > > + *free_entries =3D capacity + r->cons.tail - op.pos.head; > > > + > > > + /* check that we have enough room in ring */ > > > + if (unlikely(n > *free_entries)) > > > + n =3D (behavior =3D=3D RTE_RING_QUEUE_FIXED) ? > > > + 0 : *free_entries; > > > + > > > + if (n =3D=3D 0) > > > + break; > > > + > > > + np.pos.tail =3D op.pos.tail; > > > + np.pos.head =3D op.pos.head + n; > > > + > > > + } while (rte_atomic64_cmpset(&r->hts_prod.ht.raw, > > > + op.raw, np.raw) =3D=3D 0); > > I think we can use 32b atomic operation here and just update the head. >=20 > I think we have to do proper 64 bit CAS here, otherwise ABA race could ar= ise: > Thread reads head/tail values, then get suspended just before CAS instruc= tion > for a while. > Thread resumes when ring head value is equal to thread's local head value= , > but tail differs (some other thread enqueuing into the ring). Good point, ACK > If we'll do CAS just for head - it would succeed, though it shouldn't. > I understand that with 32-bit head/tail values probability of such situat= ion is > really low, but still. Using 64b values would be good. Both Arm and x86 support 128b CAS, not sure= about POWER. >=20 > > > > > + > > > + *old_head =3D op.pos.head; > > > + return n; > > > +} > > > +