From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B3808A04F3; Thu, 9 Jan 2020 00:40:17 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4349D1DB0B; Thu, 9 Jan 2020 00:40:17 +0100 (CET) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60047.outbound.protection.outlook.com [40.107.6.47]) by dpdk.org (Postfix) with ESMTP id 9F20B1DB05 for ; Thu, 9 Jan 2020 00:40:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kROb/Q/wJE6ZxKQ6L2VZ/hiVUCzs4BP2sWNHixGbMTo=; b=EgsYzjDuZKDF9zS/CA2S6WzMXn43m0P+spnoIKhYA/sEwoW3ca7SEj3KRGDGo9BDbyKuJeQCrB2pk5fGYd3W2Jetvhu/OBKnvnZ53HUB65vn5Dih0sUcgenzehH7/8ftw19SRBqNbWxhJami+c1fOxh4ZnKpEqFxF+gBYXnuwhg= Received: from VI1PR08CA0129.eurprd08.prod.outlook.com (2603:10a6:800:d4::31) by AM6PR08MB3095.eurprd08.prod.outlook.com (2603:10a6:209:45::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.9; Wed, 8 Jan 2020 23:40:14 +0000 Received: from AM5EUR03FT051.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::209) by VI1PR08CA0129.outlook.office365.com (2603:10a6:800:d4::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.10 via Frontend Transport; Wed, 8 Jan 2020 23:40:14 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT051.mail.protection.outlook.com (10.152.16.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2602.11 via Frontend Transport; Wed, 8 Jan 2020 23:40:14 +0000 Received: ("Tessian outbound 0eaff1016ea4:v40"); Wed, 08 Jan 2020 23:40:13 +0000 X-CR-MTA-TID: 64aa7808 Received: from a2d9e8bcc46f.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 77A393A8-63AE-4F9F-8204-D272F868A625.1; Wed, 08 Jan 2020 23:40:08 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id a2d9e8bcc46f.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 08 Jan 2020 23:40:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CDsh0wewXsZMxmZQyOBUQ+UBrL+/TF7bDCcgSxZctT3FGrvAoJH9bt1zWk31n/sIFrLyEpjPj+eVMrE0DK38PVn2hKirTU+NmmQHnvWe/8kPVr1HVOTwFM7KeEm2W8Kov0uAb4ubAtIfZlCj3LHVvyxphMQKZMW8i2EZghRjmWSCUrzdhJVcLbZAcJkGDmJPIG5frzsK/WcVyUwKRMdJiZJEE+TYm+Qd1F3OJ9/QQNM7nl1v7ml1oI1LoJBQtI4BjHPjAu5oxpMagiBSsNbU9b5xl9P1di/JT97r/VP0TFta42IwG7XfHFPzJFwZgx5LmLDuPtaQ3hBNCAU0CymKew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kROb/Q/wJE6ZxKQ6L2VZ/hiVUCzs4BP2sWNHixGbMTo=; b=kFjJheLNMhXuFUZbS/MJOY+AtiilCIFafREOMrO2PaemxThbIsuSw53+131rBGUq6mOtYS4nqEzmQCMLi4vu6eZ9Q4nHVpSSL8mBqOtWOsatklmH8FzUELZ/lIJiBKFFMR0y6UCQLRzZKtiKDoVgF69zrbN9B/eVmzS54fxfem27WlRsqCiu76gLruHrLgMIJpy03qBj3DKE64J2JZkpX4E6F97ffTPbRkv7gw+tuiyiTfKs/UcVYlXx/ObERmKLKBVTy+lbgH0M5zLNVCM6GQw4f3iLYPbnZsYEdQe0tfrHUOSt8aHGlbic4LEy1dZAGXy3MZqzcpgkqWCpgWjnxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kROb/Q/wJE6ZxKQ6L2VZ/hiVUCzs4BP2sWNHixGbMTo=; b=EgsYzjDuZKDF9zS/CA2S6WzMXn43m0P+spnoIKhYA/sEwoW3ca7SEj3KRGDGo9BDbyKuJeQCrB2pk5fGYd3W2Jetvhu/OBKnvnZ53HUB65vn5Dih0sUcgenzehH7/8ftw19SRBqNbWxhJami+c1fOxh4ZnKpEqFxF+gBYXnuwhg= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1PR08MB5056.eurprd08.prod.outlook.com (10.255.159.221) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2623.9; Wed, 8 Jan 2020 23:40:05 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::29eb:a1be:8f8f:fae2]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::29eb:a1be:8f8f:fae2%7]) with mapi id 15.20.2602.017; Wed, 8 Jan 2020 23:40:05 +0000 From: Honnappa Nagarahalli To: "Ananyev, Konstantin" , "olivier.matz@6wind.com" , "sthemmin@microsoft.com" , "jerinj@marvell.com" , "Richardson, Bruce" , "david.marchand@redhat.com" , "pbhagavatula@marvell.com" CC: "dev@dpdk.org" , Dharmik Thakkar , Ruifeng Wang , Gavin Hu , nd , Honnappa Nagarahalli , nd Thread-Topic: [PATCH v7 02/17] lib/ring: apis to support configurable element size Thread-Index: AQHVwYun8sYxhXoQvESKMlV7eC4Zb6fer7kQgAALWqCAAEpBgIAAT0pQgAAKI4CAAOSEMIAAT+uAgABqyrA= Date: Wed, 8 Jan 2020 23:40:05 +0000 Message-ID: References: <20190906190510.11146-1-honnappa.nagarahalli@arm.com> <20191220044524.32910-1-honnappa.nagarahalli@arm.com> <20191220044524.32910-3-honnappa.nagarahalli@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: d0772696-86fc-4462-9b57-8b28870d903c.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 9fff3b15-3576-4d43-6d3a-08d794941c0d X-MS-TrafficTypeDiagnostic: VE1PR08MB5056:|VE1PR08MB5056:|AM6PR08MB3095: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: True x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 02760F0D1C X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(199004)(189003)(110136005)(86362001)(54906003)(52536014)(2906002)(66946007)(26005)(186003)(66556008)(5660300002)(66446008)(64756008)(66476007)(316002)(76116006)(9686003)(33656002)(71200400001)(81166006)(8676002)(8936002)(81156014)(6506007)(55016002)(4326008)(7696005)(478600001); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB5056; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: 7jx2l1et7aL2/b8vl4R0eTMxKI0+y2/Mz5cqiSnGalAfH9jidm+ZU7QWKTtNm4bmkL878HfJpzv1RH+v35lHX6i5twFSCK5TyUeS7Xie7dfkOnQ4Fj7oPmEVPmrGFtRPKm6AtwBmYnq1oCln5ZIXrD/pRGdYTSxprvZsnCWkVp96SYNgJt7/YhH0Ne9crlOJskA8Dn1H+qPzIh6R+/XMOoCxJNsF5ajgSnQD0BYKQ5nwrVbE6LMchhhFarCEuPfXZpNECdhhZioN6BjyxpBddVXEKWPe2eW0FMMbKdzB+EFSWtY4tDe6rkKdwR5sm/zKKO171D+9sDyivrZvqxH0YNVeESm6Wn/20LLHCFTL69WpeCND3BSb3vlJD1M1lC5zBqhTBlmu3pIU6GVtmta2TV3mDynaFKZrew77ncMhDvl+wzZZQzMfG4iyjcukyDds Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5056 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT051.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(376002)(136003)(39860400002)(346002)(396003)(199004)(189003)(316002)(36906005)(7696005)(81156014)(356004)(5660300002)(478600001)(26826003)(9686003)(52536014)(336012)(54906003)(2906002)(8676002)(26005)(33656002)(186003)(8936002)(81166006)(6506007)(110136005)(55016002)(86362001)(4326008)(70586007)(70206006); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR08MB3095; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: f2f65452-a988-4138-e95b-08d7949416e2 X-Forefront-PRVS: 02760F0D1C X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7YjafBi364/6u9OnznGyou4zuANx+ctOaTpLRtoR8VLCTApChJZYyP7ospg5/MjEIiFGJB03CVdVmoHtdaRd9UqVF/h0wogyFGJiUSqQUlWpxC2mlzCD7g/t7VzeaoZdc1E4GtZQpQPkFZOPBQOKX4e7QsuG2SiiIvXFsxRqx8AhNESUzRaOR7AoD+g0DlRWCnyV9MUV3UQzhwYXcmj6PbmWvUhreC9wngppIargQqtmvaVyHtvmCxuYJlVbnGVUymaxP0iT6yq/Yf5SW+DRktUluYm3SilIPqUOMR6ygOAz/ezoO2o5uzzTtFdueIuwPSP5dal+HlxyzbzZB7lw02ScgrpaNV81D2zTJ24CRH0ZfRNwbivauEl0sWFMOJ6zwscmd/i7sHzVeP4NpEzBloNowGn/kJLzwsWJHAjOgJ5U692ldwp31RbTPO4tZJgd X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jan 2020 23:40:14.2136 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9fff3b15-3576-4d43-6d3a-08d794941c0d X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB3095 Subject: Re: [dpdk-dev] [PATCH v7 02/17] lib/ring: apis to support configurable element size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > > > > > > > + > > > > > > > > > +static __rte_always_inline void > > > > > > > > > +enqueue_elems_128(struct rte_ring *r, uint32_t > > > > > > > > > +prod_head, const void *obj_table, uint32_t n) { > > > > > > > > > +unsigned int i; const uint32_t size =3D > > > > > > > > > +r->size; uint32_t idx =3D prod_head & r->mask; > > > > > > > > > +r->__uint128_t > > > > > > > > > +*ring =3D (__uint128_t *)&r[1]; const __uint128_t *obj = =3D > > > > > > > > > +(const __uint128_t *)obj_table; if (likely(idx + n < > > > > > > > > > +size)) { for (i =3D 0; i < (n & ~0x1); i +=3D 2, idx += =3D 2) > > > > > > > > > +{ ring[idx] =3D obj[i]; ring[idx + 1] =3D obj[i + 1]; > > > > > > > > > > > > > > > > > > > > > > > > AFAIK, that implies 16B aligned obj_table... > > > > > > > > Would it always be the case? > > > > > > > I am not sure from the compiler perspective. > > > > > > > At least on Arm architecture, unaligned access (address that > > > > > > > is accessed is not aligned to the size of the data element > > > > > > > being > > > > > > > accessed) will result in faults or require additional cycles. > > > > > > > So, aligning on > > > > > 16B should be fine. > > > > > > Further, I would be changing this to use 'rte_int128_t' as > > > > > > '__uint128_t' is > > > > > not defined on 32b systems. > > > > > > > > > > What I am trying to say: with this code we imply new requirement > > > > > for elems > > > > The only existing use case in DPDK for 16B is the event ring. The > > > > event ring > > > already does similar kind of copy (using 'struct rte_event'). > > > > So, there is no change in expectations for event ring. > > > > For future code, I think this expectation should be fine since it > > > > allows for > > > optimal code. > > > > > > > > > in the ring: when sizeof(elem)=3D=3D16 it's alignment also has to= be > > > > > at least > > > 16. > > > > > Which from my perspective is not ideal. > > > > Any reasoning? > > > > > > New implicit requirement and inconsistency. > > > Code like that: > > > > > > struct ring_elem {uint64_t a, b;}; > > > .... > > > struct ring_elem elem; > > > rte_ring_dequeue_elem(ring, &elem, sizeof(elem)); > > > > > > might cause a crash. > > The alignment here is 8B. Assuming that instructions generated will > > require 16B alignment, it will result in a crash, if configured to gene= rate > exception. > > But, these instructions are not atomic instructions. At least on > > aarch64, unaligned access will not result in an exception for non-atomi= c > loads/stores. I believe it is the same behavior for x86 as well. >=20 > On IA, there are 2 types of 16B load/store instructions: aligned and unal= igned. > Aligned are a bit faster, but will cause an exception if used on non 16B = aligned > address. > As you using uint128_t * compiler will assume that both src and dst are 1= 6B > aligned and might generate code with aligned instructions. Ok, looking at few articles, I read that if the address is aligned, the una= ligned instructions do not incur the penalty. Is this understanding correct= ? I see 2 solutions here: 1) We can switch this copy to use uint32_t pointer. It would still allow th= e compiler to generate (unaligned) instructions for up to 256b load/store. = The 2 multiplications (to normalize the index and the size of copy) can use= shifts. This should make it safer. If one wants performance, they can alig= n the obj table to 16B (the ring itself is already aligned on the cache lin= e boundary). 2) Considering that performance is paramount, we could document that the ob= j table needs to be aligned on 16B boundary. This would affect event dev (i= f we go ahead with replacing the event ring implementation) significantly. Note that we have to do the same thing for 64b elements as well. >=20 > > > > > While exactly the same code with: > > > > > > struct ring_elem {uint64_t a, b, c;}; OR struct ring_elem {uint64_t > > > a, b, c, d;}; > > > > > > will work ok. > > The alignment for these structures is still 8B. Are you saying this > > will work because these will be copied using pointer to uint32_t (whose > alignment is 4B)? >=20 > Yes, as we doing uint32_t copies, compiler can't assume the data will be = 16B > aligned and will use unaligned instructions. >=20 > > > > > > > > > > > > > > Note that for elem sizes > 16 (24, 32), there is no such constrai= nt. > > > > The rest of them need to be aligned on 4B boundary. However, this > > > > should > > > not affect the existing code. > > > > The code for 8B and 16B is kept as is to ensure the performance is > > > > not > > > affected for the existing code. > >