From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8E4F5A0526; Tue, 21 Jul 2020 05:58:34 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6DFC51BFE4; Tue, 21 Jul 2020 05:58:34 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60081.outbound.protection.outlook.com [40.107.6.81]) by dpdk.org (Postfix) with ESMTP id 5FD121BFE3 for ; Tue, 21 Jul 2020 05:58:32 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cri0FIVR7yT5m9OVBSi36KLlHjsITaW/xoPbfAKwOrArNRl0cq0PK5TVcmxeT/YpZjxLhfApvAO5YoIx96tUtpumhVXxiaST077SyfPpVcAY8Qrq0/1+Vuau3XjxIAywf8cI3+9c8a9/iegloBuiQxLZFgMFASSb53llp7AwzFgpzwclNAmkZemFF9FlTKz7q6TaR/C5J3MfQGK1gpsyPKVG2Cda3zMWJo0+0OTGtKa9WCwJyRnmjj3SLv1aeR1lghX8qxcXSW3NEyyzB9DYDW70+x21ZxAU4ZV2Itgegrhw8AwlAkP5hHdjYQNq7o7WfI/pX53P78QphMqysVjzFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VhxurcbAb9jeHPpahM5jnNbYLTKAUewgf/ri65WCxE0=; b=IuBdCV3a/W8qPuBz1lV5vOMlODXnFaoH4xijG7hRVkpo+noi4D+TxX6x1cWI8pWp5VYnIXGirUyUaVaBthN2Mpzan5si+NS6uJdaDkrmgmuoVghdWPfs4Pste8YCCLDFCZmllHyn99JhSxIceYbC4IxoA+5UH4LP/vq703HoqMWoLFClrHCmnvkahyV3PH3EmE+dvXqo+1JwCGS6rI7OHUaI5gKo8qVJChko8hDIuPJpgednIo3LorGKSzrrlmn4pLbzrHYlYTnisDgwzHpuTrYo1dDAFs6ExgNXuKU3ZD15AeXa/DXaGdeNEtVbRL26Zg78AoxcElUEinMTx9KI9A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VhxurcbAb9jeHPpahM5jnNbYLTKAUewgf/ri65WCxE0=; b=dVlOk8magMsadBtQ4Jd1iAH2JP4kAspucPaUFOIMfGId2WbHnwq3orwD17kp1PmQ11NIoRG0miYrtA1U9Dl985G7Cn3V/boZx+GQB8Db+n6RqObWHeyPSXJ3f0iTmopuYuGxqneEqDFdzUg2+MNjs6Z8o/5FQ7jtbDyBxM3ebV8= Received: from AM0PR05MB4561.eurprd05.prod.outlook.com (2603:10a6:208:ad::20) by AM0PR05MB4817.eurprd05.prod.outlook.com (2603:10a6:208:cf::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.25; Tue, 21 Jul 2020 03:58:30 +0000 Received: from AM0PR05MB4561.eurprd05.prod.outlook.com ([fe80::456:89d0:7f5c:c6c3]) by AM0PR05MB4561.eurprd05.prod.outlook.com ([fe80::456:89d0:7f5c:c6c3%4]) with mapi id 15.20.3195.024; Tue, 21 Jul 2020 03:58:30 +0000 From: Alexander Kozyrev To: Phil Yang , Matan Azrad , Shahaf Shuler , Slava Ovsiienko CC: Honnappa Nagarahalli , "drc@linux.vnet.ibm.com" , nd , "dev@dpdk.org" , nd , nd Thread-Topic: [dpdk-dev] [PATCH v3] net/mlx5: relaxed ordering for multi-packet RQ buffer refcnt Thread-Index: AQHWWMITvRJ6cPOqEEiXXa7L02rE6qkRJjhAgAAsNgCAAA49EA== Date: Tue, 21 Jul 2020 03:58:30 +0000 Message-ID: References: <20200410164127.54229-7-gavin.hu@arm.com> <1592900807-13289-1-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [2607:fea8:e380:d8e0:cc8e:6267:1dfa:5b55] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 10135815-ec1c-4540-fd80-08d82d2a549b x-ms-traffictypediagnostic: AM0PR05MB4817: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7219; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: cuJklLQW30Kt3Ex6DgSZGoWoCdVUrMXR8WVgPBKOklfOPqnEmcQAG8fyBszOHdjiPuxzv1A1nIKuEZl9aUZyfbGD2nelf0xwJpdoDyE1lUgupC1JAOC9PYqd13KsqtDdUVxoOSN9hsaa7lcQgip38ZmCjE4LuBfUQ6QAA6h2+E/WV60mPd0yevdoKnJDJMqjfoC7Zk4eezANg4LASylmCHR+8AmlZ8//brW8PobXAHy+2BdQKEk+k0m/pywEX7ErC7QIGu+iM5sRVOGzJ5a1odwVGn0pf0h9WnLTfm8Vh0ENx9vCnzA0aoApkACjADg5h6wJqqpYhPDXgJeVTQ0hlA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR05MB4561.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(136003)(396003)(376002)(39860400002)(346002)(5660300002)(6636002)(7696005)(71200400001)(2906002)(478600001)(83380400001)(186003)(52536014)(76116006)(66946007)(86362001)(33656002)(66476007)(66556008)(64756008)(66446008)(8936002)(4326008)(6506007)(54906003)(316002)(55016002)(9686003)(110136005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: GNnBoI0q0IKH2bALhP5yf90H9ohEC1SWDiMMzwUqDT8TICV1hvXcf4FR5yheNHBifs5YzfH2IWQuzRW6SRag06JqXMGzdnT/szcCF2yjr7Pmec3AtVW/P/7fYhuGqWbpR+IJdU/pmrH8p6oeAoqR/FzW4p7UI7ADVexs9DbHetSKREToCdCDuk0fC2knKFt+v4kIjZ/TF7RKVXaIowsY8H7JlsHlqTS35BvTlfHZY8jbRIX3fXv5feOIwNTG0cnxU94XYURGqn6Su/bSsKagMPb642kGcZvqp4+ShduB5WfjZhaoqf+Lmdcih92VTTo7jg5JKIo6n562kNpypOe6Mkc82snRWhTs8YDz5Egyg5WnpyZP3zHGJTz8wT0Gu7fTVuvaiTmtN6SfqYXcCsRAs++PTNnhRLwvWaACeTIhAHzQQIII6f1wH3F5Thia1psV6CYxV7glU6vq4MUxyqAZMgZuhrml32yHMmpYwV+acqjR0quR13A+kYmdd6+yDsggEMqP0YE9XwswKAx55GqnNzwUIKym05pVUwLGzCEhhWAC+ypnadV8rstt9qZWYsl1 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AM0PR05MB4561.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 10135815-ec1c-4540-fd80-08d82d2a549b X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jul 2020 03:58:30.3121 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: M2Y0TQRZK1TRsFH/kPZsVwqoy/dozpIvKpSO6KIcwS0UYK8h9XIKc7/tdWUTJnWEM5E4nezMaiDnUcHWcTzQSw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR05MB4817 Subject: Re: [dpdk-dev] [PATCH v3] net/mlx5: relaxed ordering for multi-packet RQ buffer refcnt X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > Phil Yang writes: >=20 > <...> > > Subject: RE: [dpdk-dev] [PATCH v3] net/mlx5: relaxed ordering for > > multi- packet RQ buffer refcnt > > > > Hi Phil Yang, we noticed that this patch gives us 10% of performance > > degradation on ARM. > > x86 seems to be unaffected though. Do you know what may be the reason > > of this behavior? >=20 > Hi Alexander, >=20 > Thanks for your feedback. > This patch removed some expensive memory barriers on aarch64, it should g= et > better performance. > I am not sure the root cause of this degradation now, I will start the > investigation. We can profiling this issue together. > Could you share your test case(including your testbed configuration) with= us? >=20 > Thanks, > Phil I'm surprised too, Phil, but looks like it is actually making things worse.= I used Connect-X 6DX on aarch64: Linux dragon71-bf 5.4.31-mlnx.15.ge938819 #1 SMP PREEMPT Thu Jul 2 17:01:15= IDT 2020 aarch64 aarch64 aarch64 GNU/Linux Traffic generator sends 60 bytes packets and DUT executes the following com= mand: arm64-bluefield-linuxapp-gcc/build/app/test-pmd/testpmd -n 4 -w 0000:03:00= .1,mprq_en=3D1,rxqs_min_mprq=3D1 -w 0000:03:00.0,mprq_en=3D1,rxqs_min_mprq= =3D1 -c 0xe -- --burst=3D64 --mbcache=3D512 -i --nb-cores=3D1 --rxq=3D1 = --txq=3D1 --txd=3D256 --rxd=3D256 --auto-start --rss-udp Without a patch I'm getting 3.2mpps, and only 2.9mpps when the patch is app= lied. Regards, Alex