From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3E2A0A0526; Tue, 21 Jul 2020 06:04:11 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6DC4D1BFE9; Tue, 21 Jul 2020 06:04:10 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2054.outbound.protection.outlook.com [40.107.20.54]) by dpdk.org (Postfix) with ESMTP id 55A6A1BFD1 for ; Tue, 21 Jul 2020 06:04:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VycG9RojvBOCkPDYFFXmvYMYHwmwEFsv3+ho7Atdf4E=; b=IMn5XihvNUdq55g5w/0u46KID+eajaJmctZfc0eU3rYyoTiREosEM64XE+9cJWf6sc4i03UFiKT9LAajxn/JehZGDSS+voMykpCFJ0C3IvQzfkCCvMjfIt3Sqk1D0k5nmyCABbC7XUIi7Uqk1JyQE0Ieq+kHeO12o+Zn81ZYbaM= Received: from DB7PR05CA0029.eurprd05.prod.outlook.com (2603:10a6:10:36::42) by VI1PR0802MB2365.eurprd08.prod.outlook.com (2603:10a6:800:9f::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.23; Tue, 21 Jul 2020 04:04:07 +0000 Received: from DB5EUR03FT055.eop-EUR03.prod.protection.outlook.com (2603:10a6:10:36:cafe::e3) by DB7PR05CA0029.outlook.office365.com (2603:10a6:10:36::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.18 via Frontend Transport; Tue, 21 Jul 2020 04:04:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT055.mail.protection.outlook.com (10.152.21.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.18 via Frontend Transport; Tue, 21 Jul 2020 04:04:07 +0000 Received: ("Tessian outbound 1c27ecaec3d6:v62"); Tue, 21 Jul 2020 04:04:07 +0000 X-CR-MTA-TID: 64aa7808 Received: from c2c8508a1fd1.3 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 02815F0C-C9EE-407C-8E27-1BD8A5710AEC.1; Tue, 21 Jul 2020 04:04:02 +0000 Received: from EUR02-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id c2c8508a1fd1.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 21 Jul 2020 04:04:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gj0XzmV/r2v7RiDf7R7vOc7kXyR6RCm2pinmIP/rxHXjfPIXjQ9x2p53n+wh3TByjBbHhAj4kvOPTdl0BtjKq7R82MEJwSQwp3pQPmrH5wQiN7BtM4RgZSd9ZUFDmZ3Ae1X3vKm4StSfrfskXjjqIN0Li4g4SiZx5QsLy8ZGnfZwWzUfjgworYfNW8ovPcrkUxoIk46vV4EelNYyfozBfdqZLcKwZRBlllpL41/HoGYfhsefsBH+nIx4w7MD2//yydb3blqyCTCCzYkHgPsPEjTCj8ciuB14Xz6+qPYWj+yoCzp3J8IeEHxGUlf7Qec663tpSCfXLvBqRJCaAMpH5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VycG9RojvBOCkPDYFFXmvYMYHwmwEFsv3+ho7Atdf4E=; b=J+LdcRUuosVcoZlJ5fpVyEKPKsW87mTBEhS/RygUIts138ZKUzC4auS6h1TMoOmlRVasqhuou7Ln+HReAMsFcHvU4bJZ8sg4C5H2mrD0V+bUAHlwSaaSdCWZLX2sBXA+80zrp8YCs1R4PF4AuxxkVoW1iJQunxMwAH1ehh/lYaM1ZCjAvZU9OsioyDAJNCciQkpYeqC9pKrb1K2HBZBM47728WqeKwS+h/4sLR28lXPJI+5NQ6yLcz76y5/XEbMkjcC+ZYvv/qNu4CPSrDCj9WPdkExk+fkxI4MnJHJ52LsWt9ipQsILOZRMeO6jdQEc0SCvlEkYKiybYMwELvguyA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VycG9RojvBOCkPDYFFXmvYMYHwmwEFsv3+ho7Atdf4E=; b=IMn5XihvNUdq55g5w/0u46KID+eajaJmctZfc0eU3rYyoTiREosEM64XE+9cJWf6sc4i03UFiKT9LAajxn/JehZGDSS+voMykpCFJ0C3IvQzfkCCvMjfIt3Sqk1D0k5nmyCABbC7XUIi7Uqk1JyQE0Ieq+kHeO12o+Zn81ZYbaM= Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com (2603:10a6:4:85::9) by DBBPR08MB4696.eurprd08.prod.outlook.com (2603:10a6:10:f3::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3195.17; Tue, 21 Jul 2020 04:03:58 +0000 Received: from DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::9d1d:207b:e89d:199d]) by DB6PR0802MB2216.eurprd08.prod.outlook.com ([fe80::9d1d:207b:e89d:199d%10]) with mapi id 15.20.3195.026; Tue, 21 Jul 2020 04:03:58 +0000 From: Honnappa Nagarahalli To: Alexander Kozyrev , Phil Yang , Matan Azrad , Shahaf Shuler , Slava Ovsiienko CC: "drc@linux.vnet.ibm.com" , nd , "dev@dpdk.org" , Honnappa Nagarahalli , nd Thread-Topic: [dpdk-dev] [PATCH v3] net/mlx5: relaxed ordering for multi-packet RQ buffer refcnt Thread-Index: AQHWXxP1VsVFa6zZkk6D6GqPamJJOw== Date: Tue, 21 Jul 2020 04:03:58 +0000 Message-ID: References: <20200410164127.54229-7-gavin.hu@arm.com> <1592900807-13289-1-git-send-email-phil.yang@arm.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 120e15af-61bc-4b8b-9acb-a8bc40d58bda.0 x-checkrecipientchecked: true Authentication-Results-Original: mellanox.com; dkim=none (message not signed) header.d=none; mellanox.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [70.112.90.121] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 3493c9d8-ea9b-4e2a-8ec3-08d82d2b1d56 x-ms-traffictypediagnostic: DBBPR08MB4696:|VI1PR0802MB2365: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:7219;OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: AqbGs946LchvleEl5dnkpI64wp5s2eSqfsBGZH97RJQR/V9Mw1oQ7RB0HrICcuHkJFSZTjVk+sklToxCrD60X5UUNwZGYdXF2fPyG5KhRIpulSTqMAd13KlAPleEBxxQ/z4aAtEkjA8jQes0+hXfFRdgMuOL2mhIcfA7nkjpDJLGpgVg+LdBDsfbmqyN9kc4WUiZwV6jAXwnw5Ce9qFNCY8k/cLCBXjSuj8dBq4p4AL2N7f8CHMtGXHCV2y2S35sF2tN/83yFHptvwn88SH+jozuVxSO6mrnGxHBSoHdyEUHjkjsxTIiEn83G/JK7wZHz+K9gtvjOvx1xuL4TZRbZw== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DB6PR0802MB2216.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(366004)(376002)(136003)(346002)(396003)(83380400001)(7696005)(8936002)(316002)(186003)(2906002)(478600001)(5660300002)(4326008)(71200400001)(33656002)(6506007)(76116006)(26005)(110136005)(66946007)(55016002)(54906003)(66476007)(66446008)(86362001)(66556008)(52536014)(64756008)(9686003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: Tt3yEl04KxmX1ECQs8BN4j19OZzJDrKi3+eH2sR4XlV8LeiCUve9TsXhPzUXVLM5fPZOsONsBfk/u/ZhYcd66dlvOeR0a6f0vyr1av7AQTcx/HCckbKupGdJ6FocLtp6sNLoNs7t7OSQa0S8VbVVBBaGkcjWuRnk2hQiuCQzbfyNyvc4cck3V9EZ9cAuxdd8B/ue3CgOmmJbbTYv0zfKt9HPUZcvaAanZVx3kzedSu4a0h2z9I8EMr9rY9qr+X/i2c2NWfPXwOmRH445Ia0cG2DRVhsus0XnBCoXvHIM+YQSrAV/r5webbGTgSPqC5ulDhIFE3RNmJ3eFf4OiBvkWJWfEbJynelnhRXopGQjcxVXgFaerzzEikzOyAJRMCTwvEPEleaoeJzUwmrqxtYDzMDR+1/QF5M8igs2I3oc90uSws23JOgobTGdrb3+qjYuhoJ+fj46qzbGzwio8CmObgVtpKh0ZtLaYONLTa01AQfoTyBoHF0EIk3YsleFqeJE Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4696 Original-Authentication-Results: mellanox.com; dkim=none (message not signed) header.d=none; mellanox.com; dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT055.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFTY:; SFS:(4636009)(396003)(346002)(376002)(136003)(39860400002)(46966005)(82310400002)(47076004)(478600001)(83380400001)(8936002)(186003)(52536014)(336012)(5660300002)(33656002)(9686003)(55016002)(81166007)(356005)(82740400003)(2906002)(4326008)(7696005)(316002)(54906003)(110136005)(6506007)(26005)(70206006)(86362001)(70586007); DIR:OUT; SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 8fa25357-8d90-4c31-b0c8-08d82d2b180f X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3KLwTk07tH/nr5EdErwIt7dB7FfJT5zQrWbrW6HkoWrAgaQtniI+2MUs+LhGWxv8AQKXpAupIlKMJ4CRD+NPFxiQBUhnmnmD45iD751yJrEjuEl6kOCvYJwxon7kCa/IaoludvzlQ/R1qGbTDZXCSlyOIUalZdZ0lDyGKJTQDTiwTZFzmvyKUfBQ5GYAwqOFtvbMPxLwWv8Sqa5ghloCbzh7wnUXVSIWB+A5f+79PYY1gpkio3kkuTXamHL3/G9O8UlGSPcDpbFMc5cVXw66YXMkrXJO/OGjJQuBCid09BAfKZdBOEIrH0BrJqiDuHz4bXbY2s7kubJQ96sH2RqUmqvjBAasZMBrc5MsDLf6UL7bM7xy2fjFRg8YS1xit69j0DZZnJnwWNkekv7Twhzijg== X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jul 2020 04:04:07.1915 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3493c9d8-ea9b-4e2a-8ec3-08d82d2b1d56 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT055.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2365 Subject: Re: [dpdk-dev] [PATCH v3] net/mlx5: relaxed ordering for multi-packet RQ buffer refcnt X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > > Subject: RE: [dpdk-dev] [PATCH v3] net/mlx5: relaxed ordering for > > > multi- packet RQ buffer refcnt > > > > > > Hi Phil Yang, we noticed that this patch gives us 10% of performance > > > degradation on ARM. > > > x86 seems to be unaffected though. Do you know what may be the > > > reason of this behavior? > > > > Hi Alexander, > > > > Thanks for your feedback. > > This patch removed some expensive memory barriers on aarch64, it > > should get better performance. > > I am not sure the root cause of this degradation now, I will start the > > investigation. We can profiling this issue together. > > Could you share your test case(including your testbed configuration) wi= th > us? > > > > Thanks, > > Phil >=20 > I'm surprised too, Phil, but looks like it is actually making things wors= e. I used > Connect-X 6DX on aarch64: > Linux dragon71-bf 5.4.31-mlnx.15.ge938819 #1 SMP PREEMPT Thu Jul 2 > 17:01:15 IDT 2020 aarch64 aarch64 aarch64 GNU/Linux Traffic generator > sends 60 bytes packets and DUT executes the following command: > arm64-bluefield-linuxapp-gcc/build/app/test-pmd/testpmd -n 4 -w > 0000:03:00.1,mprq_en=3D1,rxqs_min_mprq=3D1 -w > 0000:03:00.0,mprq_en=3D1,rxqs_min_mprq=3D1 -c 0xe -- --burst=3D64 -- > mbcache=3D512 -i --nb-cores=3D1 --rxq=3D1 --txq=3D1 --txd=3D256 --rxd= =3D256 --auto- > start --rss-udp Without a patch I'm getting 3.2mpps, and only 2.9mpps whe= n > the patch is applied. You are running on A72 cores, is that correct? >=20 > Regards, > Alex