From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DF70DA046B for ; Thu, 22 Aug 2019 04:30:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1F2C31BF16; Thu, 22 Aug 2019 04:30:24 +0200 (CEST) Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10057.outbound.protection.outlook.com [40.107.1.57]) by dpdk.org (Postfix) with ESMTP id 57ADE1BF0F for ; Thu, 22 Aug 2019 04:30:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JLkzFWRU6C1mZuD7ZpB8/2b56x6n/Jbb/IMkg28K/bU=; b=OTZ0XMJqnYyCLs7cqTjgptwV8OFZCC902OhrRrmRAGgWn8NiqX+EgMZhgzaYnaHYqIjKbCpfSFxWICW7EsexeEemKbc/ZVKlNwig0kR70yJHLItCaevI1lAyFfWk51YIP9Ze08uV9HIZh0VxtLFm/ZDliIYU8Bg/3vx23JSniPM= Received: from VI1PR0801CA0083.eurprd08.prod.outlook.com (2603:10a6:800:7d::27) by AM5PR0802MB2593.eurprd08.prod.outlook.com (2603:10a6:203:94::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2178.18; Thu, 22 Aug 2019 02:30:20 +0000 Received: from AM5EUR03FT058.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e08::202) by VI1PR0801CA0083.outlook.office365.com (2603:10a6:800:7d::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2199.14 via Frontend Transport; Thu, 22 Aug 2019 02:30:20 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=temperror action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT058.mail.protection.outlook.com (10.152.17.48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2199.13 via Frontend Transport; Thu, 22 Aug 2019 02:30:18 +0000 Received: ("Tessian outbound a1fd2c3cfdb0:v26"); Thu, 22 Aug 2019 02:30:18 +0000 X-CR-MTA-TID: 64aa7808 Received: from 1bada2e0e350.3 (cr-mta-lb-1.cr-mta-net [104.47.12.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 71D607A2-0429-4540-A472-6DEEC6B5D3A6.1; Thu, 22 Aug 2019 02:30:13 +0000 Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2055.outbound.protection.outlook.com [104.47.12.55]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 1bada2e0e350.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Thu, 22 Aug 2019 02:30:13 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G4u9Ef+/a7U8DRifg+PcuHEDHzbIUhw6z/l2cei8LB2M7WU+kOQ3NOUMFr259YofETatlr0hQFhuLH0mKggegXSKTsLq8/Cuyb5XHakkeecYged1AneUxEQbEh1BIR9a6vTlXdWIS0vKfyg3sqvKH45gz7zQ2j2P81Bmlx6F1rBeAHifZgalJtHmybZGcMiW6TehF5962G4MW18SDhQCLZCvsJi0hRMsv9CzJH/N8ty4zq9jjBCe8l7qIOeP94qmnyq5yPBvpB0B0sRyTJ4OhxJCNm1Bz9/HcZuvyeYkEAecioKrgghd/8va/q6gb2UWbKDWMsJc224RqTCPJFIogw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JLkzFWRU6C1mZuD7ZpB8/2b56x6n/Jbb/IMkg28K/bU=; b=GT0h/qLMy59SeEx+WYeFlB0Cj4RtE58OMPbkA6PBxaFnY/HYnGFchiHPq+J2yVNHQ7OSaIjr91U4vzolZThXP5hM9rUDWxYtRPgRweSrY6nFXBE5unDmispOK5LvjGttllAQhSKU712Q1vrWQoYkNynghok83VFotanRrmOhVVCr8kACHPxxhdwclnQDM+sPRnA+uAps0tEKlcYkiJc/wPPC5qfwklOjLE6vekpjFMFcrbdwShj51krPK2RCy3Snxj78v6itHG9XQdqdhduuVJXEHlaFkH8GoK5sNQiOAPG/Gc7R5IXej9oiJxnvJn85F+813Lb1EwdZrKa0KaePgQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JLkzFWRU6C1mZuD7ZpB8/2b56x6n/Jbb/IMkg28K/bU=; b=OTZ0XMJqnYyCLs7cqTjgptwV8OFZCC902OhrRrmRAGgWn8NiqX+EgMZhgzaYnaHYqIjKbCpfSFxWICW7EsexeEemKbc/ZVKlNwig0kR70yJHLItCaevI1lAyFfWk51YIP9Ze08uV9HIZh0VxtLFm/ZDliIYU8Bg/3vx23JSniPM= Received: from VE1PR08MB4640.eurprd08.prod.outlook.com (10.255.27.75) by VE1PR08MB4816.eurprd08.prod.outlook.com (10.255.112.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2178.18; Thu, 22 Aug 2019 02:30:11 +0000 Received: from VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::e5ba:d190:d546:6ee3]) by VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::e5ba:d190:d546:6ee3%4]) with mapi id 15.20.2178.018; Thu, 22 Aug 2019 02:30:11 +0000 From: "Phil Yang (Arm Technology China)" To: "Phil Yang (Arm Technology China)" , Honnappa Nagarahalli CC: "dev@dpdk.org" , nd , nd Thread-Topic: [dpdk-dev] Sync up status for Mellanox PMD barrier investigation Thread-Index: AdVTU+BzSgbFe8XTToWEakhqRMznOwEsHykAACJ3vDA= Date: Thu, 22 Aug 2019 02:30:11 +0000 Message-ID: References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: b27c16ff-61c7-4089-82e2-8731c9b270e9.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 4daa28fe-a11a-4d6e-bbd2-08d726a8ac60 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600148)(711020)(4605104)(1401327)(4618075)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:VE1PR08MB4816; X-MS-TrafficTypeDiagnostic: VE1PR08MB4816:|AM5PR0802MB2593: X-MS-Exchange-PUrlCount: 5 x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:3513;OLM:3513; x-forefront-prvs: 01371B902F X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(136003)(39850400004)(376002)(396003)(366004)(346002)(13464003)(199004)(189003)(305945005)(6246003)(81156014)(81166006)(86362001)(8676002)(6116002)(74316002)(6636002)(11346002)(476003)(7736002)(3846002)(53936002)(14454004)(25786009)(486006)(8936002)(186003)(6306002)(102836004)(76176011)(9686003)(14444005)(2906002)(99286004)(26005)(316002)(7696005)(256004)(5024004)(52536014)(54906003)(55016002)(4326008)(5660300002)(71200400001)(71190400001)(6506007)(55236004)(478600001)(66446008)(53546011)(76116006)(66476007)(110136005)(66946007)(64756008)(446003)(66066001)(33656002)(229853002)(6436002)(966005)(66556008); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4816; H:VE1PR08MB4640.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: 2vnIdhvrtPIvaHJez1GyABjsbsMr9PiUjb5c9kSMdwjynlajZo/vmZjud24Tw2For1YcGOooLCzZLaKrME/sP4MQa55uhLjeWmo/Z/7zzluZeLx9PdtP/yDSAawVfX77aI+O6X+17SZ0NDtHKC2j8kCrNlPvn1ovcHdhyXbeDqN5fZc6ahmPnUzB0ETMI4Y9Zz1eMfAcXnSaXfV2VP7tP8M70+1hSn7GhJYIL/cC9JpZOOVCTpwRDwQbZrl7VZd/mSGBPlzPlIa4cdk3VQ+CU/p4XnkTm4Q6KP9Wfkxys6w1t33A+HVhnZwRevRVf7VcD5pfn39oVXe7G4XW2yWMa21nFCORpMhE0UKQ8e2hdhr/zDpCi8iyaPp1sTgfIODYieUe8JUd/FIGEQZdvsV+Bb8hj8hiMgDOLm8H0rWDv00= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4816 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT058.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(39860400002)(376002)(346002)(396003)(136003)(2980300002)(13464003)(199004)(189003)(99286004)(6636002)(356004)(8936002)(8746002)(229853002)(81156014)(81166006)(76176011)(8676002)(74316002)(5660300002)(446003)(102836004)(63370400001)(63350400001)(14454004)(76130400001)(97756001)(52536014)(53546011)(6306002)(14444005)(5024004)(6506007)(70206006)(126002)(11346002)(476003)(486006)(9686003)(23726003)(3846002)(6116002)(7696005)(22756006)(70586007)(46406003)(86362001)(55016002)(336012)(54906003)(316002)(47776003)(478600001)(186003)(26005)(305945005)(50466002)(26826003)(110136005)(7736002)(2906002)(25786009)(66066001)(966005)(33656002)(4326008)(6246003)(36906005); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0802MB2593; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: c73801a3-511d-4edc-2b43-08d726a8a854 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(710020)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:AM5PR0802MB2593; NoDisclaimer: True X-Forefront-PRVS: 01371B902F X-Microsoft-Antispam-Message-Info: mbDUtroLyzqedFLfgbhPNXRfHHC4l/uMU7V3skEWbLEXfFpOLq2atxsN2w5sTSTgw5nBhXdh4ECWHXPlAtfrLv+LmqmqC9ngHUkkuMV5UgX0sxrNlOLWmivb85vlYuTv0h4GhEAPzVJUQy3JaLg7EjfcksY21sMNJ41bAYc6nQ6RmY12y+gObLBLo+XIN32MAG62QTHkIaow7IdlBFJUJ71+/7OlyOiw8OCyvCeTLexCPF/GJpokgAFJk3WxMWLbLH3BSZEQv/7sR/RvaJ2RASOOjmbY1M09VZVN5V5tDQUueJGByoMvYnvkRL3P0bM4VXiiEHBtuTfE1En/0sIM8vgkE20OIYDxQUJAs03rFw/aUZLIKm2ZefoS1zNzXSpT0cIAn8/1zt/fuo0wLGHTUlWlLPgA2y5Cuuy4IwjxZsA= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Aug 2019 02:30:18.3815 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4daa28fe-a11a-4d6e-bbd2-08d726a8ac60 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0802MB2593 Subject: Re: [dpdk-dev] Sync up status for Mellanox PMD barrier investigation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Please disregard my last message. It was mistakenly sent to the wrong group= .=20 Sorry about that. Thanks, Phil Yang > -----Original Message----- > From: dev On Behalf Of Phil Yang (Arm > Technology China) > Sent: Wednesday, August 21, 2019 5:58 PM > To: Honnappa Nagarahalli > Cc: dev@dpdk.org; nd > Subject: Re: [dpdk-dev] Sync up status for Mellanox PMD barrier > investigation >=20 > Some update for this thread. >=20 > In the most critical datapath of mlx5 PMD, there are some rte_cio_w/rmb, > 'dmb osh' on aarch64, in use. > C11 atomic is good for replacing the rte_smp_r/wmb to relax the data > synchronization barrier between CPUs. > However, mlx5 PMD needs to write data back to the HW, so it used a lot o= f > rte_cio_r/wmb to synchronize data. >=20 > Please check details below. All comments are welcomed. Thanks. >=20 > //////////////////// Data path /////////////////////////// > drivers/net/mlx5/mlx5_rxtx.c=3D950=3Dmlx5_rx_err_handle(struct > mlx5_rxq_data *rxq, uint8_t mbuf_prepare) > drivers/net/mlx5/mlx5_rxtx.c:1002: rte_cio_wmb(); > drivers/net/mlx5/mlx5_rxtx.c:1004: rte_cio_wmb(); > drivers/net/mlx5/mlx5_rxtx.c:1010: rte_cio_wmb(); > drivers/net/mlx5/mlx5_rxtx.c=3D1272=3Dmlx5_rx_burst(void *dpdk_rxq, struc= t > rte_mbuf **pkts, uint16_t pkts_n) > drivers/net/mlx5/mlx5_rxtx.c:1385: rte_cio_wmb(); > drivers/net/mlx5/mlx5_rxtx.c:1387: rte_cio_wmb(); > drivers/net/mlx5/mlx5_rxtx.c=3D1549=3Dmlx5_rx_burst_mprq(void *dpdk_rxq, > struct rte_mbuf **pkts, uint16_t pkts_n) > drivers/net/mlx5/mlx5_rxtx.c:1741: rte_cio_wmb(); > drivers/net/mlx5/mlx5_rxtx.c:1745: rte_cio_wmb(); > drivers/net/mlx5/mlx5_rxtx_vec_neon.h=3D366=3Drxq_burst_v(struct > mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, > drivers/net/mlx5/mlx5_rxtx_vec_neon.h:530: rte_cio_rmb(); >=20 > Commit messages: > net/mlx5: cleanup memory barriers: mlx5_rx_burst > https://git.dpdk.org/dpdk/commit/?id=3D9afa3f74658afc0e21fbe5c3884c55a21 > ff49299 >=20 > net/mlx5: add Multi-Packet Rx support : mlx5_rx_burst_mprq > https://git.dpdk.org/dpdk/commit/?id=3D7d6bf6b866b8c25ec06539b3eeed1db > 4f785577c >=20 > net/mlx5: use coherent I/O memory barrier > https://git.dpdk.org/dpdk/commit/drivers/net/mlx5/mlx5_rxtx.c?id=3D0cfdc1= 8 > 08de82357a924a479dc3f89de88cd91c2 >=20 > net/mlx5: extend Rx completion with error handling > https://git.dpdk.org/dpdk/commit/drivers/net/mlx5/mlx5_rxtx.c?id=3D88c073 > 3535d6a7ce79045d4d57a1d78d904067c8 >=20 > net/mlx5: fix synchronization on polling Rx completions > https://git.dpdk.org/dpdk/commit/?id=3D1742c2d9fab07e66209f2d14e7daa508 > 29fc4423 >=20 >=20 > Thanks, > Phil Yang >=20 > From: Phil Yang (Arm Technology China) > Sent: Thursday, August 15, 2019 6:35 PM > To: Honnappa Nagarahalli > Subject: Sync up status for Mellanox PMD barrier investigation >=20 > Hi Honnappa, >=20 > I have checked all the barriers in mlx5 PMD data path. In my understandin= g, it > used the barrier correctly (Use DMB to synchronize the memory data > between CPUs). > The attachment is the list of positions of these barriers. > I just want to sync up with you the status. Do you have any idea or > suggestion on which part should we start to optimization? >=20 > Best Regards, > Phil Yang