From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 19F3FA046B for ; Wed, 21 Aug 2019 11:58:10 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D7F6B1BEEE; Wed, 21 Aug 2019 11:58:08 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140074.outbound.protection.outlook.com [40.107.14.74]) by dpdk.org (Postfix) with ESMTP id 286041BEEC for ; Wed, 21 Aug 2019 11:58:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uXFQSgQCdy7Zn6l2C0Ejtv1BO0Sol1hKPWE+0YWMzJ0=; b=dqbqEKh+jwD5l5bKlz17LjRlEzGQDmgXpLai5vgB7KZeB1ssy4T5Tu42V0DQI9Qxzrh1fyR+uqUEpyYWUAdG5cJQkYm4ljwgXLmPN8xf60/m/UbyNvQGYluXt9ew8lhBfi2rNk7Y1noBbqtI+UQJcGuM/UtQQccpZW8CVD9x03I= Received: from VI1PR08CA0096.eurprd08.prod.outlook.com (10.175.228.22) by AM5PR0801MB1844.eurprd08.prod.outlook.com (10.169.246.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2178.16; Wed, 21 Aug 2019 09:58:05 +0000 Received: from VE1EUR03FT061.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::206) by VI1PR08CA0096.outlook.office365.com (2603:10a6:800:d3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2178.16 via Frontend Transport; Wed, 21 Aug 2019 09:58:05 +0000 Authentication-Results: spf=temperror (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=temperror action=none header.from=arm.com; Received-SPF: TempError (protection.outlook.com: error in processing during lookup of arm.com: DNS Timeout) Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT061.mail.protection.outlook.com (10.152.19.220) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2178.16 via Frontend Transport; Wed, 21 Aug 2019 09:58:04 +0000 Received: ("Tessian outbound cc8a947d4660:v26"); Wed, 21 Aug 2019 09:58:03 +0000 X-CR-MTA-TID: 64aa7808 Received: from 9c8ba6a02a0b.1 (cr-mta-lb-1.cr-mta-net [104.47.6.57]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id BBE2D178-76A3-44B3-9104-F38C318EA183.1; Wed, 21 Aug 2019 09:57:58 +0000 Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-ve1eur02lp2057.outbound.protection.outlook.com [104.47.6.57]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9c8ba6a02a0b.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384); Wed, 21 Aug 2019 09:57:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=czfh9ifQLejDvVjBSxPTNNlDo9DlQ+UGtcvbeHUT0tSd90QVYkJUQYIfTDFyO7fmdg7yHHZWqHNmCfm1+AGMeHukJA6tqYA7ZGn00HGDNhCVLwvkxkfY4lliStzGVncmaKVak7KlzE/NTHZb3hrFi9J+k1npxWNW1PllEhszIFV3URX7WiHmRvjQ+rP9Qkw/N2expxyMqzJ2OP/r3F9+bNvZUxRjsyromzlG/PgSlxByqxjpSpANtfc7mS6oH3jA8tOoaDtjTKEAOY7K4SY95uLz6o49Y6ilZG5npBWUmG/d9907xVjFORbmUg2jkLDz94KbLsK/S0MVDs6ASnqp6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uXFQSgQCdy7Zn6l2C0Ejtv1BO0Sol1hKPWE+0YWMzJ0=; b=WuiBUiQgqPxqjufAizADQ74o4snpXw9xwqhwln+lwbuLhUimt0jtt9Q55e+CT90vbXMW8g/Bg40PcZmfH+6nI6aoKIrwG6Kdnuqc2fDvPWZ1Fd4fJnn4rquVkoR3FQWoczt61VVC6T2nns9SBBmyAVwMnJkm83UeRJOOdj5FOC6oUg0Dsb7QagQeHG37soRmHaW5tQ0N5ii5f+b3QR14OOvIu0YLIjJ0h8TI24De0dqMy/qHrsxa5TeMkUbCgl0POYhVu2C/rsuBc+Li3xomlCAiYT0e2S79CtHpili5a9AI3GJ2Ht7qm5iYXkRgfqa4NONFGhK8GfURMyyOu6lxUQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uXFQSgQCdy7Zn6l2C0Ejtv1BO0Sol1hKPWE+0YWMzJ0=; b=dqbqEKh+jwD5l5bKlz17LjRlEzGQDmgXpLai5vgB7KZeB1ssy4T5Tu42V0DQI9Qxzrh1fyR+uqUEpyYWUAdG5cJQkYm4ljwgXLmPN8xf60/m/UbyNvQGYluXt9ew8lhBfi2rNk7Y1noBbqtI+UQJcGuM/UtQQccpZW8CVD9x03I= Received: from VE1PR08MB4640.eurprd08.prod.outlook.com (10.255.27.75) by VE1PR08MB4701.eurprd08.prod.outlook.com (10.255.114.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2178.18; Wed, 21 Aug 2019 09:57:54 +0000 Received: from VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::e5ba:d190:d546:6ee3]) by VE1PR08MB4640.eurprd08.prod.outlook.com ([fe80::e5ba:d190:d546:6ee3%4]) with mapi id 15.20.2178.018; Wed, 21 Aug 2019 09:57:54 +0000 From: "Phil Yang (Arm Technology China)" To: Honnappa Nagarahalli CC: "dev@dpdk.org" , nd Thread-Topic: Sync up status for Mellanox PMD barrier investigation Thread-Index: AdVTU+BzSgbFe8XTToWEakhqRMznOwEsHykA Date: Wed, 21 Aug 2019 09:57:54 +0000 Message-ID: References: In-Reply-To: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 0ac834ea-b89a-4e0b-9d11-778c09a464d0.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-Correlation-Id: 184bebad-bb84-4875-fb29-08d7261e0f4b X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam-Untrusted: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600148)(711020)(4605104)(1401327)(4618075)(4534185)(7168020)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:VE1PR08MB4701; X-MS-TrafficTypeDiagnostic: VE1PR08MB4701:|AM5PR0801MB1844: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:3044;OLM:3044; x-forefront-prvs: 0136C1DDA4 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(376002)(39860400002)(366004)(136003)(346002)(396003)(189003)(199004)(26005)(2906002)(6116002)(790700001)(11346002)(186003)(53546011)(74316002)(102836004)(81156014)(476003)(9686003)(66066001)(8936002)(33656002)(99286004)(5660300002)(3846002)(6506007)(486006)(55236004)(52536014)(6306002)(55016002)(6436002)(6246003)(53936002)(6862004)(86362001)(14454004)(6636002)(446003)(54896002)(7696005)(4326008)(4743002)(478600001)(71190400001)(76176011)(66556008)(8676002)(25786009)(5024004)(54906003)(71200400001)(316002)(7736002)(229853002)(14444005)(66476007)(64756008)(256004)(966005)(81166006)(66946007)(76116006)(66446008); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1PR08MB4701; H:VE1PR08MB4640.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Message-Info-Original: VMA6EzkrKahAxsVMhJRgmvL9abSErZDh9Pm4uLEGHn4KWZrvua6Fnmv/Gy2V8cklPAzyYiasRrhz45ydo1BspCAew0ke1X69mqIS2lkFHFLu8huUCth5pD/k5EcPLsVY1Pw8uk3e90HIkbqsrt1DJGGRbH6uAddj3YmdGgB1Gyu/5uaqKh+gaHw4l/1Wr7Xk7mFUqirbSHRW9TSfatMu+4n3nx6YSLTb3qKbquo4JBA9ULLINxzU9wtN9CWbknmp4Loxhoam6TODljQSTMuh3xsu+oEFaEMUikvfMlEH9Zo1n04MU+1Ox25VirArdE89moqCt+9SCF+qPBIdnpMWS3YSOJJJg4tHxghJ3PQH4czLa6X8H1mUN9dUb9k7MgIrIDJFb1xNpSkIEUXgSs8qlOY0N13VNchZZWcDU4VyZXg= MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4701 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Phil.Yang@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT061.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(346002)(136003)(396003)(39860400002)(376002)(2980300002)(199004)(189003)(22756006)(76130400001)(26005)(52536014)(476003)(126002)(54906003)(26826003)(36906005)(5660300002)(16586007)(186003)(486006)(478600001)(9686003)(54896002)(6306002)(316002)(55016002)(7696005)(33656002)(70586007)(53546011)(102836004)(76176011)(6506007)(6636002)(14454004)(966005)(70206006)(99286004)(63370400001)(446003)(63350400001)(66066001)(8936002)(25786009)(2906002)(8676002)(6246003)(86362001)(71190400001)(4326008)(4743002)(336012)(74316002)(11346002)(229853002)(3846002)(7736002)(6116002)(790700001)(81166006)(14444005)(6862004)(5024004)(81156014)(356004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0801MB1844; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:TempError; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 827c2d21-3af6-41ce-6294-08d7261e0933 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(710020)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:AM5PR0801MB1844; X-MS-Exchange-PUrlCount: 5 NoDisclaimer: True X-Forefront-PRVS: 0136C1DDA4 X-Microsoft-Antispam-Message-Info: JSHBBqjSFENCmTjdClqIOcyXOundkZHAzfqWeAmHvJg382ofaJNRxH6WMz17nq0ZuFAa7JuUCgHveBcfs+SMcRcDZzEamJ9LsAjH+mSEEcabrqq68jd5VWSbQBPoDYmuQD2fNHWqBS72sQ5lXJIqGJqkSpfeWuzgMd7SerWsdWbuwC5wXPDGdbP6+NvM2lG6GgPXDTbwhWrvb03Wv8ref+zWZ2Jjq6pVOIJucYqg2Tl31V76VZ+euGAWz4IZmRBpWF8z7E0ZEZBwazBn+NcLEnrR1EirLjxMEQRcXKbv6EYFhwZObAfOhxIUfyA0fBqwGLzOtNcNsc4RRZbeELJmfo95m5wUg7d8PIZWJv/+5FuPACLKI1iaZxkKAAROapWqPR1QD7PNKdhTPQhHH/4jqOZN2lynJf8Yq6Qg6sH5niI= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2019 09:58:04.1845 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 184bebad-bb84-4875-fb29-08d7261e0f4b X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0801MB1844 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] Sync up status for Mellanox PMD barrier investigation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Some update for this thread. In the most critical datapath of mlx5 PMD, there are some rte_cio_w/rmb, 'd= mb osh' on aarch64, in use. C11 atomic is good for replacing the rte_smp_r/wmb to relax the data synchr= onization barrier between CPUs. However, mlx5 PMD needs to write data back to the HW, so it used a lot of = rte_cio_r/wmb to synchronize data. Please check details below. All comments are welcomed. Thanks. //////////////////// Data path /////////////////////////// drivers/net/mlx5/mlx5_rxtx.c=3D950=3Dmlx5_rx_err_handle(struct mlx5_rxq_dat= a *rxq, uint8_t mbuf_prepare) drivers/net/mlx5/mlx5_rxtx.c:1002: rte_cio_wmb(); drivers/net/mlx5/mlx5_rxtx.c:1004: rte_cio_wmb(); drivers/net/mlx5/mlx5_rxtx.c:1010: rte_cio_wmb(); drivers/net/mlx5/mlx5_rxtx.c=3D1272=3Dmlx5_rx_burst(void *dpdk_rxq, struct = rte_mbuf **pkts, uint16_t pkts_n) drivers/net/mlx5/mlx5_rxtx.c:1385: rte_cio_wmb(); drivers/net/mlx5/mlx5_rxtx.c:1387: rte_cio_wmb(); drivers/net/mlx5/mlx5_rxtx.c=3D1549=3Dmlx5_rx_burst_mprq(void *dpdk_rxq, st= ruct rte_mbuf **pkts, uint16_t pkts_n) drivers/net/mlx5/mlx5_rxtx.c:1741: rte_cio_wmb(); drivers/net/mlx5/mlx5_rxtx.c:1745: rte_cio_wmb(); drivers/net/mlx5/mlx5_rxtx_vec_neon.h=3D366=3Drxq_burst_v(struct mlx5_rxq_d= ata *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, drivers/net/mlx5/mlx5_rxtx_vec_neon.h:530: rte_cio_rmb(); Commit messages: net/mlx5: cleanup memory barriers: mlx5_rx_burst https://git.dpdk.org/dpdk/commit/?id=3D9afa3f74658afc0e21fbe5c3884c55a21ff4= 9299 net/mlx5: add Multi-Packet Rx support : mlx5_rx_burst_mprq https://git.dpdk.org/dpdk/commit/?id=3D7d6bf6b866b8c25ec06539b3eeed1db4f785= 577c net/mlx5: use coherent I/O memory barrier https://git.dpdk.org/dpdk/commit/drivers/net/mlx5/mlx5_rxtx.c?id=3D0cfdc180= 8de82357a924a479dc3f89de88cd91c2 net/mlx5: extend Rx completion with error handling https://git.dpdk.org/dpdk/commit/drivers/net/mlx5/mlx5_rxtx.c?id=3D88c07335= 35d6a7ce79045d4d57a1d78d904067c8 net/mlx5: fix synchronization on polling Rx completions https://git.dpdk.org/dpdk/commit/?id=3D1742c2d9fab07e66209f2d14e7daa50829fc= 4423 Thanks, Phil Yang From: Phil Yang (Arm Technology China) Sent: Thursday, August 15, 2019 6:35 PM To: Honnappa Nagarahalli Subject: Sync up status for Mellanox PMD barrier investigation Hi Honnappa, I have checked all the barriers in mlx5 PMD data path. In my understanding,= it used the barrier correctly (Use DMB to synchronize the memory data betw= een CPUs). The attachment is the list of positions of these barriers. I just want to sync up with you the status. Do you have any idea or suggest= ion on which part should we start to optimization? Best Regards, Phil Yang