From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40074.outbound.protection.outlook.com [40.107.4.74]) by dpdk.org (Postfix) with ESMTP id 643231B1FF; Wed, 9 Jan 2019 11:11:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+VmpaR3c9fsr5klS+8jgI3k3rU8NLnREw5+n2hOYW/U=; b=ru4kjvrlvy8M7LLJFecbEhUty78uhFkz4xwzZ+F2XkFN9i2CLMmb4VCTKR00mg3kNEP4GjsZKLwT7Y6AJNyltIOD0YzJ46hn6WtbPXxG2HYfjfOyfTyk1oa+CLzN0h0j7ejqGWBKK8FIwQ/q6q/Mg4AfZdRgfFmvPH1YuTw7XXM= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB4091.eurprd05.prod.outlook.com (52.134.68.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1516.13; Wed, 9 Jan 2019 10:11:16 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::d43a:3775:8af7:29c6]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::d43a:3775:8af7:29c6%4]) with mapi id 15.20.1495.011; Wed, 9 Jan 2019 10:11:15 +0000 From: Yongseok Koh To: David Marchand CC: Olivier Matz , Shahaf Shuler , "dev@dpdk.org" , "stable@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] net/mlx5: fix instruction hotspot on replenishing Rx buffer Thread-Index: AQHUp/8QJWOIY9EhFUmeyBFvcPXbJaWmsfaAgAABUoCAAAJzgIAAAZWA Date: Wed, 9 Jan 2019 10:11:15 +0000 Message-ID: <5C96B931-B5F5-4BAA-A23C-E87505B423D0@mellanox.com> References: <20190109085426.39965-1-yskoh@mellanox.com> <20190109095205.us53xaocvokx4jog@glumotte.dev.6wind.com> <45D5A06B-4014-428E-A588-6E188C87A5AA@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-originating-ip: [69.181.245.183] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB3PR0502MB4091; 6:YAvQP+ZkfzsW5MetDh7BWfYdb5Q02GjX76pFn9dqoyCExxdwb6czDs5zYhSP8EfJgd2gejtAPcAy+lbEo3FgqmnLkLUJ684GGY9bSd61ZhPrKiXZw/K67d+EVczklchKFZGJnBfqKa/alQa9LB8E+gLiP1sfGv2JC2vf7Snbwwp63rdA5IkUqLKmxXmAdFVVlunWdh+Fq9okYU64X4tv4orRJZV4BVvLHDVEtWbBslWkDnkU94HVCMJXRSgGnC6+/ZBfQ57mgeQQDfi5foChqgYP2SWBfyXtXaQIvhPl76wav9Zl3HmQyuCpqpKzzEcttyvmhlhf9XM1ZRmQRmdu/iKbdfhqQWx7tz6gZcSZU7qTgkGq6xoVMsDCh8Q4fenYVP+EDH1uTdGPSQva/0U4+ZnFenGEodQr6g1613W9DZocBWJ8T3h536/FoaXmXFVDy1okQLDywkREdobVwBz0Rw==; 5:v/sXPKSqRLOISGowgU1aJ4+WitRrkyPmcxAn6ulDULvnVbUtd+k5vYBa5UkyaCJPv5dqi0DluGZ83uQjzonPd7I3Q4LYg+qo4kzAthT+619tu2sYNMDytIWmajIs1vhxYQacERG4B49gYbo8PJKFfNkTKldJA1fCLzYLvkFYCaPA1iJ30sSfkE+lq9KV3SK0J2tGt0MyHlBcLYOrypzStQ==; 7:x4VEBNX80lqwa2Eba7sBDjupHRotz+vP53LBkbVLeIFl2ftbUdohEcdtfOsKgFrypX3v1PDzxtasLj7yEJ9iPojkgdt2v4y4GGuuWGRe8mVjCybcJwB4l0UTW1rbjCtlERzeE6mizHj/IGX4Y/RT5Q== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 9cbbeea8-0916-4bc9-e6f7-08d6761aca8f x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB4091; x-ms-traffictypediagnostic: DB3PR0502MB4091: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(8220060)(2401047)(8121501046)(10201501046)(3002001)(3231475)(944501520)(52105112)(93006095)(93001095)(6055026)(6041310)(20161123562045)(20161123560045)(20161123558120)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:DB3PR0502MB4091; BCL:0; PCL:0; RULEID:; SRVR:DB3PR0502MB4091; x-forefront-prvs: 0912297777 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(376002)(396003)(366004)(136003)(346002)(51914003)(199004)(189003)(99286004)(256004)(105586002)(106356001)(486006)(6436002)(6486002)(6512007)(25786009)(53936002)(229853002)(4326008)(6246003)(2906002)(6916009)(5660300001)(6116002)(3846002)(97736004)(66066001)(476003)(11346002)(2616005)(446003)(316002)(54906003)(7736002)(86362001)(305945005)(26005)(6506007)(53546011)(102836004)(186003)(93886005)(76176011)(71200400001)(71190400001)(83716004)(81156014)(81166006)(8676002)(33656002)(82746002)(8936002)(36756003)(68736007)(478600001)(14454004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB4091; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: HDKVX4WQMEIIQPyLSdxi799fYnoJ7EB6briQKvhhb3/M4cOBHS8Wk3KWq7/G5/eFTuzH5AFpjEfO1FpiTdSc9/hWnpS+APi6PirDZrVxYIgEIueGEa+GrLuVZspie+376PnhgoHJ2qFT0REmt1XD4yGQ8q9JLhpb+AE8bt1T19lnbo2cO2rhQV3tC6WgEy+5G7Dsunoi1CwM+ESRosPtghSlSyeyn0cjETsl7Ij8EjTpuouQ6AhRJnAsoeVxFQ2oP53pDTCnA5LaN4PBygoMD/9j8l/aqIp7D1+/y2jRxxUECfDf5yWadFdOfm/qCQfx spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9cbbeea8-0916-4bc9-e6f7-08d6761aca8f X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jan 2019 10:11:15.7138 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB4091 Subject: Re: [dpdk-dev] [PATCH] net/mlx5: fix instruction hotspot on replenishing Rx buffer X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2019 10:11:17 -0000 > On Jan 9, 2019, at 2:05 AM, David Marchand wr= ote: >=20 >=20 >=20 > On Wed, Jan 9, 2019 at 10:56 AM Yongseok Koh wrote: >=20 > > On Jan 9, 2019, at 1:52 AM, Olivier Matz wrote= : > >=20 > > On Wed, Jan 09, 2019 at 10:38:07AM +0100, David Marchand wrote: > >> On Wed, Jan 9, 2019 at 9:54 AM Yongseok Koh wrote= : > >>=20 > >>> On replenishing Rx buffers for vectorized Rx, mbuf->buf_addr isn't ne= eded > >>> to be accessed as it is static and easily calculated from the mbuf ad= dress. > >>> Accessing the mbuf content causes unnecessary load stall and it is wo= rsened > >>> on ARM. > >>>=20 > >>> Fixes: 545b884b1da3 ("net/mlx5: fix buffer address posting in SSE Rx"= ) > >>> Cc: stable@dpdk.org > >>>=20 > >>> Signed-off-by: Yongseok Koh > >>> --- > >>> drivers/net/mlx5/mlx5_rxtx_vec.h | 8 ++++++-- > >>> 1 file changed, 6 insertions(+), 2 deletions(-) > >>>=20 > >>> diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h > >>> b/drivers/net/mlx5/mlx5_rxtx_vec.h > >>> index fda7004e2d..ced5547307 100644 > >>> --- a/drivers/net/mlx5/mlx5_rxtx_vec.h > >>> +++ b/drivers/net/mlx5/mlx5_rxtx_vec.h > >>> @@ -102,8 +102,12 @@ mlx5_rx_replenish_bulk_mbuf(struct mlx5_rxq_data > >>> *rxq, uint16_t n) > >>> return; > >>> } > >>> for (i =3D 0; i < n; ++i) { > >>> - wq[i].addr =3D rte_cpu_to_be_64((uintptr_t)elts[i]->b= uf_addr > >>> + > >>> - RTE_PKTMBUF_HEADROOM); > >>> + uintptr_t buf_addr =3D > >>> + (uintptr_t)elts[i] + sizeof(struct rte_mbuf) = + > >>> + rte_pktmbuf_priv_size(rxq->mp) + > >>> RTE_PKTMBUF_HEADROOM; > >>> + > >>> + assert(buf_addr =3D=3D (uintptr_t)elts[i]->buf_addr); > >>> + wq[i].addr =3D rte_cpu_to_be_64(buf_addr); > >>> /* If there's only one MR, no need to replace LKey in = WQE. > >>> */ > >>> if (unlikely(mlx5_mr_btree_len(&rxq->mr_ctrl.cache_bh)= > > >>> 1)) > >>> wq[i].lkey =3D mlx5_rx_mb2mr(rxq, elts[i]); > >>> -- > >>> 2.11.0 > >>>=20 > >>>=20 > >> How about having a macro / inline in the mbuf api to get this informat= ion > >> in a consistent/unique way ? > >> I can see we have this calculation at least in rte_pktmbuf_init() and > >> rte_pktmbuf_detach(). > >=20 > > Agree. Maybe rte_mbuf_default_buf_addr(m) ? >=20 > I'm also okay to add. Will come up with a new patch. >=20 > > Side note, is the assert() correct in the patch? I'd say there's a > > difference of RTE_PKTMBUF_HEADROOM between the 2 values. >=20 > Oops, my fault. Thanks for the catch, you saved a crash. :-) >=20 > Is this assert really necessary if we have a common macro ? > I was under the impression that this assert is there to catch misaligneme= nt between the mbuf api and the driver. It is still good to have. This can catch corruption of mbuf content which s= ometimes happens due to wrong mbuf handling in PMD or potential HW memory corruption= . Thanks, Yongseok