From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40080.outbound.protection.outlook.com [40.107.4.80]) by dpdk.org (Postfix) with ESMTP id AB1091B1F1; Wed, 9 Jan 2019 10:56:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zYZh8/TBY4g35j05aYb/e2RlBb5BuuzfaLcVAmJ+8Q8=; b=tWrdm7h41yNoyW3rh8qCOOlLoiayNiT3Tq0+Hdy/AG6eepTxVuLUoaIYUoPgCWPq6G569MeURAbMpVLAxNQBPLHBkqBxfqI1WM8yydZyc4GGnhSWDCYTZZGMduGAO4ZrS9iCQYmFWLRjtlPlBQ/k9t+lwU5NSRt0xasH2nPJLwI= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB4028.eurprd05.prod.outlook.com (52.134.68.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1495.7; Wed, 9 Jan 2019 09:56:50 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::d43a:3775:8af7:29c6]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::d43a:3775:8af7:29c6%4]) with mapi id 15.20.1495.011; Wed, 9 Jan 2019 09:56:50 +0000 From: Yongseok Koh To: Olivier Matz CC: David Marchand , Shahaf Shuler , "dev@dpdk.org" , "stable@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] net/mlx5: fix instruction hotspot on replenishing Rx buffer Thread-Index: AQHUp/8QJWOIY9EhFUmeyBFvcPXbJaWmsfaAgAABUoA= Date: Wed, 9 Jan 2019 09:56:50 +0000 Message-ID: <45D5A06B-4014-428E-A588-6E188C87A5AA@mellanox.com> References: <20190109085426.39965-1-yskoh@mellanox.com> <20190109095205.us53xaocvokx4jog@glumotte.dev.6wind.com> In-Reply-To: <20190109095205.us53xaocvokx4jog@glumotte.dev.6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-originating-ip: [69.181.245.183] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB3PR0502MB4028; 6:ophUA1BmX9tqnn46XILnB0TxvsiZT7jVQFaLOu+B1VUurvoMGVRMLucxjQgK5kvyuOfirdcTUQM7OIol9nkq2clVlg/XVBAyAZJ6Z1O+nRCAFRpem91xieT+wVBpBOwqQq47RY1N6VVT10aCj6rWiuJKmzta0QqosZ76y9hVFflcs8pM3PP+9yzKKKmhWo7fo1rDWWPQN1aaTlqiwz7ny1b6FyEhxQhcKLCklLVHG5COoK2kZXzC46zETdnBvq3gNn4f1ptibktqJHrYdgiVENXQtERwH22Iizem6rwazefrEGY/K4ndQxi+R3qlO9/0ni7kOoGg5FZyq8tIKCa5xpGkKoxrQEUjgVLGsBlkm+Eh3f5wgk/nYdwwQl8RdNUO8jMdAungy+Sq5XB44JJnRZDQ2tHwJi0tt2JQsVxhVdbX/p2z5fWtKsaiiPx937I1V9FsqFoVkjttCpjXUddP3g==; 5:b6j7h0oHgEFK1CPlji+cM+VD09t+ZcqNuBh/dcKEfaOKq3Zt+3Xfa5YV8FsWmqP2g1ESCyuEeHIs7NwiQK6X8wcoP9Tu5NX9TwcqayX9lX+hfHdmstwPVzlf+Ryb216ndAQ9OxGrr3NeFnXcz0A7cOnAyTMlSkZUNyaHqKGMxnd7+CfFHcNLfT+WTFyHh0uU3FNTr4ShRKlpiCfVnhiykA==; 7:07c/D7Stnwu1Hf6bHIMD5qb3N7YZNpMshUcS/XJuDp005HOMTOH8oNRIW3AD7OSX5AjXkQiUEuVQGJEnXfV8tTQLW0gvOkfZxKNTKpJarY7bQAvXh9YXRNIEMgLLI5dZt8tIFLURq/HzQ60f/nymxQ== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: ef3db613-5cb0-41b4-bf13-08d67618c6ed x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB4028; x-ms-traffictypediagnostic: DB3PR0502MB4028: x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(908002)(999002)(5005026)(6040522)(8220060)(2401047)(8121501046)(10201501046)(93006095)(93001095)(3002001)(3231475)(944501520)(52105112)(6055026)(6041310)(20161123560045)(20161123558120)(20161123564045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:DB3PR0502MB4028; BCL:0; PCL:0; RULEID:; SRVR:DB3PR0502MB4028; x-forefront-prvs: 0912297777 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(136003)(39860400002)(366004)(376002)(346002)(189003)(199004)(51914003)(97736004)(33656002)(2906002)(53936002)(6512007)(86362001)(36756003)(8676002)(14454004)(76176011)(6486002)(486006)(81166006)(81156014)(7736002)(256004)(305945005)(8936002)(53546011)(71190400001)(229853002)(82746002)(316002)(71200400001)(6436002)(68736007)(99286004)(54906003)(66066001)(478600001)(6506007)(2616005)(476003)(102836004)(106356001)(83716004)(446003)(5660300001)(11346002)(186003)(26005)(6116002)(3846002)(105586002)(6246003)(25786009)(6916009)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB4028; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: i75Blwr1wTubYtVIDy5DeOeshePWTKZPsfHlOQ2rzh73LXwXO+FtzqXcMiFmonv+OZt2jJzlMV1PB1BHXblgJRDbprLMkLBR4pNpHk56AKt8U5WSo5YEsZ5zWzUzpSjQXlOFetz6HC1XVNXVcxg2Q80rjt1m1dd16EHRO1tkbc5+qcMYlZa2B6Kua5GBRJE4n1Zquxzo2+0kAm1wo/P5GXkVVayWQ6neP3V5qKS5Bhj6mth+UWqr4i3trp/4CWZzaU7fNTy37lX+tQzZFZ+Bk/0LSrp5Xqk0NDp5iZ/cNAtnrL3XqbdjKfYqo27lwmKc spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: ef3db613-5cb0-41b4-bf13-08d67618c6ed X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jan 2019 09:56:50.6715 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB4028 Subject: Re: [dpdk-dev] [PATCH] net/mlx5: fix instruction hotspot on replenishing Rx buffer X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2019 09:56:52 -0000 > On Jan 9, 2019, at 1:52 AM, Olivier Matz wrote: >=20 > On Wed, Jan 09, 2019 at 10:38:07AM +0100, David Marchand wrote: >> On Wed, Jan 9, 2019 at 9:54 AM Yongseok Koh wrote: >>=20 >>> On replenishing Rx buffers for vectorized Rx, mbuf->buf_addr isn't need= ed >>> to be accessed as it is static and easily calculated from the mbuf addr= ess. >>> Accessing the mbuf content causes unnecessary load stall and it is wors= ened >>> on ARM. >>>=20 >>> Fixes: 545b884b1da3 ("net/mlx5: fix buffer address posting in SSE Rx") >>> Cc: stable@dpdk.org >>>=20 >>> Signed-off-by: Yongseok Koh >>> --- >>> drivers/net/mlx5/mlx5_rxtx_vec.h | 8 ++++++-- >>> 1 file changed, 6 insertions(+), 2 deletions(-) >>>=20 >>> diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h >>> b/drivers/net/mlx5/mlx5_rxtx_vec.h >>> index fda7004e2d..ced5547307 100644 >>> --- a/drivers/net/mlx5/mlx5_rxtx_vec.h >>> +++ b/drivers/net/mlx5/mlx5_rxtx_vec.h >>> @@ -102,8 +102,12 @@ mlx5_rx_replenish_bulk_mbuf(struct mlx5_rxq_data >>> *rxq, uint16_t n) >>> return; >>> } >>> for (i =3D 0; i < n; ++i) { >>> - wq[i].addr =3D rte_cpu_to_be_64((uintptr_t)elts[i]->buf= _addr >>> + >>> - RTE_PKTMBUF_HEADROOM); >>> + uintptr_t buf_addr =3D >>> + (uintptr_t)elts[i] + sizeof(struct rte_mbuf) + >>> + rte_pktmbuf_priv_size(rxq->mp) + >>> RTE_PKTMBUF_HEADROOM; >>> + >>> + assert(buf_addr =3D=3D (uintptr_t)elts[i]->buf_addr); >>> + wq[i].addr =3D rte_cpu_to_be_64(buf_addr); >>> /* If there's only one MR, no need to replace LKey in WQ= E. >>> */ >>> if (unlikely(mlx5_mr_btree_len(&rxq->mr_ctrl.cache_bh) > >>> 1)) >>> wq[i].lkey =3D mlx5_rx_mb2mr(rxq, elts[i]); >>> -- >>> 2.11.0 >>>=20 >>>=20 >> How about having a macro / inline in the mbuf api to get this informatio= n >> in a consistent/unique way ? >> I can see we have this calculation at least in rte_pktmbuf_init() and >> rte_pktmbuf_detach(). >=20 > Agree. Maybe rte_mbuf_default_buf_addr(m) ? I'm also okay to add. Will come up with a new patch. > Side note, is the assert() correct in the patch? I'd say there's a > difference of RTE_PKTMBUF_HEADROOM between the 2 values. Oops, my fault. Thanks for the catch, you saved a crash. :-) Thanks, Yongseok