From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40057.outbound.protection.outlook.com [40.107.4.57]) by dpdk.org (Postfix) with ESMTP id 2D3081B31C for ; Mon, 12 Feb 2018 13:10:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=P6PoUkX4uEvWQ1VM33bjpWJWk5iaEokx7I6Z7B8qn44=; b=PbHn6mekw2MCRmPiSi09FG0cFFZoAEqjlcZaKKa9Pyqq9twWsZ5AN78ACnMA5Io358oz8qiq8GEMvFATE6AwAr2nSvOH9tjtxV41USbqlGmPBCv23XY6kFTWP2b1ZNSiChjmqGdSo/te0SIySyFaEzt52mWT6EKS6sGzFB2qeyA= Received: from AM4PR0501MB2657.eurprd05.prod.outlook.com (10.172.215.19) by AM4PR0501MB2257.eurprd05.prod.outlook.com (10.165.82.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.485.10; Mon, 12 Feb 2018 12:10:13 +0000 Received: from AM4PR0501MB2657.eurprd05.prod.outlook.com ([fe80::80c6:df5:b1b0:ff05]) by AM4PR0501MB2657.eurprd05.prod.outlook.com ([fe80::80c6:df5:b1b0:ff05%17]) with mapi id 15.20.0485.013; Mon, 12 Feb 2018 12:10:13 +0000 From: Matan Azrad To: Jerin Jacob CC: "dev@dpdk.org" , "ferruh.yigit@intel.com" , Thomas Monjalon , "Konstantin Ananyev" , Pavan Nikhilesh Thread-Topic: [dpdk-dev] [PATCH v2] ethdev: make ethdev data cache aligned Thread-Index: AQHTo8YSOfZN9JQRQ0q03fPDgObqg6OgdnKAgAAI7YCAAAGAoIAADdoAgAAVEGA= Date: Mon, 12 Feb 2018 12:10:13 +0000 Message-ID: References: <20180210094220.16201-1-jerin.jacob@caviumnetworks.com> <20180212055439.6462-1-jerin.jacob@caviumnetworks.com> <20180212092544.GA24831@jerin> <20180212102041.GA29193@jerin> In-Reply-To: <20180212102041.GA29193@jerin> Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM4PR0501MB2257; 7:1ayJYY3O/dzF5Nzu0G2jgNh7bBphXH0G2LwkVbPboBNmWzRUDyri3KOk2PVdCGjDhoxsXOzl2KU3PmlCP2/v1CWuXwPlLYOcZGKQ1sgKf8YmL3A3v4YX1KJe8vnphyVIyAX1S6KDViNozC/r1B3tGQCfLLv6dZ/5i1MEi2yJDcVepYelw0PhRw5uLLElHaj69aKrIk9J/A83sh0BBy/TTxx0wvJYY6jKqGPCPguy8VgM+dGjPQt+TVkNDR5qgW3N x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 1e94421c-9c28-4ea4-a9a0-08d57211924d x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603307)(7153060)(7193020); SRVR:AM4PR0501MB2257; x-ms-traffictypediagnostic: AM4PR0501MB2257: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(60795455431006)(131327999870524)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040501)(2401047)(5005006)(8121501046)(3231101)(944501161)(3002001)(93006095)(93001095)(10201501046)(6055026)(6041288)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123558120)(20161123564045)(6072148)(201708071742011); SRVR:AM4PR0501MB2257; BCL:0; PCL:0; RULEID:; SRVR:AM4PR0501MB2257; x-forefront-prvs: 0581B5AB35 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(376002)(396003)(366004)(346002)(39380400002)(189003)(199004)(51444003)(13464003)(25786009)(186003)(5250100002)(3846002)(7696005)(478600001)(6116002)(53936002)(81156014)(4326008)(8936002)(102836004)(3660700001)(3280700002)(6436002)(8676002)(2906002)(81166006)(86362001)(2900100001)(6506007)(59450400001)(26005)(55016002)(97736004)(6916009)(9686003)(2950100002)(76176011)(229853002)(6246003)(68736007)(93886005)(105586002)(316002)(54906003)(14454004)(5660300001)(66066001)(106356001)(7736002)(305945005)(33656002)(99286004)(74316002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR0501MB2257; H:AM4PR0501MB2657.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: i76emcArh/zrQ5sLclI7qnHz1wSP8xSFcffczKE4i+ETTR6j+hw8e/F5U5NpRLwHQsp0CsZaWMjWtz/cXwWGaw== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1e94421c-9c28-4ea4-a9a0-08d57211924d X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Feb 2018 12:10:13.6741 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR0501MB2257 Subject: Re: [dpdk-dev] [PATCH v2] ethdev: make ethdev data cache aligned X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Feb 2018 12:10:15 -0000 Hi Jerin From: Jerin Jacob, Sent: Monday, February 12, 2018 12:21 PM > -----Original Message----- > > Date: Mon, 12 Feb 2018 09:49:55 +0000 > > From: Matan Azrad > > To: Jerin Jacob > > CC: "dev@dpdk.org" , "ferruh.yigit@intel.com" > > , Thomas Monjalon , > > Konstantin Ananyev , Pavan Nikhilesh > > > > Subject: RE: [dpdk-dev] [PATCH v2] ethdev: make ethdev data cache > > aligned > > > > Hi Jerin > > > > From: Jerin Jacob, Sent: Monday, February 12, 2018 11:26 AM > > > -----Original Message----- > > > > Date: Mon, 12 Feb 2018 09:04:07 +0000 > > > > From: Matan Azrad > > > > To: Jerin Jacob , "dev@dpdk.org" > > > > > > > > CC: "ferruh.yigit@intel.com" , Thomas > > > > Monjalon , Konstantin Ananyev > > > > , Pavan Nikhilesh > > > > > > > > Subject: RE: [dpdk-dev] [PATCH v2] ethdev: make ethdev data cache > > > > aligned > > > > > > > > Hi Jerin > > > > > > > > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > > > > > Since struct rte_eth_dev_data used in the fast path, making it > > > > > as cache aligned. > > > > > > > > > > Fixes: af75078fece3 ("first public release") > > > > > Fixes: 5b7ba31148a8 ("ethdev: add port ownership") > > > > > > > > Looks like it is just improvement. > > > > No need the above "fixes" lines (also fix title is not needed as yo= u did). > > > > > > I think, It varies the way we look at it. I don't think, either way > > > it matters in the commit log. > > > > I think this commit improves " af75078fece3 ("first public release")" > > since there was no intention to aligned rte_eth_dev_data in the first > commit created it, The relevant fields in the first port probably was ali= gned > without any intention(if no , what's about the other ports?). >=20 > In my view it is a bug as it missed to align to cache line from the first= release. Can be a fix of the first release. > and before adding struct rte_vlan_filter_conf vlan_filter_conf and struct > rte_eth_dev_owner owner; it was 128B aligned just by luck for all the por= ts. > and further ("ethdev: add port ownership") changes the complete alignment > by introducing a container type on top of it. >=20 So, ("ethdev: add port ownership") and rte_vlan_filter_conf vlan_filter_c= onf patch just exposed another issue of performance... Moreover any fields adding patch to the rte_eth_dev_data structure from the= first release didn't took the performance into account.. So, all of them have a bug? I think that if you want to consider it as a fix it should be for the first= commit didn't take into account the performance alignment. And you should backport it with Cc: stable@dpdk.org. > Do you think, any reason why this fastpath structure SHOULD NOT BE cache > aligned ? >=20 I agree it should be(actually the datapath fields should be in the same cac= he line). I think also it will be good to detail the fields which should be in the sa= me cache line. > > > > My suggestion is to just explain why the rte_eth_dev_data structure > should be aligned and to align it as improvement, even to backport it to > stable branch to improve the early LTS versions for all the ports. >=20 > I don't think, There is a VERY specific reason for rte_eth_dev_data to ca= che > aligned. it applies to all fastpath functions. > IMO, We are making fastpath structure cache aligned due to, > 1) Avoid sharing the element with another cache line > 2) Compiler/CPU can access the elements in natural alignment if the top m= ost > element is aligned. Agree. =20 >=20 > It simple, 1 port, 1 queue l3fwd setup. > sudo ./examples/l3fwd/build/l3fwd -c 0xff00 -- -p 0x1 --config=3D"(0,0,9)= " >=20 > It is not black and white. it will vary based on global variable alignmen= t etc in > the binary. > i.e if apply this patch on any RANDOM change set you will not get a fixed > improvement. >=20 > Hope this clarifies. Yes, you should add the improved scenario to the commit log with the result= s.