From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09CA34310D; Sat, 26 Aug 2023 19:49:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6846F40A7D; Sat, 26 Aug 2023 19:49:14 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2045.outbound.protection.outlook.com [40.107.223.45]) by mails.dpdk.org (Postfix) with ESMTP id CCE2E4068E for ; Sat, 26 Aug 2023 19:49:12 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SaQmstrUmMewye2Ak6cR3eAZcCoG6RDTX0u5oSsMt91pR1Vlsxg02cv+RRaidalMHGBq2KgMgzpQAJsrYVkZaPO6EaBWZ1ljdYTq1Pfs1m4UIvov4rkdJ9eR8Hkeq2+f+iSJlseS00cMuUSmb5gR3I9GY4R3NSNihappIoUHq4de8ihbc4b+ekXNosxxxB67hW3+f+p0Q3U7ZHNie1/0xsBXr/RFJrWdKek7WN7631MoMDZAuttcY6RjeC38AOqnHleRlNnBRp26h740nBOr4HyoUhnx8SjBmqvO+Whk65CXG+C4g+NA9xs2vkEbQ58tOOEtx7JwDizQGfjrN4g4bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/5goTPHRBPW50M4HR8W2fWwycVYq4HjI1+yEuqFwHjw=; b=Hm8U/c5e7nh27sV97Fc7i/ngPOh7FKIBlr++2F7gjWDjyp+Ce3jPDIiWNRP9vBBAbp3qPIWy8QLNgfMgcnCnd+NK7by1ofnHQy0Ro0x8BeHK0TTtBbIKHik/PaH6TVDAEIFZOtNW0+h+8/Y4Wtzcp8JTPy4PC82ssN295x9wNVB9hHV3jcypU3tJdJpWsVjRB+hCBIe2lukYBmxNX48P2tnNwmPynDHuUWdBPLY1P0tJxyCIIKpuLcTu3QNF1XZ+VS81yp2pf380ZJQhLvuk+1Sj9YQRrxAecb20d040aJVVRy1HaowMNMBB6nHHLCjBHCrWJtYtGmitS6uW98UFDg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/5goTPHRBPW50M4HR8W2fWwycVYq4HjI1+yEuqFwHjw=; b=cnWzAEvDmmfZydXYZ3xcGu785tNWOQEklhSNG62wLVVISDix67CUWZDr8xVu61tlgI+r6LVxka7bHZtXYe+3eeT5ItClEzhbwYYRvmxbQh3k9HQgt/tjJ+eriLv8FgWBfYialfP0jdWS1+NJ/iFTjAg5lFn5gS8Gul0vTkfS+FnF2TWrZbLE3UotT17/PkksJThQywT5quIgigIET9u0uKXm9bkqW08JL2kqW4JTYMuS+XSxZSAPnh3XRtDDGtWqr2jZ9hz8MPz1fzhRsbDJDwnQl1V/dQuWztYOT0w5D1090DucLP1GoPewnub79EnAiPl7H8yWGytTVZY02AiW0Q== Received: from SA3PR12MB8811.namprd12.prod.outlook.com (2603:10b6:806:312::18) by PH7PR12MB6763.namprd12.prod.outlook.com (2603:10b6:510:1ad::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6699.34; Sat, 26 Aug 2023 17:49:08 +0000 Received: from SA3PR12MB8811.namprd12.prod.outlook.com ([fe80::fa0d:1afc:3bb0:b18a]) by SA3PR12MB8811.namprd12.prod.outlook.com ([fe80::fa0d:1afc:3bb0:b18a%4]) with mapi id 15.20.6699.034; Sat, 26 Aug 2023 17:49:08 +0000 From: Asaf Penso To: =?iso-8859-1?Q?Morten_Br=F8rup?= , "olivier.matz@6wind.com" , "NBU-Contact-Thomas Monjalon (EXTERNAL)" , Shahaf Shuler , Slava Ovsiienko CC: "dev@dpdk.org" Subject: Re: mbuf fast free optimization idea Thread-Topic: mbuf fast free optimization idea Thread-Index: AdnYBBFAEmqxv6RTQj+BJjaJnYfKUwAQYf/r Date: Sat, 26 Aug 2023 17:49:08 +0000 Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35D87B46@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87B46@smartserver.smartshare.dk> Accept-Language: en-US Content-Language: en-US X-Mentions: viacheslavo@nvidia.com X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SA3PR12MB8811:EE_|PH7PR12MB6763:EE_ x-ms-office365-filtering-correlation-id: c2d05bc0-8dcb-4911-77c2-08dba65cbf93 x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: x9wr+1d8DLpLKEJiL7K9inPYVcsQNiZkQLQ+7pDOF5rcs19hpDLsKYfnZr9VThHu0Hn1WOrCNNm7yE71mKZn3ooxPH9KltDLXNUvFRLnwpyME7Qf6fgrKowAsyMi5UgWEJlGb1fNsTWV68WzksQKT7SI2Bq85AL/gr34IPmnQmrxlSqcw+PTBnNhsMusGzLk5apKr5JCp5YqI1whKmiIykVIXqhHNb6qqkeQexWLBYIYcSSgUiiplan0egpEFMeSlD522Iu6HADFlOzgMxaDF/UNAGONWocGd9ACaSRoDZ2JErhWuDgt+qPyy9ZhShtOiQprZiVcp/+aPo+AZkQt62HabD0q5/dJSnVgiZ3JXIxbtOI8K3+1uMjFaMWk9z2BRkbHP8CPgdQqJ1yKiqW9axtyJjHiAwy88xPA16Mn1/HxVK3dQbrK8GGu9B1eSUv+7PtcNrkDR9ftVnV1h/CJbnLT9ShVPvcn/BGI4wDjJdbrGoM+iQYo5AQWAbMhYqVsngLMdh/M3/GPbrqhVYAOReyUuLMOoTDLuTnD4C1YnXvWmwWHNluYuKmhpeClFvqLZa1D5p0RzUHc/y1obOz+EpJJ1M2UYnC+qlYmiF7+xVHwOw71TmjdhGgJRBJPXqNH x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SA3PR12MB8811.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(366004)(346002)(39860400002)(136003)(396003)(376002)(186009)(451199024)(1800799009)(9686003)(7696005)(71200400001)(55016003)(53546011)(6506007)(52536014)(5660300002)(86362001)(33656002)(2906002)(76116006)(66946007)(8676002)(41300700001)(8936002)(66446008)(66476007)(66556008)(64756008)(38070700005)(110136005)(4326008)(316002)(6636002)(38100700002)(478600001)(83380400001)(122000001)(26005)(66574015); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?C6hgwjmj7CtfwXOpgXQHC/R/4vMIZByKwIF7tmAjAi/Sb0Rb2EIuS6Nyoh?= =?iso-8859-1?Q?eSOTTPw9q0R3ovfVCXl2eiz9Jex2BLncNGcvJNRh49YuP6VuHo/PDtbmnJ?= =?iso-8859-1?Q?XKL0/JL8aL95900/A+UScfBhN/PwtsFGKkjhbPtqITGdiqIrROj7b/QAr/?= =?iso-8859-1?Q?b35j7cs1hkJ6Mfe7LWo/lnM+1/i6Xf77nzQSVw5dalmCqzNd065sfqzzcr?= =?iso-8859-1?Q?Qr3kzgrxz95tqPSyl/YeJZDTgnqUJG2y/NP0n3aWULpJkXCz55TX2qAb0N?= =?iso-8859-1?Q?N3N3jarnSmeE4CglzjRNGToyUdKIkfNXwnnFqsZg8apHCeB+sVJO86NyhT?= =?iso-8859-1?Q?aaNl9Nlml3f72D0yzjaVYqJhHyblg56hwpGP5GNuxvBpS9SfNzJ6J1vSj/?= =?iso-8859-1?Q?osd6mHnUgAwNtW/zfgBOuyAcobsZno44kmLbv6hSFUBtTPt5TCFJmo5LwS?= =?iso-8859-1?Q?PpK6E5iWgUC6XonZtT0JdNgSlMT89ecCsNBf5bhfd20sOU+gA0YlIrdQ2b?= =?iso-8859-1?Q?H1pD5fRaQGNssiA1gxEf4I8PHudeLDRekE7r2B8Omby96ILTYi4elWqoAp?= =?iso-8859-1?Q?wbY8l3K/myyabRCZf2XJjm1PSrFdt5MZF6wOPz3nRkMj0HJXnpOB9wGB7z?= =?iso-8859-1?Q?ce2I2AsB0UKx+d5Vlch7UZzP4433KQ3xVh0u4HwBbJDP8YqTZRnwaBfWuV?= =?iso-8859-1?Q?07gw89TgDyKmeYgfjmWNp6I9U91II4E8avhezmHHnO2JxwUWYmCKq3vLyB?= =?iso-8859-1?Q?6HUd9kHnid3vO0ISOpAXHmyLZv9gPqtAYRfkt7zfHDmgOrAiI5wgdC6kvr?= =?iso-8859-1?Q?i/qLjOguSndWwtgTDHQEEhm3TrAGaOObOj8gVGkHAetbtnen1XFjJumWq7?= =?iso-8859-1?Q?xu7taliTFpHKQCSFMJBoXfgP1fkCNrpGVl+yCDfoKM+2Xv3jWH0bM26ChG?= =?iso-8859-1?Q?znZ/dve7PNmWZkWSLS6BCJtg0hw6uhf64yItO8k6Gx+VsupuyQQgxpK2BD?= =?iso-8859-1?Q?3mxblkUOFL/A7yCVAnYck9lQ7K9jaxpJ6tVC7/lC6IopwRIz9tQfPP/6vb?= =?iso-8859-1?Q?TXDOSdPIb4ReGihln46FzLwBtrcPF4nFOazZMXbr28G0+ZCDlZTMFrbSoc?= =?iso-8859-1?Q?QSleU2xmOnZjn+PlPI119eVBWyQvcTVfctS8BrImlLM1JQvSliJIw6H2D5?= =?iso-8859-1?Q?geIArnaBHkefozomPyfZYuFxUsjJgmbo8hrQw1vvztfvtzi6PTf15H9Y+j?= =?iso-8859-1?Q?huaTEfeg3sZvVjAB5nl+EXtu+omyS2Dup1gx5wpyZ/QAMGHj0NUNVm5Zhz?= =?iso-8859-1?Q?QUIjV+pkFqUPEOlBcUYiXBhxqi4loS2sNyJbYZ9KRZTzhZK3+Ti7ugRvQs?= =?iso-8859-1?Q?cUQfL3zb2JVWJPs6WjM8rdDWpA8RpVCV+Afjz/ZVxlt/8jLfiuOSOFM0kT?= =?iso-8859-1?Q?DeeLiUAwxaQ6u1kx2f4uLx2rOrikVVrtzNneIpKRkFVIxQrc3ABTo5SDlT?= =?iso-8859-1?Q?bhaVYLT/lKFUnQItMOZYZ5jxb3AakdDgK8YVvYmwwYKUAYmGY1JYFQpLkr?= =?iso-8859-1?Q?CN+XuYm9MjMy1LJzbVG9ca0E/O8g7+Q1BHIdB9UpyndihDVLYoVh+AAu2v?= =?iso-8859-1?Q?hx6YN47CoW81vHLS2M5+TKYwjheMFgS5qN?= Content-Type: multipart/alternative; boundary="_000_SA3PR12MB8811C2FAE64197760A295264CDE2ASA3PR12MB8811namp_" MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SA3PR12MB8811.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c2d05bc0-8dcb-4911-77c2-08dba65cbf93 X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Aug 2023 17:49:08.4078 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3rQwIqhjzwL1oyoqLZ5gSZMDgqvNdOyV23uZzFDqN8NS6NH/FkyNB9/FDkiegjxayTz7NA67QYPla8umSC3Ptw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6763 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --_000_SA3PR12MB8811C2FAE64197760A295264CDE2ASA3PR12MB8811namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable + @Slava Ovsiienko Regards, Asaf Penso ________________________________ From: Morten Br=F8rup Sent: Saturday, August 26, 2023 1:04:18 PM To: olivier.matz@6wind.com ; NBU-Contact-Thomas Mon= jalon (EXTERNAL) ; Shahaf Shuler Cc: dev@dpdk.org Subject: mbuf fast free optimization idea When RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE is used, the usage pattern of some o= f the mbuf fields differs: - The "pool" pointer becomes rarely used. (Only one packet per burst.) - The "next" pointer becomes unused. - The "tx_offload" field is still used. (Set by application, read by driver= .) This means that the 2nd cache line in the mbuf now only contains one hot fi= eld, "tx_offload". So, if we swap the "tx_offload" field (in the 2nd cache line) with the "poo= l" field (in the 1st cache line), the 2nd cache line becomes rarely used. This should provide a performance improvement for applications (using RTE_E= TH_TX_OFFLOAD_MBUF_FAST_FREE) when under cache pressure. PS: Back in the days, the mbuf description said that the 1st cache line was= for RX purposes, and the 2nd cache line was for TX purposes. This is not t= rue anymore, so let's not limit out thinking by that (obsolete) design. Med venlig hilsen / Kind regards, -Morten Br=F8rup --_000_SA3PR12MB8811C2FAE64197760A295264CDE2ASA3PR12MB8811namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable + @Slava Ovsiienko

Regards,
Asaf Penso
From: Morten Br=F8rup <m= b@smartsharesystems.com>
Sent: Saturday, August 26, 2023 1:04:18 PM
To: olivier.matz@6wind.com <olivier.matz@6wind.com>; NBU-Conta= ct-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Shahaf Shuler &l= t;shahafs@nvidia.com>
Cc: dev@dpdk.org <dev@dpdk.org>
Subject: mbuf fast free optimization idea
 
When RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE is used, th= e usage pattern of some of the mbuf fields differs:

- The "pool" pointer becomes rarely used. (Only one packet per bu= rst.)
- The "next" pointer becomes unused.
- The "tx_offload" field is still used. (Set by application, read= by driver.)

This means that the 2nd cache line in the mbuf now only contains one hot fi= eld, "tx_offload".

So, if we swap the "tx_offload" field (in the 2nd cache line) wit= h the "pool" field (in the 1st cache line), the 2nd cache line be= comes rarely used.

This should provide a performance improvement for applications (using RTE_E= TH_TX_OFFLOAD_MBUF_FAST_FREE) when under cache pressure.

PS: Back in the days, the mbuf description said that the 1st cache line was= for RX purposes, and the 2nd cache line was for TX purposes. This is not t= rue anymore, so let's not limit out thinking by that (obsolete) design.


Med venlig hilsen / Kind regards,
-Morten Br=F8rup

--_000_SA3PR12MB8811C2FAE64197760A295264CDE2ASA3PR12MB8811namp_--