From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5FDB7436D3; Tue, 12 Dec 2023 16:38:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4F9EB42E51; Tue, 12 Dec 2023 16:38:12 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id 1D4AC40270; Tue, 12 Dec 2023 16:38:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1702395490; x=1733931490; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=6bZOSSojUPQ54cr+zKoQ9fK4B1R6hySVLOcf+Iu8fh4=; b=h4IS5RWLuPgkYcxMRzyIcHmfo5Tn1n9bq5SMtZNeKvgt/TWPafc5wIHi hHsmky3pT1ID6B85tQc5twl53v4zWkXTpsli9Db/yUt2lgxhOnh91J24B UqdzuEFanJNvAxlCLNc5pERU+T6DG0RVhBrRIRHNoHMRwKJPERoJuRO2Y SAUeqO/TfDA6Icw+HxONqzraWWLoaMk7N7owVpSooKYkJtuPYM1F0uMF0 3vy6BFMZunqUIXvYN3n/Kh4PZDdvqIKDICIh2ro4c2AkDp40sfF8dHVzA QvxpC8s5MOHeRgU4OhWKS/StS9L0NM70UPblHRMxWaXL0GipNnv81KFQD Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10922"; a="394570879" X-IronPort-AV: E=Sophos;i="6.04,270,1695711600"; d="scan'208";a="394570879" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Dec 2023 07:37:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10922"; a="946803563" X-IronPort-AV: E=Sophos;i="6.04,270,1695711600"; d="scan'208";a="946803563" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by orsmga005.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 12 Dec 2023 07:37:38 -0800 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Tue, 12 Dec 2023 07:37:38 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Tue, 12 Dec 2023 07:37:38 -0800 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.41) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Tue, 12 Dec 2023 07:37:38 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QVt1X6atQloKtTC5ka5QELj4bNvD/ZpzOjohApBSnIH3twy1zyRt+Bntz+IItaC95/n0vphlJUjPBKJyykUWOiFIFVaB3ibYuVUz9TbfY4SR3Tz/RBLr5QMRpMaZAEg5TX4fELzvSP7bXuHiFbOmoF94ytMfN3iFEYgZNSf4BFHc5ssDu9FTOiOT4XNRfXI51gP+WzklKZ3QCe6MpEcsxK98TcHr+Px0tcTx5C4aFgcQKtEgxN4gtmRmvXX2H2hfEZN5tPWBpoTS+o/PPpYbnJf3fiOTnlqt+cLfL8Qg013oabPOQjf52tSb+qcS12/a6eQALOD9MBYbaZTHextStg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OecQkbosjYn1EIt3Xf6cRiVhodBsqIy00GzRvA9+mFk=; b=Pau/n8EQrb/ykkBysUc6QR7wh75dsEisMjsqlcROQAi1MX0gkB6h5JXbfyxhA55d0+VK0/xEKvUDXNoebnksbgSh9WIL+Yu8lsBCrxBhgYOIiPytmRAMO4R56+8Xko3bjCWTrGv5PLFOHkrveM8ot92V6gLfGRNHJ55+9317XPOPJRmJsRb+WVItoOK3GDBXlBTY+Uljt8IPd0BnkWVFt3ap5+NfnhRQOaqCCExrwmVDG45WZ2YKeb3Qbj942VgiSm58ETRUkC2jz2WJMzx3Cb+WKv9Bji8PWjbNTSItq3MRag5wxhr4jpUuVD2cCXNHVg7OlD1XkpBIBbYIRwGISg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by SA1PR11MB8317.namprd11.prod.outlook.com (2603:10b6:806:38d::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.26; Tue, 12 Dec 2023 15:37:35 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::4782:d54a:209d:cb49]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::4782:d54a:209d:cb49%7]) with mapi id 15.20.7068.033; Tue, 12 Dec 2023 15:37:34 +0000 Date: Tue, 12 Dec 2023 15:37:29 +0000 From: Bruce Richardson To: Morten =?iso-8859-1?Q?Br=F8rup?= CC: Ferruh Yigit , Vipin Varghese , , , , Thiyagrajan P Subject: Re: [PATCH] app/dma-perf: replace pktmbuf with mempool objects Message-ID: References: <20231212103746.1910-1-vipin.varghese@amd.com> <98CBD80474FA8B44BF855DF32C47DC35E9F0BF@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35E9F0C0@smartserver.smartshare.dk> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F0C0@smartserver.smartshare.dk> X-ClientProxiedBy: DU7PR01CA0019.eurprd01.prod.exchangelabs.com (2603:10a6:10:50f::22) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|SA1PR11MB8317:EE_ X-MS-Office365-Filtering-Correlation-Id: f66ec618-e19a-4053-e0fb-08dbfb28430c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /GaO7ogABL3z3Rimk4AZvXAsYJlU8aqYYxoXtzLHVhOvsNV7pQpYmMoLCngGOyqtv/KikPL6C7Y2rGBbqFdbvCaKrrjSG9H4p20yqSBk7aRYe0dhur0Ky7H+umLvJA5EHLW8wS9nvmq7Obnq1ApswH6OVGSEuMGW9DauKSfYfmC2Z2/NCL180vFILr9ipije6xqOxG02KwZ2And3t2TSHLubzbaCqVfRgLJUn00TXUBmpvvG3u1xGLUzqvyikOBWAnaHstdNwToC3c2bjM901jsLyCB4xmo1gvw8GUD99BQALWHi9x/qw8+j/ALZAgvxtWQYaxBRju3pwPpRK9FrY4I56yfiTTPggRwb4iVBCJodfNZRZ5UWh00PSW3oO059II//hd+4+llTAHL/L64EJqY8GVGE1bOn+StbJ7Ot0CABfcZG2YdzMfLbCFWj0koSTjFPQybmFp0KbtqdQGa+B2zxtTQkJK5U/rmJdaOxIo6z/IGgdZMwlI2eFMDTmH7Okda4ofZ7g9vPWfMqh0l241TzdOzHLVjXkG60tChzAbwwv84vErCTwJpjFl+GVAcQPNjr6WnkpCdKTrQ17a8q3TX21HSjMekEEZYkPceGTCk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39860400002)(366004)(136003)(376002)(396003)(346002)(230922051799003)(451199024)(1800799012)(64100799003)(186009)(66574015)(26005)(6512007)(6506007)(53546011)(44832011)(8936002)(5660300002)(4326008)(41300700001)(2906002)(6486002)(8676002)(6666004)(316002)(478600001)(6916009)(54906003)(38100700002)(66476007)(66556008)(86362001)(82960400001)(66946007); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?fFbe/a9Yec6AkYXQU4q77lzgLXeHTY0KycxjBxLH4bTAjAWldcDJHiVZbi?= =?iso-8859-1?Q?kDwfNaaRnMOZvFUK1vXkr1SG8EtFpJBgqxS71QhzCX44ZGctjfFNF6rJ8Q?= =?iso-8859-1?Q?Mg9sgihzl/GgKIRiQzuHurN55TSile4+Kb/+NVg5AclNDXy3QtCQm0crz1?= =?iso-8859-1?Q?lhZFYSrfoi7oovab7ta+aR2a6JykWRR5o9sF0KmMn+cEYMwkjbN/hxy4jp?= =?iso-8859-1?Q?QiLwMesMlfHJN7mOjr7LpBo/+3RNHZA9KQa5iOSzZr8RTlXjYM9OV2VlNf?= =?iso-8859-1?Q?i5NTmZ8Asnw79WX3wPkhCcj4ZwvKS9drs+eKGkBUXrMNyA3vlSJndjI7k8?= =?iso-8859-1?Q?3OPdxpgkDMPiJJPYbeeP5o/yynuGet76oXMn691JyXg8tIbfPu00SRj9yR?= =?iso-8859-1?Q?o3UIoAxmTgolpcjQCLvFYcIGem1q+Wd6ky5AuokrooSFh5HiwY5RTbv28A?= =?iso-8859-1?Q?06mmKa4pXpjAvdZmZcCANu7mHd0apiZu4lfohdO8YWI5C5L0WxTAa+BHLT?= =?iso-8859-1?Q?AIimd6JVXUWctnn3iQNdOUTUGYJCfHSCrNGEuKiZyWLM2YEEleisQE9E0m?= =?iso-8859-1?Q?bfrjsCHx7CzbdMsoJ+1quDqnqcWAArJ0UuBEB9W2yx+Z7dEhU0qZ61m985?= =?iso-8859-1?Q?znYqQXGWQ2tmPGw1VqB3ivMX8xH3YOQ9dcjkflE2+hU9TKo4GnZyY9yrSI?= =?iso-8859-1?Q?jWiRIfTp647CqV3vygm931gAuPWTG//8pdKWNbF+nC1JY3lr+2YxB7zP0Z?= =?iso-8859-1?Q?+WjAJ6Yq6SaKEysypQmYgNAtpPnTJW4EvaLaGoV1Bmmb6TXdyNDNQl5a4A?= =?iso-8859-1?Q?0Hl6VNqyeekMq4prmP7kFTQ08Ro0JoD1vJyDg6qDNLyaVQKEyqZcKBOk9i?= =?iso-8859-1?Q?NprWCMvKiNVq1utc0zcKJ0RyO3S3VPXMcpOfNBHN70ElqIktSoN7AxFJwo?= =?iso-8859-1?Q?GR6h5XhzooskLAv5wcsoaPBewWMOvg5k7VfX55gzObj+ucuSpgTvCGsK7x?= =?iso-8859-1?Q?CANhugTQcnM7ygERVW7IYmKtVEUU9x4RF6qaKO4EOlesieGmxgNdzkIp1U?= =?iso-8859-1?Q?XB3X2g5YHqWXnu0l8vQZ3oBMf+lE3F5zkCLDLE8tOEScKhbEX3gCYNAme0?= =?iso-8859-1?Q?weLNBQrehDim13bb+BG7MgwjTgWJqa5wS4jRrC/dbwmoXYW/pF7wTCjd/l?= =?iso-8859-1?Q?mJXskLTpvCJzL8XqGA22Po+R9VV6z4StOUxuqZTFdR62E7SKljTuUlkpEz?= =?iso-8859-1?Q?ip2wGYlNYNeRIEJIXWBrCIunejDuUOhvY6eNjiymRoYXNe76LrXKa8FB7y?= =?iso-8859-1?Q?03o7BGy+wJGprCq4IvHjXRziEKaxVAbc5iwvCOzS4BIoX6BGW4X4bEoiwt?= =?iso-8859-1?Q?/zeYeJY6JbnpSfA1dxOr5VvxckC6QvSPdyBnkp9kvyyzZr7rZX9yYlB52M?= =?iso-8859-1?Q?j4pjsqPR1t8o51p3TBi31ZAR6NHR0t0ZoN/fifaFE7g/h8H8VadFXnpW+T?= =?iso-8859-1?Q?8lts3566yj5lp8pmlCMNCfBIlaJ4ZA/X4tiQju20d+g8oI+i93hZeBbDFj?= =?iso-8859-1?Q?1k0ufoNWPlG964kqK8/iQITOKxGw7h6tidzkKCXVkZqqyXsQKwNCLKCrB9?= =?iso-8859-1?Q?6CCupAutkZ+VEuDLmcwUqFns8N2IIUqg9hrvSQgy0A9GMCeIF/99SLgQ?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: f66ec618-e19a-4053-e0fb-08dbfb28430c X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2023 15:37:34.7456 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XgE0bFajIkNEX6+TSnIlL4HAkPF8Hrlwq4E8Zd/3CzcJC9pbk7Zp/fnZv8M+LrXfefJBgNmvOejxUeLRPtX+Z0z8e26rDoUcOwXR1qanYog= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB8317 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Dec 12, 2023 at 04:16:20PM +0100, Morten Brørup wrote: > +TO: Bruce, please stop me if I'm completely off track here. > > > From: Ferruh Yigit [mailto:ferruh.yigit@amd.com] Sent: Tuesday, 12 > > December 2023 15.38 > > > > On 12/12/2023 11:40 AM, Morten Brørup wrote: > > >> From: Vipin Varghese [mailto:vipin.varghese@amd.com] Sent: Tuesday, > > >> 12 December 2023 11.38 > > >> > > >> Replace pktmbuf pool with mempool, this allows increase in MOPS > > >> especially in lower buffer size. Using Mempool, allows to reduce the > > >> extra CPU cycles. > > > > > > I get the point of this change: It tests the performance of copying > > raw memory objects using respectively rte_memcpy and DMA, without the > > mbuf indirection overhead. > > > > > > However, I still consider the existing test relevant: The performance > > of copying packets using respectively rte_memcpy and DMA. > > > > > > > This is DMA performance test application and packets are not used, > > using pktmbuf just introduces overhead to the main focus of the > > application. > > > > I am not sure if pktmuf selected intentionally for this test > > application, but I assume it is there because of historical reasons. > > I think pktmbuf was selected intentionally, to provide more accurate > results for application developers trying to determine when to use > rte_memcpy and when to use DMA. Much like the "copy breakpoint" in Linux > Ethernet drivers is used to determine which code path to take for each > received packet. > > Most applications will be working with pktmbufs, so these applications > will also experience the pktmbuf overhead. Performance testing with the > same overhead as the application will be better to help the application > developer determine when to use rte_memcpy and when to use DMA when > working with pktmbufs. > > (Furthermore, for the pktmbuf tests, I wonder if copying performance > could also depend on IOVA mode and RTE_IOVA_IN_MBUF.) > > Nonetheless, there may also be use cases where raw mempool objects are > being copied by rte_memcpy or DMA, so adding tests for these use cases > are useful. > > > @Bruce, you were also deeply involved in the DMA library, and probably > have more up-to-date practical experience with it. Am I right that > pktmbuf overhead in these tests provides more "real life use"-like > results? Or am I completely off track with my thinking here, i.e. the > pktmbuf overhead is only noise? > I'm actually not that familiar with the dma-test application, so can't comment on the specific overhead involved here. In the general case, if we are just talking about the overhead of dereferencing the mbufs then I would expect the overhead to be negligible. However, if we are looking to include the cost of allocation and freeing of buffers, I'd try to avoid that as it is a cost that would have to be paid for both SW copies and HW copies, so should not count when calculating offload cost. /Bruce