From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 14F99425F0; Wed, 20 Sep 2023 15:12:31 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 036C742686; Wed, 20 Sep 2023 15:12:31 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id BD14A410F2 for ; Wed, 20 Sep 2023 15:12:29 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PQqKeiXmF35lRyGVJjyt/Pb/ZnVOVj6mfUwOadpF2r/KcXSbItgld8TvD/NpQJ7X3y9larRIrCjOi+o+6tNJdHI5VbTxNzkQXj6iEd7mzGyBsaJPw/xFYMvkJhnJZCIFSk+MW4C3egl5KDmaVwdLiz4FmuzvLDS/UvYtE5KCF8uGzORI/m91dFz8v8kz8ugdUtP+puwwYQpX6ERxv+a3BFlwVHlZtzJLIzP+hi2IaP6bFnQVu6Hc25SPbtj1gEJcTPkd9xXXGK36rZJMujjv4kgrBvzji3zLs5GrqZ9cJYpfX6e4yGJRKJhfLSqyBHFny1M0StjosxNiADt/AFSv7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jd7x4JPjIGTbSWDBa6x64DDuC+gvCEc5kbzOBAkqw9M=; b=DW0Yo0oLt/vlowm1fr1U/bl8V2HxvFZ2xDEFV7hO6843qSonfo+YRHyKjcXslBfzyG5dl67hcnaY8t8H4jOd3xLTfEoukwnBbVtKSkxwJhIqVJkI8oTuFAcrq5zL35TV1/qCJ0xAaWMKFeFeACEsVTylVBEvUI/tX0QpEssA6QFhsOAVvrfyS0pcf1CfAAsAKdvofDhSHHTdLWZ/cmAsHP+d8uNjs9bSkNKHsKZbHim4TX5g4ruOfEyCAnq91zSrbaNNVskuTRAatPtkUKGI4k9jqboJJWFC4BR6I8UIi9bdjtqVly8LUZrXtp13Ej91V6OsdPPpT7uuqifJVkuT1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jd7x4JPjIGTbSWDBa6x64DDuC+gvCEc5kbzOBAkqw9M=; b=hSCJb7SPwFmBk02fWV5ZsCoOI/quEsxkT3PFsXzlcROev0Xn828jFmD/TZopFID5DWFtHHGCK58Hi0AzCa8sIjwJnIK2pGh6qnAwcs1AMzbFlVR9L4EzPAGjGXe4lgDd4sM8OP4ajErGNea0rSuYrDJaJMcS4aDXcgh6J8OJbrE= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by MW4PR12MB8612.namprd12.prod.outlook.com (2603:10b6:303:1ec::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.19; Wed, 20 Sep 2023 13:12:25 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::102f:c69b:d3e5:7fe8]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::102f:c69b:d3e5:7fe8%4]) with mapi id 15.20.6792.026; Wed, 20 Sep 2023 13:12:24 +0000 Message-ID: <4c33779b-6c78-4578-a789-e73343a193c8@amd.com> Date: Wed, 20 Sep 2023 14:12:23 +0100 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v12 0/4] Recycle mbufs from Tx queue into Rx queue Content-Language: en-US To: Feifei Wang Cc: dev@dpdk.org, nd@arm.com References: <20220420081650.2043183-1-feifei.wang2@arm.com> <20230824073629.2577864-1-feifei.wang2@arm.com> From: Ferruh Yigit Autocrypt: addr=ferruh.yigit@amd.com; keydata= xsFNBGJDD3EBEAC/M7Tk/DfQSmP1K96vyzdhfSBzlCaGtcxNXorq4fALruqVsD3oi0yfyEz9 4YN8x7py0o9EL8ZdpOX0skc0AMCDAaw033uWhCn0GLMeGRKUbfOAPvL6ecSDvGD7CJIO9j0J eZUvasBgPdM/435PEr9DmC6Ggzdzt8IuG4PoLi5jpFSfcqxZFCCxLUDEo/w0nuguk2FTuYJg B2zEZ4JTBZrw7hIHiFh8D8hr6YA6a5uTofq1tr+l048lbtdFUl8TR0aIExVzE4Z8qKZlcE+9 RQaewjK5Al1jLE4sHdmd3GN+IvgDF3D/fLsi25SKJDeGSdeHkOmaX0qGeM4WKIfU6iARRCiQ N3AmBIxZ/A7UXBKLaOyZ+/i3sE6Wb53nrO4i8+0K2Qwyh6LjTeiJAIjYKN43ppxz3DaI+QwQ vI+uyHr4Gg0Da9EPPz/YyKauSeOZCfCB5gIfICO0j6x0SCl8uQ2nLpjxcZkf0gjcwUzP3h+S 3x6NfDji9YEij0zczW/dcSpGgZ6vsFpPrtnP9ZXy6J53yp0kJtOJoOlkEFFdU2yCZnCDseum CoudmGLZVvS0/DzHDJejq+3kK3FDGktZBOxZIIpal+nFqS7lVgOZc4+huVv3jyhzoAUOEyXA XK5j6o7g8STUY+z33QNnHpdLvecMwuzmvqy0jR54yAbZ64mB9QARAQABzSNGZXJydWggWWln aXQgPGZlcnJ1aC55aWdpdEBhbWQuY29tPsLBlwQTAQgAQQIbAwULCQgHAgYVCgkICwIEFgID AQIeAQIXgAIZARYhBEm7aYjps5XGsPHCElRTPtCKKm/6BQJjb9DJBQkC+3/YAAoJEFRTPtCK Km/6d94P/irGq3mPa7LamXMIioQ8i6ppMSjpr8g+SxH9RnzbmoUjUY6hVzCpXYxEejiJHubg 7lwD+bOocYpiU8Pe0UncVBhIPNk/dIWQAyH0IWK1nd+hOnjxHv3AQpP80Be2o1mUn3oq/b+B QYiyvvre2gIugYq0hzLcG7z4zREeT+Nl0DMDfnLx+Tj2FAOHrOfudUqjdr/VjF5PoTK0bxnb Brqulp0I0ft7hNsufhzf+TlqaB5l0eQO4gDo5xGTP97TT1cGYsXVMsyDhHqW6P3cuj8kkLkO Ch4oq8OLL8GElgRy5y5svU6CJu3f8hT9aiqIoaRpfNv6N8iEk6g4/HNN+uydb9YCA+1pbwBx skmDhtxQrmXpI+Fmfq0aX2NmqHcy6JR8Ekm/nfwLAW/aDpwoMImVN5pPXtOBSjYi4pj7hiJk UA0ZFi8HCW+fic85p8MXn1GPDmbO82Mm3JVJ1uJNAzGsBbW5SP1ol6+XTeaSxcmzFlrDs8vL XQJMRTpcG6Pti7GbbsOtldvmxRQpq9PmCgQg9IrPXpMoaKE3WwtiSiCALs0kYj4F+hhgAfjI RsOuaAJTnKYW31pC/QdroMNAEqIpLveql03jj+xD9ntJIVAsXiCitsLIXzKP0L5tcHkoVLC3 NAjVYq6XkAEssPF4mMRLoEwpvxOytHe1BCJoRl0YA0aQzsFNBGJDD3EBEAC8fBFQHej8qgIG CBzoIEd1cZgPIARlIhRudODXoNDbwA+zJMKtOVwol3Hh1qJ2/yZP11nZsqrP4fyUvMxrwhDe WBWFVDbWHLnqXMnKuUU1vQMujbzgq/4Rb9wSMW5vBL6YxhZng+h71JgS/9nVtzyaTtsOTrJi 6nzFSDx6Wbza2jYvL9rlK0yxJcMEiKwZQ/if4KcOesD0rtxomU/iSEv6DATcJbGXP6T93nPl 90XksijRKAmOwvdu3A8IIlxiSSVRP0lxiHOeR35y6PjHY2usfEDZZOVOfDfhlCVAIBZUZALv VmFOVSTYXeKgYa6Ooaf72+cHM3SgJIbYnevJfFv8YQW0MEAJ/IXE7B1Lk+pHNxwU3VBCrKnA fd/PTvviesuYRkrRD6qqZnINeu3b2DouVGGt2fVcGA38BujCd3p8i7azoGc7A6cgF7z9ETnr ANrbg1/dJyDmkDxOxVrVquTBbxJbDy2HaIe9wyJTEK2Sznpy62DaHVY+gfDQzexBXM10geHC IIUhEnOUYVaq65X3ZDjyAQnNDBQ4uMqSHZk8DpJ22X+T+IMzWzWl+VyU4UZXjkLKPvlqPjJk 1RbKScek5L2GhxHQbPaD76Hx4Jiel0vm2G+4wei8Ay1+0YRFkhySxogU/uQVXHTv63KzQMak oIfnN/V2R0ucarsvMBW+gwARAQABwsF8BBgBCAAmAhsMFiEESbtpiOmzlcaw8cISVFM+0Ioq b/oFAmNv0PAFCQL7f/8ACgkQVFM+0Ioqb/oU9hAAisAJJ09j/kiEeA1HGKUg3DyFnYW22zRL z1IHYmcHQanMx4+FAwsb1bonVldQMIYNJ9z69UNIU16zIqLZt3D5QK7Je5F5q4NZgfP2jtjX rc3jyu0PuDerFZQyxNcYhhmlF9JO9NV976WYbBAM3AAr0TDohUk+YicYTKab+ZHYOABXbqHX qi+bzDCH1vKBDfpJviprthhK1tIABUK5lsp9aFwFU2jfu2J3XX3pwQhPMjtn2C4v37XUnOqM SJPr2HLU27IW5I3BBpJn7dZE/BkmDmnceHF8E4tRlXCQ/cf81+eErL832sfhVJo1MpubKh3B j89ZLmw0cDXZP2hqlC9vdXnoWWHI8PDPkiw9z11yLrT9Wd4cTJTjFV49Z9G96rgnOtWDcZ1J xHBAnXYQ7V2k9abY+ZqY1PtAPr/smqzQfPHXyTAhRVZp2f3yQmj7UqB07FJuLQjJ4CGk97Qx ///qeSg7D8x/sGEA8Yp2jgIj6u7tspuz+RgBYrWpfI0VIHhWED0jNXzqztrVX18CzA6r0ReY SG+CSyycKkco6UI9ZeUrXFwnoUJqIJY9wTJo5gD9EgGIbItv3qWTq29dihLYpzd6dqB708k7 4RBz9051oLaFVGkguFw/tXmA17nx+c2uR5jx8wb4j68umG2X++0dSta5eHAVhmtXvrqPW6Ku 3FQ= In-Reply-To: <20230824073629.2577864-1-feifei.wang2@arm.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: LO4P265CA0298.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:391::8) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|MW4PR12MB8612:EE_ X-MS-Office365-Filtering-Correlation-Id: 3144d8ae-f1d9-497d-9032-08dbb9db3b49 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZcvOutDYsTkgOqGfc/gfsuGMH4KPRAlaQjkGi+BkmOPRoN/GvrFcxGh3WqE46xx+5+I7b9LE10KB6IxlMha7WQ5qqW6LhZjWkrG/ob88/lXZDJKbT6PrGRe5wkf8B5nQsjgfwOoWZUgsDS6nn/1ykAXQZAEyxALhAupKLZQy95Y+5JSJ2ltsAiXPL4oZ8ONSSQNpOsqA0znKzTTZfKIy590V57D68FFLsMt1u2r0A7tHfVIWq6oxjHcbS5CAX5DBN5J7/XF00+28eup/F6HXVHYFOmb7EablLHl8wbmDJd+/7VgQQJd2/uzBp8ucqR87DpiSp+0gLkgZTo88n+nWMY8rEmOWKHQt2sKLUcT+7tMtO9KTwsMxrk3u6hryaaSu8fJKCkhztjqyWKI1dck1by0MDLRYBBDcLmHt6fT1qQkm33pqyl70D9aB3Dr0YxSPnfCLkAms4r9Ozb6VPfqV7pqjSv2mFaHNCYaQ2DHG6zSID/xF9NKWogiKthjnPmlDN8w2J2s0uHEDuDBdfJEfDrLFDjOUd+Rv72wzMI0b+0VmzEkTQuoTHqNvx+KdIvLp47S6Y4Sv55/H6hkoK7gKX6u7rFZIiA/VbFDymdJSSYsrvLEZGWxtUTsh1toTD9d8exdaaXz0Bj9Cb35bXh8Zyu9eW/YHcawkuzM8FHfnMbjtzFn34xQUTOfQ6CXTL5C3 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39860400002)(366004)(396003)(136003)(376002)(346002)(1800799009)(186009)(451199024)(53546011)(6506007)(6486002)(6512007)(478600001)(83380400001)(8936002)(8676002)(44832011)(66946007)(31686004)(66556008)(66476007)(966005)(6916009)(316002)(5660300002)(2616005)(36756003)(31696002)(86362001)(4326008)(2906002)(26005)(38100700002)(41300700001)(43740500002)(45980500001)(414714003)(473944003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YXp5MU9JME03YS9vVDBKeXA0MXQ3WEtIZ2dwdUU4U3JHMEdoVWF6UUNaK2RM?= =?utf-8?B?WGtLdWF5WTRwQ0h0c3RRZUw2UnlhRGdWMk51aUhjek9OaHFmTjkvSEpSdzNt?= =?utf-8?B?TkNrSVBndURkWS9rcWRidmlJRkNPditzMmNtbWRpejFaeVVCREx0YUFKdC9D?= =?utf-8?B?NXVWY0RZT0h1dDZwOVVDMHBlNHBGYllKaUJTQ1RXVWQ3c2k1d1pnY29zcUx6?= =?utf-8?B?UVBkS0ZlR0tmVURVcmRvSGFGUFdRUUlPa1A0LzBhdFRpU0VTNkhwRGNMNFd6?= =?utf-8?B?Q0xJNkYzanVKbm5WWWt3S3lPc2hsYnhBQ1pxMzdsb2J5MWthMStTR1A3ajgv?= =?utf-8?B?V1krNi9qWFcwTm1RMlc5SnFhd2d2WWpVQWozNGVoME5pK0d6aEloNmh2djI3?= =?utf-8?B?ZUprZ0k4eWVZVTZjbld1N1dNOW5IU1dYdW9VZjFRZk4yZVhPbXJydFpOeFRM?= =?utf-8?B?UVFWYkN0SUluMHdieEp5bDB5UEM5VytkNll6UHJpT0hrN00yMzZJYTdGKzRI?= =?utf-8?B?Lzd1NHZLbi9WQkdFclF2WEZJRS9CNUR1WWU3RU91WmtiQkU4VDFWVkJUYlRG?= =?utf-8?B?L00zS09DQUNuSVQyS2NvSEs1Q2M1OUFhbGp6c1BzYWdnckcwYjZiejFTNnRz?= =?utf-8?B?YVdibjIzR250NVQ3WCthOGVOK2hhZTFZd296alVxTFpzRi9tRDY3Y0sxc2tu?= =?utf-8?B?bytnZnNtaTJkRGpIK1Y3blI3WWxwTTVNb1hIaGM5NUh0YnhZV09IMmpheGpQ?= =?utf-8?B?d25FNVBQM25YczMrMlI2SzRjTk9IcHQ5dE5EdTV3OWpXdHppbmxGelpwWUlJ?= =?utf-8?B?S2VpZjRvTUNjY3lxeUx1bTZVUllIUHhtWlF0RFJlajhFU25nYnpFWGtIcXlr?= =?utf-8?B?VXlxVUtjQzRnUmVyemhWaE5UYTQ2RHd5VUJ4TGFZc2hVeURoT20vdlVHRUZy?= =?utf-8?B?eVJyRWNzYUR4ejBhVytRcWxuazJVQk5vUDF0U3oxUllCMDhyVy9zU25OQUJl?= =?utf-8?B?bWd5bVdrVHVCY1FhOXlCV1hFVXRDYnJLU1V4R3V4azVCUnluVG1DclNNckVk?= =?utf-8?B?dVJ3Rm5lT3V0RUZkNUU2eXN6aEpnZnpzbERQSDBtdkM4QnpJcHQ0aTFVczRZ?= =?utf-8?B?elUycG1qL0RpYkcvV25vUlcrekhseDhsNG9RNmJEVlpaTFZHNk9zSWVSSEVE?= =?utf-8?B?SUt3ek1BT0gwNDNUajhUWmh0eGNSRHRPSWx4U0RFQU1makg5UzhIMUxnS2Jp?= =?utf-8?B?bkdPRWRiNTNRMTRDNzJYaE1VVHlNbE4xd1l6L0NFL0dCTWE2ZUJzSTBpRW40?= =?utf-8?B?TVdsYWFkWTh0UCtQVUhhcGV2TTlEbm4zVmlacXB0NWNUVEdKei9DZURWVHhW?= =?utf-8?B?WWNlaFphWis2NmJaMVRYRjRMVEtMNFF4M0VOdXFsWnAzK2cwaGZMNkgwRVMv?= =?utf-8?B?V2grWGpGck1qZmhaQTZHOGdUWVpZSjRhSUFVbVlnYzVuL014NkdlN3RxeW5C?= =?utf-8?B?dGpvZy9oZUIwTVFjM0JUMEJCYllFUjFNYUJMMTVLYWtzTWZFMkxLUUlJSFBB?= =?utf-8?B?N3dybE1sa1NTWWZoUys3N2NPempHV0xpazNrMFNqNERHalpkOUxWVk9rOGZK?= =?utf-8?B?aHRMaTlGZVlFWDZxeXVEOU9uRHFlTkxiWXZYRFJkaTg4WTc0U3RLQ1hMT0di?= =?utf-8?B?RHc0d1Y4UmFMTUFlVEZyVlZHM2xoSFZDUmRKRDRjOG1aMmY5STY1OGxBK2cz?= =?utf-8?B?elVVL3VBV2Q4QzBucUowNmp4K1RiTlVVbnY3UDhLL3ZObHY1WXprUWZmYVlP?= =?utf-8?B?SEg5ZFhOcnk1bFVnL2taTUhrd3FyMkZDNTNPazZjd1crWnFZYkpRV3U5d0xR?= =?utf-8?B?azB5VTR2Q240TWRzQ2hBdGtGY21Ld3UzcWVIZE9pMk8zaDdiVDhhdXA3aVJT?= =?utf-8?B?M0l0VEdQbkVSVDMxZ0MvbUF5aFhEbzFRa1h3MSsyblREb0FtanFiUDRpNGg3?= =?utf-8?B?TE1VaHc0SzhDRFljZ01uWmFjdWI0ZG5jTGJ3UGYyTUk2cUtTVUFVaHgvbkh4?= =?utf-8?B?dHJCejJpUDNZWUo4ekVRT0JMM21UYU84L09QR3Vib0E0c2Mrc0xQSEtVNGlN?= =?utf-8?Q?YI+Vf6fCf8PvWUmTYolJtYB+g?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3144d8ae-f1d9-497d-9032-08dbb9db3b49 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 13:12:24.8744 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lBjywkMc+MpHvs1JaCkW4yjsAvMQuMI51LQIIGasyfvxphJrocwH1TUH0PKHMdlU X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB8612 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 8/24/2023 8:36 AM, Feifei Wang wrote: > Currently, the transmit side frees the buffers into the lcore cache and > the receive side allocates buffers from the lcore cache. The transmit > side typically frees 32 buffers resulting in 32*8=256B of stores to > lcore cache. The receive side allocates 32 buffers and stores them in > the receive side software ring, resulting in 32*8=256B of stores and > 256B of load from the lcore cache. > > This patch proposes a mechanism to avoid freeing to/allocating from > the lcore cache. i.e. the receive side will free the buffers from > transmit side directly into its software ring. This will avoid the 256B > of loads and stores introduced by the lcore cache. It also frees up the > cache lines used by the lcore cache. And we can call this mode as mbufs > recycle mode. > > In the latest version, mbufs recycle mode is packaged as a separate API. > This allows for the users to change rxq/txq pairing in real time in data plane, > according to the analysis of the packet flow by the application, for example: > ----------------------------------------------------------------------- > Step 1: upper application analyse the flow direction > Step 2: recycle_rxq_info = rte_eth_recycle_rx_queue_info_get(rx_portid, rx_queueid) > Step 3: rte_eth_recycle_mbufs(rx_portid, rx_queueid, tx_portid, tx_queueid, recycle_rxq_info); > Step 4: rte_eth_rx_burst(rx_portid,rx_queueid); > Step 5: rte_eth_tx_burst(tx_portid,tx_queueid); > ----------------------------------------------------------------------- > Above can support user to change rxq/txq pairing at run-time and user does not need to > know the direction of flow in advance. This can effectively expand mbufs recycle mode's > use scenarios. > > Furthermore, mbufs recycle mode is no longer limited to the same pmd, > it can support moving mbufs between different vendor pmds, even can put the mbufs > anywhere into your Rx mbuf ring as long as the address of the mbuf ring can be provided. > In the latest version, we enable mbufs recycle mode in i40e pmd and ixgbe pmd, and also try to > use i40e driver in Rx, ixgbe driver in Tx, and then achieve 7-9% performance improvement > by mbufs recycle mode. > > Difference between mbuf recycle, ZC API used in mempool and general path > For general path: > Rx: 32 pkts memcpy from mempool cache to rx_sw_ring > Tx: 32 pkts memcpy from tx_sw_ring to temporary variable + 32 pkts memcpy from temporary variable to mempool cache > For ZC API used in mempool: > Rx: 32 pkts memcpy from mempool cache to rx_sw_ring > Tx: 32 pkts memcpy from tx_sw_ring to zero-copy mempool cache > Refer link: http://patches.dpdk.org/project/dpdk/patch/20230221055205.22984-2-kamalakshitha.aligeri@arm.com/ > For mbufs recycle: > Rx/Tx: 32 pkts memcpy from tx_sw_ring to rx_sw_ring > Thus we can see in the one loop, compared to general path, mbufs recycle mode reduces 32+32=64 pkts memcpy; > Compared to ZC API used in mempool, we can see mbufs recycle mode reduce 32 pkts memcpy in each loop. > So, mbufs recycle has its own benefits. > > Testing status: > (1) dpdk l3fwd test with multiple drivers: > port 0: 82599 NIC port 1: XL710 NIC > ------------------------------------------------------------- > Without fast free With fast free > Thunderx2: +7.53% +13.54% > ------------------------------------------------------------- > > (2) dpdk l3fwd test with same driver: > port 0 && 1: XL710 NIC > ------------------------------------------------------------- > Without fast free With fast free > Ampere altra: +12.61% +11.42% > n1sdp: +8.30% +3.85% > x86-sse: +8.43% +3.72% > ------------------------------------------------------------- > > (3) Performance comparison with ZC_mempool used > port 0 && 1: XL710 NIC > with fast free > ------------------------------------------------------------- > With recycle buffer With zc_mempool > Ampere altra: 11.42% 3.54% > ------------------------------------------------------------- > > Furthermore, we add recycle_mbuf engine in testpmd. Due to XL710 NIC has > I/O bottleneck in testpmd in ampere altra, we can not see throughput change > compared with I/O fwd engine. However, using record cmd in testpmd: > '$set record-burst-stats on' > we can see the ratio of 'Rx/Tx burst size of 32' is reduced. This > indicate mbufs recycle can save CPU cycles. > > V2: > 1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa) > 2. Add 'txq_data_get' API to get txq info for Rx (Konstantin) > 3. Use input parameter to enable direct rearm in l3fwd (Konstantin) > 4. Add condition detection for direct rearm API (Morten, Andrew Rybchenko) > > V3: > 1. Seperate Rx and Tx operation with two APIs in direct-rearm (Konstantin) > 2. Delete L3fwd change for direct rearm (Jerin) > 3. enable direct rearm in ixgbe driver in Arm > > v4: > 1. Rename direct-rearm as buffer recycle. Based on this, function name > and variable name are changed to let this mode more general for all > drivers. (Konstantin, Morten) > 2. Add ring wrapping check (Konstantin) > > v5: > 1. some change for ethdev API (Morten) > 2. add support for avx2, sse, altivec path > > v6: > 1. fix ixgbe build issue in ppc > 2. remove 'recycle_tx_mbufs_reuse' and 'recycle_rx_descriptors_refill' > API wrapper (Tech Board meeting) > 3. add recycle_mbufs engine in testpmd (Tech Board meeting) > 4. add namespace in the functions related to mbufs recycle(Ferruh) > > v7: > 1. move 'rxq/txq data' pointers to the beginning of eth_dev structure, > in order to keep them in the same cache line as rx/tx_burst function > pointers (Morten) > 2. add the extra description for 'rte_eth_recycle_mbufs' to show it can > support feeding 1 Rx queue from 2 Tx queues in the same thread > (Konstantin) > 3. For i40e/ixgbe driver, make the previous copied buffers as invalid if > there are Tx buffers refcnt > 1 or from unexpected mempool (Konstantin) > 4. add check for the return value of 'rte_eth_recycle_rx_queue_info_get' > in testpmd fwd engine (Morten) > > v8: > 1. add arm/x86 build option to fix ixgbe build issue in ppc > > v9: > 1. delete duplicate file name for ixgbe > > v10: > 1. fix compile issue on windows > > v11: > 1. fix doc warning > > v12: > 1. replace rx queue check code with eth_dev_validate_rx_queue > function (Stephen) > 2. put port and queue check before function call (Konstantin) > > Feifei Wang (4): > ethdev: add API for mbufs recycle mode > net/i40e: implement mbufs recycle mode > net/ixgbe: implement mbufs recycle mode > app/testpmd: add recycle mbufs engine > Thanks for dedication to improve the patchset and finding better solution, it is appreciated. Series applied to dpdk-next-net/main, thanks.