From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BAA41A00C4; Thu, 29 Sep 2022 12:29:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A53D540694; Thu, 29 Sep 2022 12:29:09 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2068.outbound.protection.outlook.com [40.107.237.68]) by mails.dpdk.org (Postfix) with ESMTP id 29D2640395 for ; Thu, 29 Sep 2022 12:29:08 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KtHeV8np6BPuwrwUYSX2q+P5oG7Ab0M+gqCYNze/grbSvz3Ye4ny5/CQ5CLTwRgCXjPsUyljlBeU/y+eYTxhGXCAwNIC+VJG6zbvp26HUGGxOnjKBLwzcLvU/NSudzVSPNhMcK3U2u6lBz14XoTjaUv/NCfoq5s5D4CS9/ZimPZ3/R3dQx+Uy/L9cBK1nIrwNp6si+5I7rgHXCWCY6KMf1dxOnyCfXKOcwSwEy6Uctr/uU+J8M5dkfoJJf094D2rV8PXMEMC8LjvrI3UKyhnz3y3Z5bnjE4yWgL3n92G+mJfiZUqZF2hsBm/C5Wn319NrjELsV0nWAk207F+W56fig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JmTBjXCcQTpHlnewRsWW3ZiYYjMHkz7GBSNvOgytB7k=; b=AakxUR65Ep7ooPd0952XY5pK44c6H5dPoK34uD2l9fDHW6Vu3eHPC8p6Ng2vMYQMLcp3Qq8EC5/uyXXhx6x8oxNH/VO0Pq/h0qtBV3EpCO3fjfaL2FL1qiOkaEv3SqzymoZmc5f/YlZh2X3SD53LBe16n9L+CbvSt6jISQwy2jpbeprZK6ZQH46B/oGIum6onPrSmvvef+pvwXG1zHSR0RdoAjfmLDEMOitGfzhWowBedgIqjlzygQrFKIyH1jFKVaT/5VSyuzBv0lxYIy8hUgpRv0F4Z06tO6ly+B44cKVQs8bhKf8QB9uwdEc7EyqwsisU1Vs1Xb6m7+n6DyeMDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=softfail (sender ip is 149.199.80.198) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=fail (p=quarantine sp=quarantine pct=100) action=quarantine header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JmTBjXCcQTpHlnewRsWW3ZiYYjMHkz7GBSNvOgytB7k=; b=plN1Tb1Tbn6zYWhCY4DcjrzYefXkOdYh5MBwkPXwQSypf4XQbZI95O10hhnrO7bA9FOyII33JY0ZvnmGUm0u4B4jU1O5DMdV1sQxMbfnfkFMkScPFbMvV2b5D+vL2cqW3kR9Pzy8KeGndAfkD5Bbuqx11JxFGtNlnlETSDt2UyQ= Received: from DM6PR13CA0035.namprd13.prod.outlook.com (2603:10b6:5:bc::48) by BY5PR02MB6692.namprd02.prod.outlook.com (2603:10b6:a03:212::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.25; Thu, 29 Sep 2022 10:29:06 +0000 Received: from DM3NAM02FT029.eop-nam02.prod.protection.outlook.com (2603:10b6:5:bc:cafe::80) by DM6PR13CA0035.outlook.office365.com (2603:10b6:5:bc::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5676.15 via Frontend Transport; Thu, 29 Sep 2022 10:29:05 +0000 X-MS-Exchange-Authentication-Results: spf=softfail (sender IP is 149.199.80.198) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=fail action=quarantine header.from=amd.com; Received-SPF: SoftFail (protection.outlook.com: domain of transitioning amd.com discourages use of 149.199.80.198 as permitted sender) Received: from xir-pvapexch02.xlnx.xilinx.com (149.199.80.198) by DM3NAM02FT029.mail.protection.outlook.com (10.13.4.98) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5676.17 via Frontend Transport; Thu, 29 Sep 2022 10:29:04 +0000 Received: from xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 29 Sep 2022 11:29:03 +0100 Received: from smtp.xilinx.com (172.21.105.198) by xir-pvapexch02.xlnx.xilinx.com (172.21.17.17) with Microsoft SMTP Server id 15.1.2375.24 via Frontend Transport; Thu, 29 Sep 2022 11:29:03 +0100 Envelope-to: Feifei.Wang2@arm.com, konstantin.v.ananyev@yandex.ru, andrew.rybchenko@oktetlabs.ru, mb@smartsharesystems.com, dev@dpdk.org, nd@arm.com Received: from [172.21.36.148] (port=65117) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1odqmt-0005dF-Ih; Thu, 29 Sep 2022 11:29:03 +0100 Message-ID: Date: Thu, 29 Sep 2022 11:29:03 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 Subject: =?UTF-8?B?UmU6IOWbnuWkjTogW1BBVENIIHYyIDAvM10gRGlyZWN0IHJlLWFybWlu?= =?UTF-8?Q?g_of_buffers_on_receive_side?= Content-Language: en-US To: Feifei Wang , Konstantin Ananyev , Andrew Rybchenko , "mb@smartsharesystems.com" CC: "dev@dpdk.org" , nd References: <20220927024756.947272-1-feifei.wang2@arm.com> From: Ferruh Yigit In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM3NAM02FT029:EE_|BY5PR02MB6692:EE_ X-MS-Office365-Filtering-Correlation-Id: e95ec59a-79eb-43c7-b47e-08daa2056f24 X-MS-Exchange-SenderADCheck: 2 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Qrd2SvwHicAfv8F9CIAVubRTlv8hx4a9+jA+ZQq0+A9F6+0pA6iGG1angk9c8BooAaaGz9QuJVX9w5wECG1S/IFRFd0D0r8ol5hNQOcbNpJZa4iJn6XiNEM5ZowQuupolKMAT1uXljNkiXd+g1Kg8v4ZhDOtG7COtqQVAhAGZC7WSIKHcPuahbqlLHkDSyoLCEm61M+7l6XKifHPU2jUsJuv/Ws5aM3kIxUL+5U3v5SpK5cObFCVM/uUygLh54WGpOQTz/UfdY3r/eiFXnqNJ7fHzaBZMZl6+aKbEh4xjk37EkFbg1cFFYSOoPoynLGbJSfMf6nslQYZlg68elsEgJMzR5NncgLlqWm1P7xmVa3G2sRLT9N53Y5oyPJuxEnrCR5ioMDX5qu1ENcTwSwNhURC4OphlN+xMFpRu90bloJB62qQt5ujndYzUap1MxXH1IKfpuWLub1teStbNp4Z8NluyFkMFyxz1aBGmxKk2O+O+UseRflx8tj4R7RfaPtyUYLCl4Y/shCIBaeJIAY9QUjsVL3jzTX9Nwcrc/b408TbytTetajOJHjprxAy3cZhbh4EGRJCce0kulW7uVSbZitOzahQO7XGVuAW1YH4z5ZErXwgpG6DBHQud49delaiJr33jda+hy5tQ1MsKkd8+E7sRNMHFQjJYjK47QgOW52opGWrH6W++5tA2AD8ZMEgRZVVtFLN28x5tDPyMfQQqDF4EJWOYnGX9UeNe7ipWrYdQcKs1Sp3+yNy3+dlIPGhJ+iYu0eYPCxI1iUCKCQ+PnWsBEXvaTcsEdSZd55EWBFhvoONFKiBByu9AzqJ3kWnE8irX+kjZ0VPZCAHIroCO7oxcRxki1yJ/ymUAWnRniU= X-Forefront-Antispam-Report: CIP:149.199.80.198; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:xir-pvapexch02.xlnx.xilinx.com; PTR:unknown-80-198.xilinx.com; CAT:NONE; SFS:(13230022)(4636009)(136003)(376002)(39860400002)(346002)(396003)(451199015)(40470700004)(46966006)(26005)(336012)(47076005)(82310400005)(498600001)(356005)(2616005)(70586007)(316002)(7636003)(70206006)(40460700003)(4326008)(2906002)(35950700001)(82740400003)(36756003)(41300700001)(44832011)(224303003)(110136005)(8936002)(53546011)(9786002)(966005)(31686004)(83380400001)(5660300002)(31696002)(54906003)(40480700001)(86362001)(50156003)(43740500002); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.onmicrosoft.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2022 10:29:04.7215 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e95ec59a-79eb-43c7-b47e-08daa2056f24 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.80.198]; Helo=[xir-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: DM3NAM02FT029.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR02MB6692 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 9/29/2022 7:19 AM, Feifei Wang wrote: >> -----邮件原件----- >> 发件人: Feifei Wang >> 发送时间: Tuesday, September 27, 2022 10:48 AM >> 抄送: dev@dpdk.org; nd ; Feifei Wang >> >> 主题: [PATCH v2 0/3] Direct re-arming of buffers on receive side >> >> Currently, the transmit side frees the buffers into the lcore cache and the >> receive side allocates buffers from the lcore cache. The transmit side typically >> frees 32 buffers resulting in 32*8=256B of stores to lcore cache. The receive >> side allocates 32 buffers and stores them in the receive side software ring, >> resulting in 32*8=256B of stores and 256B of load from the lcore cache. >> >> This patch proposes a mechanism to avoid freeing to/allocating from the >> lcore cache. i.e. the receive side will free the buffers from transmit side >> directly into it's software ring. This will avoid the 256B of loads and stores >> introduced by the lcore cache. It also frees up the cache lines used by the >> lcore cache. >> >> However, this solution poses several constraints: >> >> 1)The receive queue needs to know which transmit queue it should take the >> buffers from. The application logic decides which transmit port to use to send >> out the packets. In many use cases the NIC might have a single port ([1], [2], >> [3]), in which case a given transmit queue is always mapped to a single >> receive queue (1:1 Rx queue: Tx queue). This is easy to configure. >> >> If the NIC has 2 ports (there are several references), then we will have >> 1:2 (RX queue: TX queue) mapping which is still easy to configure. >> However, if this is generalized to 'N' ports, the configuration can be long. >> More over the PMD would have to scan a list of transmit queues to pull the >> buffers from. >> >> 2)The other factor that needs to be considered is 'run-to-completion' vs >> 'pipeline' models. In the run-to-completion model, the receive side and the >> transmit side are running on the same lcore serially. In the pipeline model. >> The receive side and transmit side might be running on different lcores in >> parallel. This requires locking. This is not supported at this point. >> >> 3)Tx and Rx buffers must be from the same mempool. And we also must >> ensure Tx buffer free number is equal to Rx buffer free number: >> (txq->tx_rs_thresh == RTE_I40E_RXQ_REARM_THRESH) Thus, 'tx_next_dd' >> can be updated correctly in direct-rearm mode. This is due to tx_next_dd is a >> variable to compute tx sw-ring free location. >> Its value will be one more round than the position where next time free >> starts. >> >> Current status in this patch: >> 1)Two APIs are added for users to enable direct-rearm mode: >> In control plane, users can call 'rte_eth_txq_data_get' to get Tx sw_ring >> pointer and its txq_info (This avoid Rx load Tx data directly); >> >> In data plane, users can call 'rte_eth_rx_direct_rearm' to rearm Rx >> buffers and free Tx buffers at the same time (Currently it supports 1:1 >> (rxq:txq) mapping:) >> ----------------------------------------------------------------------- >> control plane: >> rte_eth_txq_data_get(*txq_data); >> data plane: >> loop { >> rte_eth_rx_direct_rearm(*txq_data){ >> for (i = 0; i <= 32; i++) { >> rx.mbuf[i] = tx.mbuf[i]; >> initialize descs[i]; >> } >> } >> rte_eth_rx_burst; >> rte_eth_tx_burst; >> } >> ----------------------------------------------------------------------- >> 2)The i40e driver is changed to do the direct re-arm of the receive >> side. >> 3)L3fwd application is modified to enable direct rearm mode. Users can >> enable direct-rearm and map queues by input parameters. >> >> Testing status: >> 1.The testing results for L3fwd are as follows: >> ------------------------------------------------------------------- >> enabled direct rearm >> ------------------------------------------------------------------- >> Arm: >> N1SDP(neon path): >> without fast-free mode with fast-free mode >> +15.09% +4.2% >> >> Ampere Altra(neon path): >> without fast-free mode with fast-free mode >> +10.9% +14.6% >> ------------------------------------------------------------------- >> >> 2.The testing results for VPP-L3fwd are as follows: >> ------------------------------------------------------------------- >> Arm: >> N1SDP(neon path): >> with direct re-arm mode enabled >> +4.5% >> >> Ampere Altra(neon path): >> with direct re-arm mode enabled >> +6.5% >> ------------------------------------------------------------------- >> >> Reference: >> [1] https://store.nvidia.com/en- >> us/networking/store/product/MCX623105AN- >> CDAT/NVIDIAMCX623105ANCDATConnectX6DxENAdapterCard100GbECrypt >> oDisabled/ >> [2] https://www.intel.com/content/www/us/en/products/sku/192561/intel- >> ethernet-network-adapter-e810cqda1/specifications.html >> [3] https://www.broadcom.com/products/ethernet-connectivity/network- >> adapters/100gb-nic-ocp/n1100g >> >> V2: >> 1. Use data-plane API to enable direct-rearm (Konstantin, Honnappa) 2. Add >> 'txq_data_get' API to get txq info for Rx (Konstantin) 3. Use input parameter >> to enable direct rearm in l3fwd (Konstantin) 4. Add condition detection for >> direct rearm API (Morten, Andrew Rybchenko) >> > PING > > Hi, > > Would you please give some comments for this version? > Thanks very much. > Hi Feifei, Because of the restrictions this feature has, although it gives a performance improvement on some specific tests I believe it doesn't have much practical usage, so I have constraints about the feature.