From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 13C59A0093; Wed, 11 May 2022 00:02:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EEE2D406B4; Wed, 11 May 2022 00:02:30 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140071.outbound.protection.outlook.com [40.107.14.71]) by mails.dpdk.org (Postfix) with ESMTP id 093EB4067E for ; Wed, 11 May 2022 00:02:30 +0200 (CEST) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=jZr9u0999fVi5HZsXjLR0HpNKfpLPCdEf7ZzppcKm39XQQLlsIcbRrKCCZBEaxND4cLr2VhfNHZBLlMqC28YiKdqre95puaE87poL0R11ciHTjhPe7L6rblgG0ZmQC11S5Z2BfXJroNI8Rk6xmvu9n4mB8AOiqC9oP8099Zt8Z05RZ/rSguTDqV39rYXj7SeCBEjEL22zAQfq+4jIP7PTykRIki/Q2r1D8U/jKNJL36LIEv2t5U0qeOyCsnMMr9AUDZm+gw1GVV7NYyZTTiVBWp7lZqI74/MKRwoTodDcpW1qd7G+husbGYLj1kC8M7fIUD57kJ3o/fY2m40zUCvmw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=; b=XkSFWNVJvtfJkc3zf0aRlE6mjZOS3ZKphVuNckTVyZ2mjAXvFiJyuSoznF4L3KYpLhtf7kA/CwBJDQ63WEwgNvE8yQHVxr8m7DLfJaFd+MAhUaaMNisHqwwQsypeVcSeg4z+ncIRmU1r9hASI+u9HQD4aKZ9VlK4Af4KcfBsADHViypSpYUhJuoSADiA3EQTysBv5+vOdRRW7KznwjLG7cLFhI96P2tpM27ACpyWT+GzQZ9BsRERA/g4v54pXmDpE8ZEb2yTEm+Sm+V+cmkipvon7AUoIhvr6cw4gs7Gp22XnoarcVo6IKH9cp+KPQgbRdWYRd2XfIEKTxl2zLmFdg== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=; b=M8Lf0utPL2BAhoat014HcPloA4xUlSCk7/SmJqFkNpNS0S/uX/FgWEOw5RjDNmpf2Y2AptjMZ1vAm1kB8kwnsU4O9TAPB4/L8GhYgr2st0tuZe7s8ie5djlGODuLNYNiMjJkP4PNCd13NyuWjDcb9zSqCJINIV5UldM7nYLHxLQ= Received: from DB6PR1001CA0035.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:4:55::21) by AM6PR08MB4184.eurprd08.prod.outlook.com (2603:10a6:20b:a0::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.21; Tue, 10 May 2022 22:02:28 +0000 Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com (2603:10a6:4:55:cafe::b2) by DB6PR1001CA0035.outlook.office365.com (2603:10a6:4:55::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13 via Frontend Transport; Tue, 10 May 2022 22:02:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.15 via Frontend Transport; Tue, 10 May 2022 22:02:28 +0000 Received: ("Tessian outbound 9511859e950a:v118"); Tue, 10 May 2022 22:02:28 +0000 X-CR-MTA-TID: 64aa7808 Received: from 65567cf88d6d.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id D416272A-6BE5-4924-A3C8-56128908F7F9.1; Tue, 10 May 2022 22:02:22 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 65567cf88d6d.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 10 May 2022 22:02:22 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kl+uOFs09M4kJJOoS96LQSCObku7RtHnq+HRyN2VSz3ekNTK5Bl4KpUWHVMyxMP0/+36c0sKpG8YV639pPoBVEM0OTkM0ODrOjzNqz4dc6sxFitzOG5UfaCvEn5wHbEryXB9RC5EWj0ak+v9esM1wNibyK/aQtwCXkxZAEthiGaFBKipRPYQBwK6NqDQJUc0R08FuRXjsou8nG5Ac9679MRJPrV/SgE3FHg5S71Yw2JtnClDjfyZTV8NOQdbjwupnFZSpDkhqbvoGGT8JZrovXZrDCzC2z8KpUS3BVvuPkiyYNDSdNA+es9UscNsq7cLcn4Zh+w20GoGLyJ+kavH3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=; b=N7u3hvYEmTVp+gzge/I/d97ShHtyrXirTZxV7YnHn898ebcBVCAdxxNVBbyGPv3zYOMNDjHf/0Kz5JnAAj/BRg2fqp8tMri6Ds/5ZY079mt4OHSSv+0XBNs68zYo8Cm3QsPLD8YZCbEB1IrLoWLd/csR5/kxPqy4RhKioZujGJUZ52INTSiysr/W8GndHxzKBMJu91pGNzy6bpMRd8g+of8pYL7z9SLidy4F/LSO0K1nQDklkuWkG+/gtnr3bn26ap91XyZ43hrvYdHGrEoSuC8UKkOw8Q0ynbX2LuKgZLI/C1clM8LedGViUqNeAJf467G8Eq5zC/mjz4JQrtHJZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=; b=M8Lf0utPL2BAhoat014HcPloA4xUlSCk7/SmJqFkNpNS0S/uX/FgWEOw5RjDNmpf2Y2AptjMZ1vAm1kB8kwnsU4O9TAPB4/L8GhYgr2st0tuZe7s8ie5djlGODuLNYNiMjJkP4PNCd13NyuWjDcb9zSqCJINIV5UldM7nYLHxLQ= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by AM0PR08MB5490.eurprd08.prod.outlook.com (2603:10a6:208:184::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13; Tue, 10 May 2022 22:01:55 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::44cf:1f8d:e477:1f99]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::44cf:1f8d:e477:1f99%8]) with mapi id 15.20.5227.023; Tue, 10 May 2022 22:01:55 +0000 From: Honnappa Nagarahalli To: =?iso-8859-1?Q?Morten_Br=F8rup?= , Feifei Wang CC: "dev@dpdk.org" , nd , Ruifeng Wang , nd Subject: RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode Thread-Topic: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode Thread-Index: AQHYVI8Os3vmCXi1HEGk+2Bx5G4N06z4lK6AgAEIHHCAAEehgIAezJ5Q Date: Tue, 10 May 2022 22:01:55 +0000 Message-ID: References: <20220420081650.2043183-1-feifei.wang2@arm.com> <20220420081650.2043183-6-feifei.wang2@arm.com> <98CBD80474FA8B44BF855DF32C47DC35D86FE4@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D86FE8@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D86FE8@smartserver.smartshare.dk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 39D9400ED234EF4F8C793F9DE8DDF702.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: 743f6ac7-b559-4dee-941c-08da32d0c5e2 x-ms-traffictypediagnostic: AM0PR08MB5490:EE_|DB5EUR03FT026:EE_|AM6PR08MB4184:EE_ X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: CRhkkywmWGangP6PMpimy1FRpCmSkCARG68y3xVoKlrlLcKuxJiQnqYYhUhOdgXmKc6kTDLAmqi34LsWxPD9RyBVto07viJNXaHNQvCP28Je1mckssqrLomJci6OKftRmCVpcbims8dutDoNb4leXt52szBRaNavqP99yz7kEUOD6BTgn6YYTDtI6jRCFDlzpuJas8wTFXGv3Q1vTzXe4duMMnn17a0y2/VmBsukZTTJ7Sr/JS6/fsBZuwBnh29R9Dg/KRES5gm5WyL2Xt5rNz3eGDkadh1MjGlTRK9JKxQeZJN9n3CbeRofMnA58S6cZmATPGhQKZQjUm3mvlQQOwOOa0KmR4m2Hkc2MIopA2vw9Iomwoj0ldZML2RTQFa/hmiM6P30AJly2GOnnEbzwZf35S6sjWdvC/ScEVxLXM3Zg9F03m3I4HwTz2WvZ4CJv42Ds7tfQ+W6otzuz3IkY/S8xZ9BHf8VmNzIcosSaT1C8e98FYP0H7xUpPDWmHgafTld2HtL2YjLKg4mVkRniqo8Px8eT99XRDkHSfMEk4u6kF51PlNiYikP53maDk5/J4gDrxrBNFfJpIwCjmYpaJgamURffHjuoHj6N7VB5ZYFgboVRh34BLx5/i8AFUs0h4ePafqWM7NYlhpGuN3FYwB+QwkfZYfLt6tMjWWwwQnoa5lrWCq8VC6s3dlsgC3g7QyI0tkW8eXNPh8EKuRz7A== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(55016003)(4326008)(5660300002)(66556008)(6506007)(64756008)(76116006)(508600001)(8936002)(26005)(66946007)(9686003)(66476007)(66446008)(7696005)(122000001)(52536014)(38100700002)(38070700005)(8676002)(83380400001)(86362001)(71200400001)(54906003)(186003)(6636002)(316002)(2906002)(110136005)(33656002); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5490 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 1406bfbc-3626-42ba-3cbd-08da32d0b283 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: XMhM8pke4CCcK8ZvvAnPhWJSPwITEHVmTqzAn9OUG+FL8NwQACaRqrEb02C4l5HhDd/WbOEih65W31hJ+bRoW7QrmESowROOAfyqEZ1TEK5uq+5b+1v82m1Z4V6Eh+/wmxsz3j0RWtsKUoXRovGryoQBSPadre6BEGMj2osuuEFVOL0eXpbkZ4BA1q5XGnJGfeS1j+Da9LAW4+jI9MIh4f9r6Zkia7umEADvbYIpPQ6NieHq+G44ufMvrA7VFgslWpud5dt5BvIwlDmTz4ELEoLCiz1Da8OHz+cp1oRmffzwzDRDi+xg91NZu27WnGhlxPDiHXa8ofzGoVDYJk5zGnq9YN2PfW/avF6QAt/E+m1B6/pPQuUrGGo73pzxhZKMjPBSPPmWcjEAPl0xcz1Be0sjYL7IoU/+zNS3aGJNQGmpwuhDBjEb8JmwuB97CNMeEiR/DYzTmtwNIgnrXNurnl4Evo0eY52Ti1N9hmEY2KhS1sz/Odz1vwC9US7OzeBjZHsZet4cWqBcIXFx7SI+eJm/UxDoTnZhfQGeJgUOOeenO/sBbbuez9cU7VhNgVPVIsD2DVw0nZb6tUlEHPyPJFTFOeOzRYSAdnR0CT/LNuU1sx3x3mfhK3iPGYy5zx1UuEdSvbeUJQni+wR27lN81rzcns2+izvQGzPad7eTzCzmLcCQU3kZPxZRXvvVflS6 X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(336012)(47076005)(186003)(4326008)(33656002)(8676002)(70586007)(81166007)(70206006)(83380400001)(508600001)(26005)(9686003)(55016003)(86362001)(110136005)(82310400005)(54906003)(316002)(6506007)(7696005)(6636002)(356005)(40460700003)(36860700001)(8936002)(52536014)(2906002)(5660300002); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2022 22:02:28.0737 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 743f6ac7-b559-4dee-941c-08da32d0c5e2 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4184 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org (apologies for the late response, this one slipped my mind) Appreciate if others could weigh their opinions. >=20 > > From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com] > > Sent: Thursday, 21 April 2022 04.35 > > > > > > > From: Feifei Wang [mailto:feifei.wang2@arm.com] > > > > Sent: Wednesday, 20 April 2022 10.17 > > > > > > > > Enable direct rearm mode. The mapping is decided in the data plane > > > > based on the first packet received. > > > > > > I usually don't care much about l3fwd, but putting configuration > > changes in the > > > fast path is just wrong! > > I would say it depends. In this case the cycles consumed by the API > > are very less and configuration data is very small and is already in > > the cache as PMD has accessed the same data structure. > > > > If the configuration needs more cycles than a typical (depending on > > the > > application) data plane packet processing needs or brings in enormous > > amount of data in to the cache, it should not be done on the data > > plane. > > >=20 > As a matter of principle, configuration changes should be done outside th= e fast > path. >=20 > If we allow an exception for this feature, it will set a bad precedent ab= out > where to put configuration code. I think there are other examples though not exactly the same. For ex: the s= eqlock, we cannot have a scheduled out writer while holding the lock. But, = it was mentioned that this can be over come easily by running the writer on= an isolated core (which to me breaks some principles). >=20 > > > > > > Also, l3fwd is often used for benchmarking, and this small piece of > > code in the > > > fast path will affect benchmark results (although only very little). > > We do not see any impact on the performance numbers. The reason for > > putting in the data plane was it covers wider use case in this L3fwd > > application. If the app were to be simple, the configuration could be > > done from the control plane. Unfortunately, the performance of L3fwd > > application matters. > > >=20 > Let's proceed down that path for the sake of discussion... Then the fast = path is > missing runtime verification that all preconditions for using remapping a= re > present at any time. Agree, few checks (ensuring that TX and RX buffers are from the same pool, = ensuring tx_rs_thresh is same as RX rearm threshold) are missing. We will add these, it is possible to add these checks outside the packet pr= ocessing loop. >=20 > > > > > > Please move it out of the fast path. >=20 > BTW, this patch does not call the rte_eth_direct_rxrearm_enable() to enab= le > the feature. >=20 > And finally, this feature should be disabled by default, and only enabled= by a > command line parameter or similar. Otherwise, future l3fwd NIC performanc= e > reports will provide misleading performance results, if the feature is ut= ilized. > Application developers, when comparing NIC performance results, don't car= e > about the performance for this unique use case; they care about the > performance for the generic use case. >=20 I think this feature is similar to fast free feature (RTE_ETH_TX_OFFLOAD_MB= UF_FAST_FREE) as you have mentioned in the other thread. It should be handl= ed similar to how fast free feature is handled.