From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 13C59A0093;
Wed, 11 May 2022 00:02:31 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id EEE2D406B4;
Wed, 11 May 2022 00:02:30 +0200 (CEST)
Received: from EUR01-VE1-obe.outbound.protection.outlook.com
(mail-eopbgr140071.outbound.protection.outlook.com [40.107.14.71])
by mails.dpdk.org (Postfix) with ESMTP id 093EB4067E
for <dev@dpdk.org>; Wed, 11 May 2022 00:02:30 +0200 (CEST)
ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass;
b=jZr9u0999fVi5HZsXjLR0HpNKfpLPCdEf7ZzppcKm39XQQLlsIcbRrKCCZBEaxND4cLr2VhfNHZBLlMqC28YiKdqre95puaE87poL0R11ciHTjhPe7L6rblgG0ZmQC11S5Z2BfXJroNI8Rk6xmvu9n4mB8AOiqC9oP8099Zt8Z05RZ/rSguTDqV39rYXj7SeCBEjEL22zAQfq+4jIP7PTykRIki/Q2r1D8U/jKNJL36LIEv2t5U0qeOyCsnMMr9AUDZm+gw1GVV7NYyZTTiVBWp7lZqI74/MKRwoTodDcpW1qd7G+husbGYLj1kC8M7fIUD57kJ3o/fY2m40zUCvmw==
ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=;
b=XkSFWNVJvtfJkc3zf0aRlE6mjZOS3ZKphVuNckTVyZ2mjAXvFiJyuSoznF4L3KYpLhtf7kA/CwBJDQ63WEwgNvE8yQHVxr8m7DLfJaFd+MAhUaaMNisHqwwQsypeVcSeg4z+ncIRmU1r9hASI+u9HQD4aKZ9VlK4Af4KcfBsADHViypSpYUhJuoSADiA3EQTysBv5+vOdRRW7KznwjLG7cLFhI96P2tpM27ACpyWT+GzQZ9BsRERA/g4v54pXmDpE8ZEb2yTEm+Sm+V+cmkipvon7AUoIhvr6cw4gs7Gp22XnoarcVo6IKH9cp+KPQgbRdWYRd2XfIEKTxl2zLmFdg==
ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is
63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass
(p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass
(signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1
ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com]
dmarc=[1,1,header.from=arm.com])
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=;
b=M8Lf0utPL2BAhoat014HcPloA4xUlSCk7/SmJqFkNpNS0S/uX/FgWEOw5RjDNmpf2Y2AptjMZ1vAm1kB8kwnsU4O9TAPB4/L8GhYgr2st0tuZe7s8ie5djlGODuLNYNiMjJkP4PNCd13NyuWjDcb9zSqCJINIV5UldM7nYLHxLQ=
Received: from DB6PR1001CA0035.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:4:55::21)
by AM6PR08MB4184.eurprd08.prod.outlook.com (2603:10a6:20b:a0::28)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.21; Tue, 10 May
2022 22:02:28 +0000
Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
(2603:10a6:4:55:cafe::b2) by DB6PR1001CA0035.outlook.office365.com
(2603:10a6:4:55::21) with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13 via Frontend
Transport; Tue, 10 May 2022 22:02:28 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123)
smtp.mailfrom=arm.com; dkim=pass (signature was verified)
header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
63.35.35.123 as permitted sender) receiver=protection.outlook.com;
client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by
DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with
Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.5227.15 via Frontend Transport; Tue, 10 May 2022 22:02:28 +0000
Received: ("Tessian outbound 9511859e950a:v118");
Tue, 10 May 2022 22:02:28 +0000
X-CR-MTA-TID: 64aa7808
Received: from 65567cf88d6d.1
by 64aa7808-outbound-1.mta.getcheckrecipient.com id
D416272A-6BE5-4924-A3C8-56128908F7F9.1;
Tue, 10 May 2022 22:02:22 +0000
Received: from EUR05-DB8-obe.outbound.protection.outlook.com
by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 65567cf88d6d.1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384);
Tue, 10 May 2022 22:02:22 +0000
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=Kl+uOFs09M4kJJOoS96LQSCObku7RtHnq+HRyN2VSz3ekNTK5Bl4KpUWHVMyxMP0/+36c0sKpG8YV639pPoBVEM0OTkM0ODrOjzNqz4dc6sxFitzOG5UfaCvEn5wHbEryXB9RC5EWj0ak+v9esM1wNibyK/aQtwCXkxZAEthiGaFBKipRPYQBwK6NqDQJUc0R08FuRXjsou8nG5Ac9679MRJPrV/SgE3FHg5S71Yw2JtnClDjfyZTV8NOQdbjwupnFZSpDkhqbvoGGT8JZrovXZrDCzC2z8KpUS3BVvuPkiyYNDSdNA+es9UscNsq7cLcn4Zh+w20GoGLyJ+kavH3A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=;
b=N7u3hvYEmTVp+gzge/I/d97ShHtyrXirTZxV7YnHn898ebcBVCAdxxNVBbyGPv3zYOMNDjHf/0Kz5JnAAj/BRg2fqp8tMri6Ds/5ZY079mt4OHSSv+0XBNs68zYo8Cm3QsPLD8YZCbEB1IrLoWLd/csR5/kxPqy4RhKioZujGJUZ52INTSiysr/W8GndHxzKBMJu91pGNzy6bpMRd8g+of8pYL7z9SLidy4F/LSO0K1nQDklkuWkG+/gtnr3bn26ap91XyZ43hrvYdHGrEoSuC8UKkOw8Q0ynbX2LuKgZLI/C1clM8LedGViUqNeAJf467G8Eq5zC/mjz4JQrtHJZQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass
header.d=arm.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com;
s=selector2-armh-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=Fke22pQ5prljRBSpmoD4wNeDZr+bsSuNuu4VyeKOUDk=;
b=M8Lf0utPL2BAhoat014HcPloA4xUlSCk7/SmJqFkNpNS0S/uX/FgWEOw5RjDNmpf2Y2AptjMZ1vAm1kB8kwnsU4O9TAPB4/L8GhYgr2st0tuZe7s8ie5djlGODuLNYNiMjJkP4PNCd13NyuWjDcb9zSqCJINIV5UldM7nYLHxLQ=
Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6)
by AM0PR08MB5490.eurprd08.prod.outlook.com (2603:10a6:208:184::23) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5250.13; Tue, 10 May
2022 22:01:55 +0000
Received: from DBAPR08MB5814.eurprd08.prod.outlook.com
([fe80::44cf:1f8d:e477:1f99]) by DBAPR08MB5814.eurprd08.prod.outlook.com
([fe80::44cf:1f8d:e477:1f99%8]) with mapi id 15.20.5227.023; Tue, 10 May 2022
22:01:55 +0000
From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
To: =?iso-8859-1?Q?Morten_Br=F8rup?= <mb@smartsharesystems.com>, Feifei Wang
<Feifei.Wang2@arm.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, nd <nd@arm.com>, Ruifeng Wang
<Ruifeng.Wang@arm.com>, nd <nd@arm.com>
Subject: RE: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
Thread-Topic: [PATCH v1 5/5] examples/l3fwd: enable direct rearm mode
Thread-Index: AQHYVI8Os3vmCXi1HEGk+2Bx5G4N06z4lK6AgAEIHHCAAEehgIAezJ5Q
Date: Tue, 10 May 2022 22:01:55 +0000
Message-ID: <DBAPR08MB58142E7A3CF788182F35540298C99@DBAPR08MB5814.eurprd08.prod.outlook.com>
References: <20220420081650.2043183-1-feifei.wang2@arm.com>
<20220420081650.2043183-6-feifei.wang2@arm.com>
<98CBD80474FA8B44BF855DF32C47DC35D86FE4@smartserver.smartshare.dk>
<DBAPR08MB581470A4DE41AC755F5FD25898F49@DBAPR08MB5814.eurprd08.prod.outlook.com>
<98CBD80474FA8B44BF855DF32C47DC35D86FE8@smartserver.smartshare.dk>
In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D86FE8@smartserver.smartshare.dk>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ts-tracking-id: 39D9400ED234EF4F8C793F9DE8DDF702.0
x-checkrecipientchecked: true
Authentication-Results-Original: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=arm.com;
X-MS-Office365-Filtering-Correlation-Id: 743f6ac7-b559-4dee-941c-08da32d0c5e2
x-ms-traffictypediagnostic: AM0PR08MB5490:EE_|DB5EUR03FT026:EE_|AM6PR08MB4184:EE_
X-Microsoft-Antispam-PRVS: <AM6PR08MB41842589760A323B2B031CCE98C99@AM6PR08MB4184.eurprd08.prod.outlook.com>
x-checkrecipientrouted: true
nodisclaimer: true
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam-Untrusted: BCL:0;
X-Microsoft-Antispam-Message-Info-Original: CRhkkywmWGangP6PMpimy1FRpCmSkCARG68y3xVoKlrlLcKuxJiQnqYYhUhOdgXmKc6kTDLAmqi34LsWxPD9RyBVto07viJNXaHNQvCP28Je1mckssqrLomJci6OKftRmCVpcbims8dutDoNb4leXt52szBRaNavqP99yz7kEUOD6BTgn6YYTDtI6jRCFDlzpuJas8wTFXGv3Q1vTzXe4duMMnn17a0y2/VmBsukZTTJ7Sr/JS6/fsBZuwBnh29R9Dg/KRES5gm5WyL2Xt5rNz3eGDkadh1MjGlTRK9JKxQeZJN9n3CbeRofMnA58S6cZmATPGhQKZQjUm3mvlQQOwOOa0KmR4m2Hkc2MIopA2vw9Iomwoj0ldZML2RTQFa/hmiM6P30AJly2GOnnEbzwZf35S6sjWdvC/ScEVxLXM3Zg9F03m3I4HwTz2WvZ4CJv42Ds7tfQ+W6otzuz3IkY/S8xZ9BHf8VmNzIcosSaT1C8e98FYP0H7xUpPDWmHgafTld2HtL2YjLKg4mVkRniqo8Px8eT99XRDkHSfMEk4u6kF51PlNiYikP53maDk5/J4gDrxrBNFfJpIwCjmYpaJgamURffHjuoHj6N7VB5ZYFgboVRh34BLx5/i8AFUs0h4ePafqWM7NYlhpGuN3FYwB+QwkfZYfLt6tMjWWwwQnoa5lrWCq8VC6s3dlsgC3g7QyI0tkW8eXNPh8EKuRz7A==
X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en;
SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com;
PTR:; CAT:NONE;
SFS:(13230001)(4636009)(366004)(55016003)(4326008)(5660300002)(66556008)(6506007)(64756008)(76116006)(508600001)(8936002)(26005)(66946007)(9686003)(66476007)(66446008)(7696005)(122000001)(52536014)(38100700002)(38070700005)(8676002)(83380400001)(86362001)(71200400001)(54906003)(186003)(6636002)(316002)(2906002)(110136005)(33656002);
DIR:OUT; SFP:1101;
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR08MB5490
Original-Authentication-Results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs: 1406bfbc-3626-42ba-3cbd-08da32d0b283
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: XMhM8pke4CCcK8ZvvAnPhWJSPwITEHVmTqzAn9OUG+FL8NwQACaRqrEb02C4l5HhDd/WbOEih65W31hJ+bRoW7QrmESowROOAfyqEZ1TEK5uq+5b+1v82m1Z4V6Eh+/wmxsz3j0RWtsKUoXRovGryoQBSPadre6BEGMj2osuuEFVOL0eXpbkZ4BA1q5XGnJGfeS1j+Da9LAW4+jI9MIh4f9r6Zkia7umEADvbYIpPQ6NieHq+G44ufMvrA7VFgslWpud5dt5BvIwlDmTz4ELEoLCiz1Da8OHz+cp1oRmffzwzDRDi+xg91NZu27WnGhlxPDiHXa8ofzGoVDYJk5zGnq9YN2PfW/avF6QAt/E+m1B6/pPQuUrGGo73pzxhZKMjPBSPPmWcjEAPl0xcz1Be0sjYL7IoU/+zNS3aGJNQGmpwuhDBjEb8JmwuB97CNMeEiR/DYzTmtwNIgnrXNurnl4Evo0eY52Ti1N9hmEY2KhS1sz/Odz1vwC9US7OzeBjZHsZet4cWqBcIXFx7SI+eJm/UxDoTnZhfQGeJgUOOeenO/sBbbuez9cU7VhNgVPVIsD2DVw0nZb6tUlEHPyPJFTFOeOzRYSAdnR0CT/LNuU1sx3x3mfhK3iPGYy5zx1UuEdSvbeUJQni+wR27lN81rzcns2+izvQGzPad7eTzCzmLcCQU3kZPxZRXvvVflS6
X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:;
IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com;
PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE;
SFS:(13230001)(4636009)(36840700001)(40470700004)(46966006)(336012)(47076005)(186003)(4326008)(33656002)(8676002)(70586007)(81166007)(70206006)(83380400001)(508600001)(26005)(9686003)(55016003)(86362001)(110136005)(82310400005)(54906003)(316002)(6506007)(7696005)(6636002)(356005)(40460700003)(36860700001)(8936002)(52536014)(2906002)(5660300002);
DIR:OUT; SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2022 22:02:28.0737 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 743f6ac7-b559-4dee-941c-08da32d0c5e2
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123];
Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource: DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4184
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
(apologies for the late response, this one slipped my mind)
Appreciate if others could weigh their opinions.
<snip>
>=20
> > From: Honnappa Nagarahalli [mailto:Honnappa.Nagarahalli@arm.com]
> > Sent: Thursday, 21 April 2022 04.35
> > >
> > > > From: Feifei Wang [mailto:feifei.wang2@arm.com]
> > > > Sent: Wednesday, 20 April 2022 10.17
> > > >
> > > > Enable direct rearm mode. The mapping is decided in the data plane
> > > > based on the first packet received.
> > >
> > > I usually don't care much about l3fwd, but putting configuration
> > changes in the
> > > fast path is just wrong!
> > I would say it depends. In this case the cycles consumed by the API
> > are very less and configuration data is very small and is already in
> > the cache as PMD has accessed the same data structure.
> >
> > If the configuration needs more cycles than a typical (depending on
> > the
> > application) data plane packet processing needs or brings in enormous
> > amount of data in to the cache, it should not be done on the data
> > plane.
> >
>=20
> As a matter of principle, configuration changes should be done outside th=
e fast
> path.
>=20
> If we allow an exception for this feature, it will set a bad precedent ab=
out
> where to put configuration code.
I think there are other examples though not exactly the same. For ex: the s=
eqlock, we cannot have a scheduled out writer while holding the lock. But, =
it was mentioned that this can be over come easily by running the writer on=
an isolated core (which to me breaks some principles).
>=20
> > >
> > > Also, l3fwd is often used for benchmarking, and this small piece of
> > code in the
> > > fast path will affect benchmark results (although only very little).
> > We do not see any impact on the performance numbers. The reason for
> > putting in the data plane was it covers wider use case in this L3fwd
> > application. If the app were to be simple, the configuration could be
> > done from the control plane. Unfortunately, the performance of L3fwd
> > application matters.
> >
>=20
> Let's proceed down that path for the sake of discussion... Then the fast =
path is
> missing runtime verification that all preconditions for using remapping a=
re
> present at any time.
Agree, few checks (ensuring that TX and RX buffers are from the same pool, =
ensuring tx_rs_thresh is same as RX rearm threshold) are missing.
We will add these, it is possible to add these checks outside the packet pr=
ocessing loop.
>=20
> > >
> > > Please move it out of the fast path.
>=20
> BTW, this patch does not call the rte_eth_direct_rxrearm_enable() to enab=
le
> the feature.
>=20
> And finally, this feature should be disabled by default, and only enabled=
by a
> command line parameter or similar. Otherwise, future l3fwd NIC performanc=
e
> reports will provide misleading performance results, if the feature is ut=
ilized.
> Application developers, when comparing NIC performance results, don't car=
e
> about the performance for this unique use case; they care about the
> performance for the generic use case.
>=20
I think this feature is similar to fast free feature (RTE_ETH_TX_OFFLOAD_MB=
UF_FAST_FREE) as you have mentioned in the other thread. It should be handl=
ed similar to how fast free feature is handled.