From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 7D77BA04FF;
Tue, 24 May 2022 04:50:07 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 6A31442685;
Tue, 24 May 2022 04:50:07 +0200 (CEST)
Received: from NAM11-BN8-obe.outbound.protection.outlook.com
(mail-bn8nam11on2060.outbound.protection.outlook.com [40.107.236.60])
by mails.dpdk.org (Postfix) with ESMTP id 1E64E42670
for <dev@dpdk.org>; Tue, 24 May 2022 04:50:06 +0200 (CEST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=hG7Htunr12EV4+liTuQBMGLRy/1INX1ekigbjA+tD0nmO6Jq8BQ4Oql38hR8rcSTgwWIigEzdjORLmeeV15RAppnI0/HO1Vmc3DnFhaOUOUr60/Cj4ZbtghAVvdYraH/sVH29JxQI5P0lC+RwY4oangRzFmLw8eTErKYUiH0GkjJta3inpA6p8ibwXEoepZ3lRCKcCTEjUj0P9HP+pLr3Ob+j9CNgt8WLh4UfmczKz/x1aizhQIQN80f4Pp6vcSfY3CkO7Wpvrgbh0ZCKyQPx39Aiz/4RAWqb7f8SDqcfFEzkyR2fHKAAMN2WyL5KV2ZBOo7pyI719xxme29a4lb3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=Ae5VL7dJigbODtKgbA/85bFxKJxPQIXI++geTzHG8hA=;
b=VojQQKpCtmTNINskKoFnXHMfHXP0sQonTDy+M0o+Y3mbfLe6N5Lx21jsIkFYVGyC461r8kOtCPZOzo6TbhzmuYyccDxGUvAdaPQhcetZ8UqZgfCodEL9cIwtIMQ7q2be6c7/X7s8aINv3tJVHDHunLZpLEbw5caFQlQTx+DuY5WiJW1pTf/VAa7e2299bAFsXlCIkzfKmB99xZXlQdc1NyHfaaPIkcBuHh0gpiChqMDZQa/OHbfrJjg6q/FZhqtiIU5zpYoj5mqa39WtW6HKTFCRVY8VPMPXwkNMdWyxXF12tG5Fh4Nx/74XEweYVTCC778N5bDSBeqBzN/BdCOhAw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com;
dkim=pass header.d=nvidia.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com;
s=selector2;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=Ae5VL7dJigbODtKgbA/85bFxKJxPQIXI++geTzHG8hA=;
b=JKuHG0PiBLyqX+7tOiRNwxjqEf0cATIm1Ax7yxQz1KRU2mXc4uC9NcDW8LDcXDsG3Qs1Zg0jmjaeCd5g4ZMV9gMCuKYHnldvoyCZY3eEgcsY9RzG0CCyDklD7+BuFTp/tOQinXZ3VZMoIbeIbXGl1HBxQAkbHEnzs408hr3mGSWwXai7gtij42HOnQHsflAbP1ei5sqMeB85DWbeUvBlbQAf2ctvWSM5eNr9WgF25LpPEWInerEhWzC/SVnLNhsUTtaJ21TaDXualc7Ck6q0q4RbdUpe0+gveiad2c0LzXtXOX6hygeo9vfaKB3BRG+C6FAa8XvPAn7wHBHdwBquyQ==
Received: from MN2PR12MB3647.namprd12.prod.outlook.com (2603:10b6:208:c4::17)
by DM6PR12MB4893.namprd12.prod.outlook.com (2603:10b6:5:1bd::19) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.16; Tue, 24 May
2022 02:50:04 +0000
Received: from MN2PR12MB3647.namprd12.prod.outlook.com
([fe80::f831:cec1:9c49:5988]) by MN2PR12MB3647.namprd12.prod.outlook.com
([fe80::f831:cec1:9c49:5988%6]) with mapi id 15.20.5273.022; Tue, 24 May 2022
02:50:04 +0000
From: Spike Du <spiked@nvidia.com>
To: "NBU-Contact-Thomas Monjalon (EXTERNAL)" <thomas@monjalon.net>, Stephen
Hemminger <stephen@networkplumber.org>
CC: "dev@dpdk.org" <dev@dpdk.org>, Matan Azrad <matan@nvidia.com>, Slava
Ovsiienko <viacheslavo@nvidia.com>, Ori Kam <orika@nvidia.com>, Raslan
Darawsheh <rasland@nvidia.com>, "ferruh.yigit@amd.com"
<ferruh.yigit@amd.com>, "andrew.rybchenko@oktetlabs.ru"
<andrew.rybchenko@oktetlabs.ru>
Subject: RE: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark
Thread-Topic: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark
Thread-Index: AQHYbe/iAz+l86Nfv02itzUhoBl7mq0rus/ggAFGMgCAAFKO4A==
Date: Tue, 24 May 2022 02:50:04 +0000
Message-ID: <MN2PR12MB3647A056A4721B7283DC245FA8D79@MN2PR12MB3647.namprd12.prod.outlook.com>
References: <20220506035645.4101714-1-spiked@nvidia.com>
<20220522082321.3cdb7693@hermes.local>
<MN2PR12MB3647125617695A255DB326DCA8D49@MN2PR12MB3647.namprd12.prod.outlook.com>
<1836405.jbyF5MZJ3u@thomas>
In-Reply-To: <1836405.jbyF5MZJ3u@thomas>
Accept-Language: zh-CN, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: dkim=none (message not signed)
header.d=none;dmarc=none action=none header.from=nvidia.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 09716aa0-a593-4236-3c68-08da3d301ae5
x-ms-traffictypediagnostic: DM6PR12MB4893:EE_
x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr
x-microsoft-antispam-prvs: <DM6PR12MB4893C1E922D2EBDED64D5B99A8D79@DM6PR12MB4893.namprd12.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: UsIv7K6S4zETCbQkRpWiu20qZH749/GEpJ+9BN/jLQoJaFnPE3JuONVO1kZRMFBNXI3+4EksovSI5dCtHXiff/ge7F8YGw2uACNR8bGtV5CnNU+cZ1jcNVbJARD3NjQTkLgbyvsjsYy8hed10VPxdhbGwpqxNvlIo4J9zfkM7R5Rgnq6n8HvpMCh91yLXtOz5X053B+CC/bYOYDTWz6YqshPzSQ81aIH7HD3/bbO29m8HxWaJc6F/00NO9a6/o7mCtEyy8bP3sXpeqxuDY4fPZerPVFKSdqce9cu6tOmwHA9VMegj1b+0peQ8d3KOYNDsf1S4m1xgVhTwH3QA42by4ePLz5ZkLuflAdfYaJreLHcGcKnNkAz1oF3/HkS7QHwq6r5YYtu0noh4P1ZCFFB5GBMeUYNKijddnIuvfXeG4z3XpOys3gXv+3/Cd4g0AXvX0TSqK5zADHcV8sSKhvjCJelAAAlVs9m2G/1FbamK5rGwd7jbvR//0aOcM3NvEV1Me5kPh1GX4Q5Vil84HKZ5tZWuQzMNWV+ExI3oVFz4DSONutVystDMqUvVmaoGKi3tExncx/+LMB70ce2E6Fuf/gz0KujsLmFdK/HqVoYcRztisc+rQx2rWdex12St3ETMXDqjYkbSkIIfJKlulyEGEQd+63RKomdE+XowiKn+IHa9RMpQjnplfh3DyUVb4NarN+rSxe6w+slT5dPQMpfpQ==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:MN2PR12MB3647.namprd12.prod.outlook.com; PTR:; CAT:NONE;
SFS:(13230001)(4636009)(366004)(66556008)(71200400001)(2906002)(9686003)(26005)(110136005)(316002)(54906003)(52536014)(38070700005)(186003)(4326008)(38100700002)(122000001)(8936002)(33656002)(7696005)(55016003)(53546011)(508600001)(6506007)(5660300002)(86362001)(83380400001)(76116006)(8676002)(66446008)(64756008)(66946007)(66476007);
DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?AHOnM1H6cIx0aErEPVI/82JpVVrigtnflo/UFbPWXLWTzGbS/3rjwL9gJlrC?=
=?us-ascii?Q?L+TiKwz7nUAOhYkEFVhvOYJW3DCtInzeRWgGhjYLUpRCW4xOSwZlD1qV0fWd?=
=?us-ascii?Q?Mt1ytSuEfJVFO6EGd7SHJy7a0fK/UpWHWMl4ojw2Ib7ftAhAbcm6yz3oi2WB?=
=?us-ascii?Q?PnPtwGmgsZ0WLOulMCc9N2I7VhL7cVQKemvog3rOI8Ew6r4NwdU3nOcvJFhe?=
=?us-ascii?Q?IH+FIJ63+VGhEtVf9+rNFcMHfoKgezyavnyRnO24bC0q0CgriqDE5JyHAWyj?=
=?us-ascii?Q?j5LB+xwCBDPwwXZd562PcFANV/9+e4wPG6VjmU9EeggMENfXvMvoW2jwyfwX?=
=?us-ascii?Q?noLEg5X6mVTAlLmALDwts8PIFee+sThVTXO24dy4Glp4slWCJrgTFO1+Swje?=
=?us-ascii?Q?7OF9iVHb3DKQzowycF8kd26EHnW1WgUA0lyiXMpKMF37jXaqtyYkkR7WHorj?=
=?us-ascii?Q?dXS+3WFastiajCr6C+mZ68DjjpyeVUHOFB+sZRLhJra1lmYABLbYt3NNbp1+?=
=?us-ascii?Q?RMdq861nkBAGczDdHl66S0/FrmwIm10ularMyk0CA+3qWt3aUWUXsjBPmZ/u?=
=?us-ascii?Q?/mCKsnqTHHBWMCpVjcbgjwyts3E5vzxNogizij9P2QLHez8APE6IgOyKByH1?=
=?us-ascii?Q?aUAhW6NQrEuiPmopeYh/v8p2pqyA66KxHPAAgqzMGd1LZ4K33k9onLxIQN34?=
=?us-ascii?Q?z5WGmc55nHMwVxT6b5U3Kig8IvuiFrSyl0DOEvQ7Y4Sr5QnahQ6KOJaTRhr8?=
=?us-ascii?Q?yIICDscZcKj8mDfnSUweLtvy0KVpmJnEJaFmgcgUMYgL0YBmbBmvLYOPtPDW?=
=?us-ascii?Q?GB5hJ3zN373gi0kvf1DaryTd+THDKPSwIE5qW3iV8AGjIE6m4XTTVv+Ohnv1?=
=?us-ascii?Q?ibn8SjYKXhJgcYD30+y7JE5sTxuePQXCttadkFzrvs9waSz5oz/Q8aV/jtqP?=
=?us-ascii?Q?NvwmHuhv7qxuccxJmplmcS9CU4/bQmtjWNPogVA5V4JlFqJcSWrCIpfPaAWf?=
=?us-ascii?Q?b4BManCsmkICEKZ35i59shfqFKSo7erkTYkDLp3asnDNeXsoCgO6ZDvApRmE?=
=?us-ascii?Q?kChFec5QC4oinjaoCrFd7rH7w81cCcyNFlPdKasrKnhHXzEDrUgf5BDisb6E?=
=?us-ascii?Q?NY0aPRVsjiE686vqzVY6g4Q0YIOUVbt+H3Gp+3zaqSJF2BA1sLiDtv8dhCxw?=
=?us-ascii?Q?W2oqiZwtX9UFyA9XsmMdDfj8kYD4ucMgXOdplb3u8Z0jrHim/g9Us5vocNFL?=
=?us-ascii?Q?GkDzIBqxJTb1ko5fQWMhmc5SJ3SJrD1iqRS43Bw96XLE+VyDyLSeNhkSFP16?=
=?us-ascii?Q?psPHcaFYsuIaOvfaibCi11YqWsMNSBZ3zz2HR9FrvQpXkIigFiz76o2M+yu0?=
=?us-ascii?Q?XQjYMQrMjHfusMbZT5UAIgbZ/yrzq2HCCcnzd8wmjf8LhrefnF3K5K68egYB?=
=?us-ascii?Q?QMUVz7jads7yX8V/FXHePb5KpOzyKzicSK992aroCe+m9DPwElpUAo+zj4xF?=
=?us-ascii?Q?Mk/u/StIkcWP/nUqTKi7JE3bGKsUygXSf3zL5LZiUGKeIc/EIzLiSyCS8899?=
=?us-ascii?Q?gEO5W/GScFj/SOG1DqeZbo5S/Vz1SPQ1BOWZDNmTdzHfHhDJ9JH5qJ2Wm5SY?=
=?us-ascii?Q?RuZeooLl8aI3wA/izegs+w2bFqbzAnQJr9DnOajSRKiZM3mvbXiY57W97p1N?=
=?us-ascii?Q?98C7A/zI41E3Z0JfBRrD0yVAOWA9yyo3t2sA1G0SaSqfWUMV?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB3647.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 09716aa0-a593-4236-3c68-08da3d301ae5
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 May 2022 02:50:04.4676 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: rHHwgwiaeEuXqU7/BFX5QTCB93nBUIQEjeEv6Hv1EPkDPitnAYcMWoHgY4FrGttHxBR3oyPbbsJvd+WlS9vXsQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4893
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, May 24, 2022 5:46 AM
> To: Stephen Hemminger <stephen@networkplumber.org>; Spike Du
> <spiked@nvidia.com>
> Cc: dev@dpdk.org; Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>; Raslan Darawsheh
> <rasland@nvidia.com>; ferruh.yigit@amd.com;
> andrew.rybchenko@oktetlabs.ru
> Subject: Re: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermar=
k
>=20
> External email: Use caution opening links or attachments
>=20
>=20
> 23/05/2022 05:01, Spike Du:
> > From: Stephen Hemminger <stephen@networkplumber.org>
> > > Spike Du <spiked@nvidia.com> wrote:
> > > > --- a/lib/ethdev/rte_ethdev.h
> > > > +++ b/lib/ethdev/rte_ethdev.h
> > > > @@ -1249,7 +1249,16 @@ struct rte_eth_rxconf {
> > > > */
> > > > union rte_eth_rxseg *rx_seg;
> > > >
> > > > - uint64_t reserved_64s[2]; /**< Reserved for future fields */
> > > > + /**
> > > > + * Per-queue Rx limit watermark defined as percentage of Rx q=
ueue
> > > > + * size. If Rx queue receives traffic higher than this percen=
tage,
> > > > + * the event RTE_ETH_EVENT_RX_LWM is triggered.
> > > > + */
> > > > + uint8_t lwm;
> > > > +
> > > > + uint8_t reserved_bits[3];
> > > > + uint32_t reserved_32s;
> > > > + uint64_t reserved_64s;
> > >
> > > Ok but, this is an ABI risk about this because reserved stuff was
> > > never required before.
>=20
> An ABI compatibility issue would be for an application compiled with an o=
ld
> DPDK, and loading a new DPDK at runtime.
> Let's think what would happen in such a case.
>=20
> > > Whenever is a reserved field is introduced the code (in this case
> > > rte_ethdev_configure).
>=20
> rte_eth_rx_queue_setup() is called with rx_conf->lwm not initialized.
> Then the library and drivers may interpret a wrong value.
>=20
> > > Best practice would have been to have the code require all reserved
> > > fields be
> > > 0 in earlier releases. In this case an application is like to define
> > > a watermark of zero; how will your code handle it.
> >
> > Having watermark of 0 is desired, which is the default. LWM of 0 means
> > the Rx Queue's watermark is not monitored, hence no LWM event is
> generated.
>=20
> The problem is to have a value not initialized.
> I think the best approach is to not expose the LWM value through this
> configuration structure.
> If the need is to get the current value, we should better add a field in =
the
> struct rte_eth_rxq_info.
At least from all the dpdk app/example code, rxconf is initialized to 0 the=
n setup
The Rx queue, if user follows these examples we should not have ABI issue.
Since many people are concerned about rxconf change, it's ok to remove the =
LWM
Field there.
Yes, I think we can add lwm into rte_eth_rxq_info. If we can set Rx queue's=
attribute,
We should have a way to get it.
>=20
> [...]
> >
> > > Why introduce new query/set operations? This should just be part of
> > > the overall device configuration.
>=20
> Thanks to the "set" function, we can avoid the ABI compat issue.
>=20
> > Due to different implementation. LWM can be a dynamic configuration
> which can help user design a flexible flow control.
> > User may feel ok with LWM of 80% to get high throughput, or later on wi=
th
> 50% to throttle the traffic responsively by handling LWM event in order t=
o
> reduce drop.
> > Some driver like mlx5 may implement LWM event as one-time shot. When
> > you receive LWM event, you need to reconfigure LWM in order to receive
> the event again, thus you will not likely to be overwhelmed by the events=
.
> > These all require set operation.
>=20
> Yes it is better to allow dynamic watermark configuration, not using the
> function rte_eth_rx_queue_setup().
>=20
> > For the query operation. The rte_event API
> rte_eth_dev_callback_process() is per-port API, it doesn't carry much
> information when an event happens.
> > When a LWM event happens, we need to know in which Rx queue it
> happens or optionally what's the current LWM percentage of this queue.
> > The query operation serves this purpose.
>=20
> Yes "query" has to be called in the event handler because event structure=
is
> not specific to any event type.
>=20
>=20