From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D77BA04FF; Tue, 24 May 2022 04:50:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6A31442685; Tue, 24 May 2022 04:50:07 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2060.outbound.protection.outlook.com [40.107.236.60]) by mails.dpdk.org (Postfix) with ESMTP id 1E64E42670 for ; Tue, 24 May 2022 04:50:06 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hG7Htunr12EV4+liTuQBMGLRy/1INX1ekigbjA+tD0nmO6Jq8BQ4Oql38hR8rcSTgwWIigEzdjORLmeeV15RAppnI0/HO1Vmc3DnFhaOUOUr60/Cj4ZbtghAVvdYraH/sVH29JxQI5P0lC+RwY4oangRzFmLw8eTErKYUiH0GkjJta3inpA6p8ibwXEoepZ3lRCKcCTEjUj0P9HP+pLr3Ob+j9CNgt8WLh4UfmczKz/x1aizhQIQN80f4Pp6vcSfY3CkO7Wpvrgbh0ZCKyQPx39Aiz/4RAWqb7f8SDqcfFEzkyR2fHKAAMN2WyL5KV2ZBOo7pyI719xxme29a4lb3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ae5VL7dJigbODtKgbA/85bFxKJxPQIXI++geTzHG8hA=; b=VojQQKpCtmTNINskKoFnXHMfHXP0sQonTDy+M0o+Y3mbfLe6N5Lx21jsIkFYVGyC461r8kOtCPZOzo6TbhzmuYyccDxGUvAdaPQhcetZ8UqZgfCodEL9cIwtIMQ7q2be6c7/X7s8aINv3tJVHDHunLZpLEbw5caFQlQTx+DuY5WiJW1pTf/VAa7e2299bAFsXlCIkzfKmB99xZXlQdc1NyHfaaPIkcBuHh0gpiChqMDZQa/OHbfrJjg6q/FZhqtiIU5zpYoj5mqa39WtW6HKTFCRVY8VPMPXwkNMdWyxXF12tG5Fh4Nx/74XEweYVTCC778N5bDSBeqBzN/BdCOhAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ae5VL7dJigbODtKgbA/85bFxKJxPQIXI++geTzHG8hA=; b=JKuHG0PiBLyqX+7tOiRNwxjqEf0cATIm1Ax7yxQz1KRU2mXc4uC9NcDW8LDcXDsG3Qs1Zg0jmjaeCd5g4ZMV9gMCuKYHnldvoyCZY3eEgcsY9RzG0CCyDklD7+BuFTp/tOQinXZ3VZMoIbeIbXGl1HBxQAkbHEnzs408hr3mGSWwXai7gtij42HOnQHsflAbP1ei5sqMeB85DWbeUvBlbQAf2ctvWSM5eNr9WgF25LpPEWInerEhWzC/SVnLNhsUTtaJ21TaDXualc7Ck6q0q4RbdUpe0+gveiad2c0LzXtXOX6hygeo9vfaKB3BRG+C6FAa8XvPAn7wHBHdwBquyQ== Received: from MN2PR12MB3647.namprd12.prod.outlook.com (2603:10b6:208:c4::17) by DM6PR12MB4893.namprd12.prod.outlook.com (2603:10b6:5:1bd::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.16; Tue, 24 May 2022 02:50:04 +0000 Received: from MN2PR12MB3647.namprd12.prod.outlook.com ([fe80::f831:cec1:9c49:5988]) by MN2PR12MB3647.namprd12.prod.outlook.com ([fe80::f831:cec1:9c49:5988%6]) with mapi id 15.20.5273.022; Tue, 24 May 2022 02:50:04 +0000 From: Spike Du To: "NBU-Contact-Thomas Monjalon (EXTERNAL)" , Stephen Hemminger CC: "dev@dpdk.org" , Matan Azrad , Slava Ovsiienko , Ori Kam , Raslan Darawsheh , "ferruh.yigit@amd.com" , "andrew.rybchenko@oktetlabs.ru" Subject: RE: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark Thread-Topic: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark Thread-Index: AQHYbe/iAz+l86Nfv02itzUhoBl7mq0rus/ggAFGMgCAAFKO4A== Date: Tue, 24 May 2022 02:50:04 +0000 Message-ID: References: <20220506035645.4101714-1-spiked@nvidia.com> <20220522082321.3cdb7693@hermes.local> <1836405.jbyF5MZJ3u@thomas> In-Reply-To: <1836405.jbyF5MZJ3u@thomas> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 09716aa0-a593-4236-3c68-08da3d301ae5 x-ms-traffictypediagnostic: DM6PR12MB4893:EE_ x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: UsIv7K6S4zETCbQkRpWiu20qZH749/GEpJ+9BN/jLQoJaFnPE3JuONVO1kZRMFBNXI3+4EksovSI5dCtHXiff/ge7F8YGw2uACNR8bGtV5CnNU+cZ1jcNVbJARD3NjQTkLgbyvsjsYy8hed10VPxdhbGwpqxNvlIo4J9zfkM7R5Rgnq6n8HvpMCh91yLXtOz5X053B+CC/bYOYDTWz6YqshPzSQ81aIH7HD3/bbO29m8HxWaJc6F/00NO9a6/o7mCtEyy8bP3sXpeqxuDY4fPZerPVFKSdqce9cu6tOmwHA9VMegj1b+0peQ8d3KOYNDsf1S4m1xgVhTwH3QA42by4ePLz5ZkLuflAdfYaJreLHcGcKnNkAz1oF3/HkS7QHwq6r5YYtu0noh4P1ZCFFB5GBMeUYNKijddnIuvfXeG4z3XpOys3gXv+3/Cd4g0AXvX0TSqK5zADHcV8sSKhvjCJelAAAlVs9m2G/1FbamK5rGwd7jbvR//0aOcM3NvEV1Me5kPh1GX4Q5Vil84HKZ5tZWuQzMNWV+ExI3oVFz4DSONutVystDMqUvVmaoGKi3tExncx/+LMB70ce2E6Fuf/gz0KujsLmFdK/HqVoYcRztisc+rQx2rWdex12St3ETMXDqjYkbSkIIfJKlulyEGEQd+63RKomdE+XowiKn+IHa9RMpQjnplfh3DyUVb4NarN+rSxe6w+slT5dPQMpfpQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB3647.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(66556008)(71200400001)(2906002)(9686003)(26005)(110136005)(316002)(54906003)(52536014)(38070700005)(186003)(4326008)(38100700002)(122000001)(8936002)(33656002)(7696005)(55016003)(53546011)(508600001)(6506007)(5660300002)(86362001)(83380400001)(76116006)(8676002)(66446008)(64756008)(66946007)(66476007); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?AHOnM1H6cIx0aErEPVI/82JpVVrigtnflo/UFbPWXLWTzGbS/3rjwL9gJlrC?= =?us-ascii?Q?L+TiKwz7nUAOhYkEFVhvOYJW3DCtInzeRWgGhjYLUpRCW4xOSwZlD1qV0fWd?= =?us-ascii?Q?Mt1ytSuEfJVFO6EGd7SHJy7a0fK/UpWHWMl4ojw2Ib7ftAhAbcm6yz3oi2WB?= =?us-ascii?Q?PnPtwGmgsZ0WLOulMCc9N2I7VhL7cVQKemvog3rOI8Ew6r4NwdU3nOcvJFhe?= =?us-ascii?Q?IH+FIJ63+VGhEtVf9+rNFcMHfoKgezyavnyRnO24bC0q0CgriqDE5JyHAWyj?= =?us-ascii?Q?j5LB+xwCBDPwwXZd562PcFANV/9+e4wPG6VjmU9EeggMENfXvMvoW2jwyfwX?= =?us-ascii?Q?noLEg5X6mVTAlLmALDwts8PIFee+sThVTXO24dy4Glp4slWCJrgTFO1+Swje?= =?us-ascii?Q?7OF9iVHb3DKQzowycF8kd26EHnW1WgUA0lyiXMpKMF37jXaqtyYkkR7WHorj?= =?us-ascii?Q?dXS+3WFastiajCr6C+mZ68DjjpyeVUHOFB+sZRLhJra1lmYABLbYt3NNbp1+?= =?us-ascii?Q?RMdq861nkBAGczDdHl66S0/FrmwIm10ularMyk0CA+3qWt3aUWUXsjBPmZ/u?= =?us-ascii?Q?/mCKsnqTHHBWMCpVjcbgjwyts3E5vzxNogizij9P2QLHez8APE6IgOyKByH1?= =?us-ascii?Q?aUAhW6NQrEuiPmopeYh/v8p2pqyA66KxHPAAgqzMGd1LZ4K33k9onLxIQN34?= =?us-ascii?Q?z5WGmc55nHMwVxT6b5U3Kig8IvuiFrSyl0DOEvQ7Y4Sr5QnahQ6KOJaTRhr8?= =?us-ascii?Q?yIICDscZcKj8mDfnSUweLtvy0KVpmJnEJaFmgcgUMYgL0YBmbBmvLYOPtPDW?= =?us-ascii?Q?GB5hJ3zN373gi0kvf1DaryTd+THDKPSwIE5qW3iV8AGjIE6m4XTTVv+Ohnv1?= =?us-ascii?Q?ibn8SjYKXhJgcYD30+y7JE5sTxuePQXCttadkFzrvs9waSz5oz/Q8aV/jtqP?= =?us-ascii?Q?NvwmHuhv7qxuccxJmplmcS9CU4/bQmtjWNPogVA5V4JlFqJcSWrCIpfPaAWf?= =?us-ascii?Q?b4BManCsmkICEKZ35i59shfqFKSo7erkTYkDLp3asnDNeXsoCgO6ZDvApRmE?= =?us-ascii?Q?kChFec5QC4oinjaoCrFd7rH7w81cCcyNFlPdKasrKnhHXzEDrUgf5BDisb6E?= =?us-ascii?Q?NY0aPRVsjiE686vqzVY6g4Q0YIOUVbt+H3Gp+3zaqSJF2BA1sLiDtv8dhCxw?= =?us-ascii?Q?W2oqiZwtX9UFyA9XsmMdDfj8kYD4ucMgXOdplb3u8Z0jrHim/g9Us5vocNFL?= =?us-ascii?Q?GkDzIBqxJTb1ko5fQWMhmc5SJ3SJrD1iqRS43Bw96XLE+VyDyLSeNhkSFP16?= =?us-ascii?Q?psPHcaFYsuIaOvfaibCi11YqWsMNSBZ3zz2HR9FrvQpXkIigFiz76o2M+yu0?= =?us-ascii?Q?XQjYMQrMjHfusMbZT5UAIgbZ/yrzq2HCCcnzd8wmjf8LhrefnF3K5K68egYB?= =?us-ascii?Q?QMUVz7jads7yX8V/FXHePb5KpOzyKzicSK992aroCe+m9DPwElpUAo+zj4xF?= =?us-ascii?Q?Mk/u/StIkcWP/nUqTKi7JE3bGKsUygXSf3zL5LZiUGKeIc/EIzLiSyCS8899?= =?us-ascii?Q?gEO5W/GScFj/SOG1DqeZbo5S/Vz1SPQ1BOWZDNmTdzHfHhDJ9JH5qJ2Wm5SY?= =?us-ascii?Q?RuZeooLl8aI3wA/izegs+w2bFqbzAnQJr9DnOajSRKiZM3mvbXiY57W97p1N?= =?us-ascii?Q?98C7A/zI41E3Z0JfBRrD0yVAOWA9yyo3t2sA1G0SaSqfWUMV?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB3647.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 09716aa0-a593-4236-3c68-08da3d301ae5 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 May 2022 02:50:04.4676 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rHHwgwiaeEuXqU7/BFX5QTCB93nBUIQEjeEv6Hv1EPkDPitnAYcMWoHgY4FrGttHxBR3oyPbbsJvd+WlS9vXsQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4893 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Thomas Monjalon > Sent: Tuesday, May 24, 2022 5:46 AM > To: Stephen Hemminger ; Spike Du > > Cc: dev@dpdk.org; Matan Azrad ; Slava Ovsiienko > ; Ori Kam ; Raslan Darawsheh > ; ferruh.yigit@amd.com; > andrew.rybchenko@oktetlabs.ru > Subject: Re: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermar= k >=20 > External email: Use caution opening links or attachments >=20 >=20 > 23/05/2022 05:01, Spike Du: > > From: Stephen Hemminger > > > Spike Du wrote: > > > > --- a/lib/ethdev/rte_ethdev.h > > > > +++ b/lib/ethdev/rte_ethdev.h > > > > @@ -1249,7 +1249,16 @@ struct rte_eth_rxconf { > > > > */ > > > > union rte_eth_rxseg *rx_seg; > > > > > > > > - uint64_t reserved_64s[2]; /**< Reserved for future fields */ > > > > + /** > > > > + * Per-queue Rx limit watermark defined as percentage of Rx q= ueue > > > > + * size. If Rx queue receives traffic higher than this percen= tage, > > > > + * the event RTE_ETH_EVENT_RX_LWM is triggered. > > > > + */ > > > > + uint8_t lwm; > > > > + > > > > + uint8_t reserved_bits[3]; > > > > + uint32_t reserved_32s; > > > > + uint64_t reserved_64s; > > > > > > Ok but, this is an ABI risk about this because reserved stuff was > > > never required before. >=20 > An ABI compatibility issue would be for an application compiled with an o= ld > DPDK, and loading a new DPDK at runtime. > Let's think what would happen in such a case. >=20 > > > Whenever is a reserved field is introduced the code (in this case > > > rte_ethdev_configure). >=20 > rte_eth_rx_queue_setup() is called with rx_conf->lwm not initialized. > Then the library and drivers may interpret a wrong value. >=20 > > > Best practice would have been to have the code require all reserved > > > fields be > > > 0 in earlier releases. In this case an application is like to define > > > a watermark of zero; how will your code handle it. > > > > Having watermark of 0 is desired, which is the default. LWM of 0 means > > the Rx Queue's watermark is not monitored, hence no LWM event is > generated. >=20 > The problem is to have a value not initialized. > I think the best approach is to not expose the LWM value through this > configuration structure. > If the need is to get the current value, we should better add a field in = the > struct rte_eth_rxq_info. At least from all the dpdk app/example code, rxconf is initialized to 0 the= n setup The Rx queue, if user follows these examples we should not have ABI issue. Since many people are concerned about rxconf change, it's ok to remove the = LWM Field there. Yes, I think we can add lwm into rte_eth_rxq_info. If we can set Rx queue's= attribute, We should have a way to get it. >=20 > [...] > > > > > Why introduce new query/set operations? This should just be part of > > > the overall device configuration. >=20 > Thanks to the "set" function, we can avoid the ABI compat issue. >=20 > > Due to different implementation. LWM can be a dynamic configuration > which can help user design a flexible flow control. > > User may feel ok with LWM of 80% to get high throughput, or later on wi= th > 50% to throttle the traffic responsively by handling LWM event in order t= o > reduce drop. > > Some driver like mlx5 may implement LWM event as one-time shot. When > > you receive LWM event, you need to reconfigure LWM in order to receive > the event again, thus you will not likely to be overwhelmed by the events= . > > These all require set operation. >=20 > Yes it is better to allow dynamic watermark configuration, not using the > function rte_eth_rx_queue_setup(). >=20 > > For the query operation. The rte_event API > rte_eth_dev_callback_process() is per-port API, it doesn't carry much > information when an event happens. > > When a LWM event happens, we need to know in which Rx queue it > happens or optionally what's the current LWM percentage of this queue. > > The query operation serves this purpose. >=20 > Yes "query" has to be called in the event handler because event structure= is > not specific to any event type. >=20 >=20