From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BD972A04FD; Mon, 23 May 2022 05:01:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9FFEF40156; Mon, 23 May 2022 05:01:25 +0200 (CEST) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2075.outbound.protection.outlook.com [40.107.244.75]) by mails.dpdk.org (Postfix) with ESMTP id AB8A54014F for ; Mon, 23 May 2022 05:01:23 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kql+PD/SNSQOG0N2KEn0VGUgNYrwCb7AtlTXmhJg/NpzbqNmExIYxwPlBRcxoWMgYo2iN0P5CM6VIqXGdRLsdNkwIJbjZWlFs/Hr43XEzaDKvlkgXVkav35H9aO5dSB2dMsxKnwm1NYRAH7z2D0Z3NCQ4bTBXFR6xJH99J5gmFZzdI1hrcJJzzjsaoE/mWwvSmQDgwqI2AmI7wZU6/iX6s0fNa3OTrf6LMhOdY+9vSYQGbwLqhuOb/4PiwIzPxHVhE7ep2lpuWNoDsCbrGvTPCkD55QpThDFb8xJpdZWPBI757tJ6wTydNQUXoN9g0sI2A2SdqY7+xfAQloNMBiqRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/rA7egiWE1FDFxFNkEgdgRUVHlN2h/y/EA+GQY2a3D4=; b=k9ImyeIYPp/9bWXHBo/cETLKGPvNccPnCW6iOe0nWXvnJCrnsQbfjzuum6H7A416CY6HGkZ+xeSzamZ/IWRLFFShJj4PzGaLhLwFdVTC5E3ZTVl8GAXyCzFfnSpSaMk/kRWFz4OHbyI31q5RDCUkBUO0nxpJORW36w/TVNGJqjTTrGLUfv6rPK2OCwB8Z1nSaBJ0ZdKQsrBEddy0eE0TNgAk0TDR+waIl2fFLOZVFgGFd7diSCBRQpOs3pllsZXTzR1lh196O+MuTgIbBBa97+/t3QGtxv99hoijUvFnGdhuhQ0YHtuYlwX5u3HaRz0338jp051p10D/oo1NddgEXw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/rA7egiWE1FDFxFNkEgdgRUVHlN2h/y/EA+GQY2a3D4=; b=i7jEJiBb5KE+VItNSDfSIu3654jfX3osWUDMIxaWRPZ7BpLfX1Qm/lOvONbUhJOOJ6fQaOtyZQSX7BeRjBxerQidCDz4VQXFuBZA0HjCEFKHnR/N3IsR7d+4HYlLgQ1gOU5h5r4Dwnr4jZID8xrWrVjGe8JlCn5huGh8VJkJMQWTIgmMhd9f2NaDWR7tEbF5arbbGiepGgia3Ay63K2N9n5Cu1sYpJe2u2KjBWcy7BEOFhT/el+S9bNDb1q6GX9fzD52MJpFOokkoywHjQke/YHxOvtL8MhxPznOezqGm1CbiHI3Mc395m9FlWICL5RUJqB7aiH0TbmvVa0uMzpmSg== Received: from MN2PR12MB3647.namprd12.prod.outlook.com (2603:10b6:208:c4::17) by PH7PR12MB5975.namprd12.prod.outlook.com (2603:10b6:510:1da::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5273.21; Mon, 23 May 2022 03:01:20 +0000 Received: from MN2PR12MB3647.namprd12.prod.outlook.com ([fe80::f831:cec1:9c49:5988]) by MN2PR12MB3647.namprd12.prod.outlook.com ([fe80::f831:cec1:9c49:5988%6]) with mapi id 15.20.5273.022; Mon, 23 May 2022 03:01:20 +0000 From: Spike Du To: Stephen Hemminger CC: Matan Azrad , Slava Ovsiienko , Ori Kam , "NBU-Contact-Thomas Monjalon (EXTERNAL)" , "dev@dpdk.org" , Raslan Darawsheh Subject: RE: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark Thread-Topic: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark Thread-Index: AQHYbe/iAz+l86Nfv02itzUhoBl7mq0rus/g Date: Mon, 23 May 2022 03:01:20 +0000 Message-ID: References: <20220506035645.4101714-1-spiked@nvidia.com> <20220522055900.417282-1-spiked@nvidia.com> <20220522055900.417282-4-spiked@nvidia.com> <20220522082321.3cdb7693@hermes.local> In-Reply-To: <20220522082321.3cdb7693@hermes.local> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a900cda5-67d9-4761-ee34-08da3c68834e x-ms-traffictypediagnostic: PH7PR12MB5975:EE_ x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: s+mgmp9uavy8YqjcgMjWBFJgiJVQatwvfUgSPbYXrNbVWi0qXX6V+oOrhbpUFYADGPI8CM1W8CjXab4XDsIGpvJV7+nxGEtTMDyAqF7MhTZS8V06MFqKktKu8SoW55P/34puLt6UKaF3UOHcLhgL5maRj8+cBQ2VKfpubc+eyOaDJmT4vqGgr/XUbCBkLaUp24dPjILtD2fukHOLrroShkcOJ881k95mUzY/gBFdsaHfoh1zu+FySNU0mwYEPG4AC3CYTL45hZkJ5nVDSED5JRLkFdIW3dvvMoSzvjCGHh8zqRn6x/l0Zg/RnrVlQFlTwKKOqjTE5lsL2B4wmx3QYhAi3CgENqLA1OhGdbTltt9bJQt8HmdAAp4SpPVcuOfMGIoMMteYqrRqWb8I7hdhw6ctqPm14/1/U9aHxX45xr+C1RH6Hhs8KdUp5zOaiEJXRyotzVNycf8I28I72o93GuDg2DVd7P6JKwyAaf0AzHZqUNglz3EsEASU6ycx1qwdXOHvAozoZLK3hJpwEWo5yc3cGRsdPp4WIjYZULIGlTsgLOL/fqj6aVfZyIljdIoZU+/qlu5pzWuEXSQPF3wu0A4lJU/DPm9uSUjfLvEcOVAhpOwRa1jC9rYhZLG/24OHmpump3lgwjy5AJLtM70YtM9DpfJgmgcbMjVlJCa1OznWJkx4WyCowoWxH1rK1TAUDXk0mAhBOPEpH6PRwe1P0g== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MN2PR12MB3647.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(66446008)(64756008)(8676002)(66476007)(4326008)(71200400001)(8936002)(5660300002)(53546011)(52536014)(6506007)(76116006)(66946007)(66556008)(54906003)(6916009)(316002)(122000001)(7696005)(38070700005)(86362001)(508600001)(83380400001)(107886003)(186003)(55016003)(9686003)(33656002)(2906002)(38100700002)(26005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?8jJnsSYB3v3HCDmepDQOTWJpvDbGCgA3/kApB+ul4fPiVEEXIlcqWKKS7+ec?= =?us-ascii?Q?Wbi538es08NjJz+7sow6kcRwMLmJDSVoQA/iCd3Spacp4QFQNl5oRsRAKF5L?= =?us-ascii?Q?wunwYneqiYNzTp2VlAEfujYyjNKCHHhAI/DzQcEYuuoLQyL/AXKpXLUVSPdS?= =?us-ascii?Q?1GzWfZm5GDmc3mOKnsJoFCkpC8jzvsXymbsVYtTl+LIOgGFH3F0EJVoqP6nA?= =?us-ascii?Q?5hYOl9xSNF/pXKSColH5QeBJIzGyDsLvNIW+rTnLUUUy5/5OQcKd6VuLiO+J?= =?us-ascii?Q?Z7Tf5Qi2UhRDI6O9QE0RNY13q3ChNrM7EH/1w2RPNO2JChbOMUXHodu0khh6?= =?us-ascii?Q?KVfFYa6A4NQ8vKd4PMx0N9qHoFwn3LRmCqaEheindv9O3Uy/CsUs+Svy5bUM?= =?us-ascii?Q?7bSCXhgMe2cmQicks3Yy8oeF59lKeBXbIBagW/sFzM1DDiy46Q7Lbk6oSPTR?= =?us-ascii?Q?kNmKnEo1Iiu0Xf3RYL8nE2v4KXx4ivrTtCltSmk+ZJPeWf56H8Iiuk7QRBXA?= =?us-ascii?Q?utI4vEkqawX9TL/xiikv+y8d6J1bgUqq83R0/vv6BXtr4XEAFXNuYyNx71r/?= =?us-ascii?Q?oYchh8Ys262f5WnLF6As0YF/f2SM5yc+7otHeZ+QqtYPwqtKUhR6LPJAYYGY?= =?us-ascii?Q?kD0IVd31uq1FFgmUj75W0CyF1SRsR7a/zL7G7AgomFd68+gbJFfBPlc3itRO?= =?us-ascii?Q?WJLStz7c8Kp3TqMOlULzLnSYP4o13nfunyqKyFut3oCfWBcHl8u6Oot24Iky?= =?us-ascii?Q?jiIZM6TdEscMVvLKxpH3Xjm7DVOCGS5KojTajsDo6/bpkLEh2XUuNZ7C/cC2?= =?us-ascii?Q?nwjNU1aQNpJaiDaz4QZHfO5bL7pfTN+vsJSUYz9eVpTHSM+ra+DriLNwuKjl?= =?us-ascii?Q?irxCiTYCyfsAB1mbwRjhzC7PIPvWHRchQcrlO45BqEBa3r2sx3pDVaEkcvE7?= =?us-ascii?Q?2ikDseIMMzbKz1fJySIqQw7f2JEIVRhL8Vk5m3fVGAjX6MhYvXRPaooy5hFS?= =?us-ascii?Q?droVHsq/VxNEmv0K7tx02oWl4OKqtjahZGVazqHp4YBo4Vp3C2KifkM0AwUR?= =?us-ascii?Q?vk+1ECnIoOegYMKQsrH37sDSlAqObGjcowip7aPq8A0z6mObRjyEYn01YIx/?= =?us-ascii?Q?iJet+WzmUYaC4VOWi1QeeMv1UlgJPISAtjPXdu0YoonHCP6a767Nlr7/VRqQ?= =?us-ascii?Q?AgSyS2dtgY7qaH3Wi9Gwm2/je+6MmZ2Jw8E2qxbMAqTfOZj94VldqYeqqtPm?= =?us-ascii?Q?r5vmqpGQGuCBnEDqJYB0UwFBEu4TCCYzwgMDls7zvygnT+dnvVeJ3g8ozf+Y?= =?us-ascii?Q?pn/8GTXGwOqHCj3s2/VJLs0k79FMQip6WXtzvs26KqKYdUrMtJWvHROq3XhU?= =?us-ascii?Q?zya2Ax6b3Z9c9KxOgmgnBEMoZPgFMLnHSwlJ299TG91FwMNNIot6SQ5z2bon?= =?us-ascii?Q?7A5vOH8kI9t7qsA48HTEIODarpkhK0ZTJoP3a3G9i13v6FQm+vQnvgYU0k9E?= =?us-ascii?Q?oT/TKV14lnIBizfmsvvgXtbFTt4/7yDAgpZA7eHZ2TRqIr5/08o8rM4Fz+RQ?= =?us-ascii?Q?9yrss+VNpLtZ72anwolM78weV5C0LJu74BI/I7IfF+d5ETYljrPQne7YQziJ?= =?us-ascii?Q?GgUTjHsVMr05HhkxbVg5KlLpma5OS/xxUJht03hr/2X5EUeAihcW1oQHNLqa?= =?us-ascii?Q?RYK/GK9fcdmPmQ9u+iz6WWfyMN1O1wPV8gpKaQIH06LgdCaP?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB3647.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a900cda5-67d9-4761-ee34-08da3c68834e X-MS-Exchange-CrossTenant-originalarrivaltime: 23 May 2022 03:01:20.3443 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 3hnAAey4dpqxUDiUtVJYYQBOvFk61MZPt2+wyhjTUPDo6JRw3lzT7NYsErfX+ZGva94y2SpGmiIQH39S2E8XYg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5975 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi, pls see below. > -----Original Message----- > From: Stephen Hemminger > Sent: Sunday, May 22, 2022 11:23 PM > To: Spike Du > Cc: Matan Azrad ; Slava Ovsiienko > ; Ori Kam ; NBU-Contact- > Thomas Monjalon (EXTERNAL) ; dev@dpdk.org; > Raslan Darawsheh > Subject: Re: [RFC v2 3/7] ethdev: introduce Rx queue based limit watermar= k >=20 > External email: Use caution opening links or attachments >=20 >=20 > On Sun, 22 May 2022 08:58:56 +0300 > Spike Du wrote: >=20 > > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index > > 04cff8ee10..687ae5ff29 100644 > > --- a/lib/ethdev/rte_ethdev.h > > +++ b/lib/ethdev/rte_ethdev.h > > @@ -1249,7 +1249,16 @@ struct rte_eth_rxconf { > > */ > > union rte_eth_rxseg *rx_seg; > > > > - uint64_t reserved_64s[2]; /**< Reserved for future fields */ > > + /** > > + * Per-queue Rx limit watermark defined as percentage of Rx queue > > + * size. If Rx queue receives traffic higher than this percentage= , > > + * the event RTE_ETH_EVENT_RX_LWM is triggered. > > + */ > > + uint8_t lwm; > > + > > + uint8_t reserved_bits[3]; > > + uint32_t reserved_32s; > > + uint64_t reserved_64s; > > void *reserved_ptrs[2]; /**< Reserved for future fields */ > > }; > > >=20 > Ok but, this is an ABI risk about this because reserved stuff was never > required before. > Whenever is a reserved field is introduced the code (in this case > rte_ethdev_configure). >=20 > Best practice would have been to have the code require all reserved field= s be > 0 in earlier releases. In this case an application is like to define a wa= termark of > zero; how will your code handle it. Having watermark of 0 is desired, which is the default. LWM of 0 means the = Rx Queue's watermark is not monitored, hence no LWM event is generated. >=20 > Also, using 8 bits as percentage is different than how other API's handle= this. > Since Rx queue size is in packets, why is this not in packets? The short answer is to simply the LWM configuration. Rx queue descriptor is complex nowadays.=20 For normal queue, user may configure LWM according to queue descriptor numb= er easily. But for below queues, it's not easy: Take mprq as example, the testpmd cmd options can be " -a 0000:03:00.0,rxq= s_min_mprq=3D1,mprq_en=3D1,mprq_max_memcpy_len=3D465,mprq_log_stride_size= =3D8,mprq_log_stride_num=3D3 -- --mbcache=3D512 -i --nb-cores=3D7 --txd=3D1024 --rxd=3D1024 ",=20 For MLX5 implementation, the minimum "unit" in queue has 64 descriptors, t= he "unit" number is 16, if you configure according to descriptor number(10= 24) Here, you may easily set LWM as something like 512, but HW doesn't allow it= , because 512 > 16. If you want the watermark to be half, the correct value= is 8. The same issue happens to feature like "Rx queue buffer split" where a pack= et can be split to multiple descriptors. Using percentage doesn't have such issues, PMD will cover all the details. > Also document what behavior of 0 is. Sure. The behavior is like the old days without this feature, pls see above= . > Why introduce new query/set operations? This should just be part of the > overall device configuration. Due to different implementation. LWM can be a dynamic configuration which c= an help user design a flexible flow control. User may feel ok with LWM of 80% to get high throughput, or later on with 5= 0% to throttle the traffic responsively by handling LWM event in order to r= educe drop. Some driver like mlx5 may implement LWM event as one-time shot. When you re= ceive LWM event, you need to reconfigure LWM in order to receive the event = again, thus you will not likely to be overwhelmed by the events. These all require set operation. For the query operation. The rte_event API rte_eth_dev_callback_process() i= s per-port API, it doesn't carry much information when an event happens. When a LWM event happens, we need to know in which Rx queue it happens or o= ptionally what's the current LWM percentage of this queue. The query operation serves this purpose. Regards, Spike.