From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AE1DFA04FD; Tue, 10 May 2022 08:41:00 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 56E65406B4; Tue, 10 May 2022 08:41:00 +0200 (CEST) Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mails.dpdk.org (Postfix) with ESMTP id 4BD064069D for ; Tue, 10 May 2022 08:40:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652164858; x=1683700858; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=HMAGzvmrXEUyqMmCwWj1b2AgZBlH/GsOMxF9mqE9deg=; b=Xx4hM/l8uZ55rXPX8E6ZfCRRKs/5GRWogqg1WaR/ggATtB29TmGuAPxL Y4RmETtI0Gk1nZU1ZPkRTrRtfZSQLuUDC0LgZpioYScJogH0Jq9MwLIC2 CeXN3+Ot5X+hIHOOl7MN6qEl8DzMjMnh9wPNn3a7GYDRtIbyllmDV0AIl yewlKpTlh9zoz8WV8UtTCKHd9vTVjiSrDQGBg7StquNOb/Qtp52AZP6q8 Bb2N/w+it/sq1KxAEwKz6aKzhyfDn6Po9fN0xw8Bf+L5ZPsrA1B0JbtRk C4yOjZeR9aXLa8JpziNbtXRa/N5amhkHM7tck7FpzPb3QxgteBp+9v8wj w==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="268946331" X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="268946331" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 23:40:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,213,1647327600"; d="scan'208";a="623320083" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by fmsmga008.fm.intel.com with ESMTP; 09 May 2022 23:40:56 -0700 Received: from orsmsx610.amr.corp.intel.com (10.22.229.23) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Mon, 9 May 2022 23:40:56 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx610.amr.corp.intel.com (10.22.229.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Mon, 9 May 2022 23:40:56 -0700 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (104.47.51.48) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Mon, 9 May 2022 23:40:55 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jy91u1YPipB5VyKcAeWYlpQ5oUdyS13xhUK4zqn8yZpGoFNSbuTHK9txbHBMmjxUFCG49F7Gzn7A3YJjsvHsWsDI07g0ScjA6CYGP2E2SKYS0f0fC34h88P6ltRTNsivf4XJ6GvLRYMyLR/ccf1icTQIOPwYOQoSOYFx74Dca+gnuiGlwXTh86etMtM94zQzWCest/jscqNjEJJ861ZueFm1LuNdYWE7XbaUVLzSlxUIsaCjGk6tH9YBlo+BNmP48kGCkE1EO5j8XoiMYPZxi1GXREm1Bk8/LlJ9Hal+Xc207t57GICdTm0Oo+DZTU7Q1llRvuekmcquib5AxWuyRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=g5rDfcPRQUOA6ITXjjjKlhjZvJo6D32+2hlB6SO7JC0=; b=OJ/N5Z91ZcXwxQx3tzT1oPxt5XxvvEpRksMS1O4kzj6O16j/xU3rVWM/oAxmq0N2zk1h+MH9fSxL0Lt8xP3dz4hqB3uES/8bjWiUj+2uViDIVI0kN0SpdOjXelJbSrvA6Rkc5l3kx/h2Smd+ZUXfgywF5wctE/RARBz1QYUA5dpdszfMUyTXDN2lGGvW4zxc5gSYmLLH3JGfgrC3pBiJORDsITQXGG/DVNVBo7KjmDd3XzJ36FurhQR0Z1iXXXcnf7z1JyTUA5vPGbIjnmizrSAwrhKvDPyU0M+1AFAgf4dTTsb3bAYScHydWxO7AaGQLPqETfyoYU8jTy/pIdbE1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from SA0PR11MB4670.namprd11.prod.outlook.com (2603:10b6:806:9a::8) by DM4PR11MB6287.namprd11.prod.outlook.com (2603:10b6:8:a6::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.22; Tue, 10 May 2022 06:40:53 +0000 Received: from SA0PR11MB4670.namprd11.prod.outlook.com ([fe80::e525:c6b6:8be5:133e]) by SA0PR11MB4670.namprd11.prod.outlook.com ([fe80::e525:c6b6:8be5:133e%4]) with mapi id 15.20.5227.023; Tue, 10 May 2022 06:40:53 +0000 From: "Ajmera, Megha" To: "Dumitrescu, Cristian" , "Danilewicz, MarcinX" , "dev@dpdk.org" , "Singh, Jasvinder" CC: "Thakur, Sham Singh" Subject: RE: [PATCH v3] sched: enable/disable TC OV at runtime Thread-Topic: [PATCH v3] sched: enable/disable TC OV at runtime Thread-Index: AQHYWhiO16l8WcVRE0mAn+B+frOTY60XDBOAgACwEtA= Date: Tue, 10 May 2022 06:40:53 +0000 Message-ID: References: <20220407145153.238969-2-marcinx.danilewicz@intel.com> <20220427092357.491720-1-marcinx.danilewicz@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.401.20 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: eccf2990-ba64-4ef3-1933-08da325007de x-ms-traffictypediagnostic: DM4PR11MB6287:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: OH+vUdJegf1VRh8GreYKhDjU4HkOEjevlS/NGRAw8DwiKA05hAAeXK4si8XpUBDsWTow+9w9dG18rTz76G2+szf4LGvWIgX8UCvRU5byyN3yv/BHvIBnqmtTWzVWBCYDen68Q8J9e0uI8ILkjwPcK0nJqrlOG2f2f7iHFyAbUe5R6JL3g7taBY7uiOPupAVlGAYJopGtHWX2eFzKmf0bklYae9WziTFhoSonZdeswvOwum+qufUfwohxTjdZM85uhTd+P3SAuofFHL0f6dCo7fLD0X7MTCBqTK7WelRZSti4CLHHhFhUebtlIV4FYhfbDjnWIHXeN46djuvE4yK0DhG2xYgsZPGt6EU/I4lOJeratvyAWAtiVoDkTQY4USfKj3KuDHwsiZU8wZYwDFpYGkD1sQJXW7LRGwOCxBxNspFcezWKcbH+hNphGLN/Avyk3NuA7PU4v7Cru5Bpdh40joHS0yh0rrEjdVPzJTMZir7fOYOVl4KSriZwFwe1TDmZEXTr7Z5/3o7C8R5TDltj5GdAwPhtg7Wa5NQI5ez+Up0zx96xUzlhCo1F2njOaRBG/Zj3iN+frLT3jaIqelG7me+i60Crc5MfIRK0HqLCJNkWx9ZBTAn9H5yjjEvAobim1jXrqq5W173c8U+l0WBlomck/Lut7aWZled9ACNIBy7UgvlwnFH+8YXrY5kI9t40MqYmB2MS6AB5Dy6kLhM2qw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SA0PR11MB4670.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(2906002)(6506007)(71200400001)(7696005)(8936002)(52536014)(33656002)(86362001)(53546011)(508600001)(5660300002)(30864003)(83380400001)(38070700005)(122000001)(26005)(38100700002)(9686003)(107886003)(82960400001)(55016003)(186003)(66476007)(316002)(66946007)(76116006)(66556008)(110136005)(6636002)(66446008)(4326008)(8676002)(64756008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?CDkiX+nbIEebdp74+VWYNNs4dV9uA60cKKoKyifJrIMeu0qVqFrpwL/H6PAb?= =?us-ascii?Q?Vsc3OvEy/UmpdJ+IaVmM5h2WyCGxHrRdGEOMkCWKSQ7LE0RHxGvSTVS4vlRU?= =?us-ascii?Q?SmYheobzmXV6ilQBh5GSy9LTz9gxJLfn9eETc9HdO94LIlFcnfOfnHE3Bbn7?= =?us-ascii?Q?UVKjOkyVdIEfaJVyVolTFpAVZm4ti12WyRCwTCgtgxeuAqWqVDPVMI5Bu0jJ?= =?us-ascii?Q?f5MbFfBbQ2V9f8I+E56M0UmNP7hzrBi1sS/YUzKxunTBrzJRj+Jf7Q8HeTir?= =?us-ascii?Q?Z3060pgIG2FfCeFCz9N+dUBvGrTq3qbsIvtnnoHljJEbEAdWtfWvOR9Yp5fb?= =?us-ascii?Q?ckPYiXZOv4ODpou0l11WpUmOfC/fAN+hrJf6Woc+kg3jaio3gGklFjMjunu0?= =?us-ascii?Q?JDHRq1WvZxy+a6KYqWHrOmih7c0tkhDHS+mKZuPbL3b0kYs+foMJ1iOMRaFy?= =?us-ascii?Q?WwMa3d80yUNTCj6xzwW2rCeVTqagT3D3PbpwXsgbnD7giNiDCr+AvTaMwMEQ?= =?us-ascii?Q?ea0cWBRRWBl8sPA6vgZw9CKhS+1QoI9hm6EO7gdCH/3EcDxB7HDKdRUbxFmi?= =?us-ascii?Q?VzmzgSoic30BhMhy/r8eDAsT9bNtHn8oBvmxBJCqMOkgVUT+Ya9Q3Nvv1Tyv?= =?us-ascii?Q?pUBn2aSYJQ+e/4bGjMzd52wxD2y+A4vMDAMoW5wuFnIhFonhB54dGOExv98p?= =?us-ascii?Q?W1V/FilW9aF10AWc4bSuAV7LKzm89SytoXrlinet4VyQgjK/GGG8Z5bwZQRe?= =?us-ascii?Q?oUg5RdAnIcNts/2PWEThIzIAJX1St552B2xIydOrX80NEnStbeAc0kd58uSs?= =?us-ascii?Q?sk3aicD64ZwptSofm+GRfVz03q25gpDmqw5zvo/Som1i5cBdJ4/2saZXOEW3?= =?us-ascii?Q?7DoEtXDwqGb6wc2f/XYNWvR5o2lYIgwIDgaZU1HIgjbC6zk+X/+q4ht9fzVM?= =?us-ascii?Q?8b5d0vNSExMt8tyU4xpoMDfBY/453+3aF46It2FmnPjQtSDmznLch4n12i9d?= =?us-ascii?Q?LktK/pWcWPTshPZnZ0HqU6/O1F6QcETHj1bvKfNbvc0yegywPV7MiBoM6ufb?= =?us-ascii?Q?l3iffjk2Fdu5VKi2JrD5MpD6kvtPSNPjoBvJ3daWY+oC2BK+alwrjc4cy95x?= =?us-ascii?Q?gWgvKu10gD7RwNsQG9V+1MMXRT3nPz40OEX3PO7sx2p2oPLiE4PF4zZ3Z1yl?= =?us-ascii?Q?8+kQyqwQamhH0ZGIxrPMMICNPfO0/nGyV6bvF9TGOm52pr4S2RDGY1yBiOzn?= =?us-ascii?Q?gVsffvZ6YjMAyk7whK/hzf6NMMxUhxar4Q+kN+/T2CsC4DLi6C7RijPVK+Mn?= =?us-ascii?Q?s62GHQ4xpueCW+9I258NqGOHSttFop5Bg4kthZb3aRnFP4U+P1v55bwVm+yi?= =?us-ascii?Q?WOBY8EGaHG4rD2t6/ojddNlTncsuA3pqQGax5w994aSDFGwxrJUDlMU9bNF8?= =?us-ascii?Q?xf4GMbe0bWlw7hy8NpYgGSVv58sgRxeT2W1qhoG+u0S4gb4VSCnA+p/vbgQ6?= =?us-ascii?Q?hgP9iOYaBprcP85nK+ODbqr7GJYTdPaDuyFx8vm2yR39onvKfo0iz+FKoeMN?= =?us-ascii?Q?Yu3FzU+T0m7U5YmzRzRuzpO0aJkb7xGd3VfEKQ2NTyGtt5nOtA6bQeUW7u7P?= =?us-ascii?Q?E5DwGHPFC44hKKznb8OZv0+rHh1j7Xf0KkmyKGYDtkPR7YojNazhDOH4wByw?= =?us-ascii?Q?vwlDs0CgKwXZpzBxRflF8kvQ03cmLVHEt2nuiZrQpJiwrl9oOGMpTox9CGls?= =?us-ascii?Q?nuZQzonehw=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SA0PR11MB4670.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: eccf2990-ba64-4ef3-1933-08da325007de X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2022 06:40:53.6958 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: SB1RrsDIlA27J/+ED9FE2OX9V+hJhgVro0v00kdaVavdOBymmaSc4fRhQU/7NVVuDibJyl/Zpdugeiv71P9MAg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6287 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Cristian, Marcin, > > -----Original Message----- > > From: Danilewicz, MarcinX > > Sent: Wednesday, April 27, 2022 10:24 AM > > To: dev@dpdk.org; Singh, Jasvinder ; > > Dumitrescu, Cristian > > Cc: Ajmera, Megha > > Subject: [PATCH v3] sched: enable/disable TC OV at runtime >=20 > We are not trying to enable/disable the traffic class oversubscription fe= ature at > run-time, but at initialization. If cat, we should prohibit changing this= post- > initialization. > If we only need this to be configured at initialization time, then we can a= s well take this flag in subport config API itself. Then there will be no n= eed for a new API. The purpose of new API was to enable/disable this featur= e at runtime. =20 > Also the name of the feature should not be abbreviated in the patch title= . >=20 > I suggest you rework the title to: > [PATCH] sched: enable traffic class oversubscription conditionally >=20 > > > > Added new API to enable or disable TC over subscription for best > > effort traffic class at subport level. > > Added changes after review and increased throughput. > > > > By default TC OV is disabled. >=20 > It should be the other way around, the TC_OV should be enabled by default= . The > TC oversubscription is a more natural way to use this library, we usually= want to > disable this feature just for better performance in case this functionali= ty is not > needed. Please initialize the tc_ov flag accordingly. > In original code, this feature has always been disabled as it impacts perfo= rmance. So, in my opinion we should keep it disabled by default and let user enable= it when required. =20 > > > > Signed-off-by: Marcin Danilewicz > > --- > > lib/sched/rte_sched.c | 189 > > +++++++++++++++++++++++++++++++++++------- > > lib/sched/rte_sched.h | 18 ++++ > > lib/sched/version.map | 3 + > > 3 files changed, 178 insertions(+), 32 deletions(-) > > > > diff --git a/lib/sched/rte_sched.c b/lib/sched/rte_sched.c index > > ec74bee939..6e7d81df46 100644 > > --- a/lib/sched/rte_sched.c > > +++ b/lib/sched/rte_sched.c > > @@ -213,6 +213,9 @@ struct rte_sched_subport { > > uint8_t *bmp_array; > > struct rte_mbuf **queue_array; > > uint8_t memory[0] __rte_cache_aligned; > > + > > + /* TC oversubscription activation */ > > + int is_tc_ov_enabled; >=20 > How about we simplify the name of this variable to: tc_ov_enabled ? >=20 > > } __rte_cache_aligned; > > > > struct rte_sched_port { > > @@ -1165,6 +1168,45 @@ rte_sched_cman_config(struct rte_sched_port > > *port, } #endif > > > > +int > > +rte_sched_subport_tc_ov_config(struct rte_sched_port *port, > > + uint32_t subport_id, > > + bool tc_ov_enable) > > +{ > > + struct rte_sched_subport *s; > > + struct rte_sched_subport_profile *profile; > > + > > + if (port =3D=3D NULL) { > > + RTE_LOG(ERR, SCHED, > > + "%s: Incorrect value for parameter port\n", __func__); > > + return -EINVAL; > > + } > > + > > + if (subport_id >=3D port->n_subports_per_port) { > > + RTE_LOG(ERR, SCHED, > > + "%s: Incorrect value for parameter subport id\n", > > __func__); > > + return -EINVAL; > > + } > > + > > + s =3D port->subports[subport_id]; > > + s->is_tc_ov_enabled =3D tc_ov_enable ? 1 : 0; > > + > > + if (s->is_tc_ov_enabled) { > > + /* TC oversubscription */ > > + s->tc_ov_wm_min =3D port->mtu; > > + s->tc_ov_period_id =3D 0; > > + s->tc_ov =3D 0; > > + s->tc_ov_n =3D 0; > > + s->tc_ov_rate =3D 0; > > + > > + profile =3D port->subport_profiles + s->profile; > > + s->tc_ov_wm_max =3D rte_sched_time_ms_to_bytes(profile- > > >tc_period, > > + s->pipe_tc_be_rate_max); > > + s->tc_ov_wm =3D s->tc_ov_wm_max; > > + } > > + return 0; > > +} >=20 > This function should not exist, please remove it and keep the initial cod= e that > computes the tc_ov related variable regardless of whether tc_ov is enable= d or > not. >=20 > All the tc_ov related variables have the tc_ov particle in their name, so= there is > no clash. This is initialization code, so no performance overhead. Let's = keep the > code unmodified and compute both the tc_ov and the non-tc_ov varables at > initialization, regardless of whether the feature is enabled or not. >=20 > This comment is applicable to all the initialization code, please adjust = all the init > code accordingly. There should be no diff showing in the patch for any of= the init > code! >=20 > For this file "rte_sched.c", your patch should contain just two additiona= l run- > time functions, i.e. the non-tc-ov version of functions grinder_credits_u= pdate() > and grindler_credits_check(), and the small code required to test when to= use > the tc-ov vs. the non-tc_ov version, makes sense? >=20 > > + > > int > > is_tc_ov_enabled (struct rte_sched_port *port, > > uint32_t subport_id, > > @@ -1254,6 +1296,9 @@ rte_sched_subport_config(struct rte_sched_port > > *port, > > s->n_pipe_profiles =3D params->n_pipe_profiles; > > s->n_max_pipe_profiles =3D params->n_max_pipe_profiles; > > > > + /* TC over-subscription is disabled by default */ > > + s->is_tc_ov_enabled =3D 0; > > + >=20 > By default, this feature should be enabled: > s->is_tc_ov_enabled =3D 1; >=20 > > #ifdef RTE_SCHED_CMAN > > if (params->cman_params !=3D NULL) { > > s->cman_enabled =3D true; > > @@ -1316,13 +1361,6 @@ rte_sched_subport_config(struct rte_sched_port > > *port, > > > > for (i =3D 0; i < RTE_SCHED_PORT_N_GRINDERS; i++) > > s->grinder_base_bmp_pos[i] =3D > > RTE_SCHED_PIPE_INVALID; > > - > > - /* TC oversubscription */ > > - s->tc_ov_wm_min =3D port->mtu; > > - s->tc_ov_period_id =3D 0; > > - s->tc_ov =3D 0; > > - s->tc_ov_n =3D 0; > > - s->tc_ov_rate =3D 0; > > } > > > > { > > @@ -1342,9 +1380,6 @@ rte_sched_subport_config(struct rte_sched_port > > *port, > > else > > profile->tc_credits_per_period[i] =3D 0; > > > > - s->tc_ov_wm_max =3D rte_sched_time_ms_to_bytes(profile- > > >tc_period, > > - s- > > >pipe_tc_be_rate_max); > > - s->tc_ov_wm =3D s->tc_ov_wm_max; > > s->profile =3D subport_profile_id; > > > > } > > @@ -1417,17 +1452,20 @@ rte_sched_pipe_config(struct rte_sched_port > > *port, > > double pipe_tc_be_rate =3D > > (double) params- > > >tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE] > > / (double) params->tc_period; > > - uint32_t tc_be_ov =3D s->tc_ov; > > > > - /* Unplug pipe from its subport */ > > - s->tc_ov_n -=3D params->tc_ov_weight; > > - s->tc_ov_rate -=3D pipe_tc_be_rate; > > - s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; > > + if (s->is_tc_ov_enabled) { > > + uint32_t tc_be_ov =3D s->tc_ov; > > > > - if (s->tc_ov !=3D tc_be_ov) { > > - RTE_LOG(DEBUG, SCHED, > > - "Subport %u Best-effort TC oversubscription is > > OFF (%.4lf >=3D %.4lf)\n", > > - subport_id, subport_tc_be_rate, s- > > >tc_ov_rate); > > + /* Unplug pipe from its subport */ > > + s->tc_ov_n -=3D params->tc_ov_weight; > > + s->tc_ov_rate -=3D pipe_tc_be_rate; > > + s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; > > + > > + if (s->tc_ov !=3D tc_be_ov) { > > + RTE_LOG(DEBUG, SCHED, > > + "Subport %u Best-effort TC > > oversubscription is OFF (%.4lf >=3D %.4lf)\n", > > + subport_id, subport_tc_be_rate, s- > > >tc_ov_rate); > > + } > > } > > > > /* Reset the pipe */ > > @@ -1460,19 +1498,22 @@ rte_sched_pipe_config(struct rte_sched_port > > *port, > > double pipe_tc_be_rate =3D > > (double) params- > > >tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE] > > / (double) params->tc_period; > > - uint32_t tc_be_ov =3D s->tc_ov; > > > > - s->tc_ov_n +=3D params->tc_ov_weight; > > - s->tc_ov_rate +=3D pipe_tc_be_rate; > > - s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; > > + if (s->is_tc_ov_enabled) { > > + uint32_t tc_be_ov =3D s->tc_ov; > > + > > + s->tc_ov_n +=3D params->tc_ov_weight; > > + s->tc_ov_rate +=3D pipe_tc_be_rate; > > + s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; > > > > - if (s->tc_ov !=3D tc_be_ov) { > > - RTE_LOG(DEBUG, SCHED, > > - "Subport %u Best effort TC oversubscription is > > ON (%.4lf < %.4lf)\n", > > - subport_id, subport_tc_be_rate, s- > > >tc_ov_rate); > > + if (s->tc_ov !=3D tc_be_ov) { > > + RTE_LOG(DEBUG, SCHED, > > + "Subport %u Best effort TC > > oversubscription is ON (%.4lf < %.4lf)\n", > > + subport_id, subport_tc_be_rate, s- > > >tc_ov_rate); > > + } > > + p->tc_ov_period_id =3D s->tc_ov_period_id; > > + p->tc_ov_credits =3D s->tc_ov_wm; > > } > > - p->tc_ov_period_id =3D s->tc_ov_period_id; > > - p->tc_ov_credits =3D s->tc_ov_wm; > > } > > > > return 0; > > @@ -2318,6 +2359,45 @@ grinder_credits_update(struct rte_sched_port > > *port, > > pipe->tb_credits =3D RTE_MIN(pipe->tb_credits, params->tb_size); > > pipe->tb_time +=3D n_periods * params->tb_period; > > > > + /* Subport TCs */ > > + if (unlikely(port->time >=3D subport->tc_time)) { > > + for (i =3D 0; i < RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE; i++) > > + subport->tc_credits[i] =3D sp->tc_credits_per_period[i]; > > + > > + subport->tc_time =3D port->time + sp->tc_period; > > + } > > + > > + /* Pipe TCs */ > > + if (unlikely(port->time >=3D pipe->tc_time)) { > > + for (i =3D 0; i < RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE; i++) > > + pipe->tc_credits[i] =3D params->tc_credits_per_period[i]; > > + pipe->tc_time =3D port->time + params->tc_period; > > + } > > +} > > + > > +static inline void > > +grinder_credits_update_with_tc_ov(struct rte_sched_port *port, > > + struct rte_sched_subport *subport, uint32_t pos) { > > + struct rte_sched_grinder *grinder =3D subport->grinder + pos; > > + struct rte_sched_pipe *pipe =3D grinder->pipe; > > + struct rte_sched_pipe_profile *params =3D grinder->pipe_params; > > + struct rte_sched_subport_profile *sp =3D grinder->subport_params; > > + uint64_t n_periods; > > + uint32_t i; > > + > > + /* Subport TB */ > > + n_periods =3D (port->time - subport->tb_time) / sp->tb_period; > > + subport->tb_credits +=3D n_periods * sp->tb_credits_per_period; > > + subport->tb_credits =3D RTE_MIN(subport->tb_credits, sp->tb_size); > > + subport->tb_time +=3D n_periods * sp->tb_period; > > + > > + /* Pipe TB */ > > + n_periods =3D (port->time - pipe->tb_time) / params->tb_period; > > + pipe->tb_credits +=3D n_periods * params->tb_credits_per_period; > > + pipe->tb_credits =3D RTE_MIN(pipe->tb_credits, params->tb_size); > > + pipe->tb_time +=3D n_periods * params->tb_period; > > + > > /* Subport TCs */ > > if (unlikely(port->time >=3D subport->tc_time)) { > > subport->tc_ov_wm =3D > > @@ -2348,6 +2428,39 @@ grinder_credits_update(struct rte_sched_port > > *port, static inline int grinder_credits_check(struct rte_sched_port > > *port, > > struct rte_sched_subport *subport, uint32_t pos) > > +{ > > + struct rte_sched_grinder *grinder =3D subport->grinder + pos; > > + struct rte_sched_pipe *pipe =3D grinder->pipe; > > + struct rte_mbuf *pkt =3D grinder->pkt; > > + uint32_t tc_index =3D grinder->tc_index; > > + uint64_t pkt_len =3D pkt->pkt_len + port->frame_overhead; > > + uint64_t subport_tb_credits =3D subport->tb_credits; > > + uint64_t subport_tc_credits =3D subport->tc_credits[tc_index]; > > + uint64_t pipe_tb_credits =3D pipe->tb_credits; > > + uint64_t pipe_tc_credits =3D pipe->tc_credits[tc_index]; > > + int enough_credits; > > + > > + /* Check pipe and subport credits */ > > + enough_credits =3D (pkt_len <=3D subport_tb_credits) && > > + (pkt_len <=3D subport_tc_credits) && > > + (pkt_len <=3D pipe_tb_credits) && > > + (pkt_len <=3D pipe_tc_credits); > > + > > + if (!enough_credits) > > + return 0; > > + > > + /* Update pipe and subport credits */ > > + subport->tb_credits -=3D pkt_len; > > + subport->tc_credits[tc_index] -=3D pkt_len; > > + pipe->tb_credits -=3D pkt_len; > > + pipe->tc_credits[tc_index] -=3D pkt_len; > > + > > + return 1; > > +} > > + > > +static inline int > > +grinder_credits_check_with_tc_ov(struct rte_sched_port *port, > > + struct rte_sched_subport *subport, uint32_t pos) > > { > > struct rte_sched_grinder *grinder =3D subport->grinder + pos; > > struct rte_sched_pipe *pipe =3D grinder->pipe; @@ -2403,8 +2516,16 @@ > > grinder_schedule(struct rte_sched_port *port, > > uint32_t pkt_len =3D pkt->pkt_len + port->frame_overhead; > > uint32_t be_tc_active; > > > > - if (!grinder_credits_check(port, subport, pos)) > > - return 0; > > + switch (subport->is_tc_ov_enabled) { > > + case 1: > > + if (!grinder_credits_check_with_tc_ov(port, subport, pos)) > > + return 0; > > + break; > > + case 0: > > + if (!grinder_credits_check(port, subport, pos)) > > + return 0; > > + break; > > + } >=20 > There should be no switch statement here, please replace with an if state= ment. I > suggest the following: >=20 > int status; >=20 > status =3D subport->tc_ov_enabled ? grinder_credits_check_with_tc_ov(port= , > subport, pos) : grinder_credits_check(port, subport, pos); if (!status) > return 0; >=20 > > > > /* Advance port time */ > > port->time +=3D pkt_len; > > @@ -2770,7 +2891,11 @@ grinder_handle(struct rte_sched_port *port, > > subport->profile; > > > > grinder_prefetch_tc_queue_arrays(subport, pos); > > - grinder_credits_update(port, subport, pos); > > + > > + if (unlikely(subport->is_tc_ov_enabled)) >=20 > Please remove the "unlikely" from here, don't put any likely/unlikely her= e at all. >=20 > > + grinder_credits_update_with_tc_ov(port, subport, pos); > > + else > > + grinder_credits_update(port, subport, pos); > > > > grinder->state =3D e_GRINDER_PREFETCH_MBUF; > > return 0; > > diff --git a/lib/sched/rte_sched.h b/lib/sched/rte_sched.h index > > 5ece64e527..94febe1d94 100644 > > --- a/lib/sched/rte_sched.h > > +++ b/lib/sched/rte_sched.h > > @@ -579,6 +579,24 @@ rte_sched_port_enqueue(struct rte_sched_port > > *port, struct rte_mbuf **pkts, uint int > > rte_sched_port_dequeue(struct rte_sched_port *port, struct rte_mbuf > > **pkts, uint32_t n_pkts); > > > > +/** > > + * Hierarchical scheduler subport TC OV enable/disable config. >=20 > The name of the feature should be fully stated here: traffic class > oversubscription, not the abbreviation, please change. >=20 >=20 > > + * Note that this function is safe to use at runtime > > + * to enable/disable TC OV for subport. >=20 > We should actually forbit this rather than encourage it. Calling this fun= ction > several times does not make sense, and it can create limitations that can= come > back and byte us in the future, whenever we might need to extend this cod= e, for > no reason. >=20 > Please actually replace with: "This function should be called at the time= of > subport initialization." >=20 > > + * > > + * @param port > > + * Handle to port scheduler instance > > + * @param subport_id > > + * Subport ID > > + * @param tc_ov_enable > > + * Boolean flag to enable/disable TC OV > > + * @return > > + * 0 upon success, error code otherwise > > + */ > > +__rte_experimental > > +int > > +rte_sched_subport_tc_ov_config(struct rte_sched_port *port, uint32_t > > subport_id, bool tc_ov_enable); > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/sched/version.map b/lib/sched/version.map index > > d22c07fc9f..c6e994d8df 100644 > > --- a/lib/sched/version.map > > +++ b/lib/sched/version.map > > @@ -34,4 +34,7 @@ EXPERIMENTAL { > > # added in 21.11 > > rte_pie_rt_data_init; > > rte_pie_config_init; > > + > > + # added in 22.03 >=20 > This is not in 22.03, it will hopefully be in 22.07. >=20 > > + rte_sched_subport_tc_ov_config; > > }; > > -- > > 2.25.1 >=20 > Regards, > Cristian