From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2BBDFA034C; Mon, 9 May 2022 22:05:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C75E341156; Mon, 9 May 2022 22:05:39 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id A2659410EE for ; Mon, 9 May 2022 22:05:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652126737; x=1683662737; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=EI7uMAsfr00YBrb40o3YKVCztcKX8wyGltNSBvYaE2Y=; b=YrXkLxjWKRSX3cOXt2O8NKgKZq1IxxKgrQbR1thR1xcDj1KXJNTV0H6/ gY/D1nMPA0n0/MErj/PxOwuKvDj4heaOOhCj1VlxtbbrZeG76ryKhWJ0R EUW7QGhFwY1DoMO+bxH8UazrnoUXQDEaxYb/bVDlwnjc37tiWyHHvdsLw 5eJSx6adVQT5AXZI9b4nMwMEfm0iB3mrgebDZUPEQjerJ/4l0H8ncgviZ xatu6ikxvuEJBJEtPzuvepIX/Knt2nvmpn5UwHX6W0I2G3Bhsu92i8I7P JeRUTS1wGOIRHl0dsiHXRBKmGqD9TjEjY0dj53rha3F8l3uJCl5qKxzyI g==; X-IronPort-AV: E=McAfee;i="6400,9594,10342"; a="269295193" X-IronPort-AV: E=Sophos;i="5.91,212,1647327600"; d="scan'208";a="269295193" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2022 13:05:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,212,1647327600"; d="scan'208";a="551186836" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by orsmga002.jf.intel.com with ESMTP; 09 May 2022 13:05:34 -0700 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Mon, 9 May 2022 13:05:34 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Mon, 9 May 2022 13:05:34 -0700 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (104.47.74.44) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Mon, 9 May 2022 13:05:34 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mlCCVMjmfRMLkocj8PcJ1T9EVSCShwS3GC/YXZJTAePSqrh5qxKtnDgQYLOpLPB5wgpu2okrgl+2plrlTbxdkktOOzp4GdiC0T2br7BIX19pEN4uwu+Ukj/ICiAYm56cR9AmFzkGj0nwg4Ls2eJNm4aGHc8+hjE8VwTZZfJdoFR+PsoAJFnb5TAbENrAH3Y87jonUpGzArZ9Ja8RcEe3D9cHmpr32JCvUqc/lMn094uYThzuqHPLz4l//5r6rwmQ0sc9IQcpv0rvIcvDjkwhrAkzLAvlzsSAVSYXkTdhN88SgHUFslSdzXRKTM3WzpfBpoRUI33Edvs2y304LPDUzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=V40RHlJ7pfjOvtFez2l4QexPGRGwo20+jV6IwIyc+Nw=; b=bLYiTUP07aKhjgW1MATCitd6c774j5xsHt2RY+MboEgqpjvPzLU+XStwQN0ITJ0iqLNstIK+R2+Hp2HGWDlaOQyIF6NZY7Vs+VdGKYlNF8BiUnKQLXyh1RMq783puCKriDLAlEtoYJckh7dtE6xmHnzT7tIMLXulwqJba/UDoG1S9lRKaFy/GvnsnRF8LXharf7V06lMZ7AlYr69YBJPr8yPgjWnJi+ckmxYUB//glEKKALHYsaU/gOmY71tF9x3RdbnlbWgliy2mIkorE3jwbFPtwA+TQAOb658JYP2s3wBdnSZYPPTkglvxqrcE10SmduVCjZs10j5U+VwEKs3pg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM8PR11MB5670.namprd11.prod.outlook.com (2603:10b6:8:37::12) by BY5PR11MB4085.namprd11.prod.outlook.com (2603:10b6:a03:18d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5227.20; Mon, 9 May 2022 20:05:31 +0000 Received: from DM8PR11MB5670.namprd11.prod.outlook.com ([fe80::5dc9:53d7:3ece:fa2d]) by DM8PR11MB5670.namprd11.prod.outlook.com ([fe80::5dc9:53d7:3ece:fa2d%2]) with mapi id 15.20.5227.023; Mon, 9 May 2022 20:05:31 +0000 From: "Dumitrescu, Cristian" To: "Danilewicz, MarcinX" , "dev@dpdk.org" , "Singh, Jasvinder" CC: "Ajmera, Megha" Subject: RE: [PATCH v3] sched: enable/disable TC OV at runtime Thread-Topic: [PATCH v3] sched: enable/disable TC OV at runtime Thread-Index: AQHYWhiOjMuBu5wHJkOvKczZqJ0XNK0XAoTQ Date: Mon, 9 May 2022 20:05:31 +0000 Message-ID: References: <20220407145153.238969-2-marcinx.danilewicz@intel.com> <20220427092357.491720-1-marcinx.danilewicz@intel.com> In-Reply-To: <20220427092357.491720-1-marcinx.danilewicz@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.6.401.20 dlp-reaction: no-action dlp-product: dlpe-windows authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bdb03357-9dd5-4d52-de46-08da31f74512 x-ms-traffictypediagnostic: BY5PR11MB4085:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 6zaHw0ezHIgJX4eZFxxMxQOU09ClNZtW4MuGR8UZMpx6pbpyq4pbj5eahtq3WBClJCb7r1VN+vqkkp3WxI+qXasHAVE5DS9qW7dvrJwc4hLOAYiiegbaq08CAUhV3cjUTUKGbRmRcoQbz17KCbnOpN98ABqM68raYkxge/PTlDDhdUuxuhghNhGpJZa/hyE8g/5J+7T1f+93BgZJZIvMWpyumdVSS4tDGO5RwcayCPvmhw9qv7weFikfGCI8J1Cz0k9ozOTQxvzzudHOabDwKBeEpB/aiwULXeA4ZED5nYGHtkcN0LoQG+89lrykpu8h2P1USy0qe5lXPsF1+TorKimGQpoUI5bQ1/jVgkMBXv1wJyPoAaOdh355Tf9DdTWhCjTFzCrHSrYoe2b8kdAAcTDDDhreUC+ct/SEBu0+0DVKsM0ahT+7CkjswRRJyjwbru0BWSdBevInwlfUtz67dsgjSAFUaKkrMzjfi+4Xjk2c2q2LcNqISRYIxIE6txdWqEbzM1HW9QtHqPFaYYclBrQkyGdbbIIhm0tbsgVeulimKxGK+GMLR1rRIrmUPdbaIoN5kPHB7FR85rQlV/jMb7bBPrcuZZbjJaK9RpgiJBnO4DCs88Sn2Ioll5FTFDAcgs8Lmj/qMyZQDhZT4ACaekjwWvr3JduOS+Wmtwu+kJ/C/mziTHmwljrKeBQTVPjCljdEVulYF9WueoniBMXwrg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM8PR11MB5670.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(64756008)(8676002)(66946007)(66556008)(4326008)(6636002)(76116006)(2906002)(66476007)(66446008)(82960400001)(38100700002)(316002)(86362001)(508600001)(38070700005)(71200400001)(122000001)(83380400001)(8936002)(55016003)(30864003)(107886003)(5660300002)(9686003)(52536014)(26005)(186003)(53546011)(7696005)(6506007)(33656002)(110136005); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?AQ/jUEbq+VISxHRVjEGDK6jt2MLw9hnbWBxkWWOkzLiIknd7j9LDHEHEuz4O?= =?us-ascii?Q?g7eSSF4QHSBF5idIxnIsPLqn/OR1+43OyhDXfhierdeX+cU7p502t+2bbgn/?= =?us-ascii?Q?xGqJA9hzkRFjHfSFBIhSPJ8i0VefqMGy7EyaLti1GmQCfOYx/QADlZQyOglO?= =?us-ascii?Q?OFEiL3RDJK7cI2esKkXJhywLw4vzQlzaHgFTmLKZ7NZnRv1bj4/04empgB7t?= =?us-ascii?Q?H08X+2HAbEjBJJKsXoHMuAR26ZVetzjL6yen2rFc2k41Xcg9jDGKu2v05NGc?= =?us-ascii?Q?QDF3/091OGLZ5OC5xpTmwiVZ6J0Dx5FPS360t5XumVh4yVgHXAouIgg0n5xQ?= =?us-ascii?Q?DwFnevVNj9i9M0YSKD4WqUE2mnAmXYQVDsWNk4r6kEWYL6TVuuzc6sJ2MOan?= =?us-ascii?Q?dP36SX+XHAVTEcTg+g6m8gxOcoh8RUGC92M+sUOdKwN3qOiFsr+my10PtS1/?= =?us-ascii?Q?D4jUoq3gayz6257zzGReAhohahOUbCLs6Bi2qwX7tM0YXo2kaK27af6s0kkA?= =?us-ascii?Q?j1B70hldj+YsLIqdyoU23Su7tJ8wgpBT/04v14z7/87voSE0K9+wmbPscFEq?= =?us-ascii?Q?lhwvsxiwKave2ioTHZNDB2EnHuBsjX/bc+ls/cl/r4o4A8Uy2kRO4rMJpp4O?= =?us-ascii?Q?VEFUoqm//HTqD3U22ANMVJQw9LpGOezD/HIrJNypXmfDuRwqyEoGKIWlxzqS?= =?us-ascii?Q?t2i6zpW45KkBJIWACU7UozoT9B0+zhqLH7iUpPRAv+X58CqzC47mNebTmqyG?= =?us-ascii?Q?MLgdZ3ZZHOk+mGnX48d/bs1ZJb2jji50trXT68cekZxTjNTcGDPDtHHlXJ1c?= =?us-ascii?Q?h61VzhkYXjGbE7TvQPBI7kmkeLL9TfTW/AAavh9+lkT69xtqDcU1uRjZFReg?= =?us-ascii?Q?/00A4mXzWXLsFgJlWZZd0bxStTRcHCprdRS7oFwDsrexqZWhUHuD7NoRH11l?= =?us-ascii?Q?31Kas7wVvcBv51sFCt2CpLBdNXxPYYaPD0n2/It87IC96lXnKQAngSTRSseA?= =?us-ascii?Q?oy4029HrhUswPG2OlxeDyK5pVh+aMfcL2DBPpHWHjDtQFhGXmDckeoI1u4jc?= =?us-ascii?Q?uyNTKayfG0rBOFweM+QQ3hBTEgdrKiib0RrjSqJbhz3bvVpSHMpbvEHauV3l?= =?us-ascii?Q?8gJvyxHI3FJyL2ZGOGj++IWt0v75gk/Df1UEtYuPBugWmsa4gwcCb6LXQBXe?= =?us-ascii?Q?bkwKQmSFphCfJegiVsNTJ7IT+L9i0XqvvTmnPHizRuvQyDebz3Fp9WCcAlYW?= =?us-ascii?Q?CRfxEXkOIuUtkY6/MEnf7lS81KiU2rYaWO27XVGuCz5UbVvF/QrvJtpH/jLC?= =?us-ascii?Q?x5b90dm326s8v/trWO0if8YM6D4GSeHjLo8e33BAKpl+wXGv0E3W3r9S8cD+?= =?us-ascii?Q?7W2nA8pMnwOXw4re1qTS1C3h397CJ3aOQn41nnv72FSF1WxL/vSXC9IasZjn?= =?us-ascii?Q?KVI0eJB6ERShp9z1LRYuks/7UTRPllJw/ICWZnfY4d3f5x/b/v/yR+rCoJ1j?= =?us-ascii?Q?7wcVOdvd7KFJcEL9oNAetN0zTwXxznwf1TDWHvEsw2l+b+LmTUXbB94prwx9?= =?us-ascii?Q?Pw24TugCcrcfUaMjXOYPb+DhU9p8g5Mb/ej+9686XNegf820wcX4nHpeDgho?= =?us-ascii?Q?GcKYkEBNXtst/lC9spQCK9DIKuyhLxjcKaUdP1aWSz51dk6akhiuxU1csyyU?= =?us-ascii?Q?Qj8vt0UMO3b17wo1UE3QTnHXRLw5rxIEt3hMNClTRfIxhnCO2h/KB3hEbm60?= =?us-ascii?Q?YbZRjDxls6kjRYq32eg4QHYt8Zs5YrM=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM8PR11MB5670.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bdb03357-9dd5-4d52-de46-08da31f74512 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 May 2022 20:05:31.1269 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rN/u0mda20dVaVOBTXymKds2k+JCUuiumXtSlqWz25KZjfWkdy3z2wWaF5yzcW2F9NZE4I6SktjTiO6KSdRpZplbhEFJybUQ9/PhNo0/9Yk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB4085 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hi Marcin, > -----Original Message----- > From: Danilewicz, MarcinX > Sent: Wednesday, April 27, 2022 10:24 AM > To: dev@dpdk.org; Singh, Jasvinder ; Dumitresc= u, > Cristian > Cc: Ajmera, Megha > Subject: [PATCH v3] sched: enable/disable TC OV at runtime We are not trying to enable/disable the traffic class oversubscription feat= ure at run-time, but at initialization. If cat, we should prohibit changing= this post-initialization. Also the name of the feature should not be abbreviated in the patch title. I suggest you rework the title to: [PATCH] sched: enable traffic class oversubscription conditionally >=20 > Added new API to enable or disable TC over subscription for best > effort traffic class at subport level. > Added changes after review and increased throughput. >=20 > By default TC OV is disabled. It should be the other way around, the TC_OV should be enabled by default. = The TC oversubscription is a more natural way to use this library, we usual= ly want to disable this feature just for better performance in case this fu= nctionality is not needed. Please initialize the tc_ov flag accordingly. >=20 > Signed-off-by: Marcin Danilewicz > --- > lib/sched/rte_sched.c | 189 +++++++++++++++++++++++++++++++++++------- > lib/sched/rte_sched.h | 18 ++++ > lib/sched/version.map | 3 + > 3 files changed, 178 insertions(+), 32 deletions(-) >=20 > diff --git a/lib/sched/rte_sched.c b/lib/sched/rte_sched.c > index ec74bee939..6e7d81df46 100644 > --- a/lib/sched/rte_sched.c > +++ b/lib/sched/rte_sched.c > @@ -213,6 +213,9 @@ struct rte_sched_subport { > uint8_t *bmp_array; > struct rte_mbuf **queue_array; > uint8_t memory[0] __rte_cache_aligned; > + > + /* TC oversubscription activation */ > + int is_tc_ov_enabled; How about we simplify the name of this variable to: tc_ov_enabled ? > } __rte_cache_aligned; >=20 > struct rte_sched_port { > @@ -1165,6 +1168,45 @@ rte_sched_cman_config(struct rte_sched_port > *port, > } > #endif >=20 > +int > +rte_sched_subport_tc_ov_config(struct rte_sched_port *port, > + uint32_t subport_id, > + bool tc_ov_enable) > +{ > + struct rte_sched_subport *s; > + struct rte_sched_subport_profile *profile; > + > + if (port =3D=3D NULL) { > + RTE_LOG(ERR, SCHED, > + "%s: Incorrect value for parameter port\n", __func__); > + return -EINVAL; > + } > + > + if (subport_id >=3D port->n_subports_per_port) { > + RTE_LOG(ERR, SCHED, > + "%s: Incorrect value for parameter subport id\n", > __func__); > + return -EINVAL; > + } > + > + s =3D port->subports[subport_id]; > + s->is_tc_ov_enabled =3D tc_ov_enable ? 1 : 0; > + > + if (s->is_tc_ov_enabled) { > + /* TC oversubscription */ > + s->tc_ov_wm_min =3D port->mtu; > + s->tc_ov_period_id =3D 0; > + s->tc_ov =3D 0; > + s->tc_ov_n =3D 0; > + s->tc_ov_rate =3D 0; > + > + profile =3D port->subport_profiles + s->profile; > + s->tc_ov_wm_max =3D rte_sched_time_ms_to_bytes(profile- > >tc_period, > + s->pipe_tc_be_rate_max); > + s->tc_ov_wm =3D s->tc_ov_wm_max; > + } > + return 0; > +} This function should not exist, please remove it and keep the initial code = that computes the tc_ov related variable regardless of whether tc_ov is ena= bled or not. All the tc_ov related variables have the tc_ov particle in their name, so t= here is no clash. This is initialization code, so no performance overhead. = Let's keep the code unmodified and compute both the tc_ov and the non-tc_ov= varables at initialization, regardless of whether the feature is enabled o= r not. This comment is applicable to all the initialization code, please adjust al= l the init code accordingly. There should be no diff showing in the patch f= or any of the init code! For this file "rte_sched.c", your patch should contain just two additional = run-time functions, i.e. the non-tc-ov version of functions grinder_credits= _update() and grindler_credits_check(), and the small code required to test= when to use the tc-ov vs. the non-tc_ov version, makes sense? > + > int > is_tc_ov_enabled (struct rte_sched_port *port, > uint32_t subport_id, > @@ -1254,6 +1296,9 @@ rte_sched_subport_config(struct rte_sched_port > *port, > s->n_pipe_profiles =3D params->n_pipe_profiles; > s->n_max_pipe_profiles =3D params->n_max_pipe_profiles; >=20 > + /* TC over-subscription is disabled by default */ > + s->is_tc_ov_enabled =3D 0; > + By default, this feature should be enabled: s->is_tc_ov_enabled =3D 1; > #ifdef RTE_SCHED_CMAN > if (params->cman_params !=3D NULL) { > s->cman_enabled =3D true; > @@ -1316,13 +1361,6 @@ rte_sched_subport_config(struct rte_sched_port > *port, >=20 > for (i =3D 0; i < RTE_SCHED_PORT_N_GRINDERS; i++) > s->grinder_base_bmp_pos[i] =3D > RTE_SCHED_PIPE_INVALID; > - > - /* TC oversubscription */ > - s->tc_ov_wm_min =3D port->mtu; > - s->tc_ov_period_id =3D 0; > - s->tc_ov =3D 0; > - s->tc_ov_n =3D 0; > - s->tc_ov_rate =3D 0; > } >=20 > { > @@ -1342,9 +1380,6 @@ rte_sched_subport_config(struct rte_sched_port > *port, > else > profile->tc_credits_per_period[i] =3D 0; >=20 > - s->tc_ov_wm_max =3D rte_sched_time_ms_to_bytes(profile- > >tc_period, > - s- > >pipe_tc_be_rate_max); > - s->tc_ov_wm =3D s->tc_ov_wm_max; > s->profile =3D subport_profile_id; >=20 > } > @@ -1417,17 +1452,20 @@ rte_sched_pipe_config(struct rte_sched_port > *port, > double pipe_tc_be_rate =3D > (double) params- > >tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE] > / (double) params->tc_period; > - uint32_t tc_be_ov =3D s->tc_ov; >=20 > - /* Unplug pipe from its subport */ > - s->tc_ov_n -=3D params->tc_ov_weight; > - s->tc_ov_rate -=3D pipe_tc_be_rate; > - s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; > + if (s->is_tc_ov_enabled) { > + uint32_t tc_be_ov =3D s->tc_ov; >=20 > - if (s->tc_ov !=3D tc_be_ov) { > - RTE_LOG(DEBUG, SCHED, > - "Subport %u Best-effort TC oversubscription is > OFF (%.4lf >=3D %.4lf)\n", > - subport_id, subport_tc_be_rate, s- > >tc_ov_rate); > + /* Unplug pipe from its subport */ > + s->tc_ov_n -=3D params->tc_ov_weight; > + s->tc_ov_rate -=3D pipe_tc_be_rate; > + s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; > + > + if (s->tc_ov !=3D tc_be_ov) { > + RTE_LOG(DEBUG, SCHED, > + "Subport %u Best-effort TC > oversubscription is OFF (%.4lf >=3D %.4lf)\n", > + subport_id, subport_tc_be_rate, s- > >tc_ov_rate); > + } > } >=20 > /* Reset the pipe */ > @@ -1460,19 +1498,22 @@ rte_sched_pipe_config(struct rte_sched_port > *port, > double pipe_tc_be_rate =3D > (double) params- > >tc_credits_per_period[RTE_SCHED_TRAFFIC_CLASS_BE] > / (double) params->tc_period; > - uint32_t tc_be_ov =3D s->tc_ov; >=20 > - s->tc_ov_n +=3D params->tc_ov_weight; > - s->tc_ov_rate +=3D pipe_tc_be_rate; > - s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; > + if (s->is_tc_ov_enabled) { > + uint32_t tc_be_ov =3D s->tc_ov; > + > + s->tc_ov_n +=3D params->tc_ov_weight; > + s->tc_ov_rate +=3D pipe_tc_be_rate; > + s->tc_ov =3D s->tc_ov_rate > subport_tc_be_rate; >=20 > - if (s->tc_ov !=3D tc_be_ov) { > - RTE_LOG(DEBUG, SCHED, > - "Subport %u Best effort TC oversubscription is > ON (%.4lf < %.4lf)\n", > - subport_id, subport_tc_be_rate, s- > >tc_ov_rate); > + if (s->tc_ov !=3D tc_be_ov) { > + RTE_LOG(DEBUG, SCHED, > + "Subport %u Best effort TC > oversubscription is ON (%.4lf < %.4lf)\n", > + subport_id, subport_tc_be_rate, s- > >tc_ov_rate); > + } > + p->tc_ov_period_id =3D s->tc_ov_period_id; > + p->tc_ov_credits =3D s->tc_ov_wm; > } > - p->tc_ov_period_id =3D s->tc_ov_period_id; > - p->tc_ov_credits =3D s->tc_ov_wm; > } >=20 > return 0; > @@ -2318,6 +2359,45 @@ grinder_credits_update(struct rte_sched_port > *port, > pipe->tb_credits =3D RTE_MIN(pipe->tb_credits, params->tb_size); > pipe->tb_time +=3D n_periods * params->tb_period; >=20 > + /* Subport TCs */ > + if (unlikely(port->time >=3D subport->tc_time)) { > + for (i =3D 0; i < RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE; i++) > + subport->tc_credits[i] =3D sp->tc_credits_per_period[i]; > + > + subport->tc_time =3D port->time + sp->tc_period; > + } > + > + /* Pipe TCs */ > + if (unlikely(port->time >=3D pipe->tc_time)) { > + for (i =3D 0; i < RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE; i++) > + pipe->tc_credits[i] =3D params->tc_credits_per_period[i]; > + pipe->tc_time =3D port->time + params->tc_period; > + } > +} > + > +static inline void > +grinder_credits_update_with_tc_ov(struct rte_sched_port *port, > + struct rte_sched_subport *subport, uint32_t pos) > +{ > + struct rte_sched_grinder *grinder =3D subport->grinder + pos; > + struct rte_sched_pipe *pipe =3D grinder->pipe; > + struct rte_sched_pipe_profile *params =3D grinder->pipe_params; > + struct rte_sched_subport_profile *sp =3D grinder->subport_params; > + uint64_t n_periods; > + uint32_t i; > + > + /* Subport TB */ > + n_periods =3D (port->time - subport->tb_time) / sp->tb_period; > + subport->tb_credits +=3D n_periods * sp->tb_credits_per_period; > + subport->tb_credits =3D RTE_MIN(subport->tb_credits, sp->tb_size); > + subport->tb_time +=3D n_periods * sp->tb_period; > + > + /* Pipe TB */ > + n_periods =3D (port->time - pipe->tb_time) / params->tb_period; > + pipe->tb_credits +=3D n_periods * params->tb_credits_per_period; > + pipe->tb_credits =3D RTE_MIN(pipe->tb_credits, params->tb_size); > + pipe->tb_time +=3D n_periods * params->tb_period; > + > /* Subport TCs */ > if (unlikely(port->time >=3D subport->tc_time)) { > subport->tc_ov_wm =3D > @@ -2348,6 +2428,39 @@ grinder_credits_update(struct rte_sched_port > *port, > static inline int > grinder_credits_check(struct rte_sched_port *port, > struct rte_sched_subport *subport, uint32_t pos) > +{ > + struct rte_sched_grinder *grinder =3D subport->grinder + pos; > + struct rte_sched_pipe *pipe =3D grinder->pipe; > + struct rte_mbuf *pkt =3D grinder->pkt; > + uint32_t tc_index =3D grinder->tc_index; > + uint64_t pkt_len =3D pkt->pkt_len + port->frame_overhead; > + uint64_t subport_tb_credits =3D subport->tb_credits; > + uint64_t subport_tc_credits =3D subport->tc_credits[tc_index]; > + uint64_t pipe_tb_credits =3D pipe->tb_credits; > + uint64_t pipe_tc_credits =3D pipe->tc_credits[tc_index]; > + int enough_credits; > + > + /* Check pipe and subport credits */ > + enough_credits =3D (pkt_len <=3D subport_tb_credits) && > + (pkt_len <=3D subport_tc_credits) && > + (pkt_len <=3D pipe_tb_credits) && > + (pkt_len <=3D pipe_tc_credits); > + > + if (!enough_credits) > + return 0; > + > + /* Update pipe and subport credits */ > + subport->tb_credits -=3D pkt_len; > + subport->tc_credits[tc_index] -=3D pkt_len; > + pipe->tb_credits -=3D pkt_len; > + pipe->tc_credits[tc_index] -=3D pkt_len; > + > + return 1; > +} > + > +static inline int > +grinder_credits_check_with_tc_ov(struct rte_sched_port *port, > + struct rte_sched_subport *subport, uint32_t pos) > { > struct rte_sched_grinder *grinder =3D subport->grinder + pos; > struct rte_sched_pipe *pipe =3D grinder->pipe; > @@ -2403,8 +2516,16 @@ grinder_schedule(struct rte_sched_port *port, > uint32_t pkt_len =3D pkt->pkt_len + port->frame_overhead; > uint32_t be_tc_active; >=20 > - if (!grinder_credits_check(port, subport, pos)) > - return 0; > + switch (subport->is_tc_ov_enabled) { > + case 1: > + if (!grinder_credits_check_with_tc_ov(port, subport, pos)) > + return 0; > + break; > + case 0: > + if (!grinder_credits_check(port, subport, pos)) > + return 0; > + break; > + } There should be no switch statement here, please replace with an if stateme= nt. I suggest the following: int status; status =3D subport->tc_ov_enabled ? grinder_credits_check_with_tc_ov(port, = subport, pos) : grinder_credits_check(port, subport, pos); if (!status) return 0; >=20 > /* Advance port time */ > port->time +=3D pkt_len; > @@ -2770,7 +2891,11 @@ grinder_handle(struct rte_sched_port *port, > subport->profile; >=20 > grinder_prefetch_tc_queue_arrays(subport, pos); > - grinder_credits_update(port, subport, pos); > + > + if (unlikely(subport->is_tc_ov_enabled)) Please remove the "unlikely" from here, don't put any likely/unlikely here = at all. > + grinder_credits_update_with_tc_ov(port, subport, pos); > + else > + grinder_credits_update(port, subport, pos); >=20 > grinder->state =3D e_GRINDER_PREFETCH_MBUF; > return 0; > diff --git a/lib/sched/rte_sched.h b/lib/sched/rte_sched.h > index 5ece64e527..94febe1d94 100644 > --- a/lib/sched/rte_sched.h > +++ b/lib/sched/rte_sched.h > @@ -579,6 +579,24 @@ rte_sched_port_enqueue(struct rte_sched_port *port, > struct rte_mbuf **pkts, uint > int > rte_sched_port_dequeue(struct rte_sched_port *port, struct rte_mbuf **pk= ts, > uint32_t n_pkts); >=20 > +/** > + * Hierarchical scheduler subport TC OV enable/disable config. The name of the feature should be fully stated here: traffic class oversubs= cription, not the abbreviation, please change. > + * Note that this function is safe to use at runtime > + * to enable/disable TC OV for subport. We should actually forbit this rather than encourage it. Calling this funct= ion several times does not make sense, and it can create limitations that c= an come back and byte us in the future, whenever we might need to extend th= is code, for no reason. Please actually replace with: "This function should be called at the time o= f subport initialization." > + * > + * @param port > + * Handle to port scheduler instance > + * @param subport_id > + * Subport ID > + * @param tc_ov_enable > + * Boolean flag to enable/disable TC OV > + * @return > + * 0 upon success, error code otherwise > + */ > +__rte_experimental > +int > +rte_sched_subport_tc_ov_config(struct rte_sched_port *port, uint32_t > subport_id, bool tc_ov_enable); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/sched/version.map b/lib/sched/version.map > index d22c07fc9f..c6e994d8df 100644 > --- a/lib/sched/version.map > +++ b/lib/sched/version.map > @@ -34,4 +34,7 @@ EXPERIMENTAL { > # added in 21.11 > rte_pie_rt_data_init; > rte_pie_config_init; > + > + # added in 22.03 This is not in 22.03, it will hopefully be in 22.07. > + rte_sched_subport_tc_ov_config; > }; > -- > 2.25.1 Regards, Cristian