From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id B07531B501 for ; Fri, 30 Nov 2018 15:14:36 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 30 Nov 2018 06:14:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,298,1539673200"; d="scan'208";a="293136604" Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159]) by fmsmga006.fm.intel.com with ESMTP; 30 Nov 2018 06:14:34 -0800 Received: from irsmsx108.ger.corp.intel.com ([169.254.11.101]) by IRSMSX104.ger.corp.intel.com ([169.254.5.131]) with mapi id 14.03.0415.000; Fri, 30 Nov 2018 14:14:33 +0000 From: "Dumitrescu, Cristian" To: dev CC: "Singh, Jasvinder" Thread-Topic: [RFC] sched: new features and improvements Thread-Index: AdSIn3rXEPsv1Q/OTTC9ZiWpSxa1Sg== Date: Fri, 30 Nov 2018 14:14:32 +0000 Message-ID: <3EB4FA525960D640B5BDFFD6A3D891268E802222@IRSMSX108.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNzU4YjU2NTQtZWQyMy00NDRiLTg1MjYtODk0MTc3N2M1NjkxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiTFk1QVVMYlNZTlwvajJyVldKQkZtTER6eDZLaVYrYkhZR0NTNktCRWZGRjBqM1pmV1pZcDU2Sm5selU1eFVrV0oifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [dpdk-dev] [RFC] sched: new features and improvements X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Nov 2018 14:14:37 -0000 Hi guys, Here is a list of incremental features and improvements we are considering = to prototype and add to the DPDK hierarchical scheduler SW library. This li= st is driven by our findings as well as feedback from various users. Please= take a look and feel free to add more features to this list or comment on = the features below. Of course, all these items are subject to preserving th= e functional correctness, existing accuracy and performance of the current = implementation. 1. Pipe level: Increase number of traffic classes (TCs). Allow a more flexi= ble mapping of the pipe queues to traffic classes. Do not allocate memory f= or queues that are not needed. a) Currently, each pipe has 16 queues that are hardwired into 4 TCs schedul= ed with strict priority (SP), and each TC has exactly with 4 queues that a= re scheduled with Weighted Fair Queuing (WFQ). Specifically, TC0 =3D [Queu= e 0 .. Queue 3], TC1 =3D [Queue 4 .. Queue 7], TC2 =3D [Queue 8 .. Queue 11= ], TC3 =3D [Queue 12 .. Queue 15]. b) The plan is to support up to 16 TCs. All the high priority TCs (TC1, TC2= , ...) will have exactly 1 queue, while the lowest priority TC, called Best= Effort (BE), has 1, 4 or 8 queues. This is justified by the fact that typi= cally all the high priority TCs are fully provisioned (small to medium traf= fic rates), while most of the traffic fits into the BE class, which is usua= lly greatly oversubscribed.=20 c) This leads to the following valid options for mapping pipe queues to TCs= : i. BE class has 1 queue =3D> Max number of TCs is 16 ii. BE class has 4 queues =3D> Max number of TCs is 13 iii. BE class has 8 queues =3D> Max number of TCs is 9 d) In order to keep implementation complexity under control, it is required= that all pipes from the same subport share the same mapping of pipe queues= to TCs. e) Currently, all the 16 pipe queues have to be configured (and memory allo= cated for them internally), even if not all of them are needed. Going forwa= rd, it shall be allowed to use less than 16 queues per pipe when not all th= e 16 queues are needed, and no memory shall be allocated for the queues tha= t are not needed. 2. Subport level: Allow different subports of the same port to have differe= nt configuration in terms of number of pipes, pipe queue sizes, pipe queue = mapping to traffic classes, etc. a) In order to keep the implementation complexity under control, it is requ= ired that all pipes within the same subport share the same configuration fo= r these parameters. b) Internal implications: each subport port will likely need to have its ow= n bitmap data structure. 3. Redistribution of unused pipe BW to other pipes within the same subport:= Enable the existing oversubscription mechanism by default. a) Currently, this mechanism needs to be explicitly enabled at build time. b) This change is subject to performance impact not going to be significant= . 4. Pipe TC level: Improve shaper accuracy. a) The current pipe TC rate limiting mechanism is not robust and it can res= ult in deadlock for certain configurations. Currently, the pipe TC credits = are periodically cleared and re-initialized to a fixed value (period is con= figurable), which can result in deadlock if number of pipe TC credits is sm= aller than the MTU. b) The plan is to move the pipe TC rate limiting from the scheduler dequeue= operation (shaping) to the scheduler enqueue operation (metering), by usin= g one token bucket per pipe TC. Basically, packets that exceed the pipe TC = rate will be detected and dropped earlier rather than later, which should b= e beneficial from the perspective of not spending cycles on packets that ar= e later going to dropped anyway. c) Internal implications: Number of token buckets is multiplied 16 times. N= eed to improve the token bucket performance (e.g. by using branchless code)= in order to get back some of the performance. Best regards, Your faithful DPDK QoS implementers, Cristian and Jasvinder