From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8D189A0C3F; Mon, 28 Jun 2021 15:30:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 768AC40692; Mon, 28 Jun 2021 15:30:09 +0200 (CEST) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id 54D264068A for ; Mon, 28 Jun 2021 15:30:07 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10028"; a="188332412" X-IronPort-AV: E=Sophos;i="5.83,306,1616482800"; d="scan'208";a="188332412" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Jun 2021 06:30:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,306,1616482800"; d="scan'208";a="446565991" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by orsmga007.jf.intel.com with ESMTP; 28 Jun 2021 06:30:02 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Mon, 28 Jun 2021 06:30:02 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Mon, 28 Jun 2021 06:30:02 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4 via Frontend Transport; Mon, 28 Jun 2021 06:30:02 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.173) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.4; Mon, 28 Jun 2021 06:29:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SAxqoQCFDwd1+HheFMdEsk/wDES7x/RgAG0m/nCD2rNceQcyPDfv7po7Q4nyzDQ/xetwBQ6qYY6bHy9+1THRoGgLeOnuaCayE1jc93L/QnoA8BNClSmkpttaL+Vx+tPrzRPVo6wmWXPkHX1gmhPZkP9gCwgTWJk6hIALGC2GSUwrHnQm0aGDD318AQRhg3Og1U8fFs0j7VUIR3DNiTmiHQemtgxG/HMB00+Ajt0MLBinbOakknaX1q0jouWJ/KPyZGuO3k/GSBt6ITf2IK1uIpEIp32QNMGP26cvENXvT2njMcNDlldW2e00twObS7X3cBWKNZ4wrZs3QNDycbdKaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2/2vU7KAUbgwgR52mFi679LdgNywxhoEo9GNxMPkeaE=; b=NIB4jfGaEYOptNRorLafG8UzZmeaQvQVk9WaAlluGNfxs3LaFM8YlfgKKnfnAX6q0ziY5C68h61VG0i1sODhFdSVRWOaerBTTBjwpWyvuCNZ/yxmFCEWD0x6p1kZT/+d6AZSUDqd0UX5yT5rRU48EQSFhdd7z80pkagRjAuCTv1PmMjLOc9OnFUjHnn+pjgawv++khnO+eFPrwgUErRB2qt5vP6UseOj/YJEwfbEddenrxvFgSUw2b6kSIgKObepcRXJAkK3Ia3F4YF9bVWL1VC24YZcRwvEDwlzPUmYlGtjcYprzfJOK5EVYH1NHggn/u4zfq5MEIEhE1+Td2LWbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2/2vU7KAUbgwgR52mFi679LdgNywxhoEo9GNxMPkeaE=; b=PxPXMgxLaPtnAiTLxULoHuATS+JZ7CXpBsRFMafNEKljWnf+ZzpVDRk40Fom1UgSHDR5r7nhSXDDwFvHS0ZpVSRFgGg2Adk5DGY46JyyhBKzt046Q7Y07x1Mds+/RsjTDBgCOFU2wZ7wU/ZCS4XFbdXkDyKCmUuc5ZbCD/97f/8= Received: from DM6PR11MB4491.namprd11.prod.outlook.com (2603:10b6:5:204::19) by DM6PR11MB2636.namprd11.prod.outlook.com (2603:10b6:5:c8::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.19; Mon, 28 Jun 2021 13:29:10 +0000 Received: from DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::7dc4:66b0:f76b:6d48]) by DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::7dc4:66b0:f76b:6d48%7]) with mapi id 15.20.4264.026; Mon, 28 Jun 2021 13:29:10 +0000 From: "Ananyev, Konstantin" To: "Burakov, Anatoly" , "dev@dpdk.org" , "Hunt, David" CC: "Loftus, Ciara" Thread-Topic: [dpdk-dev] [PATCH v3 6/7] power: support monitoring multiple Rx queues Thread-Index: AQHXbBsO+Y6lPWDNwUKowUur7w9ha6spaDyw Date: Mon, 28 Jun 2021 13:29:10 +0000 Message-ID: References: <676eab0e1eb6c63acb170893675daa5a39eac29d.1624884053.git.anatoly.burakov@intel.com> In-Reply-To: <676eab0e1eb6c63acb170893675daa5a39eac29d.1624884053.git.anatoly.burakov@intel.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.5.1.3 authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [109.255.184.192] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ef91bdae-9f15-494c-1378-08d93a38b6a9 x-ms-traffictypediagnostic: DM6PR11MB2636: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: DCiZ4K9br41N3b3Ln/DKNVvWia0QjXIwZIfvpzrtY4Ep5eaaRBilotZTQ96Ii2TtqjyTq5DUpeTdRHiqFIf9MoxugjIsaKqcNIEhtDo1CMyMvqMxmL3zrzDIVcRWRObKeFsEKFtkDW9Wg+sG/mvP13pRvPv7DhdbDzdrzkph4eV7a5n9oP3d7TkDtVGaxntqOgh325olABPwSw06133lDMvP+hRf47CVmMkP69sqmz/VfJWPBmm25c8StAIbRDPRxMd1tRWcFwPqrFIpNUTXOiG5tEpBbi1Hn0rhT+J1SL3uZ1i46zqBhO5uORR7gQrm0fC/8E5YuTmTSVi5poI3/dmNatAzD1kscJcbpSRBtXdOIAxesW+Cl44cYYzmzKWppxW9PoqWlB3W/Gl9HDJoAjmp/TvUsOS4mgcMGUyy7qUS2m6ACkWJGx92xn8/ypNwxFuPrSCUEU28a4hVbWOVO7usPDZ0vw/WOi9qQCxxSwRdx7SLPq0RVCoAlsXmfXSS4s0YB+xgnO8sizBDxxnUedS4vN+GPG8hGJTTsjSJSJ6smyuWRJH7LGsVLNHwHPX0JHyYCoMdbkcE9X7VqKntZfOcrxpJCBfirR2Ui9l/viNAnIyUD2lQ8mOS39dx3F3k7kQgwizj18va6ZvL/4WMrg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4491.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(136003)(39860400002)(366004)(396003)(376002)(346002)(33656002)(122000001)(71200400001)(478600001)(55016002)(38100700002)(83380400001)(110136005)(9686003)(2906002)(52536014)(316002)(86362001)(186003)(5660300002)(7696005)(66946007)(26005)(8676002)(6636002)(76116006)(4326008)(66476007)(66556008)(66446008)(64756008)(55236004)(107886003)(6506007)(8936002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?BfLEWXwWaJY0XtlFQG8HUqI264OEI4BPh4rW685G/QVYgqB/j+ajFi3jKkS+?= =?us-ascii?Q?+ZdBVAb4POzbsMEwPjdX1gUKyb7aXIm7bJGC/kRHdmlFbX9ujj8US3my+UzU?= =?us-ascii?Q?NEJK/Eb8gLp7e7dySrFetlri/XEAvz28KVfaVqezDSATacnUSrxg6C2PGP7n?= =?us-ascii?Q?mbAUpaF2N2S6VLym2mgkfSLd4nMFdN9HwEUx+WnV5p1Th+a+BUQG3ZPl3gFo?= =?us-ascii?Q?MsFAP3NUyfZXYEMbNZOyvuskmag2G99ysSGTqJoPIAUnYa1brt5T/SUvy3Ba?= =?us-ascii?Q?UGprMgTI52nez3iT+dql3qQxAL5lBrx7Npphsb9fSzOko51sWFM+9vbTek6t?= =?us-ascii?Q?rWLalv8dYQb4vF0SMgwzU1MnhFUqG7EjyNQHrsJ92GxobaixCcA9UUcT8Ur3?= =?us-ascii?Q?xnSXZvLaQht1Y4Rd43G299EMrpAYDoV6cOQHea2T0lUWG3stQxQnHVGDiMQF?= =?us-ascii?Q?N7GvyAv2VRRaYmZXzhaG/bNrDiSQABnq4b6egvpDNCfakblmNyBY7oPq2khm?= =?us-ascii?Q?k8yUfvH7/yaC24U8Yxmqjh9Cjea2CNKr886kC5qc1wy2PoiHiXbnNdMm2bZv?= =?us-ascii?Q?a9zTWJGVinQzH43RdAW/EEawlfXKQFhEnQ8drx+CWNzMpyhDmrDvEWRFdGss?= =?us-ascii?Q?lFv2vck4VHvFgCraWQtxYGmk1geZtVJtBj5gGpiZ4lTZefGo0s7/ZzKj2Qb3?= =?us-ascii?Q?6uP4KiIzKPjR9gNsCqkBW6QhSZqeF27laCVdXivnmarl9HVV1ayRff83JFN3?= =?us-ascii?Q?n8B6EPF+LZf9yByQTue6Uj9vCH5mGl2/xgbpuwByg6X5sR/Ve/A4jwJX/Gox?= =?us-ascii?Q?t8U2cGskIUtezlVffOcyteJ8fmKTH4aLDEgMRcxkZZkljVB2Gm4GfUDStZSY?= =?us-ascii?Q?50+za0mWTKvgfXzyhNAFZ8ou2L+x+A0SfGWBSHS7fHQYatewlZ09VkGgWa5r?= =?us-ascii?Q?qO+WB+OOL13Wx4tqZrl2V+iDV+F68tyGqCM6dZtP1C4KQjNAH492v2jCBQsN?= =?us-ascii?Q?bumELtm4OzOe0FqPcYVtf4q8d0eZAheCQHRdk+iOBOvRaatkUr96CQffM2O5?= =?us-ascii?Q?1EHftWulTHRABGn1GorN1UeZA3iNBElXkswPDA4UXYBAonRQskE1IfPt7AWj?= =?us-ascii?Q?3izpsERXbItnyfXBrCLhMEc9RyPelbmkVg/1+8ZqoXYVF7NtlKz0c/71f+b1?= =?us-ascii?Q?HuYP6m6kaK+jfpZqmqOpxW1t9AVSx0vDOyUZHDhP+eU5dB9m/iXk++0mthDH?= =?us-ascii?Q?W0W1pu66GLyN3Zx0AUsrYGoz8M/UOQT95+tXEKmCMleF4gaja1Y+Yfd5XL6b?= =?us-ascii?Q?OPkT3A7Q3dtx8VnxfLmFriUK?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4491.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: ef91bdae-9f15-494c-1378-08d93a38b6a9 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jun 2021 13:29:10.5729 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Qy9G84/obgs22R34PJkuBh5N3dd279cXnfhMRSnkl67EvcTKipPgpSJZnWSU0q4kwuMTyLWlZhLN7JGCjAentpUpd7dwzSu875gkk7sfHI0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB2636 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 6/7] power: support monitoring multiple Rx queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > Use the new multi-monitor intrinsic to allow monitoring multiple ethdev > Rx queues while entering the energy efficient power state. The multi > version will be used unconditionally if supported, and the UMWAIT one > will only be used when multi-monitor is not supported by the hardware. >=20 > Signed-off-by: Anatoly Burakov > --- > doc/guides/prog_guide/power_man.rst | 9 ++-- > lib/power/rte_power_pmd_mgmt.c | 76 ++++++++++++++++++++++++++++- > 2 files changed, 80 insertions(+), 5 deletions(-) >=20 > diff --git a/doc/guides/prog_guide/power_man.rst b/doc/guides/prog_guide/= power_man.rst > index fac2c19516..3245a5ebed 100644 > --- a/doc/guides/prog_guide/power_man.rst > +++ b/doc/guides/prog_guide/power_man.rst > @@ -221,13 +221,16 @@ power saving whenever empty poll count reaches a ce= rtain number. > The "monitor" mode is only supported in the following configurations and= scenarios: >=20 > * If ``rte_cpu_get_intrinsics_support()`` function indicates that > + ``rte_power_monitor_multi()`` function is supported by the platform, t= hen > + monitoring multiple Ethernet Rx queues for traffic will be supported. > + > +* If ``rte_cpu_get_intrinsics_support()`` function indicates that only > ``rte_power_monitor()`` is supported by the platform, then monitoring = will be > limited to a mapping of 1 core 1 queue (thus, each Rx queue will have = to be > monitored from a different lcore). >=20 > -* If ``rte_cpu_get_intrinsics_support()`` function indicates that the > - ``rte_power_monitor()`` function is not supported, then monitor mode w= ill not > - be supported. > +* If ``rte_cpu_get_intrinsics_support()`` function indicates that neithe= r of the > + two monitoring functions are supported, then monitor mode will not be = supported. >=20 > * Not all Ethernet devices support monitoring, even if the underlying > platform may support the necessary CPU instructions. Please refer to > diff --git a/lib/power/rte_power_pmd_mgmt.c b/lib/power/rte_power_pmd_mgm= t.c > index 7762cd39b8..aab2d4f1ee 100644 > --- a/lib/power/rte_power_pmd_mgmt.c > +++ b/lib/power/rte_power_pmd_mgmt.c > @@ -155,6 +155,24 @@ queue_list_remove(struct pmd_core_cfg *cfg, const un= ion queue *q) > return 0; > } >=20 > +static inline int > +get_monitor_addresses(struct pmd_core_cfg *cfg, > + struct rte_power_monitor_cond *pmc) > +{ > + const struct queue_list_entry *qle; > + size_t i =3D 0; > + int ret; > + > + TAILQ_FOREACH(qle, &cfg->head, next) { > + struct rte_power_monitor_cond *cur =3D &pmc[i]; Looks like you never increment 'i' value inside that function. Also it probably will be safer to add 'num' parameter to check that we will never over-run pmc[] boundaries. > + const union queue *q =3D &qle->queue; > + ret =3D rte_eth_get_monitor_addr(q->portid, q->qid, cur); > + if (ret < 0) > + return ret; > + } > + return 0; > +} > + > static void > calc_tsc(void) > { > @@ -183,6 +201,48 @@ calc_tsc(void) > } > } >=20 > +static uint16_t > +clb_multiwait(uint16_t port_id, uint16_t qidx, > + struct rte_mbuf **pkts __rte_unused, uint16_t nb_rx, > + uint16_t max_pkts __rte_unused, void *addr __rte_unused) > +{ > + const unsigned int lcore =3D rte_lcore_id(); > + const union queue q =3D {.portid =3D port_id, .qid =3D qidx}; > + const bool empty =3D nb_rx =3D=3D 0; > + struct pmd_core_cfg *q_conf; > + > + q_conf =3D &lcore_cfg[lcore]; > + > + /* early exit */ > + if (likely(!empty)) { > + q_conf->empty_poll_stats =3D 0; > + } else { > + /* do we care about this particular queue? */ > + if (!queue_is_power_save(q_conf, &q)) > + return nb_rx; I still don't understand the need of 'special' power_save queue here... Why we can't just have a function: get_number_of_queues_whose_sequential_empty_polls_less_then_threshold(struc= t pmd_core_cfg *lcore_cfg), and then just: /* all queues have at least EMPTYPOLL_MAX sequential empty polls */ if (get_number_of_queues_whose_sequential_empty_polls_less_then_threshold(q= _conf) =3D=3D 0) { /* go into power-save mode here */ } > + > + /* > + * we can increment unconditionally here because if there were > + * non-empty polls in other queues assigned to this core, we > + * dropped the counter to zero anyway. > + */ > + q_conf->empty_poll_stats++; > + if (unlikely(q_conf->empty_poll_stats > EMPTYPOLL_MAX)) { > + struct rte_power_monitor_cond pmc[RTE_MAX_ETHPORTS]; I think you need here: struct rte_power_monitor_cond pmc[q_conf->n_queues]; > + uint16_t ret; > + > + /* gather all monitoring conditions */ > + ret =3D get_monitor_addresses(q_conf, pmc); > + > + if (ret =3D=3D 0) > + rte_power_monitor_multi(pmc, > + q_conf->n_queues, UINT64_MAX); > + } > + } > + > + return nb_rx; > +} > + > static uint16_t > clb_umwait(uint16_t port_id, uint16_t qidx, struct rte_mbuf **pkts __rte= _unused, > uint16_t nb_rx, uint16_t max_pkts __rte_unused, > @@ -348,14 +408,19 @@ static int > check_monitor(struct pmd_core_cfg *cfg, const union queue *qdata) > { > struct rte_power_monitor_cond dummy; > + bool multimonitor_supported; >=20 > /* check if rte_power_monitor is supported */ > if (!global_data.intrinsics_support.power_monitor) { > RTE_LOG(DEBUG, POWER, "Monitoring intrinsics are not supported\n"); > return -ENOTSUP; > } > + /* check if multi-monitor is supported */ > + multimonitor_supported =3D > + global_data.intrinsics_support.power_monitor_multi; >=20 > - if (cfg->n_queues > 0) { > + /* if we're adding a new queue, do we support multiple queues? */ > + if (cfg->n_queues > 0 && !multimonitor_supported) { > RTE_LOG(DEBUG, POWER, "Monitoring multiple queues is not supported\n")= ; > return -ENOTSUP; > } > @@ -371,6 +436,13 @@ check_monitor(struct pmd_core_cfg *cfg, const union = queue *qdata) > return 0; > } >=20 > +static inline rte_rx_callback_fn > +get_monitor_callback(void) > +{ > + return global_data.intrinsics_support.power_monitor_multi ? > + clb_multiwait : clb_umwait; > +} > + > int > rte_power_ethdev_pmgmt_queue_enable(unsigned int lcore_id, uint16_t port= _id, > uint16_t queue_id, enum rte_power_pmd_mgmt_type mode) > @@ -434,7 +506,7 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lcor= e_id, uint16_t port_id, > if (ret < 0) > goto end; >=20 > - clb =3D clb_umwait; > + clb =3D get_monitor_callback(); > break; > case RTE_POWER_MGMT_TYPE_SCALE: > /* check if we can add a new queue */ > -- > 2.25.1