From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id 7C815A0C41;
Tue, 22 Jun 2021 11:13:51 +0200 (CEST)
Received: from [217.70.189.124] (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id F18E14003F;
Tue, 22 Jun 2021 11:13:50 +0200 (CEST)
Received: from mga11.intel.com (mga11.intel.com [192.55.52.93])
by mails.dpdk.org (Postfix) with ESMTP id 5D2344003C
for <dev@dpdk.org>; Tue, 22 Jun 2021 11:13:48 +0200 (CEST)
IronPort-SDR: zlRgHIIlgsKBjPqdsSOJrpks4/xiLGTrB2l4iIt3OC2wBK3kDq2OTc9O06kEbggBXMP19uFEBB
N1nqmYuJbpmA==
X-IronPort-AV: E=McAfee;i="6200,9189,10022"; a="204008869"
X-IronPort-AV: E=Sophos;i="5.83,291,1616482800"; d="scan'208";a="204008869"
Received: from orsmga008.jf.intel.com ([10.7.209.65])
by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
22 Jun 2021 02:13:45 -0700
IronPort-SDR: rYAP1nYZaHx0eHPEsv1Wi2+fdcv+a1pUHSVIQpTYYKR4wjAPslkJ77j37wtF7ylIbLsgs2xz+R
dnSUscQQPaYQ==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.83,291,1616482800"; d="scan'208";a="452541759"
Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15])
by orsmga008.jf.intel.com with ESMTP; 22 Jun 2021 02:13:44 -0700
Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by
ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2242.4; Tue, 22 Jun 2021 02:13:44 -0700
Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by
ORSMSX609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.2242.4; Tue, 22 Jun 2021 02:13:43 -0700
Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by
orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4
via Frontend Transport; Tue, 22 Jun 2021 02:13:43 -0700
Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.101)
by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.1.2242.4; Tue, 22 Jun 2021 02:13:43 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=I64ojuRmxTq8T64oqigIIUA+VijLgk/9NcYJ3uJj1Mg4BFncCX7cme6B+sLM/exwwoL217rUtm49mshAQW+yBQpGlntVSSgvWrhBHPW3XcsTnrFU/K8otCpeMaANOcamNbMl2ns00K2SZgXD0J0LNxiDnxtg97bKmWFaWbKoo0vCCoxiidrNCU6I66cA35abdSr3xvjiTpXIycmrbBPCEovVHrjbuBzOY1qyNz1i0/AjXPMMNqsFauIOUoiie1huwaD5RBOItdrnU1FS37xYJXYBb7ZomX2GvzWHXVXoH2HbJtBxOwlhPj9dPUB743Hw826qnzn2qOo5gKWw+JXDvw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=5UFWs0/bTX37Muw3MJm3J+F8I+vL8gnWCiX8oGL5QgU=;
b=Rrq+X1RwlCEMokmMqGtbbwd8ZMLjrK3mu1+m9W7g49vc79mb6RiO58JN6yUfoigyWe6l/6oErZTwQgwNeA7gWMGCYfjz0UjnrVTi7nYw2uRh/PgzdeQLaYxzguylbZrD9OvdVHUL7cRzx+Iwy6rmAYSgWFHfXErpje6KK5oW7F/4UftZYV/HnHLkc0PYkICHRkVZk8cTL4jY74zlupmLTYG++4o7x7kFdRr3srBz/NTGdeFY5bLTkiQcSIZi9ob1Vz7EqeCA7CatFffwlqtqmDz9LQrvsSZZCBK5V/0s5+lO1rC9gF0lpwjb9FIhJsmP2QhyZ7TkFPVAGBpCD/2n2g==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
s=selector2-intel-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=5UFWs0/bTX37Muw3MJm3J+F8I+vL8gnWCiX8oGL5QgU=;
b=KbphHyE7+tJTNrth7lnFfIw2IhkCQq7pZ4x6RfiPAKr5ClbgOCDoDM2n4FO5lnsFVUxUOPvdrDfDBIJCCapwrVxTI5DmIzLII9BtMMujXM2p9m91Og+k09gjKgi4AWEyMckAnmb/Sy2rWN4xziXYLe6jHxhTaFW4ptZXS+W/YsI=
Received: from DM6PR11MB4491.namprd11.prod.outlook.com (2603:10b6:5:204::19)
by DM4PR11MB5440.namprd11.prod.outlook.com (2603:10b6:5:39c::7) with
Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.22; Tue, 22 Jun
2021 09:13:40 +0000
Received: from DM6PR11MB4491.namprd11.prod.outlook.com
([fe80::7dc4:66b0:f76b:6d48]) by DM6PR11MB4491.namprd11.prod.outlook.com
([fe80::7dc4:66b0:f76b:6d48%7]) with mapi id 15.20.4242.023; Tue, 22 Jun 2021
09:13:40 +0000
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>, "dev@dpdk.org"
<dev@dpdk.org>, "Hunt, David" <david.hunt@intel.com>
CC: "Loftus, Ciara" <ciara.loftus@intel.com>
Thread-Topic: [dpdk-dev] [PATCH v1 4/7] power: remove thread safety from PMD
power API's
Thread-Index: AQHXVt3s3SqCadMTUkmuRVO1oz9RIKsf3MzQ
Date: Tue, 22 Jun 2021 09:13:40 +0000
Message-ID: <DM6PR11MB44919DFCA85732B18DA74D119A099@DM6PR11MB4491.namprd11.prod.outlook.com>
References: <cover.1622548381.git.anatoly.burakov@intel.com>
<f0ea799f2908da104850c3858eea4fd059d38496.1622548381.git.anatoly.burakov@intel.com>
In-Reply-To: <f0ea799f2908da104850c3858eea4fd059d38496.1622548381.git.anatoly.burakov@intel.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.5.1.3
authentication-results: intel.com; dkim=none (message not signed)
header.d=none;intel.com; dmarc=none action=none header.from=intel.com;
x-originating-ip: [109.255.184.192]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a26c0f81-7aa8-446d-96c7-08d9355e06d4
x-ms-traffictypediagnostic: DM4PR11MB5440:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <DM4PR11MB5440202F6E8C6F2D43FE75409A099@DM4PR11MB5440.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: jnbWJoIHZgIITYnJJCXRchodpn8cElPcvpXbR7rStxXbBRoDujyKEqaSUHOqsFogd8HwSrP2OKs74DpPNSZJIjlpWOwp1a/wbrK9GdjfVLXG5clKGAEHDIRDYda3REOZNC7+U6HPcdoYOHMFJQXKn4FbDgOna5hF2BvE6IT4BszTfNYWJ15r0ISvEKeRi4X3s+G3D1LDcPgLihBKOogkPhxQKn2YTqfANLFJQ5eZspfTIQtiEiuVxvYvML/wf035g0VbMqIgQPukW+xCGoFbUiNPdDYuVF3+aBxKxVgCGJ2zsG23PO6gizy8wb8zisOrm0oKI/gIUQJ8ytGHEBzlx2S6TzxWp8CmHWIFgdhf3Rk/3KY0RQRgWdROwlrveo1aTSDf7mArp/8b6v2u6hsltE1usvR6PzjshABrRO8yDvgqZGPoEO1mnlCxF/lJ4Xtof4j4v1i+IGhUlGOTHjSErTUKfjAfN+DUU4Un3z0DzWNaJH7kLppR10mG0EKkLlD8LOM8qNODdYcpVWAT7XvYGf8q6wAPJYNQNTumZeY7l4pwJxrt8BRp0BZ5NpsvqYvEkVV5RLSFtU3a2c5JBPrlhQjp9gnPmKbz+ACx5ZhG/64=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:DM6PR11MB4491.namprd11.prod.outlook.com; PTR:; CAT:NONE;
SFS:(366004)(39860400002)(346002)(376002)(136003)(396003)(110136005)(66946007)(64756008)(66446008)(66556008)(76116006)(52536014)(122000001)(66476007)(8936002)(186003)(316002)(8676002)(5660300002)(55236004)(9686003)(107886003)(6636002)(7696005)(4326008)(71200400001)(26005)(33656002)(86362001)(2906002)(478600001)(55016002)(38100700002)(6506007)(83380400001);
DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?k5tUwSjj2MrH8MdPT6AOqLd+gROTWQItHxDBNYOxsY3thPZLZuLbdsLnQJ9e?=
=?us-ascii?Q?XuXk+UK/TTOpnXwEhR3tbvG+OUEzfgprvqk6VifW3ef6VDkO1+vfBzcLXjbI?=
=?us-ascii?Q?/nY/jSsBpU13VxtswZlgoF99CZGmdwXDed6HfUVjigdB/HoytJaVuE4/MQHZ?=
=?us-ascii?Q?JJsY5l9sE+JHFoo7gWICbkXOm2rTJiRsGWGW+sD3WnqFxuz5vjxiQYop1kfr?=
=?us-ascii?Q?/6iECJkPF8wKHSCmmSb7p7QAZ1xlQqG0J7fJBUCONK/nNjgER5egkvuOvrTS?=
=?us-ascii?Q?/bWA29ouTUXTjzwelXCIKKlTaAUNg7TjyBIKsfDnnTIujn04s5mJueii3xlb?=
=?us-ascii?Q?HaQmKm3b8/1jbqg61ZXXWVC0s6IngVW3ftH6XgTwn8mOUDEDBCtYjAZbbDcd?=
=?us-ascii?Q?baAyfm/fo+ZnKhRwwUuAaolSDGC+aPOCOSx6WXHlRJjwc7erRLfZtRitGqjj?=
=?us-ascii?Q?B8hdgbUPtmb+7bZb4E1tXm/6dJ2qA4aGB0O8SHvDIJacTOlc0qnVUWcKbkdD?=
=?us-ascii?Q?7zHu7APbrEGn25NmnP6kxsi4sQpHI1OkSPbUZLX+k2UwavKDTqcFZkZoysdr?=
=?us-ascii?Q?/AuuGSYXR0xRfMKB8tT5Q3Lv1+iBkbgbgxaHbI53gnBO0TYzdM+5mFGCATtW?=
=?us-ascii?Q?26tqh6QUxe1SKMa/bPBxFtjgzSsQl4DSUW+7FVT68go9C+z2zKntOjvMhJWP?=
=?us-ascii?Q?FQzRZd07DPjenMvfx/tKDLXVRsQRVRozQ+vNemHwDJRZJUmrKNiUbYg2LJ9p?=
=?us-ascii?Q?PXnSfuiIAyK05DED9M1itPX9S/UWGWgkfHc5tH+qRpbpNFUyaKK5FEQxR+Lo?=
=?us-ascii?Q?dSHTct0R+dE0OZQsMss+rzUk5qlgld/RqdaAXIgg/Qa4OUnKZzuvrx8H/TBX?=
=?us-ascii?Q?R2HHgDV6h58Q2I6ims9uhRckf9QT2Ov3127v5zJRiR7zjBLhDhswrColIDjU?=
=?us-ascii?Q?MgejgNOnjCvM7135HvcIEOdQGL/V/YxkAhHxKHU25Lm61kpm/vgHYLnIkDki?=
=?us-ascii?Q?MsvzJxxJA2zWZ8KHKiPQGKvxXQ7tpDsLSN2Er1npizIWs7D/p35lVXevhZza?=
=?us-ascii?Q?ASX26RU+gvJl0pypofmnMY6Mvt3m19inbZ/i2Txa5OWk6rpiYP/jwlMojRtP?=
=?us-ascii?Q?g8csbYbO2kq35RzpewTljpVuDX95SxnLDdAsdbG/VvAr78QWJGwb1TEdBvp2?=
=?us-ascii?Q?g03UqVNMZL03RJDsawtGjIRMULcRFyXh6fZyqJBO2SgswrfriEqXUidkQ9Rj?=
=?us-ascii?Q?huFht+5IO2jGInB9XCSQg5Lt3mWvl4KXVZFl1UhAqC8/iW9z8qzq24v06jAj?=
=?us-ascii?Q?ygcFQn8AcPrRaMI7cePEF2Qw?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4491.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a26c0f81-7aa8-446d-96c7-08d9355e06d4
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jun 2021 09:13:40.5900 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: tbH3oDEwc3H+zmN7dVGDJJLfXhcy/MukcwdwVeTtiTN+CGoIhMRoMtkqgO5Hieyb/PEpa3IaaxNDka+OevRRuRtgpbEsplnb0JS55gLGvZ8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5440
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [PATCH v1 4/7] power: remove thread safety from PMD
power API's
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
> Currently, we expect that only one callback can be active at any given
> moment, for a particular queue configuration, which is relatively easy
> to implement in a thread-safe way. However, we're about to add support
> for multiple queues per lcore, which will greatly increase the
> possibility of various race conditions.
>=20
> We could have used something like an RCU for this use case, but absent
> of a pressing need for thread safety we'll go the easy way and just
> mandate that the API's are to be called when all affected ports are
> stopped, and document this limitation. This greatly simplifies the
> `rte_power_monitor`-related code.
I think you need to update RN too with that.
Another thing - do you really need the whole port stopped?
>From what I understand - you work on queues, so it is enough for you
that related RX queue is stopped.
So, to make things a bit more robust, in pmgmt_queue_enable/disable=20
you can call rte_eth_rx_queue_info_get() and check queue state.
=20
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> lib/power/meson.build | 3 +
> lib/power/rte_power_pmd_mgmt.c | 106 ++++++++-------------------------
> lib/power/rte_power_pmd_mgmt.h | 6 ++
> 3 files changed, 35 insertions(+), 80 deletions(-)
>=20
> diff --git a/lib/power/meson.build b/lib/power/meson.build
> index c1097d32f1..4f6a242364 100644
> --- a/lib/power/meson.build
> +++ b/lib/power/meson.build
> @@ -21,4 +21,7 @@ headers =3D files(
> 'rte_power_pmd_mgmt.h',
> 'rte_power_guest_channel.h',
> )
> +if cc.has_argument('-Wno-cast-qual')
> + cflags +=3D '-Wno-cast-qual'
> +endif
> deps +=3D ['timer', 'ethdev']
> diff --git a/lib/power/rte_power_pmd_mgmt.c b/lib/power/rte_power_pmd_mgm=
t.c
> index db03cbf420..0707c60a4f 100644
> --- a/lib/power/rte_power_pmd_mgmt.c
> +++ b/lib/power/rte_power_pmd_mgmt.c
> @@ -40,8 +40,6 @@ struct pmd_queue_cfg {
> /**< Callback mode for this queue */
> const struct rte_eth_rxtx_callback *cur_cb;
> /**< Callback instance */
> - volatile bool umwait_in_progress;
> - /**< are we currently sleeping? */
> uint64_t empty_poll_stats;
> /**< Number of empty polls */
> } __rte_cache_aligned;
> @@ -92,30 +90,11 @@ clb_umwait(uint16_t port_id, uint16_t qidx, struct rt=
e_mbuf **pkts __rte_unused,
> struct rte_power_monitor_cond pmc;
> uint16_t ret;
>=20
> - /*
> - * we might get a cancellation request while being
> - * inside the callback, in which case the wakeup
> - * wouldn't work because it would've arrived too early.
> - *
> - * to get around this, we notify the other thread that
> - * we're sleeping, so that it can spin until we're done.
> - * unsolicited wakeups are perfectly safe.
> - */
> - q_conf->umwait_in_progress =3D true;
> -
> - rte_atomic_thread_fence(__ATOMIC_SEQ_CST);
> -
> - /* check if we need to cancel sleep */
> - if (q_conf->pwr_mgmt_state =3D=3D PMD_MGMT_ENABLED) {
> - /* use monitoring condition to sleep */
> - ret =3D rte_eth_get_monitor_addr(port_id, qidx,
> - &pmc);
> - if (ret =3D=3D 0)
> - rte_power_monitor(&pmc, UINT64_MAX);
> - }
> - q_conf->umwait_in_progress =3D false;
> -
> - rte_atomic_thread_fence(__ATOMIC_SEQ_CST);
> + /* use monitoring condition to sleep */
> + ret =3D rte_eth_get_monitor_addr(port_id, qidx,
> + &pmc);
> + if (ret =3D=3D 0)
> + rte_power_monitor(&pmc, UINT64_MAX);
> }
> } else
> q_conf->empty_poll_stats =3D 0;
> @@ -183,6 +162,7 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lcor=
e_id, uint16_t port_id,
> {
> struct pmd_queue_cfg *queue_cfg;
> struct rte_eth_dev_info info;
> + rte_rx_callback_fn clb;
> int ret;
>=20
> RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -EINVAL);
> @@ -232,17 +212,7 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lco=
re_id, uint16_t port_id,
> ret =3D -ENOTSUP;
> goto end;
> }
> - /* initialize data before enabling the callback */
> - queue_cfg->empty_poll_stats =3D 0;
> - queue_cfg->cb_mode =3D mode;
> - queue_cfg->umwait_in_progress =3D false;
> - queue_cfg->pwr_mgmt_state =3D PMD_MGMT_ENABLED;
> -
> - /* ensure we update our state before callback starts */
> - rte_atomic_thread_fence(__ATOMIC_SEQ_CST);
> -
> - queue_cfg->cur_cb =3D rte_eth_add_rx_callback(port_id, queue_id,
> - clb_umwait, NULL);
> + clb =3D clb_umwait;
> break;
> }
> case RTE_POWER_MGMT_TYPE_SCALE:
> @@ -269,16 +239,7 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lco=
re_id, uint16_t port_id,
> ret =3D -ENOTSUP;
> goto end;
> }
> - /* initialize data before enabling the callback */
> - queue_cfg->empty_poll_stats =3D 0;
> - queue_cfg->cb_mode =3D mode;
> - queue_cfg->pwr_mgmt_state =3D PMD_MGMT_ENABLED;
> -
> - /* this is not necessary here, but do it anyway */
> - rte_atomic_thread_fence(__ATOMIC_SEQ_CST);
> -
> - queue_cfg->cur_cb =3D rte_eth_add_rx_callback(port_id,
> - queue_id, clb_scale_freq, NULL);
> + clb =3D clb_scale_freq;
> break;
> }
> case RTE_POWER_MGMT_TYPE_PAUSE:
> @@ -286,18 +247,21 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lc=
ore_id, uint16_t port_id,
> if (global_data.tsc_per_us =3D=3D 0)
> calc_tsc();
>=20
> - /* initialize data before enabling the callback */
> - queue_cfg->empty_poll_stats =3D 0;
> - queue_cfg->cb_mode =3D mode;
> - queue_cfg->pwr_mgmt_state =3D PMD_MGMT_ENABLED;
> -
> - /* this is not necessary here, but do it anyway */
> - rte_atomic_thread_fence(__ATOMIC_SEQ_CST);
> -
> - queue_cfg->cur_cb =3D rte_eth_add_rx_callback(port_id, queue_id,
> - clb_pause, NULL);
> + clb =3D clb_pause;
> break;
> + default:
> + RTE_LOG(DEBUG, POWER, "Invalid power management type\n");
> + ret =3D -EINVAL;
> + goto end;
> }
> +
> + /* initialize data before enabling the callback */
> + queue_cfg->empty_poll_stats =3D 0;
> + queue_cfg->cb_mode =3D mode;
> + queue_cfg->pwr_mgmt_state =3D PMD_MGMT_ENABLED;
> + queue_cfg->cur_cb =3D rte_eth_add_rx_callback(port_id, queue_id,
> + clb, NULL);
> +
> ret =3D 0;
> end:
> return ret;
> @@ -323,27 +287,8 @@ rte_power_ethdev_pmgmt_queue_disable(unsigned int lc=
ore_id,
> /* stop any callbacks from progressing */
> queue_cfg->pwr_mgmt_state =3D PMD_MGMT_DISABLED;
>=20
> - /* ensure we update our state before continuing */
> - rte_atomic_thread_fence(__ATOMIC_SEQ_CST);
> -
> switch (queue_cfg->cb_mode) {
> - case RTE_POWER_MGMT_TYPE_MONITOR:
> - {
> - bool exit =3D false;
> - do {
> - /*
> - * we may request cancellation while the other thread
> - * has just entered the callback but hasn't started
> - * sleeping yet, so keep waking it up until we know it's
> - * done sleeping.
> - */
> - if (queue_cfg->umwait_in_progress)
> - rte_power_monitor_wakeup(lcore_id);
> - else
> - exit =3D true;
> - } while (!exit);
> - }
> - /* fall-through */
> + case RTE_POWER_MGMT_TYPE_MONITOR: /* fall-through */
> case RTE_POWER_MGMT_TYPE_PAUSE:
> rte_eth_remove_rx_callback(port_id, queue_id,
> queue_cfg->cur_cb);
> @@ -356,10 +301,11 @@ rte_power_ethdev_pmgmt_queue_disable(unsigned int l=
core_id,
> break;
> }
> /*
> - * we don't free the RX callback here because it is unsafe to do so
> - * unless we know for a fact that all data plane threads have stopped.
> + * the API doc mandates that the user stops all processing on affected
> + * ports before calling any of these API's, so we can assume that the
> + * callbacks can be freed. we're intentionally casting away const-ness.
> */
> - queue_cfg->cur_cb =3D NULL;
> + rte_free((void *)queue_cfg->cur_cb);
>=20
> return 0;
> }
> diff --git a/lib/power/rte_power_pmd_mgmt.h b/lib/power/rte_power_pmd_mgm=
t.h
> index 7a0ac24625..7557f5d7e1 100644
> --- a/lib/power/rte_power_pmd_mgmt.h
> +++ b/lib/power/rte_power_pmd_mgmt.h
> @@ -43,6 +43,9 @@ enum rte_power_pmd_mgmt_type {
> *
> * @note This function is not thread-safe.
> *
> + * @warning This function must be called when all affected Ethernet port=
s are
> + * stopped and no Rx/Tx is in progress!
> + *
> * @param lcore_id
> * The lcore the Rx queue will be polled from.
> * @param port_id
> @@ -69,6 +72,9 @@ rte_power_ethdev_pmgmt_queue_enable(unsigned int lcore_=
id,
> *
> * @note This function is not thread-safe.
> *
> + * @warning This function must be called when all affected Ethernet port=
s are
> + * stopped and no Rx/Tx is in progress!
> + *
> * @param lcore_id
> * The lcore the Rx queue is polled from.
> * @param port_id
> --
> 2.25.1