[dpdk-dev] [PATCH] net/mlx5: add power monitoring support

[dpdk-dev] [PATCH] net/mlx5: add power monitoring support

[Alexander Kozyrev]

Support the PMD power management API in MLX5 driver.
The monitor policy of this API puts a CPU core to sleep until
a data in some monitored memory address is changed by the NIC.
Implement the get_monitor_addr function to return an address
of a CQE owner bit to monitor the arrival of a new packet.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/rel_notes/release_21_05.rst |  4 ++++
 drivers/net/mlx5/mlx5.c                |  2 ++
 drivers/net/mlx5/mlx5_rxtx.c           | 19 +++++++++++++++++++
 drivers/net/mlx5/mlx5_rxtx.h           |  1 +
 4 files changed, 26 insertions(+)

diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst
index f262d48e82..928eafd92f 100644
--- a/doc/guides/rel_notes/release_21_05.rst
+++ b/doc/guides/rel_notes/release_21_05.rst
@@ -65,6 +65,10 @@ New Features
   * Added support for freeing Tx mbuf on demand.
   * Added support for copper port in Kunpeng930.
 
+* **Updated Mellanox mlx5 driver.**
+
+  * Added support for the monitor policy of Power Management API.
+
 * **Updated NXP DPAA driver.**
 
   * Added support for shared ethernet interface.
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index abd7ff70df..7b419deb2c 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1496,6 +1496,7 @@ const struct eth_dev_ops mlx5_dev_ops = {
 	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
 	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
 	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+	.get_monitor_addr = mlx5_get_monitor_addr,
 };
 
 /* Available operations from secondary process. */
@@ -1580,6 +1581,7 @@ const struct eth_dev_ops mlx5_dev_ops_isolate = {
 	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
 	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
 	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+	.get_monitor_addr = mlx5_get_monitor_addr,
 };
 
 /**
diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index e3ce9fd224..0fc0c2096a 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -712,6 +712,25 @@ mlx5_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	return rx_queue_count(rxq);
 }
 
+int mlx5_get_monitor_addr(void *rx_queue, struct rte_power_monitor_cond *pmc)
+{
+	struct mlx5_rxq_data *rxq = rx_queue;
+	const unsigned int cqe_num = 1 << rxq->cqe_n;
+	const unsigned int cqe_mask = cqe_num - 1;
+	const uint16_t idx = rxq->cq_ci & cqe_num;
+	volatile struct mlx5_cqe *cqe = &(*rxq->cqes)[rxq->cq_ci & cqe_mask];
+
+	if (unlikely(!cqe)) {
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	pmc->addr = &cqe->op_own;
+	pmc->val =  !!idx;
+	pmc->mask = MLX5_CQE_OWNER_MASK;
+	pmc->size = sizeof(uint8_t);
+	return 0;
+}
+
 #define MLX5_SYSTEM_LOG_DIR "/var/log"
 /**
  * Dump debug information to log file.
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 0fd98af9d1..35a1bba486 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -441,6 +441,7 @@ int mlx5_rx_burst_mode_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 			   struct rte_eth_burst_mode *mode);
 int mlx5_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
 			   struct rte_eth_burst_mode *mode);
+int mlx5_get_monitor_addr(void *rx_queue, struct rte_power_monitor_cond *pmc);
 
 /* Vectorized version of mlx5_rxtx.c */
 int mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq_data);
-- 
2.24.1

[dpdk-dev] [PATCH v2] net/mlx5: add power monitoring support

[Alexander Kozyrev]

Support the PMD power management API in MLX5 driver.
The monitor policy of this API puts a CPU core to sleep until
a data in some monitored memory address is changed by the NIC.
Implement the get_monitor_addr function to return an address
of a CQE owner bit to monitor the arrival of a new packet.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
v2: fixed check for empty CQE array

 doc/guides/rel_notes/release_21_05.rst |  4 ++++
 drivers/net/mlx5/mlx5.c                |  2 ++
 drivers/net/mlx5/mlx5_rxtx.c           | 19 +++++++++++++++++++
 drivers/net/mlx5/mlx5_rxtx.h           |  1 +
 4 files changed, 26 insertions(+)

diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst
index 19cec62c73..0894206b14 100644
--- a/doc/guides/rel_notes/release_21_05.rst
+++ b/doc/guides/rel_notes/release_21_05.rst
@@ -97,6 +97,10 @@ New Features
 
   * Added flow filter to support GTPU inner L3/L4 fields matching.
 
+* **Updated Mellanox mlx5 driver.**
+
+  * Added support for the monitor policy of Power Management API.
+
 * **Updated NXP DPAA driver.**
 
   * Added support for shared ethernet interface.
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 9557d06afa..c7ac224fed 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1518,6 +1518,7 @@ const struct eth_dev_ops mlx5_dev_ops = {
 	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
 	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
 	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+	.get_monitor_addr = mlx5_get_monitor_addr,
 };
 
 /* Available operations from secondary process. */
@@ -1602,6 +1603,7 @@ const struct eth_dev_ops mlx5_dev_ops_isolate = {
 	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
 	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
 	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+	.get_monitor_addr = mlx5_get_monitor_addr,
 };
 
 /**
diff --git a/drivers/net/mlx5/mlx5_rxtx.c b/drivers/net/mlx5/mlx5_rxtx.c
index c76b9951bc..1a87acb4d1 100644
--- a/drivers/net/mlx5/mlx5_rxtx.c
+++ b/drivers/net/mlx5/mlx5_rxtx.c
@@ -712,6 +712,25 @@ mlx5_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	return rx_queue_count(rxq);
 }
 
+int mlx5_get_monitor_addr(void *rx_queue, struct rte_power_monitor_cond *pmc)
+{
+	struct mlx5_rxq_data *rxq = rx_queue;
+	const unsigned int cqe_num = 1 << rxq->cqe_n;
+	const unsigned int cqe_mask = cqe_num - 1;
+	const uint16_t idx = rxq->cq_ci & cqe_num;
+	volatile struct mlx5_cqe *cqe = &(*rxq->cqes)[rxq->cq_ci & cqe_mask];
+
+	if (unlikely(rxq->cqes == NULL)) {
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	pmc->addr = &cqe->op_own;
+	pmc->val =  !!idx;
+	pmc->mask = MLX5_CQE_OWNER_MASK;
+	pmc->size = sizeof(uint8_t);
+	return 0;
+}
+
 #define MLX5_SYSTEM_LOG_DIR "/var/log"
 /**
  * Dump debug information to log file.
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 4f0fda0dec..6b8e28cea7 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -442,6 +442,7 @@ int mlx5_rx_burst_mode_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 			   struct rte_eth_burst_mode *mode);
 int mlx5_tx_burst_mode_get(struct rte_eth_dev *dev, uint16_t tx_queue_id,
 			   struct rte_eth_burst_mode *mode);
+int mlx5_get_monitor_addr(void *rx_queue, struct rte_power_monitor_cond *pmc);
 
 /* Vectorized version of mlx5_rxtx.c */
 int mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq_data);
-- 
2.24.1

[dpdk-dev] [PATCH v3] net/mlx5: add power monitoring support

[Alexander Kozyrev]

Support the PMD power management API in MLX5 driver.
The monitor policy of this API puts a CPU core to sleep until
a data in some monitored memory address is changed by the NIC.
Implement the get_monitor_addr function to return an address
of a CQE owner bit to monitor the arrival of a new packet.

Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
---
 doc/guides/rel_notes/release_21_05.rst |  1 +
 drivers/net/mlx5/mlx5.c                |  2 ++
 drivers/net/mlx5/mlx5_rx.c             | 19 +++++++++++++++++++
 drivers/net/mlx5/mlx5_rx.h             |  1 +
 4 files changed, 23 insertions(+)

diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst
index 133a0dbbf2..6957c5420f 100644
--- a/doc/guides/rel_notes/release_21_05.rst
+++ b/doc/guides/rel_notes/release_21_05.rst
@@ -165,6 +165,7 @@ New Features
   * Added support for pre-defined meter policy API.
   * Added support for ASO (Advanced Steering Operation) meter.
   * Added support for ASO metering by PPS (packet per second).
+  * Added support for the monitor policy of Power Management API.
 
 * **Updated NXP DPAA driver.**
 
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index 19ffa16769..8cd6f1eaee 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -1615,6 +1615,7 @@ const struct eth_dev_ops mlx5_dev_ops = {
 	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
 	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
 	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+	.get_monitor_addr = mlx5_get_monitor_addr,
 };
 
 /* Available operations from secondary process. */
@@ -1699,6 +1700,7 @@ const struct eth_dev_ops mlx5_dev_ops_isolate = {
 	.hairpin_queue_peer_update = mlx5_hairpin_queue_peer_update,
 	.hairpin_queue_peer_bind = mlx5_hairpin_queue_peer_bind,
 	.hairpin_queue_peer_unbind = mlx5_hairpin_queue_peer_unbind,
+	.get_monitor_addr = mlx5_get_monitor_addr,
 };
 
 /**
diff --git a/drivers/net/mlx5/mlx5_rx.c b/drivers/net/mlx5/mlx5_rx.c
index e9fcb522e2..6cd71a44eb 100644
--- a/drivers/net/mlx5/mlx5_rx.c
+++ b/drivers/net/mlx5/mlx5_rx.c
@@ -269,6 +269,25 @@ mlx5_rx_queue_count(struct rte_eth_dev *dev, uint16_t rx_queue_id)
 	return rx_queue_count(rxq);
 }
 
+int mlx5_get_monitor_addr(void *rx_queue, struct rte_power_monitor_cond *pmc)
+{
+	struct mlx5_rxq_data *rxq = rx_queue;
+	const unsigned int cqe_num = 1 << rxq->cqe_n;
+	const unsigned int cqe_mask = cqe_num - 1;
+	const uint16_t idx = rxq->cq_ci & cqe_num;
+	volatile struct mlx5_cqe *cqe = &(*rxq->cqes)[rxq->cq_ci & cqe_mask];
+
+	if (unlikely(rxq->cqes == NULL)) {
+		rte_errno = EINVAL;
+		return -rte_errno;
+	}
+	pmc->addr = &cqe->op_own;
+	pmc->val =  !!idx;
+	pmc->mask = MLX5_CQE_OWNER_MASK;
+	pmc->size = sizeof(uint8_t);
+	return 0;
+}
+
 /**
  * Translate RX completion flags to packet type.
  *
diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h
index d5a2de84d1..1b264e5994 100644
--- a/drivers/net/mlx5/mlx5_rx.h
+++ b/drivers/net/mlx5/mlx5_rx.h
@@ -263,6 +263,7 @@ void mlx5_rxq_info_get(struct rte_eth_dev *dev, uint16_t queue_id,
 		       struct rte_eth_rxq_info *qinfo);
 int mlx5_rx_burst_mode_get(struct rte_eth_dev *dev, uint16_t rx_queue_id,
 			   struct rte_eth_burst_mode *mode);
+int mlx5_get_monitor_addr(void *rx_queue, struct rte_power_monitor_cond *pmc);
 
 /* Vectorized version of mlx5_rx.c */
 int mlx5_rxq_check_vec_support(struct mlx5_rxq_data *rxq_data);
-- 
2.18.1

Re: [dpdk-dev] [PATCH v3] net/mlx5: add power monitoring support

[Slava Ovsiienko]

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Thursday, April 29, 2021 17:55
> To: dev@dpdk.org
> Cc: Raslan Darawsheh <rasland@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>;
> leif.y.johansson@ericsson.com
> Subject: [PATCH v3] net/mlx5: add power monitoring support
> 
> Support the PMD power management API in MLX5 driver.
> The monitor policy of this API puts a CPU core to sleep until a data in some
> monitored memory address is changed by the NIC.
> Implement the get_monitor_addr function to return an address of a CQE
> owner bit to monitor the arrival of a new packet.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

Re: [dpdk-dev] [PATCH v3] net/mlx5: add power monitoring support

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: Alexander Kozyrev <akozyrev@nvidia.com>
> Sent: Thursday, April 29, 2021 5:55 PM
> To: dev@dpdk.org
> Cc: Raslan Darawsheh <rasland@nvidia.com>; Matan Azrad
> <matan@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>;
> leif.y.johansson@ericsson.com
> Subject: [PATCH v3] net/mlx5: add power monitoring support
> 
> Support the PMD power management API in MLX5 driver.
> The monitor policy of this API puts a CPU core to sleep until a data in some
> monitored memory address is changed by the NIC.
> Implement the get_monitor_addr function to return an address of a CQE
> owner bit to monitor the arrival of a new packet.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> ---

Patch applied to next-net-mlx,

Kindest regards,
Raslan Darawsheh

Re: [dpdk-dev] [PATCH v3] net/mlx5: add power monitoring support

[Ferruh Yigit]

On 4/29/2021 3:55 PM, Alexander Kozyrev wrote:
> Support the PMD power management API in MLX5 driver.
> The monitor policy of this API puts a CPU core to sleep until
> a data in some monitored memory address is changed by the NIC.
> Implement the get_monitor_addr function to return an address
> of a CQE owner bit to monitor the arrival of a new packet.
> 
> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>

Hi David, Anatoly,

What to you think adding "Power Monitoring" as a new feature in the NIC feature
table? So the drivers supporting it can advertise it.
https://doc.dpdk.org/guides/nics/overview.html

For it need to,
- add "Power Monitoring" to template .ini file,
'.doc/guides/nics/features/default.ini'
- Document it in the features file, '.doc/guides/nics/features.rst'
- Update driver .ini files to advertise the feature, like 'ixgbe.ini'

Re: [dpdk-dev] [PATCH v3] net/mlx5: add power monitoring support

[Thomas Monjalon]

04/05/2021 19:40, Ferruh Yigit:
> On 4/29/2021 3:55 PM, Alexander Kozyrev wrote:
> > Support the PMD power management API in MLX5 driver.
> > The monitor policy of this API puts a CPU core to sleep until
> > a data in some monitored memory address is changed by the NIC.
> > Implement the get_monitor_addr function to return an address
> > of a CQE owner bit to monitor the arrival of a new packet.
> > 
> > Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
> 
> Hi David, Anatoly,
> 
> What to you think adding "Power Monitoring" as a new feature in the NIC feature
> table? So the drivers supporting it can advertise it.
> https://doc.dpdk.org/guides/nics/overview.html
> 
> For it need to,
> - add "Power Monitoring" to template .ini file,
> '.doc/guides/nics/features/default.ini'
> - Document it in the features file, '.doc/guides/nics/features.rst'
> - Update driver .ini files to advertise the feature, like 'ixgbe.ini'

+1

Re: [dpdk-dev] [PATCH v3] net/mlx5: add power monitoring support

[Burakov, Anatoly]

On 04-May-21 6:43 PM, Thomas Monjalon wrote:
> 04/05/2021 19:40, Ferruh Yigit:
>> On 4/29/2021 3:55 PM, Alexander Kozyrev wrote:
>>> Support the PMD power management API in MLX5 driver.
>>> The monitor policy of this API puts a CPU core to sleep until
>>> a data in some monitored memory address is changed by the NIC.
>>> Implement the get_monitor_addr function to return an address
>>> of a CQE owner bit to monitor the arrival of a new packet.
>>>
>>> Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
>>
>> Hi David, Anatoly,
>>
>> What to you think adding "Power Monitoring" as a new feature in the NIC feature
>> table? So the drivers supporting it can advertise it.
>> https://doc.dpdk.org/guides/nics/overview.html
>>
>> For it need to,
>> - add "Power Monitoring" to template .ini file,
>> '.doc/guides/nics/features/default.ini'
>> - Document it in the features file, '.doc/guides/nics/features.rst'
>> - Update driver .ini files to advertise the feature, like 'ixgbe.ini'
> 
> +1
> 
> 

It would be good to have discoverability for this feature. Seems like 
having a first non-Intel user is a good excuse to do just that :)

-- 
Thanks,
Anatoly