DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] l3fwd-power: make interrupt wakeup log thread safe
@ 2020-10-02 12:03 Anatoly Burakov
  2020-10-02 12:07 ` [dpdk-dev] [PATCH v2] " Anatoly Burakov
  0 siblings, 1 reply; 7+ messages in thread
From: Anatoly Burakov @ 2020-10-02 12:03 UTC (permalink / raw)
  To: dev; +Cc: David Hunt, anatoly.burakov

Currently, the interrupt status notification prevents log spam by
remembering whether previous interrupt wakeup was due to traffic or due
to timeout expiring. However, it is a single variable that can
potentially be accessed from multiple threads, so it is not thread-safe.

Fix it by having per-lcore interrupt status.

Fixes: f4d1e19c293d ("examples/l3fwd-power: add Rx interrupt timeout")
Cc: anatoly.burakov@intel.com

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 examples/l3fwd-power/main.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
index d0e6c9bd77..46eac7c3e0 100644
--- a/examples/l3fwd-power/main.c
+++ b/examples/l3fwd-power/main.c
@@ -821,20 +821,23 @@ power_freq_scaleup_heuristic(unsigned lcore_id,
  *  0 on success
  */
 static int
-sleep_until_rx_interrupt(int num)
+sleep_until_rx_interrupt(int num, int lcore)
 {
 	/*
 	 * we want to track when we are woken up by traffic so that we can go
-	 * back to sleep again without log spamming.
+	 * back to sleep again without log spamming. Avoid cache line sharing
+	 * to prevent threads stepping on each others' toes.
 	 */
-	static bool timeout;
+	static struct {
+		bool wakeup;
+	} __rte_cache_aligned status[RTE_MAX_LCORE];
 	struct rte_epoll_event event[num];
 	int n, i;
 	uint16_t port_id;
 	uint8_t queue_id;
 	void *data;
 
-	if (!timeout) {
+	if (!status[lcore].wakeup) {
 		RTE_LOG(INFO, L3FWD_POWER,
 				"lcore %u sleeps until interrupt triggers\n",
 				rte_lcore_id());
@@ -851,7 +854,7 @@ sleep_until_rx_interrupt(int num)
 			" port %d queue %d\n",
 			rte_lcore_id(), port_id, queue_id);
 	}
-	timeout = n == 0;
+	status[lcore].wakeup = n == 0;
 
 	return 0;
 }
@@ -1050,7 +1053,8 @@ static int main_intr_loop(__rte_unused void *dummy)
 				if (intr_en) {
 					turn_on_off_intr(qconf, 1);
 					sleep_until_rx_interrupt(
-							qconf->n_rx_queue);
+							qconf->n_rx_queue,
+							lcore_id);
 					turn_on_off_intr(qconf, 0);
 					/**
 					 * start receiving packets immediately
@@ -1473,7 +1477,8 @@ main_legacy_loop(__rte_unused void *dummy)
 				if (intr_en) {
 					turn_on_off_intr(qconf, 1);
 					sleep_until_rx_interrupt(
-						qconf->n_rx_queue);
+							qconf->n_rx_queue,
+							lcore_id);
 					turn_on_off_intr(qconf, 0);
 					/**
 					 * start receiving packets immediately
-- 
2.17.1

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log thread safe
  2020-10-02 12:03 [dpdk-dev] [PATCH] l3fwd-power: make interrupt wakeup log thread safe Anatoly Burakov
@ 2020-10-02 12:07 ` Anatoly Burakov
  2020-10-07 15:06   ` David Hunt
                     ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Anatoly Burakov @ 2020-10-02 12:07 UTC (permalink / raw)
  To: dev; +Cc: David Hunt, reshma.pattan, anatoly.burakov

Currently, the interrupt status notification prevents log spam by
remembering whether previous interrupt wakeup was due to traffic or due
to timeout expiring. However, it is a single variable that can
potentially be accessed from multiple threads, so it is not thread-safe.

Fix it by having per-lcore interrupt status.

Fixes: f4d1e19c293d ("examples/l3fwd-power: add Rx interrupt timeout")
Cc: anatoly.burakov@intel.com

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---

Notes:
    v2:
    - Fix confusing variable naming

 examples/l3fwd-power/main.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
index d0e6c9bd77..526af0db29 100644
--- a/examples/l3fwd-power/main.c
+++ b/examples/l3fwd-power/main.c
@@ -821,20 +821,23 @@ power_freq_scaleup_heuristic(unsigned lcore_id,
  *  0 on success
  */
 static int
-sleep_until_rx_interrupt(int num)
+sleep_until_rx_interrupt(int num, int lcore)
 {
 	/*
 	 * we want to track when we are woken up by traffic so that we can go
-	 * back to sleep again without log spamming.
+	 * back to sleep again without log spamming. Avoid cache line sharing
+	 * to prevent threads stepping on each others' toes.
 	 */
-	static bool timeout;
+	static struct {
+		bool wakeup;
+	} __rte_cache_aligned status[RTE_MAX_LCORE];
 	struct rte_epoll_event event[num];
 	int n, i;
 	uint16_t port_id;
 	uint8_t queue_id;
 	void *data;
 
-	if (!timeout) {
+	if (status[lcore].wakeup) {
 		RTE_LOG(INFO, L3FWD_POWER,
 				"lcore %u sleeps until interrupt triggers\n",
 				rte_lcore_id());
@@ -851,7 +854,7 @@ sleep_until_rx_interrupt(int num)
 			" port %d queue %d\n",
 			rte_lcore_id(), port_id, queue_id);
 	}
-	timeout = n == 0;
+	status[lcore].wakeup = n != 0;
 
 	return 0;
 }
@@ -1050,7 +1053,8 @@ static int main_intr_loop(__rte_unused void *dummy)
 				if (intr_en) {
 					turn_on_off_intr(qconf, 1);
 					sleep_until_rx_interrupt(
-							qconf->n_rx_queue);
+							qconf->n_rx_queue,
+							lcore_id);
 					turn_on_off_intr(qconf, 0);
 					/**
 					 * start receiving packets immediately
@@ -1473,7 +1477,8 @@ main_legacy_loop(__rte_unused void *dummy)
 				if (intr_en) {
 					turn_on_off_intr(qconf, 1);
 					sleep_until_rx_interrupt(
-						qconf->n_rx_queue);
+							qconf->n_rx_queue,
+							lcore_id);
 					turn_on_off_intr(qconf, 0);
 					/**
 					 * start receiving packets immediately
-- 
2.17.1

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log thread safe
  2020-10-02 12:07 ` [dpdk-dev] [PATCH v2] " Anatoly Burakov
@ 2020-10-07 15:06   ` David Hunt
  2020-11-02 10:35     ` Xie, WeiX
  2020-10-09  6:26   ` Xie, WeiX
  2020-11-02 10:46   ` David Marchand
  2 siblings, 1 reply; 7+ messages in thread
From: David Hunt @ 2020-10-07 15:06 UTC (permalink / raw)
  To: Anatoly Burakov, dev; +Cc: reshma.pattan

Hi Anatoly,

On 2/10/2020 1:07 PM, Anatoly Burakov wrote:
> Currently, the interrupt status notification prevents log spam by
> remembering whether previous interrupt wakeup was due to traffic or due
> to timeout expiring. However, it is a single variable that can
> potentially be accessed from multiple threads, so it is not thread-safe.
>
> Fix it by having per-lcore interrupt status.
>
> Fixes: f4d1e19c293d ("examples/l3fwd-power: add Rx interrupt timeout")
> Cc: anatoly.burakov@intel.com
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---


Makes sense. Thanks.

Acked-by: David Hunt <david.hunt@intel.com>





^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log thread safe
  2020-10-02 12:07 ` [dpdk-dev] [PATCH v2] " Anatoly Burakov
  2020-10-07 15:06   ` David Hunt
@ 2020-10-09  6:26   ` Xie, WeiX
  2020-10-30 12:46     ` David Marchand
  2020-11-02 10:46   ` David Marchand
  2 siblings, 1 reply; 7+ messages in thread
From: Xie, WeiX @ 2020-10-09  6:26 UTC (permalink / raw)
  To: Burakov, Anatoly, dev; +Cc: Hunt, David, Pattan, Reshma, Burakov, Anatoly

Tested-by: Zhang, XiX <xix.zhang@intel.com>

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anatoly Burakov
> Sent: Friday, October 2, 2020 8:07 PM
> To: dev@dpdk.org
> Cc: Hunt, David <david.hunt@intel.com>; Pattan, Reshma
> <reshma.pattan@intel.com>; Burakov, Anatoly <anatoly.burakov@intel.com>
> Subject: [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log
> thread safe


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log thread safe
  2020-10-09  6:26   ` Xie, WeiX
@ 2020-10-30 12:46     ` David Marchand
  0 siblings, 0 replies; 7+ messages in thread
From: David Marchand @ 2020-10-30 12:46 UTC (permalink / raw)
  To: Xie, WeiX; +Cc: Burakov, Anatoly, dev, Hunt, David, Pattan, Reshma

On Fri, Oct 9, 2020 at 8:27 AM Xie, WeiX <weix.xie@intel.com> wrote:
>
> Tested-by: Zhang, XiX <xix.zhang@intel.com>

A bit surprised to read a Tested-by: token sent for someone else.
Is this a typo?

Besides, it should be "Xi Zhang", no comma.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log thread safe
  2020-10-07 15:06   ` David Hunt
@ 2020-11-02 10:35     ` Xie, WeiX
  0 siblings, 0 replies; 7+ messages in thread
From: Xie, WeiX @ 2020-11-02 10:35 UTC (permalink / raw)
  To: Hunt, David, Burakov, Anatoly, dev; +Cc: Pattan, Reshma

Tested-by: Zhang, XiX <xix.zhang@intel.com>

The tag is correct and I  send the email on behalf of him, his mailbox can't receive dev patch. Thx.


Regards,
Xie Wei


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of David Hunt
> Sent: Wednesday, October 7, 2020 11:06 PM
> To: Burakov, Anatoly <anatoly.burakov@intel.com>; dev@dpdk.org
> Cc: Pattan, Reshma <reshma.pattan@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log
> thread safe
> 
> Hi Anatoly,
> 
> On 2/10/2020 1:07 PM, Anatoly Burakov wrote:
> > Currently, the interrupt status notification prevents log spam by
> > remembering whether previous interrupt wakeup was due to traffic or
> > due to timeout expiring. However, it is a single variable that can
> > potentially be accessed from multiple threads, so it is not thread-safe.
> >
> > Fix it by having per-lcore interrupt status.
> >
> > Fixes: f4d1e19c293d ("examples/l3fwd-power: add Rx interrupt timeout")
> > Cc: anatoly.burakov@intel.com
> >
> > Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> > ---
> 
> 
> Makes sense. Thanks.
> 
> Acked-by: David Hunt <david.hunt@intel.com>
> 
> 
> 


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH v2] l3fwd-power: make interrupt wakeup log thread safe
  2020-10-02 12:07 ` [dpdk-dev] [PATCH v2] " Anatoly Burakov
  2020-10-07 15:06   ` David Hunt
  2020-10-09  6:26   ` Xie, WeiX
@ 2020-11-02 10:46   ` David Marchand
  2 siblings, 0 replies; 7+ messages in thread
From: David Marchand @ 2020-11-02 10:46 UTC (permalink / raw)
  To: Anatoly Burakov; +Cc: dev, David Hunt, Pattan, Reshma

On Fri, Oct 2, 2020 at 2:07 PM Anatoly Burakov
<anatoly.burakov@intel.com> wrote:
>
> Currently, the interrupt status notification prevents log spam by
> remembering whether previous interrupt wakeup was due to traffic or due
> to timeout expiring. However, it is a single variable that can
> potentially be accessed from multiple threads, so it is not thread-safe.
>
> Fix it by having per-lcore interrupt status.
>
> Fixes: f4d1e19c293d ("examples/l3fwd-power: add Rx interrupt timeout")
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>
Tested-by: Xi Zhang <xix.zhang@intel.com>

Applied, thanks.

-- 
David Marchand


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2020-11-02 10:46 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-02 12:03 [dpdk-dev] [PATCH] l3fwd-power: make interrupt wakeup log thread safe Anatoly Burakov
2020-10-02 12:07 ` [dpdk-dev] [PATCH v2] " Anatoly Burakov
2020-10-07 15:06   ` David Hunt
2020-11-02 10:35     ` Xie, WeiX
2020-10-09  6:26   ` Xie, WeiX
2020-10-30 12:46     ` David Marchand
2020-11-02 10:46   ` David Marchand

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git