DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end
@ 2020-12-21  5:34 Feifei Wang
  2020-12-21  9:32 ` [dpdk-dev] [PATCH v2] " Feifei Wang
                   ` (3 more replies)
  0 siblings, 4 replies; 14+ messages in thread
From: Feifei Wang @ 2020-12-21  5:34 UTC (permalink / raw)
  To: Harry van Haaren, Nikhil Rao, Pavan Nikhilesh
  Cc: dev, nd, Feifei Wang, pbhagavatula, stable

Move eth stop code from "signal_handler" function to the end of "main"
function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and clear.
Based on this change, after receiving the interrupt signal, "fdata->done"
is set as 1. Then the main thread will wait all worker lcores to jump out
of the loop. Finally, the main thread will stop and then close eth dev port.

Second, for older version, the main thread first stops eth dev port and then
waits the end of worker lcore. This may cause errors because it may stop the
eth dev port which worker lcores are using. This moving change can fix this
by waiting all worker threads to exit and then stop the eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1
into fdata->done. And then the worker lcores load "fdata->done" and jump out
of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
Cc: pbhagavatula@marvell.com
Cc: stable@dpdk.org

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
---
 examples/eventdev_pipeline/main.c | 15 ++++-----------
 1 file changed, 4 insertions(+), 11 deletions(-)

diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c
index 823f8b51c..ac2ace5c4 100644
--- a/examples/eventdev_pipeline/main.c
+++ b/examples/eventdev_pipeline/main.c
@@ -291,17 +291,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -465,6 +454,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] [PATCH v2] examples/eventdev: move eth stop to the end
  2020-12-21  5:34 [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end Feifei Wang
@ 2020-12-21  9:32 ` Feifei Wang
  2020-12-21  9:56 ` [dpdk-dev] [PATCH] " Van Haaren, Harry
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 14+ messages in thread
From: Feifei Wang @ 2020-12-21  9:32 UTC (permalink / raw)
  To: Harry van Haaren, Nikhil Rao, Pavan Nikhilesh
  Cc: dev, nd, Feifei Wang, pbhagavatula, stable

Move eth stop code from "signal_handler" function to the end of "main"
function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and
clear. Based on this change, after receiving the interrupt signal,
"fdata->done" is set as 1. Then the main thread will wait all worker
lcores to jump out of the loop. Finally, the main thread will stop and
then close eth dev port.

Second, for older version, the main thread first stops eth dev port and
then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the
eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1
into fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
Cc: pbhagavatula@marvell.com
Cc: stable@dpdk.org

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
---

v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues

 examples/eventdev_pipeline/main.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c
index 823f8b51c..fdbaf667b 100644
--- a/examples/eventdev_pipeline/main.c
+++ b/examples/eventdev_pipeline/main.c
@@ -280,7 +280,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -291,17 +290,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -465,6 +453,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end
  2020-12-21  5:34 [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end Feifei Wang
  2020-12-21  9:32 ` [dpdk-dev] [PATCH v2] " Feifei Wang
@ 2020-12-21  9:56 ` Van Haaren, Harry
  2020-12-22  5:31   ` [dpdk-dev] 回复: " Feifei Wang
  2021-01-05  5:14 ` [dpdk-dev] [PATCH v3] examples/eventdev: refactor ethdev port stop Feifei Wang
  2021-01-14 10:30 ` [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev Feifei Wang
  3 siblings, 1 reply; 14+ messages in thread
From: Van Haaren, Harry @ 2020-12-21  9:56 UTC (permalink / raw)
  To: Feifei Wang, Rao, Nikhil; +Cc: dev, nd, pbhagavatula, stable, jerinj

> -----Original Message-----
> From: Feifei Wang <feifei.wang2@arm.com>
> Sent: Monday, December 21, 2020 5:35 AM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>; Rao, Nikhil
> <nikhil.rao@intel.com>; Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
> Cc: dev@dpdk.org; nd@arm.com; Feifei Wang <feifei.wang2@arm.com>;
> pbhagavatula@marvell.com; stable@dpdk.org
> Subject: [PATCH] examples/eventdev: move eth stop to the end

Suggested title improvement?
examples/eventdev_pipeline: refactor ethdev port stop

> Move eth stop code from "signal_handler" function to the end of "main"
> function. There are two reasons for this:
> 
> First, this improves code maintenance and makes code look simple and clear.
> Based on this change, after receiving the interrupt signal, "fdata->done"
> is set as 1. Then the main thread will wait all worker lcores to jump out
> of the loop. Finally, the main thread will stop and then close eth dev port.
> 
> Second, for older version, the main thread first stops eth dev port and then
> waits the end of worker lcore. This may cause errors because it may stop the
> eth dev port which worker lcores are using. This moving change can fix this
> by waiting all worker threads to exit and then stop the eth dev port.

I'm OK with the above changes, and agree that moving eth dev port close to
after lcores return is a worthy change.

> In the meanwhile, remove wmb in signal_handler.
> 
> This is because when the main lcore receive the stop signal, it stores 1
> into fdata->done. And then the worker lcores load "fdata->done" and jump out
> of the loop to stop running. Nothing should be stored after updating
> fdata->done, so the wmb is unnecessary.
>
> Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
> Cc: pbhagavatula@marvell.com
> Cc: stable@dpdk.org
> 
> Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>

+Cc Jerin for Eventdev tree;
-Cc Pavan's old email address

Ack-ed by: Harry van Haaren <harry.van.haaren@intel.com>

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] 回复: [PATCH] examples/eventdev: move eth stop to the end
  2020-12-21  9:56 ` [dpdk-dev] [PATCH] " Van Haaren, Harry
@ 2020-12-22  5:31   ` Feifei Wang
  0 siblings, 0 replies; 14+ messages in thread
From: Feifei Wang @ 2020-12-22  5:31 UTC (permalink / raw)
  To: Van Haaren, Harry, Rao, Nikhil; +Cc: dev, nd, pbhagavatula, stable, jerinj, nd

Hi, Van Haaren

> -----邮件原件-----
> 发件人: Van Haaren, Harry <harry.van.haaren@intel.com>
> 发送时间: 2020年12月21日 17:57
> 收件人: Feifei Wang <Feifei.Wang2@arm.com>; Rao, Nikhil
> <nikhil.rao@intel.com>
> 抄送: dev@dpdk.org; nd <nd@arm.com>; pbhagavatula@marvell.com;
> stable@dpdk.org; jerinj@marvell.com
> 主题: RE: [PATCH] examples/eventdev: move eth stop to the end
> 
> > -----Original Message-----
> > From: Feifei Wang <feifei.wang2@arm.com>
> > Sent: Monday, December 21, 2020 5:35 AM
> > To: Van Haaren, Harry <harry.van.haaren@intel.com>; Rao, Nikhil
> > <nikhil.rao@intel.com>; Pavan Nikhilesh
> > <pbhagavatula@caviumnetworks.com>
> > Cc: dev@dpdk.org; nd@arm.com; Feifei Wang <feifei.wang2@arm.com>;
> > pbhagavatula@marvell.com; stable@dpdk.org
> > Subject: [PATCH] examples/eventdev: move eth stop to the end
> 
> Suggested title improvement?
> examples/eventdev_pipeline: refactor ethdev port stop
That's OK. I will apply this in the next version.
> 
> > Move eth stop code from "signal_handler" function to the end of "main"
> > function. There are two reasons for this:
> >
> > First, this improves code maintenance and makes code look simple and
> clear.
> > Based on this change, after receiving the interrupt signal, "fdata->done"
> > is set as 1. Then the main thread will wait all worker lcores to jump
> > out of the loop. Finally, the main thread will stop and then close eth dev
> port.
> >
> > Second, for older version, the main thread first stops eth dev port
> > and then waits the end of worker lcore. This may cause errors because
> > it may stop the eth dev port which worker lcores are using. This
> > moving change can fix this by waiting all worker threads to exit and then
> stop the eth dev port.
> 
> I'm OK with the above changes, and agree that moving eth dev port close to
> after lcores return is a worthy change.
> 
> > In the meanwhile, remove wmb in signal_handler.
> >
> > This is because when the main lcore receive the stop signal, it stores
> > 1 into fdata->done. And then the worker lcores load "fdata->done" and
> > jump out of the loop to stop running. Nothing should be stored after
> > updating
> > fdata->done, so the wmb is unnecessary.
> >
> > Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
> > Cc: pbhagavatula@marvell.com
> > Cc: stable@dpdk.org
> >
> > Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> > Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
> 
> +Cc Jerin for Eventdev tree;
> -Cc Pavan's old email address
Thanks very much for this change.
> 
> Ack-ed by: Harry van Haaren <harry.van.haaren@intel.com>

Best Regards
Feifei

^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] [PATCH v3] examples/eventdev: refactor ethdev port stop
  2020-12-21  5:34 [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end Feifei Wang
  2020-12-21  9:32 ` [dpdk-dev] [PATCH v2] " Feifei Wang
  2020-12-21  9:56 ` [dpdk-dev] [PATCH] " Van Haaren, Harry
@ 2021-01-05  5:14 ` Feifei Wang
  2021-01-05 10:09   ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
  2021-01-14 10:30 ` [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev Feifei Wang
  3 siblings, 1 reply; 14+ messages in thread
From: Feifei Wang @ 2021-01-05  5:14 UTC (permalink / raw)
  To: Harry van Haaren, Nikhil Rao, Pavan Nikhilesh
  Cc: dev, jerinj, nd, Feifei Wang, pbhagavatula, stable, Ruifeng Wang,
	Honnappa Nagarahalli

Move eth stop code from "signal_handler" function to the end of "main"
function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and
clear. Based on this change, after receiving the interrupt signal,
"fdata->done" is set as 1. Then the main thread will wait all worker
lcores to jump out of the loop. Finally, the main thread will stop and
then close eth dev port.

Second, for older version, the main thread first stops eth dev port and
then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the
eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1
into fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
Cc: pbhagavatula@marvell.com
Cc: stable@dpdk.org

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
---

v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues

v3:
1. Title improvement (Van Haaren)

 examples/eventdev_pipeline/main.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c
index 823f8b51c..fdbaf667b 100644
--- a/examples/eventdev_pipeline/main.c
+++ b/examples/eventdev_pipeline/main.c
@@ -280,7 +280,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -291,17 +290,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -465,6 +453,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [dpdk-dev] [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
  2021-01-05  5:14 ` [dpdk-dev] [PATCH v3] examples/eventdev: refactor ethdev port stop Feifei Wang
@ 2021-01-05 10:09   ` Pavan Nikhilesh Bhagavatula
  2021-01-14  6:24     ` [dpdk-dev] 回复: " Feifei Wang
  0 siblings, 1 reply; 14+ messages in thread
From: Pavan Nikhilesh Bhagavatula @ 2021-01-05 10:09 UTC (permalink / raw)
  To: Feifei Wang, Harry van Haaren, Nikhil Rao, Pavan Nikhilesh
  Cc: dev, Jerin Jacob Kollanukkaran, nd, stable, Ruifeng Wang,
	Honnappa Nagarahalli

Hi Feifei,

>Move eth stop code from "signal_handler" function to the end of
>"main"
>function. There are two reasons for this:
>
>First, this improves code maintenance and makes code look simple and
>clear. Based on this change, after receiving the interrupt signal,
>"fdata->done" is set as 1. Then the main thread will wait all worker
>lcores to jump out of the loop. Finally, the main thread will stop and
>then close eth dev port.
>
>Second, for older version, the main thread first stops eth dev port and
>then waits the end of worker lcore. This may cause errors because it
>may
>stop the eth dev port which worker lcores are using. This moving
>change
>can fix this by waiting all worker threads to exit and then stop the
>eth dev port.

Apologies for the delayed reply,

In case of event dev the workers don't interact with eth device directly, 
Instead eth device "injects" packets into event device and event device
is responsible for scheduling them to the workers.

If the producer is not stopped i.e. in this case eth device then the worker
threads might never exit and the main core would wait indefinitely for
workers to exit. This will be predominantly seen in cases where there are
only a few flows and large number of workers causing a lot of intra thread
dependency.

Regards,
Pavan.

>
>In the meanwhile, remove wmb in signal_handler.
>
>This is because when the main lcore receive the stop signal, it stores 1
>into fdata->done. And then the worker lcores load "fdata->done" and
>jump
>out of the loop to stop running. Nothing should be stored after
>updating
>fdata->done, so the wmb is unnecessary.
>
>Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx
>adapter")
>Cc: pbhagavatula@marvell.com
>Cc: stable@dpdk.org
>
>Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
>Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
>Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>Reviewed-by: Honnappa Nagarahalli
><honnappa.nagarahalli@arm.com>
>Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
>---
>
>v2:
>1. Delete unused variable to fix build error
>2. Reduce commit message length to fix coding style issues
>
>v3:
>1. Title improvement (Van Haaren)
>
> examples/eventdev_pipeline/main.c | 16 ++++------------
> 1 file changed, 4 insertions(+), 12 deletions(-)
>
>diff --git a/examples/eventdev_pipeline/main.c
>b/examples/eventdev_pipeline/main.c
>index 823f8b51c..fdbaf667b 100644
>--- a/examples/eventdev_pipeline/main.c
>+++ b/examples/eventdev_pipeline/main.c
>@@ -280,7 +280,6 @@ static void
> signal_handler(int signum)
> {
> 	static uint8_t once;
>-	uint16_t portid;
>
> 	if (fdata->done)
> 		rte_exit(1, "Exiting on signal %d\n", signum);
>@@ -291,17 +290,6 @@ signal_handler(int signum)
> 			rte_event_dev_dump(0, stdout);
> 		once = 1;
> 		fdata->done = 1;
>-		rte_smp_wmb();
>-
>-		RTE_ETH_FOREACH_DEV(portid) {
>-			rte_event_eth_rx_adapter_stop(portid);
>-			rte_event_eth_tx_adapter_stop(portid);
>-			if (rte_eth_dev_stop(portid) < 0)
>-				printf("Failed to stop port %u", portid);
>-		}
>-
>-		rte_eal_mp_wait_lcore();
>-
> 	}
> 	if (signum == SIGTSTP)
> 		rte_event_dev_dump(0, stdout);
>@@ -465,6 +453,10 @@ main(int argc, char **argv)
> 	}
>
> 	RTE_ETH_FOREACH_DEV(portid) {
>+		rte_event_eth_rx_adapter_stop(portid);
>+		rte_event_eth_tx_adapter_stop(portid);
>+		if (rte_eth_dev_stop(portid) < 0)
>+			printf("Failed to stop port %u", portid);
> 		rte_eth_dev_close(portid);
> 	}
>
>--
>2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
  2021-01-05 10:09   ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
@ 2021-01-14  6:24     ` Feifei Wang
  2021-01-14  8:50       ` Feifei Wang
  0 siblings, 1 reply; 14+ messages in thread
From: Feifei Wang @ 2021-01-14  6:24 UTC (permalink / raw)
  To: Pavan Nikhilesh Bhagavatula, Harry van Haaren, Nikhil Rao,
	Pavan Nikhilesh
  Cc: dev, jerinj, nd, stable, Ruifeng Wang, Honnappa Nagarahalli, nd

> -----邮件原件-----
> 发件人: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>
> 发送时间: 2021年1月5日 18:09
> 收件人: Feifei Wang <Feifei.Wang2@arm.com>; Harry van Haaren
> <harry.van.haaren@intel.com>; Nikhil Rao <nikhil.rao@intel.com>; Pavan
> Nikhilesh <pbhagavatula@caviumnetworks.com>
> 抄送: dev@dpdk.org; jerinj@marvell.com; nd <nd@arm.com>;
> stable@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>
> 主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
> 
Hi, Pavan

> Hi Feifei,
> 
> >Move eth stop code from "signal_handler" function to the end of "main"
> >function. There are two reasons for this:
> >
> >First, this improves code maintenance and makes code look simple and
> >clear. Based on this change, after receiving the interrupt signal,
> >"fdata->done" is set as 1. Then the main thread will wait all worker
> >lcores to jump out of the loop. Finally, the main thread will stop and
> >then close eth dev port.
> >
> >Second, for older version, the main thread first stops eth dev port and
> >then waits the end of worker lcore. This may cause errors because it
> >may stop the eth dev port which worker lcores are using. This moving
> >change can fix this by waiting all worker threads to exit and then stop
> >the eth dev port.
> 
> Apologies for the delayed reply,
> 
> In case of event dev the workers don't interact with eth device directly,
> Instead eth device "injects" packets into event device and event device is
> responsible for scheduling them to the workers.
> 
> If the producer is not stopped i.e. in this case eth device then the worker
> threads might never exit and the main core would wait indefinitely for
> workers to exit. This will be predominantly seen in cases where there are
> only a few flows and large number of workers causing a lot of intra thread
> dependency.

For the case that the event device scheduling packets to the workers. Though the producer
is not stopped (eth device), when the main core receive the interrupt signal, the "fdata->done"
will be set as 1. Then all the workers load the value 1 of "fdata->done" and jump out of the loop
to finish their thread.

Best Regards
Feifei
> 
> Regards,
> Pavan.
> 
> >
> >In the meanwhile, remove wmb in signal_handler.
> >
> >This is because when the main lcore receive the stop signal, it stores
> >1 into fdata->done. And then the worker lcores load "fdata->done" and
> >jump out of the loop to stop running. Nothing should be stored after
> >updating
> >fdata->done, so the wmb is unnecessary.
> >
> >Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx
> >adapter")
> >Cc: pbhagavatula@marvell.com
> >Cc: stable@dpdk.org
> >
> >Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> >Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >Reviewed-by: Honnappa Nagarahalli
> ><honnappa.nagarahalli@arm.com>
> >Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
> >---
> >
> >v2:
> >1. Delete unused variable to fix build error 2. Reduce commit message
> >length to fix coding style issues
> >
> >v3:
> >1. Title improvement (Van Haaren)
> >
> > examples/eventdev_pipeline/main.c | 16 ++++------------
> > 1 file changed, 4 insertions(+), 12 deletions(-)
> >
> >diff --git a/examples/eventdev_pipeline/main.c
> >b/examples/eventdev_pipeline/main.c
> >index 823f8b51c..fdbaf667b 100644
> >--- a/examples/eventdev_pipeline/main.c
> >+++ b/examples/eventdev_pipeline/main.c
> >@@ -280,7 +280,6 @@ static void
> > signal_handler(int signum)
> > {
> > 	static uint8_t once;
> >-	uint16_t portid;
> >
> > 	if (fdata->done)
> > 		rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17
> +290,6 @@
> >signal_handler(int signum)
> > 			rte_event_dev_dump(0, stdout);
> > 		once = 1;
> > 		fdata->done = 1;
> >-		rte_smp_wmb();
> >-
> >-		RTE_ETH_FOREACH_DEV(portid) {
> >-			rte_event_eth_rx_adapter_stop(portid);
> >-			rte_event_eth_tx_adapter_stop(portid);
> >-			if (rte_eth_dev_stop(portid) < 0)
> >-				printf("Failed to stop port %u", portid);
> >-		}
> >-
> >-		rte_eal_mp_wait_lcore();
> >-
> > 	}
> > 	if (signum == SIGTSTP)
> > 		rte_event_dev_dump(0, stdout);
> >@@ -465,6 +453,10 @@ main(int argc, char **argv)
> > 	}
> >
> > 	RTE_ETH_FOREACH_DEV(portid) {
> >+		rte_event_eth_rx_adapter_stop(portid);
> >+		rte_event_eth_tx_adapter_stop(portid);
> >+		if (rte_eth_dev_stop(portid) < 0)
> >+			printf("Failed to stop port %u", portid);
> > 		rte_eth_dev_close(portid);
> > 	}
> >
> >--
> >2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
  2021-01-14  6:24     ` [dpdk-dev] 回复: " Feifei Wang
@ 2021-01-14  8:50       ` Feifei Wang
  0 siblings, 0 replies; 14+ messages in thread
From: Feifei Wang @ 2021-01-14  8:50 UTC (permalink / raw)
  To: Pavan Nikhilesh Bhagavatula, Harry van Haaren, Nikhil Rao,
	Pavan Nikhilesh
  Cc: dev, jerinj, nd, stable, Ruifeng Wang, Honnappa Nagarahalli, nd, nd



> -----邮件原件-----
> 发件人: Feifei Wang
> 发送时间: 2021年1月14日 14:24
> 收件人: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Pavan Nikhilesh
> <pbhagavatula@caviumnetworks.com>
> 抄送: dev@dpdk.org; jerinj@marvell.com; nd <nd@arm.com>;
> stable@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> 主题: 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
> 
> > -----邮件原件-----
> > 发件人: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>
> > 发送时间: 2021年1月5日 18:09
> > 收件人: Feifei Wang <Feifei.Wang2@arm.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Nikhil Rao <nikhil.rao@intel.com>; Pavan
> > Nikhilesh <pbhagavatula@caviumnetworks.com>
> > 抄送: dev@dpdk.org; jerinj@marvell.com; nd <nd@arm.com>;
> > stable@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa
> > Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > 主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
> >
> Hi, Pavan
> 
> > Hi Feifei,
> >
> > >Move eth stop code from "signal_handler" function to the end of "main"
> > >function. There are two reasons for this:
> > >
> > >First, this improves code maintenance and makes code look simple and
> > >clear. Based on this change, after receiving the interrupt signal,
> > >"fdata->done" is set as 1. Then the main thread will wait all worker
> > >lcores to jump out of the loop. Finally, the main thread will stop
> > >and then close eth dev port.
> > >
> > >Second, for older version, the main thread first stops eth dev port
> > >and then waits the end of worker lcore. This may cause errors because
> > >it may stop the eth dev port which worker lcores are using. This
> > >moving change can fix this by waiting all worker threads to exit and
> > >then stop the eth dev port.
> >
> > Apologies for the delayed reply,
> >
> > In case of event dev the workers don't interact with eth device
> > directly, Instead eth device "injects" packets into event device and
> > event device is responsible for scheduling them to the workers.
> >
> > If the producer is not stopped i.e. in this case eth device then the
> > worker threads might never exit and the main core would wait
> > indefinitely for workers to exit. This will be predominantly seen in
> > cases where there are only a few flows and large number of workers
> > causing a lot of intra thread dependency.
> 
> For the case that the event device scheduling packets to the workers.
> Though the producer is not stopped (eth device), when the main core
> receive the interrupt signal, the "fdata->done"
> will be set as 1. Then all the workers load the value 1 of "fdata->done" and
> jump out of the loop to finish their thread.
> 

And I also test the case you said above with this patch, when I send interrupt signal(ctrl + c), IXIA
is running and eth device is also working, then the program can exit normally.

HW:
Traffic generator: IXIA 
Nics: ixgbe 82599ES 10-Gigabit
Architecture: aarch64
CPU: Cortex-A72

With this patch:
______________________________________________________________________________________________
Command: sudo ./dpdk-eventdev_pipeline -l 0-2,8-11 --vdev=event_sw0 -- -r2 -t2 -e4 -w F00 -s4 -n0 -c32 -W1000 -D
.........
Config:
        ports: 2
        workers: 4
        packets: 9223372036854775807
        Queue-prio: 0
        qid0 type: atomic
        Cores available: 7
        Cores used: 6
        Eventdev 0: event_sw
  Stages:
        Stage 0, Type Atomic    Priority = 128
        Stage 1, Type Atomic    Priority = 128
        Stage 2, Type Atomic    Priority = 128
        Stage 3, Type Atomic    Priority = 128

Port 0 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34
Port 0 MAC: 90 e2 ba 56 ed 6c
Port 1 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34
Port 1 MAC: 90 e2 ba 56 ed 6d
[dump_core_info()] lcore 1 executing NIC Rx
[dump_core_info()] lcore 1 executing NIC Tx
[dump_core_info()] lcore 2 executing scheduler
[dump_core_info()] lcore 8 executing worker, using eventdev port 0
[dump_core_info()] lcore 9 executing worker, using eventdev port 1
[dump_core_info()] lcore 10 executing worker, using eventdev port 2
[dump_core_info()] lcore 11 executing worker, using eventdev port 3


Command: ctrl + c
.........
  worker 1 thread done. RX=0 TX=0
  worker 2 thread done. RX=0 TX=0
  worker 11 thread done. RX=1844023 TX=1844023
  worker 9 thread done. RX=1844888 TX=1844888
  worker 8 thread done. RX=1845490 TX=1845490
  worker 10 thread done. RX=1844903 TX=1844903

Port Workload distribution:
worker 0 :      25.0 % (1845474 pkts)
worker 1 :      25.0 % (1844872 pkts)
worker 2 :      25.0 % (1844887 pkts)
worker 3 :      25.0 % (1844007 pkts)
Invalid port_id=0
Invalid port_id=1
______________________________________________________________________________________________

> Best Regards
> Feifei
> >
> > Regards,
> > Pavan.
> >
> > >
> > >In the meanwhile, remove wmb in signal_handler.
> > >
> > >This is because when the main lcore receive the stop signal, it
> > >stores
> > >1 into fdata->done. And then the worker lcores load "fdata->done" and
> > >jump out of the loop to stop running. Nothing should be stored after
> > >updating
> > >fdata->done, so the wmb is unnecessary.
> > >
> > >Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx
> > >adapter")
> > >Cc: pbhagavatula@marvell.com
> > >Cc: stable@dpdk.org
> > >
> > >Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> > >Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >Reviewed-by: Honnappa Nagarahalli
> > ><honnappa.nagarahalli@arm.com>
> > >Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
> > >---
> > >
> > >v2:
> > >1. Delete unused variable to fix build error 2. Reduce commit message
> > >length to fix coding style issues
> > >
> > >v3:
> > >1. Title improvement (Van Haaren)
> > >
> > > examples/eventdev_pipeline/main.c | 16 ++++------------
> > > 1 file changed, 4 insertions(+), 12 deletions(-)
> > >
> > >diff --git a/examples/eventdev_pipeline/main.c
> > >b/examples/eventdev_pipeline/main.c
> > >index 823f8b51c..fdbaf667b 100644
> > >--- a/examples/eventdev_pipeline/main.c
> > >+++ b/examples/eventdev_pipeline/main.c
> > >@@ -280,7 +280,6 @@ static void
> > > signal_handler(int signum)
> > > {
> > > 	static uint8_t once;
> > >-	uint16_t portid;
> > >
> > > 	if (fdata->done)
> > > 		rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17
> > +290,6 @@
> > >signal_handler(int signum)
> > > 			rte_event_dev_dump(0, stdout);
> > > 		once = 1;
> > > 		fdata->done = 1;
> > >-		rte_smp_wmb();
> > >-
> > >-		RTE_ETH_FOREACH_DEV(portid) {
> > >-			rte_event_eth_rx_adapter_stop(portid);
> > >-			rte_event_eth_tx_adapter_stop(portid);
> > >-			if (rte_eth_dev_stop(portid) < 0)
> > >-				printf("Failed to stop port %u", portid);
> > >-		}
> > >-
> > >-		rte_eal_mp_wait_lcore();
> > >-
> > > 	}
> > > 	if (signum == SIGTSTP)
> > > 		rte_event_dev_dump(0, stdout);
> > >@@ -465,6 +453,10 @@ main(int argc, char **argv)
> > > 	}
> > >
> > > 	RTE_ETH_FOREACH_DEV(portid) {
> > >+		rte_event_eth_rx_adapter_stop(portid);
> > >+		rte_event_eth_tx_adapter_stop(portid);
> > >+		if (rte_eth_dev_stop(portid) < 0)
> > >+			printf("Failed to stop port %u", portid);
> > > 		rte_eth_dev_close(portid);
> > > 	}
> > >
> > >--
> > >2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev
  2020-12-21  5:34 [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end Feifei Wang
                   ` (2 preceding siblings ...)
  2021-01-05  5:14 ` [dpdk-dev] [PATCH v3] examples/eventdev: refactor ethdev port stop Feifei Wang
@ 2021-01-14 10:30 ` Feifei Wang
  2021-01-14 10:30   ` [dpdk-dev] [PATCH v4 1/3] doc: fix core enabled bug for eventdev pipeline example Feifei Wang
                     ` (2 more replies)
  3 siblings, 3 replies; 14+ messages in thread
From: Feifei Wang @ 2021-01-14 10:30 UTC (permalink / raw)
  Cc: dev, nd, Feifei Wang

Fix some bugs for eventdev pipeline example.

v2:
1. Delete unused variable to fix build error
2. Reduce commit message length to fix coding style issues

v3:
1. Title improvement (Van Haaren)

v4:
1. fix core enabled bug
2. add info typo for main core

Feifei Wang (3):
  doc: fix core enabled bug for eventdev pipeline example
  examples/eventdev: add info typo for main core
  examples/eventdev: move eth stop to the end

 .../sample_app_ug/eventdev_pipeline.rst       |  5 +-
 examples/eventdev_pipeline/main.c             | 78 +++++++++++--------
 2 files changed, 47 insertions(+), 36 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] [PATCH v4 1/3] doc: fix core enabled bug for eventdev pipeline example
  2021-01-14 10:30 ` [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev Feifei Wang
@ 2021-01-14 10:30   ` Feifei Wang
  2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 2/3] examples/eventdev: add info output for main core Feifei Wang
  2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 3/3] examples/eventdev: move eth stop to the end Feifei Wang
  2 siblings, 0 replies; 14+ messages in thread
From: Feifei Wang @ 2021-01-14 10:30 UTC (permalink / raw)
  To: Harry van Haaren, David Hunt, Jerin Jacob, John McNamara
  Cc: dev, nd, Feifei Wang, stable, Ruifeng Wang

In the case that the cores are isolated, if "-l" or "-c" parameter is not
added, the cores will not be enabled and can not launch worker function
correctly. In the meanwhile, no error information is reported.

For example:
totally CPUs:16
isolated CPUs:1-8
command: sudo gdb -args ./dpdk-eventdev_pipeline --vdev event_sw0 \
        -- -r1 -t1 -e4 -w F00 -s4 -n0 -c32 -W1000 -D

cores information:
rte_config->lcore_role = {ROLE_RTE, ROLE_OFF, ROLE_OFF, ROLE_OFF,
                          ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF,
                          ROLE_OFF, ROLE_RTE, ROLE_RTE, ROLE_RTE,
                          ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE}

output information:
...
[main()] lcore 9 executing worker, using eventdev port 0
[main()] lcore 10 executing worker, using eventdev port 1
[main()] lcore 11 executing worker, using eventdev port 2

This is because "RTE_LCORE_FOREACH_WORKER" chooses the enabled core. In
the case that the cores are isolated, "the lcore_role" flag of isolated
cores are set as "ROLE_OFF" by default(not enabled). So if we choose
these isolated cores as workers, "RTE_LCORE_FOREACH_WORKER" will ignore
these cores and not launch worker functions on them.

To fix this, add "-l" parameters to doc and add lcore enabled check.

Fixes: 1094ca96689c ("doc: add SW eventdev pipeline to sample app guide")
Cc: harry.van.haaren@intel.com
Cc: stable@dpdk.org

Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
---
 doc/guides/sample_app_ug/eventdev_pipeline.rst | 5 +++--
 examples/eventdev_pipeline/main.c              | 7 ++++++-
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/doc/guides/sample_app_ug/eventdev_pipeline.rst b/doc/guides/sample_app_ug/eventdev_pipeline.rst
index 4508c3dcc..19ff53803 100644
--- a/doc/guides/sample_app_ug/eventdev_pipeline.rst
+++ b/doc/guides/sample_app_ug/eventdev_pipeline.rst
@@ -34,6 +34,7 @@ options.
 An example eventdev pipeline running with the software eventdev PMD using
 these settings is shown below:
 
+ * ``-l 0,2,8-15``: lcore to use
  * ``-r1``: core mask 0x1 for RX
  * ``-t1``: core mask 0x1 for TX
  * ``-e4``: core mask 0x4 for the software scheduler
@@ -46,8 +47,8 @@ these settings is shown below:
 
 .. code-block:: console
 
-    ./<build_dir>/examples/dpdk-eventdev_pipeline --vdev event_sw0 -- -r1 -t1 \
-    -e4 -w FF00 -s4 -n0 -c32 -W1000 -D
+    ./<build_dir>/examples/dpdk-eventdev_pipeline -l 0,2,8-15 --vdev event_sw0 \
+    -- -r1 -t1 -e4 -w FF00 -s4 -n0 -c32 -W1000 -D
 
 The application has some sanity checking built-in, so if there is a function
 (e.g.; the RX core) which doesn't have a cpu core mask assigned, the application
diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c
index 823f8b51c..e9a591134 100644
--- a/examples/eventdev_pipeline/main.c
+++ b/examples/eventdev_pipeline/main.c
@@ -239,8 +239,13 @@ parse_app_args(int argc, char **argv)
 
 		if (fdata->worker_core[i])
 			cdata.num_workers++;
-		if (core_in_use(i))
+		if (core_in_use(i)) {
+			if (!rte_lcore_is_enabled(i)) {
+				printf("error: lcore %d is not enabled in lcore list\n", i);
+				rte_exit(EXIT_FAILURE, "check lcore params failed\n");
+			}
 			cdata.active_cores++;
+		}
 	}
 }
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] [PATCH v4 2/3] examples/eventdev: add info output for main core
  2021-01-14 10:30 ` [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev Feifei Wang
  2021-01-14 10:30   ` [dpdk-dev] [PATCH v4 1/3] doc: fix core enabled bug for eventdev pipeline example Feifei Wang
@ 2021-01-14 10:31   ` Feifei Wang
  2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 3/3] examples/eventdev: move eth stop to the end Feifei Wang
  2 siblings, 0 replies; 14+ messages in thread
From: Feifei Wang @ 2021-01-14 10:31 UTC (permalink / raw)
  To: Harry van Haaren, John McNamara, Jerin Jacob, David Hunt
  Cc: dev, nd, Feifei Wang, stable, Ruifeng Wang

When the main core is set as tx/rx/sched/worker core, it also needs to
print some information to show this. Thus, add info output for the main
core, and add a "dump" function to print core information for the sake
of code simplicity and easy maintenance.

In the meanwhile, fix the count error. For the variable "worker_idx", it
should be incremented when the core is set as worker core. However, when
the main core is set as rx/tx/sched core, the worker_idx is also
incremented. Though this error may not have a substantial impact due to
that the main core is the last launched core, but it should be corrected
from the perspective of code correctness.

Fixes: 1094ca96689c ("doc: add SW eventdev pipeline to sample app guide")
Cc: harry.van.haaren@intel.com
Cc: stable@dpdk.org

Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
---
 examples/eventdev_pipeline/main.c | 55 +++++++++++++++++++------------
 1 file changed, 34 insertions(+), 21 deletions(-)

diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c
index e9a591134..3526d4d3d 100644
--- a/examples/eventdev_pipeline/main.c
+++ b/examples/eventdev_pipeline/main.c
@@ -22,6 +22,32 @@ struct config_data cdata = {
 	.worker_cq_depth = 16
 };
 
+static void
+dump_core_info(unsigned int lcore_id, struct worker_data *data,
+		unsigned int worker_idx)
+{
+	if (fdata->rx_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing NIC Rx\n",
+			__func__, lcore_id);
+
+	if (fdata->tx_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing NIC Tx\n",
+			__func__, lcore_id);
+
+	if (fdata->sched_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing scheduler\n",
+			__func__, lcore_id);
+
+	if (fdata->worker_core[lcore_id])
+		printf(
+			"[%s()] lcore %d executing worker, using eventdev port %u\n",
+			__func__, lcore_id,
+			data[worker_idx].port_id);
+}
+
 static bool
 core_in_use(unsigned int lcore_id) {
 	return (fdata->rx_core[lcore_id] || fdata->sched_core[lcore_id] ||
@@ -411,25 +437,7 @@ main(int argc, char **argv)
 			!fdata->sched_core[lcore_id])
 			continue;
 
-		if (fdata->rx_core[lcore_id])
-			printf(
-				"[%s()] lcore %d executing NIC Rx\n",
-				__func__, lcore_id);
-
-		if (fdata->tx_core[lcore_id])
-			printf(
-				"[%s()] lcore %d executing NIC Tx\n",
-				__func__, lcore_id);
-
-		if (fdata->sched_core[lcore_id])
-			printf("[%s()] lcore %d executing scheduler\n",
-					__func__, lcore_id);
-
-		if (fdata->worker_core[lcore_id])
-			printf(
-				"[%s()] lcore %d executing worker, using eventdev port %u\n",
-				__func__, lcore_id,
-				worker_data[worker_idx].port_id);
+		dump_core_info(lcore_id, worker_data, worker_idx);
 
 		err = rte_eal_remote_launch(fdata->cap.worker,
 				&worker_data[worker_idx], lcore_id);
@@ -444,8 +452,13 @@ main(int argc, char **argv)
 
 	lcore_id = rte_lcore_id();
 
-	if (core_in_use(lcore_id))
-		fdata->cap.worker(&worker_data[worker_idx++]);
+	if (core_in_use(lcore_id)) {
+		dump_core_info(lcore_id, worker_data, worker_idx);
+		fdata->cap.worker(&worker_data[worker_idx]);
+
+		if (fdata->worker_core[lcore_id])
+			worker_idx++;
+	}
 
 	rte_eal_mp_wait_lcore();
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [dpdk-dev] [PATCH v4 3/3] examples/eventdev: move eth stop to the end
  2021-01-14 10:30 ` [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev Feifei Wang
  2021-01-14 10:30   ` [dpdk-dev] [PATCH v4 1/3] doc: fix core enabled bug for eventdev pipeline example Feifei Wang
  2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 2/3] examples/eventdev: add info output for main core Feifei Wang
@ 2021-01-14 10:31   ` Feifei Wang
  2021-01-25 17:51     ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
  2 siblings, 1 reply; 14+ messages in thread
From: Feifei Wang @ 2021-01-14 10:31 UTC (permalink / raw)
  To: Harry van Haaren, Pavan Nikhilesh, Nikhil Rao
  Cc: dev, nd, Feifei Wang, pbhagavatula, stable, Ruifeng Wang,
	Honnappa Nagarahalli

Move eth stop code from "signal_handler" function to the end of "main"
function. There are two reasons for this:

First, this improves code maintenance and makes code look simple and
clear. Based on this change, after receiving the interrupt signal,
"fdata->done" is set as 1. Then the main thread will wait all worker
lcores to jump out of the loop. Finally, the main thread will stop and
then close eth dev port.

Second, for older version, the main thread first stops eth dev port and
then waits the end of worker lcore. This may cause errors because it may
stop the eth dev port which worker lcores are using. This moving change
can fix this by waiting all worker threads to exit and then stop the
eth dev port.

In the meanwhile, remove wmb in signal_handler.

This is because when the main lcore receive the stop signal, it stores 1
into fdata->done. And then the worker lcores load "fdata->done" and jump
out of the loop to stop running. Nothing should be stored after updating
fdata->done, so the wmb is unnecessary.

Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx adapter")
Cc: pbhagavatula@marvell.com
Cc: stable@dpdk.org

Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
---
 examples/eventdev_pipeline/main.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/examples/eventdev_pipeline/main.c b/examples/eventdev_pipeline/main.c
index 3526d4d3d..4621e8a89 100644
--- a/examples/eventdev_pipeline/main.c
+++ b/examples/eventdev_pipeline/main.c
@@ -311,7 +311,6 @@ static void
 signal_handler(int signum)
 {
 	static uint8_t once;
-	uint16_t portid;
 
 	if (fdata->done)
 		rte_exit(1, "Exiting on signal %d\n", signum);
@@ -322,17 +321,6 @@ signal_handler(int signum)
 			rte_event_dev_dump(0, stdout);
 		once = 1;
 		fdata->done = 1;
-		rte_smp_wmb();
-
-		RTE_ETH_FOREACH_DEV(portid) {
-			rte_event_eth_rx_adapter_stop(portid);
-			rte_event_eth_tx_adapter_stop(portid);
-			if (rte_eth_dev_stop(portid) < 0)
-				printf("Failed to stop port %u", portid);
-		}
-
-		rte_eal_mp_wait_lcore();
-
 	}
 	if (signum == SIGTSTP)
 		rte_event_dev_dump(0, stdout);
@@ -483,6 +471,10 @@ main(int argc, char **argv)
 	}
 
 	RTE_ETH_FOREACH_DEV(portid) {
+		rte_event_eth_rx_adapter_stop(portid);
+		rte_event_eth_tx_adapter_stop(portid);
+		if (rte_eth_dev_stop(portid) < 0)
+			printf("Failed to stop port %u", portid);
 		rte_eth_dev_close(portid);
 	}
 
-- 
2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [dpdk-dev] [EXT] [PATCH v4 3/3] examples/eventdev: move eth stop to the end
  2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 3/3] examples/eventdev: move eth stop to the end Feifei Wang
@ 2021-01-25 17:51     ` Pavan Nikhilesh Bhagavatula
  2021-01-26 13:37       ` Jerin Jacob
  0 siblings, 1 reply; 14+ messages in thread
From: Pavan Nikhilesh Bhagavatula @ 2021-01-25 17:51 UTC (permalink / raw)
  To: Feifei Wang, Harry van Haaren, Pavan Nikhilesh, Nikhil Rao
  Cc: dev, nd, stable, Ruifeng Wang, Honnappa Nagarahalli

>Move eth stop code from "signal_handler" function to the end of
>"main"
>function. There are two reasons for this:
>
>First, this improves code maintenance and makes code look simple and
>clear. Based on this change, after receiving the interrupt signal,
>"fdata->done" is set as 1. Then the main thread will wait all worker
>lcores to jump out of the loop. Finally, the main thread will stop and
>then close eth dev port.
>
>Second, for older version, the main thread first stops eth dev port and
>then waits the end of worker lcore. This may cause errors because it
>may
>stop the eth dev port which worker lcores are using. This moving
>change
>can fix this by waiting all worker threads to exit and then stop the
>eth dev port.
>
>In the meanwhile, remove wmb in signal_handler.
>
>This is because when the main lcore receive the stop signal, it stores 1
>into fdata->done. And then the worker lcores load "fdata->done" and
>jump
>out of the loop to stop running. Nothing should be stored after
>updating
>fdata->done, so the wmb is unnecessary.
>
>Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx
>adapter")
>Cc: pbhagavatula@marvell.com
>Cc: stable@dpdk.org
>
>Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
>Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
>Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
>Reviewed-by: Honnappa Nagarahalli
><honnappa.nagarahalli@arm.com>
>Acked-by: Harry van Haaren <harry.van.haaren@intel.com>

Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

>---
> examples/eventdev_pipeline/main.c | 16 ++++------------
> 1 file changed, 4 insertions(+), 12 deletions(-)
>
>diff --git a/examples/eventdev_pipeline/main.c
>b/examples/eventdev_pipeline/main.c
>index 3526d4d3d..4621e8a89 100644
>--- a/examples/eventdev_pipeline/main.c
>+++ b/examples/eventdev_pipeline/main.c
>@@ -311,7 +311,6 @@ static void
> signal_handler(int signum)
> {
> 	static uint8_t once;
>-	uint16_t portid;
>
> 	if (fdata->done)
> 		rte_exit(1, "Exiting on signal %d\n", signum);
>@@ -322,17 +321,6 @@ signal_handler(int signum)
> 			rte_event_dev_dump(0, stdout);
> 		once = 1;
> 		fdata->done = 1;
>-		rte_smp_wmb();
>-
>-		RTE_ETH_FOREACH_DEV(portid) {
>-			rte_event_eth_rx_adapter_stop(portid);
>-			rte_event_eth_tx_adapter_stop(portid);
>-			if (rte_eth_dev_stop(portid) < 0)
>-				printf("Failed to stop port %u", portid);
>-		}
>-
>-		rte_eal_mp_wait_lcore();
>-
> 	}
> 	if (signum == SIGTSTP)
> 		rte_event_dev_dump(0, stdout);
>@@ -483,6 +471,10 @@ main(int argc, char **argv)
> 	}
>
> 	RTE_ETH_FOREACH_DEV(portid) {
>+		rte_event_eth_rx_adapter_stop(portid);
>+		rte_event_eth_tx_adapter_stop(portid);
>+		if (rte_eth_dev_stop(portid) < 0)
>+			printf("Failed to stop port %u", portid);
> 		rte_eth_dev_close(portid);
> 	}
>
>--
>2.25.1


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [dpdk-dev] [EXT] [PATCH v4 3/3] examples/eventdev: move eth stop to the end
  2021-01-25 17:51     ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
@ 2021-01-26 13:37       ` Jerin Jacob
  0 siblings, 0 replies; 14+ messages in thread
From: Jerin Jacob @ 2021-01-26 13:37 UTC (permalink / raw)
  To: Pavan Nikhilesh Bhagavatula
  Cc: Feifei Wang, Harry van Haaren, Pavan Nikhilesh, Nikhil Rao, dev,
	nd, stable, Ruifeng Wang, Honnappa Nagarahalli

On Mon, Jan 25, 2021 at 11:22 PM Pavan Nikhilesh Bhagavatula
<pbhagavatula@marvell.com> wrote:
>
> >Move eth stop code from "signal_handler" function to the end of
> >"main"
> >function. There are two reasons for this:
> >
> >First, this improves code maintenance and makes code look simple and
> >clear. Based on this change, after receiving the interrupt signal,
> >"fdata->done" is set as 1. Then the main thread will wait all worker
> >lcores to jump out of the loop. Finally, the main thread will stop and
> >then close eth dev port.
> >
> >Second, for older version, the main thread first stops eth dev port and
> >then waits the end of worker lcore. This may cause errors because it
> >may
> >stop the eth dev port which worker lcores are using. This moving
> >change
> >can fix this by waiting all worker threads to exit and then stop the
> >eth dev port.
> >
> >In the meanwhile, remove wmb in signal_handler.
> >
> >This is because when the main lcore receive the stop signal, it stores 1
> >into fdata->done. And then the worker lcores load "fdata->done" and
> >jump
> >out of the loop to stop running. Nothing should be stored after
> >updating
> >fdata->done, so the wmb is unnecessary.
> >
> >Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx
> >adapter")
> >Cc: pbhagavatula@marvell.com
> >Cc: stable@dpdk.org
> >
> >Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> >Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> >Reviewed-by: Honnappa Nagarahalli
> ><honnappa.nagarahalli@arm.com>
> >Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
>
> Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>


Series applied to dpdk-next-eventdev/for-main. Thanks.




>
> >---
> > examples/eventdev_pipeline/main.c | 16 ++++------------
> > 1 file changed, 4 insertions(+), 12 deletions(-)
> >
> >diff --git a/examples/eventdev_pipeline/main.c
> >b/examples/eventdev_pipeline/main.c
> >index 3526d4d3d..4621e8a89 100644
> >--- a/examples/eventdev_pipeline/main.c
> >+++ b/examples/eventdev_pipeline/main.c
> >@@ -311,7 +311,6 @@ static void
> > signal_handler(int signum)
> > {
> >       static uint8_t once;
> >-      uint16_t portid;
> >
> >       if (fdata->done)
> >               rte_exit(1, "Exiting on signal %d\n", signum);
> >@@ -322,17 +321,6 @@ signal_handler(int signum)
> >                       rte_event_dev_dump(0, stdout);
> >               once = 1;
> >               fdata->done = 1;
> >-              rte_smp_wmb();
> >-
> >-              RTE_ETH_FOREACH_DEV(portid) {
> >-                      rte_event_eth_rx_adapter_stop(portid);
> >-                      rte_event_eth_tx_adapter_stop(portid);
> >-                      if (rte_eth_dev_stop(portid) < 0)
> >-                              printf("Failed to stop port %u", portid);
> >-              }
> >-
> >-              rte_eal_mp_wait_lcore();
> >-
> >       }
> >       if (signum == SIGTSTP)
> >               rte_event_dev_dump(0, stdout);
> >@@ -483,6 +471,10 @@ main(int argc, char **argv)
> >       }
> >
> >       RTE_ETH_FOREACH_DEV(portid) {
> >+              rte_event_eth_rx_adapter_stop(portid);
> >+              rte_event_eth_tx_adapter_stop(portid);
> >+              if (rte_eth_dev_stop(portid) < 0)
> >+                      printf("Failed to stop port %u", portid);
> >               rte_eth_dev_close(portid);
> >       }
> >
> >--
> >2.25.1
>

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2021-01-26 13:37 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-21  5:34 [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end Feifei Wang
2020-12-21  9:32 ` [dpdk-dev] [PATCH v2] " Feifei Wang
2020-12-21  9:56 ` [dpdk-dev] [PATCH] " Van Haaren, Harry
2020-12-22  5:31   ` [dpdk-dev] 回复: " Feifei Wang
2021-01-05  5:14 ` [dpdk-dev] [PATCH v3] examples/eventdev: refactor ethdev port stop Feifei Wang
2021-01-05 10:09   ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2021-01-14  6:24     ` [dpdk-dev] 回复: " Feifei Wang
2021-01-14  8:50       ` Feifei Wang
2021-01-14 10:30 ` [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev Feifei Wang
2021-01-14 10:30   ` [dpdk-dev] [PATCH v4 1/3] doc: fix core enabled bug for eventdev pipeline example Feifei Wang
2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 2/3] examples/eventdev: add info output for main core Feifei Wang
2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 3/3] examples/eventdev: move eth stop to the end Feifei Wang
2021-01-25 17:51     ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2021-01-26 13:37       ` Jerin Jacob

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git