DPDK patches and discussions
 help / color / mirror / Atom feed
From: Feifei Wang <Feifei.Wang2@arm.com>
To: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>,
	Harry van Haaren <harry.van.haaren@intel.com>,
	Nikhil Rao <nikhil.rao@intel.com>,
	Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
	"jerinj@marvell.com" <jerinj@marvell.com>,  nd <nd@arm.com>,
	"stable@dpdk.org" <stable@dpdk.org>,
	Ruifeng Wang <Ruifeng.Wang@arm.com>,
	Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>,
	nd <nd@arm.com>, nd <nd@arm.com>
Subject: [dpdk-dev] 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
Date: Thu, 14 Jan 2021 08:50:15 +0000
Message-ID: <DBBPR08MB441116DAD4B194567D0E00BBC8A80@DBBPR08MB4411.eurprd08.prod.outlook.com> (raw)
In-Reply-To: <DBBPR08MB441108E37ABB790273F5E3D1C8A80@DBBPR08MB4411.eurprd08.prod.outlook.com>



> -----邮件原件-----
> 发件人: Feifei Wang
> 发送时间: 2021年1月14日 14:24
> 收件人: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>; Harry
> van Haaren <harry.van.haaren@intel.com>; Nikhil Rao
> <nikhil.rao@intel.com>; Pavan Nikhilesh
> <pbhagavatula@caviumnetworks.com>
> 抄送: dev@dpdk.org; jerinj@marvell.com; nd <nd@arm.com>;
> stable@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> 主题: 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
> 
> > -----邮件原件-----
> > 发件人: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>
> > 发送时间: 2021年1月5日 18:09
> > 收件人: Feifei Wang <Feifei.Wang2@arm.com>; Harry van Haaren
> > <harry.van.haaren@intel.com>; Nikhil Rao <nikhil.rao@intel.com>; Pavan
> > Nikhilesh <pbhagavatula@caviumnetworks.com>
> > 抄送: dev@dpdk.org; jerinj@marvell.com; nd <nd@arm.com>;
> > stable@dpdk.org; Ruifeng Wang <Ruifeng.Wang@arm.com>; Honnappa
> > Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > 主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
> >
> Hi, Pavan
> 
> > Hi Feifei,
> >
> > >Move eth stop code from "signal_handler" function to the end of "main"
> > >function. There are two reasons for this:
> > >
> > >First, this improves code maintenance and makes code look simple and
> > >clear. Based on this change, after receiving the interrupt signal,
> > >"fdata->done" is set as 1. Then the main thread will wait all worker
> > >lcores to jump out of the loop. Finally, the main thread will stop
> > >and then close eth dev port.
> > >
> > >Second, for older version, the main thread first stops eth dev port
> > >and then waits the end of worker lcore. This may cause errors because
> > >it may stop the eth dev port which worker lcores are using. This
> > >moving change can fix this by waiting all worker threads to exit and
> > >then stop the eth dev port.
> >
> > Apologies for the delayed reply,
> >
> > In case of event dev the workers don't interact with eth device
> > directly, Instead eth device "injects" packets into event device and
> > event device is responsible for scheduling them to the workers.
> >
> > If the producer is not stopped i.e. in this case eth device then the
> > worker threads might never exit and the main core would wait
> > indefinitely for workers to exit. This will be predominantly seen in
> > cases where there are only a few flows and large number of workers
> > causing a lot of intra thread dependency.
> 
> For the case that the event device scheduling packets to the workers.
> Though the producer is not stopped (eth device), when the main core
> receive the interrupt signal, the "fdata->done"
> will be set as 1. Then all the workers load the value 1 of "fdata->done" and
> jump out of the loop to finish their thread.
> 

And I also test the case you said above with this patch, when I send interrupt signal(ctrl + c), IXIA
is running and eth device is also working, then the program can exit normally.

HW:
Traffic generator: IXIA 
Nics: ixgbe 82599ES 10-Gigabit
Architecture: aarch64
CPU: Cortex-A72

With this patch:
______________________________________________________________________________________________
Command: sudo ./dpdk-eventdev_pipeline -l 0-2,8-11 --vdev=event_sw0 -- -r2 -t2 -e4 -w F00 -s4 -n0 -c32 -W1000 -D
.........
Config:
        ports: 2
        workers: 4
        packets: 9223372036854775807
        Queue-prio: 0
        qid0 type: atomic
        Cores available: 7
        Cores used: 6
        Eventdev 0: event_sw
  Stages:
        Stage 0, Type Atomic    Priority = 128
        Stage 1, Type Atomic    Priority = 128
        Stage 2, Type Atomic    Priority = 128
        Stage 3, Type Atomic    Priority = 128

Port 0 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34
Port 0 MAC: 90 e2 ba 56 ed 6c
Port 1 modified RSS hash function based on hardware support,requested:0x3afbc configured:0x38d34
Port 1 MAC: 90 e2 ba 56 ed 6d
[dump_core_info()] lcore 1 executing NIC Rx
[dump_core_info()] lcore 1 executing NIC Tx
[dump_core_info()] lcore 2 executing scheduler
[dump_core_info()] lcore 8 executing worker, using eventdev port 0
[dump_core_info()] lcore 9 executing worker, using eventdev port 1
[dump_core_info()] lcore 10 executing worker, using eventdev port 2
[dump_core_info()] lcore 11 executing worker, using eventdev port 3


Command: ctrl + c
.........
  worker 1 thread done. RX=0 TX=0
  worker 2 thread done. RX=0 TX=0
  worker 11 thread done. RX=1844023 TX=1844023
  worker 9 thread done. RX=1844888 TX=1844888
  worker 8 thread done. RX=1845490 TX=1845490
  worker 10 thread done. RX=1844903 TX=1844903

Port Workload distribution:
worker 0 :      25.0 % (1845474 pkts)
worker 1 :      25.0 % (1844872 pkts)
worker 2 :      25.0 % (1844887 pkts)
worker 3 :      25.0 % (1844007 pkts)
Invalid port_id=0
Invalid port_id=1
______________________________________________________________________________________________

> Best Regards
> Feifei
> >
> > Regards,
> > Pavan.
> >
> > >
> > >In the meanwhile, remove wmb in signal_handler.
> > >
> > >This is because when the main lcore receive the stop signal, it
> > >stores
> > >1 into fdata->done. And then the worker lcores load "fdata->done" and
> > >jump out of the loop to stop running. Nothing should be stored after
> > >updating
> > >fdata->done, so the wmb is unnecessary.
> > >
> > >Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx
> > >adapter")
> > >Cc: pbhagavatula@marvell.com
> > >Cc: stable@dpdk.org
> > >
> > >Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
> > >Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> > >Reviewed-by: Honnappa Nagarahalli
> > ><honnappa.nagarahalli@arm.com>
> > >Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
> > >---
> > >
> > >v2:
> > >1. Delete unused variable to fix build error 2. Reduce commit message
> > >length to fix coding style issues
> > >
> > >v3:
> > >1. Title improvement (Van Haaren)
> > >
> > > examples/eventdev_pipeline/main.c | 16 ++++------------
> > > 1 file changed, 4 insertions(+), 12 deletions(-)
> > >
> > >diff --git a/examples/eventdev_pipeline/main.c
> > >b/examples/eventdev_pipeline/main.c
> > >index 823f8b51c..fdbaf667b 100644
> > >--- a/examples/eventdev_pipeline/main.c
> > >+++ b/examples/eventdev_pipeline/main.c
> > >@@ -280,7 +280,6 @@ static void
> > > signal_handler(int signum)
> > > {
> > > 	static uint8_t once;
> > >-	uint16_t portid;
> > >
> > > 	if (fdata->done)
> > > 		rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17
> > +290,6 @@
> > >signal_handler(int signum)
> > > 			rte_event_dev_dump(0, stdout);
> > > 		once = 1;
> > > 		fdata->done = 1;
> > >-		rte_smp_wmb();
> > >-
> > >-		RTE_ETH_FOREACH_DEV(portid) {
> > >-			rte_event_eth_rx_adapter_stop(portid);
> > >-			rte_event_eth_tx_adapter_stop(portid);
> > >-			if (rte_eth_dev_stop(portid) < 0)
> > >-				printf("Failed to stop port %u", portid);
> > >-		}
> > >-
> > >-		rte_eal_mp_wait_lcore();
> > >-
> > > 	}
> > > 	if (signum == SIGTSTP)
> > > 		rte_event_dev_dump(0, stdout);
> > >@@ -465,6 +453,10 @@ main(int argc, char **argv)
> > > 	}
> > >
> > > 	RTE_ETH_FOREACH_DEV(portid) {
> > >+		rte_event_eth_rx_adapter_stop(portid);
> > >+		rte_event_eth_tx_adapter_stop(portid);
> > >+		if (rte_eth_dev_stop(portid) < 0)
> > >+			printf("Failed to stop port %u", portid);
> > > 		rte_eth_dev_close(portid);
> > > 	}
> > >
> > >--
> > >2.25.1


  reply	other threads:[~2021-01-14  8:50 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-21  5:34 [dpdk-dev] [PATCH] examples/eventdev: move eth stop to the end Feifei Wang
2020-12-21  9:32 ` [dpdk-dev] [PATCH v2] " Feifei Wang
2020-12-21  9:56 ` [dpdk-dev] [PATCH] " Van Haaren, Harry
2020-12-22  5:31   ` [dpdk-dev] 回复: " Feifei Wang
2021-01-05  5:14 ` [dpdk-dev] [PATCH v3] examples/eventdev: refactor ethdev port stop Feifei Wang
2021-01-05 10:09   ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2021-01-14  6:24     ` [dpdk-dev] 回复: " Feifei Wang
2021-01-14  8:50       ` Feifei Wang [this message]
2021-01-14 10:30 ` [dpdk-dev] [PATCH v4 0/3] fix bugs for examples/evnentdev Feifei Wang
2021-01-14 10:30   ` [dpdk-dev] [PATCH v4 1/3] doc: fix core enabled bug for eventdev pipeline example Feifei Wang
2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 2/3] examples/eventdev: add info output for main core Feifei Wang
2021-01-14 10:31   ` [dpdk-dev] [PATCH v4 3/3] examples/eventdev: move eth stop to the end Feifei Wang
2021-01-25 17:51     ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2021-01-26 13:37       ` Jerin Jacob

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DBBPR08MB441116DAD4B194567D0E00BBC8A80@DBBPR08MB4411.eurprd08.prod.outlook.com \
    --to=feifei.wang2@arm.com \
    --cc=Honnappa.Nagarahalli@arm.com \
    --cc=Ruifeng.Wang@arm.com \
    --cc=dev@dpdk.org \
    --cc=harry.van.haaren@intel.com \
    --cc=jerinj@marvell.com \
    --cc=nd@arm.com \
    --cc=nikhil.rao@intel.com \
    --cc=pbhagavatula@caviumnetworks.com \
    --cc=pbhagavatula@marvell.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git