DPDK patches and discussions
 help / color / mirror / Atom feed
From: Ferruh Yigit <ferruh.yigit@intel.com>
To: "Min Hu (Connor)" <humin29@huawei.com>,
	dev@dpdk.org, John McNamara <john.mcnamara@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>
Cc: anatoly.burakov@intel.com, andrew.rybchenko@oktetlabs.ru,
	thomas@monjalon.net, xiaoyun.li@intel.com
Subject: Re: [dpdk-dev] [PATCH v11] app/testpmd: support multi-process
Date: Mon, 19 Apr 2021 14:42:25 +0100
Message-ID: <b2936465-667a-8414-e524-2072eef5f0a0@intel.com> (raw)
In-Reply-To: <1618794213-44460-1-git-send-email-humin29@huawei.com>

On 4/19/2021 2:03 AM, Min Hu (Connor) wrote:
> This patch adds multi-process support for testpmd.
> The test cmd example as follows:
> the primary cmd:
> ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i \
> --rxq=4 --txq=4 --num-procs=2 --proc-id=0
> 
> the secondary cmd:
> ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i \
> --rxq=4 --txq=4 --num-procs=2 --proc-id=1
> 

Hi Connor,

I put some comments on the documentation, can you please check them?

Meanwhile, John, Marko can you please help on the doc review?

Thanks,
ferruh

> Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
> Signed-off-by: Lijun Ou <oulijun@huawei.com>
> Acked-by: Xiaoyun Li <xiaoyun.li@intel.com>
> Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

<...>

> @@ -2626,16 +2701,16 @@ start_port(portid_t pid)
>   		cnt_pi++;
>   
>   		/* start port */
> -		diag = rte_eth_dev_start(pi);
> +		diag = eth_dev_start_mp(pi);
>   		if (diag < 0) {
>   			printf("Fail to start port %d: %s\n", pi,
>   			       rte_strerror(-diag));
>   
>   			/* Fail to setup rx queue, return */
>   			if (rte_atomic16_cmpset(&(port->port_status),
> -				RTE_PORT_HANDLING, RTE_PORT_STOPPED) == 0)
> -				printf("Port %d can not be set back to "
> -							"stopped\n", pi);
> +			RTE_PORT_HANDLING, RTE_PORT_STOPPED) == 0)
> +				printf("Port %d can not be set back to stopped\n",
> +					pi);

Indentation is wrong.

<...>

> --- a/doc/guides/testpmd_app_ug/run_app.rst
> +++ b/doc/guides/testpmd_app_ug/run_app.rst
> @@ -543,3 +543,89 @@ The command line options are:
>       bit 1 - two hairpin ports paired
>       bit 0 - two hairpin ports loop
>       The default value is 0. Hairpin will use single port mode and implicit Tx flow mode.
> +
> +
> +Testpmd Support Multi Process Command-line Options
> +--------------------------------------------------

What do you think making this a sub-section of the "Testpmd Command-line Options"?

And the section name can be "Testpmd Multi-Process Command-line Options"

> +
> +The following are the command-line options for the testpmd applications(support

Space before '('.

> +multi process).They must be separated from the EAL options, shown in the previous
> +section, with a ``--`` separator:

If this becomes sub-section of the "Testpmd Command-line Options", above will 
become duplication, and I think it can be simplified as:

"The following are the command-line options for testpmd multi-process support:"


> +
> +.. code-block:: console
> +
> +	primary process:
> +	sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i --rxq=4 --txq=4 \
> +        --num-procs=2 --proc-id=0
> +
> +	secondary process:
> +	sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i --rxq=4 --txq=4 \
> +        --num-procs=2 --proc-id=1
> +
> +The command line options are:
> +
> +*   ``--rxq=N``
> +
> +    Set the number of Rx queues per port to N. N is the sum of queues used by primary
> +    and secondary process. Primary process and secondary process should have separate
> +    queues, and each should occupy at least one queue. Where N should be the multiple
> +    of number of processes.
> +
> +*   ``--txq=N``
> +
> +    Set the number of Tx queues per port to N. N is the sum of queues used by primary
> +    and secondary process. Primary process and secondary process should have separate
> +    queues, and each should occupy at least one queue. Where N should be the multiple
> +    of number of processes.
> +
> +*   ``--num-procs=N``
> +
> +    The number of processes which will be used.
> +
> +*   ``--proc-id=id``
> +
> +    The id of the current process (id < num-procs). id should be different in primary
> +    process and secondary process.
> +

The famous question, does it start from '0' or '1'?

> +Calculation rule for queue:
> +All queues are allocated to different processes based on proc_num and proc_id.

Can highlight proc_num and proc_id, as ``proc_num`` and ``proc_id``.

> +Calculation rule for the Testpmd to allocate queues to each process:

Not sure if testpmd should start with uppercase.

> +start(queue start id) = proc_id * nb_q / num_procs;
> +end(queue end id) = start + nb_q / num_procs;
> +

Can you put above into a code-block.

> +For example, if supports 4 txq and rxq

".., if testpmp supports 4 Tx and Rx queues"

> +the 0~1 for primary process
> +the 2~3 for secondary process
> +
> +The number of rings should be a multiple of the number of processes. If not,
> +redundant queues will exist after queues are allocated to processes. After RSS is
> +enabled, packet loss occurs when traffic is sent to all processes at the same time.
> +Some traffic enters redundant queues and cannot be forwarded.
> +
> +Most dev ops is supported in primary and secondary process. While secondary process

"Most of the device operations are supported in ..."?
"While the secondary process ..."?

> +is not permitted to allocate or release shared memory, so some ops are not supported
> +as follows:
> +``dev_configure``
> +``dev_start``
> +``dev_stop``
> +``rx_queue_setup``
> +``tx_queue_setup``
> +``rx_queue_release``
> +``tx_queue_release``
> +

This list displayed as single line in the html output, if the intention is to 
have them listed as been in the source, may need following update:

  -as follows:
  +as follows::
  +

> +So, any command from testpmd which calls those APIs will not be supported in secondary
> +process, like:
> +``port config all rxq|txq|rxd|txd <value>``
> +``port config <port_id> rx_offload xxx on/off ``
> +``port config <port_id> tx_offload xxx on/off``
> +etc.

Same here for the list view.

> +
> +RTE_FLOW supported, it applies only on its own process on SW side, but all on HW size.

You may use "Flow API", instead of RTE_FLOW, so it becomes: "Flow API is supported."

> +stats supported, stats will not change when one quit and start, As they share the same
> +buffer to store the stats. 

The above stats related sentences looks like squeezed within the flow API 
related description, it may be good to move it.

> Flow rules are maintained in process level: primary and secondary
> +has its own flow list(but one flow list in HW). The two can see all the queues, so setting

Space before '('.

> +the flow rules for the other is OK. Of course, io(receive or transmit packets) in the queue
> +from others is not permitted.

I understand what you mean, but wording can be improved, can you please try to 
reword it?

> +
> +RSS supported, Primary process and secondary process has separate queues to use, RSS

"RSS is supported"? And start 'primary' with lowercase.

> +will work in their own queues whether primary and secondary process.
> 

There is a chance that when RSS is enabled, the packet may end up in other 
process' queues?

  reply	other threads:[~2021-04-19 13:42 UTC|newest]

Thread overview: 77+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-08  9:46 [dpdk-dev] [RFC] " Lijun Ou
2021-01-08 10:28 ` Ferruh Yigit
2021-01-09  9:54   ` oulijun
2021-01-10 12:32 ` Wisam Monther
2021-01-12 14:13   ` oulijun
2021-01-12 14:21     ` Wisam Monther
2021-01-14  2:46       ` oulijun
2021-01-20 14:06 ` [dpdk-dev] [RFC V2] " Lijun Ou
2021-03-05  1:04   ` [dpdk-dev] [PATCH] " Lijun Ou
2021-03-05  4:05     ` Ajit Khaparde
2021-03-10 11:11       ` Min Hu (Connor)
2021-03-11  2:47     ` [dpdk-dev] [PATCH v2] " Min Hu (Connor)
2021-03-22  2:27       ` Ajit Khaparde
2021-03-22  6:35         ` Min Hu (Connor)
2021-06-15 12:23       ` [dpdk-dev] [PATCH v14] " Min Hu (Connor)
2021-07-02 12:09       ` [dpdk-dev] [PATCH v15] " Andrew Rybchenko
2021-07-02 12:47         ` Andrew Rybchenko
2021-07-08 12:20           ` Min Hu (Connor)
2021-07-08 12:30             ` Andrew Rybchenko
2021-07-08 12:51               ` Min Hu (Connor)
2021-07-10  3:50       ` [dpdk-dev] [PATCH v16] " Min Hu (Connor)
2021-07-24 11:45         ` Thomas Monjalon
2021-07-26  0:26           ` Min Hu (Connor)
2021-07-26  6:30             ` Thomas Monjalon
2021-07-26  7:28               ` Min Hu (Connor)
2021-08-02  1:51                 ` Min Hu (Connor)
2021-08-02  8:03                   ` Thomas Monjalon
2021-08-16 18:12                     ` Singh, Aman Deep
2021-08-24 12:18                       ` Ferruh Yigit
2021-08-24 13:27                         ` Min Hu (Connor)
2021-08-25  2:06       ` [dpdk-dev] [PATCH v17] " Min Hu (Connor)
2021-09-07 13:23         ` Ferruh Yigit
2021-09-08  0:48           ` Min Hu (Connor)
2021-03-11  9:07     ` [dpdk-dev] [PATCH v3] " Min Hu (Connor)
2021-03-20  0:58       ` Min Hu (Connor)
2021-03-22  7:07     ` [dpdk-dev] [PATCH v4] " Min Hu (Connor)
2021-03-22 11:19       ` Ferruh Yigit
2021-03-24  8:08       ` Li, Xiaoyun
2021-03-25 13:32         ` Min Hu (Connor)
2021-03-25 23:25           ` Ajit Khaparde
2021-03-26  6:46             ` Min Hu (Connor)
2021-03-25 13:17     ` [dpdk-dev] [PATCH v5] " Min Hu (Connor)
2021-03-26  6:46     ` [dpdk-dev] [PATCH v6] " Min Hu (Connor)
2021-03-26  8:52     ` [dpdk-dev] [PATCH v7] " Min Hu (Connor)
2021-03-29  7:51       ` Li, Xiaoyun
2021-03-30  1:48         ` Min Hu (Connor)
2021-03-30  1:48     ` [dpdk-dev] [PATCH v8] " Min Hu (Connor)
2021-03-30  2:17       ` Li, Xiaoyun
2021-03-30  6:36         ` Min Hu (Connor)
2021-03-30  3:11       ` Ajit Khaparde
2021-03-30  6:41         ` Min Hu (Connor)
2021-03-30 10:19           ` Ferruh Yigit
2021-03-30 10:43             ` Min Hu (Connor)
2021-04-08 10:32               ` Min Hu (Connor)
2021-04-08 13:27                 ` Ferruh Yigit
2021-04-09  0:45                   ` Min Hu (Connor)
2021-04-12 16:37       ` Ferruh Yigit
2021-04-15  7:54         ` Ferruh Yigit
2021-04-16  2:20           ` Min Hu (Connor)
2021-04-16  1:52     ` [dpdk-dev] [PATCH v9] " Min Hu (Connor)
2021-04-16 16:30       ` Ferruh Yigit
2021-04-17  6:12         ` Min Hu (Connor)
2021-04-17  6:12     ` [dpdk-dev] [PATCH v10] " Min Hu (Connor)
2021-04-17 22:21       ` Ferruh Yigit
2021-04-19  1:03         ` Min Hu (Connor)
2021-04-19  1:03     ` [dpdk-dev] [PATCH v11] " Min Hu (Connor)
2021-04-19 13:42       ` Ferruh Yigit [this message]
2021-04-21  9:08         ` Min Hu (Connor)
2021-04-21  8:36     ` [dpdk-dev] [PATCH v12] " Min Hu (Connor)
2021-04-22  1:18     ` [dpdk-dev] [PATCH v13] " Min Hu (Connor)
2021-06-08  8:42       ` Andrew Rybchenko
2021-06-08 10:22         ` Thomas Monjalon
2021-06-08 10:39           ` Andrew Rybchenko
2021-06-08 12:02             ` Thomas Monjalon
2021-06-08 12:36             ` Ferruh Yigit
2021-06-15 12:04         ` Min Hu (Connor)
2021-04-21  5:59 [dpdk-dev] [PATCH v11] " Singh, Aman Deep

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b2936465-667a-8414-e524-2072eef5f0a0@intel.com \
    --to=ferruh.yigit@intel.com \
    --cc=anatoly.burakov@intel.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=dev@dpdk.org \
    --cc=humin29@huawei.com \
    --cc=john.mcnamara@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=thomas@monjalon.net \
    --cc=xiaoyun.li@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git