Re: [dpdk-dev] [PATCH] app/flow-perf: added option to support flow priority

[Satheesh Paul <psatheesh@marvell.com>]

Hi,

Thanks for the review. Please find the reply inline.

-----Original Message-----
From: Wisam Monther <wisamm@nvidia.com> 
Sent: 01 November 2021 02:37 PM
To: Satheesh Paul <psatheesh@marvell.com>
Cc: dev@dpdk.org; Asaf Penso <asafp@nvidia.com>
Subject: [EXT] RE: [dpdk-dev] [PATCH] app/flow-perf: added option to support flow priority

External Email

----------------------------------------------------------------------
Hi,

Please see my inline comments, thanks 😊 

> -----Original Message-----
> From: psatheesh@marvell.com <psatheesh@marvell.com>
> Sent: Friday, October 29, 2021 7:09 AM
> To: Wisam Monther <wisamm@nvidia.com>
> Cc: dev@dpdk.org; Satheesh Paul <psatheesh@marvell.com>
> Subject: [dpdk-dev] [PATCH] app/flow-perf: added option to support 
> flow priority
> 
> From: Satheesh Paul <psatheesh@marvell.com>
> 
> Added support to specify maximum flow priority option. When this 
> option is given, each flow will be created with the priority attribute 
> and the priority will be random between
> 0 to max-flow-priority. This is useful to measure performance on NICs 
> which may have to rearrange flows to honor flow priority.
> 
> Removed the lower limit of 100000 flows per batch.
> 
> Signed-off-by: Satheesh Paul <psatheesh@marvell.com>
> ---
>  app/test-flow-perf/flow_gen.c  |  7 ++++--  
> app/test-flow-perf/flow_gen.h
> |  1 +
>  app/test-flow-perf/main.c      | 44 +++++++++++++++++++++-------------
>  doc/guides/tools/flow-perf.rst |  5 ++++
>  4 files changed, 38 insertions(+), 19 deletions(-)
> 
> diff --git a/app/test-flow-perf/flow_gen.c 
> b/app/test-flow-perf/flow_gen.c index 51871dbfdc..414ec0bc5e 100644
> --- a/app/test-flow-perf/flow_gen.c
> +++ b/app/test-flow-perf/flow_gen.c
> @@ -18,7 +18,7 @@
> 
>  static void
>  fill_attributes(struct rte_flow_attr *attr,
> -	uint64_t *flow_attrs, uint16_t group)
> +	uint64_t *flow_attrs, uint16_t group, uint8_t max_priority)
>  {
>  	uint8_t i;
>  	for (i = 0; i < MAX_ATTRS_NUM; i++) { @@ -32,6 +32,8 @@ 
> fill_attributes(struct rte_flow_attr *attr,
>  			attr->transfer = 1;
>  	}
>  	attr->group = group;
> +	if (max_priority)
> +		attr->priority = rte_rand_max(max_priority);

No need to do this, since it's static and default is zero you can always set it without the condition

Ack.

>  }
> 
>  struct rte_flow *
> @@ -48,6 +50,7 @@ generate_flow(uint16_t port_id,
>  	uint8_t core_idx,
>  	uint8_t rx_queues_count,
>  	bool unique_data,
> +	uint8_t max_priority,

--max-priority=N is good, but I think to reflect the exact usage of it we need something like:
--random-priority=lower_hand-upper_hand
That do random prio from lower to upper.

The spec says priority 0 must be supported. Since we have one side of the interval, it may not be useful to take "lower_hand" as input. 

>  	struct rte_flow_error *error)
>  {
>  	struct rte_flow_attr attr;
> @@ -59,7 +62,7 @@ generate_flow(uint16_t port_id,
>  	memset(actions, 0, sizeof(actions));
>  	memset(&attr, 0, sizeof(struct rte_flow_attr));
> 
> -	fill_attributes(&attr, flow_attrs, group);
> +	fill_attributes(&attr, flow_attrs, group, max_priority);
> 
>  	fill_actions(actions, flow_actions,
>  		outer_ip_src, next_table, hairpinq, diff --git 
> a/app/test-flow-perf/flow_gen.h b/app/test-flow-perf/flow_gen.h index 
> 1118a9fc14..40eeceae6e 100644
> --- a/app/test-flow-perf/flow_gen.h
> +++ b/app/test-flow-perf/flow_gen.h
> @@ -37,6 +37,7 @@ generate_flow(uint16_t port_id,
>  	uint8_t core_idx,
>  	uint8_t rx_queues_count,
>  	bool unique_data,
> +	uint8_t max_priority,
>  	struct rte_flow_error *error);
> 
>  #endif /* FLOW_PERF_FLOW_GEN */
> diff --git a/app/test-flow-perf/main.c b/app/test-flow-perf/main.c 
> index 3ebc025fb2..1d91f308fd 100644
> --- a/app/test-flow-perf/main.c
> +++ b/app/test-flow-perf/main.c
> @@ -77,6 +77,7 @@ static uint32_t rules_count;  static uint32_t 
> rules_batch; static uint32_t hairpin_queues_num; /* total hairpin q 
> number - default: 0 */ static uint32_t nb_lcores;
> +static uint8_t max_priority;
> 
>  #define MAX_PKT_BURST    32
>  #define LCORE_MODE_PKT    1
> @@ -140,6 +141,7 @@ usage(char *progname)
>  	printf("  --enable-fwd: To enable packets forwarding"
>  		" after insertion\n");
>  	printf("  --portmask=N: hexadecimal bitmask of ports used\n");
> +	printf("  --max-priority=N: Maximum priority level for flows\n");

It talks also about random values, you need to mention that it will do the random also.

Ack.

>  	printf("  --unique-data: flag to set using unique data for all"
>  		" actions that support data, such as header modify and encap 
> actions\n");
> 
> @@ -589,6 +591,7 @@ args_parse(int argc, char **argv)
>  		{ "unique-data",                0, 0, 0 },
>  		{ "portmask",                   1, 0, 0 },
>  		{ "cores",                      1, 0, 0 },
> +		{ "max-priority",               1, 0, 0 },
>  		{ "meter-profile-alg",          1, 0, 0 },
>  		{ "rxq",                        1, 0, 0 },
>  		{ "txq",                        1, 0, 0 },
> @@ -767,26 +770,21 @@ args_parse(int argc, char **argv)
>  			/* Control */
>  			if (strcmp(lgopts[opt_idx].name,
>  					"rules-batch") == 0) {
> -				n = atoi(optarg);
> -				if (n >= DEFAULT_RULES_BATCH)
> -					rules_batch = n;
> -				else {
> -					rte_exit(EXIT_FAILURE,
> -						"rules_batch should be >=
> %d\n",
> -						DEFAULT_RULES_BATCH);
> -				}
> +				rules_batch = atoi(optarg);
>  			}
>  			if (strcmp(lgopts[opt_idx].name,
>  					"rules-count") == 0) {
> -				n = atoi(optarg);
> -				if (n >= (int) rules_batch)
> -					rules_count = n;
> -				else {
> -					rte_exit(EXIT_FAILURE,
> -						"rules_count should be >=
> %d\n",
> -						rules_batch);
> +				rules_count = atoi(optarg);
> +			}
> +			if (strcmp(lgopts[opt_idx].name, "max-priority") ==
> 0) {
> +				max_priority = atoi(optarg);
> +				if (max_priority > 32) {
> +					rte_exit(
> +						EXIT_FAILURE,
> +						"max-priority cannot be >
> 32\n");

The 32 is some PMD limitation? I don't think all PMDs will have the same limitation, please remove this condition.

Ack.

>  				}
>  			}
> +
>  			if (strcmp(lgopts[opt_idx].name,
>  					"dump-iterations") == 0)
>  				dump_iterations = true;
> @@ -862,6 +860,16 @@ args_parse(int argc, char **argv)
>  			break;
>  		}
>  	}
> +	if (rules_count % rules_batch != 0) {
> +		rte_exit(EXIT_FAILURE,
> +			 "rules_count %% rules_batch should be 0\n");
> +	}
> +	if (rules_count / rules_batch > MAX_BATCHES_COUNT) {
> +		rte_exit(EXIT_FAILURE,
> +			 "rules_count / rules_batch should be <= %d\n",
> +			 MAX_BATCHES_COUNT);
> +	}
> +

Thanks for this 😊 

>  	printf("end_flow\n");
>  }
> 
> @@ -1227,7 +1235,7 @@ insert_flows(int port_id, uint8_t core_id)
>  		flow = generate_flow(port_id, 0, flow_attrs,
>  			global_items, global_actions,
>  			flow_group, 0, 0, 0, 0, core_id, rx_queues_count,
> -			unique_data, &error);
> +			unique_data, max_priority, &error);
> 
>  		if (flow == NULL) {
>  			print_flow_error(error);
> @@ -1244,7 +1252,7 @@ insert_flows(int port_id, uint8_t core_id)
>  			hairpin_queues_num,
>  			encap_data, decap_data,
>  			core_id, rx_queues_count,
> -			unique_data, &error);
> +			unique_data, max_priority, &error);
> 
>  		if (!counter) {
>  			first_flow_latency = (double)
> (rte_get_timer_cycles() - start_batch); @@ -1975,6 +1983,8 @@ main(int 
> argc, char **argv)
> 
>  	printf(":: Flows Count per port: %d\n\n", rules_count);
> 
> +	rte_srand(rte_rdtsc());

This doesn't make sense, the set for pseudo random generator need to store the seed to make sure that you can reproduce the Same numbers.

The current usage is per core and also it passes rte_rdtsc() which is not pre-known value.
I suggest to pass the seed from the same parameter:

--random-priority=lower-upper;seed

Otherwise you cannot replicate the same test case twice.

Ack. But can we have it as "--random-priority=N,S" which would mean random priority between (0, N - 1) will be used in flow creation and S is the seed for pseudo-random number generator?

> +
>  	if (has_meter())
>  		create_meter_profile();
>  	rte_eal_mp_remote_launch(run_rte_flow_handler_cores, NULL, 
> CALL_MAIN); diff --git a/doc/guides/tools/flow-perf.rst 
> b/doc/guides/tools/flow-perf.rst index 0855f88689..0dfdeab832 100644
> --- a/doc/guides/tools/flow-perf.rst
> +++ b/doc/guides/tools/flow-perf.rst
> @@ -100,6 +100,11 @@ The command line options are:
>  	Set the number of needed cores to insert/delete rte_flow rules.
>  	Default cores count is 1.
> 
> +*       ``--max-priority=N``
> +        Maximum priority level for flows. Flows will be created with the
> +        priority attribute set randomly between 0 to N - 1.
> +        Default is 0 and maximum is 32.

I think the max here should be removed also.

Ack.

> +
>  *	``--meter-profile-alg``
>  	Set the traffic metering algorithm.
>  	Example: meter-profile-alg=srtcmp, default algorithm is
> srtcm_rfc2697
> --
> 2.25.4

BRs,
Wisam Jaddo