[dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Add new Rx offload flag `DEV_RX_OFFLOAD_FLOW_MARK` that can be used to
enable/disable PMDs write to `rte_mbuf::hash::fdir::hi` and
`rte_mbuf::ol_flags` when flow actions `RTE_FLOW_ACTION_MARK` and
`RTE_FLOW_ACTION_FLAG` are enabled.

PMDs notify the validity of `rte_mbuf::hash:fdir::hi` to the applcation
by enabling `PKT_RX_FDIR_ID` flag in `rte_mbuf::ol_flags`.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
---
 doc/guides/nics/features.rst           | 12 ++++++++++++
 doc/guides/rel_notes/release_19_11.rst |  9 +++++++++
 lib/librte_ethdev/rte_ethdev.c         |  1 +
 lib/librte_ethdev/rte_ethdev.h         |  1 +
 lib/librte_ethdev/rte_flow.h           |  6 ++++--
 5 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
index d96696801..0d8d08c18 100644
--- a/doc/guides/nics/features.rst
+++ b/doc/guides/nics/features.rst
@@ -588,6 +588,18 @@ Supports packet type parsing and returns a list of supported types.
 * **[related]    API**: ``rte_eth_dev_get_supported_ptypes()``.
 
 
+.. _nic_features_flow_flag_mark:
+
+Flow flag/mark update
+---------------------
+
+Supports flow action type update to ``mbuf.ol_flags`` and ``mbuf.hash.fdir.hi``.
+
+* **[uses]     rte_eth_rxconf,rte_eth_rxmode**: ``offloads:DEV_RX_OFFLOAD_FLOW_MARK``.
+* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_FDIR``, ``mbuf.ol_flags:PKT_RX_FDIR_ID;``,
+  ``mbuf.hash.fdir.hi``
+
+
 .. _nic_features_timesync:
 
 Timesync
diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index faf69b0d9..8593fea0e 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -185,6 +185,15 @@ New Features
   * Added a console command to testpmd app, ``show port (port_id) ptypes`` which
     gives ability to print port supported ptypes in different protocol layers.
 
+* **Added Rx offload flag to enable or disable flow action type update**
+
+  *  Add new Rx offload flag `DEV_RX_OFFLOAD_FLOW_MARK` that can be used to
+     enable/disable PMDs write to `rte_mbuf::hash::fdir::hi` and
+     `rte_mbuf::ol_flags` when flow actions `RTE_FLOW_ACTION_MARK` and
+     `RTE_FLOW_ACTION_FLAG` are enabled.
+  *  PMDs notify the validity of `rte_mbuf::hash:fdir::hi` to the application
+     by enabling `PKT_RX_FDIR_ID` flag in `rte_mbuf::ol_flags`.
+
 
 Removed Items
 -------------
diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 7743205d3..f70065043 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -129,6 +129,7 @@ static const struct {
 	RTE_RX_OFFLOAD_BIT2STR(KEEP_CRC),
 	RTE_RX_OFFLOAD_BIT2STR(SCTP_CKSUM),
 	RTE_RX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM),
+	RTE_RX_OFFLOAD_BIT2STR(FLOW_MARK),
 };
 
 #undef RTE_RX_OFFLOAD_BIT2STR
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index c36c1b631..c4e626da7 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -1048,6 +1048,7 @@ struct rte_eth_conf {
 #define DEV_RX_OFFLOAD_KEEP_CRC		0x00010000
 #define DEV_RX_OFFLOAD_SCTP_CKSUM	0x00020000
 #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
+#define DEV_RX_OFFLOAD_FLOW_MARK	0x00100000
 
 #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
 				 DEV_RX_OFFLOAD_UDP_CKSUM | \
diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
index 4fee10559..6698640fb 100644
--- a/lib/librte_ethdev/rte_flow.h
+++ b/lib/librte_ethdev/rte_flow.h
@@ -1538,7 +1538,8 @@ enum rte_flow_action_type {
 
 	/**
 	 * Attaches an integer value to packets and sets PKT_RX_FDIR and
-	 * PKT_RX_FDIR_ID mbuf flags.
+	 * PKT_RX_FDIR_ID mbuf flags when
+	 * `rx_mode:offloads:DEV_RX_OFFLOAD_FLOW_MARK` is enabled.
 	 *
 	 * See struct rte_flow_action_mark.
 	 */
@@ -1546,7 +1547,8 @@ enum rte_flow_action_type {
 
 	/**
 	 * Flags packets. Similar to MARK without a specific value; only
-	 * sets the PKT_RX_FDIR mbuf flag.
+	 * sets the PKT_RX_FDIR mbuf flag when
+	 * `rx_mode:offloads:DEV_RX_OFFLOAD_FLOW_MARK` is enabled.
 	 *
 	 * No associated configuration structure.
 	 */
-- 
2.17.1

[dpdk-dev] [PATCH 2/2] drivers/net: update Rx flow flag and mark capabilities

[pbhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Add DEV_RX_OFFLOAD_FLOW_MARK flag for all PMDs that support flow action
flag and mark.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
---
 drivers/net/bnxt/bnxt_ethdev.c          |  3 ++-
 drivers/net/enic/enic_res.c             |  3 ++-
 drivers/net/i40e/i40e_ethdev.c          |  3 ++-
 drivers/net/iavf/iavf_ethdev.c          |  3 ++-
 drivers/net/ice/ice_ethdev.c            |  3 ++-
 drivers/net/ixgbe/ixgbe_rxtx.c          |  3 ++-
 drivers/net/mlx5/mlx5_rxq.c             |  3 ++-
 drivers/net/octeontx2/otx2_ethdev.h     | 11 ++++++-----
 drivers/net/octeontx2/otx2_flow.c       |  9 ++-------
 drivers/net/octeontx2/otx2_flow.h       |  1 -
 drivers/net/octeontx2/otx2_flow_parse.c |  5 +----
 drivers/net/sfc/sfc_ef10_essb_rx.c      |  2 +-
 12 files changed, 24 insertions(+), 25 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 7d9459f0a..c6de6ae91 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -123,7 +123,8 @@ static const struct rte_pci_id bnxt_pci_id_map[] = {
 				     DEV_RX_OFFLOAD_KEEP_CRC | \
 				     DEV_RX_OFFLOAD_VLAN_EXTEND | \
 				     DEV_RX_OFFLOAD_TCP_LRO | \
-				     DEV_RX_OFFLOAD_SCATTER)
+				     DEV_RX_OFFLOAD_SCATTER | \
+				     DEV_RX_OFFLOAD_FLOW_MARK)
 
 static int bnxt_vlan_offload_set_op(struct rte_eth_dev *dev, int mask);
 static void bnxt_print_link_info(struct rte_eth_dev *eth_dev);
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index f403a0b66..1d15c58bc 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -205,7 +205,8 @@ int enic_get_vnic_config(struct enic *enic)
 		DEV_RX_OFFLOAD_VLAN_STRIP |
 		DEV_RX_OFFLOAD_IPV4_CKSUM |
 		DEV_RX_OFFLOAD_UDP_CKSUM |
-		DEV_RX_OFFLOAD_TCP_CKSUM;
+		DEV_RX_OFFLOAD_TCP_CKSUM |
+		DEV_RX_OFFLOAD_FLOW_MARK;
 	enic->tx_offload_mask =
 		PKT_TX_IPV6 |
 		PKT_TX_IPV4 |
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index 77a46832c..df8da5c21 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -3613,7 +3613,8 @@ i40e_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 		DEV_RX_OFFLOAD_SCATTER |
 		DEV_RX_OFFLOAD_VLAN_EXTEND |
 		DEV_RX_OFFLOAD_VLAN_FILTER |
-		DEV_RX_OFFLOAD_JUMBO_FRAME;
+		DEV_RX_OFFLOAD_JUMBO_FRAME |
+		DEV_RX_OFFLOAD_FLOW_MARK;
 
 	dev_info->tx_queue_offload_capa = DEV_TX_OFFLOAD_MBUF_FAST_FREE;
 	dev_info->tx_offload_capa =
diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index eebc49ade..f1491da20 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -522,7 +522,8 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 		DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM |
 		DEV_RX_OFFLOAD_SCATTER |
 		DEV_RX_OFFLOAD_JUMBO_FRAME |
-		DEV_RX_OFFLOAD_VLAN_FILTER;
+		DEV_RX_OFFLOAD_VLAN_FILTER |
+		DEV_RX_OFFLOAD_FLOW_MARK;
 	dev_info->tx_offload_capa =
 		DEV_TX_OFFLOAD_VLAN_INSERT |
 		DEV_TX_OFFLOAD_QINQ_INSERT |
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index d74675842..b5b2c865c 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2800,7 +2800,8 @@ ice_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 			DEV_RX_OFFLOAD_TCP_CKSUM |
 			DEV_RX_OFFLOAD_QINQ_STRIP |
 			DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM |
-			DEV_RX_OFFLOAD_VLAN_EXTEND;
+			DEV_RX_OFFLOAD_VLAN_EXTEND |
+			DEV_RX_OFFLOAD_FLOW_MARK;
 		dev_info->tx_offload_capa |=
 			DEV_TX_OFFLOAD_QINQ_INSERT |
 			DEV_TX_OFFLOAD_IPV4_CKSUM |
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
index edcfa60ce..a86c93cea 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx.c
@@ -2872,7 +2872,8 @@ ixgbe_get_rx_port_offloads(struct rte_eth_dev *dev)
 		   DEV_RX_OFFLOAD_KEEP_CRC    |
 		   DEV_RX_OFFLOAD_JUMBO_FRAME |
 		   DEV_RX_OFFLOAD_VLAN_FILTER |
-		   DEV_RX_OFFLOAD_SCATTER;
+		   DEV_RX_OFFLOAD_SCATTER |
+		   DEV_RX_OFFLOAD_FLOW_MARK;
 
 	if (hw->mac.type == ixgbe_mac_82598EB)
 		offloads |= DEV_RX_OFFLOAD_VLAN_STRIP;
diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c
index f0ab8438d..f8f2e2b22 100644
--- a/drivers/net/mlx5/mlx5_rxq.c
+++ b/drivers/net/mlx5/mlx5_rxq.c
@@ -368,7 +368,8 @@ mlx5_get_rx_queue_offloads(struct rte_eth_dev *dev)
 	struct mlx5_dev_config *config = &priv->config;
 	uint64_t offloads = (DEV_RX_OFFLOAD_SCATTER |
 			     DEV_RX_OFFLOAD_TIMESTAMP |
-			     DEV_RX_OFFLOAD_JUMBO_FRAME);
+			     DEV_RX_OFFLOAD_JUMBO_FRAME |
+			     DEV_RX_OFFLOAD_FLOW_MARK);
 
 	if (config->hw_fcs_strip)
 		offloads |= DEV_RX_OFFLOAD_KEEP_CRC;
diff --git a/drivers/net/octeontx2/otx2_ethdev.h b/drivers/net/octeontx2/otx2_ethdev.h
index 4d9ed4870..ed293e2cd 100644
--- a/drivers/net/octeontx2/otx2_ethdev.h
+++ b/drivers/net/octeontx2/otx2_ethdev.h
@@ -140,11 +140,12 @@
 	DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM | \
 	DEV_RX_OFFLOAD_SCATTER		| \
 	DEV_RX_OFFLOAD_JUMBO_FRAME	| \
-	DEV_RX_OFFLOAD_OUTER_UDP_CKSUM | \
-	DEV_RX_OFFLOAD_VLAN_STRIP | \
-	DEV_RX_OFFLOAD_VLAN_FILTER | \
-	DEV_RX_OFFLOAD_QINQ_STRIP | \
-	DEV_RX_OFFLOAD_TIMESTAMP)
+	DEV_RX_OFFLOAD_OUTER_UDP_CKSUM	| \
+	DEV_RX_OFFLOAD_VLAN_STRIP	| \
+	DEV_RX_OFFLOAD_VLAN_FILTER	| \
+	DEV_RX_OFFLOAD_QINQ_STRIP	| \
+	DEV_RX_OFFLOAD_TIMESTAMP	| \
+	DEV_RX_OFFLOAD_FLOW_MARK)
 
 #define NIX_DEFAULT_RSS_CTX_GROUP  0
 #define NIX_DEFAULT_RSS_MCAM_IDX  -1
diff --git a/drivers/net/octeontx2/otx2_flow.c b/drivers/net/octeontx2/otx2_flow.c
index bdbf123a9..ea4e380b1 100644
--- a/drivers/net/octeontx2/otx2_flow.c
+++ b/drivers/net/octeontx2/otx2_flow.c
@@ -524,11 +524,8 @@ otx2_flow_destroy(struct rte_eth_dev *dev,
 		NIX_RX_ACT_MATCH_MASK;
 
 	if (match_id && match_id < OTX2_FLOW_ACTION_FLAG_DEFAULT) {
-		if (rte_atomic32_read(&npc->mark_actions) == 0)
-			return -EINVAL;
-
-		/* Clear mark offload flag if there are no more mark actions */
-		if (rte_atomic32_sub_return(&npc->mark_actions, 1) == 0) {
+		/* Clear mark offload flag if there is no more mark action */
+		if (hw->rx_offloads & DEV_RX_OFFLOAD_FLOW_MARK) {
 			hw->rx_offload_flags &= ~NIX_RX_OFFLOAD_MARK_UPDATE_F;
 			otx2_eth_set_rx_function(dev);
 		}
@@ -821,8 +818,6 @@ otx2_flow_init(struct otx2_eth_dev *hw)
 		return rc;
 	}
 
-	rte_atomic32_init(&npc->mark_actions);
-
 	npc->mcam_entries = NPC_MCAM_TOT_ENTRIES >> npc->keyw[NPC_MCAM_RX];
 	/* Free, free_rev, live and live_rev entries */
 	bmap_sz = rte_bitmap_get_memory_footprint(npc->mcam_entries);
diff --git a/drivers/net/octeontx2/otx2_flow.h b/drivers/net/octeontx2/otx2_flow.h
index ab068b088..85129cc9d 100644
--- a/drivers/net/octeontx2/otx2_flow.h
+++ b/drivers/net/octeontx2/otx2_flow.h
@@ -160,7 +160,6 @@ TAILQ_HEAD(otx2_flow_list, rte_flow);
 
 /* Accessed from ethdev private - otx2_eth_dev */
 struct otx2_npc_flow_info {
-	rte_atomic32_t mark_actions;
 	uint32_t keyx_supp_nmask[NPC_MAX_INTF];/* nibble mask */
 	uint32_t keyx_len[NPC_MAX_INTF];	/* per intf key len in bits */
 	uint32_t datax_len[NPC_MAX_INTF];	/* per intf data len in bits */
diff --git a/drivers/net/octeontx2/otx2_flow_parse.c b/drivers/net/octeontx2/otx2_flow_parse.c
index 2cba0a447..44f1b0cac 100644
--- a/drivers/net/octeontx2/otx2_flow_parse.c
+++ b/drivers/net/octeontx2/otx2_flow_parse.c
@@ -761,7 +761,6 @@ otx2_flow_parse_actions(struct rte_eth_dev *dev,
 			struct rte_flow *flow)
 {
 	struct otx2_eth_dev *hw = dev->data->dev_private;
-	struct otx2_npc_flow_info *npc = &hw->npc_flow;
 	const struct rte_flow_action_count *act_count;
 	const struct rte_flow_action_mark *act_mark;
 	const struct rte_flow_action_queue *act_q;
@@ -795,13 +794,11 @@ otx2_flow_parse_actions(struct rte_eth_dev *dev,
 			}
 			mark = act_mark->id + 1;
 			req_act |= OTX2_FLOW_ACT_MARK;
-			rte_atomic32_inc(&npc->mark_actions);
 			break;
 
 		case RTE_FLOW_ACTION_TYPE_FLAG:
 			mark = OTX2_FLOW_FLAG_VAL;
 			req_act |= OTX2_FLOW_ACT_FLAG;
-			rte_atomic32_inc(&npc->mark_actions);
 			break;
 
 		case RTE_FLOW_ACTION_TYPE_COUNT:
@@ -979,7 +976,7 @@ otx2_flow_parse_actions(struct rte_eth_dev *dev,
 	if (mark)
 		flow->npc_action |= (uint64_t)mark << 40;
 
-	if (rte_atomic32_read(&npc->mark_actions) == 1) {
+	if (hw->rx_offloads & DEV_RX_OFFLOAD_FLOW_MARK) {
 		hw->rx_offload_flags |=
 			NIX_RX_OFFLOAD_MARK_UPDATE_F;
 		otx2_eth_set_rx_function(dev);
diff --git a/drivers/net/sfc/sfc_ef10_essb_rx.c b/drivers/net/sfc/sfc_ef10_essb_rx.c
index 63da807ea..8c69fd8dd 100644
--- a/drivers/net/sfc/sfc_ef10_essb_rx.c
+++ b/drivers/net/sfc/sfc_ef10_essb_rx.c
@@ -716,7 +716,7 @@ struct sfc_dp_rx sfc_ef10_essb_rx = {
 	.features		= SFC_DP_RX_FEAT_FLOW_FLAG |
 				  SFC_DP_RX_FEAT_FLOW_MARK,
 	.dev_offload_capa	= DEV_RX_OFFLOAD_CHECKSUM,
-	.queue_offload_capa	= 0,
+	.queue_offload_capa	= DEV_RX_OFFLOAD_FLOW_MARK,
 	.get_dev_info		= sfc_ef10_essb_rx_get_dev_info,
 	.pool_ops_supported	= sfc_ef10_essb_rx_pool_ops_supported,
 	.qsize_up_rings		= sfc_ef10_essb_rx_qsize_up_rings,
-- 
2.17.1

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

Hi Pavan,

Sorry for jumping in late.

I don't understand why we need this feature. If the user didn't set any flow with MARK
then the user doesn't need to check it.

Also it breaks compatibility. 

If my understanding is correct the MARK field is going to be moved to dynamic field, and this 
will be way to control the use of MARK.

Other option is inside the PMD if one flow uses mark only than copy the mark to mbuf.

Please also not that we have deprecation notice on the FDIR.

Thanks,
Ori



> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of pbhagavatula@marvell.com
> Sent: Friday, October 25, 2019 6:22 PM
> To: ferruh.yigit@intel.com; arybchenko@solarflare.com; jerinj@marvell.com;
> John McNamara <john.mcnamara@intel.com>; Marko Kovacevic
> <marko.kovacevic@intel.com>; Thomas Monjalon <thomas@monjalon.net>;
> Adrien Mazarguil <adrien.mazarguil@6wind.com>
> Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
> Subject: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Add new Rx offload flag `DEV_RX_OFFLOAD_FLOW_MARK` that can be used to
> enable/disable PMDs write to `rte_mbuf::hash::fdir::hi` and
> `rte_mbuf::ol_flags` when flow actions `RTE_FLOW_ACTION_MARK` and
> `RTE_FLOW_ACTION_FLAG` are enabled.
> 
> PMDs notify the validity of `rte_mbuf::hash:fdir::hi` to the applcation
> by enabling `PKT_RX_FDIR_ID` flag in `rte_mbuf::ol_flags`.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
> ---
>  doc/guides/nics/features.rst           | 12 ++++++++++++
>  doc/guides/rel_notes/release_19_11.rst |  9 +++++++++
>  lib/librte_ethdev/rte_ethdev.c         |  1 +
>  lib/librte_ethdev/rte_ethdev.h         |  1 +
>  lib/librte_ethdev/rte_flow.h           |  6 ++++--
>  5 files changed, 27 insertions(+), 2 deletions(-)
> 
> diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
> index d96696801..0d8d08c18 100644
> --- a/doc/guides/nics/features.rst
> +++ b/doc/guides/nics/features.rst
> @@ -588,6 +588,18 @@ Supports packet type parsing and returns a list of
> supported types.
>  * **[related]    API**: ``rte_eth_dev_get_supported_ptypes()``.
> 
> 
> +.. _nic_features_flow_flag_mark:
> +
> +Flow flag/mark update
> +---------------------
> +
> +Supports flow action type update to ``mbuf.ol_flags`` and ``mbuf.hash.fdir.hi``.
> +
> +* **[uses]     rte_eth_rxconf,rte_eth_rxmode**:
> ``offloads:DEV_RX_OFFLOAD_FLOW_MARK``.
> +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_FDIR``,
> ``mbuf.ol_flags:PKT_RX_FDIR_ID;``,
> +  ``mbuf.hash.fdir.hi``
> +
> +
>  .. _nic_features_timesync:
> 
>  Timesync
> diff --git a/doc/guides/rel_notes/release_19_11.rst
> b/doc/guides/rel_notes/release_19_11.rst
> index faf69b0d9..8593fea0e 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -185,6 +185,15 @@ New Features
>    * Added a console command to testpmd app, ``show port (port_id) ptypes``
> which
>      gives ability to print port supported ptypes in different protocol layers.
> 
> +* **Added Rx offload flag to enable or disable flow action type update**
> +
> +  *  Add new Rx offload flag `DEV_RX_OFFLOAD_FLOW_MARK` that can be
> used to
> +     enable/disable PMDs write to `rte_mbuf::hash::fdir::hi` and
> +     `rte_mbuf::ol_flags` when flow actions `RTE_FLOW_ACTION_MARK` and
> +     `RTE_FLOW_ACTION_FLAG` are enabled.
> +  *  PMDs notify the validity of `rte_mbuf::hash:fdir::hi` to the application
> +     by enabling `PKT_RX_FDIR_ID` flag in `rte_mbuf::ol_flags`.
> +
> 
>  Removed Items
>  -------------
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 7743205d3..f70065043 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -129,6 +129,7 @@ static const struct {
>  	RTE_RX_OFFLOAD_BIT2STR(KEEP_CRC),
>  	RTE_RX_OFFLOAD_BIT2STR(SCTP_CKSUM),
>  	RTE_RX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM),
> +	RTE_RX_OFFLOAD_BIT2STR(FLOW_MARK),
>  };
> 
>  #undef RTE_RX_OFFLOAD_BIT2STR
> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
> index c36c1b631..c4e626da7 100644
> --- a/lib/librte_ethdev/rte_ethdev.h
> +++ b/lib/librte_ethdev/rte_ethdev.h
> @@ -1048,6 +1048,7 @@ struct rte_eth_conf {
>  #define DEV_RX_OFFLOAD_KEEP_CRC		0x00010000
>  #define DEV_RX_OFFLOAD_SCTP_CKSUM	0x00020000
>  #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
> +#define DEV_RX_OFFLOAD_FLOW_MARK	0x00100000
> 
>  #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
>  				 DEV_RX_OFFLOAD_UDP_CKSUM | \
> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
> index 4fee10559..6698640fb 100644
> --- a/lib/librte_ethdev/rte_flow.h
> +++ b/lib/librte_ethdev/rte_flow.h
> @@ -1538,7 +1538,8 @@ enum rte_flow_action_type {
> 
>  	/**
>  	 * Attaches an integer value to packets and sets PKT_RX_FDIR and
> -	 * PKT_RX_FDIR_ID mbuf flags.
> +	 * PKT_RX_FDIR_ID mbuf flags when
> +	 * `rx_mode:offloads:DEV_RX_OFFLOAD_FLOW_MARK` is enabled.
>  	 *
>  	 * See struct rte_flow_action_mark.
>  	 */
> @@ -1546,7 +1547,8 @@ enum rte_flow_action_type {
> 
>  	/**
>  	 * Flags packets. Similar to MARK without a specific value; only
> -	 * sets the PKT_RX_FDIR mbuf flag.
> +	 * sets the PKT_RX_FDIR mbuf flag when
> +	 * `rx_mode:offloads:DEV_RX_OFFLOAD_FLOW_MARK` is enabled.
>  	 *
>  	 * No associated configuration structure.
>  	 */
> --
> 2.17.1

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

Hi Ori,

On 10/28/19 1:50 PM, Ori Kam wrote:
> Hi Pavan,
>
> Sorry for jumping in late.
>
> I don't understand why we need this feature. If the user didn't set any flow with MARK
> then the user doesn't need to check it.

There is pretty long discussion on the topic already, please, read [1].

[1] 
http://inbox.dpdk.org/dev/3251fc00-7598-1c4f-fc2a-380065f0a435@solarflare.com/

> Also it breaks compatibility.

Yes, there is a deprecation notice for it.

> If my understanding is correct the MARK field is going to be moved to dynamic field, and this
> will be way to control the use of MARK.

Yes and I think the offload should used to request dynamic
field register. Similar to timestamp in dynamic mbuf examples.
Application requests Rx timestamp offload, PMD registers dynamic
filed.

> Other option is inside the PMD if one flow uses mark only than copy the mark to mbuf.
>
> Please also not that we have deprecation notice on the FDIR.
>
> Thanks,
> Ori
>
>
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of pbhagavatula@marvell.com
>> Sent: Friday, October 25, 2019 6:22 PM
>> To: ferruh.yigit@intel.com; arybchenko@solarflare.com; jerinj@marvell.com;
>> John McNamara <john.mcnamara@intel.com>; Marko Kovacevic
>> <marko.kovacevic@intel.com>; Thomas Monjalon <thomas@monjalon.net>;
>> Adrien Mazarguil <adrien.mazarguil@6wind.com>
>> Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
>> Subject: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
>> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
>>
>> Add new Rx offload flag `DEV_RX_OFFLOAD_FLOW_MARK` that can be used to
>> enable/disable PMDs write to `rte_mbuf::hash::fdir::hi` and
>> `rte_mbuf::ol_flags` when flow actions `RTE_FLOW_ACTION_MARK` and
>> `RTE_FLOW_ACTION_FLAG` are enabled.
>>
>> PMDs notify the validity of `rte_mbuf::hash:fdir::hi` to the applcation
>> by enabling `PKT_RX_FDIR_ID` flag in `rte_mbuf::ol_flags`.
>>
>> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
>> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
>> ---
>>   doc/guides/nics/features.rst           | 12 ++++++++++++
>>   doc/guides/rel_notes/release_19_11.rst |  9 +++++++++
>>   lib/librte_ethdev/rte_ethdev.c         |  1 +
>>   lib/librte_ethdev/rte_ethdev.h         |  1 +
>>   lib/librte_ethdev/rte_flow.h           |  6 ++++--
>>   5 files changed, 27 insertions(+), 2 deletions(-)
>>
>> diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
>> index d96696801..0d8d08c18 100644
>> --- a/doc/guides/nics/features.rst
>> +++ b/doc/guides/nics/features.rst
>> @@ -588,6 +588,18 @@ Supports packet type parsing and returns a list of
>> supported types.
>>   * **[related]    API**: ``rte_eth_dev_get_supported_ptypes()``.
>>
>>
>> +.. _nic_features_flow_flag_mark:
>> +
>> +Flow flag/mark update
>> +---------------------
>> +
>> +Supports flow action type update to ``mbuf.ol_flags`` and ``mbuf.hash.fdir.hi``.
>> +
>> +* **[uses]     rte_eth_rxconf,rte_eth_rxmode**:
>> ``offloads:DEV_RX_OFFLOAD_FLOW_MARK``.
>> +* **[provides] mbuf**: ``mbuf.ol_flags:PKT_RX_FDIR``,
>> ``mbuf.ol_flags:PKT_RX_FDIR_ID;``,
>> +  ``mbuf.hash.fdir.hi``
>> +
>> +
>>   .. _nic_features_timesync:
>>
>>   Timesync
>> diff --git a/doc/guides/rel_notes/release_19_11.rst
>> b/doc/guides/rel_notes/release_19_11.rst
>> index faf69b0d9..8593fea0e 100644
>> --- a/doc/guides/rel_notes/release_19_11.rst
>> +++ b/doc/guides/rel_notes/release_19_11.rst
>> @@ -185,6 +185,15 @@ New Features
>>     * Added a console command to testpmd app, ``show port (port_id) ptypes``
>> which
>>       gives ability to print port supported ptypes in different protocol layers.
>>
>> +* **Added Rx offload flag to enable or disable flow action type update**
>> +
>> +  *  Add new Rx offload flag `DEV_RX_OFFLOAD_FLOW_MARK` that can be
>> used to
>> +     enable/disable PMDs write to `rte_mbuf::hash::fdir::hi` and
>> +     `rte_mbuf::ol_flags` when flow actions `RTE_FLOW_ACTION_MARK` and
>> +     `RTE_FLOW_ACTION_FLAG` are enabled.
>> +  *  PMDs notify the validity of `rte_mbuf::hash:fdir::hi` to the application
>> +     by enabling `PKT_RX_FDIR_ID` flag in `rte_mbuf::ol_flags`.
>> +
>>
>>   Removed Items
>>   -------------
>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index 7743205d3..f70065043 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -129,6 +129,7 @@ static const struct {
>>   	RTE_RX_OFFLOAD_BIT2STR(KEEP_CRC),
>>   	RTE_RX_OFFLOAD_BIT2STR(SCTP_CKSUM),
>>   	RTE_RX_OFFLOAD_BIT2STR(OUTER_UDP_CKSUM),
>> +	RTE_RX_OFFLOAD_BIT2STR(FLOW_MARK),
>>   };
>>
>>   #undef RTE_RX_OFFLOAD_BIT2STR
>> diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
>> index c36c1b631..c4e626da7 100644
>> --- a/lib/librte_ethdev/rte_ethdev.h
>> +++ b/lib/librte_ethdev/rte_ethdev.h
>> @@ -1048,6 +1048,7 @@ struct rte_eth_conf {
>>   #define DEV_RX_OFFLOAD_KEEP_CRC		0x00010000
>>   #define DEV_RX_OFFLOAD_SCTP_CKSUM	0x00020000
>>   #define DEV_RX_OFFLOAD_OUTER_UDP_CKSUM  0x00040000
>> +#define DEV_RX_OFFLOAD_FLOW_MARK	0x00100000
>>
>>   #define DEV_RX_OFFLOAD_CHECKSUM (DEV_RX_OFFLOAD_IPV4_CKSUM | \
>>   				 DEV_RX_OFFLOAD_UDP_CKSUM | \
>> diff --git a/lib/librte_ethdev/rte_flow.h b/lib/librte_ethdev/rte_flow.h
>> index 4fee10559..6698640fb 100644
>> --- a/lib/librte_ethdev/rte_flow.h
>> +++ b/lib/librte_ethdev/rte_flow.h
>> @@ -1538,7 +1538,8 @@ enum rte_flow_action_type {
>>
>>   	/**
>>   	 * Attaches an integer value to packets and sets PKT_RX_FDIR and
>> -	 * PKT_RX_FDIR_ID mbuf flags.
>> +	 * PKT_RX_FDIR_ID mbuf flags when
>> +	 * `rx_mode:offloads:DEV_RX_OFFLOAD_FLOW_MARK` is enabled.
>>   	 *
>>   	 * See struct rte_flow_action_mark.
>>   	 */
>> @@ -1546,7 +1547,8 @@ enum rte_flow_action_type {
>>
>>   	/**
>>   	 * Flags packets. Similar to MARK without a specific value; only
>> -	 * sets the PKT_RX_FDIR mbuf flag.
>> +	 * sets the PKT_RX_FDIR mbuf flag when
>> +	 * `rx_mode:offloads:DEV_RX_OFFLOAD_FLOW_MARK` is enabled.
>>   	 *
>>   	 * No associated configuration structure.
>>   	 */
>> --
>> 2.17.1

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

Hi Andrew,

> -----Original Message-----
> From: Andrew Rybchenko <arybchenko@solarflare.com>
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> Hi Ori,
> 
> On 10/28/19 1:50 PM, Ori Kam wrote:
> > Hi Pavan,
> >
> > Sorry for jumping in late.
> >
> > I don't understand why we need this feature. If the user didn't set any flow
> with MARK
> > then the user doesn't need to check it.
> 
> There is pretty long discussion on the topic already, please, read [1].
> 
> [1]
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
> 

Thanks for the link, it was an interesting reading.

> > Also it breaks compatibility.
> 
> Yes, there is a deprecation notice for it.
> 
> > If my understanding is correct the MARK field is going to be moved to
> dynamic field, and this
> > will be way to control the use of MARK.
> 
> Yes and I think the offload should used to request dynamic
> field register. Similar to timestamp in dynamic mbuf examples.
> Application requests Rx timestamp offload, PMD registers dynamic
> filed.
> 

In general it was decided that there will be no capability for rte_flow API, due to the fact that 
it is impossible to support all possible combinations. For example a PMD can allow mark on Rx
while not supporting it on e-switch (transfer) or on Tx.
The only way to validate it is validating a flow. If the flow is validated then the action is supported.
This is the exact approach we are implementing with the Meta feature.
So as I see it, the logic should be something like this:
1. run devconfigure.
2. allocate mempool
3. setup queues.
4. run rte_flow_validate with mark action.
If flow validated register mark in mbuf else don't register.
If the PMD needs some special setting for mark he can update the queue when he gets the flow to validate.
At this stage the device is not started so any change is allowed.





> > Other option is inside the PMD if one flow uses mark only than copy the mark
> to mbuf.
> >
> > Please also not that we have deprecation notice on the FDIR.
> >
> > Thanks,
> > Ori

Thanks,
Ori 

[snip]

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 10/28/19 5:00 PM, Ori Kam wrote:
> Hi Andrew,
>
>> -----Original Message-----
>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
>> Hi Ori,
>>
>> On 10/28/19 1:50 PM, Ori Kam wrote:
>>> Hi Pavan,
>>>
>>> Sorry for jumping in late.
>>>
>>> I don't understand why we need this feature. If the user didn't set any flow
>> with MARK
>>> then the user doesn't need to check it.
>> There is pretty long discussion on the topic already, please, read [1].
>>
>> [1]
>> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
>> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
>> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
>> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
>>
> Thanks for the link, it was an interesting reading.
>
>>> Also it breaks compatibility.
>> Yes, there is a deprecation notice for it.
>>
>>> If my understanding is correct the MARK field is going to be moved to
>> dynamic field, and this
>>> will be way to control the use of MARK.
>> Yes and I think the offload should used to request dynamic
>> field register. Similar to timestamp in dynamic mbuf examples.
>> Application requests Rx timestamp offload, PMD registers dynamic
>> filed.
>>
> In general it was decided that there will be no capability for rte_flow API, due to the fact that
> it is impossible to support all possible combinations. For example a PMD can allow mark on Rx
> while not supporting it on e-switch (transfer) or on Tx.
> The only way to validate it is validating a flow. If the flow is validated then the action is supported.
> This is the exact approach we are implementing with the Meta feature.
> So as I see it, the logic should be something like this:
> 1. run devconfigure.
> 2. allocate mempool
> 3. setup queues.
> 4. run rte_flow_validate with mark action.
> If flow validated register mark in mbuf else don't register.
> If the PMD needs some special setting for mark he can update the queue when he gets the flow to validate.
> At this stage the device is not started so any change is allowed.

I understand why there is capability reporting in rte_flow API when
it is about rte_flow API itself. The problem appears when rte_flow
API starts to interact with other functionality.
Which pattern/actions should application try in order to decide
if MARK is supported or not. The right answer is a pattern/action
which will be really used, but what to do if there are many
combinations or if these combinations are not know in advance.
Minimal? But I easily imagine cases when minimal is not supported,
but more complex real life patterns are supported.

The main idea behind the offload is as much as you know in advance
as much you can optimize without overcomplicating drivers and HW.

In the case of OVS, absence MARK offload would mean that OVS
should not even try to use partial offload even if it is enabled.
So, no efforts are required to try to convert flow into pattern and
validate the flow rule.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

31/10/2019 10:49, Andrew Rybchenko:
> On 10/28/19 5:00 PM, Ori Kam wrote:
> >> -----Original Message-----
> > From: Andrew Rybchenko <arybchenko@solarflare.com>
> >> On 10/28/19 1:50 PM, Ori Kam wrote:
> >>> Hi Pavan,
> >>>
> >>> Sorry for jumping in late.
> >>>
> >>> I don't understand why we need this feature. If the user didn't set any flow
> >> with MARK
> >>> then the user doesn't need to check it.
> >> There is pretty long discussion on the topic already, please, read [1].
> >>
> >> [1]
> >> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
> >> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
> >> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
> >> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
> >> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
> >> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
> >>
> > Thanks for the link, it was an interesting reading.
> >
> >>> Also it breaks compatibility.
> >> Yes, there is a deprecation notice for it.
> >>
> >>> If my understanding is correct the MARK field is going to be moved to
> >> dynamic field, and this
> >>> will be way to control the use of MARK.
> >> Yes and I think the offload should used to request dynamic
> >> field register. Similar to timestamp in dynamic mbuf examples.
> >> Application requests Rx timestamp offload, PMD registers dynamic
> >> filed.
> >>
> > In general it was decided that there will be no capability for rte_flow API, due to the fact that
> > it is impossible to support all possible combinations. For example a PMD can allow mark on Rx
> > while not supporting it on e-switch (transfer) or on Tx.
> > The only way to validate it is validating a flow. If the flow is validated then the action is supported.
> > This is the exact approach we are implementing with the Meta feature.
> > So as I see it, the logic should be something like this:
> > 1. run devconfigure.
> > 2. allocate mempool
> > 3. setup queues.
> > 4. run rte_flow_validate with mark action.
> > If flow validated register mark in mbuf else don't register.
> > If the PMD needs some special setting for mark he can update the queue when he gets the flow to validate.
> > At this stage the device is not started so any change is allowed.
> 
> I understand why there is capability reporting in rte_flow API when
> it is about rte_flow API itself. The problem appears when rte_flow
> API starts to interact with other functionality.
> Which pattern/actions should application try in order to decide
> if MARK is supported or not.

Why application should decide whether MARK is supported or not?
In my understanding it can be enabled dynamically per flow.

> The right answer is a pattern/action
> which will be really used, but what to do if there are many
> combinations or if these combinations are not know in advance.
> Minimal? But I easily imagine cases when minimal is not supported,
> but more complex real life patterns are supported.
> 
> The main idea behind the offload is as much as you know in advance
> as much you can optimize without overcomplicating drivers and HW.
> 
> In the case of OVS, absence MARK offload would mean that OVS
> should not even try to use partial offload even if it is enabled.
> So, no efforts are required to try to convert flow into pattern and
> validate the flow rule.

That's an interesting feedback.
I would like to understand why OVS cannot adapt its datapath on demand
per port, per queue and per flow?

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Zhang, Qi Z]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> Sent: Thursday, October 31, 2019 10:50 PM
> To: Andrew Rybchenko <arybchenko@solarflare.com>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>;
> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>; Kovacevic,
> Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> 31/10/2019 10:49, Andrew Rybchenko:
> > On 10/28/19 5:00 PM, Ori Kam wrote:
> > >> -----Original Message-----
> > > From: Andrew Rybchenko <arybchenko@solarflare.com>
> > >> On 10/28/19 1:50 PM, Ori Kam wrote:
> > >>> Hi Pavan,
> > >>>
> > >>> Sorry for jumping in late.
> > >>>
> > >>> I don't understand why we need this feature. If the user didn't
> > >>> set any flow
> > >> with MARK
> > >>> then the user doesn't need to check it.
> > >> There is pretty long discussion on the topic already, please, read [1].
> > >>
> > >> [1]
> > >> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fin
> > >> box.dpdk
> > >> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
> > >>
> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mella
> n
> > >>
> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a
> 4
> > >>
> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3
> C
> > >> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
> > >>
> > > Thanks for the link, it was an interesting reading.
> > >
> > >>> Also it breaks compatibility.
> > >> Yes, there is a deprecation notice for it.
> > >>
> > >>> If my understanding is correct the MARK field is going to be moved
> > >>> to
> > >> dynamic field, and this
> > >>> will be way to control the use of MARK.
> > >> Yes and I think the offload should used to request dynamic field
> > >> register. Similar to timestamp in dynamic mbuf examples.
> > >> Application requests Rx timestamp offload, PMD registers dynamic
> > >> filed.
> > >>
> > > In general it was decided that there will be no capability for
> > > rte_flow API, due to the fact that it is impossible to support all
> > > possible combinations. For example a PMD can allow mark on Rx while not
> supporting it on e-switch (transfer) or on Tx.
> > > The only way to validate it is validating a flow. If the flow is validated then
> the action is supported.
> > > This is the exact approach we are implementing with the Meta feature.
> > > So as I see it, the logic should be something like this:
> > > 1. run devconfigure.
> > > 2. allocate mempool
> > > 3. setup queues.
> > > 4. run rte_flow_validate with mark action.
> > > If flow validated register mark in mbuf else don't register.
> > > If the PMD needs some special setting for mark he can update the queue
> when he gets the flow to validate.
> > > At this stage the device is not started so any change is allowed.
> >
> > I understand why there is capability reporting in rte_flow API when it
> > is about rte_flow API itself. The problem appears when rte_flow API
> > starts to interact with other functionality.
> > Which pattern/actions should application try in order to decide if
> > MARK is supported or not.
> 
> Why application should decide whether MARK is supported or not?
> In my understanding it can be enabled dynamically per flow.

Sorry to break in the discussion, I think the mark offload will give below benefits base on some real cases.
1. for PMD which not enable mark offload on all data paths, for example, the vector PMD does not support mark, but non vector PMD does,
In this case, the offload can give driver a hint to choose the correct data path, otherwise, when vPMD is selected at dev_start, a flow with mark action has to be rejected.
2. extract the 32 bit mark from rx descriptor have considerable performance cost, especially on vPMD.
so it will be nice if the driver knows that mark offload is not necessary for application, then it can always select a faster path. 
while, driver can track when the first flow with mark are issued and the last flow with mark is delete than branch the code for mark extraction properly, but this just give driver another option to simply this

Regards
Qi

> 
> > The right answer is a pattern/action
> > which will be really used, but what to do if there are many
> > combinations or if these combinations are not know in advance.
> > Minimal? But I easily imagine cases when minimal is not supported, but
> > more complex real life patterns are supported.
> >
> > The main idea behind the offload is as much as you know in advance as
> > much you can optimize without overcomplicating drivers and HW.
> >
> > In the case of OVS, absence MARK offload would mean that OVS should
> > not even try to use partial offload even if it is enabled.
> > So, no efforts are required to try to convert flow into pattern and
> > validate the flow rule.
> 
> That's an interesting feedback.
> I would like to understand why OVS cannot adapt its datapath on demand per
> port, per queue and per flow?
> 

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 10/31/19 5:49 PM, Thomas Monjalon wrote:
> 31/10/2019 10:49, Andrew Rybchenko:
>> On 10/28/19 5:00 PM, Ori Kam wrote:
>>>> -----Original Message-----
>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>>>> On 10/28/19 1:50 PM, Ori Kam wrote:
>>>>> Hi Pavan,
>>>>>
>>>>> Sorry for jumping in late.
>>>>>
>>>>> I don't understand why we need this feature. If the user didn't set any flow
>>>> with MARK
>>>>> then the user doesn't need to check it.
>>>> There is pretty long discussion on the topic already, please, read [1].
>>>>
>>>> [1]
>>>> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
>>>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
>>>> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
>>>> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
>>>> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
>>>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
>>>>
>>> Thanks for the link, it was an interesting reading.
>>>
>>>>> Also it breaks compatibility.
>>>> Yes, there is a deprecation notice for it.
>>>>
>>>>> If my understanding is correct the MARK field is going to be moved to
>>>> dynamic field, and this
>>>>> will be way to control the use of MARK.
>>>> Yes and I think the offload should used to request dynamic
>>>> field register. Similar to timestamp in dynamic mbuf examples.
>>>> Application requests Rx timestamp offload, PMD registers dynamic
>>>> filed.
>>>>
>>> In general it was decided that there will be no capability for rte_flow API, due to the fact that
>>> it is impossible to support all possible combinations. For example a PMD can allow mark on Rx
>>> while not supporting it on e-switch (transfer) or on Tx.
>>> The only way to validate it is validating a flow. If the flow is validated then the action is supported.
>>> This is the exact approach we are implementing with the Meta feature.
>>> So as I see it, the logic should be something like this:
>>> 1. run devconfigure.
>>> 2. allocate mempool
>>> 3. setup queues.
>>> 4. run rte_flow_validate with mark action.
>>> If flow validated register mark in mbuf else don't register.
>>> If the PMD needs some special setting for mark he can update the queue when he gets the flow to validate.
>>> At this stage the device is not started so any change is allowed.
>> I understand why there is capability reporting in rte_flow API when
>> it is about rte_flow API itself. The problem appears when rte_flow
>> API starts to interact with other functionality.
>> Which pattern/actions should application try in order to decide
>> if MARK is supported or not.
> Why application should decide whether MARK is supported or not?
> In my understanding it can be enabled dynamically per flow.

Yes, it is per flow right now, but it is resource consuming to
make a flow rule just to discard it and work without offload.
The application already suffers and attempt to use hardware
offload makes it suffer even more. Of course, hardware offload
in application may be simply globall disabled, but presence of
MARK offload allows to do it dynamically based on offload
reported by PMD.

Also I think that Qi has a good example for vPMD why
MARK offload would be useful.

>> The right answer is a pattern/action
>> which will be really used, but what to do if there are many
>> combinations or if these combinations are not know in advance.
>> Minimal? But I easily imagine cases when minimal is not supported,
>> but more complex real life patterns are supported.
>>
>> The main idea behind the offload is as much as you know in advance
>> as much you can optimize without overcomplicating drivers and HW.
>>
>> In the case of OVS, absence MARK offload would mean that OVS
>> should not even try to use partial offload even if it is enabled.
>> So, no efforts are required to try to convert flow into pattern and
>> validate the flow rule.
> That's an interesting feedback.
> I would like to understand why OVS cannot adapt its datapath on demand
> per port, per queue and per flow?

I guess there is a misunderstanding here. What I'm trying to say
is that introduction of MARK offload would make code a bit more
simple and efficient. Basically it would be possible to enable
so-called hardware offload in OVS by default, but finally make
a decision per port based on MARK offload availability
(should it try to make rte_flow rule by flow and insert it?)

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

Hi,

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Andrew Rybchenko
> Sent: Friday, November 1, 2019 1:35 PM
> To: Thomas Monjalon <thomas@monjalon.net>
> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>;
> pbhagavatula@marvell.com; ferruh.yigit@intel.com; jerinj@marvell.com; John
> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
> <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> On 10/31/19 5:49 PM, Thomas Monjalon wrote:
> > 31/10/2019 10:49, Andrew Rybchenko:
> >> On 10/28/19 5:00 PM, Ori Kam wrote:
> >>>> -----Original Message-----
> >>> From: Andrew Rybchenko <arybchenko@solarflare.com>
> >>>> On 10/28/19 1:50 PM, Ori Kam wrote:
> >>>>> Hi Pavan,
> >>>>>
> >>>>> Sorry for jumping in late.
> >>>>>
> >>>>> I don't understand why we need this feature. If the user didn't set any
> flow
> >>>> with MARK
> >>>>> then the user doesn't need to check it.
> >>>> There is pretty long discussion on the topic already, please, read [1].
> >>>>
> >>>> [1]
> >>>>
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
> >>>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
> >>>>
> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
> >>>>
> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
> >>>>
> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
> >>>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
> >>>>
> >>> Thanks for the link, it was an interesting reading.
> >>>
> >>>>> Also it breaks compatibility.
> >>>> Yes, there is a deprecation notice for it.
> >>>>
> >>>>> If my understanding is correct the MARK field is going to be moved to
> >>>> dynamic field, and this
> >>>>> will be way to control the use of MARK.
> >>>> Yes and I think the offload should used to request dynamic
> >>>> field register. Similar to timestamp in dynamic mbuf examples.
> >>>> Application requests Rx timestamp offload, PMD registers dynamic
> >>>> filed.
> >>>>
> >>> In general it was decided that there will be no capability for rte_flow API,
> due to the fact that
> >>> it is impossible to support all possible combinations. For example a PMD
> can allow mark on Rx
> >>> while not supporting it on e-switch (transfer) or on Tx.
> >>> The only way to validate it is validating a flow. If the flow is validated then
> the action is supported.
> >>> This is the exact approach we are implementing with the Meta feature.
> >>> So as I see it, the logic should be something like this:
> >>> 1. run devconfigure.
> >>> 2. allocate mempool
> >>> 3. setup queues.
> >>> 4. run rte_flow_validate with mark action.
> >>> If flow validated register mark in mbuf else don't register.
> >>> If the PMD needs some special setting for mark he can update the queue
> when he gets the flow to validate.
> >>> At this stage the device is not started so any change is allowed.
> >> I understand why there is capability reporting in rte_flow API when
> >> it is about rte_flow API itself. The problem appears when rte_flow
> >> API starts to interact with other functionality.
> >> Which pattern/actions should application try in order to decide
> >> if MARK is supported or not.
> > Why application should decide whether MARK is supported or not?
> > In my understanding it can be enabled dynamically per flow.
> 
> Yes, it is per flow right now, but it is resource consuming to
> make a flow rule just to discard it and work without offload.
> The application already suffers and attempt to use hardware
> offload makes it suffer even more. Of course, hardware offload
> in application may be simply globall disabled, but presence of
> MARK offload allows to do it dynamically based on offload
> reported by PMD.
> 
> Also I think that Qi has a good example for vPMD why
> MARK offload would be useful.
> 

I don't think that creating a simple flow during startup is resource consuming.
I think as we move more and more to rte_flow we can't continue using offloads.
The fact that one PMD doesn't support mark first should be listed in a release notes,
In Qi example the application can start with it's preferred PMD and test if its support the mark action,
if not try other PMD or use some fallback. Think about it like this, assume that one PMD support some other
rte_flow while the second PMD doesn't support it. so the application should decide which is more important
to it and enable the best PMD.

> >> The right answer is a pattern/action
> >> which will be really used, but what to do if there are many
> >> combinations or if these combinations are not know in advance.
> >> Minimal? But I easily imagine cases when minimal is not supported,
> >> but more complex real life patterns are supported.
> >>
> >> The main idea behind the offload is as much as you know in advance
> >> as much you can optimize without overcomplicating drivers and HW.
> >>
> >> In the case of OVS, absence MARK offload would mean that OVS
> >> should not even try to use partial offload even if it is enabled.
> >> So, no efforts are required to try to convert flow into pattern and
> >> validate the flow rule.
> > That's an interesting feedback.
> > I would like to understand why OVS cannot adapt its datapath on demand
> > per port, per queue and per flow?
> 
> I guess there is a misunderstanding here. What I'm trying to say
> is that introduction of MARK offload would make code a bit more
> simple and efficient. Basically it would be possible to enable
> so-called hardware offload in OVS by default, but finally make
> a decision per port based on MARK offload availability
> (should it try to make rte_flow rule by flow and insert it?)

Like I said the PMD can check if it mark is avail in the buff. So he can selected the
best function.

Best,
Ori

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/3/19 1:22 PM, Ori Kam wrote:
> Hi,
>
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Andrew Rybchenko
>> Sent: Friday, November 1, 2019 1:35 PM
>> To: Thomas Monjalon <thomas@monjalon.net>
>> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>;
>> pbhagavatula@marvell.com; ferruh.yigit@intel.com; jerinj@marvell.com; John
>> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
>> <marko.kovacevic@intel.com>; Adrien Mazarguil
>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>> ktraynor@redhat.com
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
>> On 10/31/19 5:49 PM, Thomas Monjalon wrote:
>>> 31/10/2019 10:49, Andrew Rybchenko:
>>>> On 10/28/19 5:00 PM, Ori Kam wrote:
>>>>>> -----Original Message-----
>>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>>>>>> On 10/28/19 1:50 PM, Ori Kam wrote:
>>>>>>> Hi Pavan,
>>>>>>>
>>>>>>> Sorry for jumping in late.
>>>>>>>
>>>>>>> I don't understand why we need this feature. If the user didn't set any
>> flow
>>>>>> with MARK
>>>>>>> then the user doesn't need to check it.
>>>>>> There is pretty long discussion on the topic already, please, read [1].
>>>>>>
>>>>>> [1]
>>>>>>
>> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
>>>>>>> Best,
>>>>>>> Ori
>>>>>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
>>>>>>
>> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
>> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
>> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
>>>>>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
>>>>>>
>>>>> Thanks for the link, it was an interesting reading.
>>>>>
>>>>>>> Also it breaks compatibility.
>>>>>> Yes, there is a deprecation notice for it.
>>>>>>
>>>>>>> If my understanding is correct the MARK field is going to be moved to
>>>>>> dynamic field, and this
>>>>>>> will be way to control the use of MARK.
>>>>>> Yes and I think the offload should used to request dynamic
>>>>>> field register. Similar to timestamp in dynamic mbuf examples.
>>>>>> Application requests Rx timestamp offload, PMD registers dynamic
>>>>>> filed.
>>>>>>
>>>>> In general it was decided that there will be no capability for rte_flow API,
>> due to the fact that
>>>>> it is impossible to support all possible combinations. For example a PMD
>> can allow mark on Rx
>>>>> while not supporting it on e-switch (transfer) or on Tx.
>>>>> The only way to validate it is validating a flow. If the flow is validated then
>> the action is supported.
>>>>> This is the exact approach we are implementing with the Meta feature.
>>>>> So as I see it, the logic should be something like this:
>>>>> 1. run devconfigure.
>>>>> 2. allocate mempool
>>>>> 3. setup queues.
>>>>> 4. run rte_flow_validate with mark action.
>>>>> If flow validated register mark in mbuf else don't register.
>>>>> If the PMD needs some special setting for mark he can update the queue
>> when he gets the flow to validate.
>>>>> At this stage the device is not started so any change is allowed.
>>>> I understand why there is capability reporting in rte_flow API when
>>>> it is about rte_flow API itself. The problem appears when rte_flow
>>>> API starts to interact with other functionality.
>>>> Which pattern/actions should application try in order to decide
>>>> if MARK is supported or not.
>>> Why application should decide whether MARK is supported or not?
>>> In my understanding it can be enabled dynamically per flow.
>> Yes, it is per flow right now, but it is resource consuming to
>> make a flow rule just to discard it and work without offload.
>> The application already suffers and attempt to use hardware
>> offload makes it suffer even more. Of course, hardware offload
>> in application may be simply globall disabled, but presence of
>> MARK offload allows to do it dynamically based on offload
>> reported by PMD.
>>
>> Also I think that Qi has a good example for vPMD why
>> MARK offload would be useful.
>>
> I don't think that creating a simple flow during startup is resource consuming.

It is not about startup. It is for every rule which will be rejected since
MARK is not supported.

> I think as we move more and more to rte_flow we can't continue using offloads.
> The fact that one PMD doesn't support mark first should be listed in a release notes,

Release notes are nice, but it is nothing for automated processing.

> In Qi example the application can start with it's preferred PMD and test if its support the mark action,
> if not try other PMD or use some fallback.

Pretty often there is no direct control over PMD to use. It is either 
vendor specific
or no control at all. PMD choice is the result of requested offloads.

> Think about it like this, assume that one PMD support some other
> rte_flow while the second PMD doesn't support it. so the application should decide which is more important
> to it and enable the best PMD.

Again, it is not about filtering only. It is delivery of the extra 
information which requires
extra processing and extra resources.

>>>> The right answer is a pattern/action
>>>> which will be really used, but what to do if there are many
>>>> combinations or if these combinations are not know in advance.
>>>> Minimal? But I easily imagine cases when minimal is not supported,
>>>> but more complex real life patterns are supported.
>>>>
>>>> The main idea behind the offload is as much as you know in advance
>>>> as much you can optimize without overcomplicating drivers and HW.
>>>>
>>>> In the case of OVS, absence MARK offload would mean that OVS
>>>> should not even try to use partial offload even if it is enabled.
>>>> So, no efforts are required to try to convert flow into pattern and
>>>> validate the flow rule.
>>> That's an interesting feedback.
>>> I would like to understand why OVS cannot adapt its datapath on demand
>>> per port, per queue and per flow?
>> I guess there is a misunderstanding here. What I'm trying to say
>> is that introduction of MARK offload would make code a bit more
>> simple and efficient. Basically it would be possible to enable
>> so-called hardware offload in OVS by default, but finally make
>> a decision per port based on MARK offload availability
>> (should it try to make rte_flow rule by flow and insert it?)
> Like I said the PMD can check if it mark is avail in the buff. So he can selected the
> best function.

Which buff?

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

> -----Original Message-----
> From: Andrew Rybchenko <arybchenko@solarflare.com>
> Sent: Sunday, November 3, 2019 1:41 PM
> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> On 11/3/19 1:22 PM, Ori Kam wrote:
> > Hi,
> >
> >> -----Original Message-----
> >> From: dev <dev-bounces@dpdk.org> On Behalf Of Andrew Rybchenko
> >> Sent: Friday, November 1, 2019 1:35 PM
> >> To: Thomas Monjalon <thomas@monjalon.net>
> >> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>;
> >> pbhagavatula@marvell.com; ferruh.yigit@intel.com; jerinj@marvell.com;
> John
> >> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
> >> <marko.kovacevic@intel.com>; Adrien Mazarguil
> >> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> >> ktraynor@redhat.com
> >> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
> an
> >> offload
> >>
> >> On 10/31/19 5:49 PM, Thomas Monjalon wrote:
> >>> 31/10/2019 10:49, Andrew Rybchenko:
> >>>> On 10/28/19 5:00 PM, Ori Kam wrote:
> >>>>>> -----Original Message-----
> >>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
> >>>>>> On 10/28/19 1:50 PM, Ori Kam wrote:
> >>>>>>> Hi Pavan,
> >>>>>>>
> >>>>>>> Sorry for jumping in late.
> >>>>>>>
> >>>>>>> I don't understand why we need this feature. If the user didn't set any
> >> flow
> >>>>>> with MARK
> >>>>>>> then the user doesn't need to check it.
> >>>>>> There is pretty long discussion on the topic already, please, read [1].
> >>>>>>
> >>>>>> [1]
> >>>>>>
> >>
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
> >>>>>>> Best,
> >>>>>>> Ori
> >>>>>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
> >>>>>>
> >>
> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
> >>
> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
> >>
> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
> >>>>>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
> >>>>>>
> >>>>> Thanks for the link, it was an interesting reading.
> >>>>>
> >>>>>>> Also it breaks compatibility.
> >>>>>> Yes, there is a deprecation notice for it.
> >>>>>>
> >>>>>>> If my understanding is correct the MARK field is going to be moved to
> >>>>>> dynamic field, and this
> >>>>>>> will be way to control the use of MARK.
> >>>>>> Yes and I think the offload should used to request dynamic
> >>>>>> field register. Similar to timestamp in dynamic mbuf examples.
> >>>>>> Application requests Rx timestamp offload, PMD registers dynamic
> >>>>>> filed.
> >>>>>>
> >>>>> In general it was decided that there will be no capability for rte_flow
> API,
> >> due to the fact that
> >>>>> it is impossible to support all possible combinations. For example a PMD
> >> can allow mark on Rx
> >>>>> while not supporting it on e-switch (transfer) or on Tx.
> >>>>> The only way to validate it is validating a flow. If the flow is validated
> then
> >> the action is supported.
> >>>>> This is the exact approach we are implementing with the Meta feature.
> >>>>> So as I see it, the logic should be something like this:
> >>>>> 1. run devconfigure.
> >>>>> 2. allocate mempool
> >>>>> 3. setup queues.
> >>>>> 4. run rte_flow_validate with mark action.
> >>>>> If flow validated register mark in mbuf else don't register.
> >>>>> If the PMD needs some special setting for mark he can update the queue
> >> when he gets the flow to validate.
> >>>>> At this stage the device is not started so any change is allowed.
> >>>> I understand why there is capability reporting in rte_flow API when
> >>>> it is about rte_flow API itself. The problem appears when rte_flow
> >>>> API starts to interact with other functionality.
> >>>> Which pattern/actions should application try in order to decide
> >>>> if MARK is supported or not.
> >>> Why application should decide whether MARK is supported or not?
> >>> In my understanding it can be enabled dynamically per flow.
> >> Yes, it is per flow right now, but it is resource consuming to
> >> make a flow rule just to discard it and work without offload.
> >> The application already suffers and attempt to use hardware
> >> offload makes it suffer even more. Of course, hardware offload
> >> in application may be simply globall disabled, but presence of
> >> MARK offload allows to do it dynamically based on offload
> >> reported by PMD.
> >>
> >> Also I think that Qi has a good example for vPMD why
> >> MARK offload would be useful.
> >>
> > I don't think that creating a simple flow during startup is resource consuming.
> 
> It is not about startup. It is for every rule which will be rejected since
> MARK is not supported.
> 

I'm sorry but I don't understand. Like everything else in rte_flow, if flow validation failed,
the flow should not be tested again. It is pointless. If the flow got rejected then the application
should set it's logic accordingly and not use MARK flows.
Same for example the pmd doesn't support encap, so after the application tries and fails, it should
understand that it must do the encap in SW.

> > I think as we move more and more to rte_flow we can't continue using
> offloads.
> > The fact that one PMD doesn't support mark first should be listed in a release
> notes,
> 
> Release notes are nice, but it is nothing for automated processing.
> 

Agree, this is why is should be tested using rte_flow.

> > In Qi example the application can start with it's preferred PMD and test if its
> support the mark action,
> > if not try other PMD or use some fallback.
> 
> Pretty often there is no direct control over PMD to use. It is either
> vendor specific
> or no control at all. PMD choice is the result of requested offloads.
>

We have different definition for PMD 😊. If I understand you correctly you mean
which function the PMD uses internally, am I correct?
If so it is the responsibility of the PMD that when application tries to validate a flow with mark
to switch the functions / PMD to a PMD that supports this kind of action.
Again think about it if the user want meta for example, the PMD should assume that the application wants this
kind of action and use the matching PMD.  
 
> > Think about it like this, assume that one PMD support some other
> > rte_flow while the second PMD doesn't support it. so the application should
> decide which is more important
> > to it and enable the best PMD.
> 
> Again, it is not about filtering only. It is delivery of the extra
> information which requires
> extra processing and extra resources.
> 

Please see my comment above regarding metadata.
I understand your point, we have 2 different components, the rte_flow
and the mbuf. One is used by HW and one is used by SW.
A feature like type is right in the middle. I had related argument with Slava
about where to place the dynamic mbuf function for the meta. It was decided
it is more rte_flow than mbuf.
Also think about it like this, if (and I hope we can) in next generation we will be able to 
write data to the mbuf from the HW. This will mean it will be only used in rte_flow, and not 
in the rx_burst function. 

> >>>> The right answer is a pattern/action
> >>>> which will be really used, but what to do if there are many
> >>>> combinations or if these combinations are not know in advance.
> >>>> Minimal? But I easily imagine cases when minimal is not supported,
> >>>> but more complex real life patterns are supported.
> >>>>
> >>>> The main idea behind the offload is as much as you know in advance
> >>>> as much you can optimize without overcomplicating drivers and HW.
> >>>>
> >>>> In the case of OVS, absence MARK offload would mean that OVS
> >>>> should not even try to use partial offload even if it is enabled.
> >>>> So, no efforts are required to try to convert flow into pattern and
> >>>> validate the flow rule.
> >>> That's an interesting feedback.
> >>> I would like to understand why OVS cannot adapt its datapath on demand
> >>> per port, per queue and per flow?
> >> I guess there is a misunderstanding here. What I'm trying to say
> >> is that introduction of MARK offload would make code a bit more
> >> simple and efficient. Basically it would be possible to enable
> >> so-called hardware offload in OVS by default, but finally make
> >> a decision per port based on MARK offload availability
> >> (should it try to make rte_flow rule by flow and insert it?)
> > Like I said the PMD can check if it mark is avail in the buff. So he can selected
> the
> > best function.
> 
> Which buff?

Sorry mbuf. I thought it was decided also to move the mark to dynamic mbuf.

Just to summarize I'm afraid that we will have more fields like the mark (we already do with the 
metadata) and we might even get more.
I don't want to have capability over rte_flow actions, since it can be very tricky for example 
PMD can support only mark or meta but not both, while other will be able to support both.
some will have limitation on the mark action some PMD will not (for example size)
so in any case the application must validate it.
Or if you want other example we can support mark both in transfer and normal mode. This is only
true from this DPDK version. So what should PMD that supports mark only on non transfer flows report?


Best,
Ori

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/4/19 9:37 PM, Ori Kam wrote:
>> -----Original Message-----
>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>> Sent: Sunday, November 3, 2019 1:41 PM
>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>> ktraynor@redhat.com
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
>> On 11/3/19 1:22 PM, Ori Kam wrote:
>>> Hi,
>>>
>>>> -----Original Message-----
>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Andrew Rybchenko
>>>> Sent: Friday, November 1, 2019 1:35 PM
>>>> To: Thomas Monjalon <thomas@monjalon.net>
>>>> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>;
>>>> pbhagavatula@marvell.com; ferruh.yigit@intel.com; jerinj@marvell.com;
>> John
>>>> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
>>>> <marko.kovacevic@intel.com>; Adrien Mazarguil
>>>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>>>> ktraynor@redhat.com
>>>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
>> an
>>>> offload
>>>>
>>>> On 10/31/19 5:49 PM, Thomas Monjalon wrote:
>>>>> 31/10/2019 10:49, Andrew Rybchenko:
>>>>>> On 10/28/19 5:00 PM, Ori Kam wrote:
>>>>>>>> -----Original Message-----
>>>>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>>>>>>>> On 10/28/19 1:50 PM, Ori Kam wrote:
>>>>>>>>> Hi Pavan,
>>>>>>>>>
>>>>>>>>> Sorry for jumping in late.
>>>>>>>>>
>>>>>>>>> I don't understand why we need this feature. If the user didn't set any
>>>> flow
>>>>>>>> with MARK
>>>>>>>>> then the user doesn't need to check it.
>>>>>>>> There is pretty long discussion on the topic already, please, read [1].
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
>>>>>>>>> Best,
>>>>>>>>> Ori
>>>>>>>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
>>>>>>>>
>> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
>> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
>> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
>>>>>>>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
>>>>>>>>
>>>>>>> Thanks for the link, it was an interesting reading.
>>>>>>>
>>>>>>>>> Also it breaks compatibility.
>>>>>>>> Yes, there is a deprecation notice for it.
>>>>>>>>
>>>>>>>>> If my understanding is correct the MARK field is going to be moved to
>>>>>>>> dynamic field, and this
>>>>>>>>> will be way to control the use of MARK.
>>>>>>>> Yes and I think the offload should used to request dynamic
>>>>>>>> field register. Similar to timestamp in dynamic mbuf examples.
>>>>>>>> Application requests Rx timestamp offload, PMD registers dynamic
>>>>>>>> filed.
>>>>>>>>
>>>>>>> In general it was decided that there will be no capability for rte_flow
>> API,
>>>> due to the fact that
>>>>>>> it is impossible to support all possible combinations. For example a PMD
>>>> can allow mark on Rx
>>>>>>> while not supporting it on e-switch (transfer) or on Tx.
>>>>>>> The only way to validate it is validating a flow. If the flow is validated
>> then
>>>> the action is supported.
>>>>>>> This is the exact approach we are implementing with the Meta feature.
>>>>>>> So as I see it, the logic should be something like this:
>>>>>>> 1. run devconfigure.
>>>>>>> 2. allocate mempool
>>>>>>> 3. setup queues.
>>>>>>> 4. run rte_flow_validate with mark action.
>>>>>>> If flow validated register mark in mbuf else don't register.
>>>>>>> If the PMD needs some special setting for mark he can update the queue
>>>> when he gets the flow to validate.
>>>>>>> At this stage the device is not started so any change is allowed.
>>>>>> I understand why there is capability reporting in rte_flow API when
>>>>>> it is about rte_flow API itself. The problem appears when rte_flow
>>>>>> API starts to interact with other functionality.
>>>>>> Which pattern/actions should application try in order to decide
>>>>>> if MARK is supported or not.
>>>>> Why application should decide whether MARK is supported or not?
>>>>> In my understanding it can be enabled dynamically per flow.
>>>> Yes, it is per flow right now, but it is resource consuming to
>>>> make a flow rule just to discard it and work without offload.
>>>> The application already suffers and attempt to use hardware
>>>> offload makes it suffer even more. Of course, hardware offload
>>>> in application may be simply globall disabled, but presence of
>>>> MARK offload allows to do it dynamically based on offload
>>>> reported by PMD.
>>>>
>>>> Also I think that Qi has a good example for vPMD why
>>>> MARK offload would be useful.
>>>>
>>> I don't think that creating a simple flow during startup is resource consuming.
>> It is not about startup. It is for every rule which will be rejected since
>> MARK is not supported.
>>
> I'm sorry but I don't understand. Like everything else in rte_flow, if flow validation failed,
> the flow should not be tested again. It is pointless. If the flow got rejected then the application
> should set it's logic accordingly and not use MARK flows.
> Same for example the pmd doesn't support encap, so after the application tries and fails, it should
> understand that it must do the encap in SW.

The problem is that flow validation may fail because of various reasons.
May be the first validation fails because of the pattern is not
supported, or
pattern+action is not supported, pattern+action is not supported because
of another rule is installed right now, but would be supported without the
rule installed.

>>> I think as we move more and more to rte_flow we can't continue using
>> offloads.
>>> The fact that one PMD doesn't support mark first should be listed in a release
>> notes,
>>
>> Release notes are nice, but it is nothing for automated processing.
>>
> Agree, this is why is should be tested using rte_flow.
>
>>> In Qi example the application can start with it's preferred PMD and test if its
>> support the mark action,
>>> if not try other PMD or use some fallback.
>> Pretty often there is no direct control over PMD to use. It is either
>> vendor specific
>> or no control at all. PMD choice is the result of requested offloads.
>>
> We have different definition for PMD 😊. If I understand you correctly you mean
> which function the PMD uses internally, am I correct?

Yes, like vPMD is vector-based implementation of Rx/Tx routines.

> If so it is the responsibility of the PMD that when application tries to validate a flow with mark
> to switch the functions / PMD to a PMD that supports this kind of action.

It is impossible when queues are setup and port is running.
Flow rules may be and should be installed when traffic is running.

> Again think about it if the user want meta for example, the PMD should assume that the application wants this
> kind of action and use the matching PMD.  
>  
>>> Think about it like this, assume that one PMD support some other
>>> rte_flow while the second PMD doesn't support it. so the application should
>> decide which is more important
>>> to it and enable the best PMD.
>> Again, it is not about filtering only. It is delivery of the extra
>> information which requires
>> extra processing and extra resources.
>>
> Please see my comment above regarding metadata.
> I understand your point, we have 2 different components, the rte_flow
> and the mbuf. One is used by HW and one is used by SW.
> A feature like type is right in the middle. I had related argument with Slava
> about where to place the dynamic mbuf function for the meta. It was decided
> it is more rte_flow than mbuf.
> Also think about it like this, if (and I hope we can) in next generation we will be able to 
> write data to the mbuf from the HW. This will mean it will be only used in rte_flow, and not 
> in the rx_burst function. 

I think it will not make flow mark rte_flow entity only.

>>>>>> The right answer is a pattern/action
>>>>>> which will be really used, but what to do if there are many
>>>>>> combinations or if these combinations are not know in advance.
>>>>>> Minimal? But I easily imagine cases when minimal is not supported,
>>>>>> but more complex real life patterns are supported.
>>>>>>
>>>>>> The main idea behind the offload is as much as you know in advance
>>>>>> as much you can optimize without overcomplicating drivers and HW.
>>>>>>
>>>>>> In the case of OVS, absence MARK offload would mean that OVS
>>>>>> should not even try to use partial offload even if it is enabled.
>>>>>> So, no efforts are required to try to convert flow into pattern and
>>>>>> validate the flow rule.
>>>>> That's an interesting feedback.
>>>>> I would like to understand why OVS cannot adapt its datapath on demand
>>>>> per port, per queue and per flow?
>>>> I guess there is a misunderstanding here. What I'm trying to say
>>>> is that introduction of MARK offload would make code a bit more
>>>> simple and efficient. Basically it would be possible to enable
>>>> so-called hardware offload in OVS by default, but finally make
>>>> a decision per port based on MARK offload availability
>>>> (should it try to make rte_flow rule by flow and insert it?)
>>> Like I said the PMD can check if it mark is avail in the buff. So he can selected
>> the
>>> best function.
>> Which buff?
> Sorry mbuf. I thought it was decided also to move the mark to dynamic mbuf.
>
> Just to summarize I'm afraid that we will have more fields like the mark (we already do with the 
> metadata) and we might even get more.
> I don't want to have capability over rte_flow actions, since it can be very tricky for example 
> PMD can support only mark or meta but not both, while other will be able to support both.
> some will have limitation on the mark action some PMD will not (for example size)
> so in any case the application must validate it.
> Or if you want other example we can support mark both in transfer and normal mode. This is only
> true from this DPDK version. So what should PMD that supports mark only on non transfer flows report?

MARK offload is just a possibility to deliver MARK from HW to mbuf. All
rte_flow specifics
is out-of-scope. It should be delivered with the packet (or associated
with delivered packet
in some way).

The above also highlights problems of the meta vs mark design. They are very
similar and there is no any good definition of the difference and rules
which
one should be used/supported in which conditions.

Andrew.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

> -----Original Message-----
> From: Andrew Rybchenko <arybchenko@solarflare.com>
> Sent: Tuesday, November 5, 2019 8:50 AM
> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> On 11/4/19 9:37 PM, Ori Kam wrote:
> >> -----Original Message-----
> >> From: Andrew Rybchenko <arybchenko@solarflare.com>
> >> Sent: Sunday, November 3, 2019 1:41 PM
> >> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
> >> <thomas@monjalon.net>
> >> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
> >> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
> >> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
> >> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> >> ktraynor@redhat.com
> >> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
> an
> >> offload
> >>
> >> On 11/3/19 1:22 PM, Ori Kam wrote:
> >>> Hi,
> >>>
> >>>> -----Original Message-----
> >>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Andrew Rybchenko
> >>>> Sent: Friday, November 1, 2019 1:35 PM
> >>>> To: Thomas Monjalon <thomas@monjalon.net>
> >>>> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>;
> >>>> pbhagavatula@marvell.com; ferruh.yigit@intel.com; jerinj@marvell.com;
> >> John
> >>>> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
> >>>> <marko.kovacevic@intel.com>; Adrien Mazarguil
> >>>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> >>>> ktraynor@redhat.com
> >>>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update
> as
> >> an
> >>>> offload
> >>>>
> >>>> On 10/31/19 5:49 PM, Thomas Monjalon wrote:
> >>>>> 31/10/2019 10:49, Andrew Rybchenko:
> >>>>>> On 10/28/19 5:00 PM, Ori Kam wrote:
> >>>>>>>> -----Original Message-----
> >>>>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
> >>>>>>>> On 10/28/19 1:50 PM, Ori Kam wrote:
> >>>>>>>>> Hi Pavan,
> >>>>>>>>>
> >>>>>>>>> Sorry for jumping in late.
> >>>>>>>>>
> >>>>>>>>> I don't understand why we need this feature. If the user didn't set
> any
> >>>> flow
> >>>>>>>> with MARK
> >>>>>>>>> then the user doesn't need to check it.
> >>>>>>>> There is pretty long discussion on the topic already, please, read [1].
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
> >>>>>>>>> Best,
> >>>>>>>>> Ori
> >>>>>>>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
> >>>>>>>>
> >>
> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
> >>
> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
> >>
> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
> >>>>>>>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
> >>>>>>>>
> >>>>>>> Thanks for the link, it was an interesting reading.
> >>>>>>>
> >>>>>>>>> Also it breaks compatibility.
> >>>>>>>> Yes, there is a deprecation notice for it.
> >>>>>>>>
> >>>>>>>>> If my understanding is correct the MARK field is going to be moved
> to
> >>>>>>>> dynamic field, and this
> >>>>>>>>> will be way to control the use of MARK.
> >>>>>>>> Yes and I think the offload should used to request dynamic
> >>>>>>>> field register. Similar to timestamp in dynamic mbuf examples.
> >>>>>>>> Application requests Rx timestamp offload, PMD registers dynamic
> >>>>>>>> filed.
> >>>>>>>>
> >>>>>>> In general it was decided that there will be no capability for rte_flow
> >> API,
> >>>> due to the fact that
> >>>>>>> it is impossible to support all possible combinations. For example a
> PMD
> >>>> can allow mark on Rx
> >>>>>>> while not supporting it on e-switch (transfer) or on Tx.
> >>>>>>> The only way to validate it is validating a flow. If the flow is validated
> >> then
> >>>> the action is supported.
> >>>>>>> This is the exact approach we are implementing with the Meta
> feature.
> >>>>>>> So as I see it, the logic should be something like this:
> >>>>>>> 1. run devconfigure.
> >>>>>>> 2. allocate mempool
> >>>>>>> 3. setup queues.
> >>>>>>> 4. run rte_flow_validate with mark action.
> >>>>>>> If flow validated register mark in mbuf else don't register.
> >>>>>>> If the PMD needs some special setting for mark he can update the
> queue
> >>>> when he gets the flow to validate.
> >>>>>>> At this stage the device is not started so any change is allowed.
> >>>>>> I understand why there is capability reporting in rte_flow API when
> >>>>>> it is about rte_flow API itself. The problem appears when rte_flow
> >>>>>> API starts to interact with other functionality.
> >>>>>> Which pattern/actions should application try in order to decide
> >>>>>> if MARK is supported or not.
> >>>>> Why application should decide whether MARK is supported or not?
> >>>>> In my understanding it can be enabled dynamically per flow.
> >>>> Yes, it is per flow right now, but it is resource consuming to
> >>>> make a flow rule just to discard it and work without offload.
> >>>> The application already suffers and attempt to use hardware
> >>>> offload makes it suffer even more. Of course, hardware offload
> >>>> in application may be simply globall disabled, but presence of
> >>>> MARK offload allows to do it dynamically based on offload
> >>>> reported by PMD.
> >>>>
> >>>> Also I think that Qi has a good example for vPMD why
> >>>> MARK offload would be useful.
> >>>>
> >>> I don't think that creating a simple flow during startup is resource
> consuming.
> >> It is not about startup. It is for every rule which will be rejected since
> >> MARK is not supported.
> >>
> > I'm sorry but I don't understand. Like everything else in rte_flow, if flow
> validation failed,
> > the flow should not be tested again. It is pointless. If the flow got rejected
> then the application
> > should set it's logic accordingly and not use MARK flows.
> > Same for example the pmd doesn't support encap, so after the application
> tries and fails, it should
> > understand that it must do the encap in SW.
> 
> The problem is that flow validation may fail because of various reasons.
> May be the first validation fails because of the pattern is not
> supported, or
> pattern+action is not supported, pattern+action is not supported because
> of another rule is installed right now, but would be supported without the
> rule installed.
> 

You are right, so the application should test with the most basic flow, or the flow that 
represent best what the application wishes. 
Since in this stage it is only validation there is no problem with conflicting flows. 

> >>> I think as we move more and more to rte_flow we can't continue using
> >> offloads.
> >>> The fact that one PMD doesn't support mark first should be listed in a
> release
> >> notes,
> >>
> >> Release notes are nice, but it is nothing for automated processing.
> >>
> > Agree, this is why is should be tested using rte_flow.
> >
> >>> In Qi example the application can start with it's preferred PMD and test if
> its
> >> support the mark action,
> >>> if not try other PMD or use some fallback.
> >> Pretty often there is no direct control over PMD to use. It is either
> >> vendor specific
> >> or no control at all. PMD choice is the result of requested offloads.
> >>
> > We have different definition for PMD 😊. If I understand you correctly you
> mean
> > which function the PMD uses internally, am I correct?
> 
> Yes, like vPMD is vector-based implementation of Rx/Tx routines.
> 
> > If so it is the responsibility of the PMD that when application tries to validate
> a flow with mark
> > to switch the functions / PMD to a PMD that supports this kind of action.
> 
> It is impossible when queues are setup and port is running.
> Flow rules may be and should be installed when traffic is running.
> 

I'm not saying to change the PMD while the traffic is running the validation of a flow
can be done before starting traffic.

> > Again think about it if the user want meta for example, the PMD should
> assume that the application wants this
> > kind of action and use the matching PMD.
> >
> >>> Think about it like this, assume that one PMD support some other
> >>> rte_flow while the second PMD doesn't support it. so the application should
> >> decide which is more important
> >>> to it and enable the best PMD.
> >> Again, it is not about filtering only. It is delivery of the extra
> >> information which requires
> >> extra processing and extra resources.
> >>
> > Please see my comment above regarding metadata.
> > I understand your point, we have 2 different components, the rte_flow
> > and the mbuf. One is used by HW and one is used by SW.
> > A feature like type is right in the middle. I had related argument with Slava
> > about where to place the dynamic mbuf function for the meta. It was decided
> > it is more rte_flow than mbuf.
> > Also think about it like this, if (and I hope we can) in next generation we will
> be able to
> > write data to the mbuf from the HW. This will mean it will be only used in
> rte_flow, and not
> > in the rx_burst function.
> 
> I think it will not make flow mark rte_flow entity only.
> 

But it will be set only using rte_flow and used by application so the Tx/Rx function
play no part in this case.

> >>>>>> The right answer is a pattern/action
> >>>>>> which will be really used, but what to do if there are many
> >>>>>> combinations or if these combinations are not know in advance.
> >>>>>> Minimal? But I easily imagine cases when minimal is not supported,
> >>>>>> but more complex real life patterns are supported.
> >>>>>>
> >>>>>> The main idea behind the offload is as much as you know in advance
> >>>>>> as much you can optimize without overcomplicating drivers and HW.
> >>>>>>
> >>>>>> In the case of OVS, absence MARK offload would mean that OVS
> >>>>>> should not even try to use partial offload even if it is enabled.
> >>>>>> So, no efforts are required to try to convert flow into pattern and
> >>>>>> validate the flow rule.
> >>>>> That's an interesting feedback.
> >>>>> I would like to understand why OVS cannot adapt its datapath on
> demand
> >>>>> per port, per queue and per flow?
> >>>> I guess there is a misunderstanding here. What I'm trying to say
> >>>> is that introduction of MARK offload would make code a bit more
> >>>> simple and efficient. Basically it would be possible to enable
> >>>> so-called hardware offload in OVS by default, but finally make
> >>>> a decision per port based on MARK offload availability
> >>>> (should it try to make rte_flow rule by flow and insert it?)
> >>> Like I said the PMD can check if it mark is avail in the buff. So he can
> selected
> >> the
> >>> best function.
> >> Which buff?
> > Sorry mbuf. I thought it was decided also to move the mark to dynamic mbuf.
> >
> > Just to summarize I'm afraid that we will have more fields like the mark (we
> already do with the
> > metadata) and we might even get more.
> > I don't want to have capability over rte_flow actions, since it can be very
> tricky for example
> > PMD can support only mark or meta but not both, while other will be able to
> support both.
> > some will have limitation on the mark action some PMD will not (for example
> size)
> > so in any case the application must validate it.
> > Or if you want other example we can support mark both in transfer and
> normal mode. This is only
> > true from this DPDK version. So what should PMD that supports mark only on
> non transfer flows report?
> 
> MARK offload is just a possibility to deliver MARK from HW to mbuf. All
> rte_flow specifics
> is out-of-scope. It should be delivered with the packet (or associated
> with delivered packet
> in some way).
> 

Yes but like I said in Mellanox PMD for example we supported the mark only on non-transfer flows until this release.
so when the user set mark on transfer flow it was invalid. (in transfer flow if we have a miss we send the packet back to the Rx
port so the application can understand on which table the miss happened)
In this version we added the support for mark also in transfer (E-Switch) flows.
So my question before this release what should the PMD report? What should the PMD report after this release?

Your idea was our first thought when adding the Tx meta, in that case the meta was always set in application
so we thought that this offload will enable us better function selection, but as you know we removed this capability
since it is not correct any more.



> The above also highlights problems of the meta vs mark design. They are very
> similar and there is no any good definition of the difference and rules
> which
> one should be used/supported in which conditions.
> 

Mark and Meta are exactly the same, the meta is just another value that the application can use.
This is why both should act the same.

And maybe this is the wining argument, the rte_flow validation approach was used and accepted for the meta.
So lets try it also with the mark. (please also remember that we didn't have this mark until now to somehow the 
PMD worked 😊)

Like I said before, I understand your approach, and each one of them has its own advantages and draw backs.
Lets start using the rte_flow approach and see how it goes, I promise you that if I see that it doesn't scale or cause more
issues I will be first one to submit changes.
 

> Andrew.
> 

Best,
Ori

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/5/19 11:35 AM, Ori Kam wrote:
> 
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>> Sent: Tuesday, November 5, 2019 8:50 AM
>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>> ktraynor@redhat.com
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
>> On 11/4/19 9:37 PM, Ori Kam wrote:
>>>> -----Original Message-----
>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>>>> Sent: Sunday, November 3, 2019 1:41 PM
>>>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>>>> <thomas@monjalon.net>
>>>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>>>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>>>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>>>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>>>> ktraynor@redhat.com
>>>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
>> an
>>>> offload
>>>>
>>>> On 11/3/19 1:22 PM, Ori Kam wrote:
>>>>> Hi,
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Andrew Rybchenko
>>>>>> Sent: Friday, November 1, 2019 1:35 PM
>>>>>> To: Thomas Monjalon <thomas@monjalon.net>
>>>>>> Cc: dev@dpdk.org; Ori Kam <orika@mellanox.com>;
>>>>>> pbhagavatula@marvell.com; ferruh.yigit@intel.com; jerinj@marvell.com;
>>>> John
>>>>>> McNamara <john.mcnamara@intel.com>; Marko Kovacevic
>>>>>> <marko.kovacevic@intel.com>; Adrien Mazarguil
>>>>>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>>>>>> ktraynor@redhat.com
>>>>>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update
>> as
>>>> an
>>>>>> offload
>>>>>>
>>>>>> On 10/31/19 5:49 PM, Thomas Monjalon wrote:
>>>>>>> 31/10/2019 10:49, Andrew Rybchenko:
>>>>>>>> On 10/28/19 5:00 PM, Ori Kam wrote:
>>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>>>>>>>>>> On 10/28/19 1:50 PM, Ori Kam wrote:
>>>>>>>>>>> Hi Pavan,
>>>>>>>>>>>
>>>>>>>>>>> Sorry for jumping in late.
>>>>>>>>>>>
>>>>>>>>>>> I don't understand why we need this feature. If the user didn't set
>> any
>>>>>> flow
>>>>>>>>>> with MARK
>>>>>>>>>>> then the user doesn't need to check it.
>>>>>>>>>> There is pretty long discussion on the topic already, please, read [1].
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>>
>>>>
>> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Finbox.dpdk
>>>>>>>>>>> Best,
>>>>>>>>>>> Ori
>>>>>>>>>> .org%2Fdev%2F3251fc00-7598-1c4f-fc2a-
>>>>>>>>>>
>>>>
>> 380065f0a435%40solarflare.com%2F&amp;data=02%7C01%7Corika%40mellan
>>>>
>> ox.com%7Ce3f779d4b7c44b682d6508d75b9d8688%7Ca652971c7d2e4d9ba6a4
>>>>
>> d149256f461b%7C0%7C0%7C637078604439019114&amp;sdata=sYooc%2FQ3C
>>>>>>>>>> kUZG3gRFPlZrm8xMfMB9gOWWex5YIkWhMc%3D&amp;reserved=0
>>>>>>>>>>
>>>>>>>>> Thanks for the link, it was an interesting reading.
>>>>>>>>>
>>>>>>>>>>> Also it breaks compatibility.
>>>>>>>>>> Yes, there is a deprecation notice for it.
>>>>>>>>>>
>>>>>>>>>>> If my understanding is correct the MARK field is going to be moved
>> to
>>>>>>>>>> dynamic field, and this
>>>>>>>>>>> will be way to control the use of MARK.
>>>>>>>>>> Yes and I think the offload should used to request dynamic
>>>>>>>>>> field register. Similar to timestamp in dynamic mbuf examples.
>>>>>>>>>> Application requests Rx timestamp offload, PMD registers dynamic
>>>>>>>>>> filed.
>>>>>>>>>>
>>>>>>>>> In general it was decided that there will be no capability for rte_flow
>>>> API,
>>>>>> due to the fact that
>>>>>>>>> it is impossible to support all possible combinations. For example a
>> PMD
>>>>>> can allow mark on Rx
>>>>>>>>> while not supporting it on e-switch (transfer) or on Tx.
>>>>>>>>> The only way to validate it is validating a flow. If the flow is validated
>>>> then
>>>>>> the action is supported.
>>>>>>>>> This is the exact approach we are implementing with the Meta
>> feature.
>>>>>>>>> So as I see it, the logic should be something like this:
>>>>>>>>> 1. run devconfigure.
>>>>>>>>> 2. allocate mempool
>>>>>>>>> 3. setup queues.
>>>>>>>>> 4. run rte_flow_validate with mark action.
>>>>>>>>> If flow validated register mark in mbuf else don't register.
>>>>>>>>> If the PMD needs some special setting for mark he can update the
>> queue
>>>>>> when he gets the flow to validate.
>>>>>>>>> At this stage the device is not started so any change is allowed.
>>>>>>>> I understand why there is capability reporting in rte_flow API when
>>>>>>>> it is about rte_flow API itself. The problem appears when rte_flow
>>>>>>>> API starts to interact with other functionality.
>>>>>>>> Which pattern/actions should application try in order to decide
>>>>>>>> if MARK is supported or not.
>>>>>>> Why application should decide whether MARK is supported or not?
>>>>>>> In my understanding it can be enabled dynamically per flow.
>>>>>> Yes, it is per flow right now, but it is resource consuming to
>>>>>> make a flow rule just to discard it and work without offload.
>>>>>> The application already suffers and attempt to use hardware
>>>>>> offload makes it suffer even more. Of course, hardware offload
>>>>>> in application may be simply globall disabled, but presence of
>>>>>> MARK offload allows to do it dynamically based on offload
>>>>>> reported by PMD.
>>>>>>
>>>>>> Also I think that Qi has a good example for vPMD why
>>>>>> MARK offload would be useful.
>>>>>>
>>>>> I don't think that creating a simple flow during startup is resource
>> consuming.
>>>> It is not about startup. It is for every rule which will be rejected since
>>>> MARK is not supported.
>>>>
>>> I'm sorry but I don't understand. Like everything else in rte_flow, if flow
>> validation failed,
>>> the flow should not be tested again. It is pointless. If the flow got rejected
>> then the application
>>> should set it's logic accordingly and not use MARK flows.
>>> Same for example the pmd doesn't support encap, so after the application
>> tries and fails, it should
>>> understand that it must do the encap in SW.
>>
>> The problem is that flow validation may fail because of various reasons.
>> May be the first validation fails because of the pattern is not
>> supported, or
>> pattern+action is not supported, pattern+action is not supported because
>> of another rule is installed right now, but would be supported without the
>> rule installed.
>>
> 
> You are right, so the application should test with the most basic flow, or the flow that 
> represent best what the application wishes. 
> Since in this stage it is only validation there is no problem with conflicting flows. 
> 
>>>>> I think as we move more and more to rte_flow we can't continue using
>>>> offloads.
>>>>> The fact that one PMD doesn't support mark first should be listed in a
>> release
>>>> notes,
>>>>
>>>> Release notes are nice, but it is nothing for automated processing.
>>>>
>>> Agree, this is why is should be tested using rte_flow.
>>>
>>>>> In Qi example the application can start with it's preferred PMD and test if
>> its
>>>> support the mark action,
>>>>> if not try other PMD or use some fallback.
>>>> Pretty often there is no direct control over PMD to use. It is either
>>>> vendor specific
>>>> or no control at all. PMD choice is the result of requested offloads.
>>>>
>>> We have different definition for PMD 😊. If I understand you correctly you
>> mean
>>> which function the PMD uses internally, am I correct?
>>
>> Yes, like vPMD is vector-based implementation of Rx/Tx routines.
>>
>>> If so it is the responsibility of the PMD that when application tries to validate
>> a flow with mark
>>> to switch the functions / PMD to a PMD that supports this kind of action.
>>
>> It is impossible when queues are setup and port is running.
>> Flow rules may be and should be installed when traffic is running.
>>
> 
> I'm not saying to change the PMD while the traffic is running the validation of a flow
> can be done before starting traffic.
> 
>>> Again think about it if the user want meta for example, the PMD should
>> assume that the application wants this
>>> kind of action and use the matching PMD.
>>>
>>>>> Think about it like this, assume that one PMD support some other
>>>>> rte_flow while the second PMD doesn't support it. so the application should
>>>> decide which is more important
>>>>> to it and enable the best PMD.
>>>> Again, it is not about filtering only. It is delivery of the extra
>>>> information which requires
>>>> extra processing and extra resources.
>>>>
>>> Please see my comment above regarding metadata.
>>> I understand your point, we have 2 different components, the rte_flow
>>> and the mbuf. One is used by HW and one is used by SW.
>>> A feature like type is right in the middle. I had related argument with Slava
>>> about where to place the dynamic mbuf function for the meta. It was decided
>>> it is more rte_flow than mbuf.
>>> Also think about it like this, if (and I hope we can) in next generation we will
>> be able to
>>> write data to the mbuf from the HW. This will mean it will be only used in
>> rte_flow, and not
>>> in the rx_burst function.
>>
>> I think it will not make flow mark rte_flow entity only.
>>
> 
> But it will be set only using rte_flow and used by application so the Tx/Rx function
> play no part in this case.
> 
>>>>>>>> The right answer is a pattern/action
>>>>>>>> which will be really used, but what to do if there are many
>>>>>>>> combinations or if these combinations are not know in advance.
>>>>>>>> Minimal? But I easily imagine cases when minimal is not supported,
>>>>>>>> but more complex real life patterns are supported.
>>>>>>>>
>>>>>>>> The main idea behind the offload is as much as you know in advance
>>>>>>>> as much you can optimize without overcomplicating drivers and HW.
>>>>>>>>
>>>>>>>> In the case of OVS, absence MARK offload would mean that OVS
>>>>>>>> should not even try to use partial offload even if it is enabled.
>>>>>>>> So, no efforts are required to try to convert flow into pattern and
>>>>>>>> validate the flow rule.
>>>>>>> That's an interesting feedback.
>>>>>>> I would like to understand why OVS cannot adapt its datapath on
>> demand
>>>>>>> per port, per queue and per flow?
>>>>>> I guess there is a misunderstanding here. What I'm trying to say
>>>>>> is that introduction of MARK offload would make code a bit more
>>>>>> simple and efficient. Basically it would be possible to enable
>>>>>> so-called hardware offload in OVS by default, but finally make
>>>>>> a decision per port based on MARK offload availability
>>>>>> (should it try to make rte_flow rule by flow and insert it?)
>>>>> Like I said the PMD can check if it mark is avail in the buff. So he can
>> selected
>>>> the
>>>>> best function.
>>>> Which buff?
>>> Sorry mbuf. I thought it was decided also to move the mark to dynamic mbuf.
>>>
>>> Just to summarize I'm afraid that we will have more fields like the mark (we
>> already do with the
>>> metadata) and we might even get more.
>>> I don't want to have capability over rte_flow actions, since it can be very
>> tricky for example
>>> PMD can support only mark or meta but not both, while other will be able to
>> support both.
>>> some will have limitation on the mark action some PMD will not (for example
>> size)
>>> so in any case the application must validate it.
>>> Or if you want other example we can support mark both in transfer and
>> normal mode. This is only
>>> true from this DPDK version. So what should PMD that supports mark only on
>> non transfer flows report?
>>
>> MARK offload is just a possibility to deliver MARK from HW to mbuf. All
>> rte_flow specifics
>> is out-of-scope. It should be delivered with the packet (or associated
>> with delivered packet
>> in some way).
>>
> 
> Yes but like I said in Mellanox PMD for example we supported the mark only on non-transfer flows until this release.
> so when the user set mark on transfer flow it was invalid. (in transfer flow if we have a miss we send the packet back to the Rx
> port so the application can understand on which table the miss happened)
> In this version we added the support for mark also in transfer (E-Switch) flows.
> So my question before this release what should the PMD report? What should the PMD report after this release?
> 
> Your idea was our first thought when adding the Tx meta, in that case the meta was always set in application
> so we thought that this offload will enable us better function selection, but as you know we removed this capability
> since it is not correct any more.
> 
> 
> 
>> The above also highlights problems of the meta vs mark design. They are very
>> similar and there is no any good definition of the difference and rules
>> which
>> one should be used/supported in which conditions.
>>
> 
> Mark and Meta are exactly the same, the meta is just another value that the application can use.
> This is why both should act the same.
> 
> And maybe this is the wining argument, the rte_flow validation approach was used and accepted for the meta.
> So lets try it also with the mark. (please also remember that we didn't have this mark until now to somehow the 
> PMD worked 😊)
> 
> Like I said before, I understand your approach, and each one of them has its own advantages and draw backs.
> Lets start using the rte_flow approach and see how it goes, I promise you that if I see that it doesn't scale or cause more
> issues I will be first one to submit changes.

I tend to say OK, let's try. However, it must be documented
in MARK action that if an application wants to use it, a rule
with the action must be validated before device start.
PMD may use the attempt as an indication from the application
that it would like to use flow mark even if the validation
fails. Ori, please, suggest formalized pattern and actions
specification to use if application wants to utilize
validation result as a criteria to enable/disable flow
marks usage. What should be done if patterns to use and set
of actions together with MARK are not known in advance.

Andrew.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

> -----Original Message-----
> From: Andrew Rybchenko <arybchenko@solarflare.com>
> Sent: Tuesday, November 5, 2019 1:31 PM
> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 

[Snip]

> >
> > Yes but like I said in Mellanox PMD for example we supported the mark only
> on non-transfer flows until this release.
> > so when the user set mark on transfer flow it was invalid. (in transfer flow if
> we have a miss we send the packet back to the Rx
> > port so the application can understand on which table the miss happened)
> > In this version we added the support for mark also in transfer (E-Switch)
> flows.
> > So my question before this release what should the PMD report? What should
> the PMD report after this release?
> >
> > Your idea was our first thought when adding the Tx meta, in that case the
> meta was always set in application
> > so we thought that this offload will enable us better function selection, but as
> you know we removed this capability
> > since it is not correct any more.
> >
> >
> >
> >> The above also highlights problems of the meta vs mark design. They are
> very
> >> similar and there is no any good definition of the difference and rules
> >> which
> >> one should be used/supported in which conditions.
> >>
> >
> > Mark and Meta are exactly the same, the meta is just another value that the
> application can use.
> > This is why both should act the same.
> >
> > And maybe this is the wining argument, the rte_flow validation approach was
> used and accepted for the meta.
> > So lets try it also with the mark. (please also remember that we didn't have
> this mark until now to somehow the
> > PMD worked 😊)
> >
> > Like I said before, I understand your approach, and each one of them has its
> own advantages and draw backs.
> > Lets start using the rte_flow approach and see how it goes, I promise you that
> if I see that it doesn't scale or cause more
> > issues I will be first one to submit changes.
> 
> I tend to say OK, let's try. However, it must be documented
> in MARK action that if an application wants to use it, a rule
> with the action must be validated before device start.

I agree to add this to the rte_flow mark action documation.

> PMD may use the attempt as an indication from the application
> that it would like to use flow mark even if the validation
> fails. 

No if the PMD uses this validation as hint it should return success and 
use the correct PMD.

Ori, please, suggest formalized pattern and actions
> specification to use if application wants to utilize
> validation result as a criteria to enable/disable flow
> marks usage. 

I can’t do that, it depends on the application, the most basic is just "pattern eth actions mark / queue" .
In some cases where you need it for E-Switch if should be something like "transfer  items port / eth / actions mark"

> What should be done if patterns to use and set
> of actions together with MARK are not known in advance.

I think that the application knows which kind of traffic it expects and which actions it needs.
 
> Andrew.

Ori

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/5/19 7:37 PM, Ori Kam wrote:
> 
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>> Sent: Tuesday, November 5, 2019 1:31 PM
>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>> ktraynor@redhat.com
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
> 
> [Snip]
> 
>>>
>>> Yes but like I said in Mellanox PMD for example we supported the mark only
>> on non-transfer flows until this release.
>>> so when the user set mark on transfer flow it was invalid. (in transfer flow if
>> we have a miss we send the packet back to the Rx
>>> port so the application can understand on which table the miss happened)
>>> In this version we added the support for mark also in transfer (E-Switch)
>> flows.
>>> So my question before this release what should the PMD report? What should
>> the PMD report after this release?
>>>
>>> Your idea was our first thought when adding the Tx meta, in that case the
>> meta was always set in application
>>> so we thought that this offload will enable us better function selection, but as
>> you know we removed this capability
>>> since it is not correct any more.
>>>
>>>
>>>
>>>> The above also highlights problems of the meta vs mark design. They are
>> very
>>>> similar and there is no any good definition of the difference and rules
>>>> which
>>>> one should be used/supported in which conditions.
>>>>
>>>
>>> Mark and Meta are exactly the same, the meta is just another value that the
>> application can use.
>>> This is why both should act the same.
>>>
>>> And maybe this is the wining argument, the rte_flow validation approach was
>> used and accepted for the meta.
>>> So lets try it also with the mark. (please also remember that we didn't have
>> this mark until now to somehow the
>>> PMD worked 😊)
>>>
>>> Like I said before, I understand your approach, and each one of them has its
>> own advantages and draw backs.
>>> Lets start using the rte_flow approach and see how it goes, I promise you that
>> if I see that it doesn't scale or cause more
>>> issues I will be first one to submit changes.
>>
>> I tend to say OK, let's try. However, it must be documented
>> in MARK action that if an application wants to use it, a rule
>> with the action must be validated before device start.
> 
> I agree to add this to the rte_flow mark action documation.
> 
>> PMD may use the attempt as an indication from the application
>> that it would like to use flow mark even if the validation
>> fails. 
> 
> No if the PMD uses this validation as hint it should return success and 
> use the correct PMD.

It would make it too strictly dependent on pattern/actions/state.

>> Ori, please, suggest formalized pattern and actions
>> specification to use if application wants to utilize
>> validation result as a criteria to enable/disable flow
>> marks usage. 
> 
> I can’t do that, it depends on the application, the most basic is just "pattern eth actions mark / queue" .
> In some cases where you need it for E-Switch if should be something like "transfer  items port / eth / actions mark"

If so, what application author should do if even maintainers cannot
formalize it. It sounds like the approach does not work.

>> What should be done if patterns to use and set
>> of actions together with MARK are not known in advance.
> 
> I think that the application knows which kind of traffic it expects and which actions it needs.

I'm afraid it is not always true.

>> Andrew.
> 
> Ori
> 

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

> -----Original Message-----
> From: Andrew Rybchenko <arybchenko@solarflare.com>
> Sent: Wednesday, November 6, 2019 8:41 AM
> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
> <thomas@monjalon.net>
> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> On 11/5/19 7:37 PM, Ori Kam wrote:
> >
> >
> >> -----Original Message-----
> >> From: Andrew Rybchenko <arybchenko@solarflare.com>
> >> Sent: Tuesday, November 5, 2019 1:31 PM
> >> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
> >> <thomas@monjalon.net>
> >> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
> >> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
> >> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
> >> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> >> ktraynor@redhat.com
> >> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
> an
> >> offload
> >>
> >
> > [Snip]
> >
> >>>
> >>> Yes but like I said in Mellanox PMD for example we supported the mark
> only
> >> on non-transfer flows until this release.
> >>> so when the user set mark on transfer flow it was invalid. (in transfer flow
> if
> >> we have a miss we send the packet back to the Rx
> >>> port so the application can understand on which table the miss happened)
> >>> In this version we added the support for mark also in transfer (E-Switch)
> >> flows.
> >>> So my question before this release what should the PMD report? What
> should
> >> the PMD report after this release?
> >>>
> >>> Your idea was our first thought when adding the Tx meta, in that case the
> >> meta was always set in application
> >>> so we thought that this offload will enable us better function selection, but
> as
> >> you know we removed this capability
> >>> since it is not correct any more.
> >>>
> >>>
> >>>
> >>>> The above also highlights problems of the meta vs mark design. They are
> >> very
> >>>> similar and there is no any good definition of the difference and rules
> >>>> which
> >>>> one should be used/supported in which conditions.
> >>>>
> >>>
> >>> Mark and Meta are exactly the same, the meta is just another value that
> the
> >> application can use.
> >>> This is why both should act the same.
> >>>
> >>> And maybe this is the wining argument, the rte_flow validation approach
> was
> >> used and accepted for the meta.
> >>> So lets try it also with the mark. (please also remember that we didn't have
> >> this mark until now to somehow the
> >>> PMD worked 😊)
> >>>
> >>> Like I said before, I understand your approach, and each one of them has
> its
> >> own advantages and draw backs.
> >>> Lets start using the rte_flow approach and see how it goes, I promise you
> that
> >> if I see that it doesn't scale or cause more
> >>> issues I will be first one to submit changes.
> >>
> >> I tend to say OK, let's try. However, it must be documented
> >> in MARK action that if an application wants to use it, a rule
> >> with the action must be validated before device start.
> >
> > I agree to add this to the rte_flow mark action documation.
> >
> >> PMD may use the attempt as an indication from the application
> >> that it would like to use flow mark even if the validation
> >> fails.
> >
> > No if the PMD uses this validation as hint it should return success and
> > use the correct PMD.
> 
> It would make it too strictly dependent on pattern/actions/state.

I'm not sure I understand your comment.
Why would it make it to strict? I guess that mark action is doesn't care which items are in 
the flow, so just setting eth item sound good enough.
Also I guess the mark doesn't conflict with many actions maybe the meta.
So the application doesn't have a lot of choices.
In any case and this is the most important one. In any case the application must validate that the 
Nic support the mark in any case, according to RTE_FLOW definition. What is the meaning
of saying we support offload (using your segguetion) and then each flow you try to insert
will fail. So in any case the application must validate the flows it is going to use.

> 
> >> Ori, please, suggest formalized pattern and actions
> >> specification to use if application wants to utilize
> >> validation result as a criteria to enable/disable flow
> >> marks usage.
> >
> > I can’t do that, it depends on the application, the most basic is just "pattern
> eth actions mark / queue" .
> > In some cases where you need it for E-Switch if should be something like
> "transfer  items port / eth / actions mark"
> 
> If so, what application author should do if even maintainers cannot
> formalize it. It sounds like the approach does not work.

This should be very easy for the application to know. How can I a mantianer know 
what flows and what is important to the application, I need to make sure that the application
can understand if what it wants is supported.

> 
> >> What should be done if patterns to use and set
> >> of actions together with MARK are not known in advance.
> >
> > I think that the application knows which kind of traffic it expects and which
> actions it needs.
> 
> I'm afraid it is not always true.
> 

Can you please give me such example, how can you develop an application without knowing what it should do?
This means the application is not optimized and impossible to test.

> >> Andrew.
> >
> > Ori
> >

Ori

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/6/19 10:42 AM, Ori Kam wrote:
> 
>> -----Original Message-----
>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>> Sent: Wednesday, November 6, 2019 8:41 AM
>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>> ktraynor@redhat.com
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
>> On 11/5/19 7:37 PM, Ori Kam wrote:
>>>
>>>> -----Original Message-----
>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>>>> Sent: Tuesday, November 5, 2019 1:31 PM
>>>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>>>> <thomas@monjalon.net>
>>>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>>>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>>>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>>>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>>>> ktraynor@redhat.com
>>>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>>>> offload
>>>>
>>>
>>> [Snip]
>>>
>>>>>
>>>>> Yes but like I said in Mellanox PMD for example we supported the mark only on non-transfer flows until this release.
>>>>> so when the user set mark on transfer flow it was invalid. (in transfer flow if we have a miss we send the packet back to the Rx
>>>>> port so the application can understand on which table the miss happened)
>>>>> In this version we added the support for mark also in transfer (E-Switch) flows.
>>>>> So my question before this release what should the PMD report? What should the PMD report after this release?
>>>>>
>>>>> Your idea was our first thought when adding the Tx meta, in that case the meta was always set in application
>>>>> so we thought that this offload will enable us better function selection, but as you know we removed this capability
>>>>> since it is not correct any more.
>>>>>
>>>>>> The above also highlights problems of the meta vs mark design. They are very
>>>>>> similar and there is no any good definition of the difference and rules
>>>>>> which one should be used/supported in which conditions.
>>>>>>
>>>>>
>>>>> Mark and Meta are exactly the same, the meta is just another value that the application can use.
>>>>> This is why both should act the same.
>>>>>
>>>>> And maybe this is the wining argument, the rte_flow validation approach was used and accepted for the meta.
>>>>> So lets try it also with the mark. (please also remember that we didn't have this mark until now to somehow the
>>>>> PMD worked 😊)
>>>>>
>>>>> Like I said before, I understand your approach, and each one of them has its own advantages and draw backs.
>>>>> Lets start using the rte_flow approach and see how it goes, I promise you that if I see that it doesn't scale or cause more
>>>>> issues I will be first one to submit changes.
>>>>
>>>> I tend to say OK, let's try. However, it must be documented
>>>> in MARK action that if an application wants to use it, a rule
>>>> with the action must be validated before device start.
>>>
>>> I agree to add this to the rte_flow mark action documation.
>>>
>>>> PMD may use the attempt as an indication from the application
>>>> that it would like to use flow mark even if the validation
>>>> fails.
>>>
>>> No if the PMD uses this validation as hint it should return success and
>>> use the correct PMD.
>>
>> It would make it too strictly dependent on pattern/actions/state.
> 
> I'm not sure I understand your comment.
> Why would it make it to strict? I guess that mark action is doesn't care which items are in 
> the flow, so just setting eth item sound good enough.

The problem is if single pattern+actions is used to check MARK
support and let PMD know that application is going to use it and
the flow rule is not supported, but another combination is supported,
that's it. The feature is false classified as not supported.

> Also I guess the mark doesn't conflict with many actions maybe the meta.
> So the application doesn't have a lot of choices.
> In any case and this is the most important one. In any case the application must validate that the 
> Nic support the mark in any case, according to RTE_FLOW definition. What is the meaning
> of saying we support offload (using your segguetion) and then each flow you try to insert
> will fail. So in any case the application must validate the flows it is going to use.

Yes, but the decision here is per rule, not global.

>>>> Ori, please, suggest formalized pattern and actions
>>>> specification to use if application wants to utilize
>>>> validation result as a criteria to enable/disable flow
>>>> marks usage.
>>>
>>> I can’t do that, it depends on the application, the most basic is just "pattern eth actions mark / queue" .
>>> In some cases where you need it for E-Switch if should be something like"transfer  items port / eth / actions mark"
>>
>> If so, what application author should do if even maintainers cannot
>> formalize it. It sounds like the approach does not work.
> 
> This should be very easy for the application to know. How can I a mantianer know 
> what flows and what is important to the application, I need to make sure that the application
> can understand if what it wants is supported.

Yes and it is the problem of complex logic to make global
decision at start of day. Flow API allows to avoid global
decisions and verdict is returned for each specific rule,
but here we need global decision which will affect further
logic.

>>>> What should be done if patterns to use and set
>>>> of actions together with MARK are not known in advance.
>>>
>>> I think that the application knows which kind of traffic it expects and which actions it needs.
>>
>> I'm afraid it is not always true.
> 
> Can you please give me such example, how can you develop an application without knowing what it should do?
> This means the application is not optimized and impossible to test.

First of all testpmd. I know that it is a test application etc,
but I think it is still valid example.
Second OVS, if I understand it correctly. As far as I know it
allows to add a rule with various patterns and, in theory, it
could result in the pattern to be used with MARK action.
Basically any application which allows to add custom flow rules.

I think we are running in circles here. I'll try to summarize
the result of the discussion to make it easier for other
developers to follow. Please, correct me if I'm wrong below.

Sorry for a very-very long mail below, but I'd really like
to finalize the discussion.

The problem:
~~~~~~~~~~~~
PMD wants to know before port start if application wants to
to use flow MARK/FLAG in the future. It is required because:

1. HW may be configured in a different way to reserve resources
   for MARK/FLAG delivery

2. Datapath implementation choice may depend on it (e.g. vPMD
   is faster, but does not support MARK)

Discussed solutions:
~~~~~~~~~~~~~~~~~~~~
A. Explicit Rx offload suggested by the patch.

B. Implicit by validation of a flow rule with MARK/FLAG actions used.

C. Use dynamic field/flag (i.e. application registers dynamic field
   and/or flag and PMD uses lookup to solve the problem) plus part
   of (B) to discover if a feature is supported.

All solutions require changes in applications which use these
features. There is a deprecation notice in place which advertises
DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
it with solution (B) or (C). Solution (C) requires changes since
it should be combined with (B) in order to understand if
the feature is supported.

Advantages and drawbacks of solutions:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. The main drawback of (A) is a "duplication" since we already
   have a way to request flow MARK using rte_flow API.
   I don't fully agree that it is a duplication, but I agree
   that it sounds like duplication and complicates a bit flow
   MARK usage by applications. (B) complicates it as well.

2. One more drawback of the solution (A) is the necessity of
   similar solution for META and it eats one more offload bit.
   Yes, that's true and I think it is not a problem.
   It would make it easier for applications to find out if
   either MARK or META is supported.

3. The main advantage of the solution (A) is simplicity.
   It is simple for application to understand if it supported.
   It is simple in PMD to understand that it is required.
   It is simple to disable it - just reconfigure.
   Also it is easier to document it - just mention that
   the offload should be supported and enabled.

4. The main advantage of the solution (B) is no "duplication".
   I agree that it is valid argument. Solving the problem
   without extra entities is always nice, but unfortunately
   it is too complex in this case.

5. The main drawback of the solution (B) is the complexity.
   It is necessary to choose a flow rule which should be used
   as a criteria. It could be hardware dependent.
   Complex logic is require in PMD if it wants to address the
   problem and control MARK delivery based on validated flow
   rules. It adds dependency between start/stop processing and
   flow rules validation code.
   It is pretty complicated to document it.

6. Useless enabling of the offload in the case of solution (A)
   if really used flow rules do not support MARK looks like
   drawback as well, but easily mitigated by a combination
   with solution (B) and only required if the application wants
   to dive in the level of optimization and complexity and
   makes sense if application knows required flow rules in
   advance. So, it is not a problem in this case.

7. Solution (C) has drawbacks of the solution (B) for
   applications to understand if these features are supported,
   but no drawbacks in PMD, since explicit criteria is used to
   enable/disable (dynamic field/flag lookup).

8. Solution (C) is nice since it avoids "duplication".

9. The main drawback of the solution (C) is asymmetry.
   As it was discussed in the case of RX_TIMESTAMP
   (if I remember it correctly):
    - PMD advertises RX_TIMESTAMP offload capability
    - application enables the offload
    - PMD registers dynamic field for timestamp
   Solution (C):
     - PMD advertises nothing
     - application uses solution (B) to understand if
       these features are supported
     - application registers dynamic field/flag
     - PMD does lookup and solve the problem
   The asymmetry could be partially mitigated if RX_TIMESTAMP
   solution is changed to require an application to register
   dynamic fields and PMD to do lookup if the offload is
   enabled. So, the only difference will be in no offload
   in the case of flow MARK/FLAG and usage of complex logic
   to understand if it is supported or no.
   May be it would be really good since it will allow to
   have dynamic fields registered before mempool population.

10. Common drawback of solutions (B) and (C) is no granularity.
    Solution (A) may be per queue while (B) and (C) cannot be
    per queue. Moreover (C) looks global - for all devices.
    It could be really painful.

(C) is nice, but I still vote for simplicity and
granularity of (A).

Andrew.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Tom Barbette]

My two cents as a regular user of the flow and mark feature: (A) seems much better and natural.

In packet processing platform such as FastClick, I don't have a real understanding of the rules of the user, and rules can be added by the user while the application is running via a socket or web interface. So it is much easier to "pass the requirement", asking the user to give e.g. "FLOW_MARK true" to the I/O elements to enable the feature for later use that I easily translate to the offload flag. Moreover, that creates a dependency issue as I first initialize I/O elements so other elements can query some resources such as the MAC address (One could have half-initialization or a real dependency scheme but that is not supported for now). In the dynamic case (B) would require to forge a fake rule in case the feature is later used.

(C) seems even more complex... Though if you want to use a dynamic field for the mark/meta, it seems okay to have a second requirement to add a field for it.

Tom
________________________________________
De : dev <dev-bounces@dpdk.org> de la part de Andrew Rybchenko <arybchenko@solarflare.com>
Envoyé : vendredi 8 novembre 2019 09:35
À : Ori Kam; Thomas Monjalon
Cc : dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com; jerinj@marvell.com; John McNamara; Marko Kovacevic; Adrien Mazarguil; david.marchand@redhat.com; ktraynor@redhat.com; Olivier Matz
Objet : Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

On 11/6/19 10:42 AM, Ori Kam wrote:
>
>> -----Original Message-----
>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>> Sent: Wednesday, November 6, 2019 8:41 AM
>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>> <thomas@monjalon.net>
>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>> ktraynor@redhat.com
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>> offload
>>
>> On 11/5/19 7:37 PM, Ori Kam wrote:
>>>
>>>> -----Original Message-----
>>>> From: Andrew Rybchenko <arybchenko@solarflare.com>
>>>> Sent: Tuesday, November 5, 2019 1:31 PM
>>>> To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
>>>> <thomas@monjalon.net>
>>>> Cc: dev@dpdk.org; pbhagavatula@marvell.com; ferruh.yigit@intel.com;
>>>> jerinj@marvell.com; John McNamara <john.mcnamara@intel.com>; Marko
>>>> Kovacevic <marko.kovacevic@intel.com>; Adrien Mazarguil
>>>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>>>> ktraynor@redhat.com
>>>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
>>>> offload
>>>>
>>>
>>> [Snip]
>>>
>>>>>
>>>>> Yes but like I said in Mellanox PMD for example we supported the mark only on non-transfer flows until this release.
>>>>> so when the user set mark on transfer flow it was invalid. (in transfer flow if we have a miss we send the packet back to the Rx
>>>>> port so the application can understand on which table the miss happened)
>>>>> In this version we added the support for mark also in transfer (E-Switch) flows.
>>>>> So my question before this release what should the PMD report? What should the PMD report after this release?
>>>>>
>>>>> Your idea was our first thought when adding the Tx meta, in that case the meta was always set in application
>>>>> so we thought that this offload will enable us better function selection, but as you know we removed this capability
>>>>> since it is not correct any more.
>>>>>
>>>>>> The above also highlights problems of the meta vs mark design. They are very
>>>>>> similar and there is no any good definition of the difference and rules
>>>>>> which one should be used/supported in which conditions.
>>>>>>
>>>>>
>>>>> Mark and Meta are exactly the same, the meta is just another value that the application can use.
>>>>> This is why both should act the same.
>>>>>
>>>>> And maybe this is the wining argument, the rte_flow validation approach was used and accepted for the meta.
>>>>> So lets try it also with the mark. (please also remember that we didn't have this mark until now to somehow the
>>>>> PMD worked 😊)
>>>>>
>>>>> Like I said before, I understand your approach, and each one of them has its own advantages and draw backs.
>>>>> Lets start using the rte_flow approach and see how it goes, I promise you that if I see that it doesn't scale or cause more
>>>>> issues I will be first one to submit changes.
>>>>
>>>> I tend to say OK, let's try. However, it must be documented
>>>> in MARK action that if an application wants to use it, a rule
>>>> with the action must be validated before device start.
>>>
>>> I agree to add this to the rte_flow mark action documation.
>>>
>>>> PMD may use the attempt as an indication from the application
>>>> that it would like to use flow mark even if the validation
>>>> fails.
>>>
>>> No if the PMD uses this validation as hint it should return success and
>>> use the correct PMD.
>>
>> It would make it too strictly dependent on pattern/actions/state.
>
> I'm not sure I understand your comment.
> Why would it make it to strict? I guess that mark action is doesn't care which items are in
> the flow, so just setting eth item sound good enough.

The problem is if single pattern+actions is used to check MARK
support and let PMD know that application is going to use it and
the flow rule is not supported, but another combination is supported,
that's it. The feature is false classified as not supported.

> Also I guess the mark doesn't conflict with many actions maybe the meta.
> So the application doesn't have a lot of choices.
> In any case and this is the most important one. In any case the application must validate that the
> Nic support the mark in any case, according to RTE_FLOW definition. What is the meaning
> of saying we support offload (using your segguetion) and then each flow you try to insert
> will fail. So in any case the application must validate the flows it is going to use.

Yes, but the decision here is per rule, not global.

>>>> Ori, please, suggest formalized pattern and actions
>>>> specification to use if application wants to utilize
>>>> validation result as a criteria to enable/disable flow
>>>> marks usage.
>>>
>>> I can’t do that, it depends on the application, the most basic is just "pattern eth actions mark / queue" .
>>> In some cases where you need it for E-Switch if should be something like"transfer  items port / eth / actions mark"
>>
>> If so, what application author should do if even maintainers cannot
>> formalize it. It sounds like the approach does not work.
>
> This should be very easy for the application to know. How can I a mantianer know
> what flows and what is important to the application, I need to make sure that the application
> can understand if what it wants is supported.

Yes and it is the problem of complex logic to make global
decision at start of day. Flow API allows to avoid global
decisions and verdict is returned for each specific rule,
but here we need global decision which will affect further
logic.

>>>> What should be done if patterns to use and set
>>>> of actions together with MARK are not known in advance.
>>>
>>> I think that the application knows which kind of traffic it expects and which actions it needs.
>>
>> I'm afraid it is not always true.
>
> Can you please give me such example, how can you develop an application without knowing what it should do?
> This means the application is not optimized and impossible to test.

First of all testpmd. I know that it is a test application etc,
but I think it is still valid example.
Second OVS, if I understand it correctly. As far as I know it
allows to add a rule with various patterns and, in theory, it
could result in the pattern to be used with MARK action.
Basically any application which allows to add custom flow rules.

I think we are running in circles here. I'll try to summarize
the result of the discussion to make it easier for other
developers to follow. Please, correct me if I'm wrong below.

Sorry for a very-very long mail below, but I'd really like
to finalize the discussion.

The problem:
~~~~~~~~~~~~
PMD wants to know before port start if application wants to
to use flow MARK/FLAG in the future. It is required because:

1. HW may be configured in a different way to reserve resources
   for MARK/FLAG delivery

2. Datapath implementation choice may depend on it (e.g. vPMD
   is faster, but does not support MARK)

Discussed solutions:
~~~~~~~~~~~~~~~~~~~~
A. Explicit Rx offload suggested by the patch.

B. Implicit by validation of a flow rule with MARK/FLAG actions used.

C. Use dynamic field/flag (i.e. application registers dynamic field
   and/or flag and PMD uses lookup to solve the problem) plus part
   of (B) to discover if a feature is supported.

All solutions require changes in applications which use these
features. There is a deprecation notice in place which advertises
DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
it with solution (B) or (C). Solution (C) requires changes since
it should be combined with (B) in order to understand if
the feature is supported.

Advantages and drawbacks of solutions:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. The main drawback of (A) is a "duplication" since we already
   have a way to request flow MARK using rte_flow API.
   I don't fully agree that it is a duplication, but I agree
   that it sounds like duplication and complicates a bit flow
   MARK usage by applications. (B) complicates it as well.

2. One more drawback of the solution (A) is the necessity of
   similar solution for META and it eats one more offload bit.
   Yes, that's true and I think it is not a problem.
   It would make it easier for applications to find out if
   either MARK or META is supported.

3. The main advantage of the solution (A) is simplicity.
   It is simple for application to understand if it supported.
   It is simple in PMD to understand that it is required.
   It is simple to disable it - just reconfigure.
   Also it is easier to document it - just mention that
   the offload should be supported and enabled.

4. The main advantage of the solution (B) is no "duplication".
   I agree that it is valid argument. Solving the problem
   without extra entities is always nice, but unfortunately
   it is too complex in this case.

5. The main drawback of the solution (B) is the complexity.
   It is necessary to choose a flow rule which should be used
   as a criteria. It could be hardware dependent.
   Complex logic is require in PMD if it wants to address the
   problem and control MARK delivery based on validated flow
   rules. It adds dependency between start/stop processing and
   flow rules validation code.
   It is pretty complicated to document it.

6. Useless enabling of the offload in the case of solution (A)
   if really used flow rules do not support MARK looks like
   drawback as well, but easily mitigated by a combination
   with solution (B) and only required if the application wants
   to dive in the level of optimization and complexity and
   makes sense if application knows required flow rules in
   advance. So, it is not a problem in this case.

7. Solution (C) has drawbacks of the solution (B) for
   applications to understand if these features are supported,
   but no drawbacks in PMD, since explicit criteria is used to
   enable/disable (dynamic field/flag lookup).

8. Solution (C) is nice since it avoids "duplication".

9. The main drawback of the solution (C) is asymmetry.
   As it was discussed in the case of RX_TIMESTAMP
   (if I remember it correctly):
    - PMD advertises RX_TIMESTAMP offload capability
    - application enables the offload
    - PMD registers dynamic field for timestamp
   Solution (C):
     - PMD advertises nothing
     - application uses solution (B) to understand if
       these features are supported
     - application registers dynamic field/flag
     - PMD does lookup and solve the problem
   The asymmetry could be partially mitigated if RX_TIMESTAMP
   solution is changed to require an application to register
   dynamic fields and PMD to do lookup if the offload is
   enabled. So, the only difference will be in no offload
   in the case of flow MARK/FLAG and usage of complex logic
   to understand if it is supported or no.
   May be it would be really good since it will allow to
   have dynamic fields registered before mempool population.

10. Common drawback of solutions (B) and (C) is no granularity.
    Solution (A) may be per queue while (B) and (C) cannot be
    per queue. Moreover (C) looks global - for all devices.
    It could be really painful.

(C) is nice, but I still vote for simplicity and
granularity of (A).

Andrew.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

08/11/2019 09:35, Andrew Rybchenko:
> The problem:
> ~~~~~~~~~~~~
> PMD wants to know before port start if application wants to
> to use flow MARK/FLAG in the future. It is required because:
> 
> 1. HW may be configured in a different way to reserve resources
>    for MARK/FLAG delivery
> 
> 2. Datapath implementation choice may depend on it (e.g. vPMD
>    is faster, but does not support MARK)

Thank you for the clear problem statement.
I agree with it. This is a real design issue.


> Discussed solutions:
> ~~~~~~~~~~~~~~~~~~~~
> A. Explicit Rx offload suggested by the patch.
> 
> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> 
> C. Use dynamic field/flag (i.e. application registers dynamic field
>    and/or flag and PMD uses lookup to solve the problem) plus part
>    of (B) to discover if a feature is supported.

The dynamic field should be registered via a new API function
named '<feature>_init'.
It means the application must explicit request the feature.
I agree this is the way to go.

> All solutions require changes in applications which use these
> features. There is a deprecation notice in place which advertises
> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
> it with solution (B) or (C). Solution (C) requires changes since
> it should be combined with (B) in order to understand if
> the feature is supported.

I don't understand.
Application request and PMD support are two different things.
PMD support must be via rte_flow validation on a case by case anyway.


> Advantages and drawbacks of solutions:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 1. The main drawback of (A) is a "duplication" since we already
>    have a way to request flow MARK using rte_flow API.
>    I don't fully agree that it is a duplication, but I agree
>    that it sounds like duplication and complicates a bit flow
>    MARK usage by applications. (B) complicates it as well.
> 
> 2. One more drawback of the solution (A) is the necessity of
>    similar solution for META and it eats one more offload bit.
>    Yes, that's true and I think it is not a problem.
>    It would make it easier for applications to find out if
>    either MARK or META is supported.
> 
> 3. The main advantage of the solution (A) is simplicity.
>    It is simple for application to understand if it supported.
>    It is simple in PMD to understand that it is required.
>    It is simple to disable it - just reconfigure.
>    Also it is easier to document it - just mention that
>    the offload should be supported and enabled.
> 
> 4. The main advantage of the solution (B) is no "duplication".
>    I agree that it is valid argument. Solving the problem
>    without extra entities is always nice, but unfortunately
>    it is too complex in this case.
> 
> 5. The main drawback of the solution (B) is the complexity.
>    It is necessary to choose a flow rule which should be used
>    as a criteria. It could be hardware dependent.
>    Complex logic is require in PMD if it wants to address the
>    problem and control MARK delivery based on validated flow
>    rules. It adds dependency between start/stop processing and
>    flow rules validation code.
>    It is pretty complicated to document it.
> 
> 6. Useless enabling of the offload in the case of solution (A)
>    if really used flow rules do not support MARK looks like
>    drawback as well, but easily mitigated by a combination
>    with solution (B) and only required if the application wants
>    to dive in the level of optimization and complexity and
>    makes sense if application knows required flow rules in
>    advance. So, it is not a problem in this case.
> 
> 7. Solution (C) has drawbacks of the solution (B) for
>    applications to understand if these features are supported,
>    but no drawbacks in PMD, since explicit criteria is used to
>    enable/disable (dynamic field/flag lookup).
> 
> 8. Solution (C) is nice since it avoids "duplication".
> 
> 9. The main drawback of the solution (C) is asymmetry.
>    As it was discussed in the case of RX_TIMESTAMP
>    (if I remember it correctly):
>     - PMD advertises RX_TIMESTAMP offload capability
>     - application enables the offload
>     - PMD registers dynamic field for timestamp
>    Solution (C):
>      - PMD advertises nothing
>      - application uses solution (B) to understand if
>        these features are supported
>      - application registers dynamic field/flag
>      - PMD does lookup and solve the problem
>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>    solution is changed to require an application to register
>    dynamic fields and PMD to do lookup if the offload is
>    enabled. So, the only difference will be in no offload
>    in the case of flow MARK/FLAG and usage of complex logic
>    to understand if it is supported or no.
>    May be it would be really good since it will allow to
>    have dynamic fields registered before mempool population.
> 
> 10. Common drawback of solutions (B) and (C) is no granularity.
>     Solution (A) may be per queue while (B) and (C) cannot be
>     per queue. Moreover (C) looks global - for all devices.
>     It could be really painful.
> 
> (C) is nice, but I still vote for simplicity and
> granularity of (A).

I vote for clear separation of application needs and PMD support,
by using the method C (dynamic fields).
I agree timestamp must use the same path.
I agree it's complicate because we don't know in advance whether
a flow rule will be accepted, but that's the reality, config is complex.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> 08/11/2019 09:35, Andrew Rybchenko:
>> The problem:
>> ~~~~~~~~~~~~
>> PMD wants to know before port start if application wants to
>> to use flow MARK/FLAG in the future. It is required because:
>>
>> 1. HW may be configured in a different way to reserve resources
>>    for MARK/FLAG delivery
>>
>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>    is faster, but does not support MARK)
> 
> Thank you for the clear problem statement.
> I agree with it. This is a real design issue.
> 
> 
>> Discussed solutions:
>> ~~~~~~~~~~~~~~~~~~~~

May be it is not 100% clear since below are alternatives.

>> A. Explicit Rx offload suggested by the patch.
>>
>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>
>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>    of (B) to discover if a feature is supported.
> 
> The dynamic field should be registered via a new API function
> named '<feature>_init'.
> It means the application must explicit request the feature.
> I agree this is the way to go.

If I understand your statement correctly, but (C) is not ideal since it
looks global. If registered dynamic field of mbuf and is flag that
the feature should be enabled, it is a flag to all ports/devices.

>> All solutions require changes in applications which use these
>> features. There is a deprecation notice in place which advertises
>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>> it with solution (B) or (C). Solution (C) requires changes since
>> it should be combined with (B) in order to understand if
>> the feature is supported.
> 
> I don't understand.
> Application request and PMD support are two different things.
> PMD support must be via rte_flow validation on a case by case anyway.

I mean that application wants to understand if the feature is
supported. Then, it wants to enable it. In the case of (B),
if I understand the solution correctly, there is no explicit
way to enable, PMD just detects it because of discovery is done
(that's what I mean by "implicit" and it is a drawback from my
point of view, but still could be considered). (C) solves the
problem of (B).

>> Advantages and drawbacks of solutions:
>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> 1. The main drawback of (A) is a "duplication" since we already
>>    have a way to request flow MARK using rte_flow API.
>>    I don't fully agree that it is a duplication, but I agree
>>    that it sounds like duplication and complicates a bit flow
>>    MARK usage by applications. (B) complicates it as well.
>>
>> 2. One more drawback of the solution (A) is the necessity of
>>    similar solution for META and it eats one more offload bit.
>>    Yes, that's true and I think it is not a problem.
>>    It would make it easier for applications to find out if
>>    either MARK or META is supported.
>>
>> 3. The main advantage of the solution (A) is simplicity.
>>    It is simple for application to understand if it supported.
>>    It is simple in PMD to understand that it is required.
>>    It is simple to disable it - just reconfigure.
>>    Also it is easier to document it - just mention that
>>    the offload should be supported and enabled.
>>
>> 4. The main advantage of the solution (B) is no "duplication".
>>    I agree that it is valid argument. Solving the problem
>>    without extra entities is always nice, but unfortunately
>>    it is too complex in this case.
>>
>> 5. The main drawback of the solution (B) is the complexity.
>>    It is necessary to choose a flow rule which should be used
>>    as a criteria. It could be hardware dependent.
>>    Complex logic is require in PMD if it wants to address the
>>    problem and control MARK delivery based on validated flow
>>    rules. It adds dependency between start/stop processing and
>>    flow rules validation code.
>>    It is pretty complicated to document it.
>>
>> 6. Useless enabling of the offload in the case of solution (A)
>>    if really used flow rules do not support MARK looks like
>>    drawback as well, but easily mitigated by a combination
>>    with solution (B) and only required if the application wants
>>    to dive in the level of optimization and complexity and
>>    makes sense if application knows required flow rules in
>>    advance. So, it is not a problem in this case.
>>
>> 7. Solution (C) has drawbacks of the solution (B) for
>>    applications to understand if these features are supported,
>>    but no drawbacks in PMD, since explicit criteria is used to
>>    enable/disable (dynamic field/flag lookup).
>>
>> 8. Solution (C) is nice since it avoids "duplication".
>>
>> 9. The main drawback of the solution (C) is asymmetry.
>>    As it was discussed in the case of RX_TIMESTAMP
>>    (if I remember it correctly):
>>     - PMD advertises RX_TIMESTAMP offload capability
>>     - application enables the offload
>>     - PMD registers dynamic field for timestamp
>>    Solution (C):
>>      - PMD advertises nothing
>>      - application uses solution (B) to understand if
>>        these features are supported
>>      - application registers dynamic field/flag
>>      - PMD does lookup and solve the problem
>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>    solution is changed to require an application to register
>>    dynamic fields and PMD to do lookup if the offload is
>>    enabled. So, the only difference will be in no offload
>>    in the case of flow MARK/FLAG and usage of complex logic
>>    to understand if it is supported or no.
>>    May be it would be really good since it will allow to
>>    have dynamic fields registered before mempool population.
>>
>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>     Solution (A) may be per queue while (B) and (C) cannot be
>>     per queue. Moreover (C) looks global - for all devices.
>>     It could be really painful.
>>
>> (C) is nice, but I still vote for simplicity and
>> granularity of (A).
> 
> I vote for clear separation of application needs and PMD support,
> by using the method C (dynamic fields).
> I agree timestamp must use the same path.
> I agree it's complicate because we don't know in advance whether
> a flow rule will be accepted, but that's the reality, config is complex.

Do you think that global nature of the (C) is acceptable?

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

08/11/2019 11:42, Andrew Rybchenko:
> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> > 08/11/2019 09:35, Andrew Rybchenko:
> >> The problem:
> >> ~~~~~~~~~~~~
> >> PMD wants to know before port start if application wants to
> >> to use flow MARK/FLAG in the future. It is required because:
> >>
> >> 1. HW may be configured in a different way to reserve resources
> >>    for MARK/FLAG delivery
> >>
> >> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>    is faster, but does not support MARK)
> > 
> > Thank you for the clear problem statement.
> > I agree with it. This is a real design issue.
> > 
> > 
> >> Discussed solutions:
> >> ~~~~~~~~~~~~~~~~~~~~
> 
> May be it is not 100% clear since below are alternatives.
> 
> >> A. Explicit Rx offload suggested by the patch.
> >>
> >> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> >>
> >> C. Use dynamic field/flag (i.e. application registers dynamic field
> >>    and/or flag and PMD uses lookup to solve the problem) plus part
> >>    of (B) to discover if a feature is supported.
> > 
> > The dynamic field should be registered via a new API function
> > named '<feature>_init'.
> > It means the application must explicit request the feature.
> > I agree this is the way to go.
> 
> If I understand your statement correctly, but (C) is not ideal since it
> looks global. If registered dynamic field of mbuf and is flag that
> the feature should be enabled, it is a flag to all ports/devices.
> 
> >> All solutions require changes in applications which use these
> >> features. There is a deprecation notice in place which advertises
> >> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
> >> it with solution (B) or (C). Solution (C) requires changes since
> >> it should be combined with (B) in order to understand if
> >> the feature is supported.
> > 
> > I don't understand.
> > Application request and PMD support are two different things.
> > PMD support must be via rte_flow validation on a case by case anyway.
> 
> I mean that application wants to understand if the feature is
> supported. Then, it wants to enable it. In the case of (B),
> if I understand the solution correctly, there is no explicit
> way to enable, PMD just detects it because of discovery is done
> (that's what I mean by "implicit" and it is a drawback from my
> point of view, but still could be considered). (C) solves the
> problem of (B).
> 
> >> Advantages and drawbacks of solutions:
> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >> 1. The main drawback of (A) is a "duplication" since we already
> >>    have a way to request flow MARK using rte_flow API.
> >>    I don't fully agree that it is a duplication, but I agree
> >>    that it sounds like duplication and complicates a bit flow
> >>    MARK usage by applications. (B) complicates it as well.
> >>
> >> 2. One more drawback of the solution (A) is the necessity of
> >>    similar solution for META and it eats one more offload bit.
> >>    Yes, that's true and I think it is not a problem.
> >>    It would make it easier for applications to find out if
> >>    either MARK or META is supported.
> >>
> >> 3. The main advantage of the solution (A) is simplicity.
> >>    It is simple for application to understand if it supported.
> >>    It is simple in PMD to understand that it is required.
> >>    It is simple to disable it - just reconfigure.
> >>    Also it is easier to document it - just mention that
> >>    the offload should be supported and enabled.
> >>
> >> 4. The main advantage of the solution (B) is no "duplication".
> >>    I agree that it is valid argument. Solving the problem
> >>    without extra entities is always nice, but unfortunately
> >>    it is too complex in this case.
> >>
> >> 5. The main drawback of the solution (B) is the complexity.
> >>    It is necessary to choose a flow rule which should be used
> >>    as a criteria. It could be hardware dependent.
> >>    Complex logic is require in PMD if it wants to address the
> >>    problem and control MARK delivery based on validated flow
> >>    rules. It adds dependency between start/stop processing and
> >>    flow rules validation code.
> >>    It is pretty complicated to document it.
> >>
> >> 6. Useless enabling of the offload in the case of solution (A)
> >>    if really used flow rules do not support MARK looks like
> >>    drawback as well, but easily mitigated by a combination
> >>    with solution (B) and only required if the application wants
> >>    to dive in the level of optimization and complexity and
> >>    makes sense if application knows required flow rules in
> >>    advance. So, it is not a problem in this case.
> >>
> >> 7. Solution (C) has drawbacks of the solution (B) for
> >>    applications to understand if these features are supported,
> >>    but no drawbacks in PMD, since explicit criteria is used to
> >>    enable/disable (dynamic field/flag lookup).
> >>
> >> 8. Solution (C) is nice since it avoids "duplication".
> >>
> >> 9. The main drawback of the solution (C) is asymmetry.
> >>    As it was discussed in the case of RX_TIMESTAMP
> >>    (if I remember it correctly):
> >>     - PMD advertises RX_TIMESTAMP offload capability
> >>     - application enables the offload
> >>     - PMD registers dynamic field for timestamp
> >>    Solution (C):
> >>      - PMD advertises nothing
> >>      - application uses solution (B) to understand if
> >>        these features are supported
> >>      - application registers dynamic field/flag
> >>      - PMD does lookup and solve the problem
> >>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> >>    solution is changed to require an application to register
> >>    dynamic fields and PMD to do lookup if the offload is
> >>    enabled. So, the only difference will be in no offload
> >>    in the case of flow MARK/FLAG and usage of complex logic
> >>    to understand if it is supported or no.
> >>    May be it would be really good since it will allow to
> >>    have dynamic fields registered before mempool population.
> >>
> >> 10. Common drawback of solutions (B) and (C) is no granularity.
> >>     Solution (A) may be per queue while (B) and (C) cannot be
> >>     per queue. Moreover (C) looks global - for all devices.
> >>     It could be really painful.
> >>
> >> (C) is nice, but I still vote for simplicity and
> >> granularity of (A).
> > 
> > I vote for clear separation of application needs and PMD support,
> > by using the method C (dynamic fields).
> > I agree timestamp must use the same path.
> > I agree it's complicate because we don't know in advance whether
> > a flow rule will be accepted, but that's the reality, config is complex.
> 
> Do you think that global nature of the (C) is acceptable?

That's a good question.
Maybe the feature request should be per port.
In this case, we are back to solution A with a flag per port?

Note that A and C will not guarantee that the offload will be possible.
We need B (flow rule validation) anyway.

It seems A, B, C are not alternatives but all required
as pieces of a puzzle...

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Zhang, Qi Z]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> Sent: Friday, November 8, 2019 7:04 PM
> To: Andrew Rybchenko <arybchenko@solarflare.com>
> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> 08/11/2019 11:42, Andrew Rybchenko:
> > On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> > > 08/11/2019 09:35, Andrew Rybchenko:
> > >> The problem:
> > >> ~~~~~~~~~~~~
> > >> PMD wants to know before port start if application wants to to use
> > >> flow MARK/FLAG in the future. It is required because:
> > >>
> > >> 1. HW may be configured in a different way to reserve resources
> > >>    for MARK/FLAG delivery
> > >>
> > >> 2. Datapath implementation choice may depend on it (e.g. vPMD
> > >>    is faster, but does not support MARK)
> > >
> > > Thank you for the clear problem statement.
> > > I agree with it. This is a real design issue.
> > >
> > >
> > >> Discussed solutions:
> > >> ~~~~~~~~~~~~~~~~~~~~
> >
> > May be it is not 100% clear since below are alternatives.
> >
> > >> A. Explicit Rx offload suggested by the patch.
> > >>
> > >> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> > >>
> > >> C. Use dynamic field/flag (i.e. application registers dynamic field
> > >>    and/or flag and PMD uses lookup to solve the problem) plus part
> > >>    of (B) to discover if a feature is supported.
> > >
> > > The dynamic field should be registered via a new API function named
> > > '<feature>_init'.
> > > It means the application must explicit request the feature.
> > > I agree this is the way to go.
> >
> > If I understand your statement correctly, but (C) is not ideal since
> > it looks global. If registered dynamic field of mbuf and is flag that
> > the feature should be enabled, it is a flag to all ports/devices.
> >
> > >> All solutions require changes in applications which use these
> > >> features. There is a deprecation notice in place which advertises
> > >> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to
> > >> substitute it with solution (B) or (C). Solution (C) requires
> > >> changes since it should be combined with (B) in order to understand
> > >> if the feature is supported.
> > >
> > > I don't understand.
> > > Application request and PMD support are two different things.
> > > PMD support must be via rte_flow validation on a case by case anyway.
> >
> > I mean that application wants to understand if the feature is
> > supported. Then, it wants to enable it. In the case of (B), if I
> > understand the solution correctly, there is no explicit way to enable,
> > PMD just detects it because of discovery is done (that's what I mean
> > by "implicit" and it is a drawback from my point of view, but still
> > could be considered). (C) solves the problem of (B).
> >
> > >> Advantages and drawbacks of solutions:
> > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >> 1. The main drawback of (A) is a "duplication" since we already
> > >>    have a way to request flow MARK using rte_flow API.
> > >>    I don't fully agree that it is a duplication, but I agree
> > >>    that it sounds like duplication and complicates a bit flow
> > >>    MARK usage by applications. (B) complicates it as well.
> > >>
> > >> 2. One more drawback of the solution (A) is the necessity of
> > >>    similar solution for META and it eats one more offload bit.
> > >>    Yes, that's true and I think it is not a problem.
> > >>    It would make it easier for applications to find out if
> > >>    either MARK or META is supported.
> > >>
> > >> 3. The main advantage of the solution (A) is simplicity.
> > >>    It is simple for application to understand if it supported.
> > >>    It is simple in PMD to understand that it is required.
> > >>    It is simple to disable it - just reconfigure.
> > >>    Also it is easier to document it - just mention that
> > >>    the offload should be supported and enabled.
> > >>
> > >> 4. The main advantage of the solution (B) is no "duplication".
> > >>    I agree that it is valid argument. Solving the problem
> > >>    without extra entities is always nice, but unfortunately
> > >>    it is too complex in this case.
> > >>
> > >> 5. The main drawback of the solution (B) is the complexity.
> > >>    It is necessary to choose a flow rule which should be used
> > >>    as a criteria. It could be hardware dependent.
> > >>    Complex logic is require in PMD if it wants to address the
> > >>    problem and control MARK delivery based on validated flow
> > >>    rules. It adds dependency between start/stop processing and
> > >>    flow rules validation code.
> > >>    It is pretty complicated to document it.
> > >>
> > >> 6. Useless enabling of the offload in the case of solution (A)
> > >>    if really used flow rules do not support MARK looks like
> > >>    drawback as well, but easily mitigated by a combination
> > >>    with solution (B) and only required if the application wants
> > >>    to dive in the level of optimization and complexity and
> > >>    makes sense if application knows required flow rules in
> > >>    advance. So, it is not a problem in this case.
> > >>
> > >> 7. Solution (C) has drawbacks of the solution (B) for
> > >>    applications to understand if these features are supported,
> > >>    but no drawbacks in PMD, since explicit criteria is used to
> > >>    enable/disable (dynamic field/flag lookup).
> > >>
> > >> 8. Solution (C) is nice since it avoids "duplication".
> > >>
> > >> 9. The main drawback of the solution (C) is asymmetry.
> > >>    As it was discussed in the case of RX_TIMESTAMP
> > >>    (if I remember it correctly):
> > >>     - PMD advertises RX_TIMESTAMP offload capability
> > >>     - application enables the offload
> > >>     - PMD registers dynamic field for timestamp
> > >>    Solution (C):
> > >>      - PMD advertises nothing
> > >>      - application uses solution (B) to understand if
> > >>        these features are supported
> > >>      - application registers dynamic field/flag
> > >>      - PMD does lookup and solve the problem
> > >>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> > >>    solution is changed to require an application to register
> > >>    dynamic fields and PMD to do lookup if the offload is
> > >>    enabled. So, the only difference will be in no offload
> > >>    in the case of flow MARK/FLAG and usage of complex logic
> > >>    to understand if it is supported or no.
> > >>    May be it would be really good since it will allow to
> > >>    have dynamic fields registered before mempool population.
> > >>
> > >> 10. Common drawback of solutions (B) and (C) is no granularity.
> > >>     Solution (A) may be per queue while (B) and (C) cannot be
> > >>     per queue. Moreover (C) looks global - for all devices.
> > >>     It could be really painful.
> > >>
> > >> (C) is nice, but I still vote for simplicity and granularity of
> > >> (A).
> > >
> > > I vote for clear separation of application needs and PMD support, by
> > > using the method C (dynamic fields).
> > > I agree timestamp must use the same path.
> > > I agree it's complicate because we don't know in advance whether a
> > > flow rule will be accepted, but that's the reality, config is complex.
> >
> > Do you think that global nature of the (C) is acceptable?
> 
> That's a good question.
> Maybe the feature request should be per port.
> In this case, we are back to solution A with a flag per port?
> 
> Note that A and C will not guarantee that the offload will be possible.
> We need B (flow rule validation) anyway.

I may not understand how solution B can works well for all the cases.

A rte_flow rule can be issued after dev_start, which means the rx_burst function is already selected at that time, 
so does that mean the driver need to switch from a non- mark offload aware path to a mark offload aware path without stop device? or it has to reject the flow? 
The question is if we have 2 data path, one support some offload , one not but more fast, which one should be selected during dev_start? Isn't Offload widely used to solve this problem?

I think the option A solve all the problems, option C might also works, but A is looks much straightforward for me.

Regards
Qi


> 
> It seems A, B, C are not alternatives but all required as pieces of a puzzle...
> 

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/8/19 2:03 PM, Thomas Monjalon wrote:
> 08/11/2019 11:42, Andrew Rybchenko:
>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>> The problem:
>>>> ~~~~~~~~~~~~
>>>> PMD wants to know before port start if application wants to
>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>
>>>> 1. HW may be configured in a different way to reserve resources
>>>>    for MARK/FLAG delivery
>>>>
>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>    is faster, but does not support MARK)
>>>
>>> Thank you for the clear problem statement.
>>> I agree with it. This is a real design issue.
>>>
>>>
>>>> Discussed solutions:
>>>> ~~~~~~~~~~~~~~~~~~~~
>>
>> May be it is not 100% clear since below are alternatives.
>>
>>>> A. Explicit Rx offload suggested by the patch.
>>>>
>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>
>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>>>    of (B) to discover if a feature is supported.
>>>
>>> The dynamic field should be registered via a new API function
>>> named '<feature>_init'.
>>> It means the application must explicit request the feature.
>>> I agree this is the way to go.
>>
>> If I understand your statement correctly, but (C) is not ideal since it
>> looks global. If registered dynamic field of mbuf and is flag that
>> the feature should be enabled, it is a flag to all ports/devices.
>>
>>>> All solutions require changes in applications which use these
>>>> features. There is a deprecation notice in place which advertises
>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>> it should be combined with (B) in order to understand if
>>>> the feature is supported.
>>>
>>> I don't understand.
>>> Application request and PMD support are two different things.
>>> PMD support must be via rte_flow validation on a case by case anyway.
>>
>> I mean that application wants to understand if the feature is
>> supported. Then, it wants to enable it. In the case of (B),
>> if I understand the solution correctly, there is no explicit
>> way to enable, PMD just detects it because of discovery is done
>> (that's what I mean by "implicit" and it is a drawback from my
>> point of view, but still could be considered). (C) solves the
>> problem of (B).
>>
>>>> Advantages and drawbacks of solutions:
>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>    have a way to request flow MARK using rte_flow API.
>>>>    I don't fully agree that it is a duplication, but I agree
>>>>    that it sounds like duplication and complicates a bit flow
>>>>    MARK usage by applications. (B) complicates it as well.
>>>>
>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>    similar solution for META and it eats one more offload bit.
>>>>    Yes, that's true and I think it is not a problem.
>>>>    It would make it easier for applications to find out if
>>>>    either MARK or META is supported.
>>>>
>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>    It is simple for application to understand if it supported.
>>>>    It is simple in PMD to understand that it is required.
>>>>    It is simple to disable it - just reconfigure.
>>>>    Also it is easier to document it - just mention that
>>>>    the offload should be supported and enabled.
>>>>
>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>    I agree that it is valid argument. Solving the problem
>>>>    without extra entities is always nice, but unfortunately
>>>>    it is too complex in this case.
>>>>
>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>    It is necessary to choose a flow rule which should be used
>>>>    as a criteria. It could be hardware dependent.
>>>>    Complex logic is require in PMD if it wants to address the
>>>>    problem and control MARK delivery based on validated flow
>>>>    rules. It adds dependency between start/stop processing and
>>>>    flow rules validation code.
>>>>    It is pretty complicated to document it.
>>>>
>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>    if really used flow rules do not support MARK looks like
>>>>    drawback as well, but easily mitigated by a combination
>>>>    with solution (B) and only required if the application wants
>>>>    to dive in the level of optimization and complexity and
>>>>    makes sense if application knows required flow rules in
>>>>    advance. So, it is not a problem in this case.
>>>>
>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>    applications to understand if these features are supported,
>>>>    but no drawbacks in PMD, since explicit criteria is used to
>>>>    enable/disable (dynamic field/flag lookup).
>>>>
>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>
>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>    As it was discussed in the case of RX_TIMESTAMP
>>>>    (if I remember it correctly):
>>>>     - PMD advertises RX_TIMESTAMP offload capability
>>>>     - application enables the offload
>>>>     - PMD registers dynamic field for timestamp
>>>>    Solution (C):
>>>>      - PMD advertises nothing
>>>>      - application uses solution (B) to understand if
>>>>        these features are supported
>>>>      - application registers dynamic field/flag
>>>>      - PMD does lookup and solve the problem
>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>    solution is changed to require an application to register
>>>>    dynamic fields and PMD to do lookup if the offload is
>>>>    enabled. So, the only difference will be in no offload
>>>>    in the case of flow MARK/FLAG and usage of complex logic
>>>>    to understand if it is supported or no.
>>>>    May be it would be really good since it will allow to
>>>>    have dynamic fields registered before mempool population.
>>>>
>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>     Solution (A) may be per queue while (B) and (C) cannot be
>>>>     per queue. Moreover (C) looks global - for all devices.
>>>>     It could be really painful.
>>>>
>>>> (C) is nice, but I still vote for simplicity and
>>>> granularity of (A).
>>>
>>> I vote for clear separation of application needs and PMD support,
>>> by using the method C (dynamic fields).
>>> I agree timestamp must use the same path.
>>> I agree it's complicate because we don't know in advance whether
>>> a flow rule will be accepted, but that's the reality, config is complex.
>>
>> Do you think that global nature of the (C) is acceptable?
> 
> That's a good question.
> Maybe the feature request should be per port.
> In this case, we are back to solution A with a flag per port?

Offloads are natively per-queue as well, so (A) keeps the choice
between per-port vs per-queue to PMDs as usual.

> Note that A and C will not guarantee that the offload will be possible.

Yes, definitely.

> We need B (flow rule validation) anyway.

Strictly speaking (B) (checking flow rules before device
startup) is required if an application can predict flow
rules and wants to ensure that MARK offload will be usable.
Otherwise, it may be skipped.

> It seems A, B, C are not alternatives but all required
> as pieces of a puzzle...

Unfortunately true in the most complex case.
Right now it will be A with B if required as explained above.
C will come a bit later when the field migrates to dynamic.

May be it is even better if application registers dynamic
fields before an attempt to enable offload to be sure that
it will not fail because of impossibility to register
dynamic field (lack of space). I'm not sure, but it is not
not that important.

If we finally go way A, should we add offloads for META back?
I guess separate Rx and Tx are required.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

Hi Andrew,

Thanks very much for the summery, I think this makes the discussion much better.
(we even got some new folks 😊)

I vote for B and C,

Please read my comments below.

Ori

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Zhang, Qi Z
> Sent: Friday, November 8, 2019 1:41 PM
> To: Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> <arybchenko@solarflare.com>
> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
> an offload
> 
> 
> 
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> > Sent: Friday, November 8, 2019 7:04 PM
> > To: Andrew Rybchenko <arybchenko@solarflare.com>
> > Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> > pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> > jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> > Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> > <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> > ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> > Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
> an
> > offload
> >
> > 08/11/2019 11:42, Andrew Rybchenko:
> > > On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> > > > 08/11/2019 09:35, Andrew Rybchenko:
> > > >> The problem:
> > > >> ~~~~~~~~~~~~
> > > >> PMD wants to know before port start if application wants to to use
> > > >> flow MARK/FLAG in the future. It is required because:
> > > >>
> > > >> 1. HW may be configured in a different way to reserve resources
> > > >>    for MARK/FLAG delivery
> > > >>
> > > >> 2. Datapath implementation choice may depend on it (e.g. vPMD
> > > >>    is faster, but does not support MARK)
> > > >
> > > > Thank you for the clear problem statement.
> > > > I agree with it. This is a real design issue.
> > > >
> > > >
> > > >> Discussed solutions:
> > > >> ~~~~~~~~~~~~~~~~~~~~
> > >
> > > May be it is not 100% clear since below are alternatives.
> > >
> > > >> A. Explicit Rx offload suggested by the patch.
> > > >>
> > > >> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> > > >>

I don’t agree that this method is implicit it is quit explicit, do we support it in PMD 
with the use case or not. Sound very explicit. You can argue that the PMD uses 
validation flow to understand that it need to support MARK, but from every perspective
it is more correct . what does it mean if the application enabled mark and never download a flow?

> > > >> C. Use dynamic field/flag (i.e. application registers dynamic field
> > > >>    and/or flag and PMD uses lookup to solve the problem) plus part
> > > >>    of (B) to discover if a feature is supported.
> > > >
> > > > The dynamic field should be registered via a new API function named
> > > > '<feature>_init'.
> > > > It means the application must explicit request the feature.
> > > > I agree this is the way to go.
> > >
> > > If I understand your statement correctly, but (C) is not ideal since
> > > it looks global. If registered dynamic field of mbuf and is flag that
> > > the feature should be enabled, it is a flag to all ports/devices.
> > >
> > > >> All solutions require changes in applications which use these
> > > >> features. There is a deprecation notice in place which advertises
> > > >> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to
> > > >> substitute it with solution (B) or (C). Solution (C) requires
> > > >> changes since it should be combined with (B) in order to understand
> > > >> if the feature is supported.
> > > >
> > > > I don't understand.
> > > > Application request and PMD support are two different things.
> > > > PMD support must be via rte_flow validation on a case by case anyway.
> > >
> > > I mean that application wants to understand if the feature is
> > > supported. Then, it wants to enable it. In the case of (B), if I
> > > understand the solution correctly, there is no explicit way to enable,
> > > PMD just detects it because of discovery is done (that's what I mean
> > > by "implicit" and it is a drawback from my point of view, but still
> > > could be considered). (C) solves the problem of (B).
> > >
> > > >> Advantages and drawbacks of solutions:
> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >> 1. The main drawback of (A) is a "duplication" since we already
> > > >>    have a way to request flow MARK using rte_flow API.
> > > >>    I don't fully agree that it is a duplication, but I agree
> > > >>    that it sounds like duplication and complicates a bit flow
> > > >>    MARK usage by applications. (B) complicates it as well.
> > > >>
> > > >> 2. One more drawback of the solution (A) is the necessity of
> > > >>    similar solution for META and it eats one more offload bit.
> > > >>    Yes, that's true and I think it is not a problem.
> > > >>    It would make it easier for applications to find out if
> > > >>    either MARK or META is supported.
> > > >>
> > > >> 3. The main advantage of the solution (A) is simplicity.
> > > >>    It is simple for application to understand if it supported.
> > > >>    It is simple in PMD to understand that it is required.
> > > >>    It is simple to disable it - just reconfigure.
> > > >>    Also it is easier to document it - just mention that
> > > >>    the offload should be supported and enabled.
> > > >>

This is not true, for example the PMD support Mark only in Nic mode
While the application needs E-Switch then the PMD reports incorrectly.
Or for example if the mark is supported only as only action, and the application needs
decap and mark. this means that the user will never be able to use the mark
so the PMD uses slower Rx function without the mark ever being used.


> > > >> 4. The main advantage of the solution (B) is no "duplication".
> > > >>    I agree that it is valid argument. Solving the problem
> > > >>    without extra entities is always nice, but unfortunately
> > > >>    it is too complex in this case.
> > > >>

You forgot a one more very important reason, the fact that the mark 
may be supported only on some flows. So if the user needs mark with decap
and this combination is not supported the user will never we able to use mark
while using mark Rx function. So you get slow data path with no ability to use it.

> > > >> 5. The main drawback of the solution (B) is the complexity.
> > > >>    It is necessary to choose a flow rule which should be used
> > > >>    as a criteria. It could be hardware dependent.
> > > >>    Complex logic is require in PMD if it wants to address the
> > > >>    problem and control MARK delivery based on validated flow
> > > >>    rules. It adds dependency between start/stop processing and
> > > >>    flow rules validation code.
> > > >>    It is pretty complicated to document it.
> > > >>
> > > >> 6. Useless enabling of the offload in the case of solution (A)
> > > >>    if really used flow rules do not support MARK looks like
> > > >>    drawback as well, but easily mitigated by a combination
> > > >>    with solution (B) and only required if the application wants
> > > >>    to dive in the level of optimization and complexity and
> > > >>    makes sense if application knows required flow rules in
> > > >>    advance. So, it is not a problem in this case.
> > > >>
> > > >> 7. Solution (C) has drawbacks of the solution (B) for
> > > >>    applications to understand if these features are supported,
> > > >>    but no drawbacks in PMD, since explicit criteria is used to
> > > >>    enable/disable (dynamic field/flag lookup).
> > > >>
> > > >> 8. Solution (C) is nice since it avoids "duplication".
> > > >>
> > > >> 9. The main drawback of the solution (C) is asymmetry.
> > > >>    As it was discussed in the case of RX_TIMESTAMP
> > > >>    (if I remember it correctly):
> > > >>     - PMD advertises RX_TIMESTAMP offload capability
> > > >>     - application enables the offload
> > > >>     - PMD registers dynamic field for timestamp
> > > >>    Solution (C):
> > > >>      - PMD advertises nothing
> > > >>      - application uses solution (B) to understand if
> > > >>        these features are supported
> > > >>      - application registers dynamic field/flag
> > > >>      - PMD does lookup and solve the problem
> > > >>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> > > >>    solution is changed to require an application to register
> > > >>    dynamic fields and PMD to do lookup if the offload is
> > > >>    enabled. So, the only difference will be in no offload
> > > >>    in the case of flow MARK/FLAG and usage of complex logic
> > > >>    to understand if it is supported or no.
> > > >>    May be it would be really good since it will allow to
> > > >>    have dynamic fields registered before mempool population.
> > > >>
> > > >> 10. Common drawback of solutions (B) and (C) is no granularity.
> > > >>     Solution (A) may be per queue while (B) and (C) cannot be
> > > >>     per queue. Moreover (C) looks global - for all devices.
> > > >>     It could be really painful.
> > > >>
> > > >> (C) is nice, but I still vote for simplicity and granularity of
> > > >> (A).
> > > >
> > > > I vote for clear separation of application needs and PMD support, by
> > > > using the method C (dynamic fields).
> > > > I agree timestamp must use the same path.
> > > > I agree it's complicate because we don't know in advance whether a
> > > > flow rule will be accepted, but that's the reality, config is complex.
> > >
> > > Do you think that global nature of the (C) is acceptable?
> >
> > That's a good question.
> > Maybe the feature request should be per port.
> > In this case, we are back to solution A with a flag per port?
> >
> > Note that A and C will not guarantee that the offload will be possible.
> > We need B (flow rule validation) anyway.
> 
> I may not understand how solution B can works well for all the cases.
> 

That is the point of solution B it always work, it is a bit harder to use maybe,
but you can check and decide everything.

> A rte_flow rule can be issued after dev_start, which means the rx_burst
> function is already selected at that time,
> so does that mean the driver need to switch from a non- mark offload aware
> path to a mark offload aware path without stop device? or it has to reject the
> flow?
> The question is if we have 2 data path, one support some offload , one not
> but more fast, which one should be selected during dev_start? Isn't Offload
> widely used to solve this problem?
> 
> I think the option A solve all the problems, option C might also works, but A is
> looks much straightforward for me.
> 

Solution A my result in selecting incorrect datapath  please see my answer above.

As you can guess I'm in favor of B and C (C need B)

> Regards
> Qi
> 
> 
> >
> > It seems A, B, C are not alternatives but all required as pieces of a puzzle...
> >

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/8/19 3:12 PM, Ori Kam wrote:
> 
> Hi Andrew,
> 
> Thanks very much for the summery, I think this makes the discussion much better.
> (we even got some new folks 😊)
> 
> I vote for B and C,
> 
> Please read my comments below.
> 
> Ori
> 
>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Zhang, Qi Z
>> Sent: Friday, November 8, 2019 1:41 PM
>> To: Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
>> <arybchenko@solarflare.com>
>> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
>> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
>> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
>> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
>> an offload
>>
>>
>>
>>> -----Original Message-----
>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
>>> Sent: Friday, November 8, 2019 7:04 PM
>>> To: Andrew Rybchenko <arybchenko@solarflare.com>
>>> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
>>> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
>>> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
>>> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
>>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
>>> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
>>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
>> an
>>> offload
>>>
>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>> The problem:
>>>>>> ~~~~~~~~~~~~
>>>>>> PMD wants to know before port start if application wants to to use
>>>>>> flow MARK/FLAG in the future. It is required because:
>>>>>>
>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>    for MARK/FLAG delivery
>>>>>>
>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>    is faster, but does not support MARK)
>>>>>
>>>>> Thank you for the clear problem statement.
>>>>> I agree with it. This is a real design issue.
>>>>>
>>>>>
>>>>>> Discussed solutions:
>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>>
>>>> May be it is not 100% clear since below are alternatives.
>>>>
>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>
>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>
> 
> I don’t agree that this method is implicit it is quit explicit, do we support it in PMD 
> with the use case or not. Sound very explicit. You can argue that the PMD uses 
> validation flow to understand that it need to support MARK, but from every perspective
> it is more correct . what does it mean if the application enabled mark and never download a flow?

It is implicit from PMD point of view, since there is no command
to enable it. The decision to enable is the result of seen
attempt to validate mark. If few rules are validated, but an
application decides to have it disabled finally, how to achieve it?
Last validation attempt wins?
Other points are covered by (6) below.

>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>    of (B) to discover if a feature is supported.
>>>>>
>>>>> The dynamic field should be registered via a new API function named
>>>>> '<feature>_init'.
>>>>> It means the application must explicit request the feature.
>>>>> I agree this is the way to go.
>>>>
>>>> If I understand your statement correctly, but (C) is not ideal since
>>>> it looks global. If registered dynamic field of mbuf and is flag that
>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>
>>>>>> All solutions require changes in applications which use these
>>>>>> features. There is a deprecation notice in place which advertises
>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to
>>>>>> substitute it with solution (B) or (C). Solution (C) requires
>>>>>> changes since it should be combined with (B) in order to understand
>>>>>> if the feature is supported.
>>>>>
>>>>> I don't understand.
>>>>> Application request and PMD support are two different things.
>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>>
>>>> I mean that application wants to understand if the feature is
>>>> supported. Then, it wants to enable it. In the case of (B), if I
>>>> understand the solution correctly, there is no explicit way to enable,
>>>> PMD just detects it because of discovery is done (that's what I mean
>>>> by "implicit" and it is a drawback from my point of view, but still
>>>> could be considered). (C) solves the problem of (B).
>>>>
>>>>>> Advantages and drawbacks of solutions:
>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>    have a way to request flow MARK using rte_flow API.
>>>>>>    I don't fully agree that it is a duplication, but I agree
>>>>>>    that it sounds like duplication and complicates a bit flow
>>>>>>    MARK usage by applications. (B) complicates it as well.
>>>>>>
>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>    similar solution for META and it eats one more offload bit.
>>>>>>    Yes, that's true and I think it is not a problem.
>>>>>>    It would make it easier for applications to find out if
>>>>>>    either MARK or META is supported.
>>>>>>
>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>    It is simple for application to understand if it supported.
>>>>>>    It is simple in PMD to understand that it is required.
>>>>>>    It is simple to disable it - just reconfigure.
>>>>>>    Also it is easier to document it - just mention that
>>>>>>    the offload should be supported and enabled.
>>>>>>
> 
> This is not true, for example the PMD support Mark only in Nic mode
> While the application needs E-Switch then the PMD reports incorrectly.
> Or for example if the mark is supported only as only action, and the application needs
> decap and mark. this means that the user will never be able to use the mark
> so the PMD uses slower Rx function without the mark ever being used.

It is covered by (6) below.

>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>    I agree that it is valid argument. Solving the problem
>>>>>>    without extra entities is always nice, but unfortunately
>>>>>>    it is too complex in this case.
>>>>>>
> 
> You forgot a one more very important reason, the fact that the mark 
> may be supported only on some flows. So if the user needs mark with decap
> and this combination is not supported the user will never we able to use mark
> while using mark Rx function. So you get slow data path with no ability to use it.

Again, see (6) below.

>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>    It is necessary to choose a flow rule which should be used
>>>>>>    as a criteria. It could be hardware dependent.
>>>>>>    Complex logic is require in PMD if it wants to address the
>>>>>>    problem and control MARK delivery based on validated flow
>>>>>>    rules. It adds dependency between start/stop processing and
>>>>>>    flow rules validation code.
>>>>>>    It is pretty complicated to document it.
>>>>>>
>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>    if really used flow rules do not support MARK looks like
>>>>>>    drawback as well, but easily mitigated by a combination
>>>>>>    with solution (B) and only required if the application wants
>>>>>>    to dive in the level of optimization and complexity and
>>>>>>    makes sense if application knows required flow rules in
>>>>>>    advance. So, it is not a problem in this case.
>>>>>>
>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>    applications to understand if these features are supported,
>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
>>>>>>    enable/disable (dynamic field/flag lookup).
>>>>>>
>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>
>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>    As it was discussed in the case of RX_TIMESTAMP
>>>>>>    (if I remember it correctly):
>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
>>>>>>     - application enables the offload
>>>>>>     - PMD registers dynamic field for timestamp
>>>>>>    Solution (C):
>>>>>>      - PMD advertises nothing
>>>>>>      - application uses solution (B) to understand if
>>>>>>        these features are supported
>>>>>>      - application registers dynamic field/flag
>>>>>>      - PMD does lookup and solve the problem
>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>    solution is changed to require an application to register
>>>>>>    dynamic fields and PMD to do lookup if the offload is
>>>>>>    enabled. So, the only difference will be in no offload
>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
>>>>>>    to understand if it is supported or no.
>>>>>>    May be it would be really good since it will allow to
>>>>>>    have dynamic fields registered before mempool population.
>>>>>>
>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>     per queue. Moreover (C) looks global - for all devices.
>>>>>>     It could be really painful.
>>>>>>
>>>>>> (C) is nice, but I still vote for simplicity and granularity of
>>>>>> (A).
>>>>>
>>>>> I vote for clear separation of application needs and PMD support, by
>>>>> using the method C (dynamic fields).
>>>>> I agree timestamp must use the same path.
>>>>> I agree it's complicate because we don't know in advance whether a
>>>>> flow rule will be accepted, but that's the reality, config is complex.
>>>>
>>>> Do you think that global nature of the (C) is acceptable?
>>>
>>> That's a good question.
>>> Maybe the feature request should be per port.
>>> In this case, we are back to solution A with a flag per port?
>>>
>>> Note that A and C will not guarantee that the offload will be possible.
>>> We need B (flow rule validation) anyway.
>>
>> I may not understand how solution B can works well for all the cases.
>>
> 
> That is the point of solution B it always work, it is a bit harder to use maybe,
> but you can check and decide everything.
> 
>> A rte_flow rule can be issued after dev_start, which means the rx_burst
>> function is already selected at that time,
>> so does that mean the driver need to switch from a non- mark offload aware
>> path to a mark offload aware path without stop device? or it has to reject the
>> flow?
>> The question is if we have 2 data path, one support some offload , one not
>> but more fast, which one should be selected during dev_start? Isn't Offload
>> widely used to solve this problem?
>>
>> I think the option A solve all the problems, option C might also works, but A is
>> looks much straightforward for me.
>>
> 
> Solution A my result in selecting incorrect datapath  please see my answer above.
> 
> As you can guess I'm in favor of B and C (C need B)
> 
>> Regards
>> Qi
>>
>>
>>>
>>> It seems A, B, C are not alternatives but all required as pieces of a puzzle...
>>>
> 

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ori Kam]

Yes missed somehow 6.
One more general comment, what happened until now? The Mark is 
already implemented and working more then 2 years.

> -----Original Message-----
> From: Andrew Rybchenko <arybchenko@solarflare.com>
> Sent: Friday, November 8, 2019 2:20 PM
> To: Ori Kam <orika@mellanox.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> Thomas Monjalon <thomas@monjalon.net>
> Cc: dev@dpdk.org; pbhagavatula@marvell.com; Yigit, Ferruh
> <ferruh.yigit@intel.com>; jerinj@marvell.com; Mcnamara, John
> <john.mcnamara@intel.com>; Kovacevic, Marko
> <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as
> an offload
> 
> On 11/8/19 3:12 PM, Ori Kam wrote:
> >
> > Hi Andrew,
> >
> > Thanks very much for the summery, I think this makes the discussion much
> better.
> > (we even got some new folks 😊)
> >
> > I vote for B and C,
> >
> > Please read my comments below.
> >
> > Ori
> >
> >> -----Original Message-----
> >> From: dev <dev-bounces@dpdk.org> On Behalf Of Zhang, Qi Z
> >> Sent: Friday, November 8, 2019 1:41 PM
> >> To: Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> >> <arybchenko@solarflare.com>
> >> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> >> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> >> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> >> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> >> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> >> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> >> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update
> as
> >> an offload
> >>
> >>
> >>
> >>> -----Original Message-----
> >>> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> >>> Sent: Friday, November 8, 2019 7:04 PM
> >>> To: Andrew Rybchenko <arybchenko@solarflare.com>
> >>> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> >>> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> >>> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> >>> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> >>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> >>> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> >>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type
> update as
> >> an
> >>> offload
> >>>
> >>> 08/11/2019 11:42, Andrew Rybchenko:
> >>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> >>>>> 08/11/2019 09:35, Andrew Rybchenko:
> >>>>>> The problem:
> >>>>>> ~~~~~~~~~~~~
> >>>>>> PMD wants to know before port start if application wants to to use
> >>>>>> flow MARK/FLAG in the future. It is required because:
> >>>>>>
> >>>>>> 1. HW may be configured in a different way to reserve resources
> >>>>>>    for MARK/FLAG delivery
> >>>>>>
> >>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>>>>>    is faster, but does not support MARK)
> >>>>>
> >>>>> Thank you for the clear problem statement.
> >>>>> I agree with it. This is a real design issue.
> >>>>>
> >>>>>
> >>>>>> Discussed solutions:
> >>>>>> ~~~~~~~~~~~~~~~~~~~~
> >>>>
> >>>> May be it is not 100% clear since below are alternatives.
> >>>>
> >>>>>> A. Explicit Rx offload suggested by the patch.
> >>>>>>
> >>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> >>>>>>
> >
> > I don’t agree that this method is implicit it is quit explicit, do we support it in
> PMD
> > with the use case or not. Sound very explicit. You can argue that the PMD
> uses
> > validation flow to understand that it need to support MARK, but from
> every perspective
> > it is more correct . what does it mean if the application enabled mark and
> never download a flow?
> 
> It is implicit from PMD point of view, since there is no command
> to enable it. The decision to enable is the result of seen
> attempt to validate mark. If few rules are validated, but an
> application decides to have it disabled finally, how to achieve it?
> Last validation attempt wins?
> Other points are covered by (6) below.
> 
> >>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
> >>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
> >>>>>>    of (B) to discover if a feature is supported.
> >>>>>
> >>>>> The dynamic field should be registered via a new API function named
> >>>>> '<feature>_init'.
> >>>>> It means the application must explicit request the feature.
> >>>>> I agree this is the way to go.
> >>>>
> >>>> If I understand your statement correctly, but (C) is not ideal since
> >>>> it looks global. If registered dynamic field of mbuf and is flag that
> >>>> the feature should be enabled, it is a flag to all ports/devices.
> >>>>
> >>>>>> All solutions require changes in applications which use these
> >>>>>> features. There is a deprecation notice in place which advertises
> >>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to
> >>>>>> substitute it with solution (B) or (C). Solution (C) requires
> >>>>>> changes since it should be combined with (B) in order to understand
> >>>>>> if the feature is supported.
> >>>>>
> >>>>> I don't understand.
> >>>>> Application request and PMD support are two different things.
> >>>>> PMD support must be via rte_flow validation on a case by case
> anyway.
> >>>>
> >>>> I mean that application wants to understand if the feature is
> >>>> supported. Then, it wants to enable it. In the case of (B), if I
> >>>> understand the solution correctly, there is no explicit way to enable,
> >>>> PMD just detects it because of discovery is done (that's what I mean
> >>>> by "implicit" and it is a drawback from my point of view, but still
> >>>> could be considered). (C) solves the problem of (B).
> >>>>
> >>>>>> Advantages and drawbacks of solutions:
> >>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>>>>> 1. The main drawback of (A) is a "duplication" since we already
> >>>>>>    have a way to request flow MARK using rte_flow API.
> >>>>>>    I don't fully agree that it is a duplication, but I agree
> >>>>>>    that it sounds like duplication and complicates a bit flow
> >>>>>>    MARK usage by applications. (B) complicates it as well.
> >>>>>>
> >>>>>> 2. One more drawback of the solution (A) is the necessity of
> >>>>>>    similar solution for META and it eats one more offload bit.
> >>>>>>    Yes, that's true and I think it is not a problem.
> >>>>>>    It would make it easier for applications to find out if
> >>>>>>    either MARK or META is supported.
> >>>>>>
> >>>>>> 3. The main advantage of the solution (A) is simplicity.
> >>>>>>    It is simple for application to understand if it supported.
> >>>>>>    It is simple in PMD to understand that it is required.
> >>>>>>    It is simple to disable it - just reconfigure.
> >>>>>>    Also it is easier to document it - just mention that
> >>>>>>    the offload should be supported and enabled.
> >>>>>>
> >
> > This is not true, for example the PMD support Mark only in Nic mode
> > While the application needs E-Switch then the PMD reports incorrectly.
> > Or for example if the mark is supported only as only action, and the
> application needs
> > decap and mark. this means that the user will never be able to use the
> mark
> > so the PMD uses slower Rx function without the mark ever being used.
> 
> It is covered by (6) below.
> 
> >>>>>> 4. The main advantage of the solution (B) is no "duplication".
> >>>>>>    I agree that it is valid argument. Solving the problem
> >>>>>>    without extra entities is always nice, but unfortunately
> >>>>>>    it is too complex in this case.
> >>>>>>
> >
> > You forgot a one more very important reason, the fact that the mark
> > may be supported only on some flows. So if the user needs mark with
> decap
> > and this combination is not supported the user will never we able to use
> mark
> > while using mark Rx function. So you get slow data path with no ability to
> use it.
> 
> Again, see (6) below.
> 
> >>>>>> 5. The main drawback of the solution (B) is the complexity.
> >>>>>>    It is necessary to choose a flow rule which should be used
> >>>>>>    as a criteria. It could be hardware dependent.
> >>>>>>    Complex logic is require in PMD if it wants to address the
> >>>>>>    problem and control MARK delivery based on validated flow
> >>>>>>    rules. It adds dependency between start/stop processing and
> >>>>>>    flow rules validation code.
> >>>>>>    It is pretty complicated to document it.
> >>>>>>
> >>>>>> 6. Useless enabling of the offload in the case of solution (A)
> >>>>>>    if really used flow rules do not support MARK looks like
> >>>>>>    drawback as well, but easily mitigated by a combination
> >>>>>>    with solution (B) and only required if the application wants
> >>>>>>    to dive in the level of optimization and complexity and
> >>>>>>    makes sense if application knows required flow rules in
> >>>>>>    advance. So, it is not a problem in this case.
> >>>>>>
> >>>>>> 7. Solution (C) has drawbacks of the solution (B) for
> >>>>>>    applications to understand if these features are supported,
> >>>>>>    but no drawbacks in PMD, since explicit criteria is used to
> >>>>>>    enable/disable (dynamic field/flag lookup).
> >>>>>>
> >>>>>> 8. Solution (C) is nice since it avoids "duplication".
> >>>>>>
> >>>>>> 9. The main drawback of the solution (C) is asymmetry.
> >>>>>>    As it was discussed in the case of RX_TIMESTAMP
> >>>>>>    (if I remember it correctly):
> >>>>>>     - PMD advertises RX_TIMESTAMP offload capability
> >>>>>>     - application enables the offload
> >>>>>>     - PMD registers dynamic field for timestamp
> >>>>>>    Solution (C):
> >>>>>>      - PMD advertises nothing
> >>>>>>      - application uses solution (B) to understand if
> >>>>>>        these features are supported
> >>>>>>      - application registers dynamic field/flag
> >>>>>>      - PMD does lookup and solve the problem
> >>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> >>>>>>    solution is changed to require an application to register
> >>>>>>    dynamic fields and PMD to do lookup if the offload is
> >>>>>>    enabled. So, the only difference will be in no offload
> >>>>>>    in the case of flow MARK/FLAG and usage of complex logic
> >>>>>>    to understand if it is supported or no.
> >>>>>>    May be it would be really good since it will allow to
> >>>>>>    have dynamic fields registered before mempool population.
> >>>>>>
> >>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
> >>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
> >>>>>>     per queue. Moreover (C) looks global - for all devices.
> >>>>>>     It could be really painful.
> >>>>>>
> >>>>>> (C) is nice, but I still vote for simplicity and granularity of
> >>>>>> (A).
> >>>>>
> >>>>> I vote for clear separation of application needs and PMD support, by
> >>>>> using the method C (dynamic fields).
> >>>>> I agree timestamp must use the same path.
> >>>>> I agree it's complicate because we don't know in advance whether a
> >>>>> flow rule will be accepted, but that's the reality, config is complex.
> >>>>
> >>>> Do you think that global nature of the (C) is acceptable?
> >>>
> >>> That's a good question.
> >>> Maybe the feature request should be per port.
> >>> In this case, we are back to solution A with a flag per port?
> >>>
> >>> Note that A and C will not guarantee that the offload will be possible.
> >>> We need B (flow rule validation) anyway.
> >>
> >> I may not understand how solution B can works well for all the cases.
> >>
> >
> > That is the point of solution B it always work, it is a bit harder to use maybe,
> > but you can check and decide everything.
> >
> >> A rte_flow rule can be issued after dev_start, which means the rx_burst
> >> function is already selected at that time,
> >> so does that mean the driver need to switch from a non- mark offload
> aware
> >> path to a mark offload aware path without stop device? or it has to reject
> the
> >> flow?
> >> The question is if we have 2 data path, one support some offload , one
> not
> >> but more fast, which one should be selected during dev_start? Isn't
> Offload
> >> widely used to solve this problem?
> >>
> >> I think the option A solve all the problems, option C might also works, but
> A is
> >> looks much straightforward for me.
> >>
> >
> > Solution A my result in selecting incorrect datapath  please see my answer
> above.
> >
> > As you can guess I'm in favor of B and C (C need B)
> >
> >> Regards
> >> Qi
> >>
> >>
> >>>
> >>> It seems A, B, C are not alternatives but all required as pieces of a
> puzzle...
> >>>
> >

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

08/11/2019 12:40, Zhang, Qi Z:
> 
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> > Sent: Friday, November 8, 2019 7:04 PM
> > To: Andrew Rybchenko <arybchenko@solarflare.com>
> > Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> > pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> > jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> > Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> > <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> > ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> > Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> > offload
> > 
> > 08/11/2019 11:42, Andrew Rybchenko:
> > > On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> > > > 08/11/2019 09:35, Andrew Rybchenko:
> > > >> The problem:
> > > >> ~~~~~~~~~~~~
> > > >> PMD wants to know before port start if application wants to to use
> > > >> flow MARK/FLAG in the future. It is required because:
> > > >>
> > > >> 1. HW may be configured in a different way to reserve resources
> > > >>    for MARK/FLAG delivery
> > > >>
> > > >> 2. Datapath implementation choice may depend on it (e.g. vPMD
> > > >>    is faster, but does not support MARK)
> > > >
> > > > Thank you for the clear problem statement.
> > > > I agree with it. This is a real design issue.
> > > >
> > > >
> > > >> Discussed solutions:
> > > >> ~~~~~~~~~~~~~~~~~~~~
> > >
> > > May be it is not 100% clear since below are alternatives.
> > >
> > > >> A. Explicit Rx offload suggested by the patch.
> > > >>
> > > >> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> > > >>
> > > >> C. Use dynamic field/flag (i.e. application registers dynamic field
> > > >>    and/or flag and PMD uses lookup to solve the problem) plus part
> > > >>    of (B) to discover if a feature is supported.
> > > >
> > > > The dynamic field should be registered via a new API function named
> > > > '<feature>_init'.
> > > > It means the application must explicit request the feature.
> > > > I agree this is the way to go.
> > >
> > > If I understand your statement correctly, but (C) is not ideal since
> > > it looks global. If registered dynamic field of mbuf and is flag that
> > > the feature should be enabled, it is a flag to all ports/devices.
> > >
> > > >> All solutions require changes in applications which use these
> > > >> features. There is a deprecation notice in place which advertises
> > > >> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to
> > > >> substitute it with solution (B) or (C). Solution (C) requires
> > > >> changes since it should be combined with (B) in order to understand
> > > >> if the feature is supported.
> > > >
> > > > I don't understand.
> > > > Application request and PMD support are two different things.
> > > > PMD support must be via rte_flow validation on a case by case anyway.
> > >
> > > I mean that application wants to understand if the feature is
> > > supported. Then, it wants to enable it. In the case of (B), if I
> > > understand the solution correctly, there is no explicit way to enable,
> > > PMD just detects it because of discovery is done (that's what I mean
> > > by "implicit" and it is a drawback from my point of view, but still
> > > could be considered). (C) solves the problem of (B).
> > >
> > > >> Advantages and drawbacks of solutions:
> > > >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > >> 1. The main drawback of (A) is a "duplication" since we already
> > > >>    have a way to request flow MARK using rte_flow API.
> > > >>    I don't fully agree that it is a duplication, but I agree
> > > >>    that it sounds like duplication and complicates a bit flow
> > > >>    MARK usage by applications. (B) complicates it as well.
> > > >>
> > > >> 2. One more drawback of the solution (A) is the necessity of
> > > >>    similar solution for META and it eats one more offload bit.
> > > >>    Yes, that's true and I think it is not a problem.
> > > >>    It would make it easier for applications to find out if
> > > >>    either MARK or META is supported.
> > > >>
> > > >> 3. The main advantage of the solution (A) is simplicity.
> > > >>    It is simple for application to understand if it supported.
> > > >>    It is simple in PMD to understand that it is required.
> > > >>    It is simple to disable it - just reconfigure.
> > > >>    Also it is easier to document it - just mention that
> > > >>    the offload should be supported and enabled.
> > > >>
> > > >> 4. The main advantage of the solution (B) is no "duplication".
> > > >>    I agree that it is valid argument. Solving the problem
> > > >>    without extra entities is always nice, but unfortunately
> > > >>    it is too complex in this case.
> > > >>
> > > >> 5. The main drawback of the solution (B) is the complexity.
> > > >>    It is necessary to choose a flow rule which should be used
> > > >>    as a criteria. It could be hardware dependent.
> > > >>    Complex logic is require in PMD if it wants to address the
> > > >>    problem and control MARK delivery based on validated flow
> > > >>    rules. It adds dependency between start/stop processing and
> > > >>    flow rules validation code.
> > > >>    It is pretty complicated to document it.
> > > >>
> > > >> 6. Useless enabling of the offload in the case of solution (A)
> > > >>    if really used flow rules do not support MARK looks like
> > > >>    drawback as well, but easily mitigated by a combination
> > > >>    with solution (B) and only required if the application wants
> > > >>    to dive in the level of optimization and complexity and
> > > >>    makes sense if application knows required flow rules in
> > > >>    advance. So, it is not a problem in this case.
> > > >>
> > > >> 7. Solution (C) has drawbacks of the solution (B) for
> > > >>    applications to understand if these features are supported,
> > > >>    but no drawbacks in PMD, since explicit criteria is used to
> > > >>    enable/disable (dynamic field/flag lookup).
> > > >>
> > > >> 8. Solution (C) is nice since it avoids "duplication".
> > > >>
> > > >> 9. The main drawback of the solution (C) is asymmetry.
> > > >>    As it was discussed in the case of RX_TIMESTAMP
> > > >>    (if I remember it correctly):
> > > >>     - PMD advertises RX_TIMESTAMP offload capability
> > > >>     - application enables the offload
> > > >>     - PMD registers dynamic field for timestamp
> > > >>    Solution (C):
> > > >>      - PMD advertises nothing
> > > >>      - application uses solution (B) to understand if
> > > >>        these features are supported
> > > >>      - application registers dynamic field/flag
> > > >>      - PMD does lookup and solve the problem
> > > >>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> > > >>    solution is changed to require an application to register
> > > >>    dynamic fields and PMD to do lookup if the offload is
> > > >>    enabled. So, the only difference will be in no offload
> > > >>    in the case of flow MARK/FLAG and usage of complex logic
> > > >>    to understand if it is supported or no.
> > > >>    May be it would be really good since it will allow to
> > > >>    have dynamic fields registered before mempool population.
> > > >>
> > > >> 10. Common drawback of solutions (B) and (C) is no granularity.
> > > >>     Solution (A) may be per queue while (B) and (C) cannot be
> > > >>     per queue. Moreover (C) looks global - for all devices.
> > > >>     It could be really painful.
> > > >>
> > > >> (C) is nice, but I still vote for simplicity and granularity of
> > > >> (A).
> > > >
> > > > I vote for clear separation of application needs and PMD support, by
> > > > using the method C (dynamic fields).
> > > > I agree timestamp must use the same path.
> > > > I agree it's complicate because we don't know in advance whether a
> > > > flow rule will be accepted, but that's the reality, config is complex.
> > >
> > > Do you think that global nature of the (C) is acceptable?
> > 
> > That's a good question.
> > Maybe the feature request should be per port.
> > In this case, we are back to solution A with a flag per port?
> > 
> > Note that A and C will not guarantee that the offload will be possible.
> > We need B (flow rule validation) anyway.
> 
> I may not understand how solution B can works well for all the cases.

I think you didn't read above carefully.
I am not saying B will solve all, but is needed in addition of A and C.

> A rte_flow rule can be issued after dev_start, which means the rx_burst function is already selected at that time, 
> so does that mean the driver need to switch from a non- mark offload aware path to a mark offload aware path without stop device?

I agree to have the application request the offload before starting (A).

> or it has to reject the flow? 

Yes if PMD is not ready (ignored app request or app did not request),
it must reject the flow rule.

> The question is if we have 2 data path, one support some offload , one not but more fast, which one should be selected during dev_start? Isn't Offload widely used to solve this problem?
> 
> I think the option A solve all the problems, option C might also works, but A is looks much straightforward for me.

Again, the answer is below:

> > It seems A, B, C are not alternatives but all required as pieces of a puzzle...

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Zhang, Qi Z]

> -----Original Message-----
> From: Ori Kam <orika@mellanox.com>
> Sent: Friday, November 8, 2019 8:43 PM
> To: Andrew Rybchenko <arybchenko@solarflare.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Thomas Monjalon <thomas@monjalon.net>
> Cc: dev@dpdk.org; pbhagavatula@marvell.com; Yigit, Ferruh
> <ferruh.yigit@intel.com>; jerinj@marvell.com; Mcnamara, John
> <john.mcnamara@intel.com>; Kovacevic, Marko
> <marko.kovacevic@intel.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> Subject: RE: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an
> offload
> 
> Yes missed somehow 6.
> One more general comment, what happened until now? The Mark is already
> implemented and working more then 2 years.

Actually we get many complain on i40e driver does not support MARK due to vector path is selected by default
The driver always need a hint to select the mark aware data path by "--pkt-filter-mode =perfect" which is just a workaround.
though this issue has been finally solved since software mark on vPMD is enabled recently on i40e.
but For the new ice driver, we didn't get chance to enable this on vPMD yet, then we meet the same issue.

I know option B can always work, driver can always decide the function call at runtime, but that need a lot rework and may impact performance, while option A just give more flexibility,
and it does not block a driver to enable all the best thing required by option B.

Regards
Qi


> 
> > -----Original Message-----
> > From: Andrew Rybchenko <arybchenko@solarflare.com>
> > Sent: Friday, November 8, 2019 2:20 PM
> > To: Ori Kam <orika@mellanox.com>; Zhang, Qi Z <qi.z.zhang@intel.com>;
> > Thomas Monjalon <thomas@monjalon.net>
> > Cc: dev@dpdk.org; pbhagavatula@marvell.com; Yigit, Ferruh
> > <ferruh.yigit@intel.com>; jerinj@marvell.com; Mcnamara, John
> > <john.mcnamara@intel.com>; Kovacevic, Marko
> > <marko.kovacevic@intel.com>; Adrien Mazarguil
> > <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> > ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> > Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type
> > update as an offload
> >
> > On 11/8/19 3:12 PM, Ori Kam wrote:
> > >
> > > Hi Andrew,
> > >
> > > Thanks very much for the summery, I think this makes the discussion
> > > much
> > better.
> > > (we even got some new folks 😊)
> > >
> > > I vote for B and C,
> > >
> > > Please read my comments below.
> > >
> > > Ori
> > >
> > >> -----Original Message-----
> > >> From: dev <dev-bounces@dpdk.org> On Behalf Of Zhang, Qi Z
> > >> Sent: Friday, November 8, 2019 1:41 PM
> > >> To: Thomas Monjalon <thomas@monjalon.net>; Andrew Rybchenko
> > >> <arybchenko@solarflare.com>
> > >> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> > >> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> > >> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> > >> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> > >> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> > >> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> > >> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type
> > >> update
> > as
> > >> an offload
> > >>
> > >>
> > >>
> > >>> -----Original Message-----
> > >>> From: dev <dev-bounces@dpdk.org> On Behalf Of Thomas Monjalon
> > >>> Sent: Friday, November 8, 2019 7:04 PM
> > >>> To: Andrew Rybchenko <arybchenko@solarflare.com>
> > >>> Cc: Ori Kam <orika@mellanox.com>; dev@dpdk.org;
> > >>> pbhagavatula@marvell.com; Yigit, Ferruh <ferruh.yigit@intel.com>;
> > >>> jerinj@marvell.com; Mcnamara, John <john.mcnamara@intel.com>;
> > >>> Kovacevic, Marko <marko.kovacevic@intel.com>; Adrien Mazarguil
> > >>> <adrien.mazarguil@6wind.com>; david.marchand@redhat.com;
> > >>> ktraynor@redhat.com; Olivier Matz <olivier.matz@6wind.com>
> > >>> Subject: Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type
> > update as
> > >> an
> > >>> offload
> > >>>
> > >>> 08/11/2019 11:42, Andrew Rybchenko:
> > >>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> > >>>>> 08/11/2019 09:35, Andrew Rybchenko:
> > >>>>>> The problem:
> > >>>>>> ~~~~~~~~~~~~
> > >>>>>> PMD wants to know before port start if application wants to to
> > >>>>>> use flow MARK/FLAG in the future. It is required because:
> > >>>>>>
> > >>>>>> 1. HW may be configured in a different way to reserve resources
> > >>>>>>    for MARK/FLAG delivery
> > >>>>>>
> > >>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
> > >>>>>>    is faster, but does not support MARK)
> > >>>>>
> > >>>>> Thank you for the clear problem statement.
> > >>>>> I agree with it. This is a real design issue.
> > >>>>>
> > >>>>>
> > >>>>>> Discussed solutions:
> > >>>>>> ~~~~~~~~~~~~~~~~~~~~
> > >>>>
> > >>>> May be it is not 100% clear since below are alternatives.
> > >>>>
> > >>>>>> A. Explicit Rx offload suggested by the patch.
> > >>>>>>
> > >>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> > >>>>>>
> > >
> > > I don’t agree that this method is implicit it is quit explicit, do
> > > we support it in
> > PMD
> > > with the use case or not. Sound very explicit. You can argue that
> > > the PMD
> > uses
> > > validation flow to understand that it need to support MARK, but from
> > every perspective
> > > it is more correct . what does it mean if the application enabled
> > > mark and
> > never download a flow?
> >
> > It is implicit from PMD point of view, since there is no command to
> > enable it. The decision to enable is the result of seen attempt to
> > validate mark. If few rules are validated, but an application decides
> > to have it disabled finally, how to achieve it?
> > Last validation attempt wins?
> > Other points are covered by (6) below.
> >
> > >>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
> > >>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
> > >>>>>>    of (B) to discover if a feature is supported.
> > >>>>>
> > >>>>> The dynamic field should be registered via a new API function
> > >>>>> named '<feature>_init'.
> > >>>>> It means the application must explicit request the feature.
> > >>>>> I agree this is the way to go.
> > >>>>
> > >>>> If I understand your statement correctly, but (C) is not ideal
> > >>>> since it looks global. If registered dynamic field of mbuf and is
> > >>>> flag that the feature should be enabled, it is a flag to all ports/devices.
> > >>>>
> > >>>>>> All solutions require changes in applications which use these
> > >>>>>> features. There is a deprecation notice in place which
> > >>>>>> advertises DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is
> > >>>>>> OK to substitute it with solution (B) or (C). Solution (C)
> > >>>>>> requires changes since it should be combined with (B) in order
> > >>>>>> to understand if the feature is supported.
> > >>>>>
> > >>>>> I don't understand.
> > >>>>> Application request and PMD support are two different things.
> > >>>>> PMD support must be via rte_flow validation on a case by case
> > anyway.
> > >>>>
> > >>>> I mean that application wants to understand if the feature is
> > >>>> supported. Then, it wants to enable it. In the case of (B), if I
> > >>>> understand the solution correctly, there is no explicit way to
> > >>>> enable, PMD just detects it because of discovery is done (that's
> > >>>> what I mean by "implicit" and it is a drawback from my point of
> > >>>> view, but still could be considered). (C) solves the problem of (B).
> > >>>>
> > >>>>>> Advantages and drawbacks of solutions:
> > >>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >>>>>> 1. The main drawback of (A) is a "duplication" since we already
> > >>>>>>    have a way to request flow MARK using rte_flow API.
> > >>>>>>    I don't fully agree that it is a duplication, but I agree
> > >>>>>>    that it sounds like duplication and complicates a bit flow
> > >>>>>>    MARK usage by applications. (B) complicates it as well.
> > >>>>>>
> > >>>>>> 2. One more drawback of the solution (A) is the necessity of
> > >>>>>>    similar solution for META and it eats one more offload bit.
> > >>>>>>    Yes, that's true and I think it is not a problem.
> > >>>>>>    It would make it easier for applications to find out if
> > >>>>>>    either MARK or META is supported.
> > >>>>>>
> > >>>>>> 3. The main advantage of the solution (A) is simplicity.
> > >>>>>>    It is simple for application to understand if it supported.
> > >>>>>>    It is simple in PMD to understand that it is required.
> > >>>>>>    It is simple to disable it - just reconfigure.
> > >>>>>>    Also it is easier to document it - just mention that
> > >>>>>>    the offload should be supported and enabled.
> > >>>>>>
> > >
> > > This is not true, for example the PMD support Mark only in Nic mode
> > > While the application needs E-Switch then the PMD reports incorrectly.
> > > Or for example if the mark is supported only as only action, and the
> > application needs
> > > decap and mark. this means that the user will never be able to use
> > > the
> > mark
> > > so the PMD uses slower Rx function without the mark ever being used.
> >
> > It is covered by (6) below.
> >
> > >>>>>> 4. The main advantage of the solution (B) is no "duplication".
> > >>>>>>    I agree that it is valid argument. Solving the problem
> > >>>>>>    without extra entities is always nice, but unfortunately
> > >>>>>>    it is too complex in this case.
> > >>>>>>
> > >
> > > You forgot a one more very important reason, the fact that the mark
> > > may be supported only on some flows. So if the user needs mark with
> > decap
> > > and this combination is not supported the user will never we able to
> > > use
> > mark
> > > while using mark Rx function. So you get slow data path with no
> > > ability to
> > use it.
> >
> > Again, see (6) below.
> >
> > >>>>>> 5. The main drawback of the solution (B) is the complexity.
> > >>>>>>    It is necessary to choose a flow rule which should be used
> > >>>>>>    as a criteria. It could be hardware dependent.
> > >>>>>>    Complex logic is require in PMD if it wants to address the
> > >>>>>>    problem and control MARK delivery based on validated flow
> > >>>>>>    rules. It adds dependency between start/stop processing and
> > >>>>>>    flow rules validation code.
> > >>>>>>    It is pretty complicated to document it.
> > >>>>>>
> > >>>>>> 6. Useless enabling of the offload in the case of solution (A)
> > >>>>>>    if really used flow rules do not support MARK looks like
> > >>>>>>    drawback as well, but easily mitigated by a combination
> > >>>>>>    with solution (B) and only required if the application wants
> > >>>>>>    to dive in the level of optimization and complexity and
> > >>>>>>    makes sense if application knows required flow rules in
> > >>>>>>    advance. So, it is not a problem in this case.
> > >>>>>>
> > >>>>>> 7. Solution (C) has drawbacks of the solution (B) for
> > >>>>>>    applications to understand if these features are supported,
> > >>>>>>    but no drawbacks in PMD, since explicit criteria is used to
> > >>>>>>    enable/disable (dynamic field/flag lookup).
> > >>>>>>
> > >>>>>> 8. Solution (C) is nice since it avoids "duplication".
> > >>>>>>
> > >>>>>> 9. The main drawback of the solution (C) is asymmetry.
> > >>>>>>    As it was discussed in the case of RX_TIMESTAMP
> > >>>>>>    (if I remember it correctly):
> > >>>>>>     - PMD advertises RX_TIMESTAMP offload capability
> > >>>>>>     - application enables the offload
> > >>>>>>     - PMD registers dynamic field for timestamp
> > >>>>>>    Solution (C):
> > >>>>>>      - PMD advertises nothing
> > >>>>>>      - application uses solution (B) to understand if
> > >>>>>>        these features are supported
> > >>>>>>      - application registers dynamic field/flag
> > >>>>>>      - PMD does lookup and solve the problem
> > >>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> > >>>>>>    solution is changed to require an application to register
> > >>>>>>    dynamic fields and PMD to do lookup if the offload is
> > >>>>>>    enabled. So, the only difference will be in no offload
> > >>>>>>    in the case of flow MARK/FLAG and usage of complex logic
> > >>>>>>    to understand if it is supported or no.
> > >>>>>>    May be it would be really good since it will allow to
> > >>>>>>    have dynamic fields registered before mempool population.
> > >>>>>>
> > >>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
> > >>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
> > >>>>>>     per queue. Moreover (C) looks global - for all devices.
> > >>>>>>     It could be really painful.
> > >>>>>>
> > >>>>>> (C) is nice, but I still vote for simplicity and granularity of
> > >>>>>> (A).
> > >>>>>
> > >>>>> I vote for clear separation of application needs and PMD
> > >>>>> support, by using the method C (dynamic fields).
> > >>>>> I agree timestamp must use the same path.
> > >>>>> I agree it's complicate because we don't know in advance whether
> > >>>>> a flow rule will be accepted, but that's the reality, config is complex.
> > >>>>
> > >>>> Do you think that global nature of the (C) is acceptable?
> > >>>
> > >>> That's a good question.
> > >>> Maybe the feature request should be per port.
> > >>> In this case, we are back to solution A with a flag per port?
> > >>>
> > >>> Note that A and C will not guarantee that the offload will be possible.
> > >>> We need B (flow rule validation) anyway.
> > >>
> > >> I may not understand how solution B can works well for all the cases.
> > >>
> > >
> > > That is the point of solution B it always work, it is a bit harder
> > > to use maybe, but you can check and decide everything.
> > >
> > >> A rte_flow rule can be issued after dev_start, which means the
> > >> rx_burst function is already selected at that time, so does that
> > >> mean the driver need to switch from a non- mark offload
> > aware
> > >> path to a mark offload aware path without stop device? or it has to
> > >> reject
> > the
> > >> flow?
> > >> The question is if we have 2 data path, one support some offload ,
> > >> one
> > not
> > >> but more fast, which one should be selected during dev_start? Isn't
> > Offload
> > >> widely used to solve this problem?
> > >>
> > >> I think the option A solve all the problems, option C might also
> > >> works, but
> > A is
> > >> looks much straightforward for me.
> > >>
> > >
> > > Solution A my result in selecting incorrect datapath  please see my
> > > answer
> > above.
> > >
> > > As you can guess I'm in favor of B and C (C need B)
> > >
> > >> Regards
> > >> Qi
> > >>
> > >>
> > >>>
> > >>> It seems A, B, C are not alternatives but all required as pieces
> > >>> of a
> > puzzle...
> > >>>
> > >

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

08/11/2019 13:00, Andrew Rybchenko:
> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
> > 08/11/2019 11:42, Andrew Rybchenko:
> >> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> >>> 08/11/2019 09:35, Andrew Rybchenko:
> >>>> The problem:
> >>>> ~~~~~~~~~~~~
> >>>> PMD wants to know before port start if application wants to
> >>>> to use flow MARK/FLAG in the future. It is required because:
> >>>>
> >>>> 1. HW may be configured in a different way to reserve resources
> >>>>    for MARK/FLAG delivery
> >>>>
> >>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>>>    is faster, but does not support MARK)
> >>>
> >>> Thank you for the clear problem statement.
> >>> I agree with it. This is a real design issue.
> >>>
> >>>
> >>>> Discussed solutions:
> >>>> ~~~~~~~~~~~~~~~~~~~~
> >>
> >> May be it is not 100% clear since below are alternatives.
> >>
> >>>> A. Explicit Rx offload suggested by the patch.
> >>>>
> >>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> >>>>
> >>>> C. Use dynamic field/flag (i.e. application registers dynamic field
> >>>>    and/or flag and PMD uses lookup to solve the problem) plus part
> >>>>    of (B) to discover if a feature is supported.
> >>>
> >>> The dynamic field should be registered via a new API function
> >>> named '<feature>_init'.
> >>> It means the application must explicit request the feature.
> >>> I agree this is the way to go.
> >>
> >> If I understand your statement correctly, but (C) is not ideal since it
> >> looks global. If registered dynamic field of mbuf and is flag that
> >> the feature should be enabled, it is a flag to all ports/devices.
> >>
> >>>> All solutions require changes in applications which use these
> >>>> features. There is a deprecation notice in place which advertises
> >>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
> >>>> it with solution (B) or (C). Solution (C) requires changes since
> >>>> it should be combined with (B) in order to understand if
> >>>> the feature is supported.
> >>>
> >>> I don't understand.
> >>> Application request and PMD support are two different things.
> >>> PMD support must be via rte_flow validation on a case by case anyway.
> >>
> >> I mean that application wants to understand if the feature is
> >> supported. Then, it wants to enable it. In the case of (B),
> >> if I understand the solution correctly, there is no explicit
> >> way to enable, PMD just detects it because of discovery is done
> >> (that's what I mean by "implicit" and it is a drawback from my
> >> point of view, but still could be considered). (C) solves the
> >> problem of (B).
> >>
> >>>> Advantages and drawbacks of solutions:
> >>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>>> 1. The main drawback of (A) is a "duplication" since we already
> >>>>    have a way to request flow MARK using rte_flow API.
> >>>>    I don't fully agree that it is a duplication, but I agree
> >>>>    that it sounds like duplication and complicates a bit flow
> >>>>    MARK usage by applications. (B) complicates it as well.
> >>>>
> >>>> 2. One more drawback of the solution (A) is the necessity of
> >>>>    similar solution for META and it eats one more offload bit.
> >>>>    Yes, that's true and I think it is not a problem.
> >>>>    It would make it easier for applications to find out if
> >>>>    either MARK or META is supported.
> >>>>
> >>>> 3. The main advantage of the solution (A) is simplicity.
> >>>>    It is simple for application to understand if it supported.
> >>>>    It is simple in PMD to understand that it is required.
> >>>>    It is simple to disable it - just reconfigure.
> >>>>    Also it is easier to document it - just mention that
> >>>>    the offload should be supported and enabled.
> >>>>
> >>>> 4. The main advantage of the solution (B) is no "duplication".
> >>>>    I agree that it is valid argument. Solving the problem
> >>>>    without extra entities is always nice, but unfortunately
> >>>>    it is too complex in this case.
> >>>>
> >>>> 5. The main drawback of the solution (B) is the complexity.
> >>>>    It is necessary to choose a flow rule which should be used
> >>>>    as a criteria. It could be hardware dependent.
> >>>>    Complex logic is require in PMD if it wants to address the
> >>>>    problem and control MARK delivery based on validated flow
> >>>>    rules. It adds dependency between start/stop processing and
> >>>>    flow rules validation code.
> >>>>    It is pretty complicated to document it.
> >>>>
> >>>> 6. Useless enabling of the offload in the case of solution (A)
> >>>>    if really used flow rules do not support MARK looks like
> >>>>    drawback as well, but easily mitigated by a combination
> >>>>    with solution (B) and only required if the application wants
> >>>>    to dive in the level of optimization and complexity and
> >>>>    makes sense if application knows required flow rules in
> >>>>    advance. So, it is not a problem in this case.
> >>>>
> >>>> 7. Solution (C) has drawbacks of the solution (B) for
> >>>>    applications to understand if these features are supported,
> >>>>    but no drawbacks in PMD, since explicit criteria is used to
> >>>>    enable/disable (dynamic field/flag lookup).
> >>>>
> >>>> 8. Solution (C) is nice since it avoids "duplication".
> >>>>
> >>>> 9. The main drawback of the solution (C) is asymmetry.
> >>>>    As it was discussed in the case of RX_TIMESTAMP
> >>>>    (if I remember it correctly):
> >>>>     - PMD advertises RX_TIMESTAMP offload capability
> >>>>     - application enables the offload
> >>>>     - PMD registers dynamic field for timestamp
> >>>>    Solution (C):
> >>>>      - PMD advertises nothing
> >>>>      - application uses solution (B) to understand if
> >>>>        these features are supported
> >>>>      - application registers dynamic field/flag
> >>>>      - PMD does lookup and solve the problem
> >>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> >>>>    solution is changed to require an application to register
> >>>>    dynamic fields and PMD to do lookup if the offload is
> >>>>    enabled. So, the only difference will be in no offload
> >>>>    in the case of flow MARK/FLAG and usage of complex logic
> >>>>    to understand if it is supported or no.
> >>>>    May be it would be really good since it will allow to
> >>>>    have dynamic fields registered before mempool population.
> >>>>
> >>>> 10. Common drawback of solutions (B) and (C) is no granularity.
> >>>>     Solution (A) may be per queue while (B) and (C) cannot be
> >>>>     per queue. Moreover (C) looks global - for all devices.
> >>>>     It could be really painful.
> >>>>
> >>>> (C) is nice, but I still vote for simplicity and
> >>>> granularity of (A).
> >>>
> >>> I vote for clear separation of application needs and PMD support,
> >>> by using the method C (dynamic fields).
> >>> I agree timestamp must use the same path.
> >>> I agree it's complicate because we don't know in advance whether
> >>> a flow rule will be accepted, but that's the reality, config is complex.
> >>
> >> Do you think that global nature of the (C) is acceptable?
> > 
> > That's a good question.
> > Maybe the feature request should be per port.
> > In this case, we are back to solution A with a flag per port?
> 
> Offloads are natively per-queue as well, so (A) keeps the choice
> between per-port vs per-queue to PMDs as usual.
> 
> > Note that A and C will not guarantee that the offload will be possible.
> 
> Yes, definitely.
> 
> > We need B (flow rule validation) anyway.
> 
> Strictly speaking (B) (checking flow rules before device
> startup) is required if an application can predict flow
> rules and wants to ensure that MARK offload will be usable.
> Otherwise, it may be skipped.

No no, I mean flow rule validation MUST be used anyway
during the runtime before applying a rule.
I agree it is hard to predict. I speak only about real rules.

> > It seems A, B, C are not alternatives but all required
> > as pieces of a puzzle...
> 
> Unfortunately true in the most complex case.
> Right now it will be A with B if required as explained above.
> C will come a bit later when the field migrates to dynamic.
> 
> May be it is even better if application registers dynamic
> fields before an attempt to enable offload to be sure that
> it will not fail because of impossibility to register
> dynamic field (lack of space). I'm not sure, but it is not
> not that important.

Yes of course, lack of mbuf space is another reason for
disabling the feature.

> If we finally go way A, should we add offloads for META back?
> I guess separate Rx and Tx are required.

I would prefer to add it as dynamic flags.
Why rushing on a very temporary solution while it is not a new issue?

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

08/11/2019 14:16, Zhang, Qi Z:
> From: Ori Kam <orika@mellanox.com>
> > 
> > Yes missed somehow 6.
> > One more general comment, what happened until now? The Mark is already
> > implemented and working more then 2 years.
> 
> Actually we get many complain on i40e driver does not support MARK due to vector path is selected by default
> The driver always need a hint to select the mark aware data path by "--pkt-filter-mode =perfect" which is just a workaround.
> though this issue has been finally solved since software mark on vPMD is enabled recently on i40e.
> but For the new ice driver, we didn't get chance to enable this on vPMD yet, then we meet the same issue.

Qi, Intel knows the issue for a long time.
Did Intel helped with dynamic mbuf (part of the solution)?
Did Intel helped with this ethdev change?

In general, doing some PMD workaround instead of discussing API change is bad.
That's the same story for the VF config via representor, etc.
The way Intel was pushing some API which were not generic at all is also bad.
I really believe it's time to have truly generic API for all needs.
Speaking about that, we need a lot of cleanups in ethdev to remove some
vendor-specific bits and find generic replacements.

Qi, please don't take it personnaly, I am happy you enter in the discussion,
it is a good sign that we can do API improvements during next year.
I am motivated to address all ethdev issues before the next LTS 20.11.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/8/19 4:17 PM, Thomas Monjalon wrote:
> 08/11/2019 13:00, Andrew Rybchenko:
>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>> The problem:
>>>>>> ~~~~~~~~~~~~
>>>>>> PMD wants to know before port start if application wants to
>>>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>>>
>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>    for MARK/FLAG delivery
>>>>>>
>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>    is faster, but does not support MARK)
>>>>> Thank you for the clear problem statement.
>>>>> I agree with it. This is a real design issue.
>>>>>
>>>>>
>>>>>> Discussed solutions:
>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>> May be it is not 100% clear since below are alternatives.
>>>>
>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>
>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>
>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>    of (B) to discover if a feature is supported.
>>>>> The dynamic field should be registered via a new API function
>>>>> named '<feature>_init'.
>>>>> It means the application must explicit request the feature.
>>>>> I agree this is the way to go.
>>>> If I understand your statement correctly, but (C) is not ideal since it
>>>> looks global. If registered dynamic field of mbuf and is flag that
>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>
>>>>>> All solutions require changes in applications which use these
>>>>>> features. There is a deprecation notice in place which advertises
>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>>>> it should be combined with (B) in order to understand if
>>>>>> the feature is supported.
>>>>> I don't understand.
>>>>> Application request and PMD support are two different things.
>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>> I mean that application wants to understand if the feature is
>>>> supported. Then, it wants to enable it. In the case of (B),
>>>> if I understand the solution correctly, there is no explicit
>>>> way to enable, PMD just detects it because of discovery is done
>>>> (that's what I mean by "implicit" and it is a drawback from my
>>>> point of view, but still could be considered). (C) solves the
>>>> problem of (B).
>>>>
>>>>>> Advantages and drawbacks of solutions:
>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>    have a way to request flow MARK using rte_flow API.
>>>>>>    I don't fully agree that it is a duplication, but I agree
>>>>>>    that it sounds like duplication and complicates a bit flow
>>>>>>    MARK usage by applications. (B) complicates it as well.
>>>>>>
>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>    similar solution for META and it eats one more offload bit.
>>>>>>    Yes, that's true and I think it is not a problem.
>>>>>>    It would make it easier for applications to find out if
>>>>>>    either MARK or META is supported.
>>>>>>
>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>    It is simple for application to understand if it supported.
>>>>>>    It is simple in PMD to understand that it is required.
>>>>>>    It is simple to disable it - just reconfigure.
>>>>>>    Also it is easier to document it - just mention that
>>>>>>    the offload should be supported and enabled.
>>>>>>
>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>    I agree that it is valid argument. Solving the problem
>>>>>>    without extra entities is always nice, but unfortunately
>>>>>>    it is too complex in this case.
>>>>>>
>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>    It is necessary to choose a flow rule which should be used
>>>>>>    as a criteria. It could be hardware dependent.
>>>>>>    Complex logic is require in PMD if it wants to address the
>>>>>>    problem and control MARK delivery based on validated flow
>>>>>>    rules. It adds dependency between start/stop processing and
>>>>>>    flow rules validation code.
>>>>>>    It is pretty complicated to document it.
>>>>>>
>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>    if really used flow rules do not support MARK looks like
>>>>>>    drawback as well, but easily mitigated by a combination
>>>>>>    with solution (B) and only required if the application wants
>>>>>>    to dive in the level of optimization and complexity and
>>>>>>    makes sense if application knows required flow rules in
>>>>>>    advance. So, it is not a problem in this case.
>>>>>>
>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>    applications to understand if these features are supported,
>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
>>>>>>    enable/disable (dynamic field/flag lookup).
>>>>>>
>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>
>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>    As it was discussed in the case of RX_TIMESTAMP
>>>>>>    (if I remember it correctly):
>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
>>>>>>     - application enables the offload
>>>>>>     - PMD registers dynamic field for timestamp
>>>>>>    Solution (C):
>>>>>>      - PMD advertises nothing
>>>>>>      - application uses solution (B) to understand if
>>>>>>        these features are supported
>>>>>>      - application registers dynamic field/flag
>>>>>>      - PMD does lookup and solve the problem
>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>    solution is changed to require an application to register
>>>>>>    dynamic fields and PMD to do lookup if the offload is
>>>>>>    enabled. So, the only difference will be in no offload
>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
>>>>>>    to understand if it is supported or no.
>>>>>>    May be it would be really good since it will allow to
>>>>>>    have dynamic fields registered before mempool population.
>>>>>>
>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>     per queue. Moreover (C) looks global - for all devices.
>>>>>>     It could be really painful.
>>>>>>
>>>>>> (C) is nice, but I still vote for simplicity and
>>>>>> granularity of (A).
>>>>> I vote for clear separation of application needs and PMD support,
>>>>> by using the method C (dynamic fields).
>>>>> I agree timestamp must use the same path.
>>>>> I agree it's complicate because we don't know in advance whether
>>>>> a flow rule will be accepted, but that's the reality, config is complex.
>>>> Do you think that global nature of the (C) is acceptable?
>>> That's a good question.
>>> Maybe the feature request should be per port.
>>> In this case, we are back to solution A with a flag per port?
>> Offloads are natively per-queue as well, so (A) keeps the choice
>> between per-port vs per-queue to PMDs as usual.
>>
>>> Note that A and C will not guarantee that the offload will be possible.
>> Yes, definitely.
>>
>>> We need B (flow rule validation) anyway.
>> Strictly speaking (B) (checking flow rules before device
>> startup) is required if an application can predict flow
>> rules and wants to ensure that MARK offload will be usable.
>> Otherwise, it may be skipped.
> No no, I mean flow rule validation MUST be used anyway
> during the runtime before applying a rule.
> I agree it is hard to predict. I speak only about real rules.

OK, I see. Of course, flow rule validation is required at runtime.
I was rather concentrated on the stated problem solutions.

>>> It seems A, B, C are not alternatives but all required
>>> as pieces of a puzzle...
>> Unfortunately true in the most complex case.
>> Right now it will be A with B if required as explained above.
>> C will come a bit later when the field migrates to dynamic.
>>
>> May be it is even better if application registers dynamic
>> fields before an attempt to enable offload to be sure that
>> it will not fail because of impossibility to register
>> dynamic field (lack of space). I'm not sure, but it is not
>> not that important.
> Yes of course, lack of mbuf space is another reason for
> disabling the feature.
>
>> If we finally go way A, should we add offloads for META back?
>> I guess separate Rx and Tx are required.
> I would prefer to add it as dynamic flags.
> Why rushing on a very temporary solution while it is not a new issue?

Basically it means that we go just (B)+(C) in the case of META.
I have no strong opinion but thought that it could be better to
align the solution. Of course, we can wait with it. As I understand
META is an experimental feature.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

08/11/2019 14:27, Andrew Rybchenko:
> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
> > 08/11/2019 13:00, Andrew Rybchenko:
> >> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
> >>> 08/11/2019 11:42, Andrew Rybchenko:
> >>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> >>>>> 08/11/2019 09:35, Andrew Rybchenko:
> >>>>>> The problem:
> >>>>>> ~~~~~~~~~~~~
> >>>>>> PMD wants to know before port start if application wants to
> >>>>>> to use flow MARK/FLAG in the future. It is required because:
> >>>>>>
> >>>>>> 1. HW may be configured in a different way to reserve resources
> >>>>>>    for MARK/FLAG delivery
> >>>>>>
> >>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>>>>>    is faster, but does not support MARK)
> >>>>> Thank you for the clear problem statement.
> >>>>> I agree with it. This is a real design issue.
> >>>>>
> >>>>>
> >>>>>> Discussed solutions:
> >>>>>> ~~~~~~~~~~~~~~~~~~~~
> >>>> May be it is not 100% clear since below are alternatives.
> >>>>
> >>>>>> A. Explicit Rx offload suggested by the patch.
> >>>>>>
> >>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> >>>>>>
> >>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
> >>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
> >>>>>>    of (B) to discover if a feature is supported.
> >>>>> The dynamic field should be registered via a new API function
> >>>>> named '<feature>_init'.
> >>>>> It means the application must explicit request the feature.
> >>>>> I agree this is the way to go.
> >>>> If I understand your statement correctly, but (C) is not ideal since it
> >>>> looks global. If registered dynamic field of mbuf and is flag that
> >>>> the feature should be enabled, it is a flag to all ports/devices.
> >>>>
> >>>>>> All solutions require changes in applications which use these
> >>>>>> features. There is a deprecation notice in place which advertises
> >>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
> >>>>>> it with solution (B) or (C). Solution (C) requires changes since
> >>>>>> it should be combined with (B) in order to understand if
> >>>>>> the feature is supported.
> >>>>> I don't understand.
> >>>>> Application request and PMD support are two different things.
> >>>>> PMD support must be via rte_flow validation on a case by case anyway.
> >>>> I mean that application wants to understand if the feature is
> >>>> supported. Then, it wants to enable it. In the case of (B),
> >>>> if I understand the solution correctly, there is no explicit
> >>>> way to enable, PMD just detects it because of discovery is done
> >>>> (that's what I mean by "implicit" and it is a drawback from my
> >>>> point of view, but still could be considered). (C) solves the
> >>>> problem of (B).
> >>>>
> >>>>>> Advantages and drawbacks of solutions:
> >>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>>>>> 1. The main drawback of (A) is a "duplication" since we already
> >>>>>>    have a way to request flow MARK using rte_flow API.
> >>>>>>    I don't fully agree that it is a duplication, but I agree
> >>>>>>    that it sounds like duplication and complicates a bit flow
> >>>>>>    MARK usage by applications. (B) complicates it as well.
> >>>>>>
> >>>>>> 2. One more drawback of the solution (A) is the necessity of
> >>>>>>    similar solution for META and it eats one more offload bit.
> >>>>>>    Yes, that's true and I think it is not a problem.
> >>>>>>    It would make it easier for applications to find out if
> >>>>>>    either MARK or META is supported.
> >>>>>>
> >>>>>> 3. The main advantage of the solution (A) is simplicity.
> >>>>>>    It is simple for application to understand if it supported.
> >>>>>>    It is simple in PMD to understand that it is required.
> >>>>>>    It is simple to disable it - just reconfigure.
> >>>>>>    Also it is easier to document it - just mention that
> >>>>>>    the offload should be supported and enabled.
> >>>>>>
> >>>>>> 4. The main advantage of the solution (B) is no "duplication".
> >>>>>>    I agree that it is valid argument. Solving the problem
> >>>>>>    without extra entities is always nice, but unfortunately
> >>>>>>    it is too complex in this case.
> >>>>>>
> >>>>>> 5. The main drawback of the solution (B) is the complexity.
> >>>>>>    It is necessary to choose a flow rule which should be used
> >>>>>>    as a criteria. It could be hardware dependent.
> >>>>>>    Complex logic is require in PMD if it wants to address the
> >>>>>>    problem and control MARK delivery based on validated flow
> >>>>>>    rules. It adds dependency between start/stop processing and
> >>>>>>    flow rules validation code.
> >>>>>>    It is pretty complicated to document it.
> >>>>>>
> >>>>>> 6. Useless enabling of the offload in the case of solution (A)
> >>>>>>    if really used flow rules do not support MARK looks like
> >>>>>>    drawback as well, but easily mitigated by a combination
> >>>>>>    with solution (B) and only required if the application wants
> >>>>>>    to dive in the level of optimization and complexity and
> >>>>>>    makes sense if application knows required flow rules in
> >>>>>>    advance. So, it is not a problem in this case.
> >>>>>>
> >>>>>> 7. Solution (C) has drawbacks of the solution (B) for
> >>>>>>    applications to understand if these features are supported,
> >>>>>>    but no drawbacks in PMD, since explicit criteria is used to
> >>>>>>    enable/disable (dynamic field/flag lookup).
> >>>>>>
> >>>>>> 8. Solution (C) is nice since it avoids "duplication".
> >>>>>>
> >>>>>> 9. The main drawback of the solution (C) is asymmetry.
> >>>>>>    As it was discussed in the case of RX_TIMESTAMP
> >>>>>>    (if I remember it correctly):
> >>>>>>     - PMD advertises RX_TIMESTAMP offload capability
> >>>>>>     - application enables the offload
> >>>>>>     - PMD registers dynamic field for timestamp
> >>>>>>    Solution (C):
> >>>>>>      - PMD advertises nothing
> >>>>>>      - application uses solution (B) to understand if
> >>>>>>        these features are supported
> >>>>>>      - application registers dynamic field/flag
> >>>>>>      - PMD does lookup and solve the problem
> >>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> >>>>>>    solution is changed to require an application to register
> >>>>>>    dynamic fields and PMD to do lookup if the offload is
> >>>>>>    enabled. So, the only difference will be in no offload
> >>>>>>    in the case of flow MARK/FLAG and usage of complex logic
> >>>>>>    to understand if it is supported or no.
> >>>>>>    May be it would be really good since it will allow to
> >>>>>>    have dynamic fields registered before mempool population.
> >>>>>>
> >>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
> >>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
> >>>>>>     per queue. Moreover (C) looks global - for all devices.
> >>>>>>     It could be really painful.
> >>>>>>
> >>>>>> (C) is nice, but I still vote for simplicity and
> >>>>>> granularity of (A).
> >>>>> I vote for clear separation of application needs and PMD support,
> >>>>> by using the method C (dynamic fields).
> >>>>> I agree timestamp must use the same path.
> >>>>> I agree it's complicate because we don't know in advance whether
> >>>>> a flow rule will be accepted, but that's the reality, config is complex.
> >>>> Do you think that global nature of the (C) is acceptable?
> >>> That's a good question.
> >>> Maybe the feature request should be per port.
> >>> In this case, we are back to solution A with a flag per port?
> >> Offloads are natively per-queue as well, so (A) keeps the choice
> >> between per-port vs per-queue to PMDs as usual.
> >>
> >>> Note that A and C will not guarantee that the offload will be possible.
> >> Yes, definitely.
> >>
> >>> We need B (flow rule validation) anyway.
> >> Strictly speaking (B) (checking flow rules before device
> >> startup) is required if an application can predict flow
> >> rules and wants to ensure that MARK offload will be usable.
> >> Otherwise, it may be skipped.
> > No no, I mean flow rule validation MUST be used anyway
> > during the runtime before applying a rule.
> > I agree it is hard to predict. I speak only about real rules.
> 
> OK, I see. Of course, flow rule validation is required at runtime.
> I was rather concentrated on the stated problem solutions.
> 
> >>> It seems A, B, C are not alternatives but all required
> >>> as pieces of a puzzle...
> >> Unfortunately true in the most complex case.
> >> Right now it will be A with B if required as explained above.
> >> C will come a bit later when the field migrates to dynamic.
> >>
> >> May be it is even better if application registers dynamic
> >> fields before an attempt to enable offload to be sure that
> >> it will not fail because of impossibility to register
> >> dynamic field (lack of space). I'm not sure, but it is not
> >> not that important.
> > Yes of course, lack of mbuf space is another reason for
> > disabling the feature.
> >
> >> If we finally go way A, should we add offloads for META back?
> >> I guess separate Rx and Tx are required.
> > I would prefer to add it as dynamic flags.
> > Why rushing on a very temporary solution while it is not a new issue?
> 
> Basically it means that we go just (B)+(C) in the case of META.
> I have no strong opinion but thought that it could be better to
> align the solution. Of course, we can wait with it. As I understand
> META is an experimental feature.

Yes it is experimental and I think it is too late to align now.

Anyway, we will probably to discuss again these offloads TAG/MARK/META,
as requested by several people.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/8/19 4:30 PM, Thomas Monjalon wrote:
> 08/11/2019 14:27, Andrew Rybchenko:
>> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
>>> 08/11/2019 13:00, Andrew Rybchenko:
>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
>>>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>>>> The problem:
>>>>>>>> ~~~~~~~~~~~~
>>>>>>>> PMD wants to know before port start if application wants to
>>>>>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>>>>>
>>>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>>>    for MARK/FLAG delivery
>>>>>>>>
>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>>>    is faster, but does not support MARK)
>>>>>>> Thank you for the clear problem statement.
>>>>>>> I agree with it. This is a real design issue.
>>>>>>>
>>>>>>>
>>>>>>>> Discussed solutions:
>>>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>>>> May be it is not 100% clear since below are alternatives.
>>>>>>
>>>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>>>
>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>>>
>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>>>    of (B) to discover if a feature is supported.
>>>>>>> The dynamic field should be registered via a new API function
>>>>>>> named '<feature>_init'.
>>>>>>> It means the application must explicit request the feature.
>>>>>>> I agree this is the way to go.
>>>>>> If I understand your statement correctly, but (C) is not ideal since it
>>>>>> looks global. If registered dynamic field of mbuf and is flag that
>>>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>>>
>>>>>>>> All solutions require changes in applications which use these
>>>>>>>> features. There is a deprecation notice in place which advertises
>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>>>>>> it should be combined with (B) in order to understand if
>>>>>>>> the feature is supported.
>>>>>>> I don't understand.
>>>>>>> Application request and PMD support are two different things.
>>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>>>> I mean that application wants to understand if the feature is
>>>>>> supported. Then, it wants to enable it. In the case of (B),
>>>>>> if I understand the solution correctly, there is no explicit
>>>>>> way to enable, PMD just detects it because of discovery is done
>>>>>> (that's what I mean by "implicit" and it is a drawback from my
>>>>>> point of view, but still could be considered). (C) solves the
>>>>>> problem of (B).
>>>>>>
>>>>>>>> Advantages and drawbacks of solutions:
>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>>>    have a way to request flow MARK using rte_flow API.
>>>>>>>>    I don't fully agree that it is a duplication, but I agree
>>>>>>>>    that it sounds like duplication and complicates a bit flow
>>>>>>>>    MARK usage by applications. (B) complicates it as well.
>>>>>>>>
>>>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>>>    similar solution for META and it eats one more offload bit.
>>>>>>>>    Yes, that's true and I think it is not a problem.
>>>>>>>>    It would make it easier for applications to find out if
>>>>>>>>    either MARK or META is supported.
>>>>>>>>
>>>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>>>    It is simple for application to understand if it supported.
>>>>>>>>    It is simple in PMD to understand that it is required.
>>>>>>>>    It is simple to disable it - just reconfigure.
>>>>>>>>    Also it is easier to document it - just mention that
>>>>>>>>    the offload should be supported and enabled.
>>>>>>>>
>>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>>>    I agree that it is valid argument. Solving the problem
>>>>>>>>    without extra entities is always nice, but unfortunately
>>>>>>>>    it is too complex in this case.
>>>>>>>>
>>>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>>>    It is necessary to choose a flow rule which should be used
>>>>>>>>    as a criteria. It could be hardware dependent.
>>>>>>>>    Complex logic is require in PMD if it wants to address the
>>>>>>>>    problem and control MARK delivery based on validated flow
>>>>>>>>    rules. It adds dependency between start/stop processing and
>>>>>>>>    flow rules validation code.
>>>>>>>>    It is pretty complicated to document it.
>>>>>>>>
>>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>>>    if really used flow rules do not support MARK looks like
>>>>>>>>    drawback as well, but easily mitigated by a combination
>>>>>>>>    with solution (B) and only required if the application wants
>>>>>>>>    to dive in the level of optimization and complexity and
>>>>>>>>    makes sense if application knows required flow rules in
>>>>>>>>    advance. So, it is not a problem in this case.
>>>>>>>>
>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>>>    applications to understand if these features are supported,
>>>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
>>>>>>>>    enable/disable (dynamic field/flag lookup).
>>>>>>>>
>>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>>>
>>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>>>    As it was discussed in the case of RX_TIMESTAMP
>>>>>>>>    (if I remember it correctly):
>>>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
>>>>>>>>     - application enables the offload
>>>>>>>>     - PMD registers dynamic field for timestamp
>>>>>>>>    Solution (C):
>>>>>>>>      - PMD advertises nothing
>>>>>>>>      - application uses solution (B) to understand if
>>>>>>>>        these features are supported
>>>>>>>>      - application registers dynamic field/flag
>>>>>>>>      - PMD does lookup and solve the problem
>>>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>>>    solution is changed to require an application to register
>>>>>>>>    dynamic fields and PMD to do lookup if the offload is
>>>>>>>>    enabled. So, the only difference will be in no offload
>>>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
>>>>>>>>    to understand if it is supported or no.
>>>>>>>>    May be it would be really good since it will allow to
>>>>>>>>    have dynamic fields registered before mempool population.
>>>>>>>>
>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>>>     per queue. Moreover (C) looks global - for all devices.
>>>>>>>>     It could be really painful.
>>>>>>>>
>>>>>>>> (C) is nice, but I still vote for simplicity and
>>>>>>>> granularity of (A).
>>>>>>> I vote for clear separation of application needs and PMD support,
>>>>>>> by using the method C (dynamic fields).
>>>>>>> I agree timestamp must use the same path.
>>>>>>> I agree it's complicate because we don't know in advance whether
>>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
>>>>>> Do you think that global nature of the (C) is acceptable?
>>>>> That's a good question.
>>>>> Maybe the feature request should be per port.
>>>>> In this case, we are back to solution A with a flag per port?
>>>> Offloads are natively per-queue as well, so (A) keeps the choice
>>>> between per-port vs per-queue to PMDs as usual.
>>>>
>>>>> Note that A and C will not guarantee that the offload will be possible.
>>>> Yes, definitely.
>>>>
>>>>> We need B (flow rule validation) anyway.
>>>> Strictly speaking (B) (checking flow rules before device
>>>> startup) is required if an application can predict flow
>>>> rules and wants to ensure that MARK offload will be usable.
>>>> Otherwise, it may be skipped.
>>> No no, I mean flow rule validation MUST be used anyway
>>> during the runtime before applying a rule.
>>> I agree it is hard to predict. I speak only about real rules.
>>
>> OK, I see. Of course, flow rule validation is required at runtime.
>> I was rather concentrated on the stated problem solutions.
>>
>>>>> It seems A, B, C are not alternatives but all required
>>>>> as pieces of a puzzle...
>>>> Unfortunately true in the most complex case.
>>>> Right now it will be A with B if required as explained above.
>>>> C will come a bit later when the field migrates to dynamic.
>>>>
>>>> May be it is even better if application registers dynamic
>>>> fields before an attempt to enable offload to be sure that
>>>> it will not fail because of impossibility to register
>>>> dynamic field (lack of space). I'm not sure, but it is not
>>>> not that important.
>>> Yes of course, lack of mbuf space is another reason for
>>> disabling the feature.
>>>
>>>> If we finally go way A, should we add offloads for META back?
>>>> I guess separate Rx and Tx are required.
>>> I would prefer to add it as dynamic flags.
>>> Why rushing on a very temporary solution while it is not a new issue?
>>
>> Basically it means that we go just (B)+(C) in the case of META.
>> I have no strong opinion but thought that it could be better to
>> align the solution. Of course, we can wait with it. As I understand
>> META is an experimental feature.
> 
> Yes it is experimental and I think it is too late to align now.
> 
> Anyway, we will probably to discuss again these offloads TAG/MARK/META,
> as requested by several people.
> 

The series implements (A) to help to solve the problem described above.
What is the fate of the series in v19.11 in accordance with the
discussion?

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

19/11/2019 10:24, Andrew Rybchenko:
> On 11/8/19 4:30 PM, Thomas Monjalon wrote:
> > 08/11/2019 14:27, Andrew Rybchenko:
> >> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
> >>> 08/11/2019 13:00, Andrew Rybchenko:
> >>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
> >>>>> 08/11/2019 11:42, Andrew Rybchenko:
> >>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> >>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
> >>>>>>>> The problem:
> >>>>>>>> ~~~~~~~~~~~~
> >>>>>>>> PMD wants to know before port start if application wants to
> >>>>>>>> to use flow MARK/FLAG in the future. It is required because:
> >>>>>>>>
> >>>>>>>> 1. HW may be configured in a different way to reserve resources
> >>>>>>>>    for MARK/FLAG delivery
> >>>>>>>>
> >>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>>>>>>>    is faster, but does not support MARK)
> >>>>>>> Thank you for the clear problem statement.
> >>>>>>> I agree with it. This is a real design issue.
> >>>>>>>
> >>>>>>>
> >>>>>>>> Discussed solutions:
> >>>>>>>> ~~~~~~~~~~~~~~~~~~~~
> >>>>>> May be it is not 100% clear since below are alternatives.
> >>>>>>
> >>>>>>>> A. Explicit Rx offload suggested by the patch.
> >>>>>>>>
> >>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> >>>>>>>>
> >>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
> >>>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
> >>>>>>>>    of (B) to discover if a feature is supported.
> >>>>>>> The dynamic field should be registered via a new API function
> >>>>>>> named '<feature>_init'.
> >>>>>>> It means the application must explicit request the feature.
> >>>>>>> I agree this is the way to go.
> >>>>>> If I understand your statement correctly, but (C) is not ideal since it
> >>>>>> looks global. If registered dynamic field of mbuf and is flag that
> >>>>>> the feature should be enabled, it is a flag to all ports/devices.
> >>>>>>
> >>>>>>>> All solutions require changes in applications which use these
> >>>>>>>> features. There is a deprecation notice in place which advertises
> >>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
> >>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
> >>>>>>>> it should be combined with (B) in order to understand if
> >>>>>>>> the feature is supported.
> >>>>>>> I don't understand.
> >>>>>>> Application request and PMD support are two different things.
> >>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
> >>>>>> I mean that application wants to understand if the feature is
> >>>>>> supported. Then, it wants to enable it. In the case of (B),
> >>>>>> if I understand the solution correctly, there is no explicit
> >>>>>> way to enable, PMD just detects it because of discovery is done
> >>>>>> (that's what I mean by "implicit" and it is a drawback from my
> >>>>>> point of view, but still could be considered). (C) solves the
> >>>>>> problem of (B).
> >>>>>>
> >>>>>>>> Advantages and drawbacks of solutions:
> >>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
> >>>>>>>>    have a way to request flow MARK using rte_flow API.
> >>>>>>>>    I don't fully agree that it is a duplication, but I agree
> >>>>>>>>    that it sounds like duplication and complicates a bit flow
> >>>>>>>>    MARK usage by applications. (B) complicates it as well.
> >>>>>>>>
> >>>>>>>> 2. One more drawback of the solution (A) is the necessity of
> >>>>>>>>    similar solution for META and it eats one more offload bit.
> >>>>>>>>    Yes, that's true and I think it is not a problem.
> >>>>>>>>    It would make it easier for applications to find out if
> >>>>>>>>    either MARK or META is supported.
> >>>>>>>>
> >>>>>>>> 3. The main advantage of the solution (A) is simplicity.
> >>>>>>>>    It is simple for application to understand if it supported.
> >>>>>>>>    It is simple in PMD to understand that it is required.
> >>>>>>>>    It is simple to disable it - just reconfigure.
> >>>>>>>>    Also it is easier to document it - just mention that
> >>>>>>>>    the offload should be supported and enabled.
> >>>>>>>>
> >>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
> >>>>>>>>    I agree that it is valid argument. Solving the problem
> >>>>>>>>    without extra entities is always nice, but unfortunately
> >>>>>>>>    it is too complex in this case.
> >>>>>>>>
> >>>>>>>> 5. The main drawback of the solution (B) is the complexity.
> >>>>>>>>    It is necessary to choose a flow rule which should be used
> >>>>>>>>    as a criteria. It could be hardware dependent.
> >>>>>>>>    Complex logic is require in PMD if it wants to address the
> >>>>>>>>    problem and control MARK delivery based on validated flow
> >>>>>>>>    rules. It adds dependency between start/stop processing and
> >>>>>>>>    flow rules validation code.
> >>>>>>>>    It is pretty complicated to document it.
> >>>>>>>>
> >>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
> >>>>>>>>    if really used flow rules do not support MARK looks like
> >>>>>>>>    drawback as well, but easily mitigated by a combination
> >>>>>>>>    with solution (B) and only required if the application wants
> >>>>>>>>    to dive in the level of optimization and complexity and
> >>>>>>>>    makes sense if application knows required flow rules in
> >>>>>>>>    advance. So, it is not a problem in this case.
> >>>>>>>>
> >>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
> >>>>>>>>    applications to understand if these features are supported,
> >>>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
> >>>>>>>>    enable/disable (dynamic field/flag lookup).
> >>>>>>>>
> >>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
> >>>>>>>>
> >>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
> >>>>>>>>    As it was discussed in the case of RX_TIMESTAMP
> >>>>>>>>    (if I remember it correctly):
> >>>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
> >>>>>>>>     - application enables the offload
> >>>>>>>>     - PMD registers dynamic field for timestamp
> >>>>>>>>    Solution (C):
> >>>>>>>>      - PMD advertises nothing
> >>>>>>>>      - application uses solution (B) to understand if
> >>>>>>>>        these features are supported
> >>>>>>>>      - application registers dynamic field/flag
> >>>>>>>>      - PMD does lookup and solve the problem
> >>>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> >>>>>>>>    solution is changed to require an application to register
> >>>>>>>>    dynamic fields and PMD to do lookup if the offload is
> >>>>>>>>    enabled. So, the only difference will be in no offload
> >>>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
> >>>>>>>>    to understand if it is supported or no.
> >>>>>>>>    May be it would be really good since it will allow to
> >>>>>>>>    have dynamic fields registered before mempool population.
> >>>>>>>>
> >>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
> >>>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
> >>>>>>>>     per queue. Moreover (C) looks global - for all devices.
> >>>>>>>>     It could be really painful.
> >>>>>>>>
> >>>>>>>> (C) is nice, but I still vote for simplicity and
> >>>>>>>> granularity of (A).
> >>>>>>> I vote for clear separation of application needs and PMD support,
> >>>>>>> by using the method C (dynamic fields).
> >>>>>>> I agree timestamp must use the same path.
> >>>>>>> I agree it's complicate because we don't know in advance whether
> >>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
> >>>>>> Do you think that global nature of the (C) is acceptable?
> >>>>> That's a good question.
> >>>>> Maybe the feature request should be per port.
> >>>>> In this case, we are back to solution A with a flag per port?
> >>>> Offloads are natively per-queue as well, so (A) keeps the choice
> >>>> between per-port vs per-queue to PMDs as usual.
> >>>>
> >>>>> Note that A and C will not guarantee that the offload will be possible.
> >>>> Yes, definitely.
> >>>>
> >>>>> We need B (flow rule validation) anyway.
> >>>> Strictly speaking (B) (checking flow rules before device
> >>>> startup) is required if an application can predict flow
> >>>> rules and wants to ensure that MARK offload will be usable.
> >>>> Otherwise, it may be skipped.
> >>> No no, I mean flow rule validation MUST be used anyway
> >>> during the runtime before applying a rule.
> >>> I agree it is hard to predict. I speak only about real rules.
> >>
> >> OK, I see. Of course, flow rule validation is required at runtime.
> >> I was rather concentrated on the stated problem solutions.
> >>
> >>>>> It seems A, B, C are not alternatives but all required
> >>>>> as pieces of a puzzle...
> >>>> Unfortunately true in the most complex case.
> >>>> Right now it will be A with B if required as explained above.
> >>>> C will come a bit later when the field migrates to dynamic.
> >>>>
> >>>> May be it is even better if application registers dynamic
> >>>> fields before an attempt to enable offload to be sure that
> >>>> it will not fail because of impossibility to register
> >>>> dynamic field (lack of space). I'm not sure, but it is not
> >>>> not that important.
> >>> Yes of course, lack of mbuf space is another reason for
> >>> disabling the feature.
> >>>
> >>>> If we finally go way A, should we add offloads for META back?
> >>>> I guess separate Rx and Tx are required.
> >>> I would prefer to add it as dynamic flags.
> >>> Why rushing on a very temporary solution while it is not a new issue?
> >>
> >> Basically it means that we go just (B)+(C) in the case of META.
> >> I have no strong opinion but thought that it could be better to
> >> align the solution. Of course, we can wait with it. As I understand
> >> META is an experimental feature.
> > 
> > Yes it is experimental and I think it is too late to align now.
> > 
> > Anyway, we will probably to discuss again these offloads TAG/MARK/META,
> > as requested by several people.
> > 
> 
> The series implements (A) to help to solve the problem described above.
> What is the fate of the series in v19.11 in accordance with the
> discussion?

I am against adding anything related to a feature union'ed in mbuf.
The feature must move to dynamic field first.

In addition, such capability is very weak.
I am not sure it is a good idea to have some weak capabilities,
meaning a feature could be available but not in all cases.
I think we should discuss more generally how we want to handle
the rte_flow capabilities conveniently and reliably.

So regarding 19.11, as this feature is not new, it can wait 20.02.

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Andrew Rybchenko]

On 11/19/19 12:50 PM, Thomas Monjalon wrote:
> 19/11/2019 10:24, Andrew Rybchenko:
>> On 11/8/19 4:30 PM, Thomas Monjalon wrote:
>>> 08/11/2019 14:27, Andrew Rybchenko:
>>>> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
>>>>> 08/11/2019 13:00, Andrew Rybchenko:
>>>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
>>>>>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>>>>>> The problem:
>>>>>>>>>> ~~~~~~~~~~~~
>>>>>>>>>> PMD wants to know before port start if application wants to
>>>>>>>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>>>>>>>
>>>>>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>>>>>    for MARK/FLAG delivery
>>>>>>>>>>
>>>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>>>>>    is faster, but does not support MARK)
>>>>>>>>> Thank you for the clear problem statement.
>>>>>>>>> I agree with it. This is a real design issue.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Discussed solutions:
>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>>>>>> May be it is not 100% clear since below are alternatives.
>>>>>>>>
>>>>>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>>>>>
>>>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>>>>>
>>>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>>>>>    of (B) to discover if a feature is supported.
>>>>>>>>> The dynamic field should be registered via a new API function
>>>>>>>>> named '<feature>_init'.
>>>>>>>>> It means the application must explicit request the feature.
>>>>>>>>> I agree this is the way to go.
>>>>>>>> If I understand your statement correctly, but (C) is not ideal since it
>>>>>>>> looks global. If registered dynamic field of mbuf and is flag that
>>>>>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>>>>>
>>>>>>>>>> All solutions require changes in applications which use these
>>>>>>>>>> features. There is a deprecation notice in place which advertises
>>>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>>>>>>>> it should be combined with (B) in order to understand if
>>>>>>>>>> the feature is supported.
>>>>>>>>> I don't understand.
>>>>>>>>> Application request and PMD support are two different things.
>>>>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>>>>>> I mean that application wants to understand if the feature is
>>>>>>>> supported. Then, it wants to enable it. In the case of (B),
>>>>>>>> if I understand the solution correctly, there is no explicit
>>>>>>>> way to enable, PMD just detects it because of discovery is done
>>>>>>>> (that's what I mean by "implicit" and it is a drawback from my
>>>>>>>> point of view, but still could be considered). (C) solves the
>>>>>>>> problem of (B).
>>>>>>>>
>>>>>>>>>> Advantages and drawbacks of solutions:
>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>>>>>    have a way to request flow MARK using rte_flow API.
>>>>>>>>>>    I don't fully agree that it is a duplication, but I agree
>>>>>>>>>>    that it sounds like duplication and complicates a bit flow
>>>>>>>>>>    MARK usage by applications. (B) complicates it as well.
>>>>>>>>>>
>>>>>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>>>>>    similar solution for META and it eats one more offload bit.
>>>>>>>>>>    Yes, that's true and I think it is not a problem.
>>>>>>>>>>    It would make it easier for applications to find out if
>>>>>>>>>>    either MARK or META is supported.
>>>>>>>>>>
>>>>>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>>>>>    It is simple for application to understand if it supported.
>>>>>>>>>>    It is simple in PMD to understand that it is required.
>>>>>>>>>>    It is simple to disable it - just reconfigure.
>>>>>>>>>>    Also it is easier to document it - just mention that
>>>>>>>>>>    the offload should be supported and enabled.
>>>>>>>>>>
>>>>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>>>>>    I agree that it is valid argument. Solving the problem
>>>>>>>>>>    without extra entities is always nice, but unfortunately
>>>>>>>>>>    it is too complex in this case.
>>>>>>>>>>
>>>>>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>>>>>    It is necessary to choose a flow rule which should be used
>>>>>>>>>>    as a criteria. It could be hardware dependent.
>>>>>>>>>>    Complex logic is require in PMD if it wants to address the
>>>>>>>>>>    problem and control MARK delivery based on validated flow
>>>>>>>>>>    rules. It adds dependency between start/stop processing and
>>>>>>>>>>    flow rules validation code.
>>>>>>>>>>    It is pretty complicated to document it.
>>>>>>>>>>
>>>>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>>>>>    if really used flow rules do not support MARK looks like
>>>>>>>>>>    drawback as well, but easily mitigated by a combination
>>>>>>>>>>    with solution (B) and only required if the application wants
>>>>>>>>>>    to dive in the level of optimization and complexity and
>>>>>>>>>>    makes sense if application knows required flow rules in
>>>>>>>>>>    advance. So, it is not a problem in this case.
>>>>>>>>>>
>>>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>>>>>    applications to understand if these features are supported,
>>>>>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
>>>>>>>>>>    enable/disable (dynamic field/flag lookup).
>>>>>>>>>>
>>>>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>>>>>
>>>>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>>>>>    As it was discussed in the case of RX_TIMESTAMP
>>>>>>>>>>    (if I remember it correctly):
>>>>>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
>>>>>>>>>>     - application enables the offload
>>>>>>>>>>     - PMD registers dynamic field for timestamp
>>>>>>>>>>    Solution (C):
>>>>>>>>>>      - PMD advertises nothing
>>>>>>>>>>      - application uses solution (B) to understand if
>>>>>>>>>>        these features are supported
>>>>>>>>>>      - application registers dynamic field/flag
>>>>>>>>>>      - PMD does lookup and solve the problem
>>>>>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>>>>>    solution is changed to require an application to register
>>>>>>>>>>    dynamic fields and PMD to do lookup if the offload is
>>>>>>>>>>    enabled. So, the only difference will be in no offload
>>>>>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
>>>>>>>>>>    to understand if it is supported or no.
>>>>>>>>>>    May be it would be really good since it will allow to
>>>>>>>>>>    have dynamic fields registered before mempool population.
>>>>>>>>>>
>>>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>>>>>     per queue. Moreover (C) looks global - for all devices.
>>>>>>>>>>     It could be really painful.
>>>>>>>>>>
>>>>>>>>>> (C) is nice, but I still vote for simplicity and
>>>>>>>>>> granularity of (A).
>>>>>>>>> I vote for clear separation of application needs and PMD support,
>>>>>>>>> by using the method C (dynamic fields).
>>>>>>>>> I agree timestamp must use the same path.
>>>>>>>>> I agree it's complicate because we don't know in advance whether
>>>>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
>>>>>>>> Do you think that global nature of the (C) is acceptable?
>>>>>>> That's a good question.
>>>>>>> Maybe the feature request should be per port.
>>>>>>> In this case, we are back to solution A with a flag per port?
>>>>>> Offloads are natively per-queue as well, so (A) keeps the choice
>>>>>> between per-port vs per-queue to PMDs as usual.
>>>>>>
>>>>>>> Note that A and C will not guarantee that the offload will be possible.
>>>>>> Yes, definitely.
>>>>>>
>>>>>>> We need B (flow rule validation) anyway.
>>>>>> Strictly speaking (B) (checking flow rules before device
>>>>>> startup) is required if an application can predict flow
>>>>>> rules and wants to ensure that MARK offload will be usable.
>>>>>> Otherwise, it may be skipped.
>>>>> No no, I mean flow rule validation MUST be used anyway
>>>>> during the runtime before applying a rule.
>>>>> I agree it is hard to predict. I speak only about real rules.
>>>> OK, I see. Of course, flow rule validation is required at runtime.
>>>> I was rather concentrated on the stated problem solutions.
>>>>
>>>>>>> It seems A, B, C are not alternatives but all required
>>>>>>> as pieces of a puzzle...
>>>>>> Unfortunately true in the most complex case.
>>>>>> Right now it will be A with B if required as explained above.
>>>>>> C will come a bit later when the field migrates to dynamic.
>>>>>>
>>>>>> May be it is even better if application registers dynamic
>>>>>> fields before an attempt to enable offload to be sure that
>>>>>> it will not fail because of impossibility to register
>>>>>> dynamic field (lack of space). I'm not sure, but it is not
>>>>>> not that important.
>>>>> Yes of course, lack of mbuf space is another reason for
>>>>> disabling the feature.
>>>>>
>>>>>> If we finally go way A, should we add offloads for META back?
>>>>>> I guess separate Rx and Tx are required.
>>>>> I would prefer to add it as dynamic flags.
>>>>> Why rushing on a very temporary solution while it is not a new issue?
>>>> Basically it means that we go just (B)+(C) in the case of META.
>>>> I have no strong opinion but thought that it could be better to
>>>> align the solution. Of course, we can wait with it. As I understand
>>>> META is an experimental feature.
>>> Yes it is experimental and I think it is too late to align now.
>>>
>>> Anyway, we will probably to discuss again these offloads TAG/MARK/META,
>>> as requested by several people.
>>>
>> The series implements (A) to help to solve the problem described above.
>> What is the fate of the series in v19.11 in accordance with the
>> discussion?
> I am against adding anything related to a feature union'ed in mbuf.
> The feature must move to dynamic field first.
>
> In addition, such capability is very weak.
> I am not sure it is a good idea to have some weak capabilities,
> meaning a feature could be available but not in all cases.
> I think we should discuss more generally how we want to handle
> the rte_flow capabilities conveniently and reliably.

It is really unexpected outcome from the above discussion.
It is just possibility to deliver and handle marks on datapath and
request to have it. It says almost nothing about rte_flow rules
supported etc. I'll be happy to take part in the discussion.

> So regarding 19.11, as this feature is not new, it can wait 20.02.

OK, it is not critical for me, so I don't mind, however, I've seen
patches which try to use it [1] except net/octeontx2 in the second
patch of the series.

[1] https://patches.dpdk.org/patch/62415/

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

19/11/2019 11:59, Andrew Rybchenko:
> On 11/19/19 12:50 PM, Thomas Monjalon wrote:
> > 19/11/2019 10:24, Andrew Rybchenko:
> >> On 11/8/19 4:30 PM, Thomas Monjalon wrote:
> >>> 08/11/2019 14:27, Andrew Rybchenko:
> >>>> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
> >>>>> 08/11/2019 13:00, Andrew Rybchenko:
> >>>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
> >>>>>>> 08/11/2019 11:42, Andrew Rybchenko:
> >>>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> >>>>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
> >>>>>>>>>> The problem:
> >>>>>>>>>> ~~~~~~~~~~~~
> >>>>>>>>>> PMD wants to know before port start if application wants to
> >>>>>>>>>> to use flow MARK/FLAG in the future. It is required because:
> >>>>>>>>>>
> >>>>>>>>>> 1. HW may be configured in a different way to reserve resources
> >>>>>>>>>>    for MARK/FLAG delivery
> >>>>>>>>>>
> >>>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>>>>>>>>>    is faster, but does not support MARK)
> >>>>>>>>> Thank you for the clear problem statement.
> >>>>>>>>> I agree with it. This is a real design issue.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Discussed solutions:
> >>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~
> >>>>>>>> May be it is not 100% clear since below are alternatives.
> >>>>>>>>
> >>>>>>>>>> A. Explicit Rx offload suggested by the patch.
> >>>>>>>>>>
> >>>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> >>>>>>>>>>
> >>>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
> >>>>>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
> >>>>>>>>>>    of (B) to discover if a feature is supported.
> >>>>>>>>> The dynamic field should be registered via a new API function
> >>>>>>>>> named '<feature>_init'.
> >>>>>>>>> It means the application must explicit request the feature.
> >>>>>>>>> I agree this is the way to go.
> >>>>>>>> If I understand your statement correctly, but (C) is not ideal since it
> >>>>>>>> looks global. If registered dynamic field of mbuf and is flag that
> >>>>>>>> the feature should be enabled, it is a flag to all ports/devices.
> >>>>>>>>
> >>>>>>>>>> All solutions require changes in applications which use these
> >>>>>>>>>> features. There is a deprecation notice in place which advertises
> >>>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
> >>>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
> >>>>>>>>>> it should be combined with (B) in order to understand if
> >>>>>>>>>> the feature is supported.
> >>>>>>>>> I don't understand.
> >>>>>>>>> Application request and PMD support are two different things.
> >>>>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
> >>>>>>>> I mean that application wants to understand if the feature is
> >>>>>>>> supported. Then, it wants to enable it. In the case of (B),
> >>>>>>>> if I understand the solution correctly, there is no explicit
> >>>>>>>> way to enable, PMD just detects it because of discovery is done
> >>>>>>>> (that's what I mean by "implicit" and it is a drawback from my
> >>>>>>>> point of view, but still could be considered). (C) solves the
> >>>>>>>> problem of (B).
> >>>>>>>>
> >>>>>>>>>> Advantages and drawbacks of solutions:
> >>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
> >>>>>>>>>>    have a way to request flow MARK using rte_flow API.
> >>>>>>>>>>    I don't fully agree that it is a duplication, but I agree
> >>>>>>>>>>    that it sounds like duplication and complicates a bit flow
> >>>>>>>>>>    MARK usage by applications. (B) complicates it as well.
> >>>>>>>>>>
> >>>>>>>>>> 2. One more drawback of the solution (A) is the necessity of
> >>>>>>>>>>    similar solution for META and it eats one more offload bit.
> >>>>>>>>>>    Yes, that's true and I think it is not a problem.
> >>>>>>>>>>    It would make it easier for applications to find out if
> >>>>>>>>>>    either MARK or META is supported.
> >>>>>>>>>>
> >>>>>>>>>> 3. The main advantage of the solution (A) is simplicity.
> >>>>>>>>>>    It is simple for application to understand if it supported.
> >>>>>>>>>>    It is simple in PMD to understand that it is required.
> >>>>>>>>>>    It is simple to disable it - just reconfigure.
> >>>>>>>>>>    Also it is easier to document it - just mention that
> >>>>>>>>>>    the offload should be supported and enabled.
> >>>>>>>>>>
> >>>>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
> >>>>>>>>>>    I agree that it is valid argument. Solving the problem
> >>>>>>>>>>    without extra entities is always nice, but unfortunately
> >>>>>>>>>>    it is too complex in this case.
> >>>>>>>>>>
> >>>>>>>>>> 5. The main drawback of the solution (B) is the complexity.
> >>>>>>>>>>    It is necessary to choose a flow rule which should be used
> >>>>>>>>>>    as a criteria. It could be hardware dependent.
> >>>>>>>>>>    Complex logic is require in PMD if it wants to address the
> >>>>>>>>>>    problem and control MARK delivery based on validated flow
> >>>>>>>>>>    rules. It adds dependency between start/stop processing and
> >>>>>>>>>>    flow rules validation code.
> >>>>>>>>>>    It is pretty complicated to document it.
> >>>>>>>>>>
> >>>>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
> >>>>>>>>>>    if really used flow rules do not support MARK looks like
> >>>>>>>>>>    drawback as well, but easily mitigated by a combination
> >>>>>>>>>>    with solution (B) and only required if the application wants
> >>>>>>>>>>    to dive in the level of optimization and complexity and
> >>>>>>>>>>    makes sense if application knows required flow rules in
> >>>>>>>>>>    advance. So, it is not a problem in this case.
> >>>>>>>>>>
> >>>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
> >>>>>>>>>>    applications to understand if these features are supported,
> >>>>>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
> >>>>>>>>>>    enable/disable (dynamic field/flag lookup).
> >>>>>>>>>>
> >>>>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
> >>>>>>>>>>
> >>>>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
> >>>>>>>>>>    As it was discussed in the case of RX_TIMESTAMP
> >>>>>>>>>>    (if I remember it correctly):
> >>>>>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
> >>>>>>>>>>     - application enables the offload
> >>>>>>>>>>     - PMD registers dynamic field for timestamp
> >>>>>>>>>>    Solution (C):
> >>>>>>>>>>      - PMD advertises nothing
> >>>>>>>>>>      - application uses solution (B) to understand if
> >>>>>>>>>>        these features are supported
> >>>>>>>>>>      - application registers dynamic field/flag
> >>>>>>>>>>      - PMD does lookup and solve the problem
> >>>>>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
> >>>>>>>>>>    solution is changed to require an application to register
> >>>>>>>>>>    dynamic fields and PMD to do lookup if the offload is
> >>>>>>>>>>    enabled. So, the only difference will be in no offload
> >>>>>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
> >>>>>>>>>>    to understand if it is supported or no.
> >>>>>>>>>>    May be it would be really good since it will allow to
> >>>>>>>>>>    have dynamic fields registered before mempool population.
> >>>>>>>>>>
> >>>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
> >>>>>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
> >>>>>>>>>>     per queue. Moreover (C) looks global - for all devices.
> >>>>>>>>>>     It could be really painful.
> >>>>>>>>>>
> >>>>>>>>>> (C) is nice, but I still vote for simplicity and
> >>>>>>>>>> granularity of (A).
> >>>>>>>>> I vote for clear separation of application needs and PMD support,
> >>>>>>>>> by using the method C (dynamic fields).
> >>>>>>>>> I agree timestamp must use the same path.
> >>>>>>>>> I agree it's complicate because we don't know in advance whether
> >>>>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
> >>>>>>>> Do you think that global nature of the (C) is acceptable?
> >>>>>>> That's a good question.
> >>>>>>> Maybe the feature request should be per port.
> >>>>>>> In this case, we are back to solution A with a flag per port?
> >>>>>> Offloads are natively per-queue as well, so (A) keeps the choice
> >>>>>> between per-port vs per-queue to PMDs as usual.
> >>>>>>
> >>>>>>> Note that A and C will not guarantee that the offload will be possible.
> >>>>>> Yes, definitely.
> >>>>>>
> >>>>>>> We need B (flow rule validation) anyway.
> >>>>>> Strictly speaking (B) (checking flow rules before device
> >>>>>> startup) is required if an application can predict flow
> >>>>>> rules and wants to ensure that MARK offload will be usable.
> >>>>>> Otherwise, it may be skipped.
> >>>>> No no, I mean flow rule validation MUST be used anyway
> >>>>> during the runtime before applying a rule.
> >>>>> I agree it is hard to predict. I speak only about real rules.
> >>>> OK, I see. Of course, flow rule validation is required at runtime.
> >>>> I was rather concentrated on the stated problem solutions.
> >>>>
> >>>>>>> It seems A, B, C are not alternatives but all required
> >>>>>>> as pieces of a puzzle...
> >>>>>> Unfortunately true in the most complex case.
> >>>>>> Right now it will be A with B if required as explained above.
> >>>>>> C will come a bit later when the field migrates to dynamic.
> >>>>>>
> >>>>>> May be it is even better if application registers dynamic
> >>>>>> fields before an attempt to enable offload to be sure that
> >>>>>> it will not fail because of impossibility to register
> >>>>>> dynamic field (lack of space). I'm not sure, but it is not
> >>>>>> not that important.
> >>>>> Yes of course, lack of mbuf space is another reason for
> >>>>> disabling the feature.
> >>>>>
> >>>>>> If we finally go way A, should we add offloads for META back?
> >>>>>> I guess separate Rx and Tx are required.
> >>>>> I would prefer to add it as dynamic flags.
> >>>>> Why rushing on a very temporary solution while it is not a new issue?
> >>>> Basically it means that we go just (B)+(C) in the case of META.
> >>>> I have no strong opinion but thought that it could be better to
> >>>> align the solution. Of course, we can wait with it. As I understand
> >>>> META is an experimental feature.
> >>> Yes it is experimental and I think it is too late to align now.
> >>>
> >>> Anyway, we will probably to discuss again these offloads TAG/MARK/META,
> >>> as requested by several people.
> >>>
> >> The series implements (A) to help to solve the problem described above.
> >> What is the fate of the series in v19.11 in accordance with the
> >> discussion?
> > I am against adding anything related to a feature union'ed in mbuf.
> > The feature must move to dynamic field first.
> >
> > In addition, such capability is very weak.
> > I am not sure it is a good idea to have some weak capabilities,
> > meaning a feature could be available but not in all cases.
> > I think we should discuss more generally how we want to handle
> > the rte_flow capabilities conveniently and reliably.
> 
> It is really unexpected outcome from the above discussion.

I'm sorry, I thought I was clear in my request to switch to dynamic first.


> It is just possibility to deliver and handle marks on datapath and
> request to have it. It says almost nothing about rte_flow rules
> supported etc. I'll be happy to take part in the discussion.
> 
> > So regarding 19.11, as this feature is not new, it can wait 20.02.
> 
> OK, it is not critical for me, so I don't mind, however, I've seen
> patches which try to use it [1] except net/octeontx2 in the second
> patch of the series.
> 
> [1] https://patches.dpdk.org/patch/62415/

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ferruh Yigit]

On 11/19/2019 11:09 AM, Thomas Monjalon wrote:
> 19/11/2019 11:59, Andrew Rybchenko:
>> On 11/19/19 12:50 PM, Thomas Monjalon wrote:
>>> 19/11/2019 10:24, Andrew Rybchenko:
>>>> On 11/8/19 4:30 PM, Thomas Monjalon wrote:
>>>>> 08/11/2019 14:27, Andrew Rybchenko:
>>>>>> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
>>>>>>> 08/11/2019 13:00, Andrew Rybchenko:
>>>>>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
>>>>>>>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>>>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>>>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>>>>>>>> The problem:
>>>>>>>>>>>> ~~~~~~~~~~~~
>>>>>>>>>>>> PMD wants to know before port start if application wants to
>>>>>>>>>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>>>>>>>    for MARK/FLAG delivery
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>>>>>>>    is faster, but does not support MARK)
>>>>>>>>>>> Thank you for the clear problem statement.
>>>>>>>>>>> I agree with it. This is a real design issue.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Discussed solutions:
>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>> May be it is not 100% clear since below are alternatives.
>>>>>>>>>>
>>>>>>>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>>>>>>>
>>>>>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>>>>>>>
>>>>>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>>>>>>>    and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>>>>>>>    of (B) to discover if a feature is supported.
>>>>>>>>>>> The dynamic field should be registered via a new API function
>>>>>>>>>>> named '<feature>_init'.
>>>>>>>>>>> It means the application must explicit request the feature.
>>>>>>>>>>> I agree this is the way to go.
>>>>>>>>>> If I understand your statement correctly, but (C) is not ideal since it
>>>>>>>>>> looks global. If registered dynamic field of mbuf and is flag that
>>>>>>>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>>>>>>>
>>>>>>>>>>>> All solutions require changes in applications which use these
>>>>>>>>>>>> features. There is a deprecation notice in place which advertises
>>>>>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>>>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>>>>>>>>>> it should be combined with (B) in order to understand if
>>>>>>>>>>>> the feature is supported.
>>>>>>>>>>> I don't understand.
>>>>>>>>>>> Application request and PMD support are two different things.
>>>>>>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>>>>>>>> I mean that application wants to understand if the feature is
>>>>>>>>>> supported. Then, it wants to enable it. In the case of (B),
>>>>>>>>>> if I understand the solution correctly, there is no explicit
>>>>>>>>>> way to enable, PMD just detects it because of discovery is done
>>>>>>>>>> (that's what I mean by "implicit" and it is a drawback from my
>>>>>>>>>> point of view, but still could be considered). (C) solves the
>>>>>>>>>> problem of (B).
>>>>>>>>>>
>>>>>>>>>>>> Advantages and drawbacks of solutions:
>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>>>>>>>    have a way to request flow MARK using rte_flow API.
>>>>>>>>>>>>    I don't fully agree that it is a duplication, but I agree
>>>>>>>>>>>>    that it sounds like duplication and complicates a bit flow
>>>>>>>>>>>>    MARK usage by applications. (B) complicates it as well.
>>>>>>>>>>>>
>>>>>>>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>>>>>>>    similar solution for META and it eats one more offload bit.
>>>>>>>>>>>>    Yes, that's true and I think it is not a problem.
>>>>>>>>>>>>    It would make it easier for applications to find out if
>>>>>>>>>>>>    either MARK or META is supported.
>>>>>>>>>>>>
>>>>>>>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>>>>>>>    It is simple for application to understand if it supported.
>>>>>>>>>>>>    It is simple in PMD to understand that it is required.
>>>>>>>>>>>>    It is simple to disable it - just reconfigure.
>>>>>>>>>>>>    Also it is easier to document it - just mention that
>>>>>>>>>>>>    the offload should be supported and enabled.
>>>>>>>>>>>>
>>>>>>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>>>>>>>    I agree that it is valid argument. Solving the problem
>>>>>>>>>>>>    without extra entities is always nice, but unfortunately
>>>>>>>>>>>>    it is too complex in this case.
>>>>>>>>>>>>
>>>>>>>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>>>>>>>    It is necessary to choose a flow rule which should be used
>>>>>>>>>>>>    as a criteria. It could be hardware dependent.
>>>>>>>>>>>>    Complex logic is require in PMD if it wants to address the
>>>>>>>>>>>>    problem and control MARK delivery based on validated flow
>>>>>>>>>>>>    rules. It adds dependency between start/stop processing and
>>>>>>>>>>>>    flow rules validation code.
>>>>>>>>>>>>    It is pretty complicated to document it.
>>>>>>>>>>>>
>>>>>>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>>>>>>>    if really used flow rules do not support MARK looks like
>>>>>>>>>>>>    drawback as well, but easily mitigated by a combination
>>>>>>>>>>>>    with solution (B) and only required if the application wants
>>>>>>>>>>>>    to dive in the level of optimization and complexity and
>>>>>>>>>>>>    makes sense if application knows required flow rules in
>>>>>>>>>>>>    advance. So, it is not a problem in this case.
>>>>>>>>>>>>
>>>>>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>>>>>>>    applications to understand if these features are supported,
>>>>>>>>>>>>    but no drawbacks in PMD, since explicit criteria is used to
>>>>>>>>>>>>    enable/disable (dynamic field/flag lookup).
>>>>>>>>>>>>
>>>>>>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>>>>>>>
>>>>>>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>>>>>>>    As it was discussed in the case of RX_TIMESTAMP
>>>>>>>>>>>>    (if I remember it correctly):
>>>>>>>>>>>>     - PMD advertises RX_TIMESTAMP offload capability
>>>>>>>>>>>>     - application enables the offload
>>>>>>>>>>>>     - PMD registers dynamic field for timestamp
>>>>>>>>>>>>    Solution (C):
>>>>>>>>>>>>      - PMD advertises nothing
>>>>>>>>>>>>      - application uses solution (B) to understand if
>>>>>>>>>>>>        these features are supported
>>>>>>>>>>>>      - application registers dynamic field/flag
>>>>>>>>>>>>      - PMD does lookup and solve the problem
>>>>>>>>>>>>    The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>>>>>>>    solution is changed to require an application to register
>>>>>>>>>>>>    dynamic fields and PMD to do lookup if the offload is
>>>>>>>>>>>>    enabled. So, the only difference will be in no offload
>>>>>>>>>>>>    in the case of flow MARK/FLAG and usage of complex logic
>>>>>>>>>>>>    to understand if it is supported or no.
>>>>>>>>>>>>    May be it would be really good since it will allow to
>>>>>>>>>>>>    have dynamic fields registered before mempool population.
>>>>>>>>>>>>
>>>>>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>>>>>>>     Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>>>>>>>     per queue. Moreover (C) looks global - for all devices.
>>>>>>>>>>>>     It could be really painful.
>>>>>>>>>>>>
>>>>>>>>>>>> (C) is nice, but I still vote for simplicity and
>>>>>>>>>>>> granularity of (A).
>>>>>>>>>>> I vote for clear separation of application needs and PMD support,
>>>>>>>>>>> by using the method C (dynamic fields).
>>>>>>>>>>> I agree timestamp must use the same path.
>>>>>>>>>>> I agree it's complicate because we don't know in advance whether
>>>>>>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
>>>>>>>>>> Do you think that global nature of the (C) is acceptable?
>>>>>>>>> That's a good question.
>>>>>>>>> Maybe the feature request should be per port.
>>>>>>>>> In this case, we are back to solution A with a flag per port?
>>>>>>>> Offloads are natively per-queue as well, so (A) keeps the choice
>>>>>>>> between per-port vs per-queue to PMDs as usual.
>>>>>>>>
>>>>>>>>> Note that A and C will not guarantee that the offload will be possible.
>>>>>>>> Yes, definitely.
>>>>>>>>
>>>>>>>>> We need B (flow rule validation) anyway.
>>>>>>>> Strictly speaking (B) (checking flow rules before device
>>>>>>>> startup) is required if an application can predict flow
>>>>>>>> rules and wants to ensure that MARK offload will be usable.
>>>>>>>> Otherwise, it may be skipped.
>>>>>>> No no, I mean flow rule validation MUST be used anyway
>>>>>>> during the runtime before applying a rule.
>>>>>>> I agree it is hard to predict. I speak only about real rules.
>>>>>> OK, I see. Of course, flow rule validation is required at runtime.
>>>>>> I was rather concentrated on the stated problem solutions.
>>>>>>
>>>>>>>>> It seems A, B, C are not alternatives but all required
>>>>>>>>> as pieces of a puzzle...
>>>>>>>> Unfortunately true in the most complex case.
>>>>>>>> Right now it will be A with B if required as explained above.
>>>>>>>> C will come a bit later when the field migrates to dynamic.
>>>>>>>>
>>>>>>>> May be it is even better if application registers dynamic
>>>>>>>> fields before an attempt to enable offload to be sure that
>>>>>>>> it will not fail because of impossibility to register
>>>>>>>> dynamic field (lack of space). I'm not sure, but it is not
>>>>>>>> not that important.
>>>>>>> Yes of course, lack of mbuf space is another reason for
>>>>>>> disabling the feature.
>>>>>>>
>>>>>>>> If we finally go way A, should we add offloads for META back?
>>>>>>>> I guess separate Rx and Tx are required.
>>>>>>> I would prefer to add it as dynamic flags.
>>>>>>> Why rushing on a very temporary solution while it is not a new issue?
>>>>>> Basically it means that we go just (B)+(C) in the case of META.
>>>>>> I have no strong opinion but thought that it could be better to
>>>>>> align the solution. Of course, we can wait with it. As I understand
>>>>>> META is an experimental feature.
>>>>> Yes it is experimental and I think it is too late to align now.
>>>>>
>>>>> Anyway, we will probably to discuss again these offloads TAG/MARK/META,
>>>>> as requested by several people.
>>>>>
>>>> The series implements (A) to help to solve the problem described above.
>>>> What is the fate of the series in v19.11 in accordance with the
>>>> discussion?
>>> I am against adding anything related to a feature union'ed in mbuf.
>>> The feature must move to dynamic field first.
>>>
>>> In addition, such capability is very weak.
>>> I am not sure it is a good idea to have some weak capabilities,
>>> meaning a feature could be available but not in all cases.
>>> I think we should discuss more generally how we want to handle
>>> the rte_flow capabilities conveniently and reliably.
>>
>> It is really unexpected outcome from the above discussion.
> 
> I'm sorry, I thought I was clear in my request to switch to dynamic first.
> 
> 
>> It is just possibility to deliver and handle marks on datapath and
>> request to have it. It says almost nothing about rte_flow rules
>> supported etc. I'll be happy to take part in the discussion.
>>
>>> So regarding 19.11, as this feature is not new, it can wait 20.02.
>>
>> OK, it is not critical for me, so I don't mind, however, I've seen
>> patches which try to use it [1] except net/octeontx2 in the second
>> patch of the series.
>>
>> [1] https://patches.dpdk.org/patch/62415/
> 

Sorry, I have to resurrect this old (long) discussion because the patches are
still active in the patchwork [1] and the deprecation notice is still there [2].

Andrew has a good summary in the thread [3], after a year nothing seems changed.


Pavan, Thomas, Andrew, Ori,

What is our plan with this series, lets try to have a conclusion.




[1]
https://patches.dpdk.org/user/todo/dpdk/?series=7076

[2]
http://lxr.dpdk.org/dpdk/v20.05/source/doc/guides/rel_notes/deprecation.rst#L88

[3]
http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ferruh Yigit]

On 7/3/2020 3:34 PM, Ferruh Yigit wrote:
> On 11/19/2019 11:09 AM, Thomas Monjalon wrote:
>> 19/11/2019 11:59, Andrew Rybchenko:
>>> On 11/19/19 12:50 PM, Thomas Monjalon wrote:
>>>> 19/11/2019 10:24, Andrew Rybchenko:
>>>>> On 11/8/19 4:30 PM, Thomas Monjalon wrote:
>>>>>> 08/11/2019 14:27, Andrew Rybchenko:
>>>>>>> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
>>>>>>>> 08/11/2019 13:00, Andrew Rybchenko:
>>>>>>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
>>>>>>>>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>>>>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>>>>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>>>>>>>>> The problem:
>>>>>>>>>>>>> ~~~~~~~~~~~~
>>>>>>>>>>>>> PMD wants to know before port start if application wants to
>>>>>>>>>>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>>>>>>>>     for MARK/FLAG delivery
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>>>>>>>>     is faster, but does not support MARK)
>>>>>>>>>>>> Thank you for the clear problem statement.
>>>>>>>>>>>> I agree with it. This is a real design issue.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Discussed solutions:
>>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>>> May be it is not 100% clear since below are alternatives.
>>>>>>>>>>>
>>>>>>>>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>>>>>>>>
>>>>>>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>>>>>>>>     and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>>>>>>>>     of (B) to discover if a feature is supported.
>>>>>>>>>>>> The dynamic field should be registered via a new API function
>>>>>>>>>>>> named '<feature>_init'.
>>>>>>>>>>>> It means the application must explicit request the feature.
>>>>>>>>>>>> I agree this is the way to go.
>>>>>>>>>>> If I understand your statement correctly, but (C) is not ideal since it
>>>>>>>>>>> looks global. If registered dynamic field of mbuf and is flag that
>>>>>>>>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>>>>>>>>
>>>>>>>>>>>>> All solutions require changes in applications which use these
>>>>>>>>>>>>> features. There is a deprecation notice in place which advertises
>>>>>>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>>>>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>>>>>>>>>>> it should be combined with (B) in order to understand if
>>>>>>>>>>>>> the feature is supported.
>>>>>>>>>>>> I don't understand.
>>>>>>>>>>>> Application request and PMD support are two different things.
>>>>>>>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>>>>>>>>> I mean that application wants to understand if the feature is
>>>>>>>>>>> supported. Then, it wants to enable it. In the case of (B),
>>>>>>>>>>> if I understand the solution correctly, there is no explicit
>>>>>>>>>>> way to enable, PMD just detects it because of discovery is done
>>>>>>>>>>> (that's what I mean by "implicit" and it is a drawback from my
>>>>>>>>>>> point of view, but still could be considered). (C) solves the
>>>>>>>>>>> problem of (B).
>>>>>>>>>>>
>>>>>>>>>>>>> Advantages and drawbacks of solutions:
>>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>>>>>>>>     have a way to request flow MARK using rte_flow API.
>>>>>>>>>>>>>     I don't fully agree that it is a duplication, but I agree
>>>>>>>>>>>>>     that it sounds like duplication and complicates a bit flow
>>>>>>>>>>>>>     MARK usage by applications. (B) complicates it as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>>>>>>>>     similar solution for META and it eats one more offload bit.
>>>>>>>>>>>>>     Yes, that's true and I think it is not a problem.
>>>>>>>>>>>>>     It would make it easier for applications to find out if
>>>>>>>>>>>>>     either MARK or META is supported.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>>>>>>>>     It is simple for application to understand if it supported.
>>>>>>>>>>>>>     It is simple in PMD to understand that it is required.
>>>>>>>>>>>>>     It is simple to disable it - just reconfigure.
>>>>>>>>>>>>>     Also it is easier to document it - just mention that
>>>>>>>>>>>>>     the offload should be supported and enabled.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>>>>>>>>     I agree that it is valid argument. Solving the problem
>>>>>>>>>>>>>     without extra entities is always nice, but unfortunately
>>>>>>>>>>>>>     it is too complex in this case.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>>>>>>>>     It is necessary to choose a flow rule which should be used
>>>>>>>>>>>>>     as a criteria. It could be hardware dependent.
>>>>>>>>>>>>>     Complex logic is require in PMD if it wants to address the
>>>>>>>>>>>>>     problem and control MARK delivery based on validated flow
>>>>>>>>>>>>>     rules. It adds dependency between start/stop processing and
>>>>>>>>>>>>>     flow rules validation code.
>>>>>>>>>>>>>     It is pretty complicated to document it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>>>>>>>>     if really used flow rules do not support MARK looks like
>>>>>>>>>>>>>     drawback as well, but easily mitigated by a combination
>>>>>>>>>>>>>     with solution (B) and only required if the application wants
>>>>>>>>>>>>>     to dive in the level of optimization and complexity and
>>>>>>>>>>>>>     makes sense if application knows required flow rules in
>>>>>>>>>>>>>     advance. So, it is not a problem in this case.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>>>>>>>>     applications to understand if these features are supported,
>>>>>>>>>>>>>     but no drawbacks in PMD, since explicit criteria is used to
>>>>>>>>>>>>>     enable/disable (dynamic field/flag lookup).
>>>>>>>>>>>>>
>>>>>>>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>>>>>>>>
>>>>>>>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>>>>>>>>     As it was discussed in the case of RX_TIMESTAMP
>>>>>>>>>>>>>     (if I remember it correctly):
>>>>>>>>>>>>>      - PMD advertises RX_TIMESTAMP offload capability
>>>>>>>>>>>>>      - application enables the offload
>>>>>>>>>>>>>      - PMD registers dynamic field for timestamp
>>>>>>>>>>>>>     Solution (C):
>>>>>>>>>>>>>       - PMD advertises nothing
>>>>>>>>>>>>>       - application uses solution (B) to understand if
>>>>>>>>>>>>>         these features are supported
>>>>>>>>>>>>>       - application registers dynamic field/flag
>>>>>>>>>>>>>       - PMD does lookup and solve the problem
>>>>>>>>>>>>>     The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>>>>>>>>     solution is changed to require an application to register
>>>>>>>>>>>>>     dynamic fields and PMD to do lookup if the offload is
>>>>>>>>>>>>>     enabled. So, the only difference will be in no offload
>>>>>>>>>>>>>     in the case of flow MARK/FLAG and usage of complex logic
>>>>>>>>>>>>>     to understand if it is supported or no.
>>>>>>>>>>>>>     May be it would be really good since it will allow to
>>>>>>>>>>>>>     have dynamic fields registered before mempool population.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>>>>>>>>      Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>>>>>>>>      per queue. Moreover (C) looks global - for all devices.
>>>>>>>>>>>>>      It could be really painful.
>>>>>>>>>>>>>
>>>>>>>>>>>>> (C) is nice, but I still vote for simplicity and
>>>>>>>>>>>>> granularity of (A).
>>>>>>>>>>>> I vote for clear separation of application needs and PMD support,
>>>>>>>>>>>> by using the method C (dynamic fields).
>>>>>>>>>>>> I agree timestamp must use the same path.
>>>>>>>>>>>> I agree it's complicate because we don't know in advance whether
>>>>>>>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
>>>>>>>>>>> Do you think that global nature of the (C) is acceptable?
>>>>>>>>>> That's a good question.
>>>>>>>>>> Maybe the feature request should be per port.
>>>>>>>>>> In this case, we are back to solution A with a flag per port?
>>>>>>>>> Offloads are natively per-queue as well, so (A) keeps the choice
>>>>>>>>> between per-port vs per-queue to PMDs as usual.
>>>>>>>>>
>>>>>>>>>> Note that A and C will not guarantee that the offload will be possible.
>>>>>>>>> Yes, definitely.
>>>>>>>>>
>>>>>>>>>> We need B (flow rule validation) anyway.
>>>>>>>>> Strictly speaking (B) (checking flow rules before device
>>>>>>>>> startup) is required if an application can predict flow
>>>>>>>>> rules and wants to ensure that MARK offload will be usable.
>>>>>>>>> Otherwise, it may be skipped.
>>>>>>>> No no, I mean flow rule validation MUST be used anyway
>>>>>>>> during the runtime before applying a rule.
>>>>>>>> I agree it is hard to predict. I speak only about real rules.
>>>>>>> OK, I see. Of course, flow rule validation is required at runtime.
>>>>>>> I was rather concentrated on the stated problem solutions.
>>>>>>>
>>>>>>>>>> It seems A, B, C are not alternatives but all required
>>>>>>>>>> as pieces of a puzzle...
>>>>>>>>> Unfortunately true in the most complex case.
>>>>>>>>> Right now it will be A with B if required as explained above.
>>>>>>>>> C will come a bit later when the field migrates to dynamic.
>>>>>>>>>
>>>>>>>>> May be it is even better if application registers dynamic
>>>>>>>>> fields before an attempt to enable offload to be sure that
>>>>>>>>> it will not fail because of impossibility to register
>>>>>>>>> dynamic field (lack of space). I'm not sure, but it is not
>>>>>>>>> not that important.
>>>>>>>> Yes of course, lack of mbuf space is another reason for
>>>>>>>> disabling the feature.
>>>>>>>>
>>>>>>>>> If we finally go way A, should we add offloads for META back?
>>>>>>>>> I guess separate Rx and Tx are required.
>>>>>>>> I would prefer to add it as dynamic flags.
>>>>>>>> Why rushing on a very temporary solution while it is not a new issue?
>>>>>>> Basically it means that we go just (B)+(C) in the case of META.
>>>>>>> I have no strong opinion but thought that it could be better to
>>>>>>> align the solution. Of course, we can wait with it. As I understand
>>>>>>> META is an experimental feature.
>>>>>> Yes it is experimental and I think it is too late to align now.
>>>>>>
>>>>>> Anyway, we will probably to discuss again these offloads TAG/MARK/META,
>>>>>> as requested by several people.
>>>>>>
>>>>> The series implements (A) to help to solve the problem described above.
>>>>> What is the fate of the series in v19.11 in accordance with the
>>>>> discussion?
>>>> I am against adding anything related to a feature union'ed in mbuf.
>>>> The feature must move to dynamic field first.
>>>>
>>>> In addition, such capability is very weak.
>>>> I am not sure it is a good idea to have some weak capabilities,
>>>> meaning a feature could be available but not in all cases.
>>>> I think we should discuss more generally how we want to handle
>>>> the rte_flow capabilities conveniently and reliably.
>>>
>>> It is really unexpected outcome from the above discussion.
>>
>> I'm sorry, I thought I was clear in my request to switch to dynamic first.
>>
>>
>>> It is just possibility to deliver and handle marks on datapath and
>>> request to have it. It says almost nothing about rte_flow rules
>>> supported etc. I'll be happy to take part in the discussion.
>>>
>>>> So regarding 19.11, as this feature is not new, it can wait 20.02.
>>>
>>> OK, it is not critical for me, so I don't mind, however, I've seen
>>> patches which try to use it [1] except net/octeontx2 in the second
>>> patch of the series.
>>>
>>> [1] https://patches.dpdk.org/patch/62415/
>>
> 
> Sorry, I have to resurrect this old (long) discussion because the patches are
> still active in the patchwork [1] and the deprecation notice is still there [2].
> 
> Andrew has a good summary in the thread [3], after a year nothing seems changed.
> 
> 
> Pavan, Thomas, Andrew, Ori,
> 
> What is our plan with this series, lets try to have a conclusion.
> 
> 
> 
> 
> [1]
> https://patches.dpdk.org/user/todo/dpdk/?series=7076
> 
> [2]
> http://lxr.dpdk.org/dpdk/v20.05/source/doc/guides/rel_notes/deprecation.rst#L88
> 
> [3]
> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
> 
> 

I re-read the thread, will try to have a little movement while we are in the new 
release cycle, if there is no update I am planning to reject the patches.

There seems two problems:

P1) Application will keep trying to program NIC for MARK action for each flow, 
since application doesn't know if next one will succeed or not.
If only there would be a way to find out that NIC/PMD doesn't support the MARK 
action at all, this could save application to keep trying.

P2) PMD can make better internal choices if it gets more hint from application 
about MARK action may be used or not.
Application at least may say it won't use the MARK flow action at all.


This patch uses offload flags infrastructure to solve above two problems, 
solution (A) in Andrew's summary.

Although it may solve the issues, there are questions/concerns around using this 
additional flag to control flow API, I also agree it may be confusing in the 
design level although practically using flags can be simple.
And this is not generic solution, what happen with META action question is 
already hanging on in the thread, more flags? How many more can we add?

And also there is option an to use dynamic mbuf flags to detect the capability, 
solution (C) in Andrew's summary, again it may solve the problem but it looks 
again a workaround to solve same flow API design restriction, and this one is 
not as simple as (A).

Overall the discussion seems going on circles without an agreed on decision.



What about trying to solve this with flow API return values,

If a flow rule is not supported at all by the NIC/PMD, it may return 
'-ENO_WAY_JOSE', and application knows it can't be used at all, this may solve 
the (P1) above.

And if a flow rule can be supported for the given pattern, but it is not 
supported right now because current configuration or resourcing restrictions 
doesn't allow creating rule, a special error type can be returned with a 
descriptive error log for application to response:
-ECONFLICT, "Can't enable rule A when rule B is enabled"
-EDATAPATH, "Can't enable this rule when vector datapath is used"
-ERESOURCE, "Can't enable more than 3 rules"
This may solve the (P2) partially.

I am not sure about second part, but at least first part shouldn't be too hard 
to implement, and it is a generic solution, what do you think?

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Thomas Monjalon]

17/02/2021 14:45, Ferruh Yigit:
> On 7/3/2020 3:34 PM, Ferruh Yigit wrote:
> > On 11/19/2019 11:09 AM, Thomas Monjalon wrote:
> >> 19/11/2019 11:59, Andrew Rybchenko:
> >>> On 11/19/19 12:50 PM, Thomas Monjalon wrote:
> >>>> 19/11/2019 10:24, Andrew Rybchenko:
> >>>>> On 11/8/19 4:30 PM, Thomas Monjalon wrote:
> >>>>>> 08/11/2019 14:27, Andrew Rybchenko:
> >>>>>>> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
> >>>>>>>> 08/11/2019 13:00, Andrew Rybchenko:
> >>>>>>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
> >>>>>>>>>> 08/11/2019 11:42, Andrew Rybchenko:
> >>>>>>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
> >>>>>>>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
> >>>>>>>>>>>>> The problem:
> >>>>>>>>>>>>> ~~~~~~~~~~~~
> >>>>>>>>>>>>> PMD wants to know before port start if application wants to
> >>>>>>>>>>>>> to use flow MARK/FLAG in the future. It is required because:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1. HW may be configured in a different way to reserve resources
> >>>>>>>>>>>>>     for MARK/FLAG delivery
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
> >>>>>>>>>>>>>     is faster, but does not support MARK)
> >>>>>>>>>>>> Thank you for the clear problem statement.
> >>>>>>>>>>>> I agree with it. This is a real design issue.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Discussed solutions:
> >>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~
> >>>>>>>>>>> May be it is not 100% clear since below are alternatives.
> >>>>>>>>>>>
> >>>>>>>>>>>>> A. Explicit Rx offload suggested by the patch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
> >>>>>>>>>>>>>     and/or flag and PMD uses lookup to solve the problem) plus part
> >>>>>>>>>>>>>     of (B) to discover if a feature is supported.
> >>>>>>>>>>>> The dynamic field should be registered via a new API function
> >>>>>>>>>>>> named '<feature>_init'.
> >>>>>>>>>>>> It means the application must explicit request the feature.
> >>>>>>>>>>>> I agree this is the way to go.
> >>>>>>>>>>> If I understand your statement correctly, but (C) is not ideal since it
> >>>>>>>>>>> looks global. If registered dynamic field of mbuf and is flag that
> >>>>>>>>>>> the feature should be enabled, it is a flag to all ports/devices.
> >>>>>>>>>>>
> >>>>>>>>>>>>> All solutions require changes in applications which use these
> >>>>>>>>>>>>> features. There is a deprecation notice in place which advertises
> >>>>>>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
> >>>>>>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
> >>>>>>>>>>>>> it should be combined with (B) in order to understand if
> >>>>>>>>>>>>> the feature is supported.
> >>>>>>>>>>>> I don't understand.
> >>>>>>>>>>>> Application request and PMD support are two different things.
> >>>>>>>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
> >>>>>>>>>>> I mean that application wants to understand if the feature is
> >>>>>>>>>>> supported. Then, it wants to enable it. In the case of (B),
> >>>>>>>>>>> if I understand the solution correctly, there is no explicit
> >>>>>>>>>>> way to enable, PMD just detects it because of discovery is done
> >>>>>>>>>>> (that's what I mean by "implicit" and it is a drawback from my
> >>>>>>>>>>> point of view, but still could be considered). (C) solves the
> >>>>>>>>>>> problem of (B).
> >>>>>>>>>>>
> >>>>>>>>>>>>> Advantages and drawbacks of solutions:
> >>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>>>>>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
> >>>>>>>>>>>>>     have a way to request flow MARK using rte_flow API.
> >>>>>>>>>>>>>     I don't fully agree that it is a duplication, but I agree
> >>>>>>>>>>>>>     that it sounds like duplication and complicates a bit flow
> >>>>>>>>>>>>>     MARK usage by applications. (B) complicates it as well.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2. One more drawback of the solution (A) is the necessity of
> >>>>>>>>>>>>>     similar solution for META and it eats one more offload bit.
> >>>>>>>>>>>>>     Yes, that's true and I think it is not a problem.
> >>>>>>>>>>>>>     It would make it easier for applications to find out if
> >>>>>>>>>>>>>     either MARK or META is supported.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 3. The main advantage of the solution (A) is simplicity.
> >>>>>>>>>>>>>     It is simple for application to understand if it supported.
> >>>>>>>>>>>>>     It is simple in PMD to understand that it is required.
> >>>>>>>>>>>>>     It is simple to disable it - just reconfigure.
> >>>>>>>>>>>>>     Also it is easier to document it - just mention that
> >>>>>>>>>>>>>     the offload should be supported and enabled.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
> >>>>>>>>>>>>>     I agree that it is valid argument. Solving the problem
> >>>>>>>>>>>>>     without extra entities is always nice, but unfortunately
> >>>>>>>>>>>>>     it is too complex in this case.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 5. The main drawback of the solution (B) is the complexity.
> >>>>>>>>>>>>>     It is necessary to choose a flow rule which should be used
> >>>>>>>>>>>>>     as a criteria. It could be hardware dependent.
> >>>>>>>>>>>>>     Complex logic is require in PMD if it wants to address the
> >>>>>>>>>>>>>     problem and control MARK delivery based on validated flow
> >>>>>>>>>>>>>     rules. It adds dependency between start/stop processing and
> >>>>>>>>>>>>>     flow rules validation code.
> >>>>>>>>>>>>>     It is pretty complicated to document it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
> >>>>>>>>>>>>>     if really used flow rules do not support MARK looks like
> >>>>>>>>>>>>>     drawback as well, but easily mitigated by a combination
> >>>>>>>>>>>>>     with solution (B) and only required if the application wants
> >>>>>>>>>>>>>     to dive in the level of optimization and complexity and
> >>>>>>>>>>>>>     makes sense if application knows required flow rules in
> >>>>>>>>>>>>>     advance. So, it is not a problem in this case.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
> >>>>>>>>>>>>>     applications to understand if these features are supported,
> >>>>>>>>>>>>>     but no drawbacks in PMD, since explicit criteria is used to
> >>>>>>>>>>>>>     enable/disable (dynamic field/flag lookup).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
> >>>>>>>>>>>>>     As it was discussed in the case of RX_TIMESTAMP
> >>>>>>>>>>>>>     (if I remember it correctly):
> >>>>>>>>>>>>>      - PMD advertises RX_TIMESTAMP offload capability
> >>>>>>>>>>>>>      - application enables the offload
> >>>>>>>>>>>>>      - PMD registers dynamic field for timestamp
> >>>>>>>>>>>>>     Solution (C):
> >>>>>>>>>>>>>       - PMD advertises nothing
> >>>>>>>>>>>>>       - application uses solution (B) to understand if
> >>>>>>>>>>>>>         these features are supported
> >>>>>>>>>>>>>       - application registers dynamic field/flag
> >>>>>>>>>>>>>       - PMD does lookup and solve the problem
> >>>>>>>>>>>>>     The asymmetry could be partially mitigated if RX_TIMESTAMP
> >>>>>>>>>>>>>     solution is changed to require an application to register
> >>>>>>>>>>>>>     dynamic fields and PMD to do lookup if the offload is
> >>>>>>>>>>>>>     enabled. So, the only difference will be in no offload
> >>>>>>>>>>>>>     in the case of flow MARK/FLAG and usage of complex logic
> >>>>>>>>>>>>>     to understand if it is supported or no.
> >>>>>>>>>>>>>     May be it would be really good since it will allow to
> >>>>>>>>>>>>>     have dynamic fields registered before mempool population.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
> >>>>>>>>>>>>>      Solution (A) may be per queue while (B) and (C) cannot be
> >>>>>>>>>>>>>      per queue. Moreover (C) looks global - for all devices.
> >>>>>>>>>>>>>      It could be really painful.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> (C) is nice, but I still vote for simplicity and
> >>>>>>>>>>>>> granularity of (A).
> >>>>>>>>>>>> I vote for clear separation of application needs and PMD support,
> >>>>>>>>>>>> by using the method C (dynamic fields).
> >>>>>>>>>>>> I agree timestamp must use the same path.
> >>>>>>>>>>>> I agree it's complicate because we don't know in advance whether
> >>>>>>>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
> >>>>>>>>>>> Do you think that global nature of the (C) is acceptable?
> >>>>>>>>>> That's a good question.
> >>>>>>>>>> Maybe the feature request should be per port.
> >>>>>>>>>> In this case, we are back to solution A with a flag per port?
> >>>>>>>>> Offloads are natively per-queue as well, so (A) keeps the choice
> >>>>>>>>> between per-port vs per-queue to PMDs as usual.
> >>>>>>>>>
> >>>>>>>>>> Note that A and C will not guarantee that the offload will be possible.
> >>>>>>>>> Yes, definitely.
> >>>>>>>>>
> >>>>>>>>>> We need B (flow rule validation) anyway.
> >>>>>>>>> Strictly speaking (B) (checking flow rules before device
> >>>>>>>>> startup) is required if an application can predict flow
> >>>>>>>>> rules and wants to ensure that MARK offload will be usable.
> >>>>>>>>> Otherwise, it may be skipped.
> >>>>>>>> No no, I mean flow rule validation MUST be used anyway
> >>>>>>>> during the runtime before applying a rule.
> >>>>>>>> I agree it is hard to predict. I speak only about real rules.
> >>>>>>> OK, I see. Of course, flow rule validation is required at runtime.
> >>>>>>> I was rather concentrated on the stated problem solutions.
> >>>>>>>
> >>>>>>>>>> It seems A, B, C are not alternatives but all required
> >>>>>>>>>> as pieces of a puzzle...
> >>>>>>>>> Unfortunately true in the most complex case.
> >>>>>>>>> Right now it will be A with B if required as explained above.
> >>>>>>>>> C will come a bit later when the field migrates to dynamic.
> >>>>>>>>>
> >>>>>>>>> May be it is even better if application registers dynamic
> >>>>>>>>> fields before an attempt to enable offload to be sure that
> >>>>>>>>> it will not fail because of impossibility to register
> >>>>>>>>> dynamic field (lack of space). I'm not sure, but it is not
> >>>>>>>>> not that important.
> >>>>>>>> Yes of course, lack of mbuf space is another reason for
> >>>>>>>> disabling the feature.
> >>>>>>>>
> >>>>>>>>> If we finally go way A, should we add offloads for META back?
> >>>>>>>>> I guess separate Rx and Tx are required.
> >>>>>>>> I would prefer to add it as dynamic flags.
> >>>>>>>> Why rushing on a very temporary solution while it is not a new issue?
> >>>>>>> Basically it means that we go just (B)+(C) in the case of META.
> >>>>>>> I have no strong opinion but thought that it could be better to
> >>>>>>> align the solution. Of course, we can wait with it. As I understand
> >>>>>>> META is an experimental feature.
> >>>>>> Yes it is experimental and I think it is too late to align now.
> >>>>>>
> >>>>>> Anyway, we will probably to discuss again these offloads TAG/MARK/META,
> >>>>>> as requested by several people.
> >>>>>>
> >>>>> The series implements (A) to help to solve the problem described above.
> >>>>> What is the fate of the series in v19.11 in accordance with the
> >>>>> discussion?
> >>>> I am against adding anything related to a feature union'ed in mbuf.
> >>>> The feature must move to dynamic field first.
> >>>>
> >>>> In addition, such capability is very weak.
> >>>> I am not sure it is a good idea to have some weak capabilities,
> >>>> meaning a feature could be available but not in all cases.
> >>>> I think we should discuss more generally how we want to handle
> >>>> the rte_flow capabilities conveniently and reliably.
> >>>
> >>> It is really unexpected outcome from the above discussion.
> >>
> >> I'm sorry, I thought I was clear in my request to switch to dynamic first.
> >>
> >>
> >>> It is just possibility to deliver and handle marks on datapath and
> >>> request to have it. It says almost nothing about rte_flow rules
> >>> supported etc. I'll be happy to take part in the discussion.
> >>>
> >>>> So regarding 19.11, as this feature is not new, it can wait 20.02.
> >>>
> >>> OK, it is not critical for me, so I don't mind, however, I've seen
> >>> patches which try to use it [1] except net/octeontx2 in the second
> >>> patch of the series.
> >>>
> >>> [1] https://patches.dpdk.org/patch/62415/
> >>
> > 
> > Sorry, I have to resurrect this old (long) discussion because the patches are
> > still active in the patchwork [1] and the deprecation notice is still there [2].
> > 
> > Andrew has a good summary in the thread [3], after a year nothing seems changed.
> > 
> > 
> > Pavan, Thomas, Andrew, Ori,
> > 
> > What is our plan with this series, lets try to have a conclusion.
> > 
> > 
> > 
> > 
> > [1]
> > https://patches.dpdk.org/user/todo/dpdk/?series=7076
> > 
> > [2]
> > http://lxr.dpdk.org/dpdk/v20.05/source/doc/guides/rel_notes/deprecation.rst#L88
> > 
> > [3]
> > http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
> > 
> > 
> 
> I re-read the thread, will try to have a little movement while we are in the new 
> release cycle, if there is no update I am planning to reject the patches.
> 
> There seems two problems:
> 
> P1) Application will keep trying to program NIC for MARK action for each flow, 
> since application doesn't know if next one will succeed or not.
> If only there would be a way to find out that NIC/PMD doesn't support the MARK 
> action at all, this could save application to keep trying.
> 
> P2) PMD can make better internal choices if it gets more hint from application 
> about MARK action may be used or not.
> Application at least may say it won't use the MARK flow action at all.
> 
> 
> This patch uses offload flags infrastructure to solve above two problems, 
> solution (A) in Andrew's summary.
> 
> Although it may solve the issues, there are questions/concerns around using this 
> additional flag to control flow API, I also agree it may be confusing in the 
> design level although practically using flags can be simple.
> And this is not generic solution, what happen with META action question is 
> already hanging on in the thread, more flags? How many more can we add?
> 
> And also there is option an to use dynamic mbuf flags to detect the capability, 
> solution (C) in Andrew's summary, again it may solve the problem but it looks 
> again a workaround to solve same flow API design restriction, and this one is 
> not as simple as (A).
> 
> Overall the discussion seems going on circles without an agreed on decision.
> 
> 
> 
> What about trying to solve this with flow API return values,
> 
> If a flow rule is not supported at all by the NIC/PMD, it may return 
> '-ENO_WAY_JOSE', and application knows it can't be used at all, this may solve 
> the (P1) above.

I like it, but who is Jose?
We can also have a function to test if an action is supported or not at all.

> And if a flow rule can be supported for the given pattern, but it is not 
> supported right now because current configuration or resourcing restrictions 
> doesn't allow creating rule, a special error type can be returned with a 
> descriptive error log for application to response:
> -ECONFLICT, "Can't enable rule A when rule B is enabled"
> -EDATAPATH, "Can't enable this rule when vector datapath is used"
> -ERESOURCE, "Can't enable more than 3 rules"
> This may solve the (P2) partially.
> 
> I am not sure about second part, but at least first part shouldn't be too hard 
> to implement, and it is a generic solution, what do you think?

+1

Re: [dpdk-dev] [PATCH 1/2] ethdev: add flow action type update as an offload

[Ferruh Yigit]

On 2/17/2021 2:10 PM, Thomas Monjalon wrote:
> 17/02/2021 14:45, Ferruh Yigit:
>> On 7/3/2020 3:34 PM, Ferruh Yigit wrote:
>>> On 11/19/2019 11:09 AM, Thomas Monjalon wrote:
>>>> 19/11/2019 11:59, Andrew Rybchenko:
>>>>> On 11/19/19 12:50 PM, Thomas Monjalon wrote:
>>>>>> 19/11/2019 10:24, Andrew Rybchenko:
>>>>>>> On 11/8/19 4:30 PM, Thomas Monjalon wrote:
>>>>>>>> 08/11/2019 14:27, Andrew Rybchenko:
>>>>>>>>> On 11/8/19 4:17 PM, Thomas Monjalon wrote:
>>>>>>>>>> 08/11/2019 13:00, Andrew Rybchenko:
>>>>>>>>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote:
>>>>>>>>>>>> 08/11/2019 11:42, Andrew Rybchenko:
>>>>>>>>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote:
>>>>>>>>>>>>>> 08/11/2019 09:35, Andrew Rybchenko:
>>>>>>>>>>>>>>> The problem:
>>>>>>>>>>>>>>> ~~~~~~~~~~~~
>>>>>>>>>>>>>>> PMD wants to know before port start if application wants to
>>>>>>>>>>>>>>> to use flow MARK/FLAG in the future. It is required because:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1. HW may be configured in a different way to reserve resources
>>>>>>>>>>>>>>>      for MARK/FLAG delivery
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD
>>>>>>>>>>>>>>>      is faster, but does not support MARK)
>>>>>>>>>>>>>> Thank you for the clear problem statement.
>>>>>>>>>>>>>> I agree with it. This is a real design issue.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Discussed solutions:
>>>>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>>>>> May be it is not 100% clear since below are alternatives.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> A. Explicit Rx offload suggested by the patch.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions used.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field
>>>>>>>>>>>>>>>      and/or flag and PMD uses lookup to solve the problem) plus part
>>>>>>>>>>>>>>>      of (B) to discover if a feature is supported.
>>>>>>>>>>>>>> The dynamic field should be registered via a new API function
>>>>>>>>>>>>>> named '<feature>_init'.
>>>>>>>>>>>>>> It means the application must explicit request the feature.
>>>>>>>>>>>>>> I agree this is the way to go.
>>>>>>>>>>>>> If I understand your statement correctly, but (C) is not ideal since it
>>>>>>>>>>>>> looks global. If registered dynamic field of mbuf and is flag that
>>>>>>>>>>>>> the feature should be enabled, it is a flag to all ports/devices.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> All solutions require changes in applications which use these
>>>>>>>>>>>>>>> features. There is a deprecation notice in place which advertises
>>>>>>>>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to substitute
>>>>>>>>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since
>>>>>>>>>>>>>>> it should be combined with (B) in order to understand if
>>>>>>>>>>>>>>> the feature is supported.
>>>>>>>>>>>>>> I don't understand.
>>>>>>>>>>>>>> Application request and PMD support are two different things.
>>>>>>>>>>>>>> PMD support must be via rte_flow validation on a case by case anyway.
>>>>>>>>>>>>> I mean that application wants to understand if the feature is
>>>>>>>>>>>>> supported. Then, it wants to enable it. In the case of (B),
>>>>>>>>>>>>> if I understand the solution correctly, there is no explicit
>>>>>>>>>>>>> way to enable, PMD just detects it because of discovery is done
>>>>>>>>>>>>> (that's what I mean by "implicit" and it is a drawback from my
>>>>>>>>>>>>> point of view, but still could be considered). (C) solves the
>>>>>>>>>>>>> problem of (B).
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Advantages and drawbacks of solutions:
>>>>>>>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>>>>>>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already
>>>>>>>>>>>>>>>      have a way to request flow MARK using rte_flow API.
>>>>>>>>>>>>>>>      I don't fully agree that it is a duplication, but I agree
>>>>>>>>>>>>>>>      that it sounds like duplication and complicates a bit flow
>>>>>>>>>>>>>>>      MARK usage by applications. (B) complicates it as well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2. One more drawback of the solution (A) is the necessity of
>>>>>>>>>>>>>>>      similar solution for META and it eats one more offload bit.
>>>>>>>>>>>>>>>      Yes, that's true and I think it is not a problem.
>>>>>>>>>>>>>>>      It would make it easier for applications to find out if
>>>>>>>>>>>>>>>      either MARK or META is supported.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3. The main advantage of the solution (A) is simplicity.
>>>>>>>>>>>>>>>      It is simple for application to understand if it supported.
>>>>>>>>>>>>>>>      It is simple in PMD to understand that it is required.
>>>>>>>>>>>>>>>      It is simple to disable it - just reconfigure.
>>>>>>>>>>>>>>>      Also it is easier to document it - just mention that
>>>>>>>>>>>>>>>      the offload should be supported and enabled.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 4. The main advantage of the solution (B) is no "duplication".
>>>>>>>>>>>>>>>      I agree that it is valid argument. Solving the problem
>>>>>>>>>>>>>>>      without extra entities is always nice, but unfortunately
>>>>>>>>>>>>>>>      it is too complex in this case.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 5. The main drawback of the solution (B) is the complexity.
>>>>>>>>>>>>>>>      It is necessary to choose a flow rule which should be used
>>>>>>>>>>>>>>>      as a criteria. It could be hardware dependent.
>>>>>>>>>>>>>>>      Complex logic is require in PMD if it wants to address the
>>>>>>>>>>>>>>>      problem and control MARK delivery based on validated flow
>>>>>>>>>>>>>>>      rules. It adds dependency between start/stop processing and
>>>>>>>>>>>>>>>      flow rules validation code.
>>>>>>>>>>>>>>>      It is pretty complicated to document it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 6. Useless enabling of the offload in the case of solution (A)
>>>>>>>>>>>>>>>      if really used flow rules do not support MARK looks like
>>>>>>>>>>>>>>>      drawback as well, but easily mitigated by a combination
>>>>>>>>>>>>>>>      with solution (B) and only required if the application wants
>>>>>>>>>>>>>>>      to dive in the level of optimization and complexity and
>>>>>>>>>>>>>>>      makes sense if application knows required flow rules in
>>>>>>>>>>>>>>>      advance. So, it is not a problem in this case.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for
>>>>>>>>>>>>>>>      applications to understand if these features are supported,
>>>>>>>>>>>>>>>      but no drawbacks in PMD, since explicit criteria is used to
>>>>>>>>>>>>>>>      enable/disable (dynamic field/flag lookup).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 8. Solution (C) is nice since it avoids "duplication".
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 9. The main drawback of the solution (C) is asymmetry.
>>>>>>>>>>>>>>>      As it was discussed in the case of RX_TIMESTAMP
>>>>>>>>>>>>>>>      (if I remember it correctly):
>>>>>>>>>>>>>>>       - PMD advertises RX_TIMESTAMP offload capability
>>>>>>>>>>>>>>>       - application enables the offload
>>>>>>>>>>>>>>>       - PMD registers dynamic field for timestamp
>>>>>>>>>>>>>>>      Solution (C):
>>>>>>>>>>>>>>>        - PMD advertises nothing
>>>>>>>>>>>>>>>        - application uses solution (B) to understand if
>>>>>>>>>>>>>>>          these features are supported
>>>>>>>>>>>>>>>        - application registers dynamic field/flag
>>>>>>>>>>>>>>>        - PMD does lookup and solve the problem
>>>>>>>>>>>>>>>      The asymmetry could be partially mitigated if RX_TIMESTAMP
>>>>>>>>>>>>>>>      solution is changed to require an application to register
>>>>>>>>>>>>>>>      dynamic fields and PMD to do lookup if the offload is
>>>>>>>>>>>>>>>      enabled. So, the only difference will be in no offload
>>>>>>>>>>>>>>>      in the case of flow MARK/FLAG and usage of complex logic
>>>>>>>>>>>>>>>      to understand if it is supported or no.
>>>>>>>>>>>>>>>      May be it would be really good since it will allow to
>>>>>>>>>>>>>>>      have dynamic fields registered before mempool population.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity.
>>>>>>>>>>>>>>>       Solution (A) may be per queue while (B) and (C) cannot be
>>>>>>>>>>>>>>>       per queue. Moreover (C) looks global - for all devices.
>>>>>>>>>>>>>>>       It could be really painful.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (C) is nice, but I still vote for simplicity and
>>>>>>>>>>>>>>> granularity of (A).
>>>>>>>>>>>>>> I vote for clear separation of application needs and PMD support,
>>>>>>>>>>>>>> by using the method C (dynamic fields).
>>>>>>>>>>>>>> I agree timestamp must use the same path.
>>>>>>>>>>>>>> I agree it's complicate because we don't know in advance whether
>>>>>>>>>>>>>> a flow rule will be accepted, but that's the reality, config is complex.
>>>>>>>>>>>>> Do you think that global nature of the (C) is acceptable?
>>>>>>>>>>>> That's a good question.
>>>>>>>>>>>> Maybe the feature request should be per port.
>>>>>>>>>>>> In this case, we are back to solution A with a flag per port?
>>>>>>>>>>> Offloads are natively per-queue as well, so (A) keeps the choice
>>>>>>>>>>> between per-port vs per-queue to PMDs as usual.
>>>>>>>>>>>
>>>>>>>>>>>> Note that A and C will not guarantee that the offload will be possible.
>>>>>>>>>>> Yes, definitely.
>>>>>>>>>>>
>>>>>>>>>>>> We need B (flow rule validation) anyway.
>>>>>>>>>>> Strictly speaking (B) (checking flow rules before device
>>>>>>>>>>> startup) is required if an application can predict flow
>>>>>>>>>>> rules and wants to ensure that MARK offload will be usable.
>>>>>>>>>>> Otherwise, it may be skipped.
>>>>>>>>>> No no, I mean flow rule validation MUST be used anyway
>>>>>>>>>> during the runtime before applying a rule.
>>>>>>>>>> I agree it is hard to predict. I speak only about real rules.
>>>>>>>>> OK, I see. Of course, flow rule validation is required at runtime.
>>>>>>>>> I was rather concentrated on the stated problem solutions.
>>>>>>>>>
>>>>>>>>>>>> It seems A, B, C are not alternatives but all required
>>>>>>>>>>>> as pieces of a puzzle...
>>>>>>>>>>> Unfortunately true in the most complex case.
>>>>>>>>>>> Right now it will be A with B if required as explained above.
>>>>>>>>>>> C will come a bit later when the field migrates to dynamic.
>>>>>>>>>>>
>>>>>>>>>>> May be it is even better if application registers dynamic
>>>>>>>>>>> fields before an attempt to enable offload to be sure that
>>>>>>>>>>> it will not fail because of impossibility to register
>>>>>>>>>>> dynamic field (lack of space). I'm not sure, but it is not
>>>>>>>>>>> not that important.
>>>>>>>>>> Yes of course, lack of mbuf space is another reason for
>>>>>>>>>> disabling the feature.
>>>>>>>>>>
>>>>>>>>>>> If we finally go way A, should we add offloads for META back?
>>>>>>>>>>> I guess separate Rx and Tx are required.
>>>>>>>>>> I would prefer to add it as dynamic flags.
>>>>>>>>>> Why rushing on a very temporary solution while it is not a new issue?
>>>>>>>>> Basically it means that we go just (B)+(C) in the case of META.
>>>>>>>>> I have no strong opinion but thought that it could be better to
>>>>>>>>> align the solution. Of course, we can wait with it. As I understand
>>>>>>>>> META is an experimental feature.
>>>>>>>> Yes it is experimental and I think it is too late to align now.
>>>>>>>>
>>>>>>>> Anyway, we will probably to discuss again these offloads TAG/MARK/META,
>>>>>>>> as requested by several people.
>>>>>>>>
>>>>>>> The series implements (A) to help to solve the problem described above.
>>>>>>> What is the fate of the series in v19.11 in accordance with the
>>>>>>> discussion?
>>>>>> I am against adding anything related to a feature union'ed in mbuf.
>>>>>> The feature must move to dynamic field first.
>>>>>>
>>>>>> In addition, such capability is very weak.
>>>>>> I am not sure it is a good idea to have some weak capabilities,
>>>>>> meaning a feature could be available but not in all cases.
>>>>>> I think we should discuss more generally how we want to handle
>>>>>> the rte_flow capabilities conveniently and reliably.
>>>>>
>>>>> It is really unexpected outcome from the above discussion.
>>>>
>>>> I'm sorry, I thought I was clear in my request to switch to dynamic first.
>>>>
>>>>
>>>>> It is just possibility to deliver and handle marks on datapath and
>>>>> request to have it. It says almost nothing about rte_flow rules
>>>>> supported etc. I'll be happy to take part in the discussion.
>>>>>
>>>>>> So regarding 19.11, as this feature is not new, it can wait 20.02.
>>>>>
>>>>> OK, it is not critical for me, so I don't mind, however, I've seen
>>>>> patches which try to use it [1] except net/octeontx2 in the second
>>>>> patch of the series.
>>>>>
>>>>> [1] https://patches.dpdk.org/patch/62415/
>>>>
>>>
>>> Sorry, I have to resurrect this old (long) discussion because the patches are
>>> still active in the patchwork [1] and the deprecation notice is still there [2].
>>>
>>> Andrew has a good summary in the thread [3], after a year nothing seems changed.
>>>
>>>
>>> Pavan, Thomas, Andrew, Ori,
>>>
>>> What is our plan with this series, lets try to have a conclusion.
>>>
>>>
>>>
>>>
>>> [1]
>>> https://patches.dpdk.org/user/todo/dpdk/?series=7076
>>>
>>> [2]
>>> http://lxr.dpdk.org/dpdk/v20.05/source/doc/guides/rel_notes/deprecation.rst#L88
>>>
>>> [3]
>>> http://inbox.dpdk.org/dev/f170105b-9c60-1b04-cb18-52e0951ddcdb@solarflare.com/
>>>
>>>
>>
>> I re-read the thread, will try to have a little movement while we are in the new
>> release cycle, if there is no update I am planning to reject the patches.
>>
>> There seems two problems:
>>
>> P1) Application will keep trying to program NIC for MARK action for each flow,
>> since application doesn't know if next one will succeed or not.
>> If only there would be a way to find out that NIC/PMD doesn't support the MARK
>> action at all, this could save application to keep trying.
>>
>> P2) PMD can make better internal choices if it gets more hint from application
>> about MARK action may be used or not.
>> Application at least may say it won't use the MARK flow action at all.
>>
>>
>> This patch uses offload flags infrastructure to solve above two problems,
>> solution (A) in Andrew's summary.
>>
>> Although it may solve the issues, there are questions/concerns around using this
>> additional flag to control flow API, I also agree it may be confusing in the
>> design level although practically using flags can be simple.
>> And this is not generic solution, what happen with META action question is
>> already hanging on in the thread, more flags? How many more can we add?
>>
>> And also there is option an to use dynamic mbuf flags to detect the capability,
>> solution (C) in Andrew's summary, again it may solve the problem but it looks
>> again a workaround to solve same flow API design restriction, and this one is
>> not as simple as (A).
>>
>> Overall the discussion seems going on circles without an agreed on decision.
>>
>>
>>
>> What about trying to solve this with flow API return values,
>>
>> If a flow rule is not supported at all by the NIC/PMD, it may return
>> '-ENO_WAY_JOSE', and application knows it can't be used at all, this may solve
>> the (P1) above.
> 
> I like it, but who is Jose?
> We can also have a function to test if an action is supported or not at all.
> 
>> And if a flow rule can be supported for the given pattern, but it is not
>> supported right now because current configuration or resourcing restrictions
>> doesn't allow creating rule, a special error type can be returned with a
>> descriptive error log for application to response:
>> -ECONFLICT, "Can't enable rule A when rule B is enabled"
>> -EDATAPATH, "Can't enable this rule when vector datapath is used"
>> -ERESOURCE, "Can't enable more than 3 rules"
>> This may solve the (P2) partially.
>>
>> I am not sure about second part, but at least first part shouldn't be too hard
>> to implement, and it is a generic solution, what do you think?
> 
> +1
> 

This is an old patch, and there was no update/response during the release 
time-frame.
I am rejecting in patchwork, please send a new version if there is still demand 
for it.