[dpdk-dev] [PATCH] net/enic: enable GENEVE offload via VNIC configuration

[dpdk-dev] [PATCH] net/enic: enable GENEVE offload via VNIC configuration

[John Daley]

The admin-configured vNIC settings (i.e. via CIMC or UCSM) now include
Geneve offload. Use that setting to decide whether to enable or
disable Geneve offload and remove the devarg 'geneve-opt'.

Also, the firmware now allows the driver to change the Geneve port
number. So extend udp_tunnel_port_{add,del} to accept Geneve port, in
addition to VXLAN.

Fixes: 93fb21fdbe23 ("net/enic: enable overlay offload for VXLAN and GENEVE")
Cc: stable@dpdk.org

Signed-off-by: John Daley <johndale@cisco.com>
Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>
---
 doc/guides/nics/enic.rst          |  32 +++---
 drivers/net/enic/base/vnic_dev.c  |   2 +-
 drivers/net/enic/base/vnic_enet.h |   1 +
 drivers/net/enic/enic.h           |   4 +-
 drivers/net/enic/enic_ethdev.c    |  71 +++++++------
 drivers/net/enic/enic_main.c      | 161 +++++++++++++++++-------------
 drivers/net/enic/enic_res.c       |   7 +-
 7 files changed, 154 insertions(+), 124 deletions(-)

diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index 4e7629c5cd..91bdcd065a 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -294,35 +294,31 @@ inner and outer packets can be IPv4 or IPv6.
 
   RSS hash calculation, therefore queue selection, is done on inner packets.
 
-In order to enable overlay offload, the 'Enable VXLAN' box should be checked
+In order to enable overlay offload, enable VXLAN and/or Geneve on vNIC
 via CIMC or UCSM followed by a reboot of the server. When PMD successfully
-enables overlay offload, it prints the following message on the console.
+enables overlay offload, it prints one of the following messages on the console.
 
 .. code-block:: console
 
-    Overlay offload is enabled
+    Overlay offload is enabled (VxLAN)
+    Overlay offload is enabled (Geneve)
+    Overlay offload is enabled (VxLAN, Geneve)
 
 By default, PMD enables overlay offload if hardware supports it. To disable
 it, set ``devargs`` parameter ``disable-overlay=1``. For example::
 
     -a 12:00.0,disable-overlay=1
 
-By default, the NIC uses 4789 as the VXLAN port. The user may change
-it through ``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as
-the current NIC has a single VXLAN port number, the user cannot
-configure multiple port numbers.
-
-Geneve headers with non-zero options are not supported by default. To
-use Geneve with options, update the VIC firmware to the latest version
-and then set ``devargs`` parameter ``geneve-opt=1``. When Geneve with
-options is enabled, flow API cannot be used as the features are
-currently mutually exclusive. When this feature is successfully
-enabled, PMD prints the following message.
-
-.. code-block:: console
-
-    Geneve with options is enabled
+By default, the NIC uses 4789 and 6081 as the VXLAN and Geneve ports,
+respectively. The user may change them through
+``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as the current
+NIC has a single VXLAN port number and a single Geneve port number,
+the user cannot configure multiple port numbers for each tunnel type.
 
+Geneve offload support has evolved over VIC models. On older models,
+Geneve offload and advanced filters are mutually exclusive.  This is
+enforced by UCSM and CIMC, which only allow one of the two features
+to be selected at one time. Newer VIC models do not have this restriction.
 
 Ingress VLAN Rewrite
 --------------------
diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c
index 526273cef5..55c08eb3dc 100644
--- a/drivers/net/enic/base/vnic_dev.c
+++ b/drivers/net/enic/base/vnic_dev.c
@@ -1318,7 +1318,7 @@ int vnic_dev_capable_geneve(struct vnic_dev *vdev)
 	int ret;
 
 	ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait);
-	return ret == 0 && (a1 & FEATURE_GENEVE_OPTIONS);
+	return ret == 0 && !!(a1 & FEATURE_GENEVE_OPTIONS);
 }
 
 uint64_t vnic_dev_capable_cq_entry_size(struct vnic_dev *vdev)
diff --git a/drivers/net/enic/base/vnic_enet.h b/drivers/net/enic/base/vnic_enet.h
index 7687951c90..2a97a33044 100644
--- a/drivers/net/enic/base/vnic_enet.h
+++ b/drivers/net/enic/base/vnic_enet.h
@@ -55,6 +55,7 @@ struct vnic_enet_config {
 #define VENETF_NICSWITCH        0x80000 /* NICSWITCH enabled */
 #define VENETF_RSSHASH_UDPIPV4  0x100000 /* Hash on UDP + IPv4 fields */
 #define VENETF_RSSHASH_UDPIPV6  0x200000 /* Hash on UDP + IPv6 fields */
+#define VENETF_GENEVE		0x400000 /* GENEVE offload */
 
 #define VENET_INTR_TYPE_MIN	0	/* Timer specs min interrupt spacing */
 #define VENET_INTR_TYPE_IDLE	1	/* Timer specs idle time before irq */
diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index cd66348f2f..47bfdac2cf 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -118,17 +118,17 @@ struct enic {
 	uint32_t flow_filter_mode;
 	uint8_t filter_actions; /* HW supported actions */
 	uint64_t cq_entry_sizes; /* supported CQ entry sizes */
+	bool geneve;
 	bool vxlan;
 	bool cq64;            /* actually using 64B CQ entry */
 	bool cq64_request;    /* devargs cq64=1 */
 	bool disable_overlay; /* devargs disable_overlay=1 */
 	uint8_t enable_avx2_rx;  /* devargs enable-avx2-rx=1 */
-	uint8_t geneve_opt_avail;    /* Geneve with options offload available */
-	uint8_t geneve_opt_enabled;  /* Geneve with options offload enabled */
 	uint8_t geneve_opt_request;  /* devargs geneve-opt=1 */
 	bool nic_cfg_chk;     /* NIC_CFG_CHK available */
 	bool udp_rss_weak;    /* Bodega style UDP RSS */
 	uint8_t ig_vlan_rewrite_mode; /* devargs ig-vlan-rewrite */
+	uint16_t geneve_port; /* current geneve port pushed to NIC */
 	uint16_t vxlan_port;  /* current vxlan port pushed to NIC */
 	int use_simple_tx_handler;
 	int use_noscatter_vec_rx_handler;
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index d91c2cdc8c..2109c6a4b8 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -11,6 +11,7 @@
 #include <rte_bus_pci.h>
 #include <ethdev_driver.h>
 #include <ethdev_pci.h>
+#include <rte_geneve.h>
 #include <rte_kvargs.h>
 #include <rte_string_fns.h>
 
@@ -67,7 +68,6 @@ static const struct vic_speed_capa {
 #define ENIC_DEVARG_CQ64 "cq64"
 #define ENIC_DEVARG_DISABLE_OVERLAY "disable-overlay"
 #define ENIC_DEVARG_ENABLE_AVX2_RX "enable-avx2-rx"
-#define ENIC_DEVARG_GENEVE_OPT "geneve-opt"
 #define ENIC_DEVARG_IG_VLAN_REWRITE "ig-vlan-rewrite"
 #define ENIC_DEVARG_REPRESENTOR "representor"
 
@@ -81,13 +81,6 @@ enicpmd_dev_flow_ops_get(struct rte_eth_dev *dev,
 
 	ENICPMD_FUNC_TRACE();
 
-	/*
-	 * Currently, when Geneve with options offload is enabled, host
-	 * cannot insert match-action rules.
-	 */
-	if (enic->geneve_opt_enabled)
-		return -ENOTSUP;
-
 	if (enic->flow_filter_mode == FILTER_FLOWMAN)
 		*ops = &enic_fm_flow_ops;
 	else
@@ -972,26 +965,32 @@ static int enicpmd_dev_rx_queue_intr_disable(struct rte_eth_dev *eth_dev,
 static int udp_tunnel_common_check(struct enic *enic,
 				   struct rte_eth_udp_tunnel *tnl)
 {
-	if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN)
+	if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN &&
+	    tnl->prot_type != RTE_TUNNEL_TYPE_GENEVE)
 		return -ENOTSUP;
 	if (!enic->overlay_offload) {
-		ENICPMD_LOG(DEBUG, " vxlan (overlay offload) is not "
-			     "supported\n");
+		ENICPMD_LOG(DEBUG, " overlay offload is not supported\n");
 		return -ENOTSUP;
 	}
 	return 0;
 }
 
-static int update_vxlan_port(struct enic *enic, uint16_t port)
+static int update_tunnel_port(struct enic *enic, uint16_t port, bool vxlan)
 {
-	if (vnic_dev_overlay_offload_cfg(enic->vdev,
-					 OVERLAY_CFG_VXLAN_PORT_UPDATE,
-					 port)) {
-		ENICPMD_LOG(DEBUG, " failed to update vxlan port\n");
+	uint8_t cfg;
+
+	cfg = vxlan ? OVERLAY_CFG_VXLAN_PORT_UPDATE :
+		OVERLAY_CFG_GENEVE_PORT_UPDATE;
+	if (vnic_dev_overlay_offload_cfg(enic->vdev, cfg, port)) {
+		ENICPMD_LOG(DEBUG, " failed to update tunnel port\n");
 		return -EINVAL;
 	}
-	ENICPMD_LOG(DEBUG, " updated vxlan port to %u\n", port);
-	enic->vxlan_port = port;
+	ENICPMD_LOG(DEBUG, " updated %s port to %u\n",
+		    vxlan ? "vxlan" : "geneve", port);
+	if (vxlan)
+		enic->vxlan_port = port;
+	else
+		enic->geneve_port = port;
 	return 0;
 }
 
@@ -999,34 +998,48 @@ static int enicpmd_dev_udp_tunnel_port_add(struct rte_eth_dev *eth_dev,
 					   struct rte_eth_udp_tunnel *tnl)
 {
 	struct enic *enic = pmd_priv(eth_dev);
+	uint16_t port;
+	bool vxlan;
 	int ret;
 
 	ENICPMD_FUNC_TRACE();
 	ret = udp_tunnel_common_check(enic, tnl);
 	if (ret)
 		return ret;
+	vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN);
+	if (vxlan)
+		port = enic->vxlan_port;
+	else
+		port = enic->geneve_port;
 	/*
-	 * The NIC has 1 configurable VXLAN port number. "Adding" a new port
-	 * number replaces it.
+	 * The NIC has 1 configurable port number per tunnel type.
+	 * "Adding" a new port number replaces it.
 	 */
-	if (tnl->udp_port == enic->vxlan_port || tnl->udp_port == 0) {
+	if (tnl->udp_port == port || tnl->udp_port == 0) {
 		ENICPMD_LOG(DEBUG, " %u is already configured or invalid\n",
 			     tnl->udp_port);
 		return -EINVAL;
 	}
-	return update_vxlan_port(enic, tnl->udp_port);
+	return update_tunnel_port(enic, tnl->udp_port, vxlan);
 }
 
 static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev,
 					   struct rte_eth_udp_tunnel *tnl)
 {
 	struct enic *enic = pmd_priv(eth_dev);
+	uint16_t port;
+	bool vxlan;
 	int ret;
 
 	ENICPMD_FUNC_TRACE();
 	ret = udp_tunnel_common_check(enic, tnl);
 	if (ret)
 		return ret;
+	vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN);
+	if (vxlan)
+		port = enic->vxlan_port;
+	else
+		port = enic->geneve_port;
 	/*
 	 * Clear the previously set port number and restore the
 	 * hardware default port number. Some drivers disable VXLAN
@@ -1034,12 +1047,13 @@ static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev,
 	 * enic does not do that as VXLAN is part of overlay offload,
 	 * which is tied to inner RSS and TSO.
 	 */
-	if (tnl->udp_port != enic->vxlan_port) {
-		ENICPMD_LOG(DEBUG, " %u is not a configured vxlan port\n",
+	if (tnl->udp_port != port) {
+		ENICPMD_LOG(DEBUG, " %u is not a configured tunnel port\n",
 			     tnl->udp_port);
 		return -EINVAL;
 	}
-	return update_vxlan_port(enic, RTE_VXLAN_DEFAULT_PORT);
+	port = vxlan ? RTE_VXLAN_DEFAULT_PORT : RTE_GENEVE_DEFAULT_PORT;
+	return update_tunnel_port(enic, port, vxlan);
 }
 
 static int enicpmd_dev_fw_version_get(struct rte_eth_dev *eth_dev,
@@ -1140,8 +1154,6 @@ static int enic_parse_zero_one(const char *key,
 		enic->disable_overlay = b;
 	if (strcmp(key, ENIC_DEVARG_ENABLE_AVX2_RX) == 0)
 		enic->enable_avx2_rx = b;
-	if (strcmp(key, ENIC_DEVARG_GENEVE_OPT) == 0)
-		enic->geneve_opt_request = b;
 	return 0;
 }
 
@@ -1184,7 +1196,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
 		ENIC_DEVARG_CQ64,
 		ENIC_DEVARG_DISABLE_OVERLAY,
 		ENIC_DEVARG_ENABLE_AVX2_RX,
-		ENIC_DEVARG_GENEVE_OPT,
 		ENIC_DEVARG_IG_VLAN_REWRITE,
 		ENIC_DEVARG_REPRESENTOR,
 		NULL};
@@ -1196,7 +1207,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
 	enic->cq64_request = true; /* Use 64B entry if available */
 	enic->disable_overlay = false;
 	enic->enable_avx2_rx = false;
-	enic->geneve_opt_request = false;
 	enic->ig_vlan_rewrite_mode = IG_VLAN_REWRITE_MODE_PASS_THRU;
 	if (!dev->device->devargs)
 		return 0;
@@ -1209,8 +1219,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
 			       enic_parse_zero_one, enic) < 0 ||
 	    rte_kvargs_process(kvlist, ENIC_DEVARG_ENABLE_AVX2_RX,
 			       enic_parse_zero_one, enic) < 0 ||
-	    rte_kvargs_process(kvlist, ENIC_DEVARG_GENEVE_OPT,
-			       enic_parse_zero_one, enic) < 0 ||
 	    rte_kvargs_process(kvlist, ENIC_DEVARG_IG_VLAN_REWRITE,
 			       enic_parse_ig_vlan_rewrite, enic) < 0) {
 		rte_kvargs_free(kvlist);
@@ -1386,5 +1394,4 @@ RTE_PMD_REGISTER_PARAM_STRING(net_enic,
 	ENIC_DEVARG_CQ64 "=0|1"
 	ENIC_DEVARG_DISABLE_OVERLAY "=0|1 "
 	ENIC_DEVARG_ENABLE_AVX2_RX "=0|1 "
-	ENIC_DEVARG_GENEVE_OPT "=0|1 "
 	ENIC_DEVARG_IG_VLAN_REWRITE "=trunk|untag|priority|pass");
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 2a06d46872..2affd380c6 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -16,6 +16,7 @@
 #include <rte_mbuf.h>
 #include <rte_string_fns.h>
 #include <ethdev_driver.h>
+#include <rte_geneve.h>
 
 #include "enic_compat.h"
 #include "enic.h"
@@ -1719,6 +1720,85 @@ int enic_set_mtu(struct enic *enic, uint16_t new_mtu)
 	return rc;
 }
 
+static void
+enic_disable_overlay_offload(struct enic *enic)
+{
+	/*
+	 * Disabling fails if the feature is provisioned but
+	 * not enabled. So ignore result and do not log error.
+	 */
+	if (enic->vxlan) {
+		vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_DISABLE);
+	}
+	if (enic->geneve) {
+		vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_DISABLE);
+	}
+}
+
+static int
+enic_enable_overlay_offload(struct enic *enic)
+{
+	if (enic->vxlan && vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_ENABLE) != 0) {
+		dev_err(NULL, "failed to enable VXLAN offload\n");
+		return -EINVAL;
+	}
+	if (enic->geneve && vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_ENABLE) != 0) {
+		dev_err(NULL, "failed to enable Geneve offload\n");
+		return -EINVAL;
+	}
+	enic->tx_offload_capa |=
+		DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+		(enic->geneve ? DEV_TX_OFFLOAD_GENEVE_TNL_TSO : 0) |
+		(enic->vxlan ? DEV_TX_OFFLOAD_VXLAN_TNL_TSO : 0);
+	enic->tx_offload_mask |=
+		PKT_TX_OUTER_IPV6 |
+		PKT_TX_OUTER_IPV4 |
+		PKT_TX_OUTER_IP_CKSUM |
+		PKT_TX_TUNNEL_MASK;
+	enic->overlay_offload = true;
+
+	if (enic->vxlan && enic->geneve)
+		dev_info(NULL, "Overlay offload is enabled (VxLAN, Geneve)\n");
+	else if (enic->vxlan)
+		dev_info(NULL, "Overlay offload is enabled (VxLAN)\n");
+	else
+		dev_info(NULL, "Overlay offload is enabled (Geneve)\n");
+
+	return 0;
+}
+
+static int
+enic_reset_overlay_port(struct enic *enic)
+{
+	if (enic->vxlan) {
+		enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT;
+		/*
+		 * Reset the vxlan port to the default, as the NIC firmware
+		 * does not reset it automatically and keeps the old setting.
+		 */
+		if (vnic_dev_overlay_offload_cfg(enic->vdev,
+						 OVERLAY_CFG_VXLAN_PORT_UPDATE,
+						 RTE_VXLAN_DEFAULT_PORT)) {
+			dev_err(enic, "failed to update vxlan port\n");
+			return -EINVAL;
+		}
+	}
+	if (enic->geneve) {
+		enic->geneve_port = RTE_GENEVE_DEFAULT_PORT;
+		if (vnic_dev_overlay_offload_cfg(enic->vdev,
+						 OVERLAY_CFG_GENEVE_PORT_UPDATE,
+						 RTE_GENEVE_DEFAULT_PORT)) {
+			dev_err(enic, "failed to update vxlan port\n");
+			return -EINVAL;
+		}
+	}
+	return 0;
+}
+
 static int enic_dev_init(struct enic *enic)
 {
 	int err;
@@ -1785,85 +1865,32 @@ static int enic_dev_init(struct enic *enic)
 	/* set up link status checking */
 	vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */
 
+	enic->overlay_offload = false;
 	/*
-	 * When Geneve with options offload is available, always disable it
-	 * first as it can interfere with user flow rules.
+	 * First, explicitly disable overlay offload as the setting is
+	 * sticky, and resetting vNIC may not disable it.
 	 */
-	if (enic->geneve_opt_avail) {
-		/*
-		 * Disabling fails if the feature is provisioned but
-		 * not enabled. So ignore result and do not log error.
-		 */
-		vnic_dev_overlay_offload_ctrl(enic->vdev,
-			OVERLAY_FEATURE_GENEVE,
-			OVERLAY_OFFLOAD_DISABLE);
-	}
-	enic->overlay_offload = false;
-	if (enic->disable_overlay && enic->vxlan) {
-		/*
-		 * Explicitly disable overlay offload as the setting is
-		 * sticky, and resetting vNIC does not disable it.
-		 */
-		if (vnic_dev_overlay_offload_ctrl(enic->vdev,
-						  OVERLAY_FEATURE_VXLAN,
-						  OVERLAY_OFFLOAD_DISABLE)) {
-			dev_err(enic, "failed to disable overlay offload\n");
-		} else {
-			dev_info(enic, "Overlay offload is disabled\n");
-		}
-	}
-	if (!enic->disable_overlay && enic->vxlan &&
-	    /* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */
-	    vnic_dev_overlay_offload_ctrl(enic->vdev,
-					  OVERLAY_FEATURE_VXLAN,
-					  OVERLAY_OFFLOAD_ENABLE) == 0) {
-		enic->tx_offload_capa |=
-			DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
-			DEV_TX_OFFLOAD_GENEVE_TNL_TSO |
-			DEV_TX_OFFLOAD_VXLAN_TNL_TSO;
-		enic->tx_offload_mask |=
-			PKT_TX_OUTER_IPV6 |
-			PKT_TX_OUTER_IPV4 |
-			PKT_TX_OUTER_IP_CKSUM |
-			PKT_TX_TUNNEL_MASK;
-		enic->overlay_offload = true;
-		dev_info(enic, "Overlay offload is enabled\n");
-	}
-	/* Geneve with options offload requires overlay offload */
-	if (enic->overlay_offload && enic->geneve_opt_avail &&
-	    enic->geneve_opt_request) {
-		if (vnic_dev_overlay_offload_ctrl(enic->vdev,
-				OVERLAY_FEATURE_GENEVE,
-				OVERLAY_OFFLOAD_ENABLE)) {
-			dev_err(enic, "failed to enable geneve+option\n");
-		} else {
-			enic->geneve_opt_enabled = 1;
-			dev_info(enic, "Geneve with options is enabled\n");
+	enic_disable_overlay_offload(enic);
+	/* Then, enable overlay offload according to vNIC flags */
+	if (!enic->disable_overlay && (enic->vxlan || enic->geneve)) {
+		err = enic_enable_overlay_offload(enic);
+		if (err) {
+			dev_info(NULL, "failed to enable overlay offload\n");
+			return err;
 		}
 	}
 	/*
-	 * Reset the vxlan port if HW vxlan parsing is available. It
+	 * Reset the vxlan/geneve port if HW parsing is available. It
 	 * is always enabled regardless of overlay offload
 	 * enable/disable.
 	 */
-	if (enic->vxlan) {
-		enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT;
-		/*
-		 * Reset the vxlan port to the default, as the NIC firmware
-		 * does not reset it automatically and keeps the old setting.
-		 */
-		if (vnic_dev_overlay_offload_cfg(enic->vdev,
-						 OVERLAY_CFG_VXLAN_PORT_UPDATE,
-						 RTE_VXLAN_DEFAULT_PORT)) {
-			dev_err(enic, "failed to update vxlan port\n");
-			return -EINVAL;
-		}
-	}
+	err = enic_reset_overlay_port(enic);
+	if (err)
+		return err;
 
 	if (enic_fm_init(enic))
 		dev_warning(enic, "Init of flowman failed.\n");
 	return 0;
-
 }
 
 static void lock_devcmd(void *priv)
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index 689bf748ae..a8f5332a40 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -179,10 +179,9 @@ int enic_get_vnic_config(struct enic *enic)
 
 	enic->vxlan = ENIC_SETTING(enic, VXLAN) &&
 		vnic_dev_capable_vxlan(enic->vdev);
-	if (vnic_dev_capable_geneve(enic->vdev)) {
-		dev_info(NULL, "Geneve with options offload available\n");
-		enic->geneve_opt_avail = 1;
-	}
+	enic->geneve = ENIC_SETTING(enic, GENEVE) &&
+		vnic_dev_capable_geneve(enic->vdev);
+
 	/* Supported CQ entry sizes */
 	enic->cq_entry_sizes = vnic_dev_capable_cq_entry_size(enic->vdev);
 	sizes = enic->cq_entry_sizes;
-- 
2.26.2

Re: [dpdk-dev] [PATCH] net/enic: enable GENEVE offload via VNIC configuration

[Ferruh Yigit]

On 5/5/2021 7:39 PM, John Daley wrote:
> The admin-configured vNIC settings (i.e. via CIMC or UCSM) now include
> Geneve offload. Use that setting to decide whether to enable or
> disable Geneve offload and remove the devarg 'geneve-opt'.
> 
> Also, the firmware now allows the driver to change the Geneve port
> number. So extend udp_tunnel_port_{add,del} to accept Geneve port, in
> addition to VXLAN.
> 
> Fixes: 93fb21fdbe23 ("net/enic: enable overlay offload for VXLAN and GENEVE")
> Cc: stable@dpdk.org
> 
> Signed-off-by: John Daley <johndale@cisco.com>
> Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>
> ---
>  doc/guides/nics/enic.rst          |  32 +++---
>  drivers/net/enic/base/vnic_dev.c  |   2 +-
>  drivers/net/enic/base/vnic_enet.h |   1 +
>  drivers/net/enic/enic.h           |   4 +-
>  drivers/net/enic/enic_ethdev.c    |  71 +++++++------
>  drivers/net/enic/enic_main.c      | 161 +++++++++++++++++-------------
>  drivers/net/enic/enic_res.c       |   7 +-
>  7 files changed, 154 insertions(+), 124 deletions(-)

Hi John, Andrew,

Can you please update release notes too?

[dpdk-dev] [PATCH v1] net/enic: enable GENEVE offload via VNIC configuration

[John Daley]

The admin-configured vNIC settings (i.e. via CIMC or UCSM) now include
Geneve offload. Use that setting to decide whether to enable or
disable Geneve offload and remove the devarg 'geneve-opt'.

Also, the firmware now allows the driver to change the Geneve port
number. So extend udp_tunnel_port_{add,del} to accept Geneve port, in
addition to VXLAN.

Fixes: 93fb21fdbe23 ("net/enic: enable overlay offload for VXLAN and GENEVE")
Cc: stable@dpdk.org

Signed-off-by: John Daley <johndale@cisco.com>
Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>
---

v1: update release notes

 doc/guides/nics/enic.rst               |  32 +++--
 doc/guides/rel_notes/release_20_05.rst |   7 ++
 drivers/net/enic/base/vnic_dev.c       |   2 +-
 drivers/net/enic/base/vnic_enet.h      |   1 +
 drivers/net/enic/enic.h                |   4 +-
 drivers/net/enic/enic_ethdev.c         |  71 ++++++-----
 drivers/net/enic/enic_main.c           | 161 +++++++++++++++----------
 drivers/net/enic/enic_res.c            |   7 +-
 8 files changed, 161 insertions(+), 124 deletions(-)

diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index 4e7629c5cd..91bdcd065a 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -294,35 +294,31 @@ inner and outer packets can be IPv4 or IPv6.
 
   RSS hash calculation, therefore queue selection, is done on inner packets.
 
-In order to enable overlay offload, the 'Enable VXLAN' box should be checked
+In order to enable overlay offload, enable VXLAN and/or Geneve on vNIC
 via CIMC or UCSM followed by a reboot of the server. When PMD successfully
-enables overlay offload, it prints the following message on the console.
+enables overlay offload, it prints one of the following messages on the console.
 
 .. code-block:: console
 
-    Overlay offload is enabled
+    Overlay offload is enabled (VxLAN)
+    Overlay offload is enabled (Geneve)
+    Overlay offload is enabled (VxLAN, Geneve)
 
 By default, PMD enables overlay offload if hardware supports it. To disable
 it, set ``devargs`` parameter ``disable-overlay=1``. For example::
 
     -a 12:00.0,disable-overlay=1
 
-By default, the NIC uses 4789 as the VXLAN port. The user may change
-it through ``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as
-the current NIC has a single VXLAN port number, the user cannot
-configure multiple port numbers.
-
-Geneve headers with non-zero options are not supported by default. To
-use Geneve with options, update the VIC firmware to the latest version
-and then set ``devargs`` parameter ``geneve-opt=1``. When Geneve with
-options is enabled, flow API cannot be used as the features are
-currently mutually exclusive. When this feature is successfully
-enabled, PMD prints the following message.
-
-.. code-block:: console
-
-    Geneve with options is enabled
+By default, the NIC uses 4789 and 6081 as the VXLAN and Geneve ports,
+respectively. The user may change them through
+``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as the current
+NIC has a single VXLAN port number and a single Geneve port number,
+the user cannot configure multiple port numbers for each tunnel type.
 
+Geneve offload support has evolved over VIC models. On older models,
+Geneve offload and advanced filters are mutually exclusive.  This is
+enforced by UCSM and CIMC, which only allow one of the two features
+to be selected at one time. Newer VIC models do not have this restriction.
 
 Ingress VLAN Rewrite
 --------------------
diff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst
index 985c845de4..81cd4b5d2e 100644
--- a/doc/guides/rel_notes/release_20_05.rst
+++ b/doc/guides/rel_notes/release_20_05.rst
@@ -178,6 +178,13 @@ New Features
   * Added support for flow patterns with wildcard VLAN items (without VID value).
   * Updated support for matching on GTP headers, added match on GTP flags.
 
+* **Updated Cisco enic driver.**
+
+  Updated Cisco enic driver GENEVE tunneling support:
+
+  * Added support to control GENEVE tunneling via UCSM/CIMC and removed devarg.
+  * Added GENEVE port number configuration.
+
 * **Added Chacha20-Poly1305 algorithm to Cryptodev API.**
 
   Added support for Chacha20-Poly1305 AEAD algorithm in Cryptodev.
diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c
index 526273cef5..55c08eb3dc 100644
--- a/drivers/net/enic/base/vnic_dev.c
+++ b/drivers/net/enic/base/vnic_dev.c
@@ -1318,7 +1318,7 @@ int vnic_dev_capable_geneve(struct vnic_dev *vdev)
 	int ret;
 
 	ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait);
-	return ret == 0 && (a1 & FEATURE_GENEVE_OPTIONS);
+	return ret == 0 && !!(a1 & FEATURE_GENEVE_OPTIONS);
 }
 
 uint64_t vnic_dev_capable_cq_entry_size(struct vnic_dev *vdev)
diff --git a/drivers/net/enic/base/vnic_enet.h b/drivers/net/enic/base/vnic_enet.h
index 7687951c90..2a97a33044 100644
--- a/drivers/net/enic/base/vnic_enet.h
+++ b/drivers/net/enic/base/vnic_enet.h
@@ -55,6 +55,7 @@ struct vnic_enet_config {
 #define VENETF_NICSWITCH        0x80000 /* NICSWITCH enabled */
 #define VENETF_RSSHASH_UDPIPV4  0x100000 /* Hash on UDP + IPv4 fields */
 #define VENETF_RSSHASH_UDPIPV6  0x200000 /* Hash on UDP + IPv6 fields */
+#define VENETF_GENEVE		0x400000 /* GENEVE offload */
 
 #define VENET_INTR_TYPE_MIN	0	/* Timer specs min interrupt spacing */
 #define VENET_INTR_TYPE_IDLE	1	/* Timer specs idle time before irq */
diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index cd66348f2f..47bfdac2cf 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -118,17 +118,17 @@ struct enic {
 	uint32_t flow_filter_mode;
 	uint8_t filter_actions; /* HW supported actions */
 	uint64_t cq_entry_sizes; /* supported CQ entry sizes */
+	bool geneve;
 	bool vxlan;
 	bool cq64;            /* actually using 64B CQ entry */
 	bool cq64_request;    /* devargs cq64=1 */
 	bool disable_overlay; /* devargs disable_overlay=1 */
 	uint8_t enable_avx2_rx;  /* devargs enable-avx2-rx=1 */
-	uint8_t geneve_opt_avail;    /* Geneve with options offload available */
-	uint8_t geneve_opt_enabled;  /* Geneve with options offload enabled */
 	uint8_t geneve_opt_request;  /* devargs geneve-opt=1 */
 	bool nic_cfg_chk;     /* NIC_CFG_CHK available */
 	bool udp_rss_weak;    /* Bodega style UDP RSS */
 	uint8_t ig_vlan_rewrite_mode; /* devargs ig-vlan-rewrite */
+	uint16_t geneve_port; /* current geneve port pushed to NIC */
 	uint16_t vxlan_port;  /* current vxlan port pushed to NIC */
 	int use_simple_tx_handler;
 	int use_noscatter_vec_rx_handler;
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index ab64480f39..8d5797523b 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -11,6 +11,7 @@
 #include <rte_bus_pci.h>
 #include <ethdev_driver.h>
 #include <ethdev_pci.h>
+#include <rte_geneve.h>
 #include <rte_kvargs.h>
 #include <rte_string_fns.h>
 
@@ -67,7 +68,6 @@ static const struct vic_speed_capa {
 #define ENIC_DEVARG_CQ64 "cq64"
 #define ENIC_DEVARG_DISABLE_OVERLAY "disable-overlay"
 #define ENIC_DEVARG_ENABLE_AVX2_RX "enable-avx2-rx"
-#define ENIC_DEVARG_GENEVE_OPT "geneve-opt"
 #define ENIC_DEVARG_IG_VLAN_REWRITE "ig-vlan-rewrite"
 #define ENIC_DEVARG_REPRESENTOR "representor"
 
@@ -81,13 +81,6 @@ enicpmd_dev_flow_ops_get(struct rte_eth_dev *dev,
 
 	ENICPMD_FUNC_TRACE();
 
-	/*
-	 * Currently, when Geneve with options offload is enabled, host
-	 * cannot insert match-action rules.
-	 */
-	if (enic->geneve_opt_enabled)
-		return -ENOTSUP;
-
 	if (enic->flow_filter_mode == FILTER_FLOWMAN)
 		*ops = &enic_fm_flow_ops;
 	else
@@ -972,26 +965,32 @@ static int enicpmd_dev_rx_queue_intr_disable(struct rte_eth_dev *eth_dev,
 static int udp_tunnel_common_check(struct enic *enic,
 				   struct rte_eth_udp_tunnel *tnl)
 {
-	if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN)
+	if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN &&
+	    tnl->prot_type != RTE_TUNNEL_TYPE_GENEVE)
 		return -ENOTSUP;
 	if (!enic->overlay_offload) {
-		ENICPMD_LOG(DEBUG, " vxlan (overlay offload) is not "
-			     "supported\n");
+		ENICPMD_LOG(DEBUG, " overlay offload is not supported\n");
 		return -ENOTSUP;
 	}
 	return 0;
 }
 
-static int update_vxlan_port(struct enic *enic, uint16_t port)
+static int update_tunnel_port(struct enic *enic, uint16_t port, bool vxlan)
 {
-	if (vnic_dev_overlay_offload_cfg(enic->vdev,
-					 OVERLAY_CFG_VXLAN_PORT_UPDATE,
-					 port)) {
-		ENICPMD_LOG(DEBUG, " failed to update vxlan port\n");
+	uint8_t cfg;
+
+	cfg = vxlan ? OVERLAY_CFG_VXLAN_PORT_UPDATE :
+		OVERLAY_CFG_GENEVE_PORT_UPDATE;
+	if (vnic_dev_overlay_offload_cfg(enic->vdev, cfg, port)) {
+		ENICPMD_LOG(DEBUG, " failed to update tunnel port\n");
 		return -EINVAL;
 	}
-	ENICPMD_LOG(DEBUG, " updated vxlan port to %u\n", port);
-	enic->vxlan_port = port;
+	ENICPMD_LOG(DEBUG, " updated %s port to %u\n",
+		    vxlan ? "vxlan" : "geneve", port);
+	if (vxlan)
+		enic->vxlan_port = port;
+	else
+		enic->geneve_port = port;
 	return 0;
 }
 
@@ -999,34 +998,48 @@ static int enicpmd_dev_udp_tunnel_port_add(struct rte_eth_dev *eth_dev,
 					   struct rte_eth_udp_tunnel *tnl)
 {
 	struct enic *enic = pmd_priv(eth_dev);
+	uint16_t port;
+	bool vxlan;
 	int ret;
 
 	ENICPMD_FUNC_TRACE();
 	ret = udp_tunnel_common_check(enic, tnl);
 	if (ret)
 		return ret;
+	vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN);
+	if (vxlan)
+		port = enic->vxlan_port;
+	else
+		port = enic->geneve_port;
 	/*
-	 * The NIC has 1 configurable VXLAN port number. "Adding" a new port
-	 * number replaces it.
+	 * The NIC has 1 configurable port number per tunnel type.
+	 * "Adding" a new port number replaces it.
 	 */
-	if (tnl->udp_port == enic->vxlan_port || tnl->udp_port == 0) {
+	if (tnl->udp_port == port || tnl->udp_port == 0) {
 		ENICPMD_LOG(DEBUG, " %u is already configured or invalid\n",
 			     tnl->udp_port);
 		return -EINVAL;
 	}
-	return update_vxlan_port(enic, tnl->udp_port);
+	return update_tunnel_port(enic, tnl->udp_port, vxlan);
 }
 
 static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev,
 					   struct rte_eth_udp_tunnel *tnl)
 {
 	struct enic *enic = pmd_priv(eth_dev);
+	uint16_t port;
+	bool vxlan;
 	int ret;
 
 	ENICPMD_FUNC_TRACE();
 	ret = udp_tunnel_common_check(enic, tnl);
 	if (ret)
 		return ret;
+	vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN);
+	if (vxlan)
+		port = enic->vxlan_port;
+	else
+		port = enic->geneve_port;
 	/*
 	 * Clear the previously set port number and restore the
 	 * hardware default port number. Some drivers disable VXLAN
@@ -1034,12 +1047,13 @@ static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev,
 	 * enic does not do that as VXLAN is part of overlay offload,
 	 * which is tied to inner RSS and TSO.
 	 */
-	if (tnl->udp_port != enic->vxlan_port) {
-		ENICPMD_LOG(DEBUG, " %u is not a configured vxlan port\n",
+	if (tnl->udp_port != port) {
+		ENICPMD_LOG(DEBUG, " %u is not a configured tunnel port\n",
 			     tnl->udp_port);
 		return -EINVAL;
 	}
-	return update_vxlan_port(enic, RTE_VXLAN_DEFAULT_PORT);
+	port = vxlan ? RTE_VXLAN_DEFAULT_PORT : RTE_GENEVE_DEFAULT_PORT;
+	return update_tunnel_port(enic, port, vxlan);
 }
 
 static int enicpmd_dev_fw_version_get(struct rte_eth_dev *eth_dev,
@@ -1145,8 +1159,6 @@ static int enic_parse_zero_one(const char *key,
 		enic->disable_overlay = b;
 	if (strcmp(key, ENIC_DEVARG_ENABLE_AVX2_RX) == 0)
 		enic->enable_avx2_rx = b;
-	if (strcmp(key, ENIC_DEVARG_GENEVE_OPT) == 0)
-		enic->geneve_opt_request = b;
 	return 0;
 }
 
@@ -1189,7 +1201,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
 		ENIC_DEVARG_CQ64,
 		ENIC_DEVARG_DISABLE_OVERLAY,
 		ENIC_DEVARG_ENABLE_AVX2_RX,
-		ENIC_DEVARG_GENEVE_OPT,
 		ENIC_DEVARG_IG_VLAN_REWRITE,
 		ENIC_DEVARG_REPRESENTOR,
 		NULL};
@@ -1201,7 +1212,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
 	enic->cq64_request = true; /* Use 64B entry if available */
 	enic->disable_overlay = false;
 	enic->enable_avx2_rx = false;
-	enic->geneve_opt_request = false;
 	enic->ig_vlan_rewrite_mode = IG_VLAN_REWRITE_MODE_PASS_THRU;
 	if (!dev->device->devargs)
 		return 0;
@@ -1214,8 +1224,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
 			       enic_parse_zero_one, enic) < 0 ||
 	    rte_kvargs_process(kvlist, ENIC_DEVARG_ENABLE_AVX2_RX,
 			       enic_parse_zero_one, enic) < 0 ||
-	    rte_kvargs_process(kvlist, ENIC_DEVARG_GENEVE_OPT,
-			       enic_parse_zero_one, enic) < 0 ||
 	    rte_kvargs_process(kvlist, ENIC_DEVARG_IG_VLAN_REWRITE,
 			       enic_parse_ig_vlan_rewrite, enic) < 0) {
 		rte_kvargs_free(kvlist);
@@ -1391,5 +1399,4 @@ RTE_PMD_REGISTER_PARAM_STRING(net_enic,
 	ENIC_DEVARG_CQ64 "=0|1"
 	ENIC_DEVARG_DISABLE_OVERLAY "=0|1 "
 	ENIC_DEVARG_ENABLE_AVX2_RX "=0|1 "
-	ENIC_DEVARG_GENEVE_OPT "=0|1 "
 	ENIC_DEVARG_IG_VLAN_REWRITE "=trunk|untag|priority|pass");
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 2a06d46872..2affd380c6 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -16,6 +16,7 @@
 #include <rte_mbuf.h>
 #include <rte_string_fns.h>
 #include <ethdev_driver.h>
+#include <rte_geneve.h>
 
 #include "enic_compat.h"
 #include "enic.h"
@@ -1719,6 +1720,85 @@ int enic_set_mtu(struct enic *enic, uint16_t new_mtu)
 	return rc;
 }
 
+static void
+enic_disable_overlay_offload(struct enic *enic)
+{
+	/*
+	 * Disabling fails if the feature is provisioned but
+	 * not enabled. So ignore result and do not log error.
+	 */
+	if (enic->vxlan) {
+		vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_DISABLE);
+	}
+	if (enic->geneve) {
+		vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_DISABLE);
+	}
+}
+
+static int
+enic_enable_overlay_offload(struct enic *enic)
+{
+	if (enic->vxlan && vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_ENABLE) != 0) {
+		dev_err(NULL, "failed to enable VXLAN offload\n");
+		return -EINVAL;
+	}
+	if (enic->geneve && vnic_dev_overlay_offload_ctrl(enic->vdev,
+			OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_ENABLE) != 0) {
+		dev_err(NULL, "failed to enable Geneve offload\n");
+		return -EINVAL;
+	}
+	enic->tx_offload_capa |=
+		DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+		(enic->geneve ? DEV_TX_OFFLOAD_GENEVE_TNL_TSO : 0) |
+		(enic->vxlan ? DEV_TX_OFFLOAD_VXLAN_TNL_TSO : 0);
+	enic->tx_offload_mask |=
+		PKT_TX_OUTER_IPV6 |
+		PKT_TX_OUTER_IPV4 |
+		PKT_TX_OUTER_IP_CKSUM |
+		PKT_TX_TUNNEL_MASK;
+	enic->overlay_offload = true;
+
+	if (enic->vxlan && enic->geneve)
+		dev_info(NULL, "Overlay offload is enabled (VxLAN, Geneve)\n");
+	else if (enic->vxlan)
+		dev_info(NULL, "Overlay offload is enabled (VxLAN)\n");
+	else
+		dev_info(NULL, "Overlay offload is enabled (Geneve)\n");
+
+	return 0;
+}
+
+static int
+enic_reset_overlay_port(struct enic *enic)
+{
+	if (enic->vxlan) {
+		enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT;
+		/*
+		 * Reset the vxlan port to the default, as the NIC firmware
+		 * does not reset it automatically and keeps the old setting.
+		 */
+		if (vnic_dev_overlay_offload_cfg(enic->vdev,
+						 OVERLAY_CFG_VXLAN_PORT_UPDATE,
+						 RTE_VXLAN_DEFAULT_PORT)) {
+			dev_err(enic, "failed to update vxlan port\n");
+			return -EINVAL;
+		}
+	}
+	if (enic->geneve) {
+		enic->geneve_port = RTE_GENEVE_DEFAULT_PORT;
+		if (vnic_dev_overlay_offload_cfg(enic->vdev,
+						 OVERLAY_CFG_GENEVE_PORT_UPDATE,
+						 RTE_GENEVE_DEFAULT_PORT)) {
+			dev_err(enic, "failed to update vxlan port\n");
+			return -EINVAL;
+		}
+	}
+	return 0;
+}
+
 static int enic_dev_init(struct enic *enic)
 {
 	int err;
@@ -1785,85 +1865,32 @@ static int enic_dev_init(struct enic *enic)
 	/* set up link status checking */
 	vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */
 
+	enic->overlay_offload = false;
 	/*
-	 * When Geneve with options offload is available, always disable it
-	 * first as it can interfere with user flow rules.
+	 * First, explicitly disable overlay offload as the setting is
+	 * sticky, and resetting vNIC may not disable it.
 	 */
-	if (enic->geneve_opt_avail) {
-		/*
-		 * Disabling fails if the feature is provisioned but
-		 * not enabled. So ignore result and do not log error.
-		 */
-		vnic_dev_overlay_offload_ctrl(enic->vdev,
-			OVERLAY_FEATURE_GENEVE,
-			OVERLAY_OFFLOAD_DISABLE);
-	}
-	enic->overlay_offload = false;
-	if (enic->disable_overlay && enic->vxlan) {
-		/*
-		 * Explicitly disable overlay offload as the setting is
-		 * sticky, and resetting vNIC does not disable it.
-		 */
-		if (vnic_dev_overlay_offload_ctrl(enic->vdev,
-						  OVERLAY_FEATURE_VXLAN,
-						  OVERLAY_OFFLOAD_DISABLE)) {
-			dev_err(enic, "failed to disable overlay offload\n");
-		} else {
-			dev_info(enic, "Overlay offload is disabled\n");
-		}
-	}
-	if (!enic->disable_overlay && enic->vxlan &&
-	    /* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */
-	    vnic_dev_overlay_offload_ctrl(enic->vdev,
-					  OVERLAY_FEATURE_VXLAN,
-					  OVERLAY_OFFLOAD_ENABLE) == 0) {
-		enic->tx_offload_capa |=
-			DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
-			DEV_TX_OFFLOAD_GENEVE_TNL_TSO |
-			DEV_TX_OFFLOAD_VXLAN_TNL_TSO;
-		enic->tx_offload_mask |=
-			PKT_TX_OUTER_IPV6 |
-			PKT_TX_OUTER_IPV4 |
-			PKT_TX_OUTER_IP_CKSUM |
-			PKT_TX_TUNNEL_MASK;
-		enic->overlay_offload = true;
-		dev_info(enic, "Overlay offload is enabled\n");
-	}
-	/* Geneve with options offload requires overlay offload */
-	if (enic->overlay_offload && enic->geneve_opt_avail &&
-	    enic->geneve_opt_request) {
-		if (vnic_dev_overlay_offload_ctrl(enic->vdev,
-				OVERLAY_FEATURE_GENEVE,
-				OVERLAY_OFFLOAD_ENABLE)) {
-			dev_err(enic, "failed to enable geneve+option\n");
-		} else {
-			enic->geneve_opt_enabled = 1;
-			dev_info(enic, "Geneve with options is enabled\n");
+	enic_disable_overlay_offload(enic);
+	/* Then, enable overlay offload according to vNIC flags */
+	if (!enic->disable_overlay && (enic->vxlan || enic->geneve)) {
+		err = enic_enable_overlay_offload(enic);
+		if (err) {
+			dev_info(NULL, "failed to enable overlay offload\n");
+			return err;
 		}
 	}
 	/*
-	 * Reset the vxlan port if HW vxlan parsing is available. It
+	 * Reset the vxlan/geneve port if HW parsing is available. It
 	 * is always enabled regardless of overlay offload
 	 * enable/disable.
 	 */
-	if (enic->vxlan) {
-		enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT;
-		/*
-		 * Reset the vxlan port to the default, as the NIC firmware
-		 * does not reset it automatically and keeps the old setting.
-		 */
-		if (vnic_dev_overlay_offload_cfg(enic->vdev,
-						 OVERLAY_CFG_VXLAN_PORT_UPDATE,
-						 RTE_VXLAN_DEFAULT_PORT)) {
-			dev_err(enic, "failed to update vxlan port\n");
-			return -EINVAL;
-		}
-	}
+	err = enic_reset_overlay_port(enic);
+	if (err)
+		return err;
 
 	if (enic_fm_init(enic))
 		dev_warning(enic, "Init of flowman failed.\n");
 	return 0;
-
 }
 
 static void lock_devcmd(void *priv)
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index 689bf748ae..a8f5332a40 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -179,10 +179,9 @@ int enic_get_vnic_config(struct enic *enic)
 
 	enic->vxlan = ENIC_SETTING(enic, VXLAN) &&
 		vnic_dev_capable_vxlan(enic->vdev);
-	if (vnic_dev_capable_geneve(enic->vdev)) {
-		dev_info(NULL, "Geneve with options offload available\n");
-		enic->geneve_opt_avail = 1;
-	}
+	enic->geneve = ENIC_SETTING(enic, GENEVE) &&
+		vnic_dev_capable_geneve(enic->vdev);
+
 	/* Supported CQ entry sizes */
 	enic->cq_entry_sizes = vnic_dev_capable_cq_entry_size(enic->vdev);
 	sizes = enic->cq_entry_sizes;
-- 
2.26.2

Re: [dpdk-dev] [PATCH v1] net/enic: enable GENEVE offload via VNIC configuration

[Ferruh Yigit]

On 5/11/2021 8:25 PM, John Daley wrote:
> The admin-configured vNIC settings (i.e. via CIMC or UCSM) now include
> Geneve offload. Use that setting to decide whether to enable or
> disable Geneve offload and remove the devarg 'geneve-opt'.
> 
> Also, the firmware now allows the driver to change the Geneve port
> number. So extend udp_tunnel_port_{add,del} to accept Geneve port, in
> addition to VXLAN.
> 
> Fixes: 93fb21fdbe23 ("net/enic: enable overlay offload for VXLAN and GENEVE")
> Cc: stable@dpdk.org
> 
> Signed-off-by: John Daley <johndale@cisco.com>
> Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>

Applied to dpdk-next-net/main, thanks.