[dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC

[dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

The firmware on new hardware models flushes the global descriptor
cache by default. Use CMD_OPENF_IG_DESCCACHE to avoid cache
flushing. This flag has no effect on older models.

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Suggested-by: Govindarajulu Varadarajan <gvaradar@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
---
 drivers/net/enic/base/vnic_devcmd.h | 1 +
 drivers/net/enic/enic_main.c        | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h
index 6b95bc488..a63624559 100644
--- a/drivers/net/enic/base/vnic_devcmd.h
+++ b/drivers/net/enic/base/vnic_devcmd.h
@@ -600,6 +600,7 @@ enum filter_cap_mode {
 
 /* flags for CMD_OPEN */
 #define CMD_OPENF_OPROM		0x1	/* open coming from option rom */
+#define CMD_OPENF_IG_DESCCACHE	0x2	/* Do not flush IG DESC cache */
 
 /* flags for CMD_INIT */
 #define CMD_INITF_DEFAULT_MAC	0x1	/* init with default mac addr */
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 69ad42555..e908e20f2 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -983,9 +983,10 @@ static int enic_dev_wait(struct vnic_dev *vdev,
 static int enic_dev_open(struct enic *enic)
 {
 	int err;
+	int flags = CMD_OPENF_IG_DESCCACHE;
 
 	err = enic_dev_wait(enic->vdev, vnic_dev_open,
-		vnic_dev_open_done, 0);
+		vnic_dev_open_done, flags);
 	if (err)
 		dev_err(enic_get_dev(enic),
 			"vNIC device open failed, err %d\n", err);
-- 
2.16.2

[dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

Recent NIC models support overlay offload. The overlay offload
feature enables the following on the NIC.
- Rx/Tx checksum offloads for both inner and outer packets.
- Rx inner packet type classification.
- TSO.
- Inner RSS.

TX descriptors do not require any changes, except the header length
for TSO. The NIC parses outer/inner packets and performs offloads on
them as necessary. The header length for tunneled TSO includes both
inner and outer headers.

The NIC actually parses and performs the above for NVGRE as well. DPDK
currently has no offload flags for NVGRE, and the hardware has no
controls to individually enable tunnel types either. So do nothing for
now.

Add a config flag to enable overlay offload by default. To disable it,
the user should set it to 'n'.

CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y

Also update the enic guide doc.

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
---
 config/common_base                  |   1 +
 doc/guides/nics/enic.rst            |  52 ++++++++++++++++++
 drivers/net/enic/base/vnic_dev.c    |  33 ++++++++++++
 drivers/net/enic/base/vnic_dev.h    |   5 +-
 drivers/net/enic/base/vnic_devcmd.h |  12 +++++
 drivers/net/enic/base/vnic_wq.h     |   1 +
 drivers/net/enic/enic.h             |   6 +++
 drivers/net/enic/enic_ethdev.c      |  21 ++------
 drivers/net/enic/enic_main.c        |  25 +++++++++
 drivers/net/enic/enic_res.c         |  23 ++++++++
 drivers/net/enic/enic_rxtx.c        | 104 ++++++++++++++++++++++++++++--------
 11 files changed, 241 insertions(+), 42 deletions(-)

diff --git a/config/common_base b/config/common_base
index c09c7cf88..964e37b6e 100644
--- a/config/common_base
+++ b/config/common_base
@@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n
 # Compile burst-oriented Cisco ENIC PMD driver
 #
 CONFIG_RTE_LIBRTE_ENIC_PMD=y
+CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y
 
 #
 # Compile burst-oriented IGB & EM PMD drivers
diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index 0bc55936a..133b1e682 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -52,6 +52,10 @@ Configuration information
   - **CONFIG_RTE_LIBRTE_ENIC_PMD** (default y): Enables or disables inclusion
     of the ENIC PMD driver in the DPDK compilation.
 
+  - **CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD** (default y): Enables or
+    disables overlay offload.
+    Please see :ref:`Overlay Offload <overlay_offload>`.
+
 - **vNIC Configuration Parameters**
 
   - **Number of Queues**
@@ -254,6 +258,49 @@ Generic Flow API is supported. The baseline support is:
 More features may be added in future firmware and new versions of the VIC.
 Please refer to the release notes.
 
+.. _overlay_offload:
+
+Overlay Offload
+---------------
+
+Recent hardware models support overlay offload. When enabled, the NIC performs
+the following operations for VXLAN, NVGRE, and GENEVE packets. In all cases,
+inner and outer packets can be IPv4 or IPv6.
+
+- TSO for VXLAN and GENEVE packets.
+
+  Hardware supports NVGRE TSO, but DPDK currently has no NVGRE offload flags.
+
+- Tx checksum offloads.
+
+  The NIC fills in IPv4/UDP/TCP checksums for both inner and outer packets.
+
+- Rx checksum offloads.
+
+  The NIC validates IPv4/UDP/TCP checksums of both inner and outer packets.
+  Good checksum flags (e.g. ``PKT_RX_L4_CKSUM_GOOD``) indicate that the inner
+  packet has the correct checksum, and if applicable, the outer packet also
+  has the correct checksum. Bad checksum flags (e.g. ``PKT_RX_L4_CKSUM_BAD``)
+  indicate that the inner and/or outer packets have invalid checksum values.
+
+- Inner Rx packet type classification
+
+  PMD sets inner L3/L4 packet types (e.g. ``RTE_PTYPE_INNER_L4_TCP``), and
+  ``RTE_PTYPE_TUNNEL_GRENAT`` to indicate that the packet is tunneled.
+  PMD does not set L3/L4 packet types for outer packets.
+
+- Inner RSS
+
+  RSS hash calculation, therefore queue selection, is done on inner packets.
+
+In order to enable overlay offload, the 'Enable VXLAN' box should be checked
+via CIMC or UCSM followed by a reboot of the server. When PMD successfully
+enables overlay offload, it prints the following message on the console.
+
+.. code-block:: console
+
+    Overlay offload is enabled
+
 .. _enic_limitations:
 
 Limitations
@@ -373,6 +420,11 @@ Supported features
 - MTU update
 - SR-IOV on UCS managed servers connected to Fabric Interconnects
 - Flow API
+- Overlay offload
+
+  - Rx/Tx checksum offloads for VXLAN, NVGRE, GENEVE
+  - TSO for VXLAN and GENEVE packets
+  - Inner RSS
 
 Known bugs and unsupported features in this release
 ---------------------------------------------------
diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c
index 1f8d222fc..386db6411 100644
--- a/drivers/net/enic/base/vnic_dev.c
+++ b/drivers/net/enic/base/vnic_dev.c
@@ -1048,3 +1048,36 @@ int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry,
 
 	return ret;
 }
+
+int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev, u8 overlay, u8 config)
+{
+	u64 a0 = overlay;
+	u64 a1 = config;
+	int wait = 1000;
+
+	return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CTRL, &a0, &a1, wait);
+}
+
+int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay,
+				 u16 vxlan_udp_port_number)
+{
+	u64 a1 = vxlan_udp_port_number;
+	u64 a0 = overlay;
+	int wait = 1000;
+
+	return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CFG, &a0, &a1, wait);
+}
+
+int vnic_dev_capable_vxlan(struct vnic_dev *vdev)
+{
+	u64 a0 = VIC_FEATURE_VXLAN;
+	u64 a1 = 0;
+	int wait = 1000;
+	int ret;
+
+	ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait);
+	/* 1 if the NIC can do VXLAN for both IPv4 and IPv6 with multiple WQs */
+	return ret == 0 &&
+		(a1 & (FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ)) ==
+		(FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ);
+}
diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h
index 7e5736b4d..b3ce26032 100644
--- a/drivers/net/enic/base/vnic_dev.h
+++ b/drivers/net/enic/base/vnic_dev.h
@@ -178,10 +178,9 @@ int vnic_dev_deinit_done(struct vnic_dev *vdev, int *status);
 int vnic_dev_set_mac_addr(struct vnic_dev *vdev, u8 *mac_addr);
 int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry,
 	struct filter_v2 *data, struct filter_action_v2 *action_v2);
-#ifdef ENIC_VXLAN
-int vnic_dev_overlay_offload_enable_disable(struct vnic_dev *vdev,
+int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev,
 	u8 overlay, u8 config);
 int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay,
 	u16 vxlan_udp_port_number);
-#endif
+int vnic_dev_capable_vxlan(struct vnic_dev *vdev);
 #endif /* _VNIC_DEV_H_ */
diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h
index a63624559..d09113dac 100644
--- a/drivers/net/enic/base/vnic_devcmd.h
+++ b/drivers/net/enic/base/vnic_devcmd.h
@@ -1078,6 +1078,18 @@ typedef enum {
 	VIC_FEATURE_MAX,
 } vic_feature_t;
 
+/*
+ * These flags are used in args[1] of devcmd CMD_GET_SUPP_FEATURE_VER
+ * to indicate the host driver about the VxLAN and Multi WQ features
+ * supported
+ */
+#define FEATURE_VXLAN_IPV6_INNER	(1 << 0)
+#define FEATURE_VXLAN_IPV6_OUTER	(1 << 1)
+#define FEATURE_VXLAN_MULTI_WQ		(1 << 2)
+
+#define FEATURE_VXLAN_IPV6		(FEATURE_VXLAN_IPV6_INNER | \
+					 FEATURE_VXLAN_IPV6_OUTER)
+
 /*
  * CMD_CONFIG_GRPINTR subcommands
  */
diff --git a/drivers/net/enic/base/vnic_wq.h b/drivers/net/enic/base/vnic_wq.h
index 7c069c063..0135bffc5 100644
--- a/drivers/net/enic/base/vnic_wq.h
+++ b/drivers/net/enic/base/vnic_wq.h
@@ -44,6 +44,7 @@ struct vnic_wq_buf {
 
 struct vnic_wq {
 	unsigned int index;
+	uint64_t tx_offload_notsup_mask;
 	struct vnic_dev *vdev;
 	struct vnic_wq_ctrl __iomem *ctrl;              /* memory-mapped */
 	struct vnic_dev_ring ring;
diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index adccc8ac5..9d69d1e9e 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -101,6 +101,7 @@ struct enic {
 	struct vnic_dev *vdev;
 
 	unsigned int port_id;
+	int overlay_offload;
 	struct rte_eth_dev *rte_dev;
 	struct enic_fdir fdir;
 	char bdf_name[ENICPMD_BDF_LENGTH];
@@ -119,6 +120,7 @@ struct enic {
 	u8 adv_filters;
 	u32 flow_filter_mode;
 	u8 filter_tags;
+	u8 vxlan;
 
 	unsigned int flags;
 	unsigned int priv_flags;
@@ -169,6 +171,10 @@ struct enic {
 	uint64_t rss_hf; /* ETH_RSS flags */
 	union vnic_rss_key rss_key;
 	union vnic_rss_cpu rss_cpu;
+
+	uint64_t rx_offload_capa; /* DEV_RX_OFFLOAD flags */
+	uint64_t tx_offload_capa; /* DEV_TX_OFFLOAD flags */
+	uint64_t tx_offload_mask; /* PKT_TX flags accepted */
 };
 
 /* Compute ethdev's max packet size from MTU */
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 03f0c2547..08307fbf3 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -39,19 +39,6 @@ static const struct rte_pci_id pci_id_enic_map[] = {
 	{.vendor_id = 0, /* sentinel */},
 };
 
-#define ENIC_TX_OFFLOAD_CAPA (			\
-		DEV_TX_OFFLOAD_VLAN_INSERT |	\
-		DEV_TX_OFFLOAD_IPV4_CKSUM  |	\
-		DEV_TX_OFFLOAD_UDP_CKSUM   |	\
-		DEV_TX_OFFLOAD_TCP_CKSUM   |	\
-		DEV_TX_OFFLOAD_TCP_TSO)
-
-#define ENIC_RX_OFFLOAD_CAPA (			\
-		DEV_RX_OFFLOAD_VLAN_STRIP |	\
-		DEV_RX_OFFLOAD_IPV4_CKSUM |	\
-		DEV_RX_OFFLOAD_UDP_CKSUM  |	\
-		DEV_RX_OFFLOAD_TCP_CKSUM)
-
 RTE_INIT(enicpmd_init_log);
 static void
 enicpmd_init_log(void)
@@ -485,8 +472,8 @@ static void enicpmd_dev_info_get(struct rte_eth_dev *eth_dev,
 	 */
 	device_info->max_rx_pktlen = enic_mtu_to_max_rx_pktlen(enic->max_mtu);
 	device_info->max_mac_addrs = ENIC_MAX_MAC_ADDR;
-	device_info->rx_offload_capa = ENIC_RX_OFFLOAD_CAPA;
-	device_info->tx_offload_capa = ENIC_TX_OFFLOAD_CAPA;
+	device_info->rx_offload_capa = enic->rx_offload_capa;
+	device_info->tx_offload_capa = enic->tx_offload_capa;
 	device_info->default_rxconf = (struct rte_eth_rxconf) {
 		.rx_free_thresh = ENIC_DEFAULT_RX_FREE_THRESH
 	};
@@ -718,7 +705,7 @@ static void enicpmd_dev_rxq_info_get(struct rte_eth_dev *dev,
 	 * Except VLAN stripping (port setting), all the checksum offloads
 	 * are always enabled.
 	 */
-	conf->offloads = ENIC_RX_OFFLOAD_CAPA;
+	conf->offloads = enic->rx_offload_capa;
 	if (!enic->ig_vlan_strip_en)
 		conf->offloads &= ~DEV_RX_OFFLOAD_VLAN_STRIP;
 	/* rx_thresh and other fields are not applicable for enic */
@@ -733,7 +720,7 @@ static void enicpmd_dev_txq_info_get(struct rte_eth_dev *dev,
 	ENICPMD_FUNC_TRACE();
 	qinfo->nb_desc = enic->config.wq_desc_count;
 	memset(&qinfo->conf, 0, sizeof(qinfo->conf));
-	qinfo->conf.offloads = ENIC_TX_OFFLOAD_CAPA; /* not configurable */
+	qinfo->conf.offloads = enic->tx_offload_capa;
 	/* tx_thresh, and all the other fields are not applicable for enic */
 }
 
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index e908e20f2..9b254d98e 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -245,6 +245,9 @@ void enic_init_vnic_resources(struct enic *enic)
 			enic_cq_wq(enic, index),
 			error_interrupt_enable,
 			error_interrupt_offset);
+		/* Compute unsupported ol flags for enic_prep_pkts() */
+		enic->wq[index].tx_offload_notsup_mask =
+			PKT_TX_OFFLOAD_MASK ^ enic->tx_offload_mask;
 
 		cq_idx = enic_cq_wq(enic, index);
 		vnic_cq_init(&enic->cq[cq_idx],
@@ -1534,6 +1537,28 @@ static int enic_dev_init(struct enic *enic)
 	/* set up link status checking */
 	vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */
 
+	enic->overlay_offload = 0;
+#ifdef RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD
+	if (enic->vxlan &&
+	    /* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */
+	    vnic_dev_overlay_offload_ctrl(enic->vdev,
+					  OVERLAY_FEATURE_VXLAN,
+					  OVERLAY_OFFLOAD_ENABLE) == 0) {
+		enic->tx_offload_capa |=
+			DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+			DEV_TX_OFFLOAD_GENEVE_TNL_TSO |
+			DEV_TX_OFFLOAD_VXLAN_TNL_TSO;
+		/*
+		 * Do not add PKT_TX_OUTER_{IPV4,IPV6} as they are not
+		 * 'offload' flags (i.e. not part of PKT_TX_OFFLOAD_MASK).
+		 */
+		enic->tx_offload_mask |=
+			PKT_TX_OUTER_IP_CKSUM |
+			PKT_TX_TUNNEL_MASK;
+		enic->overlay_offload = 1;
+		dev_info(enic, "Overlay offload is enabled\n");
+	}
+#endif
 	return 0;
 
 }
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index e7ad6767e..47f825e51 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -148,6 +148,29 @@ int enic_get_vnic_config(struct enic *enic)
 	if (!ENIC_SETTING(enic, RSS))
 		enic->flow_type_rss_offloads = 0;
 
+	enic->vxlan = ENIC_SETTING(enic, VXLAN) &&
+		vnic_dev_capable_vxlan(enic->vdev);
+	/*
+	 * Default hardware capabilities. enic_dev_init() may add additional
+	 * flags if it enables overlay offloads.
+	 */
+	enic->tx_offload_capa =
+		DEV_TX_OFFLOAD_VLAN_INSERT |
+		DEV_TX_OFFLOAD_IPV4_CKSUM |
+		DEV_TX_OFFLOAD_UDP_CKSUM |
+		DEV_TX_OFFLOAD_TCP_CKSUM |
+		DEV_TX_OFFLOAD_TCP_TSO;
+	enic->rx_offload_capa =
+		DEV_RX_OFFLOAD_VLAN_STRIP |
+		DEV_RX_OFFLOAD_IPV4_CKSUM |
+		DEV_RX_OFFLOAD_UDP_CKSUM |
+		DEV_RX_OFFLOAD_TCP_CKSUM;
+	enic->tx_offload_mask =
+		PKT_TX_VLAN_PKT |
+		PKT_TX_IP_CKSUM |
+		PKT_TX_L4_MASK |
+		PKT_TX_TCP_SEG;
+
 	return 0;
 }
 
diff --git a/drivers/net/enic/enic_rxtx.c b/drivers/net/enic/enic_rxtx.c
index 2fe5a3fa3..aa3393700 100644
--- a/drivers/net/enic/enic_rxtx.c
+++ b/drivers/net/enic/enic_rxtx.c
@@ -15,15 +15,6 @@
 #include <rte_ip.h>
 #include <rte_tcp.h>
 
-#define	ENIC_TX_OFFLOAD_MASK (			 \
-		PKT_TX_VLAN_PKT |		 \
-		PKT_TX_IP_CKSUM |		 \
-		PKT_TX_L4_MASK |		 \
-		PKT_TX_TCP_SEG)
-
-#define	ENIC_TX_OFFLOAD_NOTSUP_MASK \
-	(PKT_TX_OFFLOAD_MASK ^ ENIC_TX_OFFLOAD_MASK)
-
 #define RTE_PMD_USE_PREFETCH
 
 #ifdef RTE_PMD_USE_PREFETCH
@@ -130,30 +121,73 @@ enic_cq_rx_check_err(struct cq_desc *cqd)
 
 /* Lookup table to translate RX CQ flags to mbuf flags. */
 static inline uint32_t
-enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd)
+enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd, uint8_t tnl)
 {
 	struct cq_enet_rq_desc *cqrd = (struct cq_enet_rq_desc *)cqd;
 	uint8_t cqrd_flags = cqrd->flags;
+	/*
+	 * Odd-numbered entries are for tunnel packets. All packet type info
+	 * applies to the inner packet, and there is no info on the outer
+	 * packet. The outer flags in these entries exist only to avoid
+	 * changing enic_cq_rx_to_pkt_flags(). They are cleared from mbuf
+	 * afterwards.
+	 *
+	 * Also, as there is no tunnel type info (VXLAN, NVGRE, or GENEVE), set
+	 * RTE_PTYPE_TUNNEL_GRENAT..
+	 */
 	static const uint32_t cq_type_table[128] __rte_cache_aligned = {
 		[0x00] = RTE_PTYPE_UNKNOWN,
 		[0x20] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG,
+		[0x21] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_NONFRAG,
 		[0x22] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
+		[0x23] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_UDP,
 		[0x24] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
+		[0x25] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_TCP,
 		[0x60] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG,
+		[0x61] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_FRAG,
 		[0x62] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
+		[0x63] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_UDP,
 		[0x64] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
+		[0x65] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_TCP,
 		[0x10] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG,
+		[0x11] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_NONFRAG,
 		[0x12] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
+		[0x13] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_UDP,
 		[0x14] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
-		[0x50] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG,
-		[0x52] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
-		[0x54] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
+		[0x15] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_TCP,
 		/* All others reserved */
 	};
 	cqrd_flags &= CQ_ENET_RQ_DESC_FLAGS_IPV4_FRAGMENT
 		| CQ_ENET_RQ_DESC_FLAGS_IPV4 | CQ_ENET_RQ_DESC_FLAGS_IPV6
 		| CQ_ENET_RQ_DESC_FLAGS_TCP | CQ_ENET_RQ_DESC_FLAGS_UDP;
-	return cq_type_table[cqrd_flags];
+	return cq_type_table[cqrd_flags + tnl];
 }
 
 static inline void
@@ -200,10 +234,18 @@ enic_cq_rx_to_pkt_flags(struct cq_desc *cqd, struct rte_mbuf *mbuf)
 			uint32_t l4_flags;
 			l4_flags = mbuf->packet_type & RTE_PTYPE_L4_MASK;
 
-			if (enic_cq_rx_desc_ipv4_csum_ok(cqrd))
-				pkt_flags |= PKT_RX_IP_CKSUM_GOOD;
-			else if (mbuf->packet_type & RTE_PTYPE_L3_IPV4)
-				pkt_flags |= PKT_RX_IP_CKSUM_BAD;
+			/*
+			 * When overlay offload is enabled, the NIC may
+			 * set ipv4_csum_ok=1 if the inner packet is IPv6..
+			 * So, explicitly check for IPv4 before checking
+			 * ipv4_csum_ok.
+			 */
+			if (mbuf->packet_type & RTE_PTYPE_L3_IPV4) {
+				if (enic_cq_rx_desc_ipv4_csum_ok(cqrd))
+					pkt_flags |= PKT_RX_IP_CKSUM_GOOD;
+				else
+					pkt_flags |= PKT_RX_IP_CKSUM_BAD;
+			}
 
 			if (l4_flags == RTE_PTYPE_L4_UDP ||
 			    l4_flags == RTE_PTYPE_L4_TCP) {
@@ -245,6 +287,7 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vnic_cq *cq;
 	volatile struct cq_desc *cqd_ptr;
 	uint8_t color;
+	uint8_t tnl;
 	uint16_t seg_length;
 	struct rte_mbuf *first_seg = sop_rq->pkt_first_seg;
 	struct rte_mbuf *last_seg = sop_rq->pkt_last_seg;
@@ -336,10 +379,21 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			continue;
 		}
 
+		/*
+		 * When overlay offload is enabled, CQ.fcoe indicates the
+		 * packet is tunnelled.
+		 */
+		tnl = enic->overlay_offload &&
+			(ciflags & CQ_ENET_RQ_DESC_FLAGS_FCOE) != 0;
 		/* cq rx flags are only valid if eop bit is set */
-		first_seg->packet_type = enic_cq_rx_flags_to_pkt_type(&cqd);
+		first_seg->packet_type =
+			enic_cq_rx_flags_to_pkt_type(&cqd, tnl);
 		enic_cq_rx_to_pkt_flags(&cqd, first_seg);
-
+		/* Wipe the outer types set by enic_cq_rx_flags_to_pkt_type() */
+		if (tnl) {
+			first_seg->packet_type &= ~(RTE_PTYPE_L3_MASK |
+						    RTE_PTYPE_L4_MASK);
+		}
 		if (unlikely(packet_error)) {
 			rte_pktmbuf_free(first_seg);
 			rte_atomic64_inc(&enic->soft_stats.rx_packet_errors);
@@ -443,9 +497,10 @@ unsigned int enic_cleanup_wq(__rte_unused struct enic *enic, struct vnic_wq *wq)
 	return 0;
 }
 
-uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+uint16_t enic_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 			uint16_t nb_pkts)
 {
+	struct vnic_wq *wq = (struct vnic_wq *)tx_queue;
 	int32_t ret;
 	uint16_t i;
 	uint64_t ol_flags;
@@ -454,7 +509,7 @@ uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
 	for (i = 0; i != nb_pkts; i++) {
 		m = tx_pkts[i];
 		ol_flags = m->ol_flags;
-		if (ol_flags & ENIC_TX_OFFLOAD_NOTSUP_MASK) {
+		if (ol_flags & wq->tx_offload_notsup_mask) {
 			rte_errno = -ENOTSUP;
 			return i;
 		}
@@ -558,6 +613,11 @@ uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 
 			offload_mode = WQ_ENET_OFFLOAD_MODE_TSO;
 			mss = tx_pkt->tso_segsz;
+			/* For tunnel, need the size of outer+inner headers */
+			if (ol_flags & PKT_TX_TUNNEL_MASK) {
+				header_len += tx_pkt->outer_l2_len +
+					tx_pkt->outer_l3_len;
+			}
 		}
 
 		if ((ol_flags & ol_flags_mask) && (header_len == 0)) {
-- 
2.16.2

Re: [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE

[Ferruh Yigit]

On 4/5/2018 12:54 AM, John Daley wrote:
> From: Hyong Youb Kim <hyonkim@cisco.com>
> 
> Recent NIC models support overlay offload. The overlay offload
> feature enables the following on the NIC.
> - Rx/Tx checksum offloads for both inner and outer packets.
> - Rx inner packet type classification.
> - TSO.
> - Inner RSS.
> 
> TX descriptors do not require any changes, except the header length
> for TSO. The NIC parses outer/inner packets and performs offloads on
> them as necessary. The header length for tunneled TSO includes both
> inner and outer headers.
> 
> The NIC actually parses and performs the above for NVGRE as well. DPDK
> currently has no offload flags for NVGRE, and the hardware has no
> controls to individually enable tunnel types either. So do nothing for
> now.
> 
> Add a config flag to enable overlay offload by default. To disable it,
> the user should set it to 'n'.
> 
> CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y
> 
> Also update the enic guide doc.
> 
> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
> Reviewed-by: John Daley <johndale@cisco.com>
> ---
>  config/common_base                  |   1 +
>  doc/guides/nics/enic.rst            |  52 ++++++++++++++++++
>  drivers/net/enic/base/vnic_dev.c    |  33 ++++++++++++
>  drivers/net/enic/base/vnic_dev.h    |   5 +-
>  drivers/net/enic/base/vnic_devcmd.h |  12 +++++
>  drivers/net/enic/base/vnic_wq.h     |   1 +
>  drivers/net/enic/enic.h             |   6 +++
>  drivers/net/enic/enic_ethdev.c      |  21 ++------
>  drivers/net/enic/enic_main.c        |  25 +++++++++
>  drivers/net/enic/enic_res.c         |  23 ++++++++
>  drivers/net/enic/enic_rxtx.c        | 104 ++++++++++++++++++++++++++++--------
>  11 files changed, 241 insertions(+), 42 deletions(-)
> 
> diff --git a/config/common_base b/config/common_base
> index c09c7cf88..964e37b6e 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n
>  # Compile burst-oriented Cisco ENIC PMD driver
>  #
>  CONFIG_RTE_LIBRTE_ENIC_PMD=y
> +CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y

Hi John,

We are trying to reduce config options we have, is overlay offload enabling can
be done via runtime argument or dedicated offload flag?

Re: [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE

[Hyong Youb Kim]

On Fri, Apr 06, 2018 at 05:15:40PM +0100, Ferruh Yigit wrote:
> On 4/5/2018 12:54 AM, John Daley wrote:
> > From: Hyong Youb Kim <hyonkim@cisco.com>
> > 
> > Recent NIC models support overlay offload. The overlay offload
> > feature enables the following on the NIC.
> > - Rx/Tx checksum offloads for both inner and outer packets.
> > - Rx inner packet type classification.
> > - TSO.
> > - Inner RSS.
> > 
> > TX descriptors do not require any changes, except the header length
> > for TSO. The NIC parses outer/inner packets and performs offloads on
> > them as necessary. The header length for tunneled TSO includes both
> > inner and outer headers.
> > 
> > The NIC actually parses and performs the above for NVGRE as well. DPDK
> > currently has no offload flags for NVGRE, and the hardware has no
> > controls to individually enable tunnel types either. So do nothing for
> > now.
> > 
> > Add a config flag to enable overlay offload by default. To disable it,
> > the user should set it to 'n'.
> > 
> > CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y
> > 
> > Also update the enic guide doc.
> > 
> > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
> > Reviewed-by: John Daley <johndale@cisco.com>
> > ---
> >  config/common_base                  |   1 +
> >  doc/guides/nics/enic.rst            |  52 ++++++++++++++++++
> >  drivers/net/enic/base/vnic_dev.c    |  33 ++++++++++++
> >  drivers/net/enic/base/vnic_dev.h    |   5 +-
> >  drivers/net/enic/base/vnic_devcmd.h |  12 +++++
> >  drivers/net/enic/base/vnic_wq.h     |   1 +
> >  drivers/net/enic/enic.h             |   6 +++
> >  drivers/net/enic/enic_ethdev.c      |  21 ++------
> >  drivers/net/enic/enic_main.c        |  25 +++++++++
> >  drivers/net/enic/enic_res.c         |  23 ++++++++
> >  drivers/net/enic/enic_rxtx.c        | 104 ++++++++++++++++++++++++++++--------
> >  11 files changed, 241 insertions(+), 42 deletions(-)
> > 
> > diff --git a/config/common_base b/config/common_base
> > index c09c7cf88..964e37b6e 100644
> > --- a/config/common_base
> > +++ b/config/common_base
> > @@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n
> >  # Compile burst-oriented Cisco ENIC PMD driver
> >  #
> >  CONFIG_RTE_LIBRTE_ENIC_PMD=y
> > +CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y
> 
> Hi John,
> 
> We are trying to reduce config options we have, is overlay offload enabling can
> be done via runtime argument or dedicated offload flag?

Hi,

Would you accept a devarg? We would enable overlay offload by default,
and disable it when the app specifies that devarg. This overlay
offload is a bunch of things under one roof, and it does not play
nicely with DPDK offload flags.

Thanks.
-Hyong

Re: [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE

[Ferruh Yigit]

On 4/7/2018 3:40 AM, Hyong Youb Kim wrote:
> On Fri, Apr 06, 2018 at 05:15:40PM +0100, Ferruh Yigit wrote:
>> On 4/5/2018 12:54 AM, John Daley wrote:
>>> From: Hyong Youb Kim <hyonkim@cisco.com>
>>>
>>> Recent NIC models support overlay offload. The overlay offload
>>> feature enables the following on the NIC.
>>> - Rx/Tx checksum offloads for both inner and outer packets.
>>> - Rx inner packet type classification.
>>> - TSO.
>>> - Inner RSS.
>>>
>>> TX descriptors do not require any changes, except the header length
>>> for TSO. The NIC parses outer/inner packets and performs offloads on
>>> them as necessary. The header length for tunneled TSO includes both
>>> inner and outer headers.
>>>
>>> The NIC actually parses and performs the above for NVGRE as well. DPDK
>>> currently has no offload flags for NVGRE, and the hardware has no
>>> controls to individually enable tunnel types either. So do nothing for
>>> now.
>>>
>>> Add a config flag to enable overlay offload by default. To disable it,
>>> the user should set it to 'n'.
>>>
>>> CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y
>>>
>>> Also update the enic guide doc.
>>>
>>> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
>>> Reviewed-by: John Daley <johndale@cisco.com>
>>> ---
>>>  config/common_base                  |   1 +
>>>  doc/guides/nics/enic.rst            |  52 ++++++++++++++++++
>>>  drivers/net/enic/base/vnic_dev.c    |  33 ++++++++++++
>>>  drivers/net/enic/base/vnic_dev.h    |   5 +-
>>>  drivers/net/enic/base/vnic_devcmd.h |  12 +++++
>>>  drivers/net/enic/base/vnic_wq.h     |   1 +
>>>  drivers/net/enic/enic.h             |   6 +++
>>>  drivers/net/enic/enic_ethdev.c      |  21 ++------
>>>  drivers/net/enic/enic_main.c        |  25 +++++++++
>>>  drivers/net/enic/enic_res.c         |  23 ++++++++
>>>  drivers/net/enic/enic_rxtx.c        | 104 ++++++++++++++++++++++++++++--------
>>>  11 files changed, 241 insertions(+), 42 deletions(-)
>>>
>>> diff --git a/config/common_base b/config/common_base
>>> index c09c7cf88..964e37b6e 100644
>>> --- a/config/common_base
>>> +++ b/config/common_base
>>> @@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n
>>>  # Compile burst-oriented Cisco ENIC PMD driver
>>>  #
>>>  CONFIG_RTE_LIBRTE_ENIC_PMD=y
>>> +CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y
>>
>> Hi John,
>>
>> We are trying to reduce config options we have, is overlay offload enabling can
>> be done via runtime argument or dedicated offload flag?
> 
> Hi,
> 
> Would you accept a devarg? We would enable overlay offload by default,
> and disable it when the app specifies that devarg. This overlay
> offload is a bunch of things under one roof, and it does not play
> nicely with DPDK offload flags.

No problem on implementing this as devargs.

> 
> Thanks.
> -Hyong
> 

[dpdk-dev] [PATCH v2] net/enic: enable overlay offload for VXLAN and GENEVE

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

Recent NIC models support overlay offload. The overlay offload
feature enables the following on the NIC.
- Rx/Tx checksum offloads for both inner and outer packets.
- Rx inner packet type classification.
- TSO.
- Inner RSS.

TX descriptors do not require any changes, except the header length
for TSO. The NIC parses outer/inner packets and performs offloads on
them as necessary. The header length for tunneled TSO includes both
inner and outer headers.

The NIC actually parses and performs the above for NVGRE as well. DPDK
currently has no offload flags for NVGRE, and the hardware has no
controls to individually enable tunnel types either. So do nothing for
now.

The driver enables overlay offload by default. Add a devargs
'disable-overlay=<0|1>' to allow the app to disable it.

Also update the enic guide doc.

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
---
v2 - use devargs insetad of config option

 doc/guides/nics/enic.rst            |  53 ++++++++++++++++++
 drivers/net/enic/base/vnic_dev.c    |  33 ++++++++++++
 drivers/net/enic/base/vnic_dev.h    |   5 +-
 drivers/net/enic/base/vnic_devcmd.h |  12 +++++
 drivers/net/enic/base/vnic_wq.h     |   1 +
 drivers/net/enic/enic.h             |   8 +++
 drivers/net/enic/enic_ethdev.c      |  71 ++++++++++++++++++------
 drivers/net/enic/enic_main.c        |  24 +++++++++
 drivers/net/enic/enic_res.c         |  23 ++++++++
 drivers/net/enic/enic_rxtx.c        | 104 ++++++++++++++++++++++++++++--------
 10 files changed, 293 insertions(+), 41 deletions(-)

diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index 3ef635fc8..49abc7e95 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -254,6 +254,54 @@ Generic Flow API is supported. The baseline support is:
 More features may be added in future firmware and new versions of the VIC.
 Please refer to the release notes.
 
+.. _overlay_offload:
+
+Overlay Offload
+---------------
+
+Recent hardware models support overlay offload. When enabled, the NIC performs
+the following operations for VXLAN, NVGRE, and GENEVE packets. In all cases,
+inner and outer packets can be IPv4 or IPv6.
+
+- TSO for VXLAN and GENEVE packets.
+
+  Hardware supports NVGRE TSO, but DPDK currently has no NVGRE offload flags.
+
+- Tx checksum offloads.
+
+  The NIC fills in IPv4/UDP/TCP checksums for both inner and outer packets.
+
+- Rx checksum offloads.
+
+  The NIC validates IPv4/UDP/TCP checksums of both inner and outer packets.
+  Good checksum flags (e.g. ``PKT_RX_L4_CKSUM_GOOD``) indicate that the inner
+  packet has the correct checksum, and if applicable, the outer packet also
+  has the correct checksum. Bad checksum flags (e.g. ``PKT_RX_L4_CKSUM_BAD``)
+  indicate that the inner and/or outer packets have invalid checksum values.
+
+- Inner Rx packet type classification
+
+  PMD sets inner L3/L4 packet types (e.g. ``RTE_PTYPE_INNER_L4_TCP``), and
+  ``RTE_PTYPE_TUNNEL_GRENAT`` to indicate that the packet is tunneled.
+  PMD does not set L3/L4 packet types for outer packets.
+
+- Inner RSS
+
+  RSS hash calculation, therefore queue selection, is done on inner packets.
+
+In order to enable overlay offload, the 'Enable VXLAN' box should be checked
+via CIMC or UCSM followed by a reboot of the server. When PMD successfully
+enables overlay offload, it prints the following message on the console.
+
+.. code-block:: console
+
+    Overlay offload is enabled
+
+By default, PMD enables overlay offload if hardware supports it. To disable
+it, set ``devargs`` parameter ``disable-overlay=1``. For example::
+
+    -w 12:00.0,disable-overlay=1
+
 .. _enic_limitations:
 
 Limitations
@@ -376,6 +424,11 @@ Supported features
 - MTU update
 - SR-IOV on UCS managed servers connected to Fabric Interconnects
 - Flow API
+- Overlay offload
+
+  - Rx/Tx checksum offloads for VXLAN, NVGRE, GENEVE
+  - TSO for VXLAN and GENEVE packets
+  - Inner RSS
 
 Known bugs and unsupported features in this release
 ---------------------------------------------------
diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c
index 364458a88..8880ab981 100644
--- a/drivers/net/enic/base/vnic_dev.c
+++ b/drivers/net/enic/base/vnic_dev.c
@@ -1062,3 +1062,36 @@ int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry,
 
 	return ret;
 }
+
+int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev, u8 overlay, u8 config)
+{
+	u64 a0 = overlay;
+	u64 a1 = config;
+	int wait = 1000;
+
+	return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CTRL, &a0, &a1, wait);
+}
+
+int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay,
+				 u16 vxlan_udp_port_number)
+{
+	u64 a1 = vxlan_udp_port_number;
+	u64 a0 = overlay;
+	int wait = 1000;
+
+	return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CFG, &a0, &a1, wait);
+}
+
+int vnic_dev_capable_vxlan(struct vnic_dev *vdev)
+{
+	u64 a0 = VIC_FEATURE_VXLAN;
+	u64 a1 = 0;
+	int wait = 1000;
+	int ret;
+
+	ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait);
+	/* 1 if the NIC can do VXLAN for both IPv4 and IPv6 with multiple WQs */
+	return ret == 0 &&
+		(a1 & (FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ)) ==
+		(FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ);
+}
diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h
index 90833c794..e7a1f8bd8 100644
--- a/drivers/net/enic/base/vnic_dev.h
+++ b/drivers/net/enic/base/vnic_dev.h
@@ -179,10 +179,9 @@ int vnic_dev_deinit_done(struct vnic_dev *vdev, int *status);
 int vnic_dev_set_mac_addr(struct vnic_dev *vdev, u8 *mac_addr);
 int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry,
 	struct filter_v2 *data, struct filter_action_v2 *action_v2);
-#ifdef ENIC_VXLAN
-int vnic_dev_overlay_offload_enable_disable(struct vnic_dev *vdev,
+int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev,
 	u8 overlay, u8 config);
 int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay,
 	u16 vxlan_udp_port_number);
-#endif
+int vnic_dev_capable_vxlan(struct vnic_dev *vdev);
 #endif /* _VNIC_DEV_H_ */
diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h
index 2294b55cf..2865eb4d4 100644
--- a/drivers/net/enic/base/vnic_devcmd.h
+++ b/drivers/net/enic/base/vnic_devcmd.h
@@ -1080,6 +1080,18 @@ typedef enum {
 	VIC_FEATURE_MAX,
 } vic_feature_t;
 
+/*
+ * These flags are used in args[1] of devcmd CMD_GET_SUPP_FEATURE_VER
+ * to indicate the host driver about the VxLAN and Multi WQ features
+ * supported
+ */
+#define FEATURE_VXLAN_IPV6_INNER	(1 << 0)
+#define FEATURE_VXLAN_IPV6_OUTER	(1 << 1)
+#define FEATURE_VXLAN_MULTI_WQ		(1 << 2)
+
+#define FEATURE_VXLAN_IPV6		(FEATURE_VXLAN_IPV6_INNER | \
+					 FEATURE_VXLAN_IPV6_OUTER)
+
 /*
  * CMD_CONFIG_GRPINTR subcommands
  */
diff --git a/drivers/net/enic/base/vnic_wq.h b/drivers/net/enic/base/vnic_wq.h
index 7c069c063..0135bffc5 100644
--- a/drivers/net/enic/base/vnic_wq.h
+++ b/drivers/net/enic/base/vnic_wq.h
@@ -44,6 +44,7 @@ struct vnic_wq_buf {
 
 struct vnic_wq {
 	unsigned int index;
+	uint64_t tx_offload_notsup_mask;
 	struct vnic_dev *vdev;
 	struct vnic_wq_ctrl __iomem *ctrl;              /* memory-mapped */
 	struct vnic_dev_ring ring;
diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index 751ddc744..32a64a51a 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -17,6 +17,7 @@
 #include "vnic_rss.h"
 #include "enic_res.h"
 #include "cq_enet_desc.h"
+#include <stdbool.h>
 #include <sys/queue.h>
 #include <rte_spinlock.h>
 
@@ -101,6 +102,7 @@ struct enic {
 	struct vnic_dev *vdev;
 
 	unsigned int port_id;
+	bool overlay_offload;
 	struct rte_eth_dev *rte_dev;
 	struct enic_fdir fdir;
 	char bdf_name[ENICPMD_BDF_LENGTH];
@@ -119,6 +121,8 @@ struct enic {
 	u8 adv_filters;
 	u32 flow_filter_mode;
 	u8 filter_actions; /* HW supported actions */
+	bool vxlan;
+	bool disable_overlay; /* devargs disable_overlay=1 */
 
 	unsigned int flags;
 	unsigned int priv_flags;
@@ -169,6 +173,10 @@ struct enic {
 	uint64_t rss_hf; /* ETH_RSS flags */
 	union vnic_rss_key rss_key;
 	union vnic_rss_cpu rss_cpu;
+
+	uint64_t rx_offload_capa; /* DEV_RX_OFFLOAD flags */
+	uint64_t tx_offload_capa; /* DEV_TX_OFFLOAD flags */
+	uint64_t tx_offload_mask; /* PKT_TX flags accepted */
 };
 
 /* Compute ethdev's max packet size from MTU */
diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c
index 801f4704c..b37384c88 100644
--- a/drivers/net/enic/enic_ethdev.c
+++ b/drivers/net/enic/enic_ethdev.c
@@ -11,6 +11,7 @@
 #include <rte_bus_pci.h>
 #include <rte_ethdev_driver.h>
 #include <rte_ethdev_pci.h>
+#include <rte_kvargs.h>
 #include <rte_string_fns.h>
 
 #include "vnic_intr.h"
@@ -39,18 +40,7 @@ static const struct rte_pci_id pci_id_enic_map[] = {
 	{.vendor_id = 0, /* sentinel */},
 };
 
-#define ENIC_TX_OFFLOAD_CAPA (			\
-		DEV_TX_OFFLOAD_VLAN_INSERT |	\
-		DEV_TX_OFFLOAD_IPV4_CKSUM  |	\
-		DEV_TX_OFFLOAD_UDP_CKSUM   |	\
-		DEV_TX_OFFLOAD_TCP_CKSUM   |	\
-		DEV_TX_OFFLOAD_TCP_TSO)
-
-#define ENIC_RX_OFFLOAD_CAPA (			\
-		DEV_RX_OFFLOAD_VLAN_STRIP |	\
-		DEV_RX_OFFLOAD_IPV4_CKSUM |	\
-		DEV_RX_OFFLOAD_UDP_CKSUM  |	\
-		DEV_RX_OFFLOAD_TCP_CKSUM)
+#define ENIC_DEVARG_DISABLE_OVERLAY "disable-overlay"
 
 RTE_INIT(enicpmd_init_log);
 static void
@@ -484,8 +474,8 @@ static void enicpmd_dev_info_get(struct rte_eth_dev *eth_dev,
 	 */
 	device_info->max_rx_pktlen = enic_mtu_to_max_rx_pktlen(enic->max_mtu);
 	device_info->max_mac_addrs = ENIC_MAX_MAC_ADDR;
-	device_info->rx_offload_capa = ENIC_RX_OFFLOAD_CAPA;
-	device_info->tx_offload_capa = ENIC_TX_OFFLOAD_CAPA;
+	device_info->rx_offload_capa = enic->rx_offload_capa;
+	device_info->tx_offload_capa = enic->tx_offload_capa;
 	device_info->default_rxconf = (struct rte_eth_rxconf) {
 		.rx_free_thresh = ENIC_DEFAULT_RX_FREE_THRESH
 	};
@@ -717,7 +707,7 @@ static void enicpmd_dev_rxq_info_get(struct rte_eth_dev *dev,
 	 * Except VLAN stripping (port setting), all the checksum offloads
 	 * are always enabled.
 	 */
-	conf->offloads = ENIC_RX_OFFLOAD_CAPA;
+	conf->offloads = enic->rx_offload_capa;
 	if (!enic->ig_vlan_strip_en)
 		conf->offloads &= ~DEV_RX_OFFLOAD_VLAN_STRIP;
 	/* rx_thresh and other fields are not applicable for enic */
@@ -732,7 +722,7 @@ static void enicpmd_dev_txq_info_get(struct rte_eth_dev *dev,
 	ENICPMD_FUNC_TRACE();
 	qinfo->nb_desc = enic->config.wq_desc_count;
 	memset(&qinfo->conf, 0, sizeof(qinfo->conf));
-	qinfo->conf.offloads = ENIC_TX_OFFLOAD_CAPA; /* not configurable */
+	qinfo->conf.offloads = enic->tx_offload_capa;
 	/* tx_thresh, and all the other fields are not applicable for enic */
 }
 
@@ -806,6 +796,49 @@ static const struct eth_dev_ops enicpmd_eth_dev_ops = {
 	.rss_hash_update      = enicpmd_dev_rss_hash_update,
 };
 
+static int enic_parse_disable_overlay(__rte_unused const char *key,
+				      const char *value,
+				      void *opaque)
+{
+	struct enic *enic;
+
+	enic = (struct enic *)opaque;
+	if (strcmp(value, "0") == 0) {
+		enic->disable_overlay = false;
+	} else if (strcmp(value, "1") == 0) {
+		enic->disable_overlay = true;
+	} else {
+		dev_err(enic, "Invalid value for " ENIC_DEVARG_DISABLE_OVERLAY
+			": expected=0|1 given=%s\n", value);
+		return -EINVAL;
+	}
+	return 0;
+}
+
+static int enic_check_devargs(struct rte_eth_dev *dev)
+{
+	static const char *const valid_keys[] = {
+		ENIC_DEVARG_DISABLE_OVERLAY, NULL};
+	struct enic *enic = pmd_priv(dev);
+	struct rte_kvargs *kvlist;
+
+	ENICPMD_FUNC_TRACE();
+
+	enic->disable_overlay = false;
+	if (!dev->device->devargs)
+		return 0;
+	kvlist = rte_kvargs_parse(dev->device->devargs->args, valid_keys);
+	if (!kvlist)
+		return -EINVAL;
+	if (rte_kvargs_process(kvlist, ENIC_DEVARG_DISABLE_OVERLAY,
+			       enic_parse_disable_overlay, enic) < 0) {
+		rte_kvargs_free(kvlist);
+		return -EINVAL;
+	}
+	rte_kvargs_free(kvlist);
+	return 0;
+}
+
 struct enic *enicpmd_list_head = NULL;
 /* Initialize the driver
  * It returns 0 on success.
@@ -815,6 +848,7 @@ static int eth_enicpmd_dev_init(struct rte_eth_dev *eth_dev)
 	struct rte_pci_device *pdev;
 	struct rte_pci_addr *addr;
 	struct enic *enic = pmd_priv(eth_dev);
+	int err;
 
 	ENICPMD_FUNC_TRACE();
 
@@ -833,6 +867,9 @@ static int eth_enicpmd_dev_init(struct rte_eth_dev *eth_dev)
 	snprintf(enic->bdf_name, ENICPMD_BDF_LENGTH, "%04x:%02x:%02x.%x",
 		addr->domain, addr->bus, addr->devid, addr->function);
 
+	err = enic_check_devargs(eth_dev);
+	if (err)
+		return err;
 	return enic_probe(enic);
 }
 
@@ -858,3 +895,5 @@ static struct rte_pci_driver rte_enic_pmd = {
 RTE_PMD_REGISTER_PCI(net_enic, rte_enic_pmd);
 RTE_PMD_REGISTER_PCI_TABLE(net_enic, pci_id_enic_map);
 RTE_PMD_REGISTER_KMOD_DEP(net_enic, "* igb_uio | uio_pci_generic | vfio-pci");
+RTE_PMD_REGISTER_PARAM_STRING(net_enic,
+			      ENIC_DEVARG_DISABLE_OVERLAY "=<0|1> ");
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 98d47752c..923150bb4 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -245,6 +245,9 @@ void enic_init_vnic_resources(struct enic *enic)
 			enic_cq_wq(enic, index),
 			error_interrupt_enable,
 			error_interrupt_offset);
+		/* Compute unsupported ol flags for enic_prep_pkts() */
+		enic->wq[index].tx_offload_notsup_mask =
+			PKT_TX_OFFLOAD_MASK ^ enic->tx_offload_mask;
 
 		cq_idx = enic_cq_wq(enic, index);
 		vnic_cq_init(&enic->cq[cq_idx],
@@ -1547,6 +1550,27 @@ static int enic_dev_init(struct enic *enic)
 	/* set up link status checking */
 	vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */
 
+	enic->overlay_offload = false;
+	if (!enic->disable_overlay && enic->vxlan &&
+	    /* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */
+	    vnic_dev_overlay_offload_ctrl(enic->vdev,
+					  OVERLAY_FEATURE_VXLAN,
+					  OVERLAY_OFFLOAD_ENABLE) == 0) {
+		enic->tx_offload_capa |=
+			DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
+			DEV_TX_OFFLOAD_GENEVE_TNL_TSO |
+			DEV_TX_OFFLOAD_VXLAN_TNL_TSO;
+		/*
+		 * Do not add PKT_TX_OUTER_{IPV4,IPV6} as they are not
+		 * 'offload' flags (i.e. not part of PKT_TX_OFFLOAD_MASK).
+		 */
+		enic->tx_offload_mask |=
+			PKT_TX_OUTER_IP_CKSUM |
+			PKT_TX_TUNNEL_MASK;
+		enic->overlay_offload = true;
+		dev_info(enic, "Overlay offload is enabled\n");
+	}
+
 	return 0;
 
 }
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index 9b5baf3ba..de17a31d0 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -158,6 +158,29 @@ int enic_get_vnic_config(struct enic *enic)
 	if (!ENIC_SETTING(enic, RSS))
 		enic->flow_type_rss_offloads = 0;
 
+	enic->vxlan = ENIC_SETTING(enic, VXLAN) &&
+		vnic_dev_capable_vxlan(enic->vdev);
+	/*
+	 * Default hardware capabilities. enic_dev_init() may add additional
+	 * flags if it enables overlay offloads.
+	 */
+	enic->tx_offload_capa =
+		DEV_TX_OFFLOAD_VLAN_INSERT |
+		DEV_TX_OFFLOAD_IPV4_CKSUM |
+		DEV_TX_OFFLOAD_UDP_CKSUM |
+		DEV_TX_OFFLOAD_TCP_CKSUM |
+		DEV_TX_OFFLOAD_TCP_TSO;
+	enic->rx_offload_capa =
+		DEV_RX_OFFLOAD_VLAN_STRIP |
+		DEV_RX_OFFLOAD_IPV4_CKSUM |
+		DEV_RX_OFFLOAD_UDP_CKSUM |
+		DEV_RX_OFFLOAD_TCP_CKSUM;
+	enic->tx_offload_mask =
+		PKT_TX_VLAN_PKT |
+		PKT_TX_IP_CKSUM |
+		PKT_TX_L4_MASK |
+		PKT_TX_TCP_SEG;
+
 	return 0;
 }
 
diff --git a/drivers/net/enic/enic_rxtx.c b/drivers/net/enic/enic_rxtx.c
index 2fe5a3fa3..aa3393700 100644
--- a/drivers/net/enic/enic_rxtx.c
+++ b/drivers/net/enic/enic_rxtx.c
@@ -15,15 +15,6 @@
 #include <rte_ip.h>
 #include <rte_tcp.h>
 
-#define	ENIC_TX_OFFLOAD_MASK (			 \
-		PKT_TX_VLAN_PKT |		 \
-		PKT_TX_IP_CKSUM |		 \
-		PKT_TX_L4_MASK |		 \
-		PKT_TX_TCP_SEG)
-
-#define	ENIC_TX_OFFLOAD_NOTSUP_MASK \
-	(PKT_TX_OFFLOAD_MASK ^ ENIC_TX_OFFLOAD_MASK)
-
 #define RTE_PMD_USE_PREFETCH
 
 #ifdef RTE_PMD_USE_PREFETCH
@@ -130,30 +121,73 @@ enic_cq_rx_check_err(struct cq_desc *cqd)
 
 /* Lookup table to translate RX CQ flags to mbuf flags. */
 static inline uint32_t
-enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd)
+enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd, uint8_t tnl)
 {
 	struct cq_enet_rq_desc *cqrd = (struct cq_enet_rq_desc *)cqd;
 	uint8_t cqrd_flags = cqrd->flags;
+	/*
+	 * Odd-numbered entries are for tunnel packets. All packet type info
+	 * applies to the inner packet, and there is no info on the outer
+	 * packet. The outer flags in these entries exist only to avoid
+	 * changing enic_cq_rx_to_pkt_flags(). They are cleared from mbuf
+	 * afterwards.
+	 *
+	 * Also, as there is no tunnel type info (VXLAN, NVGRE, or GENEVE), set
+	 * RTE_PTYPE_TUNNEL_GRENAT..
+	 */
 	static const uint32_t cq_type_table[128] __rte_cache_aligned = {
 		[0x00] = RTE_PTYPE_UNKNOWN,
 		[0x20] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG,
+		[0x21] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_NONFRAG,
 		[0x22] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
+		[0x23] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_UDP,
 		[0x24] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
+		[0x25] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_TCP,
 		[0x60] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG,
+		[0x61] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_FRAG,
 		[0x62] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
+		[0x63] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_UDP,
 		[0x64] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
+		[0x65] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_TCP,
 		[0x10] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG,
+		[0x11] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_NONFRAG,
 		[0x12] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
+		[0x13] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_UDP,
 		[0x14] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
-		[0x50] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG,
-		[0x52] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP,
-		[0x54] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP,
+		[0x15] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP |
+			 RTE_PTYPE_TUNNEL_GRENAT |
+			 RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN |
+			 RTE_PTYPE_INNER_L4_TCP,
 		/* All others reserved */
 	};
 	cqrd_flags &= CQ_ENET_RQ_DESC_FLAGS_IPV4_FRAGMENT
 		| CQ_ENET_RQ_DESC_FLAGS_IPV4 | CQ_ENET_RQ_DESC_FLAGS_IPV6
 		| CQ_ENET_RQ_DESC_FLAGS_TCP | CQ_ENET_RQ_DESC_FLAGS_UDP;
-	return cq_type_table[cqrd_flags];
+	return cq_type_table[cqrd_flags + tnl];
 }
 
 static inline void
@@ -200,10 +234,18 @@ enic_cq_rx_to_pkt_flags(struct cq_desc *cqd, struct rte_mbuf *mbuf)
 			uint32_t l4_flags;
 			l4_flags = mbuf->packet_type & RTE_PTYPE_L4_MASK;
 
-			if (enic_cq_rx_desc_ipv4_csum_ok(cqrd))
-				pkt_flags |= PKT_RX_IP_CKSUM_GOOD;
-			else if (mbuf->packet_type & RTE_PTYPE_L3_IPV4)
-				pkt_flags |= PKT_RX_IP_CKSUM_BAD;
+			/*
+			 * When overlay offload is enabled, the NIC may
+			 * set ipv4_csum_ok=1 if the inner packet is IPv6..
+			 * So, explicitly check for IPv4 before checking
+			 * ipv4_csum_ok.
+			 */
+			if (mbuf->packet_type & RTE_PTYPE_L3_IPV4) {
+				if (enic_cq_rx_desc_ipv4_csum_ok(cqrd))
+					pkt_flags |= PKT_RX_IP_CKSUM_GOOD;
+				else
+					pkt_flags |= PKT_RX_IP_CKSUM_BAD;
+			}
 
 			if (l4_flags == RTE_PTYPE_L4_UDP ||
 			    l4_flags == RTE_PTYPE_L4_TCP) {
@@ -245,6 +287,7 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 	struct vnic_cq *cq;
 	volatile struct cq_desc *cqd_ptr;
 	uint8_t color;
+	uint8_t tnl;
 	uint16_t seg_length;
 	struct rte_mbuf *first_seg = sop_rq->pkt_first_seg;
 	struct rte_mbuf *last_seg = sop_rq->pkt_last_seg;
@@ -336,10 +379,21 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
 			continue;
 		}
 
+		/*
+		 * When overlay offload is enabled, CQ.fcoe indicates the
+		 * packet is tunnelled.
+		 */
+		tnl = enic->overlay_offload &&
+			(ciflags & CQ_ENET_RQ_DESC_FLAGS_FCOE) != 0;
 		/* cq rx flags are only valid if eop bit is set */
-		first_seg->packet_type = enic_cq_rx_flags_to_pkt_type(&cqd);
+		first_seg->packet_type =
+			enic_cq_rx_flags_to_pkt_type(&cqd, tnl);
 		enic_cq_rx_to_pkt_flags(&cqd, first_seg);
-
+		/* Wipe the outer types set by enic_cq_rx_flags_to_pkt_type() */
+		if (tnl) {
+			first_seg->packet_type &= ~(RTE_PTYPE_L3_MASK |
+						    RTE_PTYPE_L4_MASK);
+		}
 		if (unlikely(packet_error)) {
 			rte_pktmbuf_free(first_seg);
 			rte_atomic64_inc(&enic->soft_stats.rx_packet_errors);
@@ -443,9 +497,10 @@ unsigned int enic_cleanup_wq(__rte_unused struct enic *enic, struct vnic_wq *wq)
 	return 0;
 }
 
-uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
+uint16_t enic_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 			uint16_t nb_pkts)
 {
+	struct vnic_wq *wq = (struct vnic_wq *)tx_queue;
 	int32_t ret;
 	uint16_t i;
 	uint64_t ol_flags;
@@ -454,7 +509,7 @@ uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts,
 	for (i = 0; i != nb_pkts; i++) {
 		m = tx_pkts[i];
 		ol_flags = m->ol_flags;
-		if (ol_flags & ENIC_TX_OFFLOAD_NOTSUP_MASK) {
+		if (ol_flags & wq->tx_offload_notsup_mask) {
 			rte_errno = -ENOTSUP;
 			return i;
 		}
@@ -558,6 +613,11 @@ uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 
 			offload_mode = WQ_ENET_OFFLOAD_MODE_TSO;
 			mss = tx_pkt->tso_segsz;
+			/* For tunnel, need the size of outer+inner headers */
+			if (ol_flags & PKT_TX_TUNNEL_MASK) {
+				header_len += tx_pkt->outer_l2_len +
+					tx_pkt->outer_l3_len;
+			}
 		}
 
 		if ((ol_flags & ol_flags_mask) && (header_len == 0)) {
-- 
2.16.2

Re: [dpdk-dev] [PATCH v2] net/enic: enable overlay offload for VXLAN and GENEVE

[Ferruh Yigit]

On 4/16/2018 10:49 PM, John Daley wrote:
> From: Hyong Youb Kim <hyonkim@cisco.com>
> 
> Recent NIC models support overlay offload. The overlay offload
> feature enables the following on the NIC.
> - Rx/Tx checksum offloads for both inner and outer packets.
> - Rx inner packet type classification.
> - TSO.
> - Inner RSS.
> 
> TX descriptors do not require any changes, except the header length
> for TSO. The NIC parses outer/inner packets and performs offloads on
> them as necessary. The header length for tunneled TSO includes both
> inner and outer headers.
> 
> The NIC actually parses and performs the above for NVGRE as well. DPDK
> currently has no offload flags for NVGRE, and the hardware has no
> controls to individually enable tunnel types either. So do nothing for
> now.
> 
> The driver enables overlay offload by default. Add a devargs
> 'disable-overlay=<0|1>' to allow the app to disable it.
> 
> Also update the enic guide doc.
> 
> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
> Reviewed-by: John Daley <johndale@cisco.com>

Applied to dpdk-next-net/master, thanks.

[dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters

[John Daley]

Recent models support IPv4/IPv6 UDP RSS. There is no control bit to
enable UDP RSS alone. Instead, the NIC enables/disables TCP and UDP
RSS together.

Signed-off-by: John Daley <johndale@cisco.com>
Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>
---
 doc/guides/nics/enic.rst         |  2 +-
 drivers/net/enic/base/vnic_dev.c | 18 ++++++++++++++++++
 drivers/net/enic/base/vnic_dev.h |  1 +
 drivers/net/enic/base/vnic_nic.h |  1 +
 drivers/net/enic/enic_main.c     | 12 ++++++++++++
 drivers/net/enic/enic_res.c      |  5 +++++
 6 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index 133b1e682..d02a69829 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -416,6 +416,7 @@ Supported features
 - VLAN filtering (supported via UCSM/CIMC only)
 - Execution of application by unprivileged system users
 - IPV4, IPV6 and TCP RSS hashing
+- UDP hashing (1400 series and later adapters)
 - Scattered Rx
 - MTU update
 - SR-IOV on UCS managed servers connected to Fabric Interconnects
@@ -434,7 +435,6 @@ Known bugs and unsupported features in this release
 - VLAN based flow direction
 - Non-IPV4 flow direction
 - Setting of extended VLAN
-- UDP RSS hashing
 - MTU update only works if Scattered Rx mode is disabled
 - Maximum receive packet length is ignored if Scattered Rx mode is used
 
diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c
index 386db6411..1d6734dc1 100644
--- a/drivers/net/enic/base/vnic_dev.c
+++ b/drivers/net/enic/base/vnic_dev.c
@@ -10,6 +10,7 @@
 #include "vnic_dev.h"
 #include "vnic_resource.h"
 #include "vnic_devcmd.h"
+#include "vnic_nic.h"
 #include "vnic_stats.h"
 
 
@@ -531,6 +532,23 @@ int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode,
 	return 0;
 }
 
+int vnic_dev_capable_udp_rss(struct vnic_dev *vdev)
+{
+	u64 a0 = CMD_NIC_CFG, a1 = 0;
+	u64 rss_hash_type;
+	int wait = 1000;
+	int err;
+
+	err = vnic_dev_cmd(vdev, CMD_CAPABILITY, &a0, &a1, wait);
+	if (err)
+		return 0;
+	if (a0 == 0)
+		return 0;
+	rss_hash_type = (a1 >> NIC_CFG_RSS_HASH_TYPE_SHIFT) &
+		NIC_CFG_RSS_HASH_TYPE_MASK_FIELD;
+	return ((rss_hash_type & NIC_CFG_RSS_HASH_TYPE_UDP) ? 1 : 0);
+}
+
 int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd)
 {
 	u64 a0 = (u32)cmd, a1 = 0;
diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h
index b3ce26032..62c09fa86 100644
--- a/drivers/net/enic/base/vnic_dev.h
+++ b/drivers/net/enic/base/vnic_dev.h
@@ -109,6 +109,7 @@ int vnic_dev_capable_adv_filters(struct vnic_dev *vdev);
 int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd);
 int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode,
 				 u8 *filter_tags);
+int vnic_dev_capable_udp_rss(struct vnic_dev *vdev);
 int vnic_dev_asic_info(struct vnic_dev *vdev, u16 *asic_type, u16 *asic_rev);
 int vnic_dev_spec(struct vnic_dev *vdev, unsigned int offset, size_t size,
 	void *value);
diff --git a/drivers/net/enic/base/vnic_nic.h b/drivers/net/enic/base/vnic_nic.h
index a753b3a5a..b20915e76 100644
--- a/drivers/net/enic/base/vnic_nic.h
+++ b/drivers/net/enic/base/vnic_nic.h
@@ -33,6 +33,7 @@
 #define NIC_CFG_RSS_HASH_TYPE_TCP_IPV6		(1 << 4)
 #define NIC_CFG_RSS_HASH_TYPE_IPV6_EX		(1 << 5)
 #define NIC_CFG_RSS_HASH_TYPE_TCP_IPV6_EX	(1 << 6)
+#define NIC_CFG_RSS_HASH_TYPE_UDP		(1 << 7)
 
 static inline void vnic_set_nic_cfg(u32 *nic_cfg,
 	u8 rss_default_cpu, u8 rss_hash_type,
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 9b254d98e..f01a77c0a 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -1178,10 +1178,22 @@ int enic_set_rss_conf(struct enic *enic, struct rte_eth_rss_conf *rss_conf)
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV4;
 		if (rss_hf & ETH_RSS_NONFRAG_IPV4_TCP)
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4;
+		if (rss_hf & ETH_RSS_NONFRAG_IPV4_UDP) {
+			/*
+			 * 'TCP' is not a typo. HW does not have a separate
+			 * enable bit for UDP RSS. The TCP bit enables both TCP
+			 * and UDP RSS..
+			 */
+			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4;
+		}
 		if (rss_hf & ETH_RSS_IPV6)
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV6;
 		if (rss_hf & ETH_RSS_NONFRAG_IPV6_TCP)
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6;
+		if (rss_hf & ETH_RSS_NONFRAG_IPV6_UDP) {
+			/* Again, 'TCP' is not a typo. */
+			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6;
+		}
 		if (rss_hf & ETH_RSS_IPV6_EX)
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV6_EX;
 		if (rss_hf & ETH_RSS_IPV6_TCP_EX)
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index 47f825e51..34532720c 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -144,6 +144,11 @@ int enic_get_vnic_config(struct enic *enic)
 		enic->flow_type_rss_offloads |= ETH_RSS_IPV6_EX;
 	if (ENIC_SETTING(enic, RSSHASH_TCPIPV6_EX))
 		enic->flow_type_rss_offloads |= ETH_RSS_IPV6_TCP_EX;
+	if (vnic_dev_capable_udp_rss(enic->vdev)) {
+		enic->flow_type_rss_offloads |=
+			ETH_RSS_NONFRAG_IPV4_UDP | ETH_RSS_NONFRAG_IPV6_UDP;
+	}
+
 	/* Zero offloads if RSS is not enabled */
 	if (!ENIC_SETTING(enic, RSS))
 		enic->flow_type_rss_offloads = 0;
-- 
2.16.2

Re: [dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters

[Ferruh Yigit]

On 4/5/2018 12:54 AM, John Daley wrote:
> Recent models support IPv4/IPv6 UDP RSS. There is no control bit to
> enable UDP RSS alone. Instead, the NIC enables/disables TCP and UDP
> RSS together.
> 
> Signed-off-by: John Daley <johndale@cisco.com>
> Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>

Applied to dpdk-next-net/master, thanks.

[dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues

[John Daley]

The enic code called from rte_eth_dev_set_mtu() was assuming that the
Rx queues are already set up via a call to rte_eth_tx_queue_setup().
OVS calls rte_eth_dev_set_mtu() before rte_eth_rx_queue_setup() and
a null pointer was dereferenced.

Fixes: c3e09182bcd6 ("net/enic: support scatter Rx in MTU update")
Cc: stable@dpdk.org

Signed-off-by: John Daley <johndale@cisco.com>
Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>
---
 drivers/net/enic/enic_main.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index f01a77c0a..f39dfbe32 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -1444,6 +1444,8 @@ int enic_set_mtu(struct enic *enic, uint16_t new_mtu)
 	/* free and reallocate RQs with the new MTU */
 	for (rq_idx = 0; rq_idx < enic->rq_count; rq_idx++) {
 		rq = &enic->rq[enic_rte_rq_idx_to_sop_idx(rq_idx)];
+		if (!rq->in_use)
+			continue;
 
 		enic_free_rq(rq);
 		rc = enic_alloc_rq(enic, rq_idx, rq->socket_id, rq->mp,
-- 
2.16.2

Re: [dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues

[Ferruh Yigit]

On 4/5/2018 12:54 AM, John Daley wrote:
> The enic code called from rte_eth_dev_set_mtu() was assuming that the
> Rx queues are already set up via a call to rte_eth_tx_queue_setup().
> OVS calls rte_eth_dev_set_mtu() before rte_eth_rx_queue_setup() and
> a null pointer was dereferenced.
> 
> Fixes: c3e09182bcd6 ("net/enic: support scatter Rx in MTU update")
> Cc: stable@dpdk.org
> 
> Signed-off-by: John Daley <johndale@cisco.com>
> Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>

Applied to dpdk-next-net/master, thanks.

[dpdk-dev] [PATCH] net/enic: support the drop flow action

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

1330 and 1400 series adapters support the drop action. Check for its
availability and set the necessary flag when creating NIC filters.

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
---
 doc/guides/nics/enic.rst            |  2 +-
 drivers/net/enic/base/vnic_dev.c    | 14 +++++---------
 drivers/net/enic/base/vnic_dev.h    |  2 +-
 drivers/net/enic/base/vnic_devcmd.h |  2 ++
 drivers/net/enic/enic.h             |  2 +-
 drivers/net/enic/enic_flow.c        | 34 +++++++++++++++++++++++++++-------
 drivers/net/enic/enic_res.c         | 11 ++++++++---
 7 files changed, 45 insertions(+), 22 deletions(-)

diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index d02a69829..df3718ecd 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -251,7 +251,7 @@ Generic Flow API is supported. The baseline support is:
 
   - Attributes: ingress
   - Items: eth, ipv4, ipv6, udp, tcp, vxlan, inner eth, ipv4, ipv6, udp, tcp
-  - Actions: queue, mark, flag and void
+  - Actions: queue, mark, drop, flag and void
   - Selectors: 'is', 'spec' and 'mask'. 'last' is not supported
   - In total, up to 64 bytes of mask is allowed across all headers
 
diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c
index 1d6734dc1..8880ab981 100644
--- a/drivers/net/enic/base/vnic_dev.c
+++ b/drivers/net/enic/base/vnic_dev.c
@@ -485,7 +485,7 @@ int vnic_dev_capable_adv_filters(struct vnic_dev *vdev)
  *   Retrun true in filter_tags if supported
  */
 int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode,
-				 u8 *filter_tags)
+				 u8 *filter_actions)
 {
 	u64 args[4];
 	int err;
@@ -493,14 +493,10 @@ int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode,
 
 	err = vnic_dev_advanced_filters_cap(vdev, args, 4);
 
-	/* determine if filter tags are available */
-	if (err)
-		*filter_tags = 0;
-	if ((args[2] == FILTER_CAP_MODE_V1) &&
-	    (args[3] & FILTER_ACTION_FILTER_ID_FLAG))
-		*filter_tags = 1;
-	else
-		*filter_tags = 0;
+	/* determine supported filter actions */
+	*filter_actions = FILTER_ACTION_RQ_STEERING_FLAG; /* always available */
+	if (args[2] == FILTER_CAP_MODE_V1)
+		*filter_actions = args[3];
 
 	if (err || ((args[0] == 1) && (args[1] == 0))) {
 		/* Adv filter Command not supported or adv filters available but
diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h
index 62c09fa86..e7a1f8bd8 100644
--- a/drivers/net/enic/base/vnic_dev.h
+++ b/drivers/net/enic/base/vnic_dev.h
@@ -108,7 +108,7 @@ int vnic_dev_fw_info(struct vnic_dev *vdev,
 int vnic_dev_capable_adv_filters(struct vnic_dev *vdev);
 int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd);
 int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode,
-				 u8 *filter_tags);
+				 u8 *filter_actions);
 int vnic_dev_capable_udp_rss(struct vnic_dev *vdev);
 int vnic_dev_asic_info(struct vnic_dev *vdev, u16 *asic_type, u16 *asic_rev);
 int vnic_dev_spec(struct vnic_dev *vdev, unsigned int offset, size_t size,
diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h
index d09113dac..2865eb4d4 100644
--- a/drivers/net/enic/base/vnic_devcmd.h
+++ b/drivers/net/enic/base/vnic_devcmd.h
@@ -841,7 +841,9 @@ struct filter_action {
 
 #define FILTER_ACTION_RQ_STEERING_FLAG	(1 << 0)
 #define FILTER_ACTION_FILTER_ID_FLAG	(1 << 1)
+#define FILTER_ACTION_DROP_FLAG		(1 << 2)
 #define FILTER_ACTION_V2_ALL		(FILTER_ACTION_RQ_STEERING_FLAG \
+					 | FILTER_ACTION_DROP_FLAG \
 					 | FILTER_ACTION_FILTER_ID_FLAG)
 
 /* Version 2 of filter action must be a strict extension of struct filter_action
diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h
index 9d69d1e9e..e4936ec13 100644
--- a/drivers/net/enic/enic.h
+++ b/drivers/net/enic/enic.h
@@ -119,7 +119,7 @@ struct enic {
 	u16 max_mtu;
 	u8 adv_filters;
 	u32 flow_filter_mode;
-	u8 filter_tags;
+	u8 filter_actions; /* HW supported actions */
 	u8 vxlan;
 
 	unsigned int flags;
diff --git a/drivers/net/enic/enic_flow.c b/drivers/net/enic/enic_flow.c
index 28923b0e2..b9f36587c 100644
--- a/drivers/net/enic/enic_flow.c
+++ b/drivers/net/enic/enic_flow.c
@@ -273,21 +273,33 @@ static const enum rte_flow_action_type enic_supported_actions_v1[] = {
 };
 
 /** Supported actions for newer NICs */
-static const enum rte_flow_action_type enic_supported_actions_v2[] = {
+static const enum rte_flow_action_type enic_supported_actions_v2_id[] = {
 	RTE_FLOW_ACTION_TYPE_QUEUE,
 	RTE_FLOW_ACTION_TYPE_MARK,
 	RTE_FLOW_ACTION_TYPE_FLAG,
 	RTE_FLOW_ACTION_TYPE_END,
 };
 
+static const enum rte_flow_action_type enic_supported_actions_v2_drop[] = {
+	RTE_FLOW_ACTION_TYPE_QUEUE,
+	RTE_FLOW_ACTION_TYPE_MARK,
+	RTE_FLOW_ACTION_TYPE_FLAG,
+	RTE_FLOW_ACTION_TYPE_DROP,
+	RTE_FLOW_ACTION_TYPE_END,
+};
+
 /** Action capabilities indexed by NIC version information */
 static const struct enic_action_cap enic_action_cap[] = {
 	[FILTER_ACTION_RQ_STEERING_FLAG] = {
 		.actions = enic_supported_actions_v1,
 		.copy_fn = enic_copy_action_v1,
 	},
-	[FILTER_ACTION_V2_ALL] = {
-		.actions = enic_supported_actions_v2,
+	[FILTER_ACTION_FILTER_ID_FLAG] = {
+		.actions = enic_supported_actions_v2_id,
+		.copy_fn = enic_copy_action_v2,
+	},
+	[FILTER_ACTION_DROP_FLAG] = {
+		.actions = enic_supported_actions_v2_drop,
 		.copy_fn = enic_copy_action_v2,
 	},
 };
@@ -1021,6 +1033,10 @@ enic_copy_action_v2(const struct rte_flow_action actions[],
 			enic_action->flags |= FILTER_ACTION_FILTER_ID_FLAG;
 			break;
 		}
+		case RTE_FLOW_ACTION_TYPE_DROP: {
+			enic_action->flags |= FILTER_ACTION_DROP_FLAG;
+			break;
+		}
 		case RTE_FLOW_ACTION_TYPE_VOID:
 			continue;
 		default:
@@ -1059,10 +1075,14 @@ enic_get_filter_cap(struct enic *enic)
 static const struct enic_action_cap *
 enic_get_action_cap(struct enic *enic)
 {
-	static const struct enic_action_cap *ea;
-
-	if (enic->filter_tags)
-		ea = &enic_action_cap[FILTER_ACTION_V2_ALL];
+	const struct enic_action_cap *ea;
+	uint8_t actions;
+
+	actions = enic->filter_actions;
+	if (actions & FILTER_ACTION_DROP_FLAG)
+		ea = &enic_action_cap[FILTER_ACTION_DROP_FLAG];
+	else if (actions & FILTER_ACTION_FILTER_ID_FLAG)
+		ea = &enic_action_cap[FILTER_ACTION_FILTER_ID_FLAG];
 	else
 		ea = &enic_action_cap[FILTER_ACTION_RQ_STEERING_FLAG];
 	return ea;
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index 34532720c..de17a31d0 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -76,19 +76,24 @@ int enic_get_vnic_config(struct enic *enic)
 		 ? "" : "not "));
 
 	err = vnic_dev_capable_filter_mode(enic->vdev, &enic->flow_filter_mode,
-					   &enic->filter_tags);
+					   &enic->filter_actions);
 	if (err) {
 		dev_err(enic_get_dev(enic),
 			"Error getting filter modes, %d\n", err);
 		return err;
 	}
 
-	dev_info(enic, "Flow api filter mode: %s, Filter tagging %savailable\n",
+	dev_info(enic, "Flow api filter mode: %s Actions: %s%s%s\n",
 		((enic->flow_filter_mode == FILTER_DPDK_1) ? "DPDK" :
 		((enic->flow_filter_mode == FILTER_USNIC_IP) ? "USNIC" :
 		((enic->flow_filter_mode == FILTER_IPV4_5TUPLE) ? "5TUPLE" :
 		"NONE"))),
-		((enic->filter_tags) ? "" : "not "));
+		((enic->filter_actions & FILTER_ACTION_RQ_STEERING_FLAG) ?
+		 "steer " : ""),
+		((enic->filter_actions & FILTER_ACTION_FILTER_ID_FLAG) ?
+		 "tag " : ""),
+		((enic->filter_actions & FILTER_ACTION_DROP_FLAG) ?
+		 "drop " : ""));
 
 	c->wq_desc_count =
 		min_t(u32, ENIC_MAX_WQ_DESCS,
-- 
2.16.2

Re: [dpdk-dev] [PATCH] net/enic: support the drop flow action

[Ferruh Yigit]

On 4/5/2018 12:54 AM, John Daley wrote:
> From: Hyong Youb Kim <hyonkim@cisco.com>
> 
> 1330 and 1400 series adapters support the drop action. Check for its
> availability and set the necessary flag when creating NIC filters.
> 
> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
> Reviewed-by: John Daley <johndale@cisco.com>

Applied to dpdk-next-net/master, thanks.

[dpdk-dev] [PATCH] doc: update the enic guide and features

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

Documentation updates including for 1400 series VIC adapters.

Remove VLAN filter from the features file as the driver does not
support that API. Hardware does support VLAN filtering, but it is not
controlled through the driver.

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
---
 doc/guides/nics/enic.rst          | 28 +++++++++++++++-------------
 doc/guides/nics/features/enic.ini |  1 -
 2 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index df3718ecd..1e2d0b56a 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -236,7 +236,7 @@ Generic Flow API is supported. The baseline support is:
   - Actions: queue and void
   - Selectors: 'is'
 
-- **1300 series VICS with advanced filters disabled**
+- **1300 and later series VICS with advanced filters disabled**
 
   With advanced filters disabled, an IPv4 or IPv6 item must be specified
   in the pattern.
@@ -247,7 +247,7 @@ Generic Flow API is supported. The baseline support is:
   - Selectors: 'is', 'spec' and 'mask'. 'last' is not supported
   - In total, up to 64 bytes of mask is allowed across all headers
 
-- **1300 series VICS with advanced filters enabled**
+- **1300 and later series VICS with advanced filters enabled**
 
   - Attributes: ingress
   - Items: eth, ipv4, ipv6, udp, tcp, vxlan, inner eth, ipv4, ipv6, udp, tcp
@@ -360,10 +360,20 @@ Limitations
 - **Statistics**
 
   - ``rx_good_bytes`` (ibytes) always includes VLAN header (4B) and CRC bytes (4B).
+    This behavior applies to 1300 and older series VIC adapters.
+    1400 series VICs do not count CRC bytes, and count VLAN header only when VLAN
+    stripping is disabled.
   - When the NIC drops a packet because the Rx queue has no free buffers,
     ``rx_good_bytes`` still increments by 4B if the packet is not VLAN tagged or
     VLAN stripping is disabled, or by 8B if the packet is VLAN tagged and stripping
     is enabled.
+    This behavior applies to 1300 and older series VIC adapters. 1400 series VICs
+    do not increment this byte counter when packets are dropped.
+
+- **RSS Hashing**
+
+  - Hardware enables and disables UDP and TCP RSS hashing together. The driver
+    cannot control UDP and TCP hashing individually.
 
 How to build the suite
 ----------------------
@@ -382,17 +392,9 @@ Supported Cisco VIC adapters
 
 ENIC PMD supports all recent generations of Cisco VIC adapters including:
 
-- VIC 1280
-- VIC 1240
-- VIC 1225
-- VIC 1285
-- VIC 1225T
-- VIC 1227
-- VIC 1227T
-- VIC 1380
-- VIC 1340
-- VIC 1385
-- VIC 1387
+- VIC 1200 series
+- VIC 1300 series
+- VIC 1400 series
 
 Supported Operating Systems
 ---------------------------
diff --git a/doc/guides/nics/features/enic.ini b/doc/guides/nics/features/enic.ini
index ea171a45b..99d37708c 100644
--- a/doc/guides/nics/features/enic.ini
+++ b/doc/guides/nics/features/enic.ini
@@ -19,7 +19,6 @@ RSS hash             = Y
 RSS key update       = Y
 RSS reta update      = Y
 SR-IOV               = Y
-VLAN filter          = Y
 CRC offload          = Y
 VLAN offload         = Y
 Flow director        = Y
-- 
2.16.2

Re: [dpdk-dev] [PATCH] doc: update the enic guide and features

[Ferruh Yigit]

On 4/5/2018 12:54 AM, John Daley wrote:
> From: Hyong Youb Kim <hyonkim@cisco.com>
> 
> Documentation updates including for 1400 series VIC adapters.
> 
> Remove VLAN filter from the features file as the driver does not
> support that API. Hardware does support VLAN filtering, but it is not
> controlled through the driver.
> 
> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
> Reviewed-by: John Daley <johndale@cisco.com>

Applied to dpdk-next-net/master, thanks.

Re: [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC

[Ferruh Yigit]

On 4/5/2018 12:54 AM, John Daley wrote:
> From: Hyong Youb Kim <hyonkim@cisco.com>
> 
> The firmware on new hardware models flushes the global descriptor
> cache by default. Use CMD_OPENF_IG_DESCCACHE to avoid cache
> flushing. This flag has no effect on older models.
> 
> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
> Suggested-by: Govindarajulu Varadarajan <gvaradar@cisco.com>
> Reviewed-by: John Daley <johndale@cisco.com>

Applied to dpdk-next-net/master, thanks.