[dpdk-dev] [PATCH 1/6] net/enic: enable RQ first and then post Rx buffers

[dpdk-dev] [PATCH 1/6] net/enic: enable RQ first and then post Rx buffers

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

Future VIC adapters may require that the driver enable RQ before
posting new buffers to the NIC. So split enic_alloc_rx_queue_mbufs()
into two functions, one that allocates buffers and fills RQ and the
other that posts them (i.e. PIO write to a doorbell). And, call the
post function only after enabling RQ.

Currently released models are not affected by this change, as they
work fine whether the driver posts buffers before or after enabling RQ.

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
Reviewed-by: Aaron Conole <aconole@redhat.com>
---
 drivers/net/enic/base/vnic_rq.h |  2 ++
 drivers/net/enic/enic_main.c    | 27 +++++++++++++++++++++++----
 2 files changed, 25 insertions(+), 4 deletions(-)

diff --git a/drivers/net/enic/base/vnic_rq.h b/drivers/net/enic/base/vnic_rq.h
index d774bb0db..9619290de 100644
--- a/drivers/net/enic/base/vnic_rq.h
+++ b/drivers/net/enic/base/vnic_rq.h
@@ -6,6 +6,7 @@
 #ifndef _VNIC_RQ_H_
 #define _VNIC_RQ_H_
 
+#include <stdbool.h>
 
 #include "vnic_dev.h"
 #include "vnic_cq.h"
@@ -69,6 +70,7 @@ struct vnic_rq {
 	struct rte_mbuf *pkt_last_seg;
 	unsigned int max_mbufs_per_pkt;
 	uint16_t tot_nb_desc;
+	bool need_initial_post;
 };
 
 static inline unsigned int vnic_rq_desc_avail(struct vnic_rq *rq)
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 2a2269794..e2adadcc9 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -315,6 +315,24 @@ enic_alloc_rx_queue_mbufs(struct enic *enic, struct vnic_rq *rq)
 				rq_buf_len);
 		rq->mbuf_ring[i] = mb;
 	}
+	/*
+	 * Do not post the buffers to the NIC until we enable the RQ via
+	 * enic_start_rq().
+	 */
+	rq->need_initial_post = true;
+	return 0;
+}
+
+/*
+ * Post the Rx buffers for the first time. enic_alloc_rx_queue_mbufs() has
+ * allocated the buffers and filled the RQ descriptor ring. Just need to push
+ * the post index to the NIC.
+ */
+static void
+enic_initial_post_rx(struct enic *enic, struct vnic_rq *rq)
+{
+	if (!rq->in_use || !rq->need_initial_post)
+		return;
 
 	/* make sure all prior writes are complete before doing the PIO write */
 	rte_rmb();
@@ -329,9 +347,7 @@ enic_alloc_rx_queue_mbufs(struct enic *enic, struct vnic_rq *rq)
 	iowrite32(rq->posted_index, &rq->ctrl->posted_index);
 	iowrite32(0, &rq->ctrl->fetch_index);
 	rte_rmb();
-
-	return 0;
-
+	rq->need_initial_post = false;
 }
 
 static void *
@@ -619,10 +635,13 @@ void enic_start_rq(struct enic *enic, uint16_t queue_idx)
 	rq_data = &enic->rq[rq_sop->data_queue_idx];
 	struct rte_eth_dev *eth_dev = enic->rte_dev;
 
-	if (rq_data->in_use)
+	if (rq_data->in_use) {
 		vnic_rq_enable(rq_data);
+		enic_initial_post_rx(enic, rq_data);
+	}
 	rte_mb();
 	vnic_rq_enable(rq_sop);
+	enic_initial_post_rx(enic, rq_sop);
 	eth_dev->data->rx_queue_state[queue_idx] = RTE_ETH_QUEUE_STATE_STARTED;
 }
 
-- 
2.16.2

[dpdk-dev] [PATCH 2/6] net/enic: fix the MTU handler to rely on max packet length

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

The RQ setup functions (enic_alloc_rq and enic_alloc_rx_queue_mbufs)
have changed to rely on max_rx_pkt_len to determine the use of scatter
and buffer size. But, the MTU handler only updates ethdev's MTU
value. So make it update max_rx_pkt_len as well. Other PMDs also
update both mtu and max_rx_pkt_len in their MTU handlers.

Also the condition for taking a short cut (scatter is disabled) in the
MTU handler is wrong. Even when scatter is disabled, a change in
max_rx_pkt_len may affect the buffer size posted to the NIC. So remove
that condition.

Finally, fix a comment and a warning message condition.

Fixes: 422ba91716a7 ("net/enic: heed the requested max Rx packet size")

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
Reviewed-by: Aaron Conole <aconole@redhat.com>
---
 drivers/net/enic/enic_main.c | 34 +++++++++++++++++++---------------
 1 file changed, 19 insertions(+), 15 deletions(-)

diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index e2adadcc9..b3b4e9626 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -287,7 +287,7 @@ enic_alloc_rx_queue_mbufs(struct enic *enic, struct vnic_rq *rq)
 		  rq->ring.desc_count);
 
 	/*
-	 * If *not* using scatter and the mbuf size is smaller than the
+	 * If *not* using scatter and the mbuf size is greater than the
 	 * requested max packet size (max_rx_pkt_len), then reduce the
 	 * posted buffer size to max_rx_pkt_len. HW still receives packets
 	 * larger than max_rx_pkt_len, but they will be truncated, which we
@@ -730,7 +730,7 @@ int enic_alloc_rq(struct enic *enic, uint16_t queue_idx,
 		 * See enic_alloc_rx_queue_mbufs().
 		 */
 		if (max_rx_pkt_len <
-		    enic_mtu_to_max_rx_pktlen(enic->rte_dev->data->mtu)) {
+		    enic_mtu_to_max_rx_pktlen(enic->max_mtu)) {
 			dev_warning(enic, "rxmode.max_rx_pkt_len is ignored"
 				    " when scatter rx mode is in use.\n");
 		}
@@ -1416,20 +1416,26 @@ int enic_set_mtu(struct enic *enic, uint16_t new_mtu)
 			"MTU (%u) is greater than value configured in NIC (%u)\n",
 			new_mtu, config_mtu);
 
-	/* The easy case is when scatter is disabled. However if the MTU
-	 * becomes greater than the mbuf data size, packet drops will ensue.
+	/* Update the MTU and maximum packet length */
+	eth_dev->data->mtu = new_mtu;
+	eth_dev->data->dev_conf.rxmode.max_rx_pkt_len =
+		enic_mtu_to_max_rx_pktlen(new_mtu);
+
+	/*
+	 * If the device has not started (enic_enable), nothing to do.
+	 * Later, enic_enable() will set up RQs reflecting the new maximum
+	 * packet length.
 	 */
-	if (!(enic->rte_dev->data->dev_conf.rxmode.offloads &
-	      DEV_RX_OFFLOAD_SCATTER)) {
-		eth_dev->data->mtu = new_mtu;
+	if (!eth_dev->data->dev_started)
 		goto set_mtu_done;
-	}
 
-	/* Rx scatter is enabled so reconfigure RQ's on the fly. The point is to
-	 * change Rx scatter mode if necessary for better performance. I.e. if
-	 * MTU was greater than the mbuf size and now it's less, scatter Rx
-	 * doesn't have to be used and vice versa.
-	  */
+	/*
+	 * The device has started, re-do RQs on the fly. In the process, we
+	 * pick up the new maximum packet length.
+	 *
+	 * Some applications rely on the ability to change MTU without stopping
+	 * the device. So keep this behavior for now.
+	 */
 	rte_spinlock_lock(&enic->mtu_lock);
 
 	/* Stop traffic on all RQs */
@@ -1454,8 +1460,6 @@ int enic_set_mtu(struct enic *enic, uint16_t new_mtu)
 
 	/* now it is safe to reconfigure the RQs */
 
-	/* update the mtu */
-	eth_dev->data->mtu = new_mtu;
 
 	/* free and reallocate RQs with the new MTU */
 	for (rq_idx = 0; rq_idx < enic->rq_count; rq_idx++) {
-- 
2.16.2

[dpdk-dev] [PATCH 3/6] net/enic: set rte errno to positive value

[John Daley]

From: johndale <johndale@cisco.com>

Related to d9fff8a31, where rte_errno should always have positive
errno values.

Technically this is an ABI change since it fixes an error code
introduced in 18.02, but is minor and inconsequential. 

Fixes: 1e81dbb5321b ("net/enic: add Tx prepare handler")

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
Reviewed-by: Aaron Conole <aconole@redhat.com>
---
 drivers/net/enic/enic_rxtx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/enic/enic_rxtx.c b/drivers/net/enic/enic_rxtx.c
index aa3393700..8853a2044 100644
--- a/drivers/net/enic/enic_rxtx.c
+++ b/drivers/net/enic/enic_rxtx.c
@@ -510,7 +510,7 @@ uint16_t enic_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
 		m = tx_pkts[i];
 		ol_flags = m->ol_flags;
 		if (ol_flags & wq->tx_offload_notsup_mask) {
-			rte_errno = -ENOTSUP;
+			rte_errno = ENOTSUP;
 			return i;
 		}
 #ifdef RTE_LIBRTE_ETHDEV_DEBUG
-- 
2.16.2

[dpdk-dev] [PATCH 4/6] doc: update the enic guide and features

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

Add more descriptions regarding SR-IOV and RSS settings.

Remove 'Multicast MAC filter' and add 'Allmulticast mode' to the
features.

Cc: stable@dpdk.org

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
Reviewed-by: Aaron Conole <aconole@redhat.com>
---
 doc/guides/nics/enic.rst          | 60 ++++++++++++++++++++++++++++++---------
 doc/guides/nics/features/enic.ini |  3 +-
 2 files changed, 49 insertions(+), 14 deletions(-)

diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst
index 49abc7e95..4505dcf48 100644
--- a/doc/guides/nics/enic.rst
+++ b/doc/guides/nics/enic.rst
@@ -124,6 +124,14 @@ Configuration information
     least one interrupt for each Rx queue. For example, if the app uses 3 Rx
     queues and wants to use per-queue interrupts, configure 4 (3 + 1) interrupts.
 
+  - **Receive Side Scaling**
+
+    In order to fully utilize RSS in DPDK, enable all RSS related settings in
+    CIMC or UCSM. These include the following items listed under
+    Receive Side Scaling:
+    TCP, IPv4, TCP-IPv4, IPv6, TCP-IPv6, IPv6 Extension, TCP-IPv6 Extension.
+
+
 .. _enic-flow-director:
 
 Flow director support
@@ -145,20 +153,21 @@ perfect filtering of the 5-tuple with no masking of fields supported.
 SR-IOV mode utilization
 -----------------------
 
-UCS blade servers configured with dynamic vNIC connection policies in UCS
-manager are capable of supporting assigned devices on virtual machines (VMs)
-through a KVM hypervisor. Assigned devices, also known as 'passthrough'
-devices, are SR-IOV virtual functions (VFs) on the host which are exposed
-to VM instances.
+UCS blade servers configured with dynamic vNIC connection policies in UCSM
+are capable of supporting SR-IOV. SR-IOV virtual functions (VFs) are
+specialized vNICs, distinct from regular Ethernet vNICs. These VFs can be
+directly assigned to virtual machines (VMs) as 'passthrough' devices.
 
-The Cisco Virtual Machine Fabric Extender (VM-FEX) gives the VM a dedicated
+In UCS, SR-IOV VFs require the use of the Cisco Virtual Machine Fabric Extender
+(VM-FEX), which gives the VM a dedicated
 interface on the Fabric Interconnect (FI). Layer 2 switching is done at
 the FI. This may eliminate the requirement for software switching on the
 host to route intra-host VM traffic.
 
 Please refer to `Creating a Dynamic vNIC Connection Policy
 <http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/vm_fex/vmware/gui/config_guide/b_GUI_VMware_VM-FEX_UCSM_Configuration_Guide/b_GUI_VMware_VM-FEX_UCSM_Configuration_Guide_chapter_010.html#task_433E01651F69464783A68E66DA8A47A5>`_
-for information on configuring SR-IOV adapter policies using UCS manager.
+for information on configuring SR-IOV adapter policies and port profiles
+using UCSM.
 
 Once the policies are in place and the host OS is rebooted, VFs should be
 visible on the host, E.g.:
@@ -175,30 +184,37 @@ visible on the host, E.g.:
      0d:00.6 Ethernet controller: Cisco Systems Inc VIC SR-IOV VF (rev a2)
      0d:00.7 Ethernet controller: Cisco Systems Inc VIC SR-IOV VF (rev a2)
 
-Enable Intel IOMMU on the host and install KVM and libvirt. A VM instance should
-be created with an assigned device. When using libvirt, this configuration can
-be done within the domain (i.e. VM) config file. For example this entry maps
-host VF 0d:00:01 into the VM.
+Enable Intel IOMMU on the host and install KVM and libvirt, and reboot again as
+required. Then, using libvirt, create a VM instance with an assigned device.
+Below is an example ``interface`` block (part of the domain configuration XML)
+that adds the host VF 0d:00:01 to the VM. ``profileid='pp-vlan-25'`` indicates
+the port profile that has been configured in UCSM.
 
 .. code-block:: console
 
     <interface type='hostdev' managed='yes'>
       <mac address='52:54:00:ac:ff:b6'/>
+      <driver name='vfio'/>
       <source>
         <address type='pci' domain='0x0000' bus='0x0d' slot='0x00' function='0x1'/>
       </source>
+      <virtualport type='802.1Qbh'>
+        <parameters profileid='pp-vlan-25'/>
+      </virtualport>
+    </interface>
+
 
 Alternatively, the configuration can be done in a separate file using the
 ``network`` keyword. These methods are described in the libvirt documentation for
 `Network XML format <https://libvirt.org/formatnetwork.html>`_.
 
-When the VM instance is started, the ENIC KVM driver will bind the host VF to
+When the VM instance is started, libvirt will bind the host VF to
 vfio, complete provisioning on the FI and bring up the link.
 
 .. note::
 
     It is not possible to use a VF directly from the host because it is not
-    fully provisioned until the hypervisor brings up the VM that it is assigned
+    fully provisioned until libvirt brings up the VM that it is assigned
     to.
 
 In the VM instance, the VF will now be visible. E.g., here the VF 00:04.0 is
@@ -212,9 +228,27 @@ seen on the VM instance and should be available for binding to a DPDK.
 Follow the normal DPDK install procedure, binding the VF to either ``igb_uio``
 or ``vfio`` in non-IOMMU mode.
 
+In the VM, the kernel enic driver may be automatically bound to the VF during
+boot. Unbinding it currently hangs due to a known issue with the driver. To
+work around the issue, blacklist the enic module as follows.
 Please see :ref:`Limitations <enic_limitations>` for limitations in
 the use of SR-IOV.
 
+.. code-block:: console
+
+     # cat /etc/modprobe.d/enic.conf
+     blacklist enic
+
+     # dracut --force
+
+.. note::
+
+    Passthrough does not require SR-IOV. If VM-FEX is not desired, the user
+    may create as many regular vNICs as necessary and assign them to VMs as
+    passthrough devices. Since these vNICs are not SR-IOV VFs, using them as
+    passthrough devices do not require libvirt, port profiles, and VM-FEX.
+
+
 .. _enic-genic-flow-api:
 
 Generic Flow API support
diff --git a/doc/guides/nics/features/enic.ini b/doc/guides/nics/features/enic.ini
index 99d37708c..ae46d299a 100644
--- a/doc/guides/nics/features/enic.ini
+++ b/doc/guides/nics/features/enic.ini
@@ -13,8 +13,9 @@ Jumbo frame          = Y
 Scattered Rx         = Y
 TSO                  = Y
 Promiscuous mode     = Y
+Allmulticast mode    = Y
 Unicast MAC filter   = Y
-Multicast MAC filter = Y
+Multicast MAC filter =
 RSS hash             = Y
 RSS key update       = Y
 RSS reta update      = Y
-- 
2.16.2

[dpdk-dev] [PATCH 5/6] net/enic: fix RSS hash type advertisement

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

The NIC can hash these RSS packet types, but they are not advertised
via flow_type_rss_offloads. So add them.
- Part of the IPv4 hash:
  ETH_RSS_FRAG_IPV4
  ETH_RSS_NONFRAG_IPV4_OTHER
- Part of the IPv6 hash:
  ETH_RSS_FRAG_IPV6
  ETH_RSS_NONFRAG_IPV6_OTHER
- Part of the UDP hash:
  ETH_RSS_IPV6_UDP_EX

Also, do not use NIC_CFG_RSS_HASH_TYPE_IPV6_EX and
NIC_CFG_RSS_HASH_TYPE_TCP_IPV6_EX, as they are not needed to enable
RSS over IPv6 with extension headers.

Fixes: c2fec27b5cb0 ("net/enic: allow to change RSS settings")

Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
Reviewed-by: Aaron Conole <aconole@redhat.com>
---
 drivers/net/enic/enic_main.c | 14 ++++++--------
 drivers/net/enic/enic_res.c  | 26 ++++++++++++++++++--------
 2 files changed, 24 insertions(+), 16 deletions(-)

diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index b3b4e9626..2b1c1347c 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -1190,7 +1190,8 @@ int enic_set_rss_conf(struct enic *enic, struct rte_eth_rss_conf *rss_conf)
 	    (eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) &&
 	    rss_hf != 0) {
 		rss_enable = 1;
-		if (rss_hf & ETH_RSS_IPV4)
+		if (rss_hf & (ETH_RSS_IPV4 | ETH_RSS_FRAG_IPV4 |
+			      ETH_RSS_NONFRAG_IPV4_OTHER))
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV4;
 		if (rss_hf & ETH_RSS_NONFRAG_IPV4_TCP)
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4;
@@ -1202,18 +1203,15 @@ int enic_set_rss_conf(struct enic *enic, struct rte_eth_rss_conf *rss_conf)
 			 */
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4;
 		}
-		if (rss_hf & ETH_RSS_IPV6)
+		if (rss_hf & (ETH_RSS_IPV6 | ETH_RSS_IPV6_EX |
+			      ETH_RSS_FRAG_IPV6 | ETH_RSS_NONFRAG_IPV6_OTHER))
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV6;
-		if (rss_hf & ETH_RSS_NONFRAG_IPV6_TCP)
+		if (rss_hf & (ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_IPV6_TCP_EX))
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6;
-		if (rss_hf & ETH_RSS_NONFRAG_IPV6_UDP) {
+		if (rss_hf & (ETH_RSS_NONFRAG_IPV6_UDP | ETH_RSS_IPV6_UDP_EX)) {
 			/* Again, 'TCP' is not a typo. */
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6;
 		}
-		if (rss_hf & ETH_RSS_IPV6_EX)
-			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV6_EX;
-		if (rss_hf & ETH_RSS_IPV6_TCP_EX)
-			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6_EX;
 	} else {
 		rss_enable = 0;
 		rss_hf = 0;
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index de17a31d0..bdda2c564 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -138,20 +138,30 @@ int enic_get_vnic_config(struct enic *enic)
 	enic->hash_key_size = ENIC_RSS_HASH_KEY_SIZE;
 	enic->flow_type_rss_offloads = 0;
 	if (ENIC_SETTING(enic, RSSHASH_IPV4))
-		enic->flow_type_rss_offloads |= ETH_RSS_IPV4;
+		/*
+		 * IPV4 hash type handles both non-frag and frag packet types.
+		 * TCP/UDP is controlled via a separate flag below.
+		 */
+		enic->flow_type_rss_offloads |= ETH_RSS_IPV4 |
+			ETH_RSS_FRAG_IPV4 | ETH_RSS_NONFRAG_IPV4_OTHER;
 	if (ENIC_SETTING(enic, RSSHASH_TCPIPV4))
 		enic->flow_type_rss_offloads |= ETH_RSS_NONFRAG_IPV4_TCP;
 	if (ENIC_SETTING(enic, RSSHASH_IPV6))
-		enic->flow_type_rss_offloads |= ETH_RSS_IPV6;
+		/*
+		 * The VIC adapter can perform RSS on IPv6 packets with and
+		 * without extension headers. An IPv6 "fragment" is an IPv6
+		 * packet with the fragment extension header.
+		 */
+		enic->flow_type_rss_offloads |= ETH_RSS_IPV6 |
+			ETH_RSS_IPV6_EX | ETH_RSS_FRAG_IPV6 |
+			ETH_RSS_NONFRAG_IPV6_OTHER;
 	if (ENIC_SETTING(enic, RSSHASH_TCPIPV6))
-		enic->flow_type_rss_offloads |= ETH_RSS_NONFRAG_IPV6_TCP;
-	if (ENIC_SETTING(enic, RSSHASH_IPV6_EX))
-		enic->flow_type_rss_offloads |= ETH_RSS_IPV6_EX;
-	if (ENIC_SETTING(enic, RSSHASH_TCPIPV6_EX))
-		enic->flow_type_rss_offloads |= ETH_RSS_IPV6_TCP_EX;
+		enic->flow_type_rss_offloads |= ETH_RSS_NONFRAG_IPV6_TCP |
+			ETH_RSS_IPV6_TCP_EX;
 	if (vnic_dev_capable_udp_rss(enic->vdev)) {
 		enic->flow_type_rss_offloads |=
-			ETH_RSS_NONFRAG_IPV4_UDP | ETH_RSS_NONFRAG_IPV6_UDP;
+			ETH_RSS_NONFRAG_IPV4_UDP | ETH_RSS_NONFRAG_IPV6_UDP |
+			ETH_RSS_IPV6_UDP_EX;
 	}
 
 	/* Zero offloads if RSS is not enabled */
-- 
2.16.2

[dpdk-dev] [PATCH 6/6] net/enic: update UDP RSS controls

[John Daley]

From: Hyong Youb Kim <hyonkim@cisco.com>

Current adapters which support UDP RSS piggyback on TCP RSS. Change
the controls to be forward compatible with future adapters, which will
have independent control of UDP and TCP.

Fixes: 9bd04182bb01 ("net/enic: support UDP RSS on 1400 series adapters")

Signed-off-by: John Daley <johndale@cisco.com>
Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>
Reviewed-by: Aaron Conole <aconole@redhat.com>
---
 drivers/net/enic/base/vnic_dev.c  | 17 -----------------
 drivers/net/enic/base/vnic_dev.h  |  1 -
 drivers/net/enic/base/vnic_enet.h |  4 ++++
 drivers/net/enic/base/vnic_nic.h  |  3 ++-
 drivers/net/enic/enic_main.c      | 20 ++++++++++++--------
 drivers/net/enic/enic_res.c       | 13 ++++++++++---
 6 files changed, 28 insertions(+), 30 deletions(-)

diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c
index 8880ab981..8483f76f3 100644
--- a/drivers/net/enic/base/vnic_dev.c
+++ b/drivers/net/enic/base/vnic_dev.c
@@ -528,23 +528,6 @@ int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode,
 	return 0;
 }
 
-int vnic_dev_capable_udp_rss(struct vnic_dev *vdev)
-{
-	u64 a0 = CMD_NIC_CFG, a1 = 0;
-	u64 rss_hash_type;
-	int wait = 1000;
-	int err;
-
-	err = vnic_dev_cmd(vdev, CMD_CAPABILITY, &a0, &a1, wait);
-	if (err)
-		return 0;
-	if (a0 == 0)
-		return 0;
-	rss_hash_type = (a1 >> NIC_CFG_RSS_HASH_TYPE_SHIFT) &
-		NIC_CFG_RSS_HASH_TYPE_MASK_FIELD;
-	return ((rss_hash_type & NIC_CFG_RSS_HASH_TYPE_UDP) ? 1 : 0);
-}
-
 int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd)
 {
 	u64 a0 = (u32)cmd, a1 = 0;
diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h
index e7a1f8bd8..3c9084304 100644
--- a/drivers/net/enic/base/vnic_dev.h
+++ b/drivers/net/enic/base/vnic_dev.h
@@ -109,7 +109,6 @@ int vnic_dev_capable_adv_filters(struct vnic_dev *vdev);
 int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd);
 int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode,
 				 u8 *filter_actions);
-int vnic_dev_capable_udp_rss(struct vnic_dev *vdev);
 int vnic_dev_asic_info(struct vnic_dev *vdev, u16 *asic_type, u16 *asic_rev);
 int vnic_dev_spec(struct vnic_dev *vdev, unsigned int offset, size_t size,
 	void *value);
diff --git a/drivers/net/enic/base/vnic_enet.h b/drivers/net/enic/base/vnic_enet.h
index 26918335f..49504a7da 100644
--- a/drivers/net/enic/base/vnic_enet.h
+++ b/drivers/net/enic/base/vnic_enet.h
@@ -52,6 +52,10 @@ struct vnic_enet_config {
 #define VENETF_VXLAN    0x10000 /* VxLAN offload */
 #define VENETF_NVGRE    0x20000 /* NVGRE offload */
 #define VENETF_GRPINTR  0x40000 /* group interrupt */
+#define VENETF_NICSWITCH        0x80000 /* NICSWITCH enabled */
+#define VENETF_RSSHASH_UDP_WEAK 0x100000 /* VIC has Bodega-style UDP RSS */
+#define VENETF_RSSHASH_UDPIPV4  0x200000 /* Hash on UDP + IPv4 fields */
+#define VENETF_RSSHASH_UDPIPV6  0x400000 /* Hash on UDP + IPv6 fields */
 
 #define VENET_INTR_TYPE_MIN	0	/* Timer specs min interrupt spacing */
 #define VENET_INTR_TYPE_IDLE	1	/* Timer specs idle time before irq */
diff --git a/drivers/net/enic/base/vnic_nic.h b/drivers/net/enic/base/vnic_nic.h
index b20915e76..e318d0cb5 100644
--- a/drivers/net/enic/base/vnic_nic.h
+++ b/drivers/net/enic/base/vnic_nic.h
@@ -27,13 +27,14 @@
 #define NIC_CFG_IG_VLAN_STRIP_EN_MASK_FIELD	1UL
 #define NIC_CFG_IG_VLAN_STRIP_EN_SHIFT		24
 
+#define NIC_CFG_RSS_HASH_TYPE_UDP_IPV4		(1 << 0)
 #define NIC_CFG_RSS_HASH_TYPE_IPV4		(1 << 1)
 #define NIC_CFG_RSS_HASH_TYPE_TCP_IPV4		(1 << 2)
 #define NIC_CFG_RSS_HASH_TYPE_IPV6		(1 << 3)
 #define NIC_CFG_RSS_HASH_TYPE_TCP_IPV6		(1 << 4)
 #define NIC_CFG_RSS_HASH_TYPE_IPV6_EX		(1 << 5)
 #define NIC_CFG_RSS_HASH_TYPE_TCP_IPV6_EX	(1 << 6)
-#define NIC_CFG_RSS_HASH_TYPE_UDP		(1 << 7)
+#define NIC_CFG_RSS_HASH_TYPE_UDP_IPV6		(1 << 7)
 
 static inline void vnic_set_nic_cfg(u32 *nic_cfg,
 	u8 rss_default_cpu, u8 rss_hash_type,
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 2b1c1347c..a25d303de 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -1196,12 +1196,15 @@ int enic_set_rss_conf(struct enic *enic, struct rte_eth_rss_conf *rss_conf)
 		if (rss_hf & ETH_RSS_NONFRAG_IPV4_TCP)
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4;
 		if (rss_hf & ETH_RSS_NONFRAG_IPV4_UDP) {
-			/*
-			 * 'TCP' is not a typo. HW does not have a separate
-			 * enable bit for UDP RSS. The TCP bit enables both TCP
-			 * and UDP RSS..
-			 */
-			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4;
+			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_UDP_IPV4;
+			if (ENIC_SETTING(enic, RSSHASH_UDP_WEAK)) {
+				/*
+				 * 'TCP' is not a typo. The "weak" version of
+				 * UDP RSS requires both the TCP and UDP bits
+				 * be set. It does enable TCP RSS as well.
+				 */
+				rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4;
+			}
 		}
 		if (rss_hf & (ETH_RSS_IPV6 | ETH_RSS_IPV6_EX |
 			      ETH_RSS_FRAG_IPV6 | ETH_RSS_NONFRAG_IPV6_OTHER))
@@ -1209,8 +1212,9 @@ int enic_set_rss_conf(struct enic *enic, struct rte_eth_rss_conf *rss_conf)
 		if (rss_hf & (ETH_RSS_NONFRAG_IPV6_TCP | ETH_RSS_IPV6_TCP_EX))
 			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6;
 		if (rss_hf & (ETH_RSS_NONFRAG_IPV6_UDP | ETH_RSS_IPV6_UDP_EX)) {
-			/* Again, 'TCP' is not a typo. */
-			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6;
+			rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_UDP_IPV6;
+			if (ENIC_SETTING(enic, RSSHASH_UDP_WEAK))
+				rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6;
 		}
 	} else {
 		rss_enable = 0;
diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c
index bdda2c564..a504de5d5 100644
--- a/drivers/net/enic/enic_res.c
+++ b/drivers/net/enic/enic_res.c
@@ -122,7 +122,10 @@ int enic_get_vnic_config(struct enic *enic)
 		"loopback tag 0x%04x\n",
 		ENIC_SETTING(enic, TXCSUM) ? "yes" : "no",
 		ENIC_SETTING(enic, RXCSUM) ? "yes" : "no",
-		ENIC_SETTING(enic, RSS) ? "yes" : "no",
+		ENIC_SETTING(enic, RSS) ?
+			(ENIC_SETTING(enic, RSSHASH_UDPIPV4) ? "+UDP" :
+			((ENIC_SETTING(enic, RSSHASH_UDP_WEAK) ? "+udp" :
+			"yes"))) : "no",
 		c->intr_mode == VENET_INTR_MODE_INTX ? "INTx" :
 		c->intr_mode == VENET_INTR_MODE_MSI ? "MSI" :
 		c->intr_mode == VENET_INTR_MODE_ANY ? "any" :
@@ -158,11 +161,15 @@ int enic_get_vnic_config(struct enic *enic)
 	if (ENIC_SETTING(enic, RSSHASH_TCPIPV6))
 		enic->flow_type_rss_offloads |= ETH_RSS_NONFRAG_IPV6_TCP |
 			ETH_RSS_IPV6_TCP_EX;
-	if (vnic_dev_capable_udp_rss(enic->vdev)) {
+	if (ENIC_SETTING(enic, RSSHASH_UDP_WEAK))
 		enic->flow_type_rss_offloads |=
 			ETH_RSS_NONFRAG_IPV4_UDP | ETH_RSS_NONFRAG_IPV6_UDP |
 			ETH_RSS_IPV6_UDP_EX;
-	}
+	if (ENIC_SETTING(enic, RSSHASH_UDPIPV4))
+		enic->flow_type_rss_offloads |= ETH_RSS_NONFRAG_IPV4_UDP;
+	if (ENIC_SETTING(enic, RSSHASH_UDPIPV6))
+		enic->flow_type_rss_offloads |= ETH_RSS_NONFRAG_IPV6_UDP |
+			ETH_RSS_IPV6_UDP_EX;
 
 	/* Zero offloads if RSS is not enabled */
 	if (!ENIC_SETTING(enic, RSS))
-- 
2.16.2

Re: [dpdk-dev] [PATCH 1/6] net/enic: enable RQ first and then post Rx buffers

[Ferruh Yigit]

On 5/3/2018 8:37 PM, John Daley wrote:
> From: Hyong Youb Kim <hyonkim@cisco.com>
> 
> Future VIC adapters may require that the driver enable RQ before
> posting new buffers to the NIC. So split enic_alloc_rx_queue_mbufs()
> into two functions, one that allocates buffers and fills RQ and the
> other that posts them (i.e. PIO write to a doorbell). And, call the
> post function only after enabling RQ.
> 
> Currently released models are not affected by this change, as they
> work fine whether the driver posts buffers before or after enabling RQ.
> 
> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
> Reviewed-by: John Daley <johndale@cisco.com>
> Reviewed-by: Aaron Conole <aconole@redhat.com>

Series applied to dpdk-next-net/master, thanks.