* [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC
@ 2018-04-04 23:54 John Daley
2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE John Daley
` (5 more replies)
0 siblings, 6 replies; 18+ messages in thread
From: John Daley @ 2018-04-04 23:54 UTC (permalink / raw)
To: ferruh.yigit; +Cc: dev, Hyong Youb Kim
From: Hyong Youb Kim <hyonkim@cisco.com>
The firmware on new hardware models flushes the global descriptor
cache by default. Use CMD_OPENF_IG_DESCCACHE to avoid cache
flushing. This flag has no effect on older models.
Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com>
Suggested-by: Govindarajulu Varadarajan <gvaradar@cisco.com>
Reviewed-by: John Daley <johndale@cisco.com>
---
drivers/net/enic/base/vnic_devcmd.h | 1 +
drivers/net/enic/enic_main.c | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h
index 6b95bc488..a63624559 100644
--- a/drivers/net/enic/base/vnic_devcmd.h
+++ b/drivers/net/enic/base/vnic_devcmd.h
@@ -600,6 +600,7 @@ enum filter_cap_mode {
/* flags for CMD_OPEN */
#define CMD_OPENF_OPROM 0x1 /* open coming from option rom */
+#define CMD_OPENF_IG_DESCCACHE 0x2 /* Do not flush IG DESC cache */
/* flags for CMD_INIT */
#define CMD_INITF_DEFAULT_MAC 0x1 /* init with default mac addr */
diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c
index 69ad42555..e908e20f2 100644
--- a/drivers/net/enic/enic_main.c
+++ b/drivers/net/enic/enic_main.c
@@ -983,9 +983,10 @@ static int enic_dev_wait(struct vnic_dev *vdev,
static int enic_dev_open(struct enic *enic)
{
int err;
+ int flags = CMD_OPENF_IG_DESCCACHE;
err = enic_dev_wait(enic->vdev, vnic_dev_open,
- vnic_dev_open_done, 0);
+ vnic_dev_open_done, flags);
if (err)
dev_err(enic_get_dev(enic),
"vNIC device open failed, err %d\n", err);
--
2.16.2
^ permalink raw reply [flat|nested] 18+ messages in thread* [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE 2018-04-04 23:54 [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC John Daley @ 2018-04-04 23:54 ` John Daley 2018-04-06 16:15 ` Ferruh Yigit 2018-04-16 21:49 ` [dpdk-dev] [PATCH v2] " John Daley 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters John Daley ` (4 subsequent siblings) 5 siblings, 2 replies; 18+ messages in thread From: John Daley @ 2018-04-04 23:54 UTC (permalink / raw) To: ferruh.yigit; +Cc: dev, Hyong Youb Kim From: Hyong Youb Kim <hyonkim@cisco.com> Recent NIC models support overlay offload. The overlay offload feature enables the following on the NIC. - Rx/Tx checksum offloads for both inner and outer packets. - Rx inner packet type classification. - TSO. - Inner RSS. TX descriptors do not require any changes, except the header length for TSO. The NIC parses outer/inner packets and performs offloads on them as necessary. The header length for tunneled TSO includes both inner and outer headers. The NIC actually parses and performs the above for NVGRE as well. DPDK currently has no offload flags for NVGRE, and the hardware has no controls to individually enable tunnel types either. So do nothing for now. Add a config flag to enable overlay offload by default. To disable it, the user should set it to 'n'. CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y Also update the enic guide doc. Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> Reviewed-by: John Daley <johndale@cisco.com> --- config/common_base | 1 + doc/guides/nics/enic.rst | 52 ++++++++++++++++++ drivers/net/enic/base/vnic_dev.c | 33 ++++++++++++ drivers/net/enic/base/vnic_dev.h | 5 +- drivers/net/enic/base/vnic_devcmd.h | 12 +++++ drivers/net/enic/base/vnic_wq.h | 1 + drivers/net/enic/enic.h | 6 +++ drivers/net/enic/enic_ethdev.c | 21 ++------ drivers/net/enic/enic_main.c | 25 +++++++++ drivers/net/enic/enic_res.c | 23 ++++++++ drivers/net/enic/enic_rxtx.c | 104 ++++++++++++++++++++++++++++-------- 11 files changed, 241 insertions(+), 42 deletions(-) diff --git a/config/common_base b/config/common_base index c09c7cf88..964e37b6e 100644 --- a/config/common_base +++ b/config/common_base @@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n # Compile burst-oriented Cisco ENIC PMD driver # CONFIG_RTE_LIBRTE_ENIC_PMD=y +CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y # # Compile burst-oriented IGB & EM PMD drivers diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index 0bc55936a..133b1e682 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -52,6 +52,10 @@ Configuration information - **CONFIG_RTE_LIBRTE_ENIC_PMD** (default y): Enables or disables inclusion of the ENIC PMD driver in the DPDK compilation. + - **CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD** (default y): Enables or + disables overlay offload. + Please see :ref:`Overlay Offload <overlay_offload>`. + - **vNIC Configuration Parameters** - **Number of Queues** @@ -254,6 +258,49 @@ Generic Flow API is supported. The baseline support is: More features may be added in future firmware and new versions of the VIC. Please refer to the release notes. +.. _overlay_offload: + +Overlay Offload +--------------- + +Recent hardware models support overlay offload. When enabled, the NIC performs +the following operations for VXLAN, NVGRE, and GENEVE packets. In all cases, +inner and outer packets can be IPv4 or IPv6. + +- TSO for VXLAN and GENEVE packets. + + Hardware supports NVGRE TSO, but DPDK currently has no NVGRE offload flags. + +- Tx checksum offloads. + + The NIC fills in IPv4/UDP/TCP checksums for both inner and outer packets. + +- Rx checksum offloads. + + The NIC validates IPv4/UDP/TCP checksums of both inner and outer packets. + Good checksum flags (e.g. ``PKT_RX_L4_CKSUM_GOOD``) indicate that the inner + packet has the correct checksum, and if applicable, the outer packet also + has the correct checksum. Bad checksum flags (e.g. ``PKT_RX_L4_CKSUM_BAD``) + indicate that the inner and/or outer packets have invalid checksum values. + +- Inner Rx packet type classification + + PMD sets inner L3/L4 packet types (e.g. ``RTE_PTYPE_INNER_L4_TCP``), and + ``RTE_PTYPE_TUNNEL_GRENAT`` to indicate that the packet is tunneled. + PMD does not set L3/L4 packet types for outer packets. + +- Inner RSS + + RSS hash calculation, therefore queue selection, is done on inner packets. + +In order to enable overlay offload, the 'Enable VXLAN' box should be checked +via CIMC or UCSM followed by a reboot of the server. When PMD successfully +enables overlay offload, it prints the following message on the console. + +.. code-block:: console + + Overlay offload is enabled + .. _enic_limitations: Limitations @@ -373,6 +420,11 @@ Supported features - MTU update - SR-IOV on UCS managed servers connected to Fabric Interconnects - Flow API +- Overlay offload + + - Rx/Tx checksum offloads for VXLAN, NVGRE, GENEVE + - TSO for VXLAN and GENEVE packets + - Inner RSS Known bugs and unsupported features in this release --------------------------------------------------- diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c index 1f8d222fc..386db6411 100644 --- a/drivers/net/enic/base/vnic_dev.c +++ b/drivers/net/enic/base/vnic_dev.c @@ -1048,3 +1048,36 @@ int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry, return ret; } + +int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev, u8 overlay, u8 config) +{ + u64 a0 = overlay; + u64 a1 = config; + int wait = 1000; + + return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CTRL, &a0, &a1, wait); +} + +int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay, + u16 vxlan_udp_port_number) +{ + u64 a1 = vxlan_udp_port_number; + u64 a0 = overlay; + int wait = 1000; + + return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CFG, &a0, &a1, wait); +} + +int vnic_dev_capable_vxlan(struct vnic_dev *vdev) +{ + u64 a0 = VIC_FEATURE_VXLAN; + u64 a1 = 0; + int wait = 1000; + int ret; + + ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait); + /* 1 if the NIC can do VXLAN for both IPv4 and IPv6 with multiple WQs */ + return ret == 0 && + (a1 & (FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ)) == + (FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ); +} diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h index 7e5736b4d..b3ce26032 100644 --- a/drivers/net/enic/base/vnic_dev.h +++ b/drivers/net/enic/base/vnic_dev.h @@ -178,10 +178,9 @@ int vnic_dev_deinit_done(struct vnic_dev *vdev, int *status); int vnic_dev_set_mac_addr(struct vnic_dev *vdev, u8 *mac_addr); int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry, struct filter_v2 *data, struct filter_action_v2 *action_v2); -#ifdef ENIC_VXLAN -int vnic_dev_overlay_offload_enable_disable(struct vnic_dev *vdev, +int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev, u8 overlay, u8 config); int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay, u16 vxlan_udp_port_number); -#endif +int vnic_dev_capable_vxlan(struct vnic_dev *vdev); #endif /* _VNIC_DEV_H_ */ diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h index a63624559..d09113dac 100644 --- a/drivers/net/enic/base/vnic_devcmd.h +++ b/drivers/net/enic/base/vnic_devcmd.h @@ -1078,6 +1078,18 @@ typedef enum { VIC_FEATURE_MAX, } vic_feature_t; +/* + * These flags are used in args[1] of devcmd CMD_GET_SUPP_FEATURE_VER + * to indicate the host driver about the VxLAN and Multi WQ features + * supported + */ +#define FEATURE_VXLAN_IPV6_INNER (1 << 0) +#define FEATURE_VXLAN_IPV6_OUTER (1 << 1) +#define FEATURE_VXLAN_MULTI_WQ (1 << 2) + +#define FEATURE_VXLAN_IPV6 (FEATURE_VXLAN_IPV6_INNER | \ + FEATURE_VXLAN_IPV6_OUTER) + /* * CMD_CONFIG_GRPINTR subcommands */ diff --git a/drivers/net/enic/base/vnic_wq.h b/drivers/net/enic/base/vnic_wq.h index 7c069c063..0135bffc5 100644 --- a/drivers/net/enic/base/vnic_wq.h +++ b/drivers/net/enic/base/vnic_wq.h @@ -44,6 +44,7 @@ struct vnic_wq_buf { struct vnic_wq { unsigned int index; + uint64_t tx_offload_notsup_mask; struct vnic_dev *vdev; struct vnic_wq_ctrl __iomem *ctrl; /* memory-mapped */ struct vnic_dev_ring ring; diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h index adccc8ac5..9d69d1e9e 100644 --- a/drivers/net/enic/enic.h +++ b/drivers/net/enic/enic.h @@ -101,6 +101,7 @@ struct enic { struct vnic_dev *vdev; unsigned int port_id; + int overlay_offload; struct rte_eth_dev *rte_dev; struct enic_fdir fdir; char bdf_name[ENICPMD_BDF_LENGTH]; @@ -119,6 +120,7 @@ struct enic { u8 adv_filters; u32 flow_filter_mode; u8 filter_tags; + u8 vxlan; unsigned int flags; unsigned int priv_flags; @@ -169,6 +171,10 @@ struct enic { uint64_t rss_hf; /* ETH_RSS flags */ union vnic_rss_key rss_key; union vnic_rss_cpu rss_cpu; + + uint64_t rx_offload_capa; /* DEV_RX_OFFLOAD flags */ + uint64_t tx_offload_capa; /* DEV_TX_OFFLOAD flags */ + uint64_t tx_offload_mask; /* PKT_TX flags accepted */ }; /* Compute ethdev's max packet size from MTU */ diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index 03f0c2547..08307fbf3 100644 --- a/drivers/net/enic/enic_ethdev.c +++ b/drivers/net/enic/enic_ethdev.c @@ -39,19 +39,6 @@ static const struct rte_pci_id pci_id_enic_map[] = { {.vendor_id = 0, /* sentinel */}, }; -#define ENIC_TX_OFFLOAD_CAPA ( \ - DEV_TX_OFFLOAD_VLAN_INSERT | \ - DEV_TX_OFFLOAD_IPV4_CKSUM | \ - DEV_TX_OFFLOAD_UDP_CKSUM | \ - DEV_TX_OFFLOAD_TCP_CKSUM | \ - DEV_TX_OFFLOAD_TCP_TSO) - -#define ENIC_RX_OFFLOAD_CAPA ( \ - DEV_RX_OFFLOAD_VLAN_STRIP | \ - DEV_RX_OFFLOAD_IPV4_CKSUM | \ - DEV_RX_OFFLOAD_UDP_CKSUM | \ - DEV_RX_OFFLOAD_TCP_CKSUM) - RTE_INIT(enicpmd_init_log); static void enicpmd_init_log(void) @@ -485,8 +472,8 @@ static void enicpmd_dev_info_get(struct rte_eth_dev *eth_dev, */ device_info->max_rx_pktlen = enic_mtu_to_max_rx_pktlen(enic->max_mtu); device_info->max_mac_addrs = ENIC_MAX_MAC_ADDR; - device_info->rx_offload_capa = ENIC_RX_OFFLOAD_CAPA; - device_info->tx_offload_capa = ENIC_TX_OFFLOAD_CAPA; + device_info->rx_offload_capa = enic->rx_offload_capa; + device_info->tx_offload_capa = enic->tx_offload_capa; device_info->default_rxconf = (struct rte_eth_rxconf) { .rx_free_thresh = ENIC_DEFAULT_RX_FREE_THRESH }; @@ -718,7 +705,7 @@ static void enicpmd_dev_rxq_info_get(struct rte_eth_dev *dev, * Except VLAN stripping (port setting), all the checksum offloads * are always enabled. */ - conf->offloads = ENIC_RX_OFFLOAD_CAPA; + conf->offloads = enic->rx_offload_capa; if (!enic->ig_vlan_strip_en) conf->offloads &= ~DEV_RX_OFFLOAD_VLAN_STRIP; /* rx_thresh and other fields are not applicable for enic */ @@ -733,7 +720,7 @@ static void enicpmd_dev_txq_info_get(struct rte_eth_dev *dev, ENICPMD_FUNC_TRACE(); qinfo->nb_desc = enic->config.wq_desc_count; memset(&qinfo->conf, 0, sizeof(qinfo->conf)); - qinfo->conf.offloads = ENIC_TX_OFFLOAD_CAPA; /* not configurable */ + qinfo->conf.offloads = enic->tx_offload_capa; /* tx_thresh, and all the other fields are not applicable for enic */ } diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c index e908e20f2..9b254d98e 100644 --- a/drivers/net/enic/enic_main.c +++ b/drivers/net/enic/enic_main.c @@ -245,6 +245,9 @@ void enic_init_vnic_resources(struct enic *enic) enic_cq_wq(enic, index), error_interrupt_enable, error_interrupt_offset); + /* Compute unsupported ol flags for enic_prep_pkts() */ + enic->wq[index].tx_offload_notsup_mask = + PKT_TX_OFFLOAD_MASK ^ enic->tx_offload_mask; cq_idx = enic_cq_wq(enic, index); vnic_cq_init(&enic->cq[cq_idx], @@ -1534,6 +1537,28 @@ static int enic_dev_init(struct enic *enic) /* set up link status checking */ vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */ + enic->overlay_offload = 0; +#ifdef RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD + if (enic->vxlan && + /* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */ + vnic_dev_overlay_offload_ctrl(enic->vdev, + OVERLAY_FEATURE_VXLAN, + OVERLAY_OFFLOAD_ENABLE) == 0) { + enic->tx_offload_capa |= + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | + DEV_TX_OFFLOAD_VXLAN_TNL_TSO; + /* + * Do not add PKT_TX_OUTER_{IPV4,IPV6} as they are not + * 'offload' flags (i.e. not part of PKT_TX_OFFLOAD_MASK). + */ + enic->tx_offload_mask |= + PKT_TX_OUTER_IP_CKSUM | + PKT_TX_TUNNEL_MASK; + enic->overlay_offload = 1; + dev_info(enic, "Overlay offload is enabled\n"); + } +#endif return 0; } diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c index e7ad6767e..47f825e51 100644 --- a/drivers/net/enic/enic_res.c +++ b/drivers/net/enic/enic_res.c @@ -148,6 +148,29 @@ int enic_get_vnic_config(struct enic *enic) if (!ENIC_SETTING(enic, RSS)) enic->flow_type_rss_offloads = 0; + enic->vxlan = ENIC_SETTING(enic, VXLAN) && + vnic_dev_capable_vxlan(enic->vdev); + /* + * Default hardware capabilities. enic_dev_init() may add additional + * flags if it enables overlay offloads. + */ + enic->tx_offload_capa = + DEV_TX_OFFLOAD_VLAN_INSERT | + DEV_TX_OFFLOAD_IPV4_CKSUM | + DEV_TX_OFFLOAD_UDP_CKSUM | + DEV_TX_OFFLOAD_TCP_CKSUM | + DEV_TX_OFFLOAD_TCP_TSO; + enic->rx_offload_capa = + DEV_RX_OFFLOAD_VLAN_STRIP | + DEV_RX_OFFLOAD_IPV4_CKSUM | + DEV_RX_OFFLOAD_UDP_CKSUM | + DEV_RX_OFFLOAD_TCP_CKSUM; + enic->tx_offload_mask = + PKT_TX_VLAN_PKT | + PKT_TX_IP_CKSUM | + PKT_TX_L4_MASK | + PKT_TX_TCP_SEG; + return 0; } diff --git a/drivers/net/enic/enic_rxtx.c b/drivers/net/enic/enic_rxtx.c index 2fe5a3fa3..aa3393700 100644 --- a/drivers/net/enic/enic_rxtx.c +++ b/drivers/net/enic/enic_rxtx.c @@ -15,15 +15,6 @@ #include <rte_ip.h> #include <rte_tcp.h> -#define ENIC_TX_OFFLOAD_MASK ( \ - PKT_TX_VLAN_PKT | \ - PKT_TX_IP_CKSUM | \ - PKT_TX_L4_MASK | \ - PKT_TX_TCP_SEG) - -#define ENIC_TX_OFFLOAD_NOTSUP_MASK \ - (PKT_TX_OFFLOAD_MASK ^ ENIC_TX_OFFLOAD_MASK) - #define RTE_PMD_USE_PREFETCH #ifdef RTE_PMD_USE_PREFETCH @@ -130,30 +121,73 @@ enic_cq_rx_check_err(struct cq_desc *cqd) /* Lookup table to translate RX CQ flags to mbuf flags. */ static inline uint32_t -enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd) +enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd, uint8_t tnl) { struct cq_enet_rq_desc *cqrd = (struct cq_enet_rq_desc *)cqd; uint8_t cqrd_flags = cqrd->flags; + /* + * Odd-numbered entries are for tunnel packets. All packet type info + * applies to the inner packet, and there is no info on the outer + * packet. The outer flags in these entries exist only to avoid + * changing enic_cq_rx_to_pkt_flags(). They are cleared from mbuf + * afterwards. + * + * Also, as there is no tunnel type info (VXLAN, NVGRE, or GENEVE), set + * RTE_PTYPE_TUNNEL_GRENAT.. + */ static const uint32_t cq_type_table[128] __rte_cache_aligned = { [0x00] = RTE_PTYPE_UNKNOWN, [0x20] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG, + [0x21] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_NONFRAG, [0x22] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, + [0x23] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_UDP, [0x24] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, + [0x25] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_TCP, [0x60] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG, + [0x61] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_FRAG, [0x62] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, + [0x63] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_UDP, [0x64] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, + [0x65] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_TCP, [0x10] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG, + [0x11] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_NONFRAG, [0x12] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, + [0x13] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_UDP, [0x14] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, - [0x50] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG, - [0x52] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, - [0x54] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, + [0x15] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_TCP, /* All others reserved */ }; cqrd_flags &= CQ_ENET_RQ_DESC_FLAGS_IPV4_FRAGMENT | CQ_ENET_RQ_DESC_FLAGS_IPV4 | CQ_ENET_RQ_DESC_FLAGS_IPV6 | CQ_ENET_RQ_DESC_FLAGS_TCP | CQ_ENET_RQ_DESC_FLAGS_UDP; - return cq_type_table[cqrd_flags]; + return cq_type_table[cqrd_flags + tnl]; } static inline void @@ -200,10 +234,18 @@ enic_cq_rx_to_pkt_flags(struct cq_desc *cqd, struct rte_mbuf *mbuf) uint32_t l4_flags; l4_flags = mbuf->packet_type & RTE_PTYPE_L4_MASK; - if (enic_cq_rx_desc_ipv4_csum_ok(cqrd)) - pkt_flags |= PKT_RX_IP_CKSUM_GOOD; - else if (mbuf->packet_type & RTE_PTYPE_L3_IPV4) - pkt_flags |= PKT_RX_IP_CKSUM_BAD; + /* + * When overlay offload is enabled, the NIC may + * set ipv4_csum_ok=1 if the inner packet is IPv6.. + * So, explicitly check for IPv4 before checking + * ipv4_csum_ok. + */ + if (mbuf->packet_type & RTE_PTYPE_L3_IPV4) { + if (enic_cq_rx_desc_ipv4_csum_ok(cqrd)) + pkt_flags |= PKT_RX_IP_CKSUM_GOOD; + else + pkt_flags |= PKT_RX_IP_CKSUM_BAD; + } if (l4_flags == RTE_PTYPE_L4_UDP || l4_flags == RTE_PTYPE_L4_TCP) { @@ -245,6 +287,7 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, struct vnic_cq *cq; volatile struct cq_desc *cqd_ptr; uint8_t color; + uint8_t tnl; uint16_t seg_length; struct rte_mbuf *first_seg = sop_rq->pkt_first_seg; struct rte_mbuf *last_seg = sop_rq->pkt_last_seg; @@ -336,10 +379,21 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, continue; } + /* + * When overlay offload is enabled, CQ.fcoe indicates the + * packet is tunnelled. + */ + tnl = enic->overlay_offload && + (ciflags & CQ_ENET_RQ_DESC_FLAGS_FCOE) != 0; /* cq rx flags are only valid if eop bit is set */ - first_seg->packet_type = enic_cq_rx_flags_to_pkt_type(&cqd); + first_seg->packet_type = + enic_cq_rx_flags_to_pkt_type(&cqd, tnl); enic_cq_rx_to_pkt_flags(&cqd, first_seg); - + /* Wipe the outer types set by enic_cq_rx_flags_to_pkt_type() */ + if (tnl) { + first_seg->packet_type &= ~(RTE_PTYPE_L3_MASK | + RTE_PTYPE_L4_MASK); + } if (unlikely(packet_error)) { rte_pktmbuf_free(first_seg); rte_atomic64_inc(&enic->soft_stats.rx_packet_errors); @@ -443,9 +497,10 @@ unsigned int enic_cleanup_wq(__rte_unused struct enic *enic, struct vnic_wq *wq) return 0; } -uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, +uint16_t enic_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) { + struct vnic_wq *wq = (struct vnic_wq *)tx_queue; int32_t ret; uint16_t i; uint64_t ol_flags; @@ -454,7 +509,7 @@ uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, for (i = 0; i != nb_pkts; i++) { m = tx_pkts[i]; ol_flags = m->ol_flags; - if (ol_flags & ENIC_TX_OFFLOAD_NOTSUP_MASK) { + if (ol_flags & wq->tx_offload_notsup_mask) { rte_errno = -ENOTSUP; return i; } @@ -558,6 +613,11 @@ uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, offload_mode = WQ_ENET_OFFLOAD_MODE_TSO; mss = tx_pkt->tso_segsz; + /* For tunnel, need the size of outer+inner headers */ + if (ol_flags & PKT_TX_TUNNEL_MASK) { + header_len += tx_pkt->outer_l2_len + + tx_pkt->outer_l3_len; + } } if ((ol_flags & ol_flags_mask) && (header_len == 0)) { -- 2.16.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE John Daley @ 2018-04-06 16:15 ` Ferruh Yigit 2018-04-07 2:40 ` Hyong Youb Kim 2018-04-16 21:49 ` [dpdk-dev] [PATCH v2] " John Daley 1 sibling, 1 reply; 18+ messages in thread From: Ferruh Yigit @ 2018-04-06 16:15 UTC (permalink / raw) To: John Daley; +Cc: dev, Hyong Youb Kim, Thomas Monjalon On 4/5/2018 12:54 AM, John Daley wrote: > From: Hyong Youb Kim <hyonkim@cisco.com> > > Recent NIC models support overlay offload. The overlay offload > feature enables the following on the NIC. > - Rx/Tx checksum offloads for both inner and outer packets. > - Rx inner packet type classification. > - TSO. > - Inner RSS. > > TX descriptors do not require any changes, except the header length > for TSO. The NIC parses outer/inner packets and performs offloads on > them as necessary. The header length for tunneled TSO includes both > inner and outer headers. > > The NIC actually parses and performs the above for NVGRE as well. DPDK > currently has no offload flags for NVGRE, and the hardware has no > controls to individually enable tunnel types either. So do nothing for > now. > > Add a config flag to enable overlay offload by default. To disable it, > the user should set it to 'n'. > > CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y > > Also update the enic guide doc. > > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> > Reviewed-by: John Daley <johndale@cisco.com> > --- > config/common_base | 1 + > doc/guides/nics/enic.rst | 52 ++++++++++++++++++ > drivers/net/enic/base/vnic_dev.c | 33 ++++++++++++ > drivers/net/enic/base/vnic_dev.h | 5 +- > drivers/net/enic/base/vnic_devcmd.h | 12 +++++ > drivers/net/enic/base/vnic_wq.h | 1 + > drivers/net/enic/enic.h | 6 +++ > drivers/net/enic/enic_ethdev.c | 21 ++------ > drivers/net/enic/enic_main.c | 25 +++++++++ > drivers/net/enic/enic_res.c | 23 ++++++++ > drivers/net/enic/enic_rxtx.c | 104 ++++++++++++++++++++++++++++-------- > 11 files changed, 241 insertions(+), 42 deletions(-) > > diff --git a/config/common_base b/config/common_base > index c09c7cf88..964e37b6e 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n > # Compile burst-oriented Cisco ENIC PMD driver > # > CONFIG_RTE_LIBRTE_ENIC_PMD=y > +CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y Hi John, We are trying to reduce config options we have, is overlay offload enabling can be done via runtime argument or dedicated offload flag? ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE 2018-04-06 16:15 ` Ferruh Yigit @ 2018-04-07 2:40 ` Hyong Youb Kim 2018-04-09 12:52 ` Ferruh Yigit 0 siblings, 1 reply; 18+ messages in thread From: Hyong Youb Kim @ 2018-04-07 2:40 UTC (permalink / raw) To: Ferruh Yigit; +Cc: John Daley, dev, Thomas Monjalon On Fri, Apr 06, 2018 at 05:15:40PM +0100, Ferruh Yigit wrote: > On 4/5/2018 12:54 AM, John Daley wrote: > > From: Hyong Youb Kim <hyonkim@cisco.com> > > > > Recent NIC models support overlay offload. The overlay offload > > feature enables the following on the NIC. > > - Rx/Tx checksum offloads for both inner and outer packets. > > - Rx inner packet type classification. > > - TSO. > > - Inner RSS. > > > > TX descriptors do not require any changes, except the header length > > for TSO. The NIC parses outer/inner packets and performs offloads on > > them as necessary. The header length for tunneled TSO includes both > > inner and outer headers. > > > > The NIC actually parses and performs the above for NVGRE as well. DPDK > > currently has no offload flags for NVGRE, and the hardware has no > > controls to individually enable tunnel types either. So do nothing for > > now. > > > > Add a config flag to enable overlay offload by default. To disable it, > > the user should set it to 'n'. > > > > CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y > > > > Also update the enic guide doc. > > > > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> > > Reviewed-by: John Daley <johndale@cisco.com> > > --- > > config/common_base | 1 + > > doc/guides/nics/enic.rst | 52 ++++++++++++++++++ > > drivers/net/enic/base/vnic_dev.c | 33 ++++++++++++ > > drivers/net/enic/base/vnic_dev.h | 5 +- > > drivers/net/enic/base/vnic_devcmd.h | 12 +++++ > > drivers/net/enic/base/vnic_wq.h | 1 + > > drivers/net/enic/enic.h | 6 +++ > > drivers/net/enic/enic_ethdev.c | 21 ++------ > > drivers/net/enic/enic_main.c | 25 +++++++++ > > drivers/net/enic/enic_res.c | 23 ++++++++ > > drivers/net/enic/enic_rxtx.c | 104 ++++++++++++++++++++++++++++-------- > > 11 files changed, 241 insertions(+), 42 deletions(-) > > > > diff --git a/config/common_base b/config/common_base > > index c09c7cf88..964e37b6e 100644 > > --- a/config/common_base > > +++ b/config/common_base > > @@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n > > # Compile burst-oriented Cisco ENIC PMD driver > > # > > CONFIG_RTE_LIBRTE_ENIC_PMD=y > > +CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y > > Hi John, > > We are trying to reduce config options we have, is overlay offload enabling can > be done via runtime argument or dedicated offload flag? Hi, Would you accept a devarg? We would enable overlay offload by default, and disable it when the app specifies that devarg. This overlay offload is a bunch of things under one roof, and it does not play nicely with DPDK offload flags. Thanks. -Hyong ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE 2018-04-07 2:40 ` Hyong Youb Kim @ 2018-04-09 12:52 ` Ferruh Yigit 0 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-04-09 12:52 UTC (permalink / raw) To: Hyong Youb Kim; +Cc: John Daley, dev, Thomas Monjalon On 4/7/2018 3:40 AM, Hyong Youb Kim wrote: > On Fri, Apr 06, 2018 at 05:15:40PM +0100, Ferruh Yigit wrote: >> On 4/5/2018 12:54 AM, John Daley wrote: >>> From: Hyong Youb Kim <hyonkim@cisco.com> >>> >>> Recent NIC models support overlay offload. The overlay offload >>> feature enables the following on the NIC. >>> - Rx/Tx checksum offloads for both inner and outer packets. >>> - Rx inner packet type classification. >>> - TSO. >>> - Inner RSS. >>> >>> TX descriptors do not require any changes, except the header length >>> for TSO. The NIC parses outer/inner packets and performs offloads on >>> them as necessary. The header length for tunneled TSO includes both >>> inner and outer headers. >>> >>> The NIC actually parses and performs the above for NVGRE as well. DPDK >>> currently has no offload flags for NVGRE, and the hardware has no >>> controls to individually enable tunnel types either. So do nothing for >>> now. >>> >>> Add a config flag to enable overlay offload by default. To disable it, >>> the user should set it to 'n'. >>> >>> CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y >>> >>> Also update the enic guide doc. >>> >>> Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> >>> Reviewed-by: John Daley <johndale@cisco.com> >>> --- >>> config/common_base | 1 + >>> doc/guides/nics/enic.rst | 52 ++++++++++++++++++ >>> drivers/net/enic/base/vnic_dev.c | 33 ++++++++++++ >>> drivers/net/enic/base/vnic_dev.h | 5 +- >>> drivers/net/enic/base/vnic_devcmd.h | 12 +++++ >>> drivers/net/enic/base/vnic_wq.h | 1 + >>> drivers/net/enic/enic.h | 6 +++ >>> drivers/net/enic/enic_ethdev.c | 21 ++------ >>> drivers/net/enic/enic_main.c | 25 +++++++++ >>> drivers/net/enic/enic_res.c | 23 ++++++++ >>> drivers/net/enic/enic_rxtx.c | 104 ++++++++++++++++++++++++++++-------- >>> 11 files changed, 241 insertions(+), 42 deletions(-) >>> >>> diff --git a/config/common_base b/config/common_base >>> index c09c7cf88..964e37b6e 100644 >>> --- a/config/common_base >>> +++ b/config/common_base >>> @@ -205,6 +205,7 @@ CONFIG_RTE_LIBRTE_ENA_COM_DEBUG=n >>> # Compile burst-oriented Cisco ENIC PMD driver >>> # >>> CONFIG_RTE_LIBRTE_ENIC_PMD=y >>> +CONFIG_RTE_LIBRTE_ENIC_ENABLE_OVERLAY_OFFLOAD=y >> >> Hi John, >> >> We are trying to reduce config options we have, is overlay offload enabling can >> be done via runtime argument or dedicated offload flag? > > Hi, > > Would you accept a devarg? We would enable overlay offload by default, > and disable it when the app specifies that devarg. This overlay > offload is a bunch of things under one roof, and it does not play > nicely with DPDK offload flags. No problem on implementing this as devargs. > > Thanks. > -Hyong > ^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH v2] net/enic: enable overlay offload for VXLAN and GENEVE 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE John Daley 2018-04-06 16:15 ` Ferruh Yigit @ 2018-04-16 21:49 ` John Daley 2018-04-17 18:05 ` Ferruh Yigit 1 sibling, 1 reply; 18+ messages in thread From: John Daley @ 2018-04-16 21:49 UTC (permalink / raw) To: ferruh.yigit; +Cc: dev, Hyong Youb Kim From: Hyong Youb Kim <hyonkim@cisco.com> Recent NIC models support overlay offload. The overlay offload feature enables the following on the NIC. - Rx/Tx checksum offloads for both inner and outer packets. - Rx inner packet type classification. - TSO. - Inner RSS. TX descriptors do not require any changes, except the header length for TSO. The NIC parses outer/inner packets and performs offloads on them as necessary. The header length for tunneled TSO includes both inner and outer headers. The NIC actually parses and performs the above for NVGRE as well. DPDK currently has no offload flags for NVGRE, and the hardware has no controls to individually enable tunnel types either. So do nothing for now. The driver enables overlay offload by default. Add a devargs 'disable-overlay=<0|1>' to allow the app to disable it. Also update the enic guide doc. Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> Reviewed-by: John Daley <johndale@cisco.com> --- v2 - use devargs insetad of config option doc/guides/nics/enic.rst | 53 ++++++++++++++++++ drivers/net/enic/base/vnic_dev.c | 33 ++++++++++++ drivers/net/enic/base/vnic_dev.h | 5 +- drivers/net/enic/base/vnic_devcmd.h | 12 +++++ drivers/net/enic/base/vnic_wq.h | 1 + drivers/net/enic/enic.h | 8 +++ drivers/net/enic/enic_ethdev.c | 71 ++++++++++++++++++------ drivers/net/enic/enic_main.c | 24 +++++++++ drivers/net/enic/enic_res.c | 23 ++++++++ drivers/net/enic/enic_rxtx.c | 104 ++++++++++++++++++++++++++++-------- 10 files changed, 293 insertions(+), 41 deletions(-) diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index 3ef635fc8..49abc7e95 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -254,6 +254,54 @@ Generic Flow API is supported. The baseline support is: More features may be added in future firmware and new versions of the VIC. Please refer to the release notes. +.. _overlay_offload: + +Overlay Offload +--------------- + +Recent hardware models support overlay offload. When enabled, the NIC performs +the following operations for VXLAN, NVGRE, and GENEVE packets. In all cases, +inner and outer packets can be IPv4 or IPv6. + +- TSO for VXLAN and GENEVE packets. + + Hardware supports NVGRE TSO, but DPDK currently has no NVGRE offload flags. + +- Tx checksum offloads. + + The NIC fills in IPv4/UDP/TCP checksums for both inner and outer packets. + +- Rx checksum offloads. + + The NIC validates IPv4/UDP/TCP checksums of both inner and outer packets. + Good checksum flags (e.g. ``PKT_RX_L4_CKSUM_GOOD``) indicate that the inner + packet has the correct checksum, and if applicable, the outer packet also + has the correct checksum. Bad checksum flags (e.g. ``PKT_RX_L4_CKSUM_BAD``) + indicate that the inner and/or outer packets have invalid checksum values. + +- Inner Rx packet type classification + + PMD sets inner L3/L4 packet types (e.g. ``RTE_PTYPE_INNER_L4_TCP``), and + ``RTE_PTYPE_TUNNEL_GRENAT`` to indicate that the packet is tunneled. + PMD does not set L3/L4 packet types for outer packets. + +- Inner RSS + + RSS hash calculation, therefore queue selection, is done on inner packets. + +In order to enable overlay offload, the 'Enable VXLAN' box should be checked +via CIMC or UCSM followed by a reboot of the server. When PMD successfully +enables overlay offload, it prints the following message on the console. + +.. code-block:: console + + Overlay offload is enabled + +By default, PMD enables overlay offload if hardware supports it. To disable +it, set ``devargs`` parameter ``disable-overlay=1``. For example:: + + -w 12:00.0,disable-overlay=1 + .. _enic_limitations: Limitations @@ -376,6 +424,11 @@ Supported features - MTU update - SR-IOV on UCS managed servers connected to Fabric Interconnects - Flow API +- Overlay offload + + - Rx/Tx checksum offloads for VXLAN, NVGRE, GENEVE + - TSO for VXLAN and GENEVE packets + - Inner RSS Known bugs and unsupported features in this release --------------------------------------------------- diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c index 364458a88..8880ab981 100644 --- a/drivers/net/enic/base/vnic_dev.c +++ b/drivers/net/enic/base/vnic_dev.c @@ -1062,3 +1062,36 @@ int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry, return ret; } + +int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev, u8 overlay, u8 config) +{ + u64 a0 = overlay; + u64 a1 = config; + int wait = 1000; + + return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CTRL, &a0, &a1, wait); +} + +int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay, + u16 vxlan_udp_port_number) +{ + u64 a1 = vxlan_udp_port_number; + u64 a0 = overlay; + int wait = 1000; + + return vnic_dev_cmd(vdev, CMD_OVERLAY_OFFLOAD_CFG, &a0, &a1, wait); +} + +int vnic_dev_capable_vxlan(struct vnic_dev *vdev) +{ + u64 a0 = VIC_FEATURE_VXLAN; + u64 a1 = 0; + int wait = 1000; + int ret; + + ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait); + /* 1 if the NIC can do VXLAN for both IPv4 and IPv6 with multiple WQs */ + return ret == 0 && + (a1 & (FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ)) == + (FEATURE_VXLAN_IPV6 | FEATURE_VXLAN_MULTI_WQ); +} diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h index 90833c794..e7a1f8bd8 100644 --- a/drivers/net/enic/base/vnic_dev.h +++ b/drivers/net/enic/base/vnic_dev.h @@ -179,10 +179,9 @@ int vnic_dev_deinit_done(struct vnic_dev *vdev, int *status); int vnic_dev_set_mac_addr(struct vnic_dev *vdev, u8 *mac_addr); int vnic_dev_classifier(struct vnic_dev *vdev, u8 cmd, u16 *entry, struct filter_v2 *data, struct filter_action_v2 *action_v2); -#ifdef ENIC_VXLAN -int vnic_dev_overlay_offload_enable_disable(struct vnic_dev *vdev, +int vnic_dev_overlay_offload_ctrl(struct vnic_dev *vdev, u8 overlay, u8 config); int vnic_dev_overlay_offload_cfg(struct vnic_dev *vdev, u8 overlay, u16 vxlan_udp_port_number); -#endif +int vnic_dev_capable_vxlan(struct vnic_dev *vdev); #endif /* _VNIC_DEV_H_ */ diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h index 2294b55cf..2865eb4d4 100644 --- a/drivers/net/enic/base/vnic_devcmd.h +++ b/drivers/net/enic/base/vnic_devcmd.h @@ -1080,6 +1080,18 @@ typedef enum { VIC_FEATURE_MAX, } vic_feature_t; +/* + * These flags are used in args[1] of devcmd CMD_GET_SUPP_FEATURE_VER + * to indicate the host driver about the VxLAN and Multi WQ features + * supported + */ +#define FEATURE_VXLAN_IPV6_INNER (1 << 0) +#define FEATURE_VXLAN_IPV6_OUTER (1 << 1) +#define FEATURE_VXLAN_MULTI_WQ (1 << 2) + +#define FEATURE_VXLAN_IPV6 (FEATURE_VXLAN_IPV6_INNER | \ + FEATURE_VXLAN_IPV6_OUTER) + /* * CMD_CONFIG_GRPINTR subcommands */ diff --git a/drivers/net/enic/base/vnic_wq.h b/drivers/net/enic/base/vnic_wq.h index 7c069c063..0135bffc5 100644 --- a/drivers/net/enic/base/vnic_wq.h +++ b/drivers/net/enic/base/vnic_wq.h @@ -44,6 +44,7 @@ struct vnic_wq_buf { struct vnic_wq { unsigned int index; + uint64_t tx_offload_notsup_mask; struct vnic_dev *vdev; struct vnic_wq_ctrl __iomem *ctrl; /* memory-mapped */ struct vnic_dev_ring ring; diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h index 751ddc744..32a64a51a 100644 --- a/drivers/net/enic/enic.h +++ b/drivers/net/enic/enic.h @@ -17,6 +17,7 @@ #include "vnic_rss.h" #include "enic_res.h" #include "cq_enet_desc.h" +#include <stdbool.h> #include <sys/queue.h> #include <rte_spinlock.h> @@ -101,6 +102,7 @@ struct enic { struct vnic_dev *vdev; unsigned int port_id; + bool overlay_offload; struct rte_eth_dev *rte_dev; struct enic_fdir fdir; char bdf_name[ENICPMD_BDF_LENGTH]; @@ -119,6 +121,8 @@ struct enic { u8 adv_filters; u32 flow_filter_mode; u8 filter_actions; /* HW supported actions */ + bool vxlan; + bool disable_overlay; /* devargs disable_overlay=1 */ unsigned int flags; unsigned int priv_flags; @@ -169,6 +173,10 @@ struct enic { uint64_t rss_hf; /* ETH_RSS flags */ union vnic_rss_key rss_key; union vnic_rss_cpu rss_cpu; + + uint64_t rx_offload_capa; /* DEV_RX_OFFLOAD flags */ + uint64_t tx_offload_capa; /* DEV_TX_OFFLOAD flags */ + uint64_t tx_offload_mask; /* PKT_TX flags accepted */ }; /* Compute ethdev's max packet size from MTU */ diff --git a/drivers/net/enic/enic_ethdev.c b/drivers/net/enic/enic_ethdev.c index 801f4704c..b37384c88 100644 --- a/drivers/net/enic/enic_ethdev.c +++ b/drivers/net/enic/enic_ethdev.c @@ -11,6 +11,7 @@ #include <rte_bus_pci.h> #include <rte_ethdev_driver.h> #include <rte_ethdev_pci.h> +#include <rte_kvargs.h> #include <rte_string_fns.h> #include "vnic_intr.h" @@ -39,18 +40,7 @@ static const struct rte_pci_id pci_id_enic_map[] = { {.vendor_id = 0, /* sentinel */}, }; -#define ENIC_TX_OFFLOAD_CAPA ( \ - DEV_TX_OFFLOAD_VLAN_INSERT | \ - DEV_TX_OFFLOAD_IPV4_CKSUM | \ - DEV_TX_OFFLOAD_UDP_CKSUM | \ - DEV_TX_OFFLOAD_TCP_CKSUM | \ - DEV_TX_OFFLOAD_TCP_TSO) - -#define ENIC_RX_OFFLOAD_CAPA ( \ - DEV_RX_OFFLOAD_VLAN_STRIP | \ - DEV_RX_OFFLOAD_IPV4_CKSUM | \ - DEV_RX_OFFLOAD_UDP_CKSUM | \ - DEV_RX_OFFLOAD_TCP_CKSUM) +#define ENIC_DEVARG_DISABLE_OVERLAY "disable-overlay" RTE_INIT(enicpmd_init_log); static void @@ -484,8 +474,8 @@ static void enicpmd_dev_info_get(struct rte_eth_dev *eth_dev, */ device_info->max_rx_pktlen = enic_mtu_to_max_rx_pktlen(enic->max_mtu); device_info->max_mac_addrs = ENIC_MAX_MAC_ADDR; - device_info->rx_offload_capa = ENIC_RX_OFFLOAD_CAPA; - device_info->tx_offload_capa = ENIC_TX_OFFLOAD_CAPA; + device_info->rx_offload_capa = enic->rx_offload_capa; + device_info->tx_offload_capa = enic->tx_offload_capa; device_info->default_rxconf = (struct rte_eth_rxconf) { .rx_free_thresh = ENIC_DEFAULT_RX_FREE_THRESH }; @@ -717,7 +707,7 @@ static void enicpmd_dev_rxq_info_get(struct rte_eth_dev *dev, * Except VLAN stripping (port setting), all the checksum offloads * are always enabled. */ - conf->offloads = ENIC_RX_OFFLOAD_CAPA; + conf->offloads = enic->rx_offload_capa; if (!enic->ig_vlan_strip_en) conf->offloads &= ~DEV_RX_OFFLOAD_VLAN_STRIP; /* rx_thresh and other fields are not applicable for enic */ @@ -732,7 +722,7 @@ static void enicpmd_dev_txq_info_get(struct rte_eth_dev *dev, ENICPMD_FUNC_TRACE(); qinfo->nb_desc = enic->config.wq_desc_count; memset(&qinfo->conf, 0, sizeof(qinfo->conf)); - qinfo->conf.offloads = ENIC_TX_OFFLOAD_CAPA; /* not configurable */ + qinfo->conf.offloads = enic->tx_offload_capa; /* tx_thresh, and all the other fields are not applicable for enic */ } @@ -806,6 +796,49 @@ static const struct eth_dev_ops enicpmd_eth_dev_ops = { .rss_hash_update = enicpmd_dev_rss_hash_update, }; +static int enic_parse_disable_overlay(__rte_unused const char *key, + const char *value, + void *opaque) +{ + struct enic *enic; + + enic = (struct enic *)opaque; + if (strcmp(value, "0") == 0) { + enic->disable_overlay = false; + } else if (strcmp(value, "1") == 0) { + enic->disable_overlay = true; + } else { + dev_err(enic, "Invalid value for " ENIC_DEVARG_DISABLE_OVERLAY + ": expected=0|1 given=%s\n", value); + return -EINVAL; + } + return 0; +} + +static int enic_check_devargs(struct rte_eth_dev *dev) +{ + static const char *const valid_keys[] = { + ENIC_DEVARG_DISABLE_OVERLAY, NULL}; + struct enic *enic = pmd_priv(dev); + struct rte_kvargs *kvlist; + + ENICPMD_FUNC_TRACE(); + + enic->disable_overlay = false; + if (!dev->device->devargs) + return 0; + kvlist = rte_kvargs_parse(dev->device->devargs->args, valid_keys); + if (!kvlist) + return -EINVAL; + if (rte_kvargs_process(kvlist, ENIC_DEVARG_DISABLE_OVERLAY, + enic_parse_disable_overlay, enic) < 0) { + rte_kvargs_free(kvlist); + return -EINVAL; + } + rte_kvargs_free(kvlist); + return 0; +} + struct enic *enicpmd_list_head = NULL; /* Initialize the driver * It returns 0 on success. @@ -815,6 +848,7 @@ static int eth_enicpmd_dev_init(struct rte_eth_dev *eth_dev) struct rte_pci_device *pdev; struct rte_pci_addr *addr; struct enic *enic = pmd_priv(eth_dev); + int err; ENICPMD_FUNC_TRACE(); @@ -833,6 +867,9 @@ static int eth_enicpmd_dev_init(struct rte_eth_dev *eth_dev) snprintf(enic->bdf_name, ENICPMD_BDF_LENGTH, "%04x:%02x:%02x.%x", addr->domain, addr->bus, addr->devid, addr->function); + err = enic_check_devargs(eth_dev); + if (err) + return err; return enic_probe(enic); } @@ -858,3 +895,5 @@ static struct rte_pci_driver rte_enic_pmd = { RTE_PMD_REGISTER_PCI(net_enic, rte_enic_pmd); RTE_PMD_REGISTER_PCI_TABLE(net_enic, pci_id_enic_map); RTE_PMD_REGISTER_KMOD_DEP(net_enic, "* igb_uio | uio_pci_generic | vfio-pci"); +RTE_PMD_REGISTER_PARAM_STRING(net_enic, + ENIC_DEVARG_DISABLE_OVERLAY "=<0|1> "); diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c index 98d47752c..923150bb4 100644 --- a/drivers/net/enic/enic_main.c +++ b/drivers/net/enic/enic_main.c @@ -245,6 +245,9 @@ void enic_init_vnic_resources(struct enic *enic) enic_cq_wq(enic, index), error_interrupt_enable, error_interrupt_offset); + /* Compute unsupported ol flags for enic_prep_pkts() */ + enic->wq[index].tx_offload_notsup_mask = + PKT_TX_OFFLOAD_MASK ^ enic->tx_offload_mask; cq_idx = enic_cq_wq(enic, index); vnic_cq_init(&enic->cq[cq_idx], @@ -1547,6 +1550,27 @@ static int enic_dev_init(struct enic *enic) /* set up link status checking */ vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */ + enic->overlay_offload = false; + if (!enic->disable_overlay && enic->vxlan && + /* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */ + vnic_dev_overlay_offload_ctrl(enic->vdev, + OVERLAY_FEATURE_VXLAN, + OVERLAY_OFFLOAD_ENABLE) == 0) { + enic->tx_offload_capa |= + DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM | + DEV_TX_OFFLOAD_GENEVE_TNL_TSO | + DEV_TX_OFFLOAD_VXLAN_TNL_TSO; + /* + * Do not add PKT_TX_OUTER_{IPV4,IPV6} as they are not + * 'offload' flags (i.e. not part of PKT_TX_OFFLOAD_MASK). + */ + enic->tx_offload_mask |= + PKT_TX_OUTER_IP_CKSUM | + PKT_TX_TUNNEL_MASK; + enic->overlay_offload = true; + dev_info(enic, "Overlay offload is enabled\n"); + } + return 0; } diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c index 9b5baf3ba..de17a31d0 100644 --- a/drivers/net/enic/enic_res.c +++ b/drivers/net/enic/enic_res.c @@ -158,6 +158,29 @@ int enic_get_vnic_config(struct enic *enic) if (!ENIC_SETTING(enic, RSS)) enic->flow_type_rss_offloads = 0; + enic->vxlan = ENIC_SETTING(enic, VXLAN) && + vnic_dev_capable_vxlan(enic->vdev); + /* + * Default hardware capabilities. enic_dev_init() may add additional + * flags if it enables overlay offloads. + */ + enic->tx_offload_capa = + DEV_TX_OFFLOAD_VLAN_INSERT | + DEV_TX_OFFLOAD_IPV4_CKSUM | + DEV_TX_OFFLOAD_UDP_CKSUM | + DEV_TX_OFFLOAD_TCP_CKSUM | + DEV_TX_OFFLOAD_TCP_TSO; + enic->rx_offload_capa = + DEV_RX_OFFLOAD_VLAN_STRIP | + DEV_RX_OFFLOAD_IPV4_CKSUM | + DEV_RX_OFFLOAD_UDP_CKSUM | + DEV_RX_OFFLOAD_TCP_CKSUM; + enic->tx_offload_mask = + PKT_TX_VLAN_PKT | + PKT_TX_IP_CKSUM | + PKT_TX_L4_MASK | + PKT_TX_TCP_SEG; + return 0; } diff --git a/drivers/net/enic/enic_rxtx.c b/drivers/net/enic/enic_rxtx.c index 2fe5a3fa3..aa3393700 100644 --- a/drivers/net/enic/enic_rxtx.c +++ b/drivers/net/enic/enic_rxtx.c @@ -15,15 +15,6 @@ #include <rte_ip.h> #include <rte_tcp.h> -#define ENIC_TX_OFFLOAD_MASK ( \ - PKT_TX_VLAN_PKT | \ - PKT_TX_IP_CKSUM | \ - PKT_TX_L4_MASK | \ - PKT_TX_TCP_SEG) - -#define ENIC_TX_OFFLOAD_NOTSUP_MASK \ - (PKT_TX_OFFLOAD_MASK ^ ENIC_TX_OFFLOAD_MASK) - #define RTE_PMD_USE_PREFETCH #ifdef RTE_PMD_USE_PREFETCH @@ -130,30 +121,73 @@ enic_cq_rx_check_err(struct cq_desc *cqd) /* Lookup table to translate RX CQ flags to mbuf flags. */ static inline uint32_t -enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd) +enic_cq_rx_flags_to_pkt_type(struct cq_desc *cqd, uint8_t tnl) { struct cq_enet_rq_desc *cqrd = (struct cq_enet_rq_desc *)cqd; uint8_t cqrd_flags = cqrd->flags; + /* + * Odd-numbered entries are for tunnel packets. All packet type info + * applies to the inner packet, and there is no info on the outer + * packet. The outer flags in these entries exist only to avoid + * changing enic_cq_rx_to_pkt_flags(). They are cleared from mbuf + * afterwards. + * + * Also, as there is no tunnel type info (VXLAN, NVGRE, or GENEVE), set + * RTE_PTYPE_TUNNEL_GRENAT.. + */ static const uint32_t cq_type_table[128] __rte_cache_aligned = { [0x00] = RTE_PTYPE_UNKNOWN, [0x20] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG, + [0x21] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_NONFRAG, [0x22] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, + [0x23] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_UDP, [0x24] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, + [0x25] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_TCP, [0x60] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG, + [0x61] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_FRAG, [0x62] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, + [0x63] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_UDP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_UDP, [0x64] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, + [0x65] = RTE_PTYPE_L3_IPV4_EXT_UNKNOWN | RTE_PTYPE_L4_TCP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV4_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_TCP, [0x10] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG, + [0x11] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_NONFRAG | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_NONFRAG, [0x12] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, + [0x13] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_UDP, [0x14] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, - [0x50] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_FRAG, - [0x52] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_UDP, - [0x54] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP, + [0x15] = RTE_PTYPE_L3_IPV6_EXT_UNKNOWN | RTE_PTYPE_L4_TCP | + RTE_PTYPE_TUNNEL_GRENAT | + RTE_PTYPE_INNER_L3_IPV6_EXT_UNKNOWN | + RTE_PTYPE_INNER_L4_TCP, /* All others reserved */ }; cqrd_flags &= CQ_ENET_RQ_DESC_FLAGS_IPV4_FRAGMENT | CQ_ENET_RQ_DESC_FLAGS_IPV4 | CQ_ENET_RQ_DESC_FLAGS_IPV6 | CQ_ENET_RQ_DESC_FLAGS_TCP | CQ_ENET_RQ_DESC_FLAGS_UDP; - return cq_type_table[cqrd_flags]; + return cq_type_table[cqrd_flags + tnl]; } static inline void @@ -200,10 +234,18 @@ enic_cq_rx_to_pkt_flags(struct cq_desc *cqd, struct rte_mbuf *mbuf) uint32_t l4_flags; l4_flags = mbuf->packet_type & RTE_PTYPE_L4_MASK; - if (enic_cq_rx_desc_ipv4_csum_ok(cqrd)) - pkt_flags |= PKT_RX_IP_CKSUM_GOOD; - else if (mbuf->packet_type & RTE_PTYPE_L3_IPV4) - pkt_flags |= PKT_RX_IP_CKSUM_BAD; + /* + * When overlay offload is enabled, the NIC may + * set ipv4_csum_ok=1 if the inner packet is IPv6.. + * So, explicitly check for IPv4 before checking + * ipv4_csum_ok. + */ + if (mbuf->packet_type & RTE_PTYPE_L3_IPV4) { + if (enic_cq_rx_desc_ipv4_csum_ok(cqrd)) + pkt_flags |= PKT_RX_IP_CKSUM_GOOD; + else + pkt_flags |= PKT_RX_IP_CKSUM_BAD; + } if (l4_flags == RTE_PTYPE_L4_UDP || l4_flags == RTE_PTYPE_L4_TCP) { @@ -245,6 +287,7 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, struct vnic_cq *cq; volatile struct cq_desc *cqd_ptr; uint8_t color; + uint8_t tnl; uint16_t seg_length; struct rte_mbuf *first_seg = sop_rq->pkt_first_seg; struct rte_mbuf *last_seg = sop_rq->pkt_last_seg; @@ -336,10 +379,21 @@ enic_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, continue; } + /* + * When overlay offload is enabled, CQ.fcoe indicates the + * packet is tunnelled. + */ + tnl = enic->overlay_offload && + (ciflags & CQ_ENET_RQ_DESC_FLAGS_FCOE) != 0; /* cq rx flags are only valid if eop bit is set */ - first_seg->packet_type = enic_cq_rx_flags_to_pkt_type(&cqd); + first_seg->packet_type = + enic_cq_rx_flags_to_pkt_type(&cqd, tnl); enic_cq_rx_to_pkt_flags(&cqd, first_seg); - + /* Wipe the outer types set by enic_cq_rx_flags_to_pkt_type() */ + if (tnl) { + first_seg->packet_type &= ~(RTE_PTYPE_L3_MASK | + RTE_PTYPE_L4_MASK); + } if (unlikely(packet_error)) { rte_pktmbuf_free(first_seg); rte_atomic64_inc(&enic->soft_stats.rx_packet_errors); @@ -443,9 +497,10 @@ unsigned int enic_cleanup_wq(__rte_unused struct enic *enic, struct vnic_wq *wq) return 0; } -uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, +uint16_t enic_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) { + struct vnic_wq *wq = (struct vnic_wq *)tx_queue; int32_t ret; uint16_t i; uint64_t ol_flags; @@ -454,7 +509,7 @@ uint16_t enic_prep_pkts(__rte_unused void *tx_queue, struct rte_mbuf **tx_pkts, for (i = 0; i != nb_pkts; i++) { m = tx_pkts[i]; ol_flags = m->ol_flags; - if (ol_flags & ENIC_TX_OFFLOAD_NOTSUP_MASK) { + if (ol_flags & wq->tx_offload_notsup_mask) { rte_errno = -ENOTSUP; return i; } @@ -558,6 +613,11 @@ uint16_t enic_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, offload_mode = WQ_ENET_OFFLOAD_MODE_TSO; mss = tx_pkt->tso_segsz; + /* For tunnel, need the size of outer+inner headers */ + if (ol_flags & PKT_TX_TUNNEL_MASK) { + header_len += tx_pkt->outer_l2_len + + tx_pkt->outer_l3_len; + } } if ((ol_flags & ol_flags_mask) && (header_len == 0)) { -- 2.16.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH v2] net/enic: enable overlay offload for VXLAN and GENEVE 2018-04-16 21:49 ` [dpdk-dev] [PATCH v2] " John Daley @ 2018-04-17 18:05 ` Ferruh Yigit 0 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-04-17 18:05 UTC (permalink / raw) To: John Daley; +Cc: dev, Hyong Youb Kim On 4/16/2018 10:49 PM, John Daley wrote: > From: Hyong Youb Kim <hyonkim@cisco.com> > > Recent NIC models support overlay offload. The overlay offload > feature enables the following on the NIC. > - Rx/Tx checksum offloads for both inner and outer packets. > - Rx inner packet type classification. > - TSO. > - Inner RSS. > > TX descriptors do not require any changes, except the header length > for TSO. The NIC parses outer/inner packets and performs offloads on > them as necessary. The header length for tunneled TSO includes both > inner and outer headers. > > The NIC actually parses and performs the above for NVGRE as well. DPDK > currently has no offload flags for NVGRE, and the hardware has no > controls to individually enable tunnel types either. So do nothing for > now. > > The driver enables overlay offload by default. Add a devargs > 'disable-overlay=<0|1>' to allow the app to disable it. > > Also update the enic guide doc. > > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> > Reviewed-by: John Daley <johndale@cisco.com> Applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters 2018-04-04 23:54 [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC John Daley 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE John Daley @ 2018-04-04 23:54 ` John Daley 2018-04-06 16:41 ` Ferruh Yigit 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues John Daley ` (3 subsequent siblings) 5 siblings, 1 reply; 18+ messages in thread From: John Daley @ 2018-04-04 23:54 UTC (permalink / raw) To: ferruh.yigit; +Cc: dev, John Daley Recent models support IPv4/IPv6 UDP RSS. There is no control bit to enable UDP RSS alone. Instead, the NIC enables/disables TCP and UDP RSS together. Signed-off-by: John Daley <johndale@cisco.com> Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com> --- doc/guides/nics/enic.rst | 2 +- drivers/net/enic/base/vnic_dev.c | 18 ++++++++++++++++++ drivers/net/enic/base/vnic_dev.h | 1 + drivers/net/enic/base/vnic_nic.h | 1 + drivers/net/enic/enic_main.c | 12 ++++++++++++ drivers/net/enic/enic_res.c | 5 +++++ 6 files changed, 38 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index 133b1e682..d02a69829 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -416,6 +416,7 @@ Supported features - VLAN filtering (supported via UCSM/CIMC only) - Execution of application by unprivileged system users - IPV4, IPV6 and TCP RSS hashing +- UDP hashing (1400 series and later adapters) - Scattered Rx - MTU update - SR-IOV on UCS managed servers connected to Fabric Interconnects @@ -434,7 +435,6 @@ Known bugs and unsupported features in this release - VLAN based flow direction - Non-IPV4 flow direction - Setting of extended VLAN -- UDP RSS hashing - MTU update only works if Scattered Rx mode is disabled - Maximum receive packet length is ignored if Scattered Rx mode is used diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c index 386db6411..1d6734dc1 100644 --- a/drivers/net/enic/base/vnic_dev.c +++ b/drivers/net/enic/base/vnic_dev.c @@ -10,6 +10,7 @@ #include "vnic_dev.h" #include "vnic_resource.h" #include "vnic_devcmd.h" +#include "vnic_nic.h" #include "vnic_stats.h" @@ -531,6 +532,23 @@ int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode, return 0; } +int vnic_dev_capable_udp_rss(struct vnic_dev *vdev) +{ + u64 a0 = CMD_NIC_CFG, a1 = 0; + u64 rss_hash_type; + int wait = 1000; + int err; + + err = vnic_dev_cmd(vdev, CMD_CAPABILITY, &a0, &a1, wait); + if (err) + return 0; + if (a0 == 0) + return 0; + rss_hash_type = (a1 >> NIC_CFG_RSS_HASH_TYPE_SHIFT) & + NIC_CFG_RSS_HASH_TYPE_MASK_FIELD; + return ((rss_hash_type & NIC_CFG_RSS_HASH_TYPE_UDP) ? 1 : 0); +} + int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd) { u64 a0 = (u32)cmd, a1 = 0; diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h index b3ce26032..62c09fa86 100644 --- a/drivers/net/enic/base/vnic_dev.h +++ b/drivers/net/enic/base/vnic_dev.h @@ -109,6 +109,7 @@ int vnic_dev_capable_adv_filters(struct vnic_dev *vdev); int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd); int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode, u8 *filter_tags); +int vnic_dev_capable_udp_rss(struct vnic_dev *vdev); int vnic_dev_asic_info(struct vnic_dev *vdev, u16 *asic_type, u16 *asic_rev); int vnic_dev_spec(struct vnic_dev *vdev, unsigned int offset, size_t size, void *value); diff --git a/drivers/net/enic/base/vnic_nic.h b/drivers/net/enic/base/vnic_nic.h index a753b3a5a..b20915e76 100644 --- a/drivers/net/enic/base/vnic_nic.h +++ b/drivers/net/enic/base/vnic_nic.h @@ -33,6 +33,7 @@ #define NIC_CFG_RSS_HASH_TYPE_TCP_IPV6 (1 << 4) #define NIC_CFG_RSS_HASH_TYPE_IPV6_EX (1 << 5) #define NIC_CFG_RSS_HASH_TYPE_TCP_IPV6_EX (1 << 6) +#define NIC_CFG_RSS_HASH_TYPE_UDP (1 << 7) static inline void vnic_set_nic_cfg(u32 *nic_cfg, u8 rss_default_cpu, u8 rss_hash_type, diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c index 9b254d98e..f01a77c0a 100644 --- a/drivers/net/enic/enic_main.c +++ b/drivers/net/enic/enic_main.c @@ -1178,10 +1178,22 @@ int enic_set_rss_conf(struct enic *enic, struct rte_eth_rss_conf *rss_conf) rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV4; if (rss_hf & ETH_RSS_NONFRAG_IPV4_TCP) rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4; + if (rss_hf & ETH_RSS_NONFRAG_IPV4_UDP) { + /* + * 'TCP' is not a typo. HW does not have a separate + * enable bit for UDP RSS. The TCP bit enables both TCP + * and UDP RSS.. + */ + rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV4; + } if (rss_hf & ETH_RSS_IPV6) rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV6; if (rss_hf & ETH_RSS_NONFRAG_IPV6_TCP) rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6; + if (rss_hf & ETH_RSS_NONFRAG_IPV6_UDP) { + /* Again, 'TCP' is not a typo. */ + rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_TCP_IPV6; + } if (rss_hf & ETH_RSS_IPV6_EX) rss_hash_type |= NIC_CFG_RSS_HASH_TYPE_IPV6_EX; if (rss_hf & ETH_RSS_IPV6_TCP_EX) diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c index 47f825e51..34532720c 100644 --- a/drivers/net/enic/enic_res.c +++ b/drivers/net/enic/enic_res.c @@ -144,6 +144,11 @@ int enic_get_vnic_config(struct enic *enic) enic->flow_type_rss_offloads |= ETH_RSS_IPV6_EX; if (ENIC_SETTING(enic, RSSHASH_TCPIPV6_EX)) enic->flow_type_rss_offloads |= ETH_RSS_IPV6_TCP_EX; + if (vnic_dev_capable_udp_rss(enic->vdev)) { + enic->flow_type_rss_offloads |= + ETH_RSS_NONFRAG_IPV4_UDP | ETH_RSS_NONFRAG_IPV6_UDP; + } + /* Zero offloads if RSS is not enabled */ if (!ENIC_SETTING(enic, RSS)) enic->flow_type_rss_offloads = 0; -- 2.16.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters John Daley @ 2018-04-06 16:41 ` Ferruh Yigit 0 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-04-06 16:41 UTC (permalink / raw) To: John Daley; +Cc: dev On 4/5/2018 12:54 AM, John Daley wrote: > Recent models support IPv4/IPv6 UDP RSS. There is no control bit to > enable UDP RSS alone. Instead, the NIC enables/disables TCP and UDP > RSS together. > > Signed-off-by: John Daley <johndale@cisco.com> > Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com> Applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues 2018-04-04 23:54 [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC John Daley 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE John Daley 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters John Daley @ 2018-04-04 23:54 ` John Daley 2018-04-06 16:41 ` Ferruh Yigit 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support the drop flow action John Daley ` (2 subsequent siblings) 5 siblings, 1 reply; 18+ messages in thread From: John Daley @ 2018-04-04 23:54 UTC (permalink / raw) To: ferruh.yigit; +Cc: dev, John Daley, stable The enic code called from rte_eth_dev_set_mtu() was assuming that the Rx queues are already set up via a call to rte_eth_tx_queue_setup(). OVS calls rte_eth_dev_set_mtu() before rte_eth_rx_queue_setup() and a null pointer was dereferenced. Fixes: c3e09182bcd6 ("net/enic: support scatter Rx in MTU update") Cc: stable@dpdk.org Signed-off-by: John Daley <johndale@cisco.com> Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com> --- drivers/net/enic/enic_main.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/net/enic/enic_main.c b/drivers/net/enic/enic_main.c index f01a77c0a..f39dfbe32 100644 --- a/drivers/net/enic/enic_main.c +++ b/drivers/net/enic/enic_main.c @@ -1444,6 +1444,8 @@ int enic_set_mtu(struct enic *enic, uint16_t new_mtu) /* free and reallocate RQs with the new MTU */ for (rq_idx = 0; rq_idx < enic->rq_count; rq_idx++) { rq = &enic->rq[enic_rte_rq_idx_to_sop_idx(rq_idx)]; + if (!rq->in_use) + continue; enic_free_rq(rq); rc = enic_alloc_rq(enic, rq_idx, rq->socket_id, rq->mp, -- 2.16.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues John Daley @ 2018-04-06 16:41 ` Ferruh Yigit 0 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-04-06 16:41 UTC (permalink / raw) To: John Daley; +Cc: dev, stable On 4/5/2018 12:54 AM, John Daley wrote: > The enic code called from rte_eth_dev_set_mtu() was assuming that the > Rx queues are already set up via a call to rte_eth_tx_queue_setup(). > OVS calls rte_eth_dev_set_mtu() before rte_eth_rx_queue_setup() and > a null pointer was dereferenced. > > Fixes: c3e09182bcd6 ("net/enic: support scatter Rx in MTU update") > Cc: stable@dpdk.org > > Signed-off-by: John Daley <johndale@cisco.com> > Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com> Applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH] net/enic: support the drop flow action 2018-04-04 23:54 [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC John Daley ` (2 preceding siblings ...) 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues John Daley @ 2018-04-04 23:54 ` John Daley 2018-04-06 16:41 ` Ferruh Yigit 2018-04-04 23:54 ` [dpdk-dev] [PATCH] doc: update the enic guide and features John Daley 2018-04-06 16:40 ` [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC Ferruh Yigit 5 siblings, 1 reply; 18+ messages in thread From: John Daley @ 2018-04-04 23:54 UTC (permalink / raw) To: ferruh.yigit; +Cc: dev, Hyong Youb Kim From: Hyong Youb Kim <hyonkim@cisco.com> 1330 and 1400 series adapters support the drop action. Check for its availability and set the necessary flag when creating NIC filters. Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> Reviewed-by: John Daley <johndale@cisco.com> --- doc/guides/nics/enic.rst | 2 +- drivers/net/enic/base/vnic_dev.c | 14 +++++--------- drivers/net/enic/base/vnic_dev.h | 2 +- drivers/net/enic/base/vnic_devcmd.h | 2 ++ drivers/net/enic/enic.h | 2 +- drivers/net/enic/enic_flow.c | 34 +++++++++++++++++++++++++++------- drivers/net/enic/enic_res.c | 11 ++++++++--- 7 files changed, 45 insertions(+), 22 deletions(-) diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index d02a69829..df3718ecd 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -251,7 +251,7 @@ Generic Flow API is supported. The baseline support is: - Attributes: ingress - Items: eth, ipv4, ipv6, udp, tcp, vxlan, inner eth, ipv4, ipv6, udp, tcp - - Actions: queue, mark, flag and void + - Actions: queue, mark, drop, flag and void - Selectors: 'is', 'spec' and 'mask'. 'last' is not supported - In total, up to 64 bytes of mask is allowed across all headers diff --git a/drivers/net/enic/base/vnic_dev.c b/drivers/net/enic/base/vnic_dev.c index 1d6734dc1..8880ab981 100644 --- a/drivers/net/enic/base/vnic_dev.c +++ b/drivers/net/enic/base/vnic_dev.c @@ -485,7 +485,7 @@ int vnic_dev_capable_adv_filters(struct vnic_dev *vdev) * Retrun true in filter_tags if supported */ int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode, - u8 *filter_tags) + u8 *filter_actions) { u64 args[4]; int err; @@ -493,14 +493,10 @@ int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode, err = vnic_dev_advanced_filters_cap(vdev, args, 4); - /* determine if filter tags are available */ - if (err) - *filter_tags = 0; - if ((args[2] == FILTER_CAP_MODE_V1) && - (args[3] & FILTER_ACTION_FILTER_ID_FLAG)) - *filter_tags = 1; - else - *filter_tags = 0; + /* determine supported filter actions */ + *filter_actions = FILTER_ACTION_RQ_STEERING_FLAG; /* always available */ + if (args[2] == FILTER_CAP_MODE_V1) + *filter_actions = args[3]; if (err || ((args[0] == 1) && (args[1] == 0))) { /* Adv filter Command not supported or adv filters available but diff --git a/drivers/net/enic/base/vnic_dev.h b/drivers/net/enic/base/vnic_dev.h index 62c09fa86..e7a1f8bd8 100644 --- a/drivers/net/enic/base/vnic_dev.h +++ b/drivers/net/enic/base/vnic_dev.h @@ -108,7 +108,7 @@ int vnic_dev_fw_info(struct vnic_dev *vdev, int vnic_dev_capable_adv_filters(struct vnic_dev *vdev); int vnic_dev_capable(struct vnic_dev *vdev, enum vnic_devcmd_cmd cmd); int vnic_dev_capable_filter_mode(struct vnic_dev *vdev, u32 *mode, - u8 *filter_tags); + u8 *filter_actions); int vnic_dev_capable_udp_rss(struct vnic_dev *vdev); int vnic_dev_asic_info(struct vnic_dev *vdev, u16 *asic_type, u16 *asic_rev); int vnic_dev_spec(struct vnic_dev *vdev, unsigned int offset, size_t size, diff --git a/drivers/net/enic/base/vnic_devcmd.h b/drivers/net/enic/base/vnic_devcmd.h index d09113dac..2865eb4d4 100644 --- a/drivers/net/enic/base/vnic_devcmd.h +++ b/drivers/net/enic/base/vnic_devcmd.h @@ -841,7 +841,9 @@ struct filter_action { #define FILTER_ACTION_RQ_STEERING_FLAG (1 << 0) #define FILTER_ACTION_FILTER_ID_FLAG (1 << 1) +#define FILTER_ACTION_DROP_FLAG (1 << 2) #define FILTER_ACTION_V2_ALL (FILTER_ACTION_RQ_STEERING_FLAG \ + | FILTER_ACTION_DROP_FLAG \ | FILTER_ACTION_FILTER_ID_FLAG) /* Version 2 of filter action must be a strict extension of struct filter_action diff --git a/drivers/net/enic/enic.h b/drivers/net/enic/enic.h index 9d69d1e9e..e4936ec13 100644 --- a/drivers/net/enic/enic.h +++ b/drivers/net/enic/enic.h @@ -119,7 +119,7 @@ struct enic { u16 max_mtu; u8 adv_filters; u32 flow_filter_mode; - u8 filter_tags; + u8 filter_actions; /* HW supported actions */ u8 vxlan; unsigned int flags; diff --git a/drivers/net/enic/enic_flow.c b/drivers/net/enic/enic_flow.c index 28923b0e2..b9f36587c 100644 --- a/drivers/net/enic/enic_flow.c +++ b/drivers/net/enic/enic_flow.c @@ -273,21 +273,33 @@ static const enum rte_flow_action_type enic_supported_actions_v1[] = { }; /** Supported actions for newer NICs */ -static const enum rte_flow_action_type enic_supported_actions_v2[] = { +static const enum rte_flow_action_type enic_supported_actions_v2_id[] = { RTE_FLOW_ACTION_TYPE_QUEUE, RTE_FLOW_ACTION_TYPE_MARK, RTE_FLOW_ACTION_TYPE_FLAG, RTE_FLOW_ACTION_TYPE_END, }; +static const enum rte_flow_action_type enic_supported_actions_v2_drop[] = { + RTE_FLOW_ACTION_TYPE_QUEUE, + RTE_FLOW_ACTION_TYPE_MARK, + RTE_FLOW_ACTION_TYPE_FLAG, + RTE_FLOW_ACTION_TYPE_DROP, + RTE_FLOW_ACTION_TYPE_END, +}; + /** Action capabilities indexed by NIC version information */ static const struct enic_action_cap enic_action_cap[] = { [FILTER_ACTION_RQ_STEERING_FLAG] = { .actions = enic_supported_actions_v1, .copy_fn = enic_copy_action_v1, }, - [FILTER_ACTION_V2_ALL] = { - .actions = enic_supported_actions_v2, + [FILTER_ACTION_FILTER_ID_FLAG] = { + .actions = enic_supported_actions_v2_id, + .copy_fn = enic_copy_action_v2, + }, + [FILTER_ACTION_DROP_FLAG] = { + .actions = enic_supported_actions_v2_drop, .copy_fn = enic_copy_action_v2, }, }; @@ -1021,6 +1033,10 @@ enic_copy_action_v2(const struct rte_flow_action actions[], enic_action->flags |= FILTER_ACTION_FILTER_ID_FLAG; break; } + case RTE_FLOW_ACTION_TYPE_DROP: { + enic_action->flags |= FILTER_ACTION_DROP_FLAG; + break; + } case RTE_FLOW_ACTION_TYPE_VOID: continue; default: @@ -1059,10 +1075,14 @@ enic_get_filter_cap(struct enic *enic) static const struct enic_action_cap * enic_get_action_cap(struct enic *enic) { - static const struct enic_action_cap *ea; - - if (enic->filter_tags) - ea = &enic_action_cap[FILTER_ACTION_V2_ALL]; + const struct enic_action_cap *ea; + uint8_t actions; + + actions = enic->filter_actions; + if (actions & FILTER_ACTION_DROP_FLAG) + ea = &enic_action_cap[FILTER_ACTION_DROP_FLAG]; + else if (actions & FILTER_ACTION_FILTER_ID_FLAG) + ea = &enic_action_cap[FILTER_ACTION_FILTER_ID_FLAG]; else ea = &enic_action_cap[FILTER_ACTION_RQ_STEERING_FLAG]; return ea; diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c index 34532720c..de17a31d0 100644 --- a/drivers/net/enic/enic_res.c +++ b/drivers/net/enic/enic_res.c @@ -76,19 +76,24 @@ int enic_get_vnic_config(struct enic *enic) ? "" : "not ")); err = vnic_dev_capable_filter_mode(enic->vdev, &enic->flow_filter_mode, - &enic->filter_tags); + &enic->filter_actions); if (err) { dev_err(enic_get_dev(enic), "Error getting filter modes, %d\n", err); return err; } - dev_info(enic, "Flow api filter mode: %s, Filter tagging %savailable\n", + dev_info(enic, "Flow api filter mode: %s Actions: %s%s%s\n", ((enic->flow_filter_mode == FILTER_DPDK_1) ? "DPDK" : ((enic->flow_filter_mode == FILTER_USNIC_IP) ? "USNIC" : ((enic->flow_filter_mode == FILTER_IPV4_5TUPLE) ? "5TUPLE" : "NONE"))), - ((enic->filter_tags) ? "" : "not ")); + ((enic->filter_actions & FILTER_ACTION_RQ_STEERING_FLAG) ? + "steer " : ""), + ((enic->filter_actions & FILTER_ACTION_FILTER_ID_FLAG) ? + "tag " : ""), + ((enic->filter_actions & FILTER_ACTION_DROP_FLAG) ? + "drop " : "")); c->wq_desc_count = min_t(u32, ENIC_MAX_WQ_DESCS, -- 2.16.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] net/enic: support the drop flow action 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support the drop flow action John Daley @ 2018-04-06 16:41 ` Ferruh Yigit 0 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-04-06 16:41 UTC (permalink / raw) To: John Daley; +Cc: dev, Hyong Youb Kim On 4/5/2018 12:54 AM, John Daley wrote: > From: Hyong Youb Kim <hyonkim@cisco.com> > > 1330 and 1400 series adapters support the drop action. Check for its > availability and set the necessary flag when creating NIC filters. > > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> > Reviewed-by: John Daley <johndale@cisco.com> Applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH] doc: update the enic guide and features 2018-04-04 23:54 [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC John Daley ` (3 preceding siblings ...) 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support the drop flow action John Daley @ 2018-04-04 23:54 ` John Daley 2018-04-06 16:42 ` Ferruh Yigit 2018-04-06 16:40 ` [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC Ferruh Yigit 5 siblings, 1 reply; 18+ messages in thread From: John Daley @ 2018-04-04 23:54 UTC (permalink / raw) To: ferruh.yigit; +Cc: dev, Hyong Youb Kim From: Hyong Youb Kim <hyonkim@cisco.com> Documentation updates including for 1400 series VIC adapters. Remove VLAN filter from the features file as the driver does not support that API. Hardware does support VLAN filtering, but it is not controlled through the driver. Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> Reviewed-by: John Daley <johndale@cisco.com> --- doc/guides/nics/enic.rst | 28 +++++++++++++++------------- doc/guides/nics/features/enic.ini | 1 - 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index df3718ecd..1e2d0b56a 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -236,7 +236,7 @@ Generic Flow API is supported. The baseline support is: - Actions: queue and void - Selectors: 'is' -- **1300 series VICS with advanced filters disabled** +- **1300 and later series VICS with advanced filters disabled** With advanced filters disabled, an IPv4 or IPv6 item must be specified in the pattern. @@ -247,7 +247,7 @@ Generic Flow API is supported. The baseline support is: - Selectors: 'is', 'spec' and 'mask'. 'last' is not supported - In total, up to 64 bytes of mask is allowed across all headers -- **1300 series VICS with advanced filters enabled** +- **1300 and later series VICS with advanced filters enabled** - Attributes: ingress - Items: eth, ipv4, ipv6, udp, tcp, vxlan, inner eth, ipv4, ipv6, udp, tcp @@ -360,10 +360,20 @@ Limitations - **Statistics** - ``rx_good_bytes`` (ibytes) always includes VLAN header (4B) and CRC bytes (4B). + This behavior applies to 1300 and older series VIC adapters. + 1400 series VICs do not count CRC bytes, and count VLAN header only when VLAN + stripping is disabled. - When the NIC drops a packet because the Rx queue has no free buffers, ``rx_good_bytes`` still increments by 4B if the packet is not VLAN tagged or VLAN stripping is disabled, or by 8B if the packet is VLAN tagged and stripping is enabled. + This behavior applies to 1300 and older series VIC adapters. 1400 series VICs + do not increment this byte counter when packets are dropped. + +- **RSS Hashing** + + - Hardware enables and disables UDP and TCP RSS hashing together. The driver + cannot control UDP and TCP hashing individually. How to build the suite ---------------------- @@ -382,17 +392,9 @@ Supported Cisco VIC adapters ENIC PMD supports all recent generations of Cisco VIC adapters including: -- VIC 1280 -- VIC 1240 -- VIC 1225 -- VIC 1285 -- VIC 1225T -- VIC 1227 -- VIC 1227T -- VIC 1380 -- VIC 1340 -- VIC 1385 -- VIC 1387 +- VIC 1200 series +- VIC 1300 series +- VIC 1400 series Supported Operating Systems --------------------------- diff --git a/doc/guides/nics/features/enic.ini b/doc/guides/nics/features/enic.ini index ea171a45b..99d37708c 100644 --- a/doc/guides/nics/features/enic.ini +++ b/doc/guides/nics/features/enic.ini @@ -19,7 +19,6 @@ RSS hash = Y RSS key update = Y RSS reta update = Y SR-IOV = Y -VLAN filter = Y CRC offload = Y VLAN offload = Y Flow director = Y -- 2.16.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] doc: update the enic guide and features 2018-04-04 23:54 ` [dpdk-dev] [PATCH] doc: update the enic guide and features John Daley @ 2018-04-06 16:42 ` Ferruh Yigit 0 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-04-06 16:42 UTC (permalink / raw) To: John Daley; +Cc: dev, Hyong Youb Kim On 4/5/2018 12:54 AM, John Daley wrote: > From: Hyong Youb Kim <hyonkim@cisco.com> > > Documentation updates including for 1400 series VIC adapters. > > Remove VLAN filter from the features file as the driver does not > support that API. Hardware does support VLAN filtering, but it is not > controlled through the driver. > > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> > Reviewed-by: John Daley <johndale@cisco.com> Applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC 2018-04-04 23:54 [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC John Daley ` (4 preceding siblings ...) 2018-04-04 23:54 ` [dpdk-dev] [PATCH] doc: update the enic guide and features John Daley @ 2018-04-06 16:40 ` Ferruh Yigit 5 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-04-06 16:40 UTC (permalink / raw) To: John Daley; +Cc: dev, Hyong Youb Kim On 4/5/2018 12:54 AM, John Daley wrote: > From: Hyong Youb Kim <hyonkim@cisco.com> > > The firmware on new hardware models flushes the global descriptor > cache by default. Use CMD_OPENF_IG_DESCCACHE to avoid cache > flushing. This flag has no effect on older models. > > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> > Suggested-by: Govindarajulu Varadarajan <gvaradar@cisco.com> > Reviewed-by: John Daley <johndale@cisco.com> Applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* [dpdk-dev] [PATCH] doc: update the enic guide and features @ 2018-07-18 2:15 John Daley 2018-07-18 10:22 ` Ferruh Yigit 0 siblings, 1 reply; 18+ messages in thread From: John Daley @ 2018-07-18 2:15 UTC (permalink / raw) To: john.mcnamara, marko.kovacevic; +Cc: dev, Hyong Youb Kim From: Hyong Youb Kim <hyonkim@cisco.com> Make a few updates in preparation for 18.08. - Use SPDX - Add 1400 series VIC adapters to supported models - Describe the VXLAN port number - Expand the description for ig-vlan-rewrite - Add inner RSS and checksum to the features Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> Reviewed-by: John Daley <johndale@cisco.com> --- doc/guides/nics/enic.rst | 76 +++++++++++++++++++++++---------------- doc/guides/nics/features/enic.ini | 3 ++ 2 files changed, 48 insertions(+), 31 deletions(-) diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index 7764c8648..438a83d5f 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -1,32 +1,7 @@ -.. BSD LICENSE +.. SPDX-License-Identifier: BSD-3-Clause Copyright (c) 2017, Cisco Systems, Inc. All rights reserved. - Redistribution and use in source and binary forms, with or without - modification, are permitted provided that the following conditions - are met: - - 1. Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - - 2. Redistributions in binary form must reproduce the above copyright - notice, this list of conditions and the following disclaimer in - the documentation and/or other materials provided with the - distribution. - - THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS - FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE - COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, - INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, - BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE - POSSIBILITY OF SUCH DAMAGE. - ENIC Poll Mode Driver ===================== @@ -336,6 +311,40 @@ it, set ``devargs`` parameter ``disable-overlay=1``. For example:: -w 12:00.0,disable-overlay=1 +By default, the NIC uses 4789 as the VXLAN port. The user may change +it through ``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as +the current NIC has a single VXLAN port number, the user cannot +configure multiple port numbers. + +Ingress VLAN Rewrite +-------------------- + +VIC adapters can tag, untag, or modify the VLAN headers of ingress +packets. The ingress VLAN rewrite mode controls this behavior. By +default, it is set to pass-through, where the NIC does not modify the +VLAN header in any way so that the application can see the original +header. This mode is sufficient for many applications, but may not be +suitable for others. Such applications may change the mode by setting +``devargs`` parameter ``ig-vlan-rewrite`` to one of the following. + +- ``pass``: Pass-through mode. The NIC does not modify the VLAN + header. This is the default mode. + +- ``priority``: Priority-tag default VLAN mode. If the ingress packet + is tagged with the default VLAN, the NIC replaces its VLAN header + with the priority tag (VLAN ID 0). + +- ``trunk``: Default trunk mode. The NIC tags untagged ingress packets + with the default VLAN. Tagged ingress packets are not modified. To + the application, every packet appears as tagged. + +- ``untag``: Untag default VLAN mode. If the ingress packet is tagged + with the default VLAN, the NIC removes or untags its VLAN header so + that the application sees an untagged packet. As a result, the + default VLAN becomes `untagged`. This mode can be useful for + applications such as OVS-DPDK performance benchmarks that utilize + only the default VLAN and want to see only untagged packets. + .. _enic_limitations: Limitations @@ -366,9 +375,9 @@ Another alternative is modify the adapter's ingress VLAN rewrite mode so that packets with the default VLAN tag are stripped by the adapter and presented to DPDK as untagged packets. In this case mbuf->vlan_tci and the PKT_RX_VLAN and PKT_RX_VLAN_STRIPPED mbuf flags would not be set. This mode is enabled with the -``devargs`` parameter ``ig-vlan-rewrite=1``. For example:: +``devargs`` parameter ``ig-vlan-rewrite=untag``. For example:: - -w 12:00.0,ig-vlan-rewrite=1 + -w 12:00.0,ig-vlan-rewrite=untag - Limited flow director support on 1200 series and 1300 series Cisco VIC adapters with old firmware. Please see :ref:`enic-flow-director`. @@ -405,10 +414,14 @@ PKT_RX_VLAN_STRIPPED mbuf flags would not be set. This mode is enabled with the - ``rx_good_bytes`` (ibytes) always includes VLAN header (4B) and CRC bytes (4B). This behavior applies to 1300 and older series VIC adapters. + 1400 series VICs do not count CRC bytes, and count VLAN header only when VLAN + stripping is disabled. - When the NIC drops a packet because the Rx queue has no free buffers, ``rx_good_bytes`` still increments by 4B if the packet is not VLAN tagged or VLAN stripping is disabled, or by 8B if the packet is VLAN tagged and stripping - is enabled. This behavior applies to 1300 and older series VIC adapters. + is enabled. + This behavior applies to 1300 and older series VIC adapters. 1400 series VICs + do not increment this byte counter when packets are dropped. - **RSS Hashing** @@ -434,6 +447,7 @@ ENIC PMD supports all recent generations of Cisco VIC adapters including: - VIC 1200 series - VIC 1300 series +- VIC 1400 series Supported Operating Systems --------------------------- @@ -457,7 +471,7 @@ Supported features - VLAN filtering (supported via UCSM/CIMC only) - Execution of application by unprivileged system users - IPV4, IPV6 and TCP RSS hashing -- UDP RSS hashing (support for upcoming adapters) +- UDP RSS hashing (1400 series and later adapters) - Scattered Rx - MTU update - SR-IOV on UCS managed servers connected to Fabric Interconnects @@ -534,4 +548,4 @@ Any questions or bugs should be reported to DPDK community and to the ENIC PMD maintainers: - John Daley <johndale@cisco.com> -- Nelson Escobar <neescoba@cisco.com> +- Hyong Youb Kim <hyonkim@cisco.com> diff --git a/doc/guides/nics/features/enic.ini b/doc/guides/nics/features/enic.ini index 31a3fb5cf..d2b866b66 100644 --- a/doc/guides/nics/features/enic.ini +++ b/doc/guides/nics/features/enic.ini @@ -20,6 +20,7 @@ Multicast MAC filter = RSS hash = Y RSS key update = Y RSS reta update = Y +Inner RSS = Y SR-IOV = Y CRC offload = Y VLAN offload = Y @@ -27,6 +28,8 @@ Flow director = Y Flow API = Y L3 checksum offload = Y L4 checksum offload = Y +Inner L3 checksum = Y +Inner L4 checksum = Y Packet type parsing = Y Basic stats = Y Multiprocess aware = Y -- 2.16.2 ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [dpdk-dev] [PATCH] doc: update the enic guide and features 2018-07-18 2:15 [dpdk-dev] [PATCH] doc: update the enic guide and features John Daley @ 2018-07-18 10:22 ` Ferruh Yigit 0 siblings, 0 replies; 18+ messages in thread From: Ferruh Yigit @ 2018-07-18 10:22 UTC (permalink / raw) To: John Daley, john.mcnamara, marko.kovacevic; +Cc: dev, Hyong Youb Kim On 7/18/2018 3:15 AM, John Daley wrote: > From: Hyong Youb Kim <hyonkim@cisco.com> > > Make a few updates in preparation for 18.08. > - Use SPDX > - Add 1400 series VIC adapters to supported models > - Describe the VXLAN port number > - Expand the description for ig-vlan-rewrite > - Add inner RSS and checksum to the features > > Signed-off-by: Hyong Youb Kim <hyonkim@cisco.com> > Reviewed-by: John Daley <johndale@cisco.com> <...> > @@ -534,4 +548,4 @@ Any questions or bugs should be reported to DPDK community and to the ENIC PMD > maintainers: > > - John Daley <johndale@cisco.com> > -- Nelson Escobar <neescoba@cisco.com> > +- Hyong Youb Kim <hyonkim@cisco.com> This information is duplication of MAINTAINERS file, what would you think removing names from here and reference to maintainers file? Applied to dpdk-next-net/master, thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2018-07-18 10:22 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-04-04 23:54 [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC John Daley 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: enable overlay offload for VXLAN and GENEVE John Daley 2018-04-06 16:15 ` Ferruh Yigit 2018-04-07 2:40 ` Hyong Youb Kim 2018-04-09 12:52 ` Ferruh Yigit 2018-04-16 21:49 ` [dpdk-dev] [PATCH v2] " John Daley 2018-04-17 18:05 ` Ferruh Yigit 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support UDP RSS on 1400 series adapters John Daley 2018-04-06 16:41 ` Ferruh Yigit 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: fix seg fault on MTU update with non-setup queues John Daley 2018-04-06 16:41 ` Ferruh Yigit 2018-04-04 23:54 ` [dpdk-dev] [PATCH] net/enic: support the drop flow action John Daley 2018-04-06 16:41 ` Ferruh Yigit 2018-04-04 23:54 ` [dpdk-dev] [PATCH] doc: update the enic guide and features John Daley 2018-04-06 16:42 ` Ferruh Yigit 2018-04-06 16:40 ` [dpdk-dev] [PATCH] net/enic: do not flush descriptor cache when opening vNIC Ferruh Yigit 2018-07-18 2:15 [dpdk-dev] [PATCH] doc: update the enic guide and features John Daley 2018-07-18 10:22 ` Ferruh Yigit
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).