From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 67FD6A04E1;
	Tue, 22 Sep 2020 10:58:23 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 11CF51DBB8;
	Tue, 22 Sep 2020 10:51:24 +0200 (CEST)
Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com
 [67.231.154.164]) by dpdk.org (Postfix) with ESMTP id DA3B71D8CE
 for <dev@dpdk.org>; Tue, 22 Sep 2020 10:50:22 +0200 (CEST)
Received: from mx1-us1.ppe-hosted.com (unknown [10.110.50.137])
 by dispatch1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id
 B376020058 for <dev@dpdk.org>; Tue, 22 Sep 2020 08:50:22 +0000 (UTC)
Received: from us4-mdac16-1.at1.mdlocal (unknown [10.110.49.147])
 by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTP id B33016009B
 for <dev@dpdk.org>; Tue, 22 Sep 2020 08:50:22 +0000 (UTC)
X-Virus-Scanned: Proofpoint Essentials engine
Received: from mx1-us1.ppe-hosted.com (unknown [10.110.50.12])
 by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id D3C63220064
 for <dev@dpdk.org>; Tue, 22 Sep 2020 08:50:21 +0000 (UTC)
Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits))
 (No client certificate requested)
 by mx1-us1.ppe-hosted.com (PPE Hosted ESMTP Server) with ESMTPS id 9CD734005E
 for <dev@dpdk.org>; Tue, 22 Sep 2020 08:50:21 +0000 (UTC)
Received: from ukex01.SolarFlarecom.com (10.17.10.4) by
 ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id
 15.0.1497.2; Tue, 22 Sep 2020 09:50:11 +0100
Received: from opal.uk.solarflarecom.com (10.17.10.1) by
 ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server id
 15.0.1497.2 via Frontend Transport; Tue, 22 Sep 2020 09:50:11 +0100
Received: from ukv-loginhost.uk.solarflarecom.com
 (ukv-loginhost.uk.solarflarecom.com [10.17.10.39])
 by opal.uk.solarflarecom.com (8.13.8/8.13.8) with ESMTP id 08M8oBJm004819;
 Tue, 22 Sep 2020 09:50:11 +0100
Received: from ukv-loginhost.uk.solarflarecom.com (localhost [127.0.0.1])
 by ukv-loginhost.uk.solarflarecom.com (Postfix) with ESMTP id C564B1613AB;
 Tue, 22 Sep 2020 09:50:11 +0100 (BST)
From: Andrew Rybchenko <arybchenko@solarflare.com>
To: <dev@dpdk.org>
CC: Igor Romanov <igor.romanov@oktetlabs.ru>
Date: Tue, 22 Sep 2020 09:49:44 +0100
Message-ID: <1600764594-14752-51-git-send-email-arybchenko@solarflare.com>
X-Mailer: git-send-email 1.8.3.1
In-Reply-To: <1600764594-14752-1-git-send-email-arybchenko@solarflare.com>
References: <1600764594-14752-1-git-send-email-arybchenko@solarflare.com>
MIME-Version: 1.0
Content-Type: text/plain
X-TM-AS-Product-Ver: SMEX-12.5.0.1300-8.6.1012-25674.003
X-TM-AS-Result: No-7.880200-8.000000-10
X-TMASE-MatchedRID: fvfYm8EmthcoS/tMLUVY9bBZAi3nrnzbMVx/3ZYby7903EU8crzuS+Z5
 Gn23AeDZuA9fFHhyLzywgcHDNo5AtPI1YbpS1+avPwKTD1v8YV5MkOX0UoduuTbpMgyAfh26u2H
 gmoVy+kry2EKN1EBlF5yUY93IDmCeO3EUo7ts5V6qNnzrkU+2mgoXSOLC5a443nbDJg0gir/vWC
 W/gP4RcCNbkVYpiM4Y57jS/hsxRY9lHEcK8cX2YMzSKGx9g8xhjs/yyyVHGh4Z9P2vDaImpqip1
 8v0DWYVJkprB3tCGSlLBme6SEq2PUobO+54PiwGlIlVPzMCz/SwqLgRdvwAipGhAvBSa2i/1GWb
 KN8y66+EbplTnExxytQfNGn/5BppeFTBDh9n8Ezi447y/nAm6KR07tNu9vNjH1bhq4z+yfSdW2C
 /Ex2sg0fivM8BUT0x0wo2BidtLYkoNJV93bkz0GFdfLBMkul8XGjQf7uckKtZ+YxyNxdzR5dtyT
 UmkMozF3zGB4TkDtw3WFHyJFVzsIy3ARqB9bsb+ACG5oWJ7tLLRD51bz5RZJGJXR6jRFu4c4iQI
 JxbfRoLwJuN5zSpTrUrIc4nkZtOGAdnzrnkM48URSScn+QSXt0H8LFZNFG7bkV4e2xSge7I/7GZ
 nYPuf4n5sqzLqwrUBGheuLZvRK8S8HrICCVHtGA7bUFBqh2V
X-TM-AS-User-Approved-Sender: Yes
X-TM-AS-User-Blocked-Sender: No
X-TMASE-Result: 10--7.880200-8.000000
X-TMASE-Version: SMEX-12.5.0.1300-8.6.1012-25674.003
X-MDID: 1600764622-U1eaTEk6hGTX
Subject: [dpdk-dev] [PATCH 50/60] common/sfc_efx/base: support UDP tunnel
	operations for EF100
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

From: Igor Romanov <igor.romanov@oktetlabs.ru>

EF100 uses VNIC encapsulation rule MCDI for configuring UDP
tunnels in HW individually. Use busy added and removed states
of UDP tunnel table entries for the implementation.

Signed-off-by: Igor Romanov <igor.romanov@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>
---
 drivers/common/sfc_efx/base/ef10_filter.c  |  10 +-
 drivers/common/sfc_efx/base/efx_impl.h     |   7 +
 drivers/common/sfc_efx/base/efx_mcdi.h     |  11 +
 drivers/common/sfc_efx/base/efx_tunnel.c   |  25 +-
 drivers/common/sfc_efx/base/meson.build    |   1 +
 drivers/common/sfc_efx/base/rhead_impl.h   |  14 +
 drivers/common/sfc_efx/base/rhead_nic.c    |  17 +
 drivers/common/sfc_efx/base/rhead_tunnel.c | 343 +++++++++++++++++++++
 8 files changed, 418 insertions(+), 10 deletions(-)
 create mode 100644 drivers/common/sfc_efx/base/rhead_tunnel.c

diff --git a/drivers/common/sfc_efx/base/ef10_filter.c b/drivers/common/sfc_efx/base/ef10_filter.c
index 51e6f1a210..0e5f04fe3b 100644
--- a/drivers/common/sfc_efx/base/ef10_filter.c
+++ b/drivers/common/sfc_efx/base/ef10_filter.c
@@ -1328,9 +1328,15 @@ ef10_filter_supported_filters(
 		rc = efx_mcdi_get_parser_disp_info(enp, &buffer[next_buf_idx],
 		    next_buf_length, B_TRUE, &mcdi_encap_list_length);
 		if (rc != 0) {
-			if (rc == ENOSPC)
+			if (rc == ENOSPC) {
 				no_space = B_TRUE;
-			else
+			} else if (rc == EINVAL) {
+				/*
+				 * Do not fail if the MCDI do not recognize the
+				 * query for encapsulated packet filters.
+				 */
+				mcdi_encap_list_length = 0;
+			} else
 				goto fail2;
 		} else {
 			for (i = next_buf_idx;
diff --git a/drivers/common/sfc_efx/base/efx_impl.h b/drivers/common/sfc_efx/base/efx_impl.h
index 1ae4eeaf88..be3f3f6bf5 100644
--- a/drivers/common/sfc_efx/base/efx_impl.h
+++ b/drivers/common/sfc_efx/base/efx_impl.h
@@ -496,11 +496,18 @@ typedef enum efx_tunnel_udp_entry_state_e {
 	EFX_TUNNEL_UDP_ENTRY_APPLIED, /* Tunnel is applied by HW */
 } efx_tunnel_udp_entry_state_t;
 
+#if EFSYS_OPT_RIVERHEAD
+typedef uint32_t	efx_vnic_encap_rule_handle_t;
+#endif /* EFSYS_OPT_RIVERHEAD */
+
 typedef struct efx_tunnel_udp_entry_s {
 	uint16_t			etue_port; /* host/cpu-endian */
 	uint16_t			etue_protocol;
 	boolean_t			etue_busy;
 	efx_tunnel_udp_entry_state_t	etue_state;
+#if EFSYS_OPT_RIVERHEAD
+	efx_vnic_encap_rule_handle_t	etue_handle;
+#endif /* EFSYS_OPT_RIVERHEAD */
 } efx_tunnel_udp_entry_t;
 
 typedef struct efx_tunnel_cfg_s {
diff --git a/drivers/common/sfc_efx/base/efx_mcdi.h b/drivers/common/sfc_efx/base/efx_mcdi.h
index 97ac8bf496..77a3d636e2 100644
--- a/drivers/common/sfc_efx/base/efx_mcdi.h
+++ b/drivers/common/sfc_efx/base/efx_mcdi.h
@@ -384,6 +384,17 @@ efx_mcdi_phy_module_get_info(
 		MC_CMD_ ## _field9, _value9,				\
 		MC_CMD_ ## _field10, _value10)
 
+/*
+ * Native setters (MCDI_IN_SET_*_NATIVE) are used when MCDI field is in
+ * network order to avoid conversion to little-endian that is done in
+ * other setters.
+ */
+#define	MCDI_IN_SET_WORD_NATIVE(_emr, _ofst, _value)			\
+	MCDI_IN2((_emr), efx_word_t, _ofst)->ew_u16[0] = (_value)
+
+#define	MCDI_IN_SET_DWORD_NATIVE(_emr, _ofst, _value)			\
+	MCDI_IN2((_emr), efx_dword_t, _ofst)->ed_u32[0] = (_value)
+
 #define	MCDI_OUT(_emr, _type, _ofst)					\
 	((_type *)((_emr).emr_out_buf + (_ofst)))
 
diff --git a/drivers/common/sfc_efx/base/efx_tunnel.c b/drivers/common/sfc_efx/base/efx_tunnel.c
index 204871e00d..02b83d97ed 100644
--- a/drivers/common/sfc_efx/base/efx_tunnel.c
+++ b/drivers/common/sfc_efx/base/efx_tunnel.c
@@ -47,13 +47,6 @@
 
 #if EFSYS_OPT_TUNNEL
 
-#if EFSYS_OPT_SIENA || EFSYS_OPT_HUNTINGTON || EFSYS_OPT_RIVERHEAD
-static const efx_tunnel_ops_t	__efx_tunnel_dummy_ops = {
-	NULL,	/* eto_reconfigure */
-	NULL,	/* eto_fini */
-};
-#endif /* EFSYS_OPT_SIENA || EFSYS_OPT_HUNTINGTON || EFSYS_OPT_RIVERHEAD */
-
 #if EFSYS_OPT_MEDFORD || EFSYS_OPT_MEDFORD2
 static	__checkReturn	boolean_t
 ef10_udp_encap_supported(
@@ -66,13 +59,29 @@ ef10_tunnel_reconfigure(
 static			void
 ef10_tunnel_fini(
 	__in		efx_nic_t *enp);
+#endif /* EFSYS_OPT_MEDFORD || EFSYS_OPT_MEDFORD2 */
 
+#if EFSYS_OPT_SIENA || EFSYS_OPT_HUNTINGTON
+static const efx_tunnel_ops_t	__efx_tunnel_dummy_ops = {
+	NULL,	/* eto_reconfigure */
+	NULL,	/* eto_fini */
+};
+#endif /* EFSYS_OPT_SIENA || EFSYS_OPT_HUNTINGTON */
+
+#if EFSYS_OPT_MEDFORD || EFSYS_OPT_MEDFORD2
 static const efx_tunnel_ops_t	__efx_tunnel_ef10_ops = {
 	ef10_tunnel_reconfigure,	/* eto_reconfigure */
 	ef10_tunnel_fini,		/* eto_fini */
 };
 #endif /* EFSYS_OPT_MEDFORD || EFSYS_OPT_MEDFORD2 */
 
+#if EFSYS_OPT_RIVERHEAD
+static const efx_tunnel_ops_t	__efx_tunnel_rhead_ops = {
+	rhead_tunnel_reconfigure,	/* eto_reconfigure */
+	rhead_tunnel_fini,		/* eto_fini */
+};
+#endif /* EFSYS_OPT_RIVERHEAD */
+
 /* Indicates that an entry is to be set */
 static	__checkReturn		boolean_t
 ef10_entry_staged(
@@ -241,7 +250,7 @@ efx_tunnel_init(
 
 #if EFSYS_OPT_RIVERHEAD
 	case EFX_FAMILY_RIVERHEAD:
-		etop = &__efx_tunnel_dummy_ops;
+		etop = &__efx_tunnel_rhead_ops;
 		break;
 #endif /* EFSYS_OPT_RIVERHEAD */
 
diff --git a/drivers/common/sfc_efx/base/meson.build b/drivers/common/sfc_efx/base/meson.build
index 21feb36c73..e0acbda993 100644
--- a/drivers/common/sfc_efx/base/meson.build
+++ b/drivers/common/sfc_efx/base/meson.build
@@ -58,6 +58,7 @@ sources = [
 	'rhead_nic.c',
 	'rhead_pci.c',
 	'rhead_rx.c',
+	'rhead_tunnel.c',
 	'rhead_tx.c',
 ]
 
diff --git a/drivers/common/sfc_efx/base/rhead_impl.h b/drivers/common/sfc_efx/base/rhead_impl.h
index c75483926c..38cd30d388 100644
--- a/drivers/common/sfc_efx/base/rhead_impl.h
+++ b/drivers/common/sfc_efx/base/rhead_impl.h
@@ -437,6 +437,20 @@ rhead_tx_qstats_update(
 
 #endif /* EFSYS_OPT_QSTATS */
 
+#if EFSYS_OPT_TUNNEL
+
+LIBEFX_INTERNAL
+extern	__checkReturn	efx_rc_t
+rhead_tunnel_reconfigure(
+	__in		efx_nic_t *enp);
+
+LIBEFX_INTERNAL
+extern			void
+rhead_tunnel_fini(
+	__in		efx_nic_t *enp);
+
+#endif /* EFSYS_OPT_TUNNEL */
+
 #if EFSYS_OPT_PCI
 
 /*
diff --git a/drivers/common/sfc_efx/base/rhead_nic.c b/drivers/common/sfc_efx/base/rhead_nic.c
index 58838ab5c7..7d7073ad5e 100644
--- a/drivers/common/sfc_efx/base/rhead_nic.c
+++ b/drivers/common/sfc_efx/base/rhead_nic.c
@@ -22,6 +22,23 @@ rhead_board_cfg(
 	if ((rc = efx_mcdi_nic_board_cfg(enp)) != 0)
 		goto fail1;
 
+	/*
+	 * The tunnel encapsulation initialization happens unconditionally
+	 * for now.
+	 */
+	encp->enc_tunnel_encapsulations_supported =
+	    (1u << EFX_TUNNEL_PROTOCOL_VXLAN) |
+	    (1u << EFX_TUNNEL_PROTOCOL_GENEVE) |
+	    (1u << EFX_TUNNEL_PROTOCOL_NVGRE);
+
+	/*
+	 * Software limitation inherited from EF10. This limit is not
+	 * increased since the hardware does not report this limit, it is
+	 * handled internally resulting in a tunnel add error when there is no
+	 * space for more UDP tunnels.
+	 */
+	encp->enc_tunnel_config_udp_entries_max = EFX_TUNNEL_MAXNENTRIES;
+
 	encp->enc_clk_mult = 1; /* not used for Riverhead */
 
 	/*
diff --git a/drivers/common/sfc_efx/base/rhead_tunnel.c b/drivers/common/sfc_efx/base/rhead_tunnel.c
new file mode 100644
index 0000000000..92f12ab4e8
--- /dev/null
+++ b/drivers/common/sfc_efx/base/rhead_tunnel.c
@@ -0,0 +1,343 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ *
+ * Copyright(c) 2020 Xilinx, Inc.
+ */
+
+#include "efx.h"
+#include "efx_impl.h"
+
+#if EFSYS_OPT_RIVERHEAD && EFSYS_OPT_TUNNEL
+
+/* Match by Ether-type */
+#define	EFX_VNIC_ENCAP_RULE_MATCH_ETHER_TYPE \
+	(1u << MC_CMD_VNIC_ENCAP_RULE_ADD_IN_MATCH_ETHER_TYPE_LBN)
+/* Match by outer VLAN ID */
+#define	EFX_VNIC_ENCAP_RULE_MATCH_OUTER_VID \
+	(1u << MC_CMD_VNIC_ENCAP_RULE_ADD_IN_MATCH_OUTER_VLAN_LBN)
+/* Match by local IP host address */
+#define	EFX_VNIC_ENCAP_RULE_MATCH_LOC_HOST \
+	(1u << MC_CMD_VNIC_ENCAP_RULE_ADD_IN_MATCH_DST_IP_LBN)
+/* Match by IP transport protocol */
+#define	EFX_VNIC_ENCAP_RULE_MATCH_IP_PROTO \
+	(1u << MC_CMD_VNIC_ENCAP_RULE_ADD_IN_MATCH_IP_PROTO_LBN)
+/* Match by local TCP/UDP port */
+#define	EFX_VNIC_ENCAP_RULE_MATCH_LOC_PORT \
+	(1u << MC_CMD_VNIC_ENCAP_RULE_ADD_IN_MATCH_DST_PORT_LBN)
+
+/*
+ * Helper structure to pass parameters to MCDI function to add a VNIC
+ * encapsulation rule.
+ */
+typedef struct efx_vnic_encap_rule_spec_s {
+	uint32_t		evers_mport_selector; /* Host-endian */
+	uint32_t		evers_match_flags; /* Host-endian */
+	uint16_t		evers_ether_type; /* Host-endian */
+	uint16_t		evers_outer_vid; /* Host-endian */
+	efx_oword_t		evers_loc_host; /* Big-endian */
+	uint8_t			evers_ip_proto;
+	uint16_t		evers_loc_port; /* Host-endian */
+	efx_tunnel_protocol_t	evers_encap_type;
+} efx_vnic_encap_rule_spec_t;
+
+static				uint32_t
+efx_tunnel_protocol2mae_encap_type(
+	__in		efx_tunnel_protocol_t proto,
+	__out		uint32_t *typep)
+{
+	efx_rc_t rc;
+
+	switch (proto) {
+	case EFX_TUNNEL_PROTOCOL_NONE:
+		*typep = MAE_MCDI_ENCAP_TYPE_NONE;
+		break;
+	case EFX_TUNNEL_PROTOCOL_VXLAN:
+		*typep = MAE_MCDI_ENCAP_TYPE_VXLAN;
+		break;
+	case EFX_TUNNEL_PROTOCOL_GENEVE:
+		*typep = MAE_MCDI_ENCAP_TYPE_GENEVE;
+		break;
+	case EFX_TUNNEL_PROTOCOL_NVGRE:
+		*typep = MAE_MCDI_ENCAP_TYPE_NVGRE;
+		break;
+	default:
+		rc = EINVAL;
+		goto fail1;
+	}
+
+	return (0);
+
+fail1:
+	EFSYS_PROBE1(fail1, efx_rc_t, rc);
+
+	return (rc);
+}
+
+static	__checkReturn		efx_rc_t
+efx_mcdi_vnic_encap_rule_add(
+	__in			efx_nic_t *enp,
+	__in			const efx_vnic_encap_rule_spec_t *spec,
+	__out			efx_vnic_encap_rule_handle_t *handle)
+
+{
+	efx_mcdi_req_t req;
+	EFX_MCDI_DECLARE_BUF(payload,
+		MC_CMD_VNIC_ENCAP_RULE_ADD_IN_LEN,
+		MC_CMD_VNIC_ENCAP_RULE_ADD_OUT_LEN);
+	uint32_t encap_type;
+	efx_rc_t rc;
+
+	req.emr_cmd = MC_CMD_VNIC_ENCAP_RULE_ADD;
+	req.emr_in_buf = payload;
+	req.emr_in_length = MC_CMD_VNIC_ENCAP_RULE_ADD_IN_LEN;
+	req.emr_out_buf = payload;
+	req.emr_out_length = MC_CMD_VNIC_ENCAP_RULE_ADD_OUT_LEN;
+
+	MCDI_IN_SET_DWORD(req, VNIC_ENCAP_RULE_ADD_IN_MPORT_SELECTOR,
+	    spec->evers_mport_selector);
+	MCDI_IN_SET_DWORD(req, VNIC_ENCAP_RULE_ADD_IN_MATCH_FLAGS,
+	    spec->evers_match_flags);
+
+	MCDI_IN_SET_WORD_NATIVE(req, VNIC_ENCAP_RULE_ADD_IN_ETHER_TYPE,
+	    __CPU_TO_BE_16(spec->evers_ether_type));
+	MCDI_IN_SET_WORD_NATIVE(req, VNIC_ENCAP_RULE_ADD_IN_OUTER_VLAN_WORD,
+	    __CPU_TO_BE_16(spec->evers_outer_vid));
+
+	/*
+	 * Address is already in network order as well as the MCDI field,
+	 * so plain copy is used.
+	 */
+	EFX_STATIC_ASSERT(sizeof (spec->evers_loc_host) ==
+	    MC_CMD_VNIC_ENCAP_RULE_ADD_IN_DST_IP_LEN);
+	memcpy(MCDI_IN2(req, uint8_t, VNIC_ENCAP_RULE_ADD_IN_DST_IP),
+	    &spec->evers_loc_host.eo_byte[0],
+	    MC_CMD_VNIC_ENCAP_RULE_ADD_IN_DST_IP_LEN);
+
+	MCDI_IN_SET_BYTE(req, VNIC_ENCAP_RULE_ADD_IN_IP_PROTO,
+	    spec->evers_ip_proto);
+	MCDI_IN_SET_WORD_NATIVE(req, VNIC_ENCAP_RULE_ADD_IN_DST_PORT,
+	    __CPU_TO_BE_16(spec->evers_loc_port));
+
+	rc = efx_tunnel_protocol2mae_encap_type(spec->evers_encap_type,
+	    &encap_type);
+	if (rc != 0)
+		goto fail1;
+
+	MCDI_IN_SET_DWORD(req, VNIC_ENCAP_RULE_ADD_IN_ENCAP_TYPE, encap_type);
+
+	efx_mcdi_execute(enp, &req);
+
+	if (req.emr_rc != 0) {
+		rc = req.emr_rc;
+		goto fail2;
+	}
+
+	if (req.emr_out_length_used != MC_CMD_VNIC_ENCAP_RULE_ADD_OUT_LEN) {
+		rc = EMSGSIZE;
+		goto fail3;
+	}
+
+	if (handle != NULL)
+		*handle = MCDI_OUT_DWORD(req, VNIC_ENCAP_RULE_ADD_OUT_HANDLE);
+
+	return (0);
+
+fail3:
+	EFSYS_PROBE(fail3);
+
+fail2:
+	EFSYS_PROBE(fail2);
+
+fail1:
+	EFSYS_PROBE1(fail1, efx_rc_t, rc);
+
+	return (rc);
+}
+
+static	__checkReturn		efx_rc_t
+efx_mcdi_vnic_encap_rule_remove(
+       __in                    efx_nic_t *enp,
+       __in                    efx_vnic_encap_rule_handle_t handle)
+
+{
+       efx_mcdi_req_t req;
+       EFX_MCDI_DECLARE_BUF(payload,
+               MC_CMD_VNIC_ENCAP_RULE_REMOVE_IN_LEN,
+               MC_CMD_VNIC_ENCAP_RULE_REMOVE_OUT_LEN);
+       efx_rc_t rc;
+
+       req.emr_cmd = MC_CMD_VNIC_ENCAP_RULE_REMOVE;
+       req.emr_in_buf = payload;
+       req.emr_in_length = MC_CMD_VNIC_ENCAP_RULE_REMOVE_IN_LEN;
+       req.emr_out_buf = payload;
+       req.emr_out_length = MC_CMD_VNIC_ENCAP_RULE_REMOVE_OUT_LEN;
+
+       MCDI_IN_SET_DWORD(req, VNIC_ENCAP_RULE_REMOVE_IN_HANDLE, handle);
+
+       efx_mcdi_execute(enp, &req);
+
+       if (req.emr_rc != 0) {
+               rc = req.emr_rc;
+               goto fail1;
+       }
+
+       if (req.emr_out_length_used != MC_CMD_VNIC_ENCAP_RULE_REMOVE_OUT_LEN) {
+               rc = EMSGSIZE;
+               goto fail2;
+       }
+
+       return (0);
+
+fail2:
+       EFSYS_PROBE(fail2);
+
+fail1:
+       EFSYS_PROBE1(fail1, efx_rc_t, rc);
+
+       return (rc);
+}
+
+static			void
+rhead_vnic_encap_rule_spec_init(
+	__in		const efx_tunnel_udp_entry_t *etuep,
+	__out		efx_vnic_encap_rule_spec_t *spec)
+{
+	memset(spec, 0, sizeof (*spec));
+
+	spec->evers_mport_selector = MAE_MPORT_SELECTOR_ASSIGNED;
+	spec->evers_match_flags = EFX_VNIC_ENCAP_RULE_MATCH_IP_PROTO |
+	    EFX_VNIC_ENCAP_RULE_MATCH_LOC_PORT;
+	spec->evers_ip_proto = EFX_IPPROTO_UDP;
+	spec->evers_loc_port = etuep->etue_port;
+	spec->evers_encap_type = etuep->etue_protocol;
+}
+
+static	__checkReturn	efx_rc_t
+rhead_udp_port_tunnel_add(
+	__in		efx_nic_t *enp,
+	__inout		efx_tunnel_udp_entry_t *etuep)
+{
+	efx_vnic_encap_rule_spec_t spec;
+
+	rhead_vnic_encap_rule_spec_init(etuep, &spec);
+	return (efx_mcdi_vnic_encap_rule_add(enp, &spec, &etuep->etue_handle));
+}
+
+static	__checkReturn	efx_rc_t
+rhead_udp_port_tunnel_remove(
+	__in		efx_nic_t *enp,
+	__in		efx_tunnel_udp_entry_t *etuep)
+{
+	return (efx_mcdi_vnic_encap_rule_remove(enp, etuep->etue_handle));
+}
+
+	__checkReturn	efx_rc_t
+rhead_tunnel_reconfigure(
+	__in		efx_nic_t *enp)
+{
+	efx_tunnel_cfg_t *etcp = &enp->en_tunnel_cfg;
+	efx_rc_t rc;
+	efsys_lock_state_t state;
+	efx_tunnel_cfg_t etc;
+	efx_tunnel_cfg_t added;
+	unsigned int i;
+	unsigned int j;
+
+	memset(&added, 0, sizeof(added));
+
+	/*
+	 * Make a local copy of UDP tunnel table to release the lock
+	 * when executing MCDIs.
+	 */
+	EFSYS_LOCK(enp->en_eslp, state);
+	memcpy(&etc, etcp, sizeof (etc));
+	EFSYS_UNLOCK(enp->en_eslp, state);
+
+	for (i = 0; i < etc.etc_udp_entries_num; i++) {
+		efx_tunnel_udp_entry_t *etc_entry = &etc.etc_udp_entries[i];
+
+		if (etc_entry->etue_busy == B_FALSE)
+			continue;
+
+		switch (etc_entry->etue_state) {
+		case EFX_TUNNEL_UDP_ENTRY_APPLIED:
+			break;
+		case EFX_TUNNEL_UDP_ENTRY_ADDED:
+			rc = rhead_udp_port_tunnel_add(enp, etc_entry);
+			if (rc != 0)
+				goto fail1;
+			added.etc_udp_entries[added.etc_udp_entries_num] =
+			    *etc_entry;
+			added.etc_udp_entries_num++;
+			break;
+		case EFX_TUNNEL_UDP_ENTRY_REMOVED:
+			rc = rhead_udp_port_tunnel_remove(enp, etc_entry);
+			if (rc != 0)
+				goto fail2;
+			break;
+		default:
+			EFSYS_ASSERT(0);
+			break;
+		}
+	}
+
+	EFSYS_LOCK(enp->en_eslp, state);
+
+	/*
+	 * Adding or removing non-busy entries does not change the
+	 * order of busy entries. Therefore one linear search iteration
+	 * suffices.
+	 */
+	for (i = 0, j = 0; i < etcp->etc_udp_entries_num; i++) {
+		efx_tunnel_udp_entry_t *cur_entry = &etcp->etc_udp_entries[i];
+		efx_tunnel_udp_entry_t *added_entry = &added.etc_udp_entries[j];
+
+		if (cur_entry->etue_state == EFX_TUNNEL_UDP_ENTRY_ADDED &&
+		    cur_entry->etue_port == added_entry->etue_port) {
+			cur_entry->etue_handle = added_entry->etue_handle;
+			j++;
+		}
+	}
+
+	EFSYS_UNLOCK(enp->en_eslp, state);
+
+	return (0);
+
+fail2:
+	EFSYS_PROBE(fail2);
+
+fail1:
+	EFSYS_PROBE1(fail1, efx_rc_t, rc);
+
+	while (i-- > 0) {
+		if (etc.etc_udp_entries[i].etue_busy == B_FALSE)
+			continue;
+
+		switch (etc.etc_udp_entries[i].etue_state) {
+		case EFX_TUNNEL_UDP_ENTRY_APPLIED:
+			break;
+		case EFX_TUNNEL_UDP_ENTRY_ADDED:
+			(void) rhead_udp_port_tunnel_remove(enp,
+					&etc.etc_udp_entries[i]);
+			break;
+		case EFX_TUNNEL_UDP_ENTRY_REMOVED:
+			(void) rhead_udp_port_tunnel_add(enp,
+					&etc.etc_udp_entries[i]);
+			break;
+		default:
+			EFSYS_ASSERT(0);
+			break;
+		}
+	}
+
+	return (rc);
+}
+
+			void
+rhead_tunnel_fini(
+	__in		efx_nic_t *enp)
+{
+	(void) efx_tunnel_config_clear(enp);
+	(void) efx_tunnel_reconfigure(enp);
+}
+
+#endif	/* EFSYS_OPT_RIVERHEAD && EFSYS_OPT_TUNNEL */
-- 
2.17.1