From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 8EBBA1B3B5 for ; Thu, 7 Feb 2019 14:28:39 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E864AC0C0551; Thu, 7 Feb 2019 13:28:38 +0000 (UTC) Received: from ktraynor.remote.csb (unknown [10.33.36.135]) by smtp.corp.redhat.com (Postfix) with ESMTP id E3E3D600C4; Thu, 7 Feb 2019 13:28:37 +0000 (UTC) From: Kevin Traynor To: Viacheslav Ovsiienko Cc: Shahaf Shuler , dpdk stable Date: Thu, 7 Feb 2019 13:26:04 +0000 Message-Id: <20190207132614.20538-58-ktraynor@redhat.com> In-Reply-To: <20190207132614.20538-1-ktraynor@redhat.com> References: <20190207132614.20538-1-ktraynor@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 07 Feb 2019 13:28:39 +0000 (UTC) Subject: [dpdk-stable] patch 'net/mlx5: fix VXLAN port registration race condition' has been queued to LTS release 18.11.1 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Feb 2019 13:28:40 -0000 Hi, FYI, your patch has been queued to LTS release 18.11.1 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 02/14/19. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Thanks. Kevin Traynor --- >>From e207f69599507de39b70b58bad472acf9487c6c4 Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Wed, 23 Jan 2019 07:51:18 +0000 Subject: [PATCH] net/mlx5: fix VXLAN port registration race condition [ upstream commit 71ab2d64723c8b7192eea01877012de7aa5c2476 ] E-Switch VXLAN tunneling rules require virtual VXLAN network devices be created. These devices are managed by MLX5 PMD and created/deleted dynamically. Kernel creates the VXLAN devices and registers VXLAN UDP ports to be hardware offloaded within the NIC kernel drivers. The registration process is being performed into context of working kernel thread and the race conditions might happen. The VXLAN device is created and success code is returned to calling application, but the UDP port registration process is not completed yet and the next applied rule might be rejected by the driver with ENOSUP code. This patch adds some timeout for new created devices, allowing port registration process to be completed. The waiting is performed once after device been created and first rule is being applied. Fixes: 95a464cecc21 ("net/mlx5: add E-switch VXLAN tunnel devices management") Signed-off-by: Viacheslav Ovsiienko Acked-by: Shahaf Shuler --- drivers/net/mlx5/mlx5_flow_tcf.c | 49 ++++++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_tcf.c b/drivers/net/mlx5/mlx5_flow_tcf.c index 94421c5a1..e677404cf 100644 --- a/drivers/net/mlx5/mlx5_flow_tcf.c +++ b/drivers/net/mlx5/mlx5_flow_tcf.c @@ -321,4 +321,9 @@ struct tc_tunnel_key { #define MLX5_VXLAN_PORT_MAX 60000 #define MLX5_VXLAN_DEVICE_PFX "vmlx_" +/** + * Timeout in milliseconds to wait VXLAN UDP offloaded port + * registration completed within the mlx5 driver. + */ +#define MLX5_VXLAN_WAIT_PORT_REG_MS 250 /** Tunnel action type, used for @p type in header structure. */ @@ -404,5 +409,6 @@ struct tcf_vtep { unsigned int ifouter; /**< Index of device attached to. */ uint16_t port; - uint8_t created; + uint32_t created:1; /**< Actually created by PMD. */ + uint32_t waitreg:1; /**< Wait for VXLAN UDP port registration. */ }; @@ -4889,4 +4895,5 @@ flow_tcf_vtep_create(struct mlx5_flow_tcf_context *tcf, */ vtep->created = 1; + vtep->waitreg = 1; } /* Try to get ifindex of created of pre-existing device. */ @@ -5404,4 +5411,6 @@ flow_tcf_apply(struct rte_eth_dev *dev, struct rte_flow *flow, struct nlmsghdr *nlh; struct tcmsg *tcm; + uint64_t start = 0; + uint64_t twait = 0; int ret; @@ -5437,6 +5446,42 @@ flow_tcf_apply(struct rte_eth_dev *dev, struct rte_flow *flow, *dev_flow->tcf.tunnel->ifindex_ptr = dev_flow->tcf.tunnel->vtep->ifindex; + if (dev_flow->tcf.tunnel->vtep->waitreg) { + /* Clear wait flag for VXLAN port registration. */ + dev_flow->tcf.tunnel->vtep->waitreg = 0; + twait = rte_get_timer_hz(); + assert(twait > MS_PER_S); + twait = twait * MLX5_VXLAN_WAIT_PORT_REG_MS; + twait = twait / MS_PER_S; + start = rte_get_timer_cycles(); + } } - ret = flow_tcf_nl_ack(ctx, nlh, flow_tcf_collect_apply_cb, nlh); + /* + * Kernel creates the VXLAN devices and registers UDP ports to + * be hardware offloaded within the NIC kernel drivers. The + * registration process is being performed into context of + * working kernel thread and the race conditions might happen. + * The VXLAN device is created and success is returned to + * calling application, but the UDP port registration process + * is not completed yet. The next applied rule may be rejected + * by the driver with ENOSUP code. We are going to wait a bit, + * allowing registration process to be completed. The waiting + * is performed once after device been created. + */ + do { + struct timespec onems; + + ret = flow_tcf_nl_ack(ctx, nlh, + flow_tcf_collect_apply_cb, nlh); + if (!ret || ret != -ENOTSUP || !twait) + break; + /* Wait one millisecond and try again till timeout. */ + onems.tv_sec = 0; + onems.tv_nsec = NS_PER_S / MS_PER_S; + nanosleep(&onems, 0); + if ((rte_get_timer_cycles() - start) > twait) { + /* Timeout elapsed, try once more and exit. */ + twait = 0; + } + } while (true); if (!ret) { if (!tcm->tcm_handle) { -- 2.19.0 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2019-02-07 13:19:56.949187144 +0000 +++ 0058-net-mlx5-fix-VXLAN-port-registration-race-condition.patch 2019-02-07 13:19:55.000000000 +0000 @@ -1,8 +1,10 @@ -From 71ab2d64723c8b7192eea01877012de7aa5c2476 Mon Sep 17 00:00:00 2001 +From e207f69599507de39b70b58bad472acf9487c6c4 Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Wed, 23 Jan 2019 07:51:18 +0000 Subject: [PATCH] net/mlx5: fix VXLAN port registration race condition +[ upstream commit 71ab2d64723c8b7192eea01877012de7aa5c2476 ] + E-Switch VXLAN tunneling rules require virtual VXLAN network devices be created. These devices are managed by MLX5 PMD and created/deleted dynamically. @@ -21,7 +23,6 @@ applied. Fixes: 95a464cecc21 ("net/mlx5: add E-switch VXLAN tunnel devices management") -Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko Acked-by: Shahaf Shuler @@ -30,11 +31,11 @@ 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/drivers/net/mlx5/mlx5_flow_tcf.c b/drivers/net/mlx5/mlx5_flow_tcf.c -index 859e99a9a..b8204df63 100644 +index 94421c5a1..e677404cf 100644 --- a/drivers/net/mlx5/mlx5_flow_tcf.c +++ b/drivers/net/mlx5/mlx5_flow_tcf.c -@@ -355,4 +355,9 @@ struct tc_tunnel_key { - #define MLX5_VXLAN_DEFAULT_VNI 1 +@@ -321,4 +321,9 @@ struct tc_tunnel_key { + #define MLX5_VXLAN_PORT_MAX 60000 #define MLX5_VXLAN_DEVICE_PFX "vmlx_" +/** + * Timeout in milliseconds to wait VXLAN UDP offloaded port @@ -43,28 +44,28 @@ +#define MLX5_VXLAN_WAIT_PORT_REG_MS 250 /** Tunnel action type, used for @p type in header structure. */ -@@ -446,5 +451,6 @@ struct tcf_vtep { - unsigned int ifindex; /**< Own interface index. */ +@@ -404,5 +409,6 @@ struct tcf_vtep { + unsigned int ifouter; /**< Index of device attached to. */ uint16_t port; - uint8_t created; + uint32_t created:1; /**< Actually created by PMD. */ + uint32_t waitreg:1; /**< Wait for VXLAN UDP port registration. */ }; -@@ -5168,4 +5174,5 @@ flow_tcf_vtep_create(struct mlx5_flow_tcf_context *tcf, +@@ -4889,4 +4895,5 @@ flow_tcf_vtep_create(struct mlx5_flow_tcf_context *tcf, */ vtep->created = 1; + vtep->waitreg = 1; } /* Try to get ifindex of created of pre-existing device. */ -@@ -5649,4 +5656,6 @@ flow_tcf_apply(struct rte_eth_dev *dev, struct rte_flow *flow, +@@ -5404,4 +5411,6 @@ flow_tcf_apply(struct rte_eth_dev *dev, struct rte_flow *flow, struct nlmsghdr *nlh; struct tcmsg *tcm; + uint64_t start = 0; + uint64_t twait = 0; int ret; -@@ -5682,6 +5691,42 @@ flow_tcf_apply(struct rte_eth_dev *dev, struct rte_flow *flow, +@@ -5437,6 +5446,42 @@ flow_tcf_apply(struct rte_eth_dev *dev, struct rte_flow *flow, *dev_flow->tcf.tunnel->ifindex_ptr = dev_flow->tcf.tunnel->vtep->ifindex; + if (dev_flow->tcf.tunnel->vtep->waitreg) {