From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C327A0C4C for ; Mon, 12 Jul 2021 15:15:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 767EC40685; Mon, 12 Jul 2021 15:15:31 +0200 (CEST) Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mails.dpdk.org (Postfix) with ESMTP id 27D854069E for ; Mon, 12 Jul 2021 15:15:30 +0200 (CEST) Received: by mail-wr1-f47.google.com with SMTP id d12so24938989wre.13 for ; Mon, 12 Jul 2021 06:15:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9UiWPX29vCVR6Utkyr4z707teOCx1W1gPqVJlVld7Qg=; b=egUG1Ty1Qb569ABdpWU9jClDke8CX3xsThBuksg5NkKOOdk4tD0nkuisRxlW6pXI+t 2uA4Gzb5SS0vJ1YSXJneFHE+Lyd5ht+1Tta/LjxLJ1wB3snNXjQj0Une2wLCqVW/PalI UaN9/HyTmaFLQO4aF2u4i+T9ZLR9a45yA57JXBw180BZRAUDDtPy3sjGwfhFROCbc4Mr gTNUxNj2IGbd48VV2Wt3G5eCkKgxzGf8Omk19HjuGIwsJ77b1Iwn/73l0jYB04wGadHa 6ZUJqY3rYHgcc2ozJ7LxCrl2jRY3m/PQG2ZY8yf4eECgc1SN5b6uNZ83q28p7sqgd8c3 Apkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9UiWPX29vCVR6Utkyr4z707teOCx1W1gPqVJlVld7Qg=; b=juvSOZjvGXbFiE3iLDgxEXrIzKPDdg/FrR7z1bkLPEihZAPBeLHMQ9FX3Vq4uYdmwi X2jpeW4O2e9ovlOgFXEqYskm5J0ODA5EmNVeLeuzX8hSbeOv1IuNTUajUijpBJ3Qzq31 qR6IWUUUB/xq8BkIXhcE8TAXAaM1/DNRsLgO0q+T8V0p0ji/sYUEx1cRTtmsZUNAomjx 7GbJdsM1pp0GzldDGZtXIEuKKKTjWlewLambAHRYrvXduObcST1fKoeht/jW4rYx7V9a ZdP9PAqMMzYw6WCJZ5PiosLUF+MkyQgHqPeVuC2P2nWxCvmNXDeHCZN/8PZw7EhKYUhJ kNIw== X-Gm-Message-State: AOAM530hwKKrxfeLzHrdQ2QtMf9RiWyXz/4aw4Z0lReEShVfgi1vmqso oQybxmjvhsRcLsx+uJCpz0g= X-Google-Smtp-Source: ABdhPJyQkZXws4XJIOCsYHiKSAX15UggS0ZxUc/P3h+zUC2eestpgxjBJWcFaHMMVq1hm4ELcDsohw== X-Received: by 2002:a05:6000:1375:: with SMTP id q21mr26736534wrz.147.1626095729876; Mon, 12 Jul 2021 06:15:29 -0700 (PDT) Received: from localhost ([137.220.125.106]) by smtp.gmail.com with ESMTPSA id 12sm21012587wme.28.2021.07.12.06.15.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jul 2021 06:15:29 -0700 (PDT) From: luca.boccassi@gmail.com To: Viacheslav Ovsiienko Cc: Matan Azrad , dpdk stable Date: Mon, 12 Jul 2021 14:05:37 +0100 Message-Id: <20210712130551.2462159-102-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210712130551.2462159-1-luca.boccassi@gmail.com> References: <20210712130551.2462159-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'common/mlx5: use new port query API if available' has been queued to stable release 20.11.3 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 20.11.3 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 07/14/21. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/245c98f97ef6763ea75ae847b76f0b5fdb68fbb6 Thanks. Luca Boccassi --- >From 245c98f97ef6763ea75ae847b76f0b5fdb68fbb6 Mon Sep 17 00:00:00 2001 From: Viacheslav Ovsiienko Date: Wed, 7 Jul 2021 18:54:27 +0300 Subject: [PATCH] common/mlx5: use new port query API if available [ upstream commit d0cf77e8c2b64319057f5f629a7a595ce6e8b556 ] In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Signed-off-by: Viacheslav Ovsiienko Acked-by: Matan Azrad --- drivers/common/mlx5/linux/meson.build | 2 + drivers/common/mlx5/linux/mlx5_glue.c | 53 +++++++++++++++++++---- drivers/common/mlx5/linux/mlx5_glue.h | 16 ++++++- drivers/net/mlx5/linux/mlx5_os.c | 60 ++++++++++++--------------- 4 files changed, 88 insertions(+), 43 deletions(-) diff --git a/drivers/common/mlx5/linux/meson.build b/drivers/common/mlx5/linux/meson.build index fa9686fdaf..8295cfbe41 100644 --- a/drivers/common/mlx5/linux/meson.build +++ b/drivers/common/mlx5/linux/meson.build @@ -94,6 +94,8 @@ has_sym_args = [ 'IBV_WQ_FLAG_RX_END_PADDING' ], [ 'HAVE_MLX5DV_DR_DEVX_PORT', 'infiniband/mlx5dv.h', 'mlx5dv_query_devx_port' ], + [ 'HAVE_MLX5DV_DR_DEVX_PORT_V35', 'infiniband/mlx5dv.h', + 'mlx5dv_query_port' ], [ 'HAVE_IBV_DEVX_OBJ', 'infiniband/mlx5dv.h', 'mlx5dv_devx_obj_create' ], [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h', diff --git a/drivers/common/mlx5/linux/mlx5_glue.c b/drivers/common/mlx5/linux/mlx5_glue.c index 964f7e74ef..e24bcdfbd1 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.c +++ b/drivers/common/mlx5/linux/mlx5_glue.c @@ -1087,17 +1087,54 @@ mlx5_glue_devx_wq_query(struct ibv_wq *wq, const void *in, size_t inlen, static int mlx5_glue_devx_port_query(struct ibv_context *ctx, uint32_t port_num, - struct mlx5dv_devx_port *mlx5_devx_port) + struct mlx5_port_info *info) { + int err = 0; + + info->query_flags = 0; +#ifdef HAVE_MLX5DV_DR_DEVX_PORT_V35 + /* The DevX port query API is implemented (rdma-core v35 and above). */ + struct mlx5_ib_uapi_query_port devx_port; + + memset(&devx_port, 0, sizeof(devx_port)); + err = mlx5dv_query_port(ctx, port_num, &devx_port); + if (err) + return err; + if (devx_port.flags & MLX5DV_QUERY_PORT_VPORT_REG_C0) { + info->vport_meta_tag = devx_port.reg_c0.value; + info->vport_meta_mask = devx_port.reg_c0.mask; + info->query_flags |= MLX5_PORT_QUERY_REG_C0; + } + if (devx_port.flags & MLX5DV_QUERY_PORT_VPORT) { + info->vport_id = devx_port.vport; + info->query_flags |= MLX5_PORT_QUERY_VPORT; + } +#else #ifdef HAVE_MLX5DV_DR_DEVX_PORT - return mlx5dv_query_devx_port(ctx, port_num, mlx5_devx_port); + /* The legacy DevX port query API is implemented (prior v35). */ + struct mlx5dv_devx_port devx_port = { + .comp_mask = MLX5DV_DEVX_PORT_VPORT | + MLX5DV_DEVX_PORT_MATCH_REG_C_0 + }; + + err = mlx5dv_query_devx_port(ctx, port_num, &devx_port); + if (err) + return err; + if (devx_port.comp_mask & MLX5DV_DEVX_PORT_MATCH_REG_C_0) { + info->vport_meta_tag = devx_port.reg_c_0.value; + info->vport_meta_mask = devx_port.reg_c_0.mask; + info->query_flags |= MLX5_PORT_QUERY_REG_C0; + } + if (devx_port.comp_mask & MLX5DV_DEVX_PORT_VPORT) { + info->vport_id = devx_port.vport_num; + info->query_flags |= MLX5_PORT_QUERY_VPORT; + } #else - (void)ctx; - (void)port_num; - (void)mlx5_devx_port; - errno = ENOTSUP; - return errno; -#endif + RTE_SET_USED(ctx); + RTE_SET_USED(port_num); +#endif /* HAVE_MLX5DV_DR_DEVX_PORT */ +#endif /* HAVE_MLX5DV_DR_DEVX_PORT_V35 */ + return err; } static int diff --git a/drivers/common/mlx5/linux/mlx5_glue.h b/drivers/common/mlx5/linux/mlx5_glue.h index 9e385be957..f08c83768c 100644 --- a/drivers/common/mlx5/linux/mlx5_glue.h +++ b/drivers/common/mlx5/linux/mlx5_glue.h @@ -84,6 +84,20 @@ struct mlx5dv_dr_action; struct mlx5dv_devx_port; #endif +#ifndef HAVE_MLX5DV_DR_DEVX_PORT_V35 +struct mlx5dv_port; +#endif + +#define MLX5_PORT_QUERY_VPORT (1u << 0) +#define MLX5_PORT_QUERY_REG_C0 (1u << 1) + +struct mlx5_port_info { + uint16_t query_flags; + uint16_t vport_id; /* Associated VF vport index (if any). */ + uint32_t vport_meta_tag; /* Used for vport index match ove VF LAG. */ + uint32_t vport_meta_mask; /* Used for vport index field match mask. */ +}; + #ifndef HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER struct mlx5dv_dr_flow_meter_attr; #endif @@ -311,7 +325,7 @@ struct mlx5_glue { void *out, size_t outlen); int (*devx_port_query)(struct ibv_context *ctx, uint32_t port_num, - struct mlx5dv_devx_port *mlx5_devx_port); + struct mlx5_port_info *info); int (*dr_dump_domain)(FILE *file, void *domain); int (*devx_query_eqn)(struct ibv_context *context, uint32_t cpus, uint32_t *eqn); diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 71837e34fe..56e2bf64f4 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -695,9 +695,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, int own_domain_id = 0; uint16_t port_id; unsigned int i; -#ifdef HAVE_MLX5DV_DR_DEVX_PORT - struct mlx5dv_devx_port devx_port = { .comp_mask = 0 }; -#endif + struct mlx5_port_info vport_info = { .query_flags = 0 }; /* Determine if this port representor is supposed to be spawned. */ if (switch_info->representor && dpdk_dev->devargs) { @@ -940,29 +938,27 @@ err_secondary: priv->vport_meta_tag = 0; priv->vport_meta_mask = 0; priv->pf_bond = spawn->pf_bond; -#ifdef HAVE_MLX5DV_DR_DEVX_PORT /* - * The DevX port query API is implemented. E-Switch may use - * either vport or reg_c[0] metadata register to match on - * vport index. The engaged part of metadata register is - * defined by mask. + * If we have E-Switch we should determine the vport attributes. + * E-Switch may use either source vport field or reg_c[0] metadata + * register to match on vport index. The engaged part of metadata + * register is defined by mask. */ if (switch_info->representor || switch_info->master) { - devx_port.comp_mask = MLX5DV_DEVX_PORT_VPORT | - MLX5DV_DEVX_PORT_MATCH_REG_C_0; - err = mlx5_glue->devx_port_query(sh->ctx, spawn->phys_port, - &devx_port); + err = mlx5_glue->devx_port_query(sh->ctx, + spawn->phys_port, + &vport_info); if (err) { DRV_LOG(WARNING, "can't query devx port %d on device %s", spawn->phys_port, mlx5_os_get_dev_device_name(spawn->phys_dev)); - devx_port.comp_mask = 0; + vport_info.query_flags = 0; } } - if (devx_port.comp_mask & MLX5DV_DEVX_PORT_MATCH_REG_C_0) { - priv->vport_meta_tag = devx_port.reg_c_0.value; - priv->vport_meta_mask = devx_port.reg_c_0.mask; + if (vport_info.query_flags & MLX5_PORT_QUERY_REG_C0) { + priv->vport_meta_tag = vport_info.vport_meta_tag; + priv->vport_meta_mask = vport_info.vport_meta_mask; if (!priv->vport_meta_mask) { DRV_LOG(ERR, "vport zero mask for port %d" " on bonding device %s", @@ -982,8 +978,8 @@ err_secondary: goto error; } } - if (devx_port.comp_mask & MLX5DV_DEVX_PORT_VPORT) { - priv->vport_id = devx_port.vport_num; + if (vport_info.query_flags & MLX5_PORT_QUERY_VPORT) { + priv->vport_id = vport_info.vport_id; } else if (spawn->pf_bond >= 0 && (switch_info->representor || switch_info->master)) { DRV_LOG(ERR, "can't deduce vport index for port %d" @@ -993,25 +989,21 @@ err_secondary: err = ENOTSUP; goto error; } else { - /* Suppose vport index in compatible way. */ + /* + * Suppose vport index in compatible way. Kernel/rdma_core + * support single E-Switch per PF configurations only and + * vport_id field contains the vport index for associated VF, + * which is deduced from representor port name. + * For example, let's have the IB device port 10, it has + * attached network device eth0, which has port name attribute + * pf0vf2, we can deduce the VF number as 2, and set vport index + * as 3 (2+1). This assigning schema should be changed if the + * multiple E-Switch instances per PF configurations or/and PCI + * subfunctions are added. + */ priv->vport_id = switch_info->representor ? switch_info->port_name + 1 : -1; } -#else - /* - * Kernel/rdma_core support single E-Switch per PF configurations - * only and vport_id field contains the vport index for - * associated VF, which is deduced from representor port name. - * For example, let's have the IB device port 10, it has - * attached network device eth0, which has port name attribute - * pf0vf2, we can deduce the VF number as 2, and set vport index - * as 3 (2+1). This assigning schema should be changed if the - * multiple E-Switch instances per PF configurations or/and PCI - * subfunctions are added. - */ - priv->vport_id = switch_info->representor ? - switch_info->port_name + 1 : -1; -#endif /* representor_id field keeps the unmodified VF index. */ priv->representor_id = switch_info->representor ? switch_info->port_name : -1; -- 2.30.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2021-07-12 13:41:41.884342345 +0100 +++ 0102-common-mlx5-use-new-port-query-API-if-available.patch 2021-07-12 13:41:36.874130610 +0100 @@ -1 +1 @@ -From d0cf77e8c2b64319057f5f629a7a595ce6e8b556 Mon Sep 17 00:00:00 2001 +From 245c98f97ef6763ea75ae847b76f0b5fdb68fbb6 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit d0cf77e8c2b64319057f5f629a7a595ce6e8b556 ] + @@ -36,2 +37,0 @@ -Cc: stable@dpdk.org - @@ -48 +48 @@ -index 3a3cb934a4..e1968a6906 100644 +index fa9686fdaf..8295cfbe41 100644 @@ -51,9 +51,9 @@ -@@ -93,6 +93,8 @@ has_sym_args = [ - 'IBV_WQ_FLAG_RX_END_PADDING' ], - [ 'HAVE_MLX5DV_DR_DEVX_PORT', 'infiniband/mlx5dv.h', - 'mlx5dv_query_devx_port' ], -+ [ 'HAVE_MLX5DV_DR_DEVX_PORT_V35', 'infiniband/mlx5dv.h', -+ 'mlx5dv_query_port' ], - [ 'HAVE_IBV_DEVX_OBJ', 'infiniband/mlx5dv.h', - 'mlx5dv_devx_obj_create' ], - [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h', +@@ -94,6 +94,8 @@ has_sym_args = [ + 'IBV_WQ_FLAG_RX_END_PADDING' ], + [ 'HAVE_MLX5DV_DR_DEVX_PORT', 'infiniband/mlx5dv.h', + 'mlx5dv_query_devx_port' ], ++ [ 'HAVE_MLX5DV_DR_DEVX_PORT_V35', 'infiniband/mlx5dv.h', ++ 'mlx5dv_query_port' ], + [ 'HAVE_IBV_DEVX_OBJ', 'infiniband/mlx5dv.h', + 'mlx5dv_devx_obj_create' ], + [ 'HAVE_IBV_FLOW_DEVX_COUNTERS', 'infiniband/mlx5dv.h', @@ -61 +61 @@ -index 145cf83fd9..b371fb7aa1 100644 +index 964f7e74ef..e24bcdfbd1 100644 @@ -128 +128 @@ -index 56246bca18..61f40d5478 100644 +index 9e385be957..f08c83768c 100644 @@ -159 +158,0 @@ - int (*dr_dump_rule)(FILE *file, void *rule); @@ -160,0 +160 @@ + uint32_t *eqn); @@ -162 +162 @@ -index b94696b379..be22d9cbd2 100644 +index 71837e34fe..56e2bf64f4 100644 @@ -165,2 +165 @@ -@@ -831,9 +831,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, - char name[RTE_ETH_NAME_MAX_LEN]; +@@ -695,9 +695,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, @@ -168,0 +168 @@ + unsigned int i; @@ -175,2 +175,2 @@ - if (switch_info->representor && dpdk_dev->devargs && -@@ -1064,29 +1062,27 @@ err_secondary: + if (switch_info->representor && dpdk_dev->devargs) { +@@ -940,29 +938,27 @@ err_secondary: @@ -217 +217 @@ -@@ -1106,8 +1102,8 @@ err_secondary: +@@ -982,8 +978,8 @@ err_secondary: @@ -228 +228 @@ -@@ -1117,25 +1113,21 @@ err_secondary: +@@ -993,25 +989,21 @@ err_secondary: @@ -263,3 +263,3 @@ - priv->representor_id = mlx5_representor_id_encode(switch_info, - eth_da->type); - /* + /* representor_id field keeps the unmodified VF index. */ + priv->representor_id = switch_info->representor ? + switch_info->port_name : -1;