* [dpdk-stable] [PATCH v2 03/17] net/mlx5: fix UAR lock sharing for multiport devices [not found] ` <1594794114-16313-1-git-send-email-viacheslavo@mellanox.com> @ 2020-07-15 6:21 ` Viacheslav Ovsiienko 0 siblings, 0 replies; 2+ messages in thread From: Viacheslav Ovsiienko @ 2020-07-15 6:21 UTC (permalink / raw) To: dev; +Cc: matan, rasland, olivier.matz, thomas, ferruh.yigit, stable The master and representors might be created over the multiport Infiniband devices and the UAR resource allocated for sibling ports might belong to the same underlying Infiniband device. Hardware requires the write access to the UAR must be performed as atomic 64-bit write, on 32-bit systems this is two sequential writes, protected by lock. Due to possibility to share the same UAR between sibling devices the locks must be moved to shared context. Fixes: f048f3d479a6 ("net/mlx5: switch to the shared IB device context") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com> --- drivers/net/mlx5/linux/mlx5_os.c | 6 ------ drivers/net/mlx5/mlx5.c | 6 ++++++ drivers/net/mlx5/mlx5.h | 10 +++++----- drivers/net/mlx5/mlx5_rxq.c | 2 +- drivers/net/mlx5/mlx5_txq.c | 2 +- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 14af468..63e9350 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -630,12 +630,6 @@ priv->mtu = RTE_ETHER_MTU; priv->mp_id.port_id = port_id; strlcpy(priv->mp_id.name, MLX5_MP_NAME, RTE_MP_MAX_NAME_LEN); -#ifndef RTE_ARCH_64 - /* Initialize UAR access locks for 32bit implementations. */ - rte_spinlock_init(&priv->uar_lock_cq); - for (i = 0; i < MLX5_UAR_PAGE_NUM_MAX; i++) - rte_spinlock_init(&priv->uar_lock[i]); -#endif /* Some internal functions rely on Netlink sockets, open them now. */ priv->nl_socket_rdma = mlx5_nl_init(NETLINK_RDMA); priv->nl_socket_route = mlx5_nl_init(NETLINK_ROUTE); diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 72e0870..0786945 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -717,6 +717,12 @@ struct mlx5_dev_ctx_shared * err = ENOMEM; goto error; } +#ifndef RTE_ARCH_64 + /* Initialize UAR access locks for 32bit implementations. */ + rte_spinlock_init(&sh->uar_lock_cq); + for (i = 0; i < MLX5_UAR_PAGE_NUM_MAX; i++) + rte_spinlock_init(&sh->uar_lock[i]); +#endif /* * Once the device is added to the list of memory event * callback, its global MR cache table cannot be expanded diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 84cd3e1..d01d7f3 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -559,6 +559,11 @@ struct mlx5_dev_ctx_shared { void *fdb_domain; /* FDB Direct Rules name space handle. */ void *rx_domain; /* RX Direct Rules name space handle. */ void *tx_domain; /* TX Direct Rules name space handle. */ +#ifndef RTE_ARCH_64 + rte_spinlock_t uar_lock_cq; /* CQs share a common distinct UAR */ + rte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX]; + /* UAR same-page access control required in 32bit implementations. */ +#endif struct mlx5_hlist *flow_tbls; /* Direct Rules tables for FDB, NIC TX+RX */ void *esw_drop_action; /* Pointer to DR E-Switch drop action. */ @@ -673,11 +678,6 @@ struct mlx5_priv { uint8_t mtr_color_reg; /* Meter color match REG_C. */ struct mlx5_mtr_profiles flow_meter_profiles; /* MTR profile list. */ struct mlx5_flow_meters flow_meters; /* MTR list. */ -#ifndef RTE_ARCH_64 - rte_spinlock_t uar_lock_cq; /* CQs share a common distinct UAR */ - rte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX]; - /* UAR same-page access control required in 32bit implementations. */ -#endif uint8_t skip_default_rss_reta; /* Skip configuration of default reta. */ uint8_t fdb_def_rule; /* Whether fdb jump to table 1 is configured. */ struct mlx5_mp_id mp_id; /* ID of a multi-process process */ diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index b436f06..2681322 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1997,7 +1997,7 @@ struct mlx5_rxq_ctrl * tmpl->rxq.elts = (struct rte_mbuf *(*)[1 << tmpl->rxq.elts_n])(tmpl + 1); #ifndef RTE_ARCH_64 - tmpl->rxq.uar_lock_cq = &priv->uar_lock_cq; + tmpl->rxq.uar_lock_cq = &priv->sh->uar_lock_cq; #endif tmpl->rxq.idx = idx; rte_atomic32_inc(&tmpl->refcnt); diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 35b3ade..e1fa24e 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -355,7 +355,7 @@ /* Assign an UAR lock according to UAR page number */ lock_idx = (txq_ctrl->uar_mmap_offset / page_size) & MLX5_UAR_PAGE_NUM_MASK; - txq_ctrl->txq.uar_lock = &priv->uar_lock[lock_idx]; + txq_ctrl->txq.uar_lock = &priv->sh->uar_lock[lock_idx]; #endif } -- 1.8.3.1 ^ permalink raw reply [flat|nested] 2+ messages in thread
[parent not found: <1594887800-6563-1-git-send-email-viacheslavo@mellanox.com>]
* [dpdk-stable] [PATCH v3 03/17] net/mlx5: fix UAR lock sharing for multiport devices [not found] ` <1594887800-6563-1-git-send-email-viacheslavo@mellanox.com> @ 2020-07-16 8:23 ` Viacheslav Ovsiienko 0 siblings, 0 replies; 2+ messages in thread From: Viacheslav Ovsiienko @ 2020-07-16 8:23 UTC (permalink / raw) To: dev; +Cc: matan, rasland, olivier.matz, thomas, ferruh.yigit, stable The master and representors might be created over the multiport Infiniband devices and the UAR resource allocated for sibling ports might belong to the same underlying Infiniband device. Hardware requires the write access to the UAR must be performed as atomic 64-bit write, on 32-bit systems this is two sequential writes, protected by lock. Due to possibility to share the same UAR between sibling devices the locks must be moved to shared context. Fixes: f048f3d479a6 ("net/mlx5: switch to the shared IB device context") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com> --- drivers/net/mlx5/linux/mlx5_os.c | 6 ------ drivers/net/mlx5/mlx5.c | 6 ++++++ drivers/net/mlx5/mlx5.h | 10 +++++----- drivers/net/mlx5/mlx5_rxq.c | 2 +- drivers/net/mlx5/mlx5_txq.c | 2 +- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 14af468..63e9350 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -630,12 +630,6 @@ priv->mtu = RTE_ETHER_MTU; priv->mp_id.port_id = port_id; strlcpy(priv->mp_id.name, MLX5_MP_NAME, RTE_MP_MAX_NAME_LEN); -#ifndef RTE_ARCH_64 - /* Initialize UAR access locks for 32bit implementations. */ - rte_spinlock_init(&priv->uar_lock_cq); - for (i = 0; i < MLX5_UAR_PAGE_NUM_MAX; i++) - rte_spinlock_init(&priv->uar_lock[i]); -#endif /* Some internal functions rely on Netlink sockets, open them now. */ priv->nl_socket_rdma = mlx5_nl_init(NETLINK_RDMA); priv->nl_socket_route = mlx5_nl_init(NETLINK_ROUTE); diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 72e0870..0786945 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -717,6 +717,12 @@ struct mlx5_dev_ctx_shared * err = ENOMEM; goto error; } +#ifndef RTE_ARCH_64 + /* Initialize UAR access locks for 32bit implementations. */ + rte_spinlock_init(&sh->uar_lock_cq); + for (i = 0; i < MLX5_UAR_PAGE_NUM_MAX; i++) + rte_spinlock_init(&sh->uar_lock[i]); +#endif /* * Once the device is added to the list of memory event * callback, its global MR cache table cannot be expanded diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 84cd3e1..d01d7f3 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -559,6 +559,11 @@ struct mlx5_dev_ctx_shared { void *fdb_domain; /* FDB Direct Rules name space handle. */ void *rx_domain; /* RX Direct Rules name space handle. */ void *tx_domain; /* TX Direct Rules name space handle. */ +#ifndef RTE_ARCH_64 + rte_spinlock_t uar_lock_cq; /* CQs share a common distinct UAR */ + rte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX]; + /* UAR same-page access control required in 32bit implementations. */ +#endif struct mlx5_hlist *flow_tbls; /* Direct Rules tables for FDB, NIC TX+RX */ void *esw_drop_action; /* Pointer to DR E-Switch drop action. */ @@ -673,11 +678,6 @@ struct mlx5_priv { uint8_t mtr_color_reg; /* Meter color match REG_C. */ struct mlx5_mtr_profiles flow_meter_profiles; /* MTR profile list. */ struct mlx5_flow_meters flow_meters; /* MTR list. */ -#ifndef RTE_ARCH_64 - rte_spinlock_t uar_lock_cq; /* CQs share a common distinct UAR */ - rte_spinlock_t uar_lock[MLX5_UAR_PAGE_NUM_MAX]; - /* UAR same-page access control required in 32bit implementations. */ -#endif uint8_t skip_default_rss_reta; /* Skip configuration of default reta. */ uint8_t fdb_def_rule; /* Whether fdb jump to table 1 is configured. */ struct mlx5_mp_id mp_id; /* ID of a multi-process process */ diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index b436f06..2681322 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -1997,7 +1997,7 @@ struct mlx5_rxq_ctrl * tmpl->rxq.elts = (struct rte_mbuf *(*)[1 << tmpl->rxq.elts_n])(tmpl + 1); #ifndef RTE_ARCH_64 - tmpl->rxq.uar_lock_cq = &priv->uar_lock_cq; + tmpl->rxq.uar_lock_cq = &priv->sh->uar_lock_cq; #endif tmpl->rxq.idx = idx; rte_atomic32_inc(&tmpl->refcnt); diff --git a/drivers/net/mlx5/mlx5_txq.c b/drivers/net/mlx5/mlx5_txq.c index 35b3ade..e1fa24e 100644 --- a/drivers/net/mlx5/mlx5_txq.c +++ b/drivers/net/mlx5/mlx5_txq.c @@ -355,7 +355,7 @@ /* Assign an UAR lock according to UAR page number */ lock_idx = (txq_ctrl->uar_mmap_offset / page_size) & MLX5_UAR_PAGE_NUM_MASK; - txq_ctrl->txq.uar_lock = &priv->uar_lock[lock_idx]; + txq_ctrl->txq.uar_lock = &priv->sh->uar_lock[lock_idx]; #endif } -- 1.8.3.1 ^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2020-07-16 8:23 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <1591771085-24959-1-git-send-email-viacheslavo@mellanox.com>
[not found] ` <1594794114-16313-1-git-send-email-viacheslavo@mellanox.com>
2020-07-15 6:21 ` [dpdk-stable] [PATCH v2 03/17] net/mlx5: fix UAR lock sharing for multiport devices Viacheslav Ovsiienko
[not found] ` <1594887800-6563-1-git-send-email-viacheslavo@mellanox.com>
2020-07-16 8:23 ` [dpdk-stable] [PATCH v3 " Viacheslav Ovsiienko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).