From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 0F5185B1E for ; Mon, 1 Apr 2019 23:18:07 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from yskoh@mellanox.com) with ESMTPS (AES256-SHA encrypted); 2 Apr 2019 00:18:04 +0300 Received: from scfae-sc-2.mti.labs.mlnx (scfae-sc-2.mti.labs.mlnx [10.101.0.96]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x31LHwjh003672; Tue, 2 Apr 2019 00:18:02 +0300 From: Yongseok Koh To: shahafs@mellanox.com Cc: dev@dpdk.org, stable@dpdk.org Date: Mon, 1 Apr 2019 14:17:53 -0700 Message-Id: <20190401211757.26241-3-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190401211757.26241-1-yskoh@mellanox.com> References: <20190307074151.18815-1-yskoh@mellanox.com> <20190401211757.26241-1-yskoh@mellanox.com> Subject: [dpdk-dev] [PATCH v3 2/6] net/mlx5: fix external memory registration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Apr 2019 21:18:08 -0000 Secondary process is not allowed to register MR due to a restriction of library and kernel driver. Fixes: 7e43a32ee060 ("net/mlx5: support externally allocated static memory") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh Acked-by: Shahaf Shuler --- doc/guides/nics/mlx5.rst | 5 +++++ drivers/net/mlx5/mlx5_mr.c | 10 ++++++++++ 2 files changed, 15 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index f4db921f7f..fa9bf73da7 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -86,6 +86,11 @@ Limitations - Forked secondary process not supported. - All mempools must be initialized before rte_eth_dev_start(). + - External memory unregistered in EAL memseg list cannot be used for DMA + unless such memory has been registered by ``mlx5_mr_update_ext_mp()`` in + primary process and remapped to the same virtual address in secondary + process. If the external memory is registered by primary process but has + different virtual address in secondary process, unexpected error may happen. - Flow pattern without any specific vlan will match for vlan packets as well: diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index 3718877299..e7f55be6e1 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -1185,6 +1185,7 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, struct mlx5_mr_cache entry; uint32_t lkey; + assert(rte_eal_process_type() == RTE_PROC_PRIMARY); /* If already registered, it should return. */ rte_rwlock_read_lock(&priv->mr.rwlock); lkey = mr_lookup_dev(dev, &entry, addr); @@ -1400,6 +1401,15 @@ mlx5_tx_update_ext_mp(struct mlx5_txq_data *txq, uintptr_t addr, struct mlx5_mr_ctrl *mr_ctrl = &txq->mr_ctrl; struct mlx5_priv *priv = txq_ctrl->priv; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + DRV_LOG(WARNING, + "port %u using address (%p) from unregistered mempool" + " having externally allocated memory" + " in secondary process, please create mempool" + " prior to rte_eth_dev_start()", + PORT_ID(priv), (void *)addr); + return UINT32_MAX; + } mlx5_mr_update_ext_mp(ETH_DEV(priv), mr_ctrl, mp); return mlx5_tx_addr2mr_bh(txq, addr); } -- 2.11.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 23916A0679 for ; Mon, 1 Apr 2019 23:18:22 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0D4E15699; Mon, 1 Apr 2019 23:18:10 +0200 (CEST) Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 0F5185B1E for ; Mon, 1 Apr 2019 23:18:07 +0200 (CEST) Received: from Internal Mail-Server by MTLPINE1 (envelope-from yskoh@mellanox.com) with ESMTPS (AES256-SHA encrypted); 2 Apr 2019 00:18:04 +0300 Received: from scfae-sc-2.mti.labs.mlnx (scfae-sc-2.mti.labs.mlnx [10.101.0.96]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x31LHwjh003672; Tue, 2 Apr 2019 00:18:02 +0300 From: Yongseok Koh To: shahafs@mellanox.com Cc: dev@dpdk.org, stable@dpdk.org Date: Mon, 1 Apr 2019 14:17:53 -0700 Message-Id: <20190401211757.26241-3-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20190401211757.26241-1-yskoh@mellanox.com> References: <20190307074151.18815-1-yskoh@mellanox.com> <20190401211757.26241-1-yskoh@mellanox.com> Subject: [dpdk-dev] [PATCH v3 2/6] net/mlx5: fix external memory registration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Content-Type: text/plain; charset="UTF-8" Message-ID: <20190401211753.sstfaFDqbJanRLVUvQ7U32Zt9-VI0tnVbR9wkpo7v00@z> Secondary process is not allowed to register MR due to a restriction of library and kernel driver. Fixes: 7e43a32ee060 ("net/mlx5: support externally allocated static memory") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh Acked-by: Shahaf Shuler --- doc/guides/nics/mlx5.rst | 5 +++++ drivers/net/mlx5/mlx5_mr.c | 10 ++++++++++ 2 files changed, 15 insertions(+) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index f4db921f7f..fa9bf73da7 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -86,6 +86,11 @@ Limitations - Forked secondary process not supported. - All mempools must be initialized before rte_eth_dev_start(). + - External memory unregistered in EAL memseg list cannot be used for DMA + unless such memory has been registered by ``mlx5_mr_update_ext_mp()`` in + primary process and remapped to the same virtual address in secondary + process. If the external memory is registered by primary process but has + different virtual address in secondary process, unexpected error may happen. - Flow pattern without any specific vlan will match for vlan packets as well: diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index 3718877299..e7f55be6e1 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -1185,6 +1185,7 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, struct mlx5_mr_cache entry; uint32_t lkey; + assert(rte_eal_process_type() == RTE_PROC_PRIMARY); /* If already registered, it should return. */ rte_rwlock_read_lock(&priv->mr.rwlock); lkey = mr_lookup_dev(dev, &entry, addr); @@ -1400,6 +1401,15 @@ mlx5_tx_update_ext_mp(struct mlx5_txq_data *txq, uintptr_t addr, struct mlx5_mr_ctrl *mr_ctrl = &txq->mr_ctrl; struct mlx5_priv *priv = txq_ctrl->priv; + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { + DRV_LOG(WARNING, + "port %u using address (%p) from unregistered mempool" + " having externally allocated memory" + " in secondary process, please create mempool" + " prior to rte_eth_dev_start()", + PORT_ID(priv), (void *)addr); + return UINT32_MAX; + } mlx5_mr_update_ext_mp(ETH_DEV(priv), mr_ctrl, mp); return mlx5_tx_addr2mr_bh(txq, addr); } -- 2.11.0