From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mellanox.co.il (mail-il-dmz.mellanox.com [193.47.165.129]) by dpdk.org (Postfix) with ESMTP id 77CB62BAB for ; Thu, 21 Feb 2019 15:51:06 +0100 (CET) Received: from Internal Mail-Server by MTLPINE1 (envelope-from shahafs@mellanox.com) with ESMTPS (AES256-SHA encrypted); 21 Feb 2019 16:51:02 +0200 Received: from unicorn01.mtl.labs.mlnx. (unicorn01.mtl.labs.mlnx [10.7.12.62]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id x1LEp2ls022817; Thu, 21 Feb 2019 16:51:02 +0200 From: Shahaf Shuler To: anatoly.burakov@intel.com, yskoh@mellanox.com, thomas@monjalon.net, ferruh.yigit@intel.com, nhorman@tuxdriver.com, gaetan.rivet@6wind.com Cc: dev@dpdk.org Date: Thu, 21 Feb 2019 16:50:48 +0200 Message-Id: <7e8e1558c191f9445eb5d700d4a2ab0b4cbc6288.1550760032.git.shahafs@mellanox.com> X-Mailer: git-send-email 2.12.0 In-Reply-To: References: In-Reply-To: References: Subject: [dpdk-dev] [PATCH v2 5/6] net/mlx5: support PCI device DMA map and unmap X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Feb 2019 14:51:07 -0000 The implementation reuses the external memory registration work done by commit[1]. Note about representors: The current representor design will not work with those map and unmap functions. The reason is that for representors we have multiple IB devices share the same PCI function, so mapping will happen only on one of the representors and not all of them. While it is possible to implement such support, the IB representor design is going to be changed during DPDK19.05. The new design will have a single IB device for all representors, hence sharing of a single memory region between all representors will be possible. [1] commit 7e43a32ee060 ("net/mlx5: support externally allocated static memory") Signed-off-by: Shahaf Shuler --- drivers/net/mlx5/mlx5.c | 2 + drivers/net/mlx5/mlx5_mr.c | 135 ++++++++++++++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_rxtx.h | 5 ++ 3 files changed, 142 insertions(+) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index a913a5955f..ff17899cb3 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1626,6 +1626,8 @@ static struct rte_pci_driver mlx5_driver = { .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, .remove = mlx5_pci_remove, + .dma_map = mlx5_dma_map, + .dma_unmap = mlx5_dma_unmap, .drv_flags = (RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV | RTE_PCI_DRV_PROBE_AGAIN), }; diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c index 32be6a5445..4047a02bc0 100644 --- a/drivers/net/mlx5/mlx5_mr.c +++ b/drivers/net/mlx5/mlx5_mr.c @@ -14,6 +14,7 @@ #include #include #include +#include #include "mlx5.h" #include "mlx5_mr.h" @@ -1215,6 +1216,140 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque, } /** + * Finds the first ethdev that match the pci device. + * The existence of multiple ethdev per pci device is only with representors. + * On such case, it is enough to get only one of the ports as they all share + * the same ibv context. + * + * @param pdev + * Pointer to the PCI device. + * + * @return + * Pointer to the ethdev if found, NULL otherwise. + */ +static struct rte_eth_dev * +pci_dev_to_eth_dev(struct rte_pci_device *pdev) +{ + struct rte_dev_iterator it; + struct rte_device *dev; + + /** + * We really need to iterate all devices regardless of + * their owner. + */ + RTE_DEV_FOREACH(dev, "class=eth", &it) + if (dev == &pdev->device) + return it.class_device; + return NULL; +} + +/** + * DPDK callback to DMA map external memory to a PCI device. + * + * @param pdev + * Pointer to the PCI device. + * @param addr + * Starting virtual address of memory to be mapped. + * @param iova + * Starting IOVA address of memory to be mapped. + * @param len + * Length of memory segment being mapped. + * + * @return + * 0 on success, negative value on error. + */ +int +mlx5_dma_map(struct rte_pci_device *pdev, void *addr, + uint64_t iova __rte_unused, size_t len) +{ + struct rte_eth_dev *dev; + struct mlx5_mr *mr; + struct priv *priv; + + dev = pci_dev_to_eth_dev(pdev); + if (!dev) { + DRV_LOG(WARNING, "unable to find matching ethdev " + "to PCI device %p", (void *)pdev); + return -1; + } + priv = dev->data->dev_private; + mr = mlx5_create_mr_ext(dev, (uintptr_t)addr, len, SOCKET_ID_ANY); + if (!mr) { + DRV_LOG(WARNING, + "port %u unable to dma map", dev->data->port_id); + return -1; + } + rte_rwlock_write_lock(&priv->mr.rwlock); + LIST_INSERT_HEAD(&priv->mr.mr_list, mr, mr); + /* Insert to the global cache table. */ + mr_insert_dev_cache(dev, mr); + rte_rwlock_write_unlock(&priv->mr.rwlock); + return 0; +} + +/** + * DPDK callback to DMA unmap external memory to a PCI device. + * + * @param pdev + * Pointer to the PCI device. + * @param addr + * Starting virtual address of memory to be unmapped. + * @param iova + * Starting IOVA address of memory to be unmapped. + * @param len + * Length of memory segment being unmapped. + * + * @return + * 0 on success, negative value on error. + */ +int +mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, + uint64_t iova __rte_unused, size_t len __rte_unused) +{ + struct rte_eth_dev *dev; + struct priv *priv; + struct mlx5_mr *mr; + struct mlx5_mr_cache entry; + + dev = pci_dev_to_eth_dev(pdev); + if (!dev) { + DRV_LOG(WARNING, "unable to find matching ethdev " + "to PCI device %p", (void *)pdev); + return -1; + } + priv = dev->data->dev_private; + rte_rwlock_read_lock(&priv->mr.rwlock); + mr = mr_lookup_dev_list(dev, &entry, (uintptr_t)addr); + if (!mr) { + DRV_LOG(WARNING, "address 0x%" PRIxPTR " wasn't registered " + "to PCI device %p", (uintptr_t)addr, + (void *)pdev); + rte_rwlock_read_unlock(&priv->mr.rwlock); + return -1; + } + LIST_REMOVE(mr, mr); + LIST_INSERT_HEAD(&priv->mr.mr_free_list, mr, mr); + DEBUG("port %u remove MR(%p) from list", dev->data->port_id, + (void *)mr); + mr_rebuild_dev_cache(dev); + /* + * Flush local caches by propagating invalidation across cores. + * rte_smp_wmb() is enough to synchronize this event. If one of + * freed memsegs is seen by other core, that means the memseg + * has been allocated by allocator, which will come after this + * free call. Therefore, this store instruction (incrementing + * generation below) will be guaranteed to be seen by other core + * before the core sees the newly allocated memory. + */ + ++priv->mr.dev_gen; + DEBUG("broadcasting local cache flush, gen=%d", + priv->mr.dev_gen); + rte_smp_wmb(); + rte_rwlock_read_unlock(&priv->mr.rwlock); + return 0; +} + +/** * Register MR for entire memory chunks in a Mempool having externally allocated * memory and fill in local cache. * diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index c2529f96bc..f3f84dbac3 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -28,6 +28,7 @@ #include #include #include +#include #include "mlx5_utils.h" #include "mlx5.h" @@ -367,6 +368,10 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr); uint32_t mlx5_tx_mb2mr_bh(struct mlx5_txq_data *txq, struct rte_mbuf *mb); uint32_t mlx5_tx_update_ext_mp(struct mlx5_txq_data *txq, uintptr_t addr, struct rte_mempool *mp); +int mlx5_dma_map(struct rte_pci_device *pdev, void *addr, uint64_t iova, + size_t len); +int mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, uint64_t iova, + size_t len); /** * Provide safe 64bit store operation to mlx5 UAR region for both 32bit and -- 2.12.0