* [dpdk-dev] [PATCH] eal: move DMA mapping from bus-specific to generic driver
@ 2021-03-31 22:45 Thomas Monjalon
2021-03-31 22:53 ` Thomas Monjalon
0 siblings, 1 reply; 4+ messages in thread
From: Thomas Monjalon @ 2021-03-31 22:45 UTC (permalink / raw)
To: dev
Cc: hemant.agrawal, rosen.xu, sthemmin, longli, jerinj, ferruh.yigit,
andrew.rybchenko, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Maxime Coquelin, Chenbo Xia
The operations of DMA mapping and unmapping are controlled in some
bus drivers, following rte_bus specification.
If the device driver don't provide any specific mapping operation,
the bus driver may have a fallback (VFIO case for PCI).
The DMA mapping done by the device drivers are called
from the bus drivers via function pointers in bus-specific structures:
rte_vdev_driver and rte_pci_driver.
The device driver DMA mapping is not specific to the bus,
so it can be generalized to all device drivers, based on rte_device.
That's why the function pointers dma_map and dma_unmap
are moved to rte_driver, avoiding useless casts of device object.
The function prototypes rte_dev_dma_map_t and rte_dev_dma_unmap_t
are removed from rte_bus.h because the definition in rte_dev.h is enough,
but they are still used in rte_bus for bus drivers,
while being added in rte_driver for device drivers,
and removed from rte_vdev_driver/rte_pci_driver.
The impacted device drivers are mlx5 (PCI) and virtio_user (vdev).
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
---
Depends-on: series-16017 ("pci: add rte prefix")
---
drivers/bus/pci/pci_common.c | 8 ++--
drivers/bus/pci/rte_bus_pci.h | 40 --------------------
drivers/bus/vdev/rte_bus_vdev.h | 40 --------------------
drivers/bus/vdev/vdev.c | 32 +++++-----------
drivers/common/mlx5/mlx5_common_pci.c | 30 ++++++++-------
drivers/net/mlx5/mlx5.c | 4 +-
drivers/net/mlx5/mlx5_mr.c | 50 +++++++++++++------------
drivers/net/mlx5/mlx5_rxtx.h | 4 +-
drivers/net/virtio/virtio_user_ethdev.c | 22 +++++------
lib/librte_eal/include/rte_bus.h | 42 ---------------------
lib/librte_eal/include/rte_dev.h | 49 +++++++++++++++++++++++-
11 files changed, 117 insertions(+), 204 deletions(-)
diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f966358..75e3f01352 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -585,8 +585,8 @@ pci_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
rte_errno = EINVAL;
return -1;
}
- if (pdev->driver->dma_map)
- return pdev->driver->dma_map(pdev, addr, iova, len);
+ if (dev->driver->dma_map)
+ return dev->driver->dma_map(dev, addr, iova, len);
/**
* In case driver don't provides any specific mapping
* try fallback to VFIO.
@@ -608,8 +608,8 @@ pci_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
rte_errno = EINVAL;
return -1;
}
- if (pdev->driver->dma_unmap)
- return pdev->driver->dma_unmap(pdev, addr, iova, len);
+ if (dev->driver->dma_unmap)
+ return dev->driver->dma_unmap(dev, addr, iova, len);
/**
* In case driver don't provides any specific mapping
* try fallback to VFIO.
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b4731..5f2bf1cc33 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -119,44 +119,6 @@ typedef int (rte_pci_probe_t)(struct rte_pci_driver *, struct rte_pci_device *);
*/
typedef int (rte_pci_remove_t)(struct rte_pci_device *);
-/**
- * Driver-specific DMA mapping. After a successful call the device
- * will be able to read/write from/to this segment.
- *
- * @param dev
- * Pointer to the PCI device.
- * @param addr
- * Starting virtual address of memory to be mapped.
- * @param iova
- * Starting IOVA address of memory to be mapped.
- * @param len
- * Length of memory segment being mapped.
- * @return
- * - 0 On success.
- * - Negative value and rte_errno is set otherwise.
- */
-typedef int (pci_dma_map_t)(struct rte_pci_device *dev, void *addr,
- uint64_t iova, size_t len);
-
-/**
- * Driver-specific DMA un-mapping. After a successful call the device
- * will not be able to read/write from/to this segment.
- *
- * @param dev
- * Pointer to the PCI device.
- * @param addr
- * Starting virtual address of memory to be unmapped.
- * @param iova
- * Starting IOVA address of memory to be unmapped.
- * @param len
- * Length of memory segment being unmapped.
- * @return
- * - 0 On success.
- * - Negative value and rte_errno is set otherwise.
- */
-typedef int (pci_dma_unmap_t)(struct rte_pci_device *dev, void *addr,
- uint64_t iova, size_t len);
-
/**
* A structure describing a PCI driver.
*/
@@ -166,8 +128,6 @@ struct rte_pci_driver {
struct rte_pci_bus *bus; /**< PCI bus reference. */
rte_pci_probe_t *probe; /**< Device probe function. */
rte_pci_remove_t *remove; /**< Device remove function. */
- pci_dma_map_t *dma_map; /**< device dma map function. */
- pci_dma_unmap_t *dma_unmap; /**< device dma unmap function. */
const struct rte_pci_id *id_table; /**< ID table, NULL terminated. */
uint32_t drv_flags; /**< Flags RTE_PCI_DRV_*. */
};
diff --git a/drivers/bus/vdev/rte_bus_vdev.h b/drivers/bus/vdev/rte_bus_vdev.h
index fc315d10fa..a5ea830e55 100644
--- a/drivers/bus/vdev/rte_bus_vdev.h
+++ b/drivers/bus/vdev/rte_bus_vdev.h
@@ -65,44 +65,6 @@ typedef int (rte_vdev_probe_t)(struct rte_vdev_device *dev);
*/
typedef int (rte_vdev_remove_t)(struct rte_vdev_device *dev);
-/**
- * Driver-specific DMA mapping. After a successful call the device
- * will be able to read/write from/to this segment.
- *
- * @param dev
- * Pointer to the Virtual device.
- * @param addr
- * Starting virtual address of memory to be mapped.
- * @param iova
- * Starting IOVA address of memory to be mapped.
- * @param len
- * Length of memory segment being mapped.
- * @return
- * - 0 On success.
- * - Negative value and rte_errno is set otherwise.
- */
-typedef int (rte_vdev_dma_map_t)(struct rte_vdev_device *dev, void *addr,
- uint64_t iova, size_t len);
-
-/**
- * Driver-specific DMA un-mapping. After a successful call the device
- * will not be able to read/write from/to this segment.
- *
- * @param dev
- * Pointer to the Virtual device.
- * @param addr
- * Starting virtual address of memory to be unmapped.
- * @param iova
- * Starting IOVA address of memory to be unmapped.
- * @param len
- * Length of memory segment being unmapped.
- * @return
- * - 0 On success.
- * - Negative value and rte_errno is set otherwise.
- */
-typedef int (rte_vdev_dma_unmap_t)(struct rte_vdev_device *dev, void *addr,
- uint64_t iova, size_t len);
-
/**
* A virtual device driver abstraction.
*/
@@ -111,8 +73,6 @@ struct rte_vdev_driver {
struct rte_driver driver; /**< Inherited general driver. */
rte_vdev_probe_t *probe; /**< Virtual device probe function. */
rte_vdev_remove_t *remove; /**< Virtual device remove function. */
- rte_vdev_dma_map_t *dma_map; /**< Virtual device DMA map function. */
- rte_vdev_dma_unmap_t *dma_unmap; /**< Virtual device DMA unmap function. */
uint32_t drv_flags; /**< Flags RTE_VDEV_DRV_*. */
};
diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c
index 9a673347ae..3ba268f783 100644
--- a/drivers/bus/vdev/vdev.c
+++ b/drivers/bus/vdev/vdev.c
@@ -137,24 +137,18 @@ vdev_parse(const char *name, void *addr)
static int
vdev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
{
- struct rte_vdev_device *vdev = RTE_DEV_TO_VDEV(dev);
- const struct rte_vdev_driver *driver;
-
- if (!vdev) {
+ if (dev == NULL) {
rte_errno = EINVAL;
return -1;
}
- if (!vdev->device.driver) {
- VDEV_LOG(DEBUG, "no driver attach to device %s", dev->name);
+ if (dev->driver == NULL) {
+ VDEV_LOG(DEBUG, "no driver attached to device %s", dev->name);
return 1;
}
- driver = container_of(vdev->device.driver, const struct rte_vdev_driver,
- driver);
-
- if (driver->dma_map)
- return driver->dma_map(vdev, addr, iova, len);
+ if (dev->driver->dma_map)
+ return dev->driver->dma_map(dev, addr, iova, len);
return 0;
}
@@ -162,24 +156,18 @@ vdev_dma_map(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
static int
vdev_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova, size_t len)
{
- struct rte_vdev_device *vdev = RTE_DEV_TO_VDEV(dev);
- const struct rte_vdev_driver *driver;
-
- if (!vdev) {
+ if (dev == NULL) {
rte_errno = EINVAL;
return -1;
}
- if (!vdev->device.driver) {
- VDEV_LOG(DEBUG, "no driver attach to device %s", dev->name);
+ if (dev->driver == NULL) {
+ VDEV_LOG(DEBUG, "no driver attached to device %s", dev->name);
return 1;
}
- driver = container_of(vdev->device.driver, const struct rte_vdev_driver,
- driver);
-
- if (driver->dma_unmap)
- return driver->dma_unmap(vdev, addr, iova, len);
+ if (dev->driver->dma_unmap)
+ return dev->driver->dma_unmap(dev, addr, iova, len);
return 0;
}
diff --git a/drivers/common/mlx5/mlx5_common_pci.c b/drivers/common/mlx5/mlx5_common_pci.c
index a7f541a90c..a9f50dca07 100644
--- a/drivers/common/mlx5/mlx5_common_pci.c
+++ b/drivers/common/mlx5/mlx5_common_pci.c
@@ -350,7 +350,7 @@ mlx5_common_pci_remove(struct rte_pci_device *pci_dev)
}
static int
-mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr,
+mlx5_common_pci_dma_map(struct rte_device *rte_dev, void *addr,
uint64_t iova, size_t len)
{
struct mlx5_pci_driver *driver = NULL;
@@ -358,14 +358,14 @@ mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr,
struct mlx5_pci_device *dev;
int ret = -EINVAL;
- dev = pci_to_mlx5_device(pci_dev);
+ dev = pci_to_mlx5_device(RTE_DEV_TO_PCI(rte_dev));
if (!dev)
return -ENODEV;
TAILQ_FOREACH(driver, &drv_list, next) {
if (device_class_enabled(dev, driver->driver_class) &&
- driver->pci_driver.dma_map) {
- ret = driver->pci_driver.dma_map(pci_dev, addr,
- iova, len);
+ driver->pci_driver.driver.dma_map) {
+ ret = driver->pci_driver.driver.dma_map(rte_dev,
+ addr, iova, len);
if (ret)
goto map_err;
}
@@ -376,14 +376,16 @@ mlx5_common_pci_dma_map(struct rte_pci_device *pci_dev, void *addr,
if (temp == driver)
break;
if (device_class_enabled(dev, temp->driver_class) &&
- temp->pci_driver.dma_map && temp->pci_driver.dma_unmap)
- temp->pci_driver.dma_unmap(pci_dev, addr, iova, len);
+ temp->pci_driver.driver.dma_map &&
+ temp->pci_driver.driver.dma_unmap)
+ temp->pci_driver.driver.dma_unmap(rte_dev,
+ addr, iova, len);
}
return ret;
}
static int
-mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr,
+mlx5_common_pci_dma_unmap(struct rte_device *rte_dev, void *addr,
uint64_t iova, size_t len)
{
struct mlx5_pci_driver *driver;
@@ -391,16 +393,16 @@ mlx5_common_pci_dma_unmap(struct rte_pci_device *pci_dev, void *addr,
int local_ret = -EINVAL;
int ret;
- dev = pci_to_mlx5_device(pci_dev);
+ dev = pci_to_mlx5_device(RTE_DEV_TO_PCI(rte_dev));
if (!dev)
return -ENODEV;
ret = 0;
/* There is no unmap error recovery in current implementation. */
TAILQ_FOREACH_REVERSE(driver, &drv_list, mlx5_pci_bus_drv_head, next) {
if (device_class_enabled(dev, driver->driver_class) &&
- driver->pci_driver.dma_unmap) {
- local_ret = driver->pci_driver.dma_unmap(pci_dev, addr,
- iova, len);
+ driver->pci_driver.driver.dma_unmap) {
+ local_ret = driver->pci_driver.driver.dma_unmap(rte_dev,
+ addr, iova, len);
if (local_ret && (ret == 0))
ret = local_ret;
}
@@ -416,11 +418,11 @@ static struct rte_pci_id *mlx5_pci_id_table;
static struct rte_pci_driver mlx5_pci_driver = {
.driver = {
.name = MLX5_DRIVER_NAME,
+ .dma_map = mlx5_common_pci_dma_map,
+ .dma_unmap = mlx5_common_pci_dma_unmap,
},
.probe = mlx5_common_pci_probe,
.remove = mlx5_common_pci_remove,
- .dma_map = mlx5_common_pci_dma_map,
- .dma_unmap = mlx5_common_pci_dma_unmap,
};
static int
diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c
index abd7ff70df..f420fc7986 100644
--- a/drivers/net/mlx5/mlx5.c
+++ b/drivers/net/mlx5/mlx5.c
@@ -2216,12 +2216,12 @@ static struct mlx5_pci_driver mlx5_driver = {
.pci_driver = {
.driver = {
.name = MLX5_DRIVER_NAME,
+ .dma_map = mlx5_dma_map,
+ .dma_unmap = mlx5_dma_unmap,
},
.id_table = mlx5_pci_id_map,
.probe = mlx5_os_pci_probe,
.remove = mlx5_pci_remove,
- .dma_map = mlx5_dma_map,
- .dma_unmap = mlx5_dma_unmap,
.drv_flags = PCI_DRV_FLAGS,
},
};
diff --git a/drivers/net/mlx5/mlx5_mr.c b/drivers/net/mlx5/mlx5_mr.c
index 3255393ca2..6caf046838 100644
--- a/drivers/net/mlx5/mlx5_mr.c
+++ b/drivers/net/mlx5/mlx5_mr.c
@@ -288,33 +288,33 @@ mlx5_mr_update_ext_mp_cb(struct rte_mempool *mp, void *opaque,
}
/**
- * Finds the first ethdev that match the pci device.
- * The existence of multiple ethdev per pci device is only with representors.
+ * Find the first ethdev that matches the device.
+ * The existence of multiple ethdev per device is only with representors.
* On such case, it is enough to get only one of the ports as they all share
* the same ibv context.
*
- * @param pdev
- * Pointer to the PCI device.
+ * @param dev
+ * Pointer to the generic device.
*
* @return
* Pointer to the ethdev if found, NULL otherwise.
*/
static struct rte_eth_dev *
-pci_dev_to_eth_dev(struct rte_pci_device *pdev)
+dev_to_eth_dev(struct rte_device *dev)
{
uint16_t port_id;
- port_id = rte_eth_find_next_of(0, &pdev->device);
- if (port_id == RTE_MAX_ETHPORTS)
- return NULL;
- return &rte_eth_devices[port_id];
+ RTE_ETH_FOREACH_DEV_OF(port_id, dev)
+ return &rte_eth_devices[port_id];
+
+ return NULL;
}
/**
* DPDK callback to DMA map external memory to a PCI device.
*
- * @param pdev
- * Pointer to the PCI device.
+ * @param rte_dev
+ * Pointer to the generic device.
* @param addr
* Starting virtual address of memory to be mapped.
* @param iova
@@ -326,7 +326,7 @@ pci_dev_to_eth_dev(struct rte_pci_device *pdev)
* 0 on success, negative value on error.
*/
int
-mlx5_dma_map(struct rte_pci_device *pdev, void *addr,
+mlx5_dma_map(struct rte_device *rte_dev, void *addr,
uint64_t iova __rte_unused, size_t len)
{
struct rte_eth_dev *dev;
@@ -334,10 +334,11 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr,
struct mlx5_priv *priv;
struct mlx5_dev_ctx_shared *sh;
- dev = pci_dev_to_eth_dev(pdev);
+ dev = dev_to_eth_dev(rte_dev);
if (!dev) {
- DRV_LOG(WARNING, "unable to find matching ethdev "
- "to PCI device %p", (void *)pdev);
+ DRV_LOG(WARNING,
+ "unable to find matching ethdev to device %s",
+ rte_dev->name);
rte_errno = ENODEV;
return -1;
}
@@ -362,8 +363,8 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr,
/**
* DPDK callback to DMA unmap external memory to a PCI device.
*
- * @param pdev
- * Pointer to the PCI device.
+ * @param rte_dev
+ * Pointer to the generic device.
* @param addr
* Starting virtual address of memory to be unmapped.
* @param iova
@@ -375,7 +376,7 @@ mlx5_dma_map(struct rte_pci_device *pdev, void *addr,
* 0 on success, negative value on error.
*/
int
-mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr,
+mlx5_dma_unmap(struct rte_device *rte_dev, void *addr,
uint64_t iova __rte_unused, size_t len __rte_unused)
{
struct rte_eth_dev *dev;
@@ -384,10 +385,11 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr,
struct mlx5_mr *mr;
struct mr_cache_entry entry;
- dev = pci_dev_to_eth_dev(pdev);
+ dev = dev_to_eth_dev(rte_dev);
if (!dev) {
- DRV_LOG(WARNING, "unable to find matching ethdev "
- "to PCI device %p", (void *)pdev);
+ DRV_LOG(WARNING,
+ "unable to find matching ethdev to device %s",
+ rte_dev->name);
rte_errno = ENODEV;
return -1;
}
@@ -397,9 +399,9 @@ mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr,
mr = mlx5_mr_lookup_list(&sh->share_cache, &entry, (uintptr_t)addr);
if (!mr) {
rte_rwlock_read_unlock(&sh->share_cache.rwlock);
- DRV_LOG(WARNING, "address 0x%" PRIxPTR " wasn't registered "
- "to PCI device %p", (uintptr_t)addr,
- (void *)pdev);
+ DRV_LOG(WARNING,
+ "address 0x%" PRIxPTR " wasn't registered to device %s",
+ (uintptr_t)addr, rte_dev->name);
rte_errno = EINVAL;
return -1;
}
diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h
index 4f0fda0dec..ba0eb28995 100644
--- a/drivers/net/mlx5/mlx5_rxtx.h
+++ b/drivers/net/mlx5/mlx5_rxtx.h
@@ -458,9 +458,9 @@ uint32_t mlx5_rx_addr2mr_bh(struct mlx5_rxq_data *rxq, uintptr_t addr);
uint32_t mlx5_tx_mb2mr_bh(struct mlx5_txq_data *txq, struct rte_mbuf *mb);
uint32_t mlx5_tx_update_ext_mp(struct mlx5_txq_data *txq, uintptr_t addr,
struct rte_mempool *mp);
-int mlx5_dma_map(struct rte_pci_device *pdev, void *addr, uint64_t iova,
+int mlx5_dma_map(struct rte_device *dev, void *addr, uint64_t iova,
size_t len);
-int mlx5_dma_unmap(struct rte_pci_device *pdev, void *addr, uint64_t iova,
+int mlx5_dma_unmap(struct rte_device *dev, void *addr, uint64_t iova,
size_t len);
/**
diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c
index 1810a54694..3d640fb951 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -712,18 +712,16 @@ virtio_user_pmd_remove(struct rte_vdev_device *vdev)
return rte_eth_dev_close(eth_dev->data->port_id);
}
-static int virtio_user_pmd_dma_map(struct rte_vdev_device *vdev, void *addr,
+static int virtio_user_pmd_dma_map(struct rte_device *rte_dev, void *addr,
uint64_t iova, size_t len)
{
- const char *name;
struct rte_eth_dev *eth_dev;
struct virtio_user_dev *dev;
- if (!vdev)
+ if (rte_dev == NULL)
return -EINVAL;
- name = rte_vdev_device_name(vdev);
- eth_dev = rte_eth_dev_allocated(name);
+ eth_dev = rte_eth_dev_allocated(rte_dev->name);
/* Port has already been released by close. */
if (!eth_dev)
return 0;
@@ -736,18 +734,16 @@ static int virtio_user_pmd_dma_map(struct rte_vdev_device *vdev, void *addr,
return 0;
}
-static int virtio_user_pmd_dma_unmap(struct rte_vdev_device *vdev, void *addr,
+static int virtio_user_pmd_dma_unmap(struct rte_device *rte_dev, void *addr,
uint64_t iova, size_t len)
{
- const char *name;
struct rte_eth_dev *eth_dev;
struct virtio_user_dev *dev;
- if (!vdev)
+ if (rte_dev == NULL)
return -EINVAL;
- name = rte_vdev_device_name(vdev);
- eth_dev = rte_eth_dev_allocated(name);
+ eth_dev = rte_eth_dev_allocated(rte_dev->name);
/* Port has already been released by close. */
if (!eth_dev)
return 0;
@@ -761,10 +757,12 @@ static int virtio_user_pmd_dma_unmap(struct rte_vdev_device *vdev, void *addr,
}
static struct rte_vdev_driver virtio_user_driver = {
+ .driver = {
+ .dma_map = virtio_user_pmd_dma_map,
+ .dma_unmap = virtio_user_pmd_dma_unmap,
+ },
.probe = virtio_user_pmd_probe,
.remove = virtio_user_pmd_remove,
- .dma_map = virtio_user_pmd_dma_map,
- .dma_unmap = virtio_user_pmd_dma_unmap,
.drv_flags = RTE_VDEV_DRV_NEED_IOVA_AS_VA,
};
diff --git a/lib/librte_eal/include/rte_bus.h b/lib/librte_eal/include/rte_bus.h
index 80b154fb98..9f48ee7728 100644
--- a/lib/librte_eal/include/rte_bus.h
+++ b/lib/librte_eal/include/rte_bus.h
@@ -139,48 +139,6 @@ typedef int (*rte_bus_unplug_t)(struct rte_device *dev);
*/
typedef int (*rte_bus_parse_t)(const char *name, void *addr);
-/**
- * Device level DMA map function.
- * After a successful call, the memory segment will be mapped to the
- * given device.
- *
- * @param dev
- * Device pointer.
- * @param addr
- * Virtual address to map.
- * @param iova
- * IOVA address to map.
- * @param len
- * Length of the memory segment being mapped.
- *
- * @return
- * 0 if mapping was successful.
- * Negative value and rte_errno is set otherwise.
- */
-typedef int (*rte_dev_dma_map_t)(struct rte_device *dev, void *addr,
- uint64_t iova, size_t len);
-
-/**
- * Device level DMA unmap function.
- * After a successful call, the memory segment will no longer be
- * accessible by the given device.
- *
- * @param dev
- * Device pointer.
- * @param addr
- * Virtual address to unmap.
- * @param iova
- * IOVA address to unmap.
- * @param len
- * Length of the memory segment being mapped.
- *
- * @return
- * 0 if un-mapping was successful.
- * Negative value and rte_errno is set otherwise.
- */
-typedef int (*rte_dev_dma_unmap_t)(struct rte_device *dev, void *addr,
- uint64_t iova, size_t len);
-
/**
* Implement a specific hot-unplug handler, which is responsible for
* handle the failure when device be hot-unplugged. When the event of
diff --git a/lib/librte_eal/include/rte_dev.h b/lib/librte_eal/include/rte_dev.h
index 6dd72c11a1..b4de2ef653 100644
--- a/lib/librte_eal/include/rte_dev.h
+++ b/lib/librte_eal/include/rte_dev.h
@@ -24,6 +24,9 @@ extern "C" {
#include <rte_compat.h>
#include <rte_log.h>
+/* Forward declaration, defined below. */
+struct rte_device;
+
/**
* The device event type.
*/
@@ -71,13 +74,55 @@ struct rte_mem_resource {
void *addr; /**< Virtual address, NULL when not mapped. */
};
+/**
+ * Driver-specific DMA mapping.
+ * After a successful call, the device will be able
+ * to read/write from/to this segment.
+ *
+ * @param dev
+ * Pointer to the device.
+ * @param addr
+ * Starting virtual address of memory to be mapped.
+ * @param iova
+ * Starting IOVA address of memory to be mapped.
+ * @param len
+ * Length of memory segment being mapped.
+ * @return
+ * - 0 On success.
+ * - Negative value and rte_errno is set otherwise.
+ */
+typedef int (*rte_dev_dma_map_t)(struct rte_device *dev,
+ void *addr, uint64_t iova, size_t len);
+
+/**
+ * Driver-specific DMA un-mapping.
+ * After a successful call, the device will not be able
+ * to read/write from/to this segment.
+ *
+ * @param dev
+ * Pointer to the device.
+ * @param addr
+ * Starting virtual address of memory to be unmapped.
+ * @param iova
+ * Starting IOVA address of memory to be unmapped.
+ * @param len
+ * Length of memory segment being unmapped.
+ * @return
+ * - 0 On success.
+ * - Negative value and rte_errno is set otherwise.
+ */
+typedef int (*rte_dev_dma_unmap_t)(struct rte_device *dev,
+ void *addr, uint64_t iova, size_t len);
+
/**
* A structure describing a device driver.
*/
struct rte_driver {
- TAILQ_ENTRY(rte_driver) next; /**< Next in list. */
- const char *name; /**< Driver name. */
+ TAILQ_ENTRY(rte_driver) next; /**< Next in list. */
+ const char *name; /**< Driver name. */
const char *alias; /**< Driver alias. */
+ rte_dev_dma_map_t dma_map; /**< Device DMA map function. */
+ rte_dev_dma_unmap_t dma_unmap; /**< Device DMA unmap function. */
};
/*
--
2.30.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-dev] [PATCH] eal: move DMA mapping from bus-specific to generic driver
2021-03-31 22:45 [dpdk-dev] [PATCH] eal: move DMA mapping from bus-specific to generic driver Thomas Monjalon
@ 2021-03-31 22:53 ` Thomas Monjalon
2021-04-01 8:40 ` Kinsella, Ray
2021-04-12 15:03 ` Kinsella, Ray
0 siblings, 2 replies; 4+ messages in thread
From: Thomas Monjalon @ 2021-03-31 22:53 UTC (permalink / raw)
To: Ray Kinsella
Cc: dev, hemant.agrawal, rosen.xu, sthemmin, longli, jerinj,
ferruh.yigit, andrew.rybchenko, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Maxime Coquelin, Chenbo Xia, xuemingl,
david.marchand
01/04/2021 00:45, Thomas Monjalon:
> The operations of DMA mapping and unmapping are controlled in some
> bus drivers, following rte_bus specification.
> If the device driver don't provide any specific mapping operation,
> the bus driver may have a fallback (VFIO case for PCI).
>
> The DMA mapping done by the device drivers are called
> from the bus drivers via function pointers in bus-specific structures:
> rte_vdev_driver and rte_pci_driver.
>
> The device driver DMA mapping is not specific to the bus,
> so it can be generalized to all device drivers, based on rte_device.
> That's why the function pointers dma_map and dma_unmap
> are moved to rte_driver, avoiding useless casts of device object.
>
> The function prototypes rte_dev_dma_map_t and rte_dev_dma_unmap_t
> are removed from rte_bus.h because the definition in rte_dev.h is enough,
> but they are still used in rte_bus for bus drivers,
> while being added in rte_driver for device drivers,
> and removed from rte_vdev_driver/rte_pci_driver.
>
> The impacted device drivers are mlx5 (PCI) and virtio_user (vdev).
>
> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> ---
> Depends-on: series-16017 ("pci: add rte prefix")
> ---
> drivers/bus/pci/pci_common.c | 8 ++--
> drivers/bus/pci/rte_bus_pci.h | 40 --------------------
> drivers/bus/vdev/rte_bus_vdev.h | 40 --------------------
> drivers/bus/vdev/vdev.c | 32 +++++-----------
> drivers/common/mlx5/mlx5_common_pci.c | 30 ++++++++-------
> drivers/net/mlx5/mlx5.c | 4 +-
> drivers/net/mlx5/mlx5_mr.c | 50 +++++++++++++------------
> drivers/net/mlx5/mlx5_rxtx.h | 4 +-
> drivers/net/virtio/virtio_user_ethdev.c | 22 +++++------
> lib/librte_eal/include/rte_bus.h | 42 ---------------------
> lib/librte_eal/include/rte_dev.h | 49 +++++++++++++++++++++++-
> 11 files changed, 117 insertions(+), 204 deletions(-)
The ABI checker reports some issues on the driver interface.
It needs to be carefully analyzed, because driver interface
should not be part of the ABI compatibility contract.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-dev] [PATCH] eal: move DMA mapping from bus-specific to generic driver
2021-03-31 22:53 ` Thomas Monjalon
@ 2021-04-01 8:40 ` Kinsella, Ray
2021-04-12 15:03 ` Kinsella, Ray
1 sibling, 0 replies; 4+ messages in thread
From: Kinsella, Ray @ 2021-04-01 8:40 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, hemant.agrawal, rosen.xu, sthemmin, longli, jerinj,
ferruh.yigit, andrew.rybchenko, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Maxime Coquelin, Chenbo Xia, xuemingl,
david.marchand
On 31/03/2021 23:53, Thomas Monjalon wrote:
> 01/04/2021 00:45, Thomas Monjalon:
>> The operations of DMA mapping and unmapping are controlled in some
>> bus drivers, following rte_bus specification.
>> If the device driver don't provide any specific mapping operation,
>> the bus driver may have a fallback (VFIO case for PCI).
>>
>> The DMA mapping done by the device drivers are called
>> from the bus drivers via function pointers in bus-specific structures:
>> rte_vdev_driver and rte_pci_driver.
>>
>> The device driver DMA mapping is not specific to the bus,
>> so it can be generalized to all device drivers, based on rte_device.
>> That's why the function pointers dma_map and dma_unmap
>> are moved to rte_driver, avoiding useless casts of device object.
>>
>> The function prototypes rte_dev_dma_map_t and rte_dev_dma_unmap_t
>> are removed from rte_bus.h because the definition in rte_dev.h is enough,
>> but they are still used in rte_bus for bus drivers,
>> while being added in rte_driver for device drivers,
>> and removed from rte_vdev_driver/rte_pci_driver.
>>
>> The impacted device drivers are mlx5 (PCI) and virtio_user (vdev).
>>
>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>> ---
>> Depends-on: series-16017 ("pci: add rte prefix")
>> ---
>> drivers/bus/pci/pci_common.c | 8 ++--
>> drivers/bus/pci/rte_bus_pci.h | 40 --------------------
>> drivers/bus/vdev/rte_bus_vdev.h | 40 --------------------
>> drivers/bus/vdev/vdev.c | 32 +++++-----------
>> drivers/common/mlx5/mlx5_common_pci.c | 30 ++++++++-------
>> drivers/net/mlx5/mlx5.c | 4 +-
>> drivers/net/mlx5/mlx5_mr.c | 50 +++++++++++++------------
>> drivers/net/mlx5/mlx5_rxtx.h | 4 +-
>> drivers/net/virtio/virtio_user_ethdev.c | 22 +++++------
>> lib/librte_eal/include/rte_bus.h | 42 ---------------------
>> lib/librte_eal/include/rte_dev.h | 49 +++++++++++++++++++++++-
>> 11 files changed, 117 insertions(+), 204 deletions(-)
>
> The ABI checker reports some issues on the driver interface.
> It needs to be carefully analyzed, because driver interface
> should not be part of the ABI compatibility contract.
>
ok - I will take a look.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-dev] [PATCH] eal: move DMA mapping from bus-specific to generic driver
2021-03-31 22:53 ` Thomas Monjalon
2021-04-01 8:40 ` Kinsella, Ray
@ 2021-04-12 15:03 ` Kinsella, Ray
1 sibling, 0 replies; 4+ messages in thread
From: Kinsella, Ray @ 2021-04-12 15:03 UTC (permalink / raw)
To: Thomas Monjalon
Cc: dev, hemant.agrawal, rosen.xu, sthemmin, longli, jerinj,
ferruh.yigit, andrew.rybchenko, Matan Azrad, Shahaf Shuler,
Viacheslav Ovsiienko, Maxime Coquelin, Chenbo Xia, xuemingl,
david.marchand
On 31/03/2021 23:53, Thomas Monjalon wrote:
> 01/04/2021 00:45, Thomas Monjalon:
>> The operations of DMA mapping and unmapping are controlled in some
>> bus drivers, following rte_bus specification.
>> If the device driver don't provide any specific mapping operation,
>> the bus driver may have a fallback (VFIO case for PCI).
>>
>> The DMA mapping done by the device drivers are called
>> from the bus drivers via function pointers in bus-specific structures:
>> rte_vdev_driver and rte_pci_driver.
>>
>> The device driver DMA mapping is not specific to the bus,
>> so it can be generalized to all device drivers, based on rte_device.
>> That's why the function pointers dma_map and dma_unmap
>> are moved to rte_driver, avoiding useless casts of device object.
>>
>> The function prototypes rte_dev_dma_map_t and rte_dev_dma_unmap_t
>> are removed from rte_bus.h because the definition in rte_dev.h is enough,
>> but they are still used in rte_bus for bus drivers,
>> while being added in rte_driver for device drivers,
>> and removed from rte_vdev_driver/rte_pci_driver.
>>
>> The impacted device drivers are mlx5 (PCI) and virtio_user (vdev).
>>
>> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
>> ---
>> Depends-on: series-16017 ("pci: add rte prefix")
>> ---
>> drivers/bus/pci/pci_common.c | 8 ++--
>> drivers/bus/pci/rte_bus_pci.h | 40 --------------------
>> drivers/bus/vdev/rte_bus_vdev.h | 40 --------------------
>> drivers/bus/vdev/vdev.c | 32 +++++-----------
>> drivers/common/mlx5/mlx5_common_pci.c | 30 ++++++++-------
>> drivers/net/mlx5/mlx5.c | 4 +-
>> drivers/net/mlx5/mlx5_mr.c | 50 +++++++++++++------------
>> drivers/net/mlx5/mlx5_rxtx.h | 4 +-
>> drivers/net/virtio/virtio_user_ethdev.c | 22 +++++------
>> lib/librte_eal/include/rte_bus.h | 42 ---------------------
>> lib/librte_eal/include/rte_dev.h | 49 +++++++++++++++++++++++-
>> 11 files changed, 117 insertions(+), 204 deletions(-)
>
> The ABI checker reports some issues on the driver interface.
> It needs to be carefully analyzed, because driver interface
> should not be part of the ABI compatibility contract.
>
The depends-on series-16017 is marked as superseded.
This patch doesn't apply cleanly, even when the depends-on series is applied before it.
Ray K
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-04-12 15:04 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-31 22:45 [dpdk-dev] [PATCH] eal: move DMA mapping from bus-specific to generic driver Thomas Monjalon
2021-03-31 22:53 ` Thomas Monjalon
2021-04-01 8:40 ` Kinsella, Ray
2021-04-12 15:03 ` Kinsella, Ray
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).