Re: [dpdk-dev] [PATCH v3 3/6] bus: introduce device level DMA memory mapping

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

On 05-Mar-19 1:59 PM, Shahaf Shuler wrote:
> The DPDK APIs expose 3 different modes to work with memory used for DMA:
> 
> 1. Use the DPDK owned memory (backed by the DPDK provided hugepages).
> This memory is allocated by the DPDK libraries, included in the DPDK
> memory system (memseg lists) and automatically DMA mapped by the DPDK
> layers.
> 
> 2. Use memory allocated by the user and register to the DPDK memory
> systems. Upon registration of memory, the DPDK layers will DMA map it
> to all needed devices. After registration, allocation of this memory
> will be done with rte_*malloc APIs.
> 
> 3. Use memory allocated by the user and not registered to the DPDK memory
> system. This is for users who wants to have tight control on this
> memory (e.g. avoid the rte_malloc header).
> The user should create a memory, register it through rte_extmem_register
> API, and call DMA map function in order to register such memory to
> the different devices.
> 
> The scope of the patch focus on #3 above.
> 
> Currently the only way to map external memory is through VFIO
> (rte_vfio_dma_map). While VFIO is common, there are other vendors
> which use different ways to map memory (e.g. Mellanox and NXP).
> 
> The work in this patch moves the DMA mapping to vendor agnostic APIs.
> Device level DMA map and unmap APIs were added. Implementation of those
> APIs was done currently only for PCI devices.
> 
> For PCI bus devices, the pci driver can expose its own map and unmap
> functions to be used for the mapping. In case the driver doesn't provide
> any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
> 
> Application usage with those APIs is quite simple:
> * allocate memory
> * call rte_extmem_register on the memory chunk.
> * take a device, and query its rte_device.
> * call the device specific mapping function for this device.
> 
> Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap
> APIs, leaving the rte device APIs as the preferred option for the user.
> 
> Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
> ---

<snip>

> diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
> index fd7f5ca7d5..08303b2f53 100644
> --- a/lib/librte_eal/common/eal_common_dev.c
> +++ b/lib/librte_eal/common/eal_common_dev.c
> @@ -756,3 +756,37 @@ rte_dev_iterator_next(struct rte_dev_iterator *it)
>   	free(cls_str);
>   	return it->device;
>   }
> +
> +int
> +rte_dev_dma_map(struct rte_device *dev, void *addr, uint64_t iova,
> +		size_t len)
> +{
> +	if (dev->bus->dma_map == NULL || len == 0) {
> +		rte_errno = EINVAL;
> +		return -1;

Here and below as well - please correct me if i'm wrong here, but should 
not having dma_map defined for a bus be an EINVAL error? EINVAL is 
reserved for invalid arguments - this seems to me that ENOTSUP would be 
more appropriate error code. User should be able to differentiate 
between failed mapping due to invalid data, and failed data due to the 
device's bus not supporting DMA remapping API in the first place.

-- 
Thanks,
Anatoly