[dpdk-dev] DMA from a NIC to memory on a PCIe device.

[dpdk-dev] DMA from a NIC to memory on a PCIe device.

[Minsung Jang]

Hi,

I am trying to dma packets from NIC to memory on a PCIE device (not buffers
on hugepage.) Memory of the device is mapped to the physical memory via PCI
BAR.

I created a kernel module allocating/freeing memory on the device, and
applications can interact with the module via IOCTL so that apps can
allocate memory on the device and know virtual and physical address of
allocated memory. In DPDK, each mbuf contains its physical
address(buf_physaddr), which will eventually be used for dma address by a
dma engine of a NIC.

My idea is to simply replace this physical address given by a mem pool
(e.g., rte_pktmbuf_pool_create) with that from my Kernel module.  This will
look like

/* allocate memory on a pci device and return its Physical address */
pci_physical_addr = myDeviceMemAlloc(...);

/* create mbuf_pool */
mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL",...);

/* retrieve mbufs to replace their physical address */
rte_mempool_get_bulk(mbuf_pool, mbufs, counts );

/* replacing the physical address */
for_each (mbuf) {  mbuf[n]->buf_physaddr = pci_physical_addr + offset[n]; }


/* put all mbufs with a new physical address back to mbuf_pool */
rte_mempool_put_bulk(mbuf_pool, mbufs, counts);

/* port init and start port. This is the same as port_init in basicfwd.c */

Since rte_eth_rx_burst in dpdk doesn't help the application receive the
packets, I wrote a function to read them from the device memory.

But unfortunately, my idea seems not to work as I intended. If I
misunderstand anything in dpdk, especially DMA, please inform me. In
addition, if this idea doesn't work, please let me know how I can dma
packets from NIC to memory in a PCIe device without intervention of host
memory.

Thank you,
Minsung















-- 
Minsung Jang