From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vk0-f41.google.com (mail-vk0-f41.google.com [209.85.213.41]) by dpdk.org (Postfix) with ESMTP id E3C2A1C52 for ; Fri, 1 Jul 2016 20:02:30 +0200 (CEST) Received: by mail-vk0-f41.google.com with SMTP id k68so76267073vkb.0 for ; Fri, 01 Jul 2016 11:02:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=9AANgAAZ1xySWocLcXbJ8jrRPIw2GTTBm0yTo8F8G8o=; b=Q5II6t/2dnbLsLEZ9Op8RHM8S4CKmyhvP90OYPJwWk4FUmoFe7E2CiaqPiM0Xkzw/B rkof1pToyUnkvBdOlWQEw1LyvN4CWdKgAtOB6ZDOg5khmpRbui2tkvbkRyyoGbLeLbq8 /3ZhR5lK4ixrYqgc3PjQ7R0j0jeG959pg393Ze5fi0G+K1e88qgSGSTcd23kpj2JBbMQ uIpiEGa4IqFPA47bZ7/fe6bqHcOZcpqh6TnmAeAZINRjObv5sQSva1LhxqAeeyM+oxNF roiy9JtVyppbcVy3UfkzTFag+DE9UVubETutMLvGpgBM0cX1sqYsAPmoB3lsW2Xr0esw SIGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=9AANgAAZ1xySWocLcXbJ8jrRPIw2GTTBm0yTo8F8G8o=; b=Bs+hiREt2EdFBkq7Jq37LUVxhif53BFNV3LLhKf1iODryltMQ0AcDKVcyyJqixQEE1 SfSiBlT2IdoK+fgep4HGb394SCB1WPbYWCZNwnrg0RcciITn1XRMHmvqjWfDVNAvpT/B 8962q5XLqEhIYwXGKMiOnfsBKeEQUHiqL0CS/3dCXzadGuFhGMGzV3oSRYUYTqw7oWRv 07+KxatLy8Uk1iItUbOZcx9YPwqy/tMsqhB5S15EiONQx9uJxBG43VIST7r7CLUQdFuL 0nJZWKgbBdrd7JkvX/dZcmCZFIrJFqCnzHvYYPsmS72QKwwrnq+r2xKGts4IJfm0cgmk uejA== X-Gm-Message-State: ALyK8tI1oszY2Qjy9fH4XsaEuSSbXUlQwMkSwFswfMaRY8ZpUWVWVpr5mOcGccAs2IqD385A0tHsNX4rCr5RBg== X-Received: by 10.31.80.199 with SMTP id e190mr10097631vkb.124.1467396150110; Fri, 01 Jul 2016 11:02:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.71.196 with HTTP; Fri, 1 Jul 2016 11:02:29 -0700 (PDT) From: Minsung Jang Date: Fri, 1 Jul 2016 14:02:29 -0400 Message-ID: To: dev@dpdk.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] DMA from a NIC to memory on a PCIe device. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Jul 2016 18:02:31 -0000 Hi, I am trying to dma packets from NIC to memory on a PCIE device (not buffers on hugepage.) Memory of the device is mapped to the physical memory via PCI BAR. I created a kernel module allocating/freeing memory on the device, and applications can interact with the module via IOCTL so that apps can allocate memory on the device and know virtual and physical address of allocated memory. In DPDK, each mbuf contains its physical address(buf_physaddr), which will eventually be used for dma address by a dma engine of a NIC. My idea is to simply replace this physical address given by a mem pool (e.g., rte_pktmbuf_pool_create) with that from my Kernel module. This will look like /* allocate memory on a pci device and return its Physical address */ pci_physical_addr = myDeviceMemAlloc(...); /* create mbuf_pool */ mbuf_pool = rte_pktmbuf_pool_create("MBUF_POOL",...); /* retrieve mbufs to replace their physical address */ rte_mempool_get_bulk(mbuf_pool, mbufs, counts ); /* replacing the physical address */ for_each (mbuf) { mbuf[n]->buf_physaddr = pci_physical_addr + offset[n]; } /* put all mbufs with a new physical address back to mbuf_pool */ rte_mempool_put_bulk(mbuf_pool, mbufs, counts); /* port init and start port. This is the same as port_init in basicfwd.c */ Since rte_eth_rx_burst in dpdk doesn't help the application receive the packets, I wrote a function to read them from the device memory. But unfortunately, my idea seems not to work as I intended. If I misunderstand anything in dpdk, especially DMA, please inform me. In addition, if this idea doesn't work, please let me know how I can dma packets from NIC to memory in a PCIe device without intervention of host memory. Thank you, Minsung -- Minsung Jang