From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp.tuxdriver.com (charlotte.tuxdriver.com [70.61.120.58]) by dpdk.org (Postfix) with ESMTP id EFFA8333 for ; Wed, 3 Sep 2014 11:59:27 +0200 (CEST) Received: from hmsreliant.think-freely.org ([2001:470:8:a08:7aac:c0ff:fec2:933b] helo=localhost) by smtp.tuxdriver.com with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1XP7Pr-0007ju-1U; Wed, 03 Sep 2014 06:03:56 -0400 Date: Wed, 3 Sep 2014 06:03:49 -0400 From: Neil Horman To: "Saygin, Artur" Message-ID: <20140903100349.GA24854@hmsreliant.think-freely.org> References: <1971750.bXvAyT2929@xps13> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-Spam-Status: No Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] DPDK and custom memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Sep 2014 09:59:28 -0000 On Wed, Sep 03, 2014 at 01:17:53AM +0000, Saygin, Artur wrote: > Thanks for prompt responses! > > To clarify, the questions is not about accessing a NIC, but about a NIC accessing a very specific block of physical memory, possibly non-kernel managed. > Still not sure what you mean here by non-kernel managed. If memory can be accessed from the CPU, then the kernel can allocate, free and access it, thats it. If the memory isn't accessible from the cpu, then this is out of our hands anyway. The only question is, how do you access it. > Per my understanding memory that rte_mempool_create API obtains is kernel managed, grabbed by DPDK via HUGETLBFS, with address selection being outside of application control. Is there a way around that? As in have DPDK allocate buffer memory from address XYZ only... Nope, the DPDK allocates blocks of memory without regard to the operation of the NIC. If you have some odd NIC that requires access to a specific physical memory range, then it is your responsibility to reserve that memory and author the PMD in such a way that it communicates with the NIC via that memory. Usually this is done via a combination of operating system facilities (e.g. the linux kernel commanline option memmap or the runtime mmap operation on the /dev/mem device). Regards Neil > > If VFIO / IOMMU is still the answer - I'll poke in that direction. If not - any additional insight is appreciated. > > -----Original Message----- > From: Alex Markuze [mailto:alex@weka.io] > Sent: Sunday, August 31, 2014 1:27 AM > To: Thomas Monjalon > Cc: Saygin, Artur; dev@dpdk.org > Subject: Re: [dpdk-dev] DPDK and custom memory > > Artur, I don't have the details of what you are trying to achieve, but > it sounds like something that is covered by IOMMU, SW or HW. The > IOMMU creates an iova (I/O Virtual address) the nic can access the > range is controlled with flags passed to the dma_map functions. > > So I understand your question this way, How does the DPDK work with > IOMMU enabled system and can you influence the mapping? > > > On Sat, Aug 30, 2014 at 4:03 PM, Thomas Monjalon > wrote: > > Hello, > > > > 2014-08-29 18:40, Saygin, Artur: > >> Imagine a PMD for an FPGA-based NIC that is limited to accessing certain > >> memory regions . > > > > Does it mean Intel is making an FPGA-based NIC? > > > >> Is there a way to make DPDK use that exact memory? > > > > Maybe I don't understand the question well, because it doesn't seem really > > different of what other PMDs do. > > Assuming your NIC is PCI, you can access it via uio (igb_uio) or VFIO. > > > >> Perhaps this is more of a hugetlbfs question than DPDK but I thought I'd > >> start here. > > > > It's a pleasure to receive new drivers. > > Welcome here :) > > > > -- > > Thomas