From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 6515FB347 for ; Fri, 19 Sep 2014 02:08:11 +0200 (CEST) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP; 18 Sep 2014 17:13:57 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,862,1389772800"; d="scan'208";a="388352721" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by FMSMGA003.fm.intel.com with ESMTP; 18 Sep 2014 17:08:22 -0700 Received: from orsmsx115.amr.corp.intel.com (10.22.240.11) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.195.1; Thu, 18 Sep 2014 17:13:56 -0700 Received: from orsmsx102.amr.corp.intel.com ([169.254.1.108]) by ORSMSX115.amr.corp.intel.com ([10.22.240.11]) with mapi id 14.03.0195.001; Thu, 18 Sep 2014 17:13:56 -0700 From: "Saygin, Artur" To: Neil Horman Thread-Topic: [dpdk-dev] DPDK and custom memory Thread-Index: Ac/DuKgCKCdw7vIpS3GfrEzm1AOQkgDpb6BUAwwJR3A= Date: Fri, 19 Sep 2014 00:13:55 +0000 Message-ID: References: <1971750.bXvAyT2929@xps13> <20140903100349.GA24854@hmsreliant.think-freely.org> In-Reply-To: <20140903100349.GA24854@hmsreliant.think-freely.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.139] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] DPDK and custom memory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Sep 2014 00:08:11 -0000 FWIW: rte_mempool_xmem_create turned out to be exactly what the use case re= quires. It's not without limitations but is probably better than having to = copy buffers between device and DPDK memory. -----Original Message----- From: Neil Horman [mailto:nhorman@tuxdriver.com]=20 Sent: Wednesday, September 03, 2014 3:04 AM To: Saygin, Artur Cc: Alex Markuze; Thomas Monjalon; dev@dpdk.org Subject: Re: [dpdk-dev] DPDK and custom memory On Wed, Sep 03, 2014 at 01:17:53AM +0000, Saygin, Artur wrote: > Thanks for prompt responses! >=20 > To clarify, the questions is not about accessing a NIC, but about a NIC a= ccessing a very specific block of physical memory, possibly non-kernel mana= ged. >=20 Still not sure what you mean here by non-kernel managed. If memory can be accessed from the CPU, then the kernel can allocate, free and access it, th= ats it. If the memory isn't accessible from the cpu, then this is out of our h= ands anyway. The only question is, how do you access it. > Per my understanding memory that rte_mempool_create API obtains is kernel= managed, grabbed by DPDK via HUGETLBFS, with address selection being outsi= de of application control. Is there a way around that? As in have DPDK allo= cate buffer memory from address XYZ only... Nope, the DPDK allocates blocks of memory without regard to the operation o= f the NIC. If you have some odd NIC that requires access to a specific physical memory range, then it is your responsibility to reserve that memory and aut= hor the PMD in such a way that it communicates with the NIC via that memory. Usually this is done via a combination of operating system facilities (e.g.= the linux kernel commanline option memmap or the runtime mmap operation on the /dev/mem device). Regards Neil >=20 > If VFIO / IOMMU is still the answer - I'll poke in that direction. If not= - any additional insight is appreciated. >=20 > -----Original Message----- > From: Alex Markuze [mailto:alex@weka.io]=20 > Sent: Sunday, August 31, 2014 1:27 AM > To: Thomas Monjalon > Cc: Saygin, Artur; dev@dpdk.org > Subject: Re: [dpdk-dev] DPDK and custom memory >=20 > Artur, I don't have the details of what you are trying to achieve, but > it sounds like something that is covered by IOMMU, SW or HW. The > IOMMU creates an iova (I/O Virtual address) the nic can access the > range is controlled with flags passed to the dma_map functions. >=20 > So I understand your question this way, How does the DPDK work with > IOMMU enabled system and can you influence the mapping? >=20 >=20 > On Sat, Aug 30, 2014 at 4:03 PM, Thomas Monjalon > wrote: > > Hello, > > > > 2014-08-29 18:40, Saygin, Artur: > >> Imagine a PMD for an FPGA-based NIC that is limited to accessing certa= in > >> memory regions . > > > > Does it mean Intel is making an FPGA-based NIC? > > > >> Is there a way to make DPDK use that exact memory? > > > > Maybe I don't understand the question well, because it doesn't seem rea= lly > > different of what other PMDs do. > > Assuming your NIC is PCI, you can access it via uio (igb_uio) or VFIO. > > > >> Perhaps this is more of a hugetlbfs question than DPDK but I thought I= 'd > >> start here. > > > > It's a pleasure to receive new drivers. > > Welcome here :) > > > > -- > > Thomas