DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Saygin, Artur" <artur.saygin@intel.com>
To: Neil Horman <nhorman@tuxdriver.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] DPDK and custom memory
Date: Fri, 19 Sep 2014 00:13:55 +0000	[thread overview]
Message-ID: <B1E02754DD4D4F4EAC7A14C1202CE62073E9CC82@ORSMSX102.amr.corp.intel.com> (raw)
In-Reply-To: <20140903100349.GA24854@hmsreliant.think-freely.org>

FWIW: rte_mempool_xmem_create turned out to be exactly what the use case requires. It's not without limitations but is probably better than having to copy buffers between device and DPDK memory.

-----Original Message-----
From: Neil Horman [mailto:nhorman@tuxdriver.com] 
Sent: Wednesday, September 03, 2014 3:04 AM
To: Saygin, Artur
Cc: Alex Markuze; Thomas Monjalon; dev@dpdk.org
Subject: Re: [dpdk-dev] DPDK and custom memory

On Wed, Sep 03, 2014 at 01:17:53AM +0000, Saygin, Artur wrote:
> Thanks for prompt responses!
> 
> To clarify, the questions is not about accessing a NIC, but about a NIC accessing a very specific block of physical memory, possibly non-kernel managed.
> 
Still not sure what you mean here by non-kernel managed.  If memory can be
accessed from the CPU, then the kernel can allocate, free and access it, thats
it.  If the memory isn't accessible from the cpu, then this is out of our hands
anyway.  The only question is, how do you access it.

> Per my understanding memory that rte_mempool_create API obtains is kernel managed, grabbed by DPDK via HUGETLBFS, with address selection being outside of application control. Is there a way around that? As in have DPDK allocate buffer memory from address XYZ only...
Nope, the DPDK allocates blocks of memory without regard to the operation of the
NIC.  If you have some odd NIC that requires access to a specific physical
memory range, then it is your responsibility to reserve that memory and author
the PMD in such a way that it communicates with the NIC via that memory.
Usually this is done via a combination of operating system facilities (e.g. the
linux kernel commanline option memmap or the runtime mmap operation on the
/dev/mem device).

Regards
Neil

> 
> If VFIO / IOMMU is still the answer - I'll poke in that direction. If not - any additional insight is appreciated.
> 
> -----Original Message-----
> From: Alex Markuze [mailto:alex@weka.io] 
> Sent: Sunday, August 31, 2014 1:27 AM
> To: Thomas Monjalon
> Cc: Saygin, Artur; dev@dpdk.org
> Subject: Re: [dpdk-dev] DPDK and custom memory
> 
> Artur, I don't have the details of what you are trying to achieve, but
> it sounds like something that is covered by IOMMU, SW or HW.  The
> IOMMU creates an iova (I/O Virtual address) the nic can access the
> range is controlled with flags passed to the dma_map functions.
> 
> So I understand your question this way, How does the DPDK work with
> IOMMU enabled system and can you influence the mapping?
> 
> 
> On Sat, Aug 30, 2014 at 4:03 PM, Thomas Monjalon
> <thomas.monjalon@6wind.com> wrote:
> > Hello,
> >
> > 2014-08-29 18:40, Saygin, Artur:
> >> Imagine a PMD for an FPGA-based NIC that is limited to accessing certain
> >> memory regions <system, PCI, etc>.
> >
> > Does it mean Intel is making an FPGA-based NIC?
> >
> >> Is there a way to make DPDK use that exact memory?
> >
> > Maybe I don't understand the question well, because it doesn't seem really
> > different of what other PMDs do.
> > Assuming your NIC is PCI, you can access it via uio (igb_uio) or VFIO.
> >
> >> Perhaps this is more of a hugetlbfs question than DPDK but I thought I'd
> >> start here.
> >
> > It's a pleasure to receive new drivers.
> > Welcome here :)
> >
> > --
> > Thomas

  reply	other threads:[~2014-09-19  0:08 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <B1E02754DD4D4F4EAC7A14C1202CE62073E8E330@ORSMSX102.amr.corp.intel.com>
2014-08-30 13:03 ` Thomas Monjalon
2014-08-31  8:27   ` Alex Markuze
     [not found]     ` <B1E02754DD4D4F4EAC7A14C1202CE62073E8E8C7@ORSMSX102.amr.corp.intel.com>
2014-09-03 10:03       ` Neil Horman
2014-09-19  0:13         ` Saygin, Artur [this message]
2014-09-19 10:18           ` Neil Horman
2014-09-02 13:47 ` Neil Horman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=B1E02754DD4D4F4EAC7A14C1202CE62073E9CC82@ORSMSX102.amr.corp.intel.com \
    --to=artur.saygin@intel.com \
    --cc=dev@dpdk.org \
    --cc=nhorman@tuxdriver.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).