DPDK patches and discussions
 help / color / mirror / Atom feed
From: Stefan Baranoff <sbaranoff@gmail.com>
To: dev@dpdk.org
Subject: [dpdk-dev] Multiprocess /dev/uio mmap Bug/possible fix
Date: Fri, 18 Jul 2014 09:22:20 -0400	[thread overview]
Message-ID: <CAHzKxpYXyYm7H+39sbz8Cr8ZTis5aM1eoEwzh5QD6o0+VmFD2g@mail.gmail.com> (raw)

All,

If this was fixed in 1.7 and I missed it I apologize (but it looks from
source to still be broken). I am using DPDK 1.6.0r2 (will be upgrading to
1.7.0 soon) on RHEL 6.4. I've converted the functions below to 1.7.0
names/locations since it looks to still be an issue there.

tl;dr -- to get examples/symmetric_mp to work at all I had to manually
force the address used to map PCI UIO devices in the primary process which
I achieved by:
In eal_memory.c :: get_virtual_area(...) -- Use directly then increment
internal_config.base_virtaddr instead of incrementing/adding baseaddr_offset
In eal_pci_uio.c in 1.7.0 :: pci_uio_map_resource -- In the primary
process, use and increment internal_config.base_virtaddr as the first
argument when calling pci_uio_map_resource (instead of NULL).


Background/reason:
Recently I was trying to get multiprocessing with each process reading from
one queue working and ran into issues with the call to mmap the memory for
/dev/uioX. Even the example 'symmetric_mp' application was failing. The
issue is that there are two memory regions which are mmap'd for each
/dev/uioX device and the second one, in the secondary process, is being
mapped to an address other than the requested address. This causes the
check for pci_map_resource(...) != uio_res->maps[i].addr to fail. I tracked
the reason for not using the requested address to the mmap call to create
the stack for a thread created during one of (I forget which - I think eal)
eal/pci/pmd_init calls which is used to handle interrupts.

With DPDK compiled as a single shared library on our system the memory
address handed for the second memory reach for each /dev/uioX in the
primary process is right in the middle of the stack for that thread.

My possible fix: use --base-virtaddr to populate the requested address for
mmaping the memory for /dev/uioX
I am not sure if this is safe but if it is I can submit a patch. My
approach was in eal_memory.c to, rather than keep an 'offset' variable
locally just increment the global base_virtaddr value each time through
that loop and then use the resulting final value when doing the /dev/uioX
mappings.

Of course in 1.6.0r2 I also ran into the errno bug with parsing
--base-virtaddr and strtoull but it looks like that was fixed in 1.7.0.


Thoughts? Thanks!
Stefan

             reply	other threads:[~2014-07-18 13:21 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-07-18 13:22 Stefan Baranoff [this message]
2014-07-18 13:29 ` Gray, Mark D

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAHzKxpYXyYm7H+39sbz8Cr8ZTis5aM1eoEwzh5QD6o0+VmFD2g@mail.gmail.com \
    --to=sbaranoff@gmail.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).