From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-f50.google.com (mail-pb0-f50.google.com [209.85.160.50]) by dpdk.org (Postfix) with ESMTP id 586A058E8 for ; Tue, 8 Apr 2014 18:05:40 +0200 (CEST) Received: by mail-pb0-f50.google.com with SMTP id md12so1231772pbc.37 for ; Tue, 08 Apr 2014 09:07:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Q83g8VrXq3oxAyvq5PXzgWuXW/wkqSvDLP1XhMFKYK4=; b=OA7JzzVTMrAI3yPXCZEWsYPrZI92tRKOZ4VuHvxWAe70xK2x35gQcM++tyIBXgSkJ4 D0oBuEh4/N0oodclBaw4aNaLKDz//D9l3RdQF6k3ZA56l0hK/iNKAXpBeoYa3cZR7xs/ NmJr3dDTqj42ajj05Y4ESfNHMVaxIyi1cwVz9Lx+xeF2y/dO56o3IkPGkcGK2LncAcRo ccPUtwQm5UyGDEpCrb1O1SleLWSVBcObxKdJpI/vViBP4yp4YlxccRo2HsUDk0JxtIJ9 UfEJzKiuRYDG0o5zrOzMRBhvOcGegwJp7kW7gjobt9qlX/3FF68AUitvqYICmmntJtIt tQjA== MIME-Version: 1.0 X-Received: by 10.68.226.35 with SMTP id rp3mr3728794pbc.73.1396973236689; Tue, 08 Apr 2014 09:07:16 -0700 (PDT) Received: by 10.66.13.5 with HTTP; Tue, 8 Apr 2014 09:07:16 -0700 (PDT) Received: by 10.66.13.5 with HTTP; Tue, 8 Apr 2014 09:07:16 -0700 (PDT) In-Reply-To: <4032A54B6BB5F04B8C08B6CFF08C59285540FFDA@FMSMSX103.amr.corp.intel.com> References: <032101cf533c$f871bb60$e9553220$@gmail.com> <4032A54B6BB5F04B8C08B6CFF08C59285540FFDA@FMSMSX103.amr.corp.intel.com> Date: Tue, 8 Apr 2014 19:07:16 +0300 Message-ID: From: elevran To: "Shaw, Jeffrey B" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Using DPDK in a multiprocess environment X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2014 16:05:40 -0000 Jeff, Thanks for the quick reply. I'll see if calling eal_init earlier resolves the problem I'm seeing. I'm not sure this will resolve the issue if shared objects are loaded before main() starts... I understand the rationale for having the same mbuf addresses across processes. And indeed they're mapped just fine (--virt-addr also gives some control over the mapping?). I was wondering if the same logic applies to the mapping of device PCI addresses. Are they shared or passed around between processes in the same way? Thanks again for the quick response, Etai =D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A 8 =D7=91=D7=90=D7=A4=D7=A8 2014 18:54,= "Shaw, Jeffrey B" =D7=9B=D7=AA=D7=91: > Have you tried calling "rte_eal_init()" closer to the beginning of the > program in your secondary process (i.e. the first thing in main())? > > The same mmap address is required. The reason is simple, if process A > thinks the virtual address of an mbuf is 123, and process B thinks the > virtual address of the same mbuf is 456, either process may segmentation > fault, accessing mbuf memory that is not actually mapped into the process= es > address space. > > Jeff > > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Etai Lev Ran > Sent: Tuesday, April 08, 2014 8:13 AM > To: dev@dpdk.org > Subject: [dpdk-dev] Using DPDK in a multiprocess environment > > Hi, > > > > I'd like to split DPDK application functionality into a setup (primary) > process and business logic (secondary) processes. > > The secondary processes access the hardware queues directly (exclusive > queue per process) and not through software rings. > > > > I'm running into an initialization problem: > > - The primary starts and sets up memory and ports and then goes = to > sleep waiting for termination signal > > - Secondary processes fail when probing the PCI bus for devices > (required, otherwise I get 0 ports visible in the secondary) > > > > The error is directly related to the secondary failing to get the *same* > virtual address for mmap'ing the UIO device fd's. > > The reason is that the secondary processes has considerably more shared > objects loaded and some of these are > > loaded and mapped into addresses which the primary used to map UIO fd's. > > The pci_map_resource() (linuxapp/eal_pci.c) code explicitly requires tha= t > the secondary processes get the same mmap'ed > > address as given to the primary. > > > > 1) Is this behavior (same mmap address) required? > > 2) If so, is there a workaround to cause PCI areas of UIO devices to > be > mapped to the same location in arbitrary processes? > > > > The samples work just fine since all primary and secondary processes have > similar set and load order for .so's > > > > Using v1.6 on Ubuntu 12.04 64b, ixgbe devices, 1GB hugepages, ASLR > disabled. > > > > Thanks, > > Etai > > > >