From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mout.gmx.com (mout.gmx.com [74.208.4.200]) by dpdk.org (Postfix) with ESMTP id A4FCC7E62 for ; Fri, 24 Oct 2014 16:54:34 +0200 (CEST) Received: from [93.146.250.201] by 3capp-mailcom-lxa11.server.lan (via HTTP); Fri, 24 Oct 2014 17:03:02 +0200 MIME-Version: 1.0 Message-ID: From: "Mario Gianni" To: "Bruce Richardson" Content-Type: text/plain; charset=UTF-8 Date: Fri, 24 Oct 2014 17:03:02 +0200 Importance: normal Sensitivity: Normal In-Reply-To: <20141024133943.GB7648@BRICHA3-MOBL> References: <20141024120803.GA5804@BRICHA3-MOBL> , <20141024133943.GB7648@BRICHA3-MOBL> Content-Transfer-Encoding: quoted-printable X-UI-Message-Type: mail X-Priority: 3 X-Provags-ID: V03:K0:w9P5VcvqnxdoHjq0LJznhaaNkslxpIuWNxOaxH2sMQ6 AUzi3ipUXu3Pw2BFUFDyTkRnzBMSYgrHJELe/2KlUZtHDFz5O+ qhhk5ToNVDNQ5YBMBrBNe62gN5LmUoY3Tz2gXmdNBINjRxxgCI sYWFEXKJGxqcvtOq+lG1jYKH0f5KKnbpTicaOOs0avXxORYJM3 0q1qN0vonmoUzgBdZQ2LX3fHt8mjKaok8ViPN6FjdgRK/IEgqU pKhCA5NFCNciIQJqY+w6Zdi4nM3XPb05Omo3CxlAcoOcwAg1ZB s8BZW0pVOtdUdGLz7B/L84GBfrs X-UI-Out-Filterresults: notjunk:1; Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Cannot mmap device resource in DPDK 1.7.0 multi-process/multi-thread X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Oct 2014 14:54:35 -0000 So you are telling me that in order to implement multi-process I should bet= ter use the l2fwd_fork example instead of client_server_mp. In fact if I use the client_server_mp with a lot of mp_client threads it g= ives me the error. If instead I use the l2fwd_fork example it doesn't give me the error. One more question at this point: Assume that I use l2fwd_fork, when I launch the secondary process, how do = I assign the lcore coremask associated with that process?=20 Mario =C2=A0 =C2=A0 Sent:=C2=A0Friday, October 24, 2014 at 3:39 PM From:=C2=A0"Bruce Richardson" To:=C2=A0"Mario Gianni" Cc:=C2=A0dev@dpdk.org Subject:=C2=A0Re: [dpdk-dev] Cannot mmap device resource in DPDK 1.7.0 mul= ti-process/multi-thread On Fri, Oct 24, 2014 at 03:04:26PM +0200, Mario Gianni wrote: > Hi Bruce, > thank you for your answer, adding cores to the primary mask didn't help,= instead it helped manually passing the --base-virtaddr parameter, setting = it to the first value of Virtual Area that EAL finds when it starts the pri= mary process. > =C2=A0 > Honestly I don't understand why it works in this way, in the experimenta= l phase this could be a patch, but in the final program I have to automate = this process, do you have any suggestions? > For example is there a way to find the virtual area before starting the = primary process? > =C2=A0 > Mario In multi-process, there is a requirement that we can map the hugepage memo= ry and the NIC BARs to the same virtual addresses in both processes. Mostly this works ok, but occasionally it needs help due to the memory regions being chosen in the primary process being used by something else pre-eal_init in the secondary process. Anything from additional threads, t= o having an additional shared library linked in can affect the amount of memory used by the secondary process and therefore affect the chances that we won't be able to get an exact mapping. As far as I know there is no way to pre-compute how much memory a given process will use, or what memory regions will be free in it, by the time rte_eal_init() is called. If you just need multiple processes, which don't need to be individually spawned, then perhaps consider using fork() to spawn the processes, since that will guarantee you idential mappings without issues. The downside obviously is that you need to have all processes use the same binary, something not required for DPDK multi-process support. /Bruce > =C2=A0 > > Sent:=C2=A0Friday, October 24, 2014 at 2:08 PM > From:=C2=A0"Bruce Richardson" > To:=C2=A0"Mario Gianni" > Cc:=C2=A0dev@dpdk.org > Subject:=C2=A0Re: [dpdk-dev] Cannot mmap device resource in DPDK 1.7.0 m= ulti-process/multi-thread > On Fri, Oct 24, 2014 at 01:21:08PM +0200, Mario Gianni wrote: > > Hi all, I have a problem since I updated to 1.7.0 version, > > I got a multi-process, multi-threaded application, > > In my application first I launch a master process, then I launch a sec= ondary process with multiple threads in it > > Well, when the number of lcores reserved for the secondary process exc= eeds a certain number (eg. 4) i got an error in rte_eal_init() on the secon= dary process when it tries to map PCI memory: > > > > EAL: pci_map_resource(): cannot mmap(12, 0x7ffff2e96000, 0x800000, 0x1= 000): Success (0x7ffff559b000) > > EAL: Cannot mmap device resource > > EAL: Error - exiting with code: 1 > > Cause: Requested device 0000:01:00.0 cannot be used > > > > Can you help me? > > This could be because the additional memory/stack space used by the pthr= eads > for the cores in the secondary process is overlapping the space used in = the > primary process for hugepage or device memory. You could perhaps try add= ing > a few cores to the primary process's coremask (and not using those cores= ) > and see if it helps things. > Alternatively there is a base-virtaddr parameter that can be passed to t= he > primary process to try and adjust the starting address for it mapping > memory. If you look at where it starts mapping memory right now, and the= n > try hinting to it to maps the pages at a slightly higher or lower addres= s > and see if it helps. > > /Bruce