From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp-fr.alcatel-lucent.com (fr-hpida-esg-02.alcatel-lucent.com [135.245.210.21]) by dpdk.org (Postfix) with ESMTP id 0F09AA10 for ; Thu, 9 Jul 2015 19:27:15 +0200 (CEST) Received: from us70uusmtp3.zam.alcatel-lucent.com (unknown [135.5.2.65]) by Websense Email Security Gateway with ESMTPS id 6FA48612A4257; Thu, 9 Jul 2015 17:27:11 +0000 (GMT) Received: from US70TWXCHHUB04.zam.alcatel-lucent.com (us70twxchhub04.zam.alcatel-lucent.com [135.5.2.36]) by us70uusmtp3.zam.alcatel-lucent.com (GMO) with ESMTP id t69HRDR1001923 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 9 Jul 2015 17:27:13 GMT Received: from US70TWXCHMBA07.zam.alcatel-lucent.com ([169.254.1.180]) by US70TWXCHHUB04.zam.alcatel-lucent.com ([135.5.2.36]) with mapi id 14.03.0195.001; Thu, 9 Jul 2015 13:27:12 -0400 From: "Assaad, Sami (Sami)" To: Bruce Richardson Thread-Topic: dev Digest, Vol 45, Issue 9 Thread-Index: AQHQp27noGOwOPbK5k+S0vRJm5IQjZ3SUfywgAEBKICAADbZ0A== Date: Thu, 9 Jul 2015 17:27:12 +0000 Message-ID: <9478F0FB69DAA249AF0A9BDA1E6ED95218848F9D@US70TWXCHMBA07.zam.alcatel-lucent.com> References: <9478F0FB69DAA249AF0A9BDA1E6ED95218848877@US70TWXCHMBA07.zam.alcatel-lucent.com> <20150709101026.GC8408@bricha3-MOBL3> In-Reply-To: <20150709101026.GC8408@bricha3-MOBL3> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.5.27.18] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] dev Digest, Vol 45, Issue 9 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jul 2015 17:27:15 -0000 Thank you! Best Regards, Sami. -----Original Message----- From: Bruce Richardson [mailto:bruce.richardson@intel.com]=20 Sent: Thursday, July 09, 2015 6:10 AM To: Assaad, Sami (Sami) Cc: dev@dpdk.org Subject: Re: dev Digest, Vol 45, Issue 9 On Wed, Jul 08, 2015 at 11:18:04PM +0000, Assaad, Sami (Sami) wrote: > Hello Bruce, >=20 > As you stated previously, I cannot guarantee the proper mapping of the=20 > virtual addresses between my primary and secondary processes. I have=20 > one primary process and up to 42 secondary processes. The application=20 > as a whole works very well. However, closing the=20 > application/restarting it continuously results with certain clients=20 > (secondary processes) failing to memory map the primary virtual=20 > addresses into /dev/zero (fd_zero). [Error "Cound not mmap 8573157376=20 > bytes in /dev/zero to requested address ..."] >=20 > So, I disabled ASLR on my virtual machine (VM). Rebooted it with ASLR per= manently disabled and ran the application. None of the clients (secondary p= rocesses) was able in memory mapping the virtual addresses derived from the= primary process. Rte_eal_config_reattach() fails with the error "Cannot mm= ap memory for rte_config". >=20 > My experience with ASLR is limited. Must I rebuild the software with ASLR= disabled or am I missing EAL configuration? > How about SELinux - must it be enabled or disabled? > Or ... must I manually configure the virtual addressing for both the prim= ary & secondary processes? An example would help. >=20 > I am running the primary process with the option --proc-type=3Dprimary an= d the 40+ clients with --proc-type=3Dsecondary. >=20 > - My OS is CentOS6.6 > - Using KVM/QEMU > - DPDK 1.8.0 > - Haswell >=20 > Thanks in advance! >=20 > Best Regards, > Sami Assaad.=20 > =20 This unfortunately can happen. To work around this we added in the EAL flag= "--base-virtaddr", to allow you to hint a suitable base address in the pri= mary process so that the address mappings can succeed in the secondary proc= esses. [What is happening with ASLR off is that the mappings are repeatable - eith= er always successful or always fail - sadly you hit the latter]. One tip might be to look at the output of the secondary processes to see wh= at address the failing file is getting mapped at, and use that as a base ad= dress hint to the primary process. You may have to play about with a few va= lues before you get it working. [One other possibility that sometimes works is to pass in additional cores = in the coremask to the primary process. Having more threads, even if they a= re idle, can also adjust the address layout due to the addition per-thread = stack space.] /Bruce >=20 > -----Original Message----- >=20 > Message: 2 > Date: Mon, 15 Jun 2015 11:21:39 +0100 > From: Bruce Richardson > To: "Assaad, Sami (Sami)" > Cc: "dev@dpdk.org" > Subject: Re: [dpdk-dev] DPDK and ASLR > Message-ID: <20150615102138.GB3872@bricha3-MOBL3> > Content-Type: text/plain; charset=3Dus-ascii >=20 > On Fri, Jun 12, 2015 at 10:53:58PM +0000, Assaad, Sami (Sami) wrote: > > When I operate a DPDK based application, the EAL always reports the fol= lowing: > > EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in t= he kernel. > > EAL: This may cause issues with mapping memory into secondary proc= esses. > >=20 > > Our application is DPDK client/server based and runs properly. > >=20 > > My questions are: > >=20 > > * Is this warning of any importance? >=20 > Yes, it's there for a reason. With ASLR, the position of the hugepage (an= d other) memory in your DPDK primary process virtual address space will mov= e about from one run to another, and the same with the secondary process. B= ecause of this, you may occasionally get instances where your application f= ails to run because an essential piece of memory is mapped at address X in = the primary, while something else is mapped at address X in the secondary p= rocess. How frequently, if ever, this happens will vary from application to= application. >=20 > If ASLR is disabled, the memory mappings created in the primary and secon= dary processes will be identical and repeatable from one run to another, so= you can know that if a set of processes starts once, it will start a secon= d time. With ASLR enabled, that guarantee cannot be made. >=20 >=20 > >=20 > > * Should ASLR be disabled? > >=20 >=20 > That is a questions we can't answer for you. ASLR is a security feature i= n the OS so you should be aware of the implications of disabling it. Howeve= r, if you need absolute guarantees of repeatabiltiy of mappings from one mu= lti-process run to another, the only way get that - that I am aware of - is= to disable ASLR. If an occasional random failure at startup is ok, then AS= LR can safely be left on. >=20 > > * Does ASLR affect DPDK performance? >=20 > No, it only affects the repeatability of memory mappings at DPDK start-up= . >=20 > Hope this clarifies things. >=20 > /Bruce >=20