From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by dpdk.org (Postfix) with ESMTP id 9C8EF68CB for ; Fri, 18 Jul 2014 15:21:20 +0200 (CEST) Received: by mail-ie0-f180.google.com with SMTP id at20so4509977iec.39 for ; Fri, 18 Jul 2014 06:22:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=g0SZIncXIIcAIGrxpXR0B+y9c56th3GTCNH6mBIc7AY=; b=YHfJ+w42rY32v7MNGXNvT73uvaiOWwUTpOrjJmaZ7IyqISsOeTA2suGVy4pkjb40zE nq+wXu51nTK4hECLs2fy3V/u+xFDO/ah3UCqKpNIepsjJeSr75b2bZ3uE1F7gIG5/Eqd puo3SdF0xYJZwSU6pGROELtCDqYjJqUu3AOlvzpz/Gy7jjhBVO4mLv3wKhdAIXLL90Sj y6P9Fs7VlUitYpdON5jfEfd0GNF2wKO7R9NsF3VXtaS+SVxTF7nzcdznojFgphgOjuYb ae9rrgr3ozuGby6xGV8b/ZbLYaiSN7Yje2wH0xnY49+u8iwBp+4VLD+iGDtmgCwU6TCn ly1A== MIME-Version: 1.0 X-Received: by 10.50.117.2 with SMTP id ka2mr8740712igb.33.1405689740476; Fri, 18 Jul 2014 06:22:20 -0700 (PDT) Received: by 10.64.28.18 with HTTP; Fri, 18 Jul 2014 06:22:20 -0700 (PDT) Date: Fri, 18 Jul 2014 09:22:20 -0400 Message-ID: From: Stefan Baranoff To: dev@dpdk.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Multiprocess /dev/uio mmap Bug/possible fix X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2014 13:21:21 -0000 All, If this was fixed in 1.7 and I missed it I apologize (but it looks from source to still be broken). I am using DPDK 1.6.0r2 (will be upgrading to 1.7.0 soon) on RHEL 6.4. I've converted the functions below to 1.7.0 names/locations since it looks to still be an issue there. tl;dr -- to get examples/symmetric_mp to work at all I had to manually force the address used to map PCI UIO devices in the primary process which I achieved by: In eal_memory.c :: get_virtual_area(...) -- Use directly then increment internal_config.base_virtaddr instead of incrementing/adding baseaddr_offset In eal_pci_uio.c in 1.7.0 :: pci_uio_map_resource -- In the primary process, use and increment internal_config.base_virtaddr as the first argument when calling pci_uio_map_resource (instead of NULL). Background/reason: Recently I was trying to get multiprocessing with each process reading from one queue working and ran into issues with the call to mmap the memory for /dev/uioX. Even the example 'symmetric_mp' application was failing. The issue is that there are two memory regions which are mmap'd for each /dev/uioX device and the second one, in the secondary process, is being mapped to an address other than the requested address. This causes the check for pci_map_resource(...) != uio_res->maps[i].addr to fail. I tracked the reason for not using the requested address to the mmap call to create the stack for a thread created during one of (I forget which - I think eal) eal/pci/pmd_init calls which is used to handle interrupts. With DPDK compiled as a single shared library on our system the memory address handed for the second memory reach for each /dev/uioX in the primary process is right in the middle of the stack for that thread. My possible fix: use --base-virtaddr to populate the requested address for mmaping the memory for /dev/uioX I am not sure if this is safe but if it is I can submit a patch. My approach was in eal_memory.c to, rather than keep an 'offset' variable locally just increment the global base_virtaddr value each time through that loop and then use the resulting final value when doing the /dev/uioX mappings. Of course in 1.6.0r2 I also ran into the errno bug with parsing --base-virtaddr and strtoull but it looks like that was fixed in 1.7.0. Thoughts? Thanks! Stefan