From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E96CBA0032; Fri, 15 Jul 2022 12:17:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8CFE640696; Fri, 15 Jul 2022 12:17:53 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id E432140042 for ; Fri, 15 Jul 2022 12:17:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657880272; x=1689416272; h=message-id:date:subject:to:references:from:in-reply-to: content-transfer-encoding:mime-version; bh=vekFrApO17u9VfeICIh5DjAiKcFMbvCRejzffbBjcAQ=; b=JPRS3ESxN5E2qcZxhi6aY2BResbSg8Udo5QHt7Jhjv7jBO+/Xg4zXRvf kbne9RDWYHNc0xKygZAoZ29WESpFD4UqyWE1p8ThxoUruUavtf1924hdh FbA4KZ103I9HccTQKOilc8ofwztz6eL4o6tdf+1wvTICLPslvfZHLcfxt pvXB9+SV14bK4ab37FZYhAcfCvl0twV6HqR6kplQeEvpo0tu5uF69ZGGZ X4rhsb30/hRmra3c6PXK8WebSPyrMVFJYIHxUewmB6Cg1C6AuFExaAFbJ 5pk8rVBZoBdkOLdNdrx0gc8Sb06O/OFaQv14MatPLcW2i5akmwURcFVuj g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="284515656" X-IronPort-AV: E=Sophos;i="5.92,273,1650956400"; d="scan'208";a="284515656" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Jul 2022 03:17:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.92,273,1650956400"; d="scan'208";a="772928937" Received: from orsmsx602.amr.corp.intel.com ([10.22.229.15]) by orsmga005.jf.intel.com with ESMTP; 15 Jul 2022 03:17:50 -0700 Received: from orsmsx602.amr.corp.intel.com (10.22.229.15) by ORSMSX602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Fri, 15 Jul 2022 03:17:50 -0700 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx602.amr.corp.intel.com (10.22.229.15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Fri, 15 Jul 2022 03:17:50 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.100) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.27; Fri, 15 Jul 2022 03:17:49 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SEtL1SWpvsX9IxgSkVNOE2ECW4s57N2c7RTxifjxsVJUWrMF76rhAGdUUd4jusIiyZqRkk8XXapFNs7+DWmqyfXJP1+IfRefi4mrr08OE7hmF9dtezXmCaEyHESMp5MzMZI/a9K/Pf4MbAMW8JXzpsTqVhEyIcdwxhnns9r6Kty17aBTVXgOm/0t7be3xYQb+7MvRKHpNT1dr7PfsjZFmyhs9YENt3Av+60t/BNskiO73CMzLfLoOik97qGEuug3dCobOY1+uHbEe9h1IxSl6RJtzPfdIhyw9RTrbc1GOUb8JsyxFbs48fTa743T9VJ3nsHXE9tj1pdzCMoUV9oWFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5/H6/66bDxzKXfQPw3+G43BQFyBokRDZE9GCqVNmXoY=; b=Yejy7N3Ga4fA58aLWmvTS5FgkLXQs/l0NKOfUQJoww2F1p6aWZBrCXA0th7IM6SCX0DJP0sP3tOXRg6EYotxZaVQopvvzWELyHkn16CK/WRMvOgJnqcpxnHpMtodf5rTEVjjEJGQFSeATXVjHQ5d6bwUwFaO9lawRcN5Cm5tvsF13s5Aa0y/H2fsCqfNy07t6R3HYr3FvT/iUOHARCJL0OhW5B7trCS68Lp5gU3vaBQkSLlC+xUCXMjY+gRBMD7uk/6IY6qdm2Ns1uUTxp719FVua4zLSaKPIZr+aRtkQ7Tkpz2xmnvUCu0sqoUH59qkuIhr1N85IWD4cgV9jv1QRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from BN6PR11MB1251.namprd11.prod.outlook.com (2603:10b6:404:48::10) by MWHPR11MB1279.namprd11.prod.outlook.com (2603:10b6:300:2a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5417.25; Fri, 15 Jul 2022 10:17:47 +0000 Received: from BN6PR11MB1251.namprd11.prod.outlook.com ([fe80::128:8fb:9d0:1a9f]) by BN6PR11MB1251.namprd11.prod.outlook.com ([fe80::128:8fb:9d0:1a9f%8]) with mapi id 15.20.5417.029; Fri, 15 Jul 2022 10:17:47 +0000 Message-ID: <9350acbd-06b3-8981-1c2a-7a2b1a5daea9@intel.com> Date: Fri, 15 Jul 2022 11:17:43 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0 Thunderbird/91.11.0 Subject: Re: DPDK 19.11.3 with multi processes and external physical memory: unable to receive traffic in the secondary processes Content-Language: en-US To: Asaf Sinai , "dev@dpdk.org" References: From: "Burakov, Anatoly" In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO2P123CA0037.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600::25) To BN6PR11MB1251.namprd11.prod.outlook.com (2603:10b6:404:48::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: abb8aeec-0a2e-4e28-beaf-08da664b43b4 X-MS-TrafficTypeDiagnostic: MWHPR11MB1279:EE_ X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: S/DeNptiS242WmvOi325UMh3N84DdZyJws/pRdyIDahI3/UKWFqrwucJS6imnmZVr/0OMO7h5bnF1rTbbXDMNmdLaBa6EJA/M7klwaMLjV3vX8rOIRDoyMhSbizFQZXP6u9CWQ+006SZpVSlRUd5Ez4nJFEZQNtrh8PFVzNn4MgTW6n7IFvLNrACWEufOpda3J2tvLzVkTOZEWd6+sEwn6WSp25iGgBWaovDx5qZqlgVGxD34Jw430QT08oXU6NRZegW0Ls7e1ulX96JAScoaXbpSwek/pkUOJfjgagUy+L2xCXR1O4YonEsHZD7+3GSDUTavPAzJ1GxLAzA++550Gz3ac7fG9b8G1TbiSCNz92EdDL1xsnXlT8onPqVJn4NBC0spoirlGJ1uiaYIbTCMv5236Y8XSIMBr3AB61SA2YrNXBSCFpxtG/SHi39AE/b8pYn1SShSkbGQzrsg5PBqAPEmyUM42/w/An1E/LmZdH06bRkczHZuAgneNBslHyE1ebDRyzToc8qfI58X+cba1aIgqxn2+gKL2JzBKF/bLuxNQHqaoQ2Gy9sEwxKIFfIEc7r9PTJ0QrI9aWBScGRq3Eg/KlDrBnWJucDmn73y8xrQ8SMWcg9yRrJH3rsOUKVMx6VTz38sbwf2wBm6mZwx2G2j1WU00rO4xjUvOzqIc7xtpEIMWWH+7Xa/8VCi/V2b80G2+0J68toC52z4Wg9yWrIGx+ESzrwpIN6fLhQkYpbQYR79E4gGSQ0RaGeeqErgszK1vWgolSJZkd0z/p8Old9khHXNWzHZKolyn9oc2iL7hVZ2176zJxcGSETNOp+BcHEukhmUu1tA45Ah2IE/rDvzAoirAKzqBP40hatut+f+JwGlxZ4rzi5/V/mZ/8JckLNsLvRdKiXOjAU9Jnnt0QLMqMT4nGqvmVGLPBCJRo= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN6PR11MB1251.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(346002)(396003)(39860400002)(136003)(366004)(376002)(82960400001)(8676002)(110136005)(41300700001)(66556008)(31686004)(36756003)(66476007)(38100700002)(53546011)(66946007)(316002)(45080400002)(6506007)(86362001)(966005)(6486002)(478600001)(2616005)(5660300002)(8936002)(31696002)(2906002)(26005)(186003)(83380400001)(6512007)(6666004)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VDVYUytDYzk2ZFJ6NEJ3NE5jNmlsa2JEcVk4TktsN2ZzTU1zRm12THVzNlNL?= =?utf-8?B?a003anBaYVZreFN6b1BJYUdEMEJOaFZQQ2V2R0hZOHFqL050c0lQYXJkR1Bh?= =?utf-8?B?U2pTY1JBTTBqdUlpMGZ0anc5ejBXT1dlVng1blFEWlU5TEt3QmhOU2tMMjJJ?= =?utf-8?B?WGk4MTVubkJJZFZzRUROemQyL09vOXE1L3A2bnFuNGtGZGx4bXR0eE9nbHdR?= =?utf-8?B?SEk3ekY3Tnc1TldUank3UFJvMVNPK2tjUkIzdTVmWEVnUkpTSDVpQzJkMFZC?= =?utf-8?B?T2RBLzV2NjVueHh6VkZrR2FGWUtoLzBNcGkwMEJHZXJRclc3TVYzQ3JmRjBt?= =?utf-8?B?MnMzS0NRVUpic2tGMndZUkxaTW5xTTYzOCtIWnRpZUZXN2hXb1NKa2U0emJF?= =?utf-8?B?ZFpLVHFPWFNRbzBxajNYK2hCNDNjeUoxK3pZU1Z5Zk4xS0gwUXJpaW1SWWsr?= =?utf-8?B?cXN3aTBRTDlwYjIyblFOelpzaGErNGVialRtTEpxUUFxTDBhMUUxdUM1aWVz?= =?utf-8?B?bWl2VzFsbGpGQ3QxTVoxVDdIbCtKVnpjNnVLbWZpNGFHTTlIZWx1S2h5ek5s?= =?utf-8?B?VElaOWVxOXZaQ1BnMHlKV2pycHJjcUhJOHVoQWQ4UmMrTkRSTm9xajF1V2Yw?= =?utf-8?B?TU9VWVpUS0hKVGJMdEw0a3dLMzh2NXVZWk5LS2RLKzltR085WkJjWmY0KzlB?= =?utf-8?B?WENwQW9nV2FqYk5IUXNPaWY0eXMwamUzR0hOcWVFSm53NEJvdi9hYWN2b0My?= =?utf-8?B?b2NoT1NyNDU2MUJHTWRqNzZFa2s4VzVlNm5oNFh6d3NuaXBxc1dmMXpTRE1R?= =?utf-8?B?cktzYi9ZT3NzbE9iZEhFdzUzVmlLOUJoZ1pxV3I2bFVoeDRuZ3c4Yzd1Mzhr?= =?utf-8?B?UHV6eEx4c3NlSEZxRmhBOCtJOHpUa1BIaFJnTUZjanZjVUYydzg2UHRFUkFI?= =?utf-8?B?Y0RJanVDdmVzaDZOaklnMnhUcE5PcTlRakRSVDNjaThBVm90WDVJdGRjTUNZ?= =?utf-8?B?SnUvK0RmemZPN1ZtYVd2NUwwQ2w1dFZwOGJWdVgwK3BuUWl6RjJSY0dyOVM4?= =?utf-8?B?bG1KY29Da0ZyZEdVMG1WWU00SW8vdGxoanYzV21MMlhZU1hnU0RGa3o2K3gy?= =?utf-8?B?K1UwZEZQTGZycHhVQTlmZUwxRzhzWDlLaStWRTZiVS9OSVc4TGFERmhRNDNx?= =?utf-8?B?Z1dyWW8xbDdCQ0NFWGRGbWl3Y2Y5TTNXMTZ2VWxlbEJaNUhGeVdaaEtGYW11?= =?utf-8?B?cnQ2S0g2bEY0NVFBYnhTZ0RRaWZrL1NoLytFZitRN2lxaU5oeVJGLzhBUGl6?= =?utf-8?B?TjBtQ3BWUTAySVEvS0NVNjJUYWVFRVY4UmdtbzhJcjV3dVhwZkMwZ2hvaHdy?= =?utf-8?B?Y3VHa2VadGtSQ3UxdnpHTmNyaGlrQ1RkQjBUTEhDQkZYNjdvOGhiOGZsd2h6?= =?utf-8?B?ZTNhQVlrRzFNazQzdVhLYjlNeFBKbXZlNzY0MlJlZW9XekppWnNicU1MZiti?= =?utf-8?B?WHd6RmRYZ0U1RXh4SUJjYTQxQWZTT3JYRXdYTXNDNVJXdi9NNGxXREFaSlc5?= =?utf-8?B?U3pHY0lKNTYvY094TVI5L0diM2FGRkJsZ2ZmVGtkMFkxRjBtSWtEa0pJY1BN?= =?utf-8?B?Z1Y5TWswemZVYjVRRnlVUU1ZS2ZxaC94cTQxQmYyeS9IVGhRbUFQMjRrRmVD?= =?utf-8?B?dEtQNDhWRnJ0UlFSS2RVNjVzTlAwMFBYdHIxTHBsNjBFTWkrZUY0eWRLWlZs?= =?utf-8?B?OGdVdXl3S0ZlTXRGZ2UvZkdGamJESitoT0N3QVZ1cHNlK2tSenNnVW1xNHFr?= =?utf-8?B?bkhrLzB6Rm9YQlNvSU43eXdaUUtuY3hXcERHTDRldWtSemQ4MjkvTFBXeW9t?= =?utf-8?B?Yjg5a0FjaFRyRUU5RHdpNWhrQlB4V2V2K0FoWWlQL082ZFkwZWNEVGticlNU?= =?utf-8?B?NVpCbUNyUDNpZnJ0azc3Vk1nR2J1R1A4QVlRRmhoTHdVSEVCYjhVVFFDOG9X?= =?utf-8?B?cE1YQmxWSXhSYlJ1NThUeUNsMU5PbDY3bUg4ZkR2akEwRUdoNHBKaXRuZjBq?= =?utf-8?B?SXZ5Y21zbElpdlRGQTZsQVlKUXcrallhZXY5VlRGN3dWMk15NDc3bkg2ZjRw?= =?utf-8?B?ckhKR0hoN3ZJa3ZxdlFmbHM3MENGU1R1UjF5SHB6UENpZ2hXc3IxK2tPNStX?= =?utf-8?B?Y0E9PQ==?= X-MS-Exchange-CrossTenant-Network-Message-Id: abb8aeec-0a2e-4e28-beaf-08da664b43b4 X-MS-Exchange-CrossTenant-AuthSource: BN6PR11MB1251.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2022 10:17:47.2564 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pbNxldTlQ3JkxpH1W04wqd7e2hmyxPEliR1RThwAARulKtOmrtIqBRmGy9exBCoV8xYuUQ40QVO0e3zebsMXEJh2oWIcLodFTBhx7sqDlUE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR11MB1279 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 14-Jul-22 11:41 AM, Asaf Sinai wrote: > As calling rte_mem_virt2iova() returns RTE_BAD_IOVA, tried to supply the > correct matching physical addresses in iova_addrs[], but it did not help > either: secondary processes still get null data in rte_mempool_lookup(). > > How does secondary process get info from primary? Via files in > /run/dpdk/rte/ ? Hi, I'm assuming the sequence of events was as follows: 1) run binary 2) fork() 3) call rte_eal_init() (note: in both primary and secondary, fork() must happen *before* EAL init!) 4) add_memory in primary, then attach_memory in secondary Can you double-check this sequence? If mempool lookup still returns NULL, that probably means that you don't have the same mempool drivers/libraries linked to your secondary process. You would have to verify that all of the same drivers/libraries are linked in both primary and secondary process. Once it is the case, at least the mempool lookup should work. > > *From:* Asaf Sinai > *Sent:* Thursday, July 14, 2022 13:25 > *To:* Burakov, Anatoly ; dev@dpdk.org > *Subject:* RE: DPDK 19.11.3 with multi processes and external physical > memory: unable to receive traffic in the secondary processes > > Hi Anatoly, > > Thanks for the comments! > > So now running with the correct initializations: first fork(), then call > rte_eal_init() in the secondary processes. > > When using hugepages, DPDK is up and running! > > Then, tried to use the external memory: > > - On primary: followed the example of create_extmem() in testpmd.c > [without calling calc_mem_size() & alloc_mem(), as we already have > virtual addresses and sizes from previous mmap() to the external memory > regions]. > >   Unfortunately, calling rte_mem_virt2iova() [which calls > rte_mem_virt2phy()] always returns RTE_BAD_IOVA. Looking at the code of that function, if you get RTE_BAD_IOVA, that means there is an error happening. Usually when something goes wrong, there are errors printed out to DPDK log, but there are two cases when that doesn't happen: when the pfn turns out to be invalid (which i assume isn't the case for you as you can see your mappings in your procfs as you have indicated in earlier mails), and when physical addresses are not available, usually because you're not running as root. Are you running DPDK as root? If not, real physical addresses will not be available, which will make it so that rte_mem_virt2phys() will return RTE_BAD_IOVA. If you are running as root, you should see the cause of the error in your EAL logs (if necessary, please run with `--log-level=*eal*:debug` to display any additional logs that might get hidden). > > - On secondaries, rte_eal_init() successfully finished, but then calling > rte_mempool_lookup() gets null data... ☹ > I'm assuming by "null data" you mean you can't look up the mempool (i.e. mempool_lookup() returns NULL)? > Do you have any idea? > > Thanks, > > Asaf > > -----Original Message----- > From: Burakov, Anatoly > > Sent: Tuesday, July 12, 2022 16:14 > To: Asaf Sinai >; > dev@dpdk.org > Subject: Re: DPDK 19.11.3 with multi processes and external physical > memory: unable to receive traffic in the secondary processes > > Hi Asaf, > > On 12-Jul-22 7:05 AM, Asaf Sinai wrote: > > > Hi, > > > > > > We run DPDK 19.11.3 with multi processes. > > > > > > When using external memory for DPDK (instead of huge pages), we are > > > unable to receive traffic in the secondary processes. > > > > > >   * We have external physical memory regions (excluded from Linux): > > > > > > /[ULP-NG]# cat /proc/cmdline/ > > > > > > /console=ttyS0,19200 isolcpus=1-127 smp_affinity=1 root=/dev/ram0 > > > rwmode=r net.ifnames=0 biosdevname=0 systemd.show_status=0/ > > > > > > /memmap=0x1000000!0x60000000 *memmap=0x90000000!0x800000000 > > > memmap=0x90000000!0x1800000000*quiet cloud-init=disabled/ > > > > > >   * We make all DPDK initializations in the primary process, including > > >     mapping the mentioned memory regions via “/dev/mem”. > > > > > > After these memory mapping, we register the external memory, as > > > described in > > > > http://doc.dpdk.org/guides/prog_guide/env_abstraction_layer.html#suppo > > > > rt-for-externally-allocated-memory > > > > > ort-for-externally-allocated-memory>, > > > and allocate memory pools: > > > > > > /.../ > > > > > > //* Map physical to virtual *// > > > > > > /int memFd = open("/dev/mem", O_RDWR);/ > > > > > > /extMemInfo[i].pageSize   = pPagesInfo->maxPageSize;/ > > > > > > /extMemInfo[i].memRegSize = 0x0000000080000000;/ > > > > > > /extMemInfo[i].memRegAddr = mmap(NULL, extMemInfo[i].memRegSize, > > > PROT_READ | PROT_WRITE,/ > > > > > > /                             MAP_SHARED, memFd, memRegPhysAddr[i]);/ > > > > > > // > > > > > > //* Create heap *// > > > > > > /sprintf(extMemInfo[i].heapName, "extMemHeapSocket_%u", i);/ > > > > > > /rv = rte_malloc_heap_create(extMemInfo[i].heapName);/ > > > > > > // > > > > > > //* Save heap socket ID *// > > > > > > /rv = rte_malloc_heap_get_socket(extMemInfo[i].heapName);/ > > > > > > /extMemInfo[i].socketId = rv;/ > > > > > > // > > > > > > //* Add memory region to heap *// > > > > > > /rv = rte_malloc_heap_memory_add(extMemInfo[i].heapName, > > > extMemInfo[i].memRegAddr,/ > > > > > > /                                extMemInfo[i].memRegSize, NULL, 0, > > > extMemInfo[i].pageSize);/ > > > > > > /.../ > > Please correct me if I'm wrong, but it seems like you're specifying NULL > as IOVA table, so that means external memory will not get IOVA addresses > set. You need to find (probably using rte_mem_virt2phys() function) the > individual page addresses, put them into an array, and pass it as an > argument to add_memory call. See documentation[1], particularly the > iova_addrs[] parameter. > > [1] > > http://doc.dpdk.org/api/rte__malloc_8h.html#af6360dea35bdf162feeb2b62cf149fd3 > > > > > > > //* Allocate memory pool *// > > > > > > /memPool = rte_mempool_create(poolName, nbufs, DPDK_MBUF_SIZE, > > > poolCacheSize,/ > > > > > > /                             sizeof(struct rte_pktmbuf_pool_private),/ > > > > > > /                             rte_pktmbuf_pool_init, NULL,/ > > > > > > /                             und_pktmbuf_init, NULL, > > > extMemInfo[i].socketId, DPDK_NO_FLAGS);/ > > > > > > /.../ > > > > > > Please note, that during calls to “/rte_malloc_heap_memory_add/”, we see > > > the following warnings: > > > > > > */EAL: WARNING! Base virtual address hint (0x2101005000 != > > > 0x7ffff4627000) not respected!/* > > > > > > /EAL:    This may cause issues with mapping memory into secondary > processes/ > > > > > > */EAL: WARNING! Base virtual address hint (0x210100b000 != > > > 0x7ffff4619000) not respected!/* > > > > > > /EAL:    This may cause issues with mapping memory into secondary > processes/ > > > > > > And after executing “/rte_mempool_create/”, physical addresses of the > > > allocated memory zones are bad: > > Yes, because you did not specify them as part of > > rte_malloc_heap_add_memory() call :) > > > > > > /[Jul 12 12:02:17] [1875] extMemConfig: heapName=extMemHeapSocket_0, > > > socketId=256, memRegAddr=0x7fff58000000, memRegSize=2415919104, > > > pageSize=2097152/ > > > > > > /[Jul 12 12:02:18] [1875] memZone: name=MP_ndPool_0, socket_id=256, > > > vaddr=0x7fffe7e7bec0-0x7fffe7ffffc0, > > > *paddr=0xffffffffffffffff-0x1840ff*, len=1589504, hugepage_sz=2MB/ > > > > > >   * In the next step, we spawn the secondary processes from the primary > > >     one, using fork(). > > > > > > This way, all DPDK data and memory mappings are the same on all > > > processes. For example: > > This is a recipe for disaster, because DPDK creates internal data > > structures that are mapped as files, and primary process expects to be > > the sole owner and user of those data structures. If you would like to > > create a secondary process using fork() at all, you should fork() first, > > and then call rte_eal_init() in the secondary process. A forked primary > > process is not what we mean when we say "secondary process". > > In addition, see documentation[1] for `rte_malloc_heap_memory_add()` - > > there's a note there that states: > > Before accessing this memory in other processes, it needs to be attached > > in each of those processes by calling rte_malloc_heap_memory_attach in > > each other process. > > So, not only you need to call secondary process EAL init, you will also > > have to call `rte_malloc_heap_memory_attach()` before you can use your > > shared external memory in your secondary process. > > Hope this is helpful! > > > > > > _Mapping in primary process:_ > > > > > > /cat /proc/1875/maps/ > > > > > > /.../ > > > > > > */7ffec8000000-7fff58000000 rw-s 1800000000 00:06 > > > 5                        /dev/mem/* > > > > > > */7fff58000000-7fffe8000000 rw-s 800000000 00:06 > > > 5                         /dev/mem/* > > > > > > /.../ > > > > > > _Mapping in secondary process:_ > > > > > > /cat /proc/2676/maps/ > > > > > > /.../ > > > > > > */7ffec8000000-7fff58000000 rw-s 1800000000 00:06 > > > 5                        /dev/mem/* > > > > > > */7fff58000000-7fffe8000000 rw-s 800000000 00:06 > > > 5                         /dev/mem/* > > > > > > /.../ > > > > > >   * Unfortunately, when traffic is received by the secondary processes, > > >     we see the following printout on the primary process: > > > > > > */i40e_dev_alarm_handler(): ICR0: malicious programming detected/* > > > > > >   * Additionally, we saw no example of using external physical shared > > >     memory on secondary processes. > > > > > > DPDK test applications show usage of anonymous private memory on the > > > primary process only. > > > > > > What are we missing? > > > > > > Can you please advise? > > > > > > Thanks, > > > > > > Asaf > > > > > > Radware > > > > > -- > > Thanks, > > Anatoly > -- Thanks, Anatoly