From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f177.google.com (mail-io0-f177.google.com [209.85.223.177]) by dpdk.org (Postfix) with ESMTP id 1FECC29D6 for ; Tue, 7 Feb 2017 11:20:22 +0100 (CET) Received: by mail-io0-f177.google.com with SMTP id j18so86349394ioe.2 for ; Tue, 07 Feb 2017 02:20:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=TwaN/4HQ8eDtqYn+PjISaRKMSwFlvvGPjAiXsovVnZc=; b=eEtGtiy2XXoZm2tduuCmG2fGoL7w0lUb6Jblq3xe58kWLVXW2xG81PwOGRaq9AYN2q D5unsanMNV1tLt1eL07XEFY4iu9NKT3Ek32NFL+Ub4ZNfM8Lkm7K8CF1tZz/p/KfTBi7 7+pNuXj+7dXEI7toqVZYs4P/Q/R0Naj0MR88zl3Z4ijR1dQQONRvAXagorFIt2p+2wKY rJGXucqxad00PqSxHjJ5kLhrjMhHvh7CWN4lCeFI3yzqYUDrEi/5JNpN+/xen1xOx0LU nwWDMLwVA1wfv70r7d1nEkAogapq1b7u1V0Z5iNGXtRsWNZ2gTGjy822kUAvnP9ARLY6 LoyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=TwaN/4HQ8eDtqYn+PjISaRKMSwFlvvGPjAiXsovVnZc=; b=Osjz+h9IrJ1JTjffMcb4BV/pTheBy5bCRUmvvMHElLXnxB5xW4ZP7rK3GmKlKI6ZYO y1YGOrrPGAYmJSjRxYEUAeJXJm11Hkoinro1P7uRT1KWDMemyVqKLwIkzeLlL9ecFayQ oEjS1X2Xp6mwKjmlN9N1Nq+lrgv6zoae8CBogd/PBKH/mxpB6Couhsuu54hNCVUY5y0V n711fftGHDcaQLPJlM4SRef9hl7m8DPRGnHIIKhkrC97t92LlnLLpdx7nAA/x7oDGm9T GrnOPuEz9F/ztaHAKYxMGqnJB7n3U6Dh39IYOEm0I47Jxg529zLnn+2x43cMsiW66LIM QQiw== X-Gm-Message-State: AMke39ls2Uo576AV32OtxGwk2CB/K9Aazr+6+U4g8Rl7dKHq+JdaZjzyQVd3k679hphwnjk2X5Gr3LWEIBcFeg== X-Received: by 10.107.47.132 with SMTP id v4mr3816927iov.172.1486462821347; Tue, 07 Feb 2017 02:20:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.64.135.161 with HTTP; Tue, 7 Feb 2017 02:19:50 -0800 (PST) In-Reply-To: References: From: Ivan Nardi Date: Tue, 7 Feb 2017 11:19:50 +0100 Message-ID: To: "Zhang, Helin" Cc: "dev@dpdk.org" , "Wu, Jingjing" Content-Type: text/plain; charset=UTF-8 Subject: Re: [dpdk-dev] i40e and memory allocations restricted on node 1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Feb 2017 10:20:22 -0000 Hi Helin no luck with the latest nvm PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.05 eetrack 800028ac RING: Cannot reserve memory HASH: memory allocation failed PMD: i40e_init_ethtype_filter_list(): Failed to create ethertype hash table! EAL: Error - exiting with code: 1 It seems to me the issue is that i40e code try to allocate hash tables (for tunnel filter list, fdir filter list and ethtype filter list) from node 0, because it doesn't explicitly set the field socket_id in struct rte_hash_parameters. If my analysis is correct, I am wondering if it should allocate memory from SOCKET_ID_ANY of from the exact node associated with the network device Thanks for looking into that Ivan On 7 February 2017 at 02:49, Zhang, Helin wrote: > > >> -----Original Message----- >> From: Ivan Nardi [mailto:nardi.ivan@gmail.com] >> Sent: Tuesday, February 7, 2017 5:04 AM >> To: dev@dpdk.org >> Cc: Zhang, Helin; Wu, Jingjing >> Subject: i40e and memory allocations restricted on node 1 >> >> Hi >> With dpdk 17.02-rc2 i40e driver doesn't load at all when memory allocation is >> restricted on numa node 1 (on system with more than 1 node, obviously) >> >> >> [root@micro ~]# /tmp/testpmd -d /home/micro/lib/librte_pmd_i40e.so -c >> 0xFFFC000 -w 0000:81:00.0 -w 0000:81:00.1 -n 4 --socket-mem=0,8192 -- -i -- >> socket-num=1 >> EAL: Detected 56 lcore(s) >> EAL: Probing VFIO support... >> EAL: PCI device 0000:81:00.0 on NUMA socket 1 >> EAL: probe driver: 8086:1572 net_i40e >> PMD: eth_i40e_dev_init(): FW 4.40 API 1.4 NVM 04.05.03 eetrack 80001cd8 > This is a very old verion of firmware, it may have compatibility issue. > So first, firmware upgrade is needed. > >> RING: Cannot reserve memory > It seems not reserve memory for ring, and not an issue specifically to i40e. > Let's double check that if it is common to all. > >> HASH: memory allocation failed >> PMD: i40e_init_ethtype_filter_list(): Failed to create ethertype hash table! >> EAL: Error - exiting with code: 1 >> Cause: Requested device 0000:81:00.0 cannot be used >> >> >> Everything is fine with 16.11, or allowing allocations from both nodes >> >> I was able to locate the code to fix (at least, I hope so), but I don't really >> know if it is better to allocate from SOCKET_ID_ANY of from the exact node >> associated with the network device, something like >> dev->data->numa_node ("pseudo" patch below) >> >> >> --- dpdk-17.02-rc2.orig/drivers/net/i40e/i40e_ethdev.c 2017-01-30 >> 23:47:11.000000000 +0100 >> +++ dpdk-17.02-rc2/drivers/net/i40e/i40e_ethdev.c 2017-02-06 >> 21:53:37.812313120 +0100 >> @@ -899,6 +899,7 @@ >> .entries = I40E_MAX_ETHERTYPE_FILTER_NUM, >> .key_len = sizeof(struct i40e_ethertype_filter_input), >> .hash_func = rte_hash_crc, >> + .socket_id = SOCKET_ID_ANY, >> }; >> >> /* Initialize ethertype filter rule list and hash */ @@ -942,6 +943,7 @@ >> .entries = I40E_MAX_TUNNEL_FILTER_NUM, >> .key_len = sizeof(struct i40e_tunnel_filter_input), >> .hash_func = rte_hash_crc, >> + .socket_id = SOCKET_ID_ANY, >> }; >> >> /* Initialize tunnel filter rule list and hash */ @@ -985,6 +987,7 @@ >> .entries = I40E_MAX_FDIR_FILTER_NUM, >> .key_len = sizeof(struct rte_eth_fdir_input), >> .hash_func = rte_hash_crc, >> + .socket_id = SOCKET_ID_ANY, >> }; >> >> /* Initialize flow director filter rule list and hash */ >> >> >> >> Any thoughts? >> Thanks in advance >> Ivan