From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0062.outbound.protection.outlook.com [104.47.41.62]) by dpdk.org (Postfix) with ESMTP id A08947D19 for ; Fri, 17 Nov 2017 18:51:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multapplied.onmicrosoft.com; s=selector1-multapplied-net; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=pRKQyywi5jW7cyfCXCnOw2gNxS54hZPyOS6tF0zuEOI=; b=Ymlb2/xK3PRR/x6suI76hh1K2hWqXw+S+XV/NqlDYjCsZWq8psA4JFGNbFCUV5k7exeu2aPbFbwg1ru8dA2khmj4deFOWE7DYtHQuCSlKxDUxYN13jzuBsNZFk7ACzB5+fne21aDrIXDLPrtCANDkZ1AfH73pOQqB9bgDVubPuY= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=ian.trick@multapplied.net; Received: from [10.100.0.160] (74.121.35.88) by DM5PR17MB1979.namprd17.prod.outlook.com (10.173.129.13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.239.5; Fri, 17 Nov 2017 17:51:22 +0000 To: "Dumitrescu, Cristian" , "users@dpdk.org" References: <3EB4FA525960D640B5BDFFD6A3D891267BAE4DD8@IRSMSX108.ger.corp.intel.com> From: Ian Trick Message-ID: <3968f1fb-0969-8cbc-3155-d5862d74560d@multapplied.net> Date: Fri, 17 Nov 2017 09:50:27 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <3EB4FA525960D640B5BDFFD6A3D891267BAE4DD8@IRSMSX108.ger.corp.intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [74.121.35.88] X-ClientProxiedBy: MWHPR21CA0040.namprd21.prod.outlook.com (10.175.142.154) To DM5PR17MB1979.namprd17.prod.outlook.com (10.173.129.13) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 730372d6-ec97-49ba-321a-08d52de3d0ff X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(4534020)(4602075)(4603075)(4627115)(201702281549075)(2017052603258); SRVR:DM5PR17MB1979; X-Microsoft-Exchange-Diagnostics: 1; DM5PR17MB1979; 3:ysk5AYwtouYxPYeWE3vFuJ+o0Bo7O1mMEfvPPWHzusEsZkykrlCL4EGRMA6MXID2Qb5u5g84VS/W2gCziNqdR4+swFRZYduxuwebH5/+qYbo4ha6N3Jnzca0Unhz+tiwlh1k7kcKCgUh4yeKCGfc2PffmKQ/6KzNX1ov+ieyTubq5Q2blnBCUeT80YYT4ON4uY3XmuNnvVhwMvuZgSRCY2iEOlyLOxT/MO09Hnxo7vi17WHrCkt6O1ydI45Z+Exu; 25:Z8ZFWa28mo6n3zPhD4baUzfLn6/n5NDnPXKyoUFRgQFPWeK4qJC+7339oF2Ov3GoEIldnsLXscTSIn+lShA2P+o3JANi8yp6JbO8dXxJtn2y+2Qfx787bygI08t5xGaSm0v3YKCGfgZz0bbdTJld0wZqoIhfDyL9q3/aF/yo5ocT/Eq5RJ+0tt4/oFNbcbU+rTQEin5vgkgx0/RXeugcn9q9pEr3/7qLDQKFOZ4v3cpdxoTN4ydkSzkB2O8VoDfYr+xYLIJTUUHYz4zNqULxjLdP2faWiVmoqALwRPNQtm3vrmcgFHWO7owOJPZvLMcm6B2Ony9jfslprZW4SQvmuw==; 31:YBT14+2yXBg/H2jJz/FP0AfhPQhFSOY8BH8ztzn+nKigHTss2Y+mhh6+Yml2ITVoAwXq/w9XIe3eSBhCc2jkSbjQ0P3SS+K/ONji9wNys3fCJKwpcKk/fqvcVp0VDkDtrHgFYfXaoV29xE6h9FsWlf6SawOs4LrJ0bMK4MD2vXAj+3E4uHCAkDXF82+6sEpXCsr0L+vpdV7CQ76WTzOLJ3e1vyGliXLYUByBFy0Ypm8= X-MS-TrafficTypeDiagnostic: DM5PR17MB1979: X-Microsoft-Exchange-Diagnostics: 1; DM5PR17MB1979; 20:C6HM8xpge03L4JkMjB7MNnE2ds+17iEh20hCVDZYbkO9F4KAAncBEJX57TlEX5E3C53jZzEUqLVFe6JpzbrmX8i2xRNeRqxjECzHvGK9fmZbw8WkvdbbUrTxe4AIlfnGSq6dZOA6CfQhVGrXJtgQsrk7G6W6n7vuplNWsGhVvViU6mbA4xMdj895QI/QPgHdFBNd3itcA4E7BW7+m9FiOyWOGj3Yhbz+uAkkq1MiNYNlL+AQ9nSVWW6z5D8hiqT+; 4:spoMwJrzWv/nnZ1Ouqkob7TkK5WKI9ivGPoWbRUCuFX87RiWJK9MMKfRjivLb7W3pqUZGX3VXsnpP2mRwZrgPt2T3dLA3G8Nbc21ReLgCOiQjWNtCZDifF6XRhE9bLWQhG4amR+/t+oGqk4vfT6xqqVN5kxQcxesITer+phsdGBHsZ+oSC95xBMK8VsUX8ywQBTBeLkUymjipjhZ+hmWsVkeV3aiT0i27TvsRseXRPbkz2nxYZX/9lTU0dJDY4WjhaVbnOPC5DGmdMlFaCTFel7HB1XzTBzmE3v2Nk6RchqRnx5GE5tXxmJe9uCv3inN X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3231022)(3002001)(100000703101)(100105400095)(6041248)(20161123562025)(20161123564025)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(2016111802025)(20161123555025)(20161123558100)(6043046)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM5PR17MB1979; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM5PR17MB1979; X-Forefront-PRVS: 049486C505 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6049001)(6009001)(39830400002)(346002)(376002)(13464003)(199003)(189002)(24454002)(377424004)(230700001)(8676002)(7736002)(33646002)(16526018)(50466002)(6116002)(3846002)(36756003)(4001150100001)(53546010)(31696002)(81156014)(81166006)(53936002)(64126003)(86362001)(5660300001)(2950100002)(2906002)(305945005)(65826007)(68736007)(47776003)(76176999)(50986999)(8936002)(54356999)(97736004)(105586002)(83506002)(106356001)(189998001)(110136005)(16576012)(229853002)(58126008)(316002)(6486002)(31686004)(25786009)(77096006)(2501003)(101416001)(66066001)(65956001)(65806001)(478600001)(90366009)(6246003)(23676003); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR17MB1979; H:[10.100.0.160]; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: multapplied.net does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtETTVQUjE3TUIxOTc5OzIzOmtlYm1OR0huQ1oyakhWMm5VbUlQK1Y3UGhl?= =?utf-8?B?V1FpRVh0ci95YUNFTEJndzdkaklpQmV0SWJRd3pJWWJ4TGt5MGJrK2tnTlJM?= =?utf-8?B?eDJyd0dPRGNma2QrTk93T214dHU4TFpzemNJWVZnS1ZueXNUL0c3SnRaNEJ1?= =?utf-8?B?VlBkME5yYmllY0RtbitOVUVtR3d2UGhXWmN4K1UrSlBGdXJOUVJpUm9mVFF3?= =?utf-8?B?aUwvWTBpZFJUSlNZV21MazQ0c2VZMUVyUEJtTWdSek94aFZISTVMZDAxSlBT?= =?utf-8?B?dTBiZDBEN1RpY1pkVXg2eU1pd250SGtOZVNlTFdNU3VMaEtXVWxjbGxMZUh0?= =?utf-8?B?ZmxWOFRiOE91d1lmTnJESWh1MURSeWYwaTFkaWpoalNVOGdsNk5WK2VKR0s5?= =?utf-8?B?QWxmeFR4ODAyWGxrQnBoRWttQmo5ZGlDdHp5enVyTGhXRmpNcWpmZXE1UUYy?= =?utf-8?B?ZTZRRmYwSm1GM2hYOUVqcVlnTjNxVXJaSXYrN2Y1bDE5MnBSbEVNaVg1NDV6?= =?utf-8?B?U2MyVGJna05ONi9aZ3NRd01HVktNNUR2dmxWb3ZyQ1RiUEo1VnJsLy9QOEgr?= =?utf-8?B?NDJkWVJqUHd3TUp5aFNZQ0ZMcUZtNkVnWngxajM4Wk11SHZCdWJNSHJVellH?= =?utf-8?B?SlZUeVFrSllXZmRhQ1IrRmVteEpvNWlVdnVEWStRYnErT1R2KzA2TEZ0ZVlt?= =?utf-8?B?Kzg3Ukt0RDVUZ2ZoMTZiS2JDSEFxT0VaOXUxOFZRQXJuQkVlb0RDZTBXOVpN?= =?utf-8?B?c2h0cDdoUkh4ZDFndnBxVzZzZnNnR1JPUit3ZDQyWFhWOC8rWE5hQ3RRcTRy?= =?utf-8?B?TzlHZkhMcDBIN3VLRkZkdmRobCtETjhldTFIa0xjTFF5aVVNeU11ZmliM0dS?= =?utf-8?B?Mlg4NFFWRzVSdS8rYm1reDQ0U1EyTDltQTlMbHBpcDRobFNCOXptbitSclVm?= =?utf-8?B?RG9jd3grMlRpTEFsY0xJbm91NWxCTTJmOWJDOTdLLzdKMjNZSk5ncVdLQ1d6?= =?utf-8?B?cXpvMkpMVHVZUnQ4N0l5MFMvNGY5SlF3a3RVTFJ1OTJyS3ZMd3QvZllhUXZ5?= =?utf-8?B?RWFkZlR3RDR5WlVuV1BCWXVLd3QzS2FkazM2Y3NzYVJYQnF1L0dqb1NSVUYw?= =?utf-8?B?OG5INDQwT0FnUU5BWFhmQTl2TjQzSFhZSldWUTYvcTFUekxSb0sxUW9XMis2?= =?utf-8?B?WWUzQVN4TUFrbzh5TFpxNkVkc0VVZnF5Qmh5NTVGb2UvaVpXdElRK3JNT1d4?= =?utf-8?B?d3RZVjZtMFdaSk50RHBBTUJhc1d0R01CKzF2Q2JBd3hTdVhSQ1Q0TkJmL3Qx?= =?utf-8?B?a3RManRJRUswYTE3YVNDSlBJL00xZVN5a1NsYm5IYW53dEwySlJEN3kvMWF5?= =?utf-8?B?Z1BvTzZjbXpqVkhodUdCa1NlckUzdXRETTVISjhGcWRLck1jeDJrOGlScGt3?= =?utf-8?B?Z2JSRDFJSWhMMzhMZGhkV1dEZzFzUTk1ZUpPZUkzbDJ0ZGlzSGlvNVhzd0Ny?= =?utf-8?B?MEs5bDAxS2pwRWRsdzg3T084WENlaGtLNU43by93VEpxeGxNVTg0NTh0QWp4?= =?utf-8?B?UFU3VlhBZXgvbkZBVCtYaVVacVhuR29LVlZWQVhocHZJK1grdW53bHNYZ3Vu?= =?utf-8?B?SlpQZUl1dnhzU1B1cFhCUmpZaWZnMHhYUk9WR1dSUWhwcldLNkkvTitWYnFs?= =?utf-8?B?SDBMK0pHZmgrcDExRlNxb3l5ekZSa0JUVHM4aDNMWGNRbUdVSGJPNTEvT2Jp?= =?utf-8?B?Wkx3S2pBWlFvNVpnTkVEdHZsS29XZGliZ0lvSTM0VTJ2Mm9JdDNRS01VTERG?= =?utf-8?B?c2ZkMUV1MG9zY1VIVEwxdWcyR0lUTkhrVitsd0hJUUdoUmQrNzhPczQzeU9K?= =?utf-8?B?SzFsN2NXVVl4VjB6MWhvVHkzSHY3bnNSa3ZpQWVuMi9uKys2L25Sb2J1REZz?= =?utf-8?B?ajFoM3MzWjNBPT0=?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR17MB1979; 6:FpUKnXFD2kK2lRwZruSsOOs3p43T7QhDHlfAFYL5EP8PsPLPxfg3vkWYznLSIrfcQVFhUdhd8v325fwJ1NaDYmSJPjSoZvURvzT1ExTPaHR5DQoFLJ6q1+SiJojGiClkkNHLCA2L+BQ/yQD+1qv2ifEOdrQI0r4RR/V2jZYHg2Fih7Mqm+cDluvtpnQOOJQdQfUNN1Tvbo4C10VoHSavyk1GOPWAs4D1MeuTF3d+h3frOPRfN6Y1VSaCXSDbOla5K4ig+5ysWYsBRJJx1VAbrer1xkyg5A01RKRK4kCpNALrHEN4k3uhZSd9CvhMzqmdijLuXf1pUbFQdnbsU+eo7faARyCEQdg2yAbknRZrzvE=; 5:WJK+B8Jjq8S6jqDG1sLqGIMtClQU3MMZKYZAadzGydhf9V6v2afhF92ULoxcfXLh6W6sQtuq4Ad+OwaMNealzd7V3SS8+Kq0rzAsqc8TIw5T8z+ZlI4H6ibcxx4beY3XXVbHZceHHH9YjiDCeFUGLYkpYq5e/h8z39t8dNf4U/I=; 24:+UbJy/u+HX62f4Fqx3sfyM4xR8oylx6mDXCk6NCtmJBI4hxBWV/25Vgdqm2n94aeswepu8Tt+IOcjbHCmjFqdkJtAR0jr26HzLEHjk7RdDg=; 7:Q9AFa2FwuQe167tlgOu5uyVdx+cxa+zW53RycE3Nqs6g0vT3kFp1xOUyM6muxZBSuQjpge6Km//6vgeo9zV5gyf3KLe/is1yS6nJMGI04DRD271GI+zTcOCtA0+A3keXXEb4CbYl21t5782rXO1xFYtzBWk0HxyzAeniNr734jUQKXfRWdgrAGbPIz1Q+IPJNsPpwgwC6VH8MWNOyIMBUJ/AiECgj716eHVCHVtWt63kt0NtCpVRWwBDelKAACOz SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: multapplied.net X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2017 17:51:22.6879 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 730372d6-ec97-49ba-321a-08d52de3d0ff X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 10c26ea1-9e95-414d-91d0-c44adf533c85 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR17MB1979 X-Mailman-Approved-At: Wed, 22 Nov 2017 09:48:16 +0100 Subject: Re: [dpdk-users] qos_sched in DPDK 17.11.0 fails to initialize mbuf pool X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2017 17:51:26 -0000 On 2017-11-17 04:19 AM, Dumitrescu, Cristian wrote: > > >> -----Original Message----- >> From: Ian Trick [mailto:ian.trick@multapplied.net] >> Sent: Friday, November 17, 2017 1:24 AM >> To: users@dpdk.org >> Cc: Dumitrescu, Cristian >> Subject: qos_sched in DPDK 17.11.0 fails to initialize mbuf pool >> >> Hi. I'm having an issue starting the qos_sched example program. >> >> # ./examples/qos_sched/build/qos_sched --no-huge -l 1,2,3 --vdev >> net_af_packet0,iface=eth1 -- --pfc "0,0,2,3" --cfg >> examples/qos_sched/profile_ov.cfg >> >> EAL: Detected 16 lcore(s) >> EAL: Probing VFIO support... >> EAL: Started without hugepages support, physical addresses not available >> EAL: PCI device 0000:08:00.0 on NUMA socket -1 >> EAL: Invalid NUMA socket, default to 0 >> EAL: probe driver: 8086:10d3 net_e1000_em >> PMD: Initializing pmd_af_packet for net_af_packet0 >> PMD: net_af_packet0: AF_PACKET MMAP parameters: >> PMD: net_af_packet0: block size 4096 >> PMD: net_af_packet0: block count 256 >> PMD: net_af_packet0: frame size 2048 >> PMD: net_af_packet0: frame count 512 >> PMD: net_af_packet0: creating AF_PACKET-backed ethdev on numa socket 0 >> EAL: Error - exiting with code: 1 >> Cause: Cannot init mbuf pool for socket 0 >> > > Personally I never used this application with --no-huge or with AF_PACKET, so I suggest you start from the configuration known to work (as detailed in the Sample App Guide) and then change/add one variable at a time to see which change triggers the mempool issue. > > This app needs large amounts of memory for the mempool, as traffic management is buffering lots of packets in lots of queues. Out typical tests are done with 4K pipes/output port (64K queues/output port) so we provision mempool to have 2M buffers for each output port. The size of the mempool is hardcoded in the application. Can I configure this to run with fewer queues or something so that it requires less memory. I thought running with profile_ov.cfg might have lower memory requirements since it includes: > number of pipes per subport = 32 compared to 4096 in the other configuration file. So I figured there would be fewer queues and buffers? But I only have 4GB available on the device I have if I want to test something that isn't AF_PACKET. > >> >> This is version 17.11.0 from the repo. My RTE_TARGET is >> x86_64-native-linuxapp-clang. eth1 is a veth. I've tried running with >> `-m` and using a low value but the issue still happens. >> >> From what I can tell, rte_pktmbuf_pool_create() is failing and rte_errno >> is set to EINVAL. >> >> In librte_mempool/rte_mempool.c, the function >> rte_mempool_populate_virt() is succeeding this test and returning -EINVAL: >> >> if (RTE_ALIGN_CEIL(len, pg_sz) != len) >> return -EINVAL; >> >> In that context, len is mz->len, the length of a memzone passed by the >> caller, rte_mempool_populate_default(). Which got it here: >> >> mz = rte_memzone_reserve_aligned(mz_name, size, >> mp->socket_id, mz_flags, align); >> /* not enough memory, retry with the biggest zone we have */ >> if (mz == NULL) >> mz = rte_memzone_reserve_aligned(mz_name, 0, >> mp->socket_id, mz_flags, align); >> >> This fails the first call, and succeeds the second when it passes 0 as >> the size. memzone_reserve_aligned_thread_unsafe(), in >> librte_eal/common/eal_common_memzone.c, gets the length this way: >> >> requested_len = find_heap_max_free_elem(&socket_id, align); >> >> So the align value is 4096. But the value returned by >> find_heap_max_free_elem() isn't aligned to that -- I think? Since it >> fails the check later on. >> >> I'm not sure if this is a thing with my environment where I don't have >> enough memory? (Although I would have expected a different error for >> that.) Or I don't have the right program arguments? Or one of these >> functions isn't doing what it's supposed to?