From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <users-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
by inbox.dpdk.org (Postfix) with ESMTP id D45E843294
for <public@inbox.dpdk.org>; Sun, 5 Nov 2023 17:01:11 +0100 (CET)
Received: from mails.dpdk.org (localhost [127.0.0.1])
by mails.dpdk.org (Postfix) with ESMTP id 426FE4029E;
Sun, 5 Nov 2023 17:01:11 +0100 (CET)
Received: from mx0a-00196b01.pphosted.com (mx0a-00196b01.pphosted.com
[67.231.149.170])
by mails.dpdk.org (Postfix) with ESMTP id F005C40278
for <users@dpdk.org>; Sun, 5 Nov 2023 17:01:07 +0100 (CET)
Received: from pps.filterd (m0072398.ppops.net [127.0.0.1])
by mx0a-00196b01.pphosted.com (8.17.1.22/8.17.1.22) with ESMTP id
3A5Aq3IS007880 for <users@dpdk.org>; Sun, 5 Nov 2023 11:01:05 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netscout.com; h=
from:to:subject:date:message-id:content-type:mime-version; s=
netscout.com.09.24.2020; bh=8RNFFEcuc9vr6tyxoJ0npfxm5RNYREpV7zBf
7cM1H+I=; b=jvDyl+zj9wTtdEAyaQMDvx1l9gjSjrRv2NPK/j2hQy5QGfTna/Q6
P9Is60ah5w/OCPvBqTd3N/KNZjbyXMnR4hm8YId2xX8qMi440L52mh7e6DiKorvn
tV2BL/5g6K0jz0mzquDFxMoocE4Sbr4h+OdRC9qcQfpNVgxDvwbpnZ5kuoagBcQ6
SsGZDVlqOvUSg5Sx4S/FIEP7PyUegpqn7dfRlZ18V5ifZK+PjBNfsitr+uP85m9c
HdTMrMuqUV54NDY4rXh/6cB9s/JP3mRpYJF4ByBAVPpM2ZNvBJiFWaRziqFZyV5z
Pnw0shSPO/kXgkOQNw6vm3YIPxiFb6xNFw==
Received: from nam10-bn7-obe.outbound.protection.outlook.com
(mail-bn7nam10lp2101.outbound.protection.outlook.com [104.47.70.101])
by mx0a-00196b01.pphosted.com (PPS) with ESMTPS id 3u5j3msbyf-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT)
for <users@dpdk.org>; Sun, 05 Nov 2023 11:01:05 -0500 (EST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=WXphltWDBUI329OJLrSOMIeSx7A33odn+topLfMzS/6sqmsoqwTRWQVLZ//NH8JmGRCxZGjoBlh2nuxWPuYbRq9i/v9FOltitWzYPh3xlEf9q4WEmnzNlWoOCZjBnSp2fec8n+trQNJK0r2n0FawC9IboclTvhXMkly8yrEZ22vBZpCEO9KQd0KpttAi/V1S4L8/hbqhxGxFbdE48klYXmP4ITfjF4BrhGuk0dGnv4uUC116Lzy0E+BPdAKj7q++L6CFf5pU3ujLMgZ8xBs4GJF3orngYqaholuJv3Ps2OrlxzqPcZoiHXWkZvhIEaMGsjw+unmtoAmAYjGrQb+mTA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=8RNFFEcuc9vr6tyxoJ0npfxm5RNYREpV7zBf7cM1H+I=;
b=LZ2KmJiZRQkzoFXv7wEZ48FMHkhc7dFhpaYN0HvIw8znrY0Y1TXObYU1obcb4VuI3tWf6CXoTOV+qLBqGce95rpQe14A4yW1t/Z3XlJNobzzw5L+V0xyXHc6soFlWBh1/Nsk3DAKtodu07kbZkH4qGFJKS6DwdzQfu1KwkmeequvKLDpz97ZHzrwQTNxv5zxTLiquIm2MhYaEGw8pob3Zu0myUdq0U2Q7jTJHhQn9A3RYnws9fJVssIuwt/5Keq3kEdTURq2aq1WRxQJUNLPmwSYzdKKk2p1Hgg4utga2YArQRNz/J9i83ZR6Bj2zLeWbhFoU917oYIR2g+Jq9kMqA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=netscout.com; dmarc=pass action=none header.from=netscout.com;
dkim=pass header.d=netscout.com; arc=none
Received: from PH0PR01MB6730.prod.exchangelabs.com (2603:10b6:510:75::16) by
MN0PR01MB7850.prod.exchangelabs.com (2603:10b6:208:37e::12) with Microsoft
SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.6954.21; Sun, 5 Nov 2023 16:01:01 +0000
Received: from PH0PR01MB6730.prod.exchangelabs.com
([fe80::f050:7c82:3f1c:2da6]) by PH0PR01MB6730.prod.exchangelabs.com
([fe80::f050:7c82:3f1c:2da6%5]) with mapi id 15.20.6954.027; Sun, 5 Nov 2023
16:00:59 +0000
From: "Lombardo, Ed" <Ed.Lombardo@netscout.com>
To: "users@dpdk.org" <users@dpdk.org>
Subject: DPDK 22.11.2 requires too much VIRT memory, how to reduce
Thread-Topic: DPDK 22.11.2 requires too much VIRT memory, how to reduce
Thread-Index: AdoP/ZNzzEBMQqwdS1qgWUB1z9DFpQ==
Date: Sun, 5 Nov 2023 16:00:57 +0000
Message-ID: <PH0PR01MB67301F1FBFD731EE7A48AD288FABA@PH0PR01MB6730.prod.exchangelabs.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: PH0PR01MB6730:EE_|MN0PR01MB7850:EE_
x-ms-office365-filtering-correlation-id: 3be5e8e2-260d-4c7c-6dde-08dbde18666a
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: DKKnzi+P6KMeUjc7fJ+0dFPJzPMyzavAJUXAf/au2ikQ+3M3tYRik+f7wCzleYyJxm2iXMlwFclwBhPIh54+1VTz7s7RSfrN3zLPnBum2or+lqlTnUbZMKpRZUEdwMGmWdDCBdHH+W/aSEOeCdEle3NTJ09xrAoZAgdnoG4HVFYTjhZ7CimtA1VQyHWD0rYHZ9vBXlGSH/ERvne2k4L7c+Y2zuiaQhr4giGZFiY1t1sT2iT52+Q8uDPGLgmzbzGzpzQuV6NIHNhwT+FH/vDmwlRyhjxXDexq4HGdwxOnPh4w7yZRx4huQjwVDlC3obx3mhbZ1gy/6aRkSjldjor6kUys25hNeD2S9vR14ylfM/wWOKqitIaJmUFqGXLcd0AITpChlPS14jtk4vebDiXd0N1M75KkbGfPdoe6wRvPWxHcyIZGMDkijiTTFgUr34h2/e0sJgw/9h7xH2S1TCH2JI0JtCSYUfCF+EfgFFtQWlUcodTFvHfRbD1yp5iRnxELYMGo9+iQbjSf84ua6h/5diuyvcaL5S8kVv/gjDv3rBOwsKoDm3NiDToG8SRmMwNRC1vy6UQCCIlVKqGhoLj5DbUo8o3YgUcSzLinYbtEGZ2D4GILAXnjO16dOQ9dkvkI
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:PH0PR01MB6730.prod.exchangelabs.com; PTR:; CAT:NONE;
SFS:(13230031)(39860400002)(396003)(346002)(376002)(366004)(136003)(230922051799003)(64100799003)(186009)(1800799009)(451199024)(38070700009)(55016003)(2906002)(122000001)(5660300002)(9686003)(83380400001)(26005)(38100700002)(86362001)(7696005)(6506007)(66946007)(76116006)(33656002)(64756008)(478600001)(71200400001)(41300700001)(66476007)(66446008)(66556008)(9326002)(8676002)(6916009)(52536014)(316002)(8936002);
DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?J1aPs1rG23/1qyzbCbVkOfGQVOoQJRxkQlgDsbdUUvZ7pMOWq1Gxn6cZFQPS?=
=?us-ascii?Q?rFzxNCdZ4YgCEAGEIoVyU2jQX1VBbPje3InLXsYOqGZoBY6IJ62YiY2o3TXv?=
=?us-ascii?Q?WzK9cXcDScVkUGcfnXdDO/R+Hbk69zhyRl4ccNpoCeaJ+p/9drxVp5NAyPHC?=
=?us-ascii?Q?1flhoOUJ24tWpVxz0Pp9oYgxwk6cjTdvW+5dwCxK8Zd7NokINw+1GoapOZcc?=
=?us-ascii?Q?9XN64on+PlFAVrxJMiViqfDTFj0Az3EtpEZ0amfWepJkKR41WGcWUxJcNPPs?=
=?us-ascii?Q?RQFcptkl42fPK3DsY25jAVKemmJ93vi4pG9mV7EW0uFEX9AzUVur1GCPMHr+?=
=?us-ascii?Q?8UNmK2Nd8+G2wgH9ZsCqoKFxi+QMUtAVk+xBHrVY3E+EHOEgnsD9tT2wn2Ll?=
=?us-ascii?Q?k+VkV1O+sQZPtOWlL89LYOHrwsqL3Zvw86nJ20W4TAi2Q4Hesm4zWWgYfaCU?=
=?us-ascii?Q?8K8C0cDKcMLpk9ZR18cWPgsW1SSGtZyMhLi27+WGBL4++NzVNDs5e7nMepr/?=
=?us-ascii?Q?nr751tS262/DVI/Ekpi52nXuX4L5KneP904X2luE0tYAOqfoAcD0d1ho/Y2K?=
=?us-ascii?Q?LbQAYu+8UTDXlr7zWSkVtt0gBAhVXrRdIyfX3BB/nO0DHNQjK7MYHSyRPhp7?=
=?us-ascii?Q?41yZEUQAGNDVYganmpzIBFRXkJsVc6IrINgAqwtXytk1Doy25u4p/eFBo2Mr?=
=?us-ascii?Q?ux3bNm0nNU2RVYDt1KUrYM00wvDZRrWwk0dSrDYOFkG8alOppmGtQNng2/fi?=
=?us-ascii?Q?HndBUh4cQdMleGcQ8d5KVK4P3Tbr8IQArXKh6dJ0Uk0cXsaxAITsxDMBQ0fJ?=
=?us-ascii?Q?PUz6Qmo4HZUWD8VUndJrto51EH2wOMHzItvqXkqyW7OMHO02+BDEersEz9n4?=
=?us-ascii?Q?JJA4NWCJhqE/t2WI5D24088MOiDt7q0Ex0FlKpMi7taPgRnxAZjBjXb8jlMn?=
=?us-ascii?Q?VkUZVOQde39x/c3T+ZM5g0dbg0XobvBhk0lBskI4j3G6A95I4RSshWBLDd67?=
=?us-ascii?Q?uOyJDVlUrHlhWKvcIGnfiDEdWt16p12J6g+JqEVAwTGMS2bZbIjqjDq0U3S8?=
=?us-ascii?Q?VPQK1At9mgBmvJQohWmw+vl5a3wYwPIMpU/EP0M1OKcp7FfbtBPrdJif/aYX?=
=?us-ascii?Q?KCw1Qq36/qyWd1ujlDK+Ts4fdQkKlCjA4d9/w7NgxbswLjPBz3cHcuriVELz?=
=?us-ascii?Q?0tfzmXPZFV2ij1prQDMftJMneMwcRDDyfKyLS/m92MihIhe6km6J9JLtW27j?=
=?us-ascii?Q?9jcAfy1X5FA8ubfuEvMi54BkrOICP/HVPzfZ6P+xVKpKK5MsTXjo43vH5pDO?=
=?us-ascii?Q?URWRLP+nErB9qDfpnt2BFaOutc9PIAvRtjVCyy8923GsXFRv6Fn3YrI9fKn2?=
=?us-ascii?Q?tkivb3f3QGIyc+Xc9GkGo9ragbjfhTSiQ719/2sMXMTv3bwqFYxZTjV4tcBr?=
=?us-ascii?Q?5CCXKUV33h4BU8K2Lahg2O/Bk1LY8pdymGCSbeeY48j0ylwwLDIPUZtznfHv?=
=?us-ascii?Q?ThKspQ7Dado0nzcGolvxQyBqRM5AcqbxOG/9hVx1olMAgjMTcp38CU1IHFvp?=
=?us-ascii?Q?Ay3Df1uHga75tDi5wdkUsYzjEAOLQtY/twg/JL9O?=
Content-Type: multipart/alternative;
boundary="_000_PH0PR01MB67301F1FBFD731EE7A48AD288FABAPH0PR01MB6730prod_"
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: ui/Ku3lz5Ee8wBkw2U/1dRRS0V088bNDsdbuLaGtl2LrpxizqkDZqMpDE8TNqVd4QIp2oFTJpWTTUbzKgwLMPYh0/mJCpnaxFzSO2+LwavmDuMQW5BU8FEEMGR+Hpvk09hFLHtE1mgE15AIvKu6A2IvAiCvipBH/PTMl7zwfMK3Gd6NaURxUQxjk3i/vh/o79rfsjh11sU9wowC0V/dPn5vwfjeNUeo6wG592vAp5epOmWTLEYPvoKTgNpnyCM7VUWgxUCSMVR7CWNB5DAjLBX43E2CeSFAiNBgpUNL1l6m8KUKDTwST17uQzKbuz564YWwu8i9trrcTJm53GcsODyMPCeFnvAM19RdDcTu4BPamQPaO1MKKgKhg0qnP2JSZWDuNFAvGbHCT+VuEa3PVHjcfZoQc2vJZscdsrh2UXPvIQcWm1Fq0ZwA5YDLzAv0tqsONkGll2NcyFT/MWyJW6wHbvMJxCOCcwkla2PVnD+IiUsmXLVqHQ/kvQbJIuOrFjbSvFD8hd1ku/KCKVFhi1J0T4aE6ghjBmpvM+ekGhN8WHz6DJUsMKOe21Mmok50JSRDq4t9zQHQHlAwTGNYHUa2TZrqCWjzf2V5fGM5jLRZ3wDvJEluUxzdGYVCxOlS59NLCDONUqb3rmIeMYaz9puoto+JV6nCN5v7npIBd/bkkn74VDrFaMRKxbNzwy9sKRHCBmwwpsfbU/onqfyoxxohmefHT3d+yNyaNL21jHpRc/f1vTHA7Afmm7FgVUICbC8J4Pj8FzKR41R3cz/iHsQ==
X-OriginatorOrg: netscout.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PH0PR01MB6730.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3be5e8e2-260d-4c7c-6dde-08dbde18666a
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Nov 2023 16:00:58.1301 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 54f11205-d4aa-4809-bd36-0b542199c5b2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: q8vCwP0nB4v82oSO5uIjwvxR1eTHs1/AjKDv/fcMc/sYwvo/7vtIWuwBNTOw4SBW+pHLYwPyJ5+bw04uxd9e8A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR01MB7850
X-Proofpoint-GUID: BHAAeEVqq05K4zfaD0SiMqElAgocOlzs
X-Proofpoint-ORIG-GUID: BHAAeEVqq05K4zfaD0SiMqElAgocOlzs
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy
score=0 clxscore=1015
priorityscore=1501 bulkscore=0 malwarescore=0 suspectscore=0
impostorscore=0 spamscore=0 lowpriorityscore=0 adultscore=0 phishscore=0
mlxscore=0 mlxlogscore=752 classifier=spam adjust=0 reason=mlx scancount=1
engine=8.19.0-2310240000 definitions=main-2311050139
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
<mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
<mailto:users-request@dpdk.org?subject=subscribe>
Errors-To: users-bounces@dpdk.org
--_000_PH0PR01MB67301F1FBFD731EE7A48AD288FABAPH0PR01MB6730prod_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Hi,
I upgraded from DPDK 17.11 to DPDK 22.11.
VM host is running Oracle 9 with kernel 5.14.0-162 in VMWARE 7.03 and PCI P=
assthrough.
Was able to detect the NIC ethernet port with DPDK API (rte_eth_dev_count_a=
vail()), however I see problems when do mmap in our application just after =
the DPDK initialization.
[root@vstreamOR9-160 bin]# /opt/dpdk/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
0000:0b:00.0 'Ethernet Controller E810-XXV for SFP 159b' drv=3Digb_uio unus=
ed=3Dice
Network devices using kernel driver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' if=3Deth0 drv=3Dvmxnet3 unu=
sed=3Digb_uio *Active*
Found errors when application is started with gdb:
EAL: Detected CPU lcores: 16
EAL: Detected NUMA nodes: 1
EAL: Detected static linkage of DPDK
[New Thread 0x7f60c8cb0640 (LWP 3872)]
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
[New Thread 0x7f60c84af640 (LWP 3873)]
EAL: Selected IOVA mode 'PA'
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open file=
s
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open file=
s
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open file=
s
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open file=
s
EAL: rte_mem_virt2phy(): cannot open /proc/self/pagemap: Too many open file=
s
EAL: eal_memalloc_alloc_seg_bulk(): couldn't find suitable memseg_list
EAL: FATAL: Cannot init memory
EAL: Cannot init memory
This may be a hint why mmap fails?
Tried the following:
* increase the VM memory from 16 GB to 32 GB,
* increase the amount of hugepages from 2GB to 4GB,
* ulimit -n shows open files =3D 1024, had to increase to 2048 to elimi=
nate the rte_mem_virt2phy() errors above when start app with gdb.
* free shows 62579808 free and available
* htop shows our application VIRT value is 66 GB. What a surprise to s=
ee this value so high. With DPDK 17.11 this value is around 5.5 GB.
* pmap -d <app PID> shows there are 4 [ anon ] of 16777216K each, which=
is what I believe is bumping the VIRT memory.
Was able to get our application to process packets from E810 once I changed=
the VM total memory to 80GB.
In HTOP the Mem value shows 5.28G/78.5G and VIRT is 69.0 GB.
When the main application executes rte_eal_init() is when the VIRT memory j=
umps to 66 GB, this is before the application configures the DPDK resources=
(worker rings, port configuration, etc).
Is there a setting in meson to reduce the DPDK resource in our application?
See the same VIRT memory issue on bare metal with same Oracle 9 and Intel I=
226 MAC. I have systems that have only 16 GB of memory and a VM requiring =
80 GB is not acceptable for our customers.
Any help will be appreciated.
Thanks,
Ed
--_000_PH0PR01MB67301F1FBFD731EE7A48AD288FABAPH0PR01MB6730prod_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1732650675;
mso-list-type:hybrid;
mso-list-template-ids:1161350398 -339453212 67698691 67698693 67698689 676=
98691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72" style=3D"word-wrap:=
break-word">
<div class=3D"WordSection1">
<p class=3D"MsoNormal">Hi,<o:p></o:p></p>
<p class=3D"MsoNormal">I upgraded from DPDK 17.11 to DPDK 22.11. <o:p=
></o:p></p>
<p class=3D"MsoNormal">VM host is running Oracle 9 with kernel 5.14.0-162 i=
n VMWARE 7.03 and PCI Passthrough.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Was able to detect the NIC ethernet port with DPDK A=
PI (rte_eth_dev_count_avail()), however I see problems when do mmap in our =
application just after the DPDK initialization.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">[root@vstreamOR9-160 bin]# /opt/dpdk/dpdk_nic_bind.p=
y --status<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Network devices using DPDK-compatible driver<o:p></o=
:p></p>
<p class=3D"MsoNormal">=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D<o:p></o:p></p>
<p class=3D"MsoNormal">0000:0b:00.0 'Ethernet Controller E810-XXV for SFP 1=
59b' drv=3Digb_uio unused=3Dice<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Network devices using kernel driver<o:p></o:p></p>
<p class=3D"MsoNormal">=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<o:p></o:p></p>
<p class=3D"MsoNormal">0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' if=
=3Deth0 drv=3Dvmxnet3 unused=3Digb_uio *Active*<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><b>Found errors when application is started with gdb=
:<o:p></o:p></b></p>
<p class=3D"MsoNormal">EAL: Detected CPU lcores: 16<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Detected NUMA nodes: 1<o:p></o:p></p>
<p class=3D"MsoNormal">EAL: Detected static linkage of DPDK<o:p></o:p></p>
<p class=3D"MsoNormal">[New Thread 0x7f60c8cb0640 (LWP 3872)]<o:p></o:p></p=
>
<p class=3D"MsoNormal">EAL: Multi-process socket /var/run/dpdk/rte/mp_socke=
t<o:p></o:p></p>
<p class=3D"MsoNormal">[New Thread 0x7f60c84af640 (LWP 3873)]<o:p></o:p></p=
>
<p class=3D"MsoNormal">EAL: Selected IOVA mode 'PA'<o:p></o:p></p>
<p class=3D"MsoNormal"><b>EAL: rte_mem_virt2phy(): cannot open /proc/self/p=
agemap: Too many open files<o:p></o:p></b></p>
<p class=3D"MsoNormal"><b>EAL: rte_mem_virt2phy(): cannot open /proc/self/p=
agemap: Too many open files<o:p></o:p></b></p>
<p class=3D"MsoNormal"><b>EAL: rte_mem_virt2phy(): cannot open /proc/self/p=
agemap: Too many open files<o:p></o:p></b></p>
<p class=3D"MsoNormal"><b>EAL: rte_mem_virt2phy(): cannot open /proc/self/p=
agemap: Too many open files<o:p></o:p></b></p>
<p class=3D"MsoNormal"><b>EAL: rte_mem_virt2phy(): cannot open /proc/self/p=
agemap: Too many open files<o:p></o:p></b></p>
<p class=3D"MsoNormal"><b>EAL: eal_memalloc_alloc_seg_bulk(): couldn't find=
suitable memseg_list<o:p></o:p></b></p>
<p class=3D"MsoNormal"><b>EAL: FATAL: Cannot init memory<o:p></o:p></b></p>
<p class=3D"MsoNormal"><b>EAL: Cannot init memory<o:p></o:p></b></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">This may be a hint why mmap fails?<o:p></o:p></p>
<p class=3D"MsoNormal">Tried the following:<o:p></o:p></p>
<ul style=3D"margin-top:0in" type=3D"disc">
<li class=3D"MsoListParagraph" style=3D"margin-left:0in;mso-list:l0 level1 =
lfo1">increase the VM memory from 16 GB to 32 GB,
<o:p></o:p></li><li class=3D"MsoListParagraph" style=3D"margin-left:0in;mso=
-list:l0 level1 lfo1">increase the amount of hugepages from 2GB to 4GB,<o:p=
></o:p></li><li class=3D"MsoListParagraph" style=3D"margin-left:0in;mso-lis=
t:l0 level1 lfo1">ulimit -n shows open files =3D 1024, had to increase to 2=
048 to eliminate the rte_mem_virt2phy() errors above when start app with gd=
b.<o:p></o:p></li><li class=3D"MsoListParagraph" style=3D"margin-left:0in;m=
so-list:l0 level1 lfo1">free shows 62579808 free and available<o:p></o:p></=
li><li class=3D"MsoListParagraph" style=3D"margin-left:0in;mso-list:l0 leve=
l1 lfo1">htop shows our application VIRT value is 66 GB. What a surpr=
ise to see this value so high. With DPDK 17.11 this value is around 5=
.5 GB.<o:p></o:p></li><li class=3D"MsoListParagraph" style=3D"margin-left:0=
in;mso-list:l0 level1 lfo1">pmap -d <app PID> shows there are 4 [ ano=
n ] of 16777216K each, which is what I believe is bumping the VIRT memory.<=
o:p></o:p></li></ul>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Was able to get our application to process packets f=
rom E810 once I changed the VM total memory to 80GB.<o:p></o:p></p>
<p class=3D"MsoNormal">In HTOP the Mem value shows 5.28G/78.5G and VIRT is =
69.0 GB.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">When the main application executes rte_eal_init() is=
when the VIRT memory jumps to 66 GB, this is before the application config=
ures the DPDK resources (worker rings, port configuration, etc).<o:p></o:p>=
</p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Is there a setting in meson to reduce the DPDK resou=
rce in our application?<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">See the same VIRT memory issue on bare metal with sa=
me Oracle 9 and Intel I226 MAC. I have systems that have only 16 GB o=
f memory and a VM requiring 80 GB is not acceptable for our customers. =
;
<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Any help will be appreciated.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Thanks,<o:p></o:p></p>
<p class=3D"MsoNormal">Ed<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>
--_000_PH0PR01MB67301F1FBFD731EE7A48AD288FABAPH0PR01MB6730prod_--