From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BEC4945E5B; Mon, 9 Dec 2024 12:03:21 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5EA304025F; Mon, 9 Dec 2024 12:03:21 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id E0D564025D for ; Mon, 9 Dec 2024 12:03:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1733742198; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ictTVHGhuEdTbd/XCRvEEpSv8rH0mXDhmsqwaCuEWYY=; b=RjGixtXRp3o7Jw/HQpW6QlW24NtdR9yd/xdFi3pk/1v42y0nb7gGBFSj9rz53wfwuenffx 80OTixQ4iZmX3ahiHR3fbEI9+LcpWWHqRXopAWpPzkdyhIAdkJTmvHQkcjLlrHTffZufCu Y1zvGzT5vrwIcHpJnFOFgsjrj04bqrs= Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-524-OYRZWljwNdyonbAF3J17ow-1; Mon, 09 Dec 2024 06:03:17 -0500 X-MC-Unique: OYRZWljwNdyonbAF3J17ow-1 X-Mimecast-MFC-AGG-ID: OYRZWljwNdyonbAF3J17ow Received: by mail-lf1-f71.google.com with SMTP id 2adb3069b0e04-5401b7d34ceso580497e87.0 for ; Mon, 09 Dec 2024 03:03:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733742195; x=1734346995; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ictTVHGhuEdTbd/XCRvEEpSv8rH0mXDhmsqwaCuEWYY=; b=pos9C5wiJXjfVMeD/iaC3Z63+V+LoHlUJcxCaYaOKU1mESU3QNVqi/l1kwPeyCzarY rKKN2enx2Ux8Z1qWyQmQI7UPua01xtvmCx3IxuQ2YhQK/QxQ8SL6VRMCdZMxj+Xbk44/ g/ZpXQtWfR4XdWBUtl+M6q16CwtdV/2Bhy1LSgKmhljqF2tdRs7Kj2xGexm1gNF6P6nL Evxy04HsY0SaJLlPBDHVrgCm7c/4vso3xrCwd3Dsr5JUqFVrxWBB8qW/2hZfEp/kSzGK YPsUMJ5KB5hGs4KhxjJMTWvR232GVv34lENmRQwB+e9mU88490ptlnfvC1oHqCEUiZxH HUsA== X-Gm-Message-State: AOJu0YwHERl4jetz/JazM0o8o1GG96YJKcJm9A3gLsMNlWkeBUt3MUnZ fHYrxdUN2o2s9qSflkPJwL3DwzAIA7O2PTuFK/JgYZCCC2aLP28Yg1VZ0i4CnYajDwDnsYdd9uH tYFBtYg4Ol26+LLwgHlMmuNWNx4WEtqQWJ3xg1M/StEnx8tlRJBpxCRwX0mhTFwhwtpdv4LWHDg x4Ztle5ZT+aysPjoc= X-Gm-Gg: ASbGnctZyaqHKRntpinckAYNm6aJ6jnzL6fEzTcucpHTI8euWalJ9Bl4VoBgZGpBOBc T/LBA/XjBMUCtiB4RDmBhZNqdyqSQ0vPMGA== X-Received: by 2002:a05:6512:402a:b0:53e:39c2:f02b with SMTP id 2adb3069b0e04-53e39c2f164mr2793367e87.42.1733742195401; Mon, 09 Dec 2024 03:03:15 -0800 (PST) X-Google-Smtp-Source: AGHT+IGcYwnaFYzrHHpBDJ6BRNJCkr22hb2L/8fRXeMQGBgxG3TJrPmDo8TY36Y1rQOIJAVYGWDuvsRZEeVyt0byaB4= X-Received: by 2002:a05:6512:402a:b0:53e:39c2:f02b with SMTP id 2adb3069b0e04-53e39c2f164mr2793354e87.42.1733742195027; Mon, 09 Dec 2024 03:03:15 -0800 (PST) MIME-Version: 1.0 References: <20241205175754.1673888-1-david.marchand@redhat.com> In-Reply-To: From: David Marchand Date: Mon, 9 Dec 2024 12:03:04 +0100 Message-ID: Subject: Re: [PATCH 0/3] Defer lcore variables allocation To: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= Cc: dev@dpdk.org, thomas@monjalon.net, frode.nordahl@canonical.com, mattias.ronnblom@ericsson.com X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: a9Cp-mFIn5hghjzUTiKdkgHpiHrsQranzSJ03bteuuE_1733742195 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hello, On Fri, Dec 6, 2024 at 12:02=E2=80=AFPM Mattias R=C3=B6nnblom wrote: > > On 2024-12-05 18:57, David Marchand wrote: > > As I had reported in rc2, the lcore variables allocation have a > > noticeable impact on applications consuming DPDK, even when such > > applications does not use DPDK, or use features associated to > > some lcore variables. > > > > While the amount has been reduced in a rush before rc2, > > there are still cases when the increased memory footprint is noticed > > like in scaling tests. > > See https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/2090931 > > > > What this bug report fails to mention is that it only affects > applications using locked memory. - By locked memory, are you referring to mlock() and friends? No ovsdb binary calls them, only the datapath cares about mlocking. - At a minimum, I understand the lcore var change introduced an increase in memory of 4kB * 128 (getpagesize() * RTE_MAX_LCORES), since lcore_var_alloc() calls memset() of the lcore var size, for every lcore. In this unit test where 1000 processes are kept alive in parallel, this means memory consumption increased by 512k * 1000, so ~500M at least. This amount of memory is probably significant in a resource-restrained env like a (Ubuntu) CI. - I went and traced this unit tests on my laptop by monitoring kmem:mm_page_alloc, though there may be a better metrics when it comes to memory consumption. # dir=3Dbuild; perf stat -e kmem:mm_page_alloc -- tests/testsuite -C $dir/tests AUTOTEST_PATH=3D$dir/utilities:$dir/vswitchd:$dir/ovsdb:$dir/vte= p:$dir/tests:$dir/ipsec:: 2154 Which gives: - 1=E2=80=AF635=E2=80=AF489 kmem:mm_page_alloc for v23.11 - 5=E2=80=AF777=E2=80=AF043 kmem:mm_page_alloc for v24.11 There is a 4M difference, where I would expect 128k. So something more happens, than a simple page allocation per lcore, though I fail to understand what. Btw, just focusing on lcore var, I did two more tests: - 1=E2=80=AF606=E2=80=AF998 kmem:mm_page_alloc for v24.11 + revert all= lcore var changes. - 1=E2=80=AF634=E2=80=AF606 kmem:mm_page_alloc for v24.11 + current se= ries with postponed allocations. --=20 David Marchand