From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F4BC45E6C; Tue, 10 Dec 2024 18:10:01 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 029B040288; Tue, 10 Dec 2024 18:10:01 +0100 (CET) Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by mails.dpdk.org (Postfix) with ESMTP id 1B0774026E for <dev@dpdk.org>; Tue, 10 Dec 2024 18:09:58 +0100 (CET) Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2166651f752so18926225ad.3 for <dev@dpdk.org>; Tue, 10 Dec 2024 09:09:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1733850598; x=1734455398; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=YSIO3j2hiPETopMEBRaWEHGqqTjDWUJEuqsjdHcwMP4=; b=uTqI8amY1h21GnR8avk+1uHWZpzOzg8wT/lMxSpipqpVkM+UYQ1jRc8f7xJNuVzlRw vhpIYSBUhkags8KkE3F+t5Je2ldM3bI4dJEwuqQRLXXa27xS0bhgN5yuyqqbZtppjwc/ hAgz/+sVIFULPYPzVwgIBZI4Mc5OwsqwarojFejC5BjtllhGZixbpng25WDyF46M7W56 O6R1rQvL5S+yDKM6QAaDmW192kg90FHw5dWZd76CkhVGghm5QnCRjY27Vu2tm12rjyaQ qwFqBGWfN+o7j60AfdOdW7TTiIWMVrTVhpwQEGmgwxGC7BHa371ypJ2qjsPhXepqGBbt oUYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733850598; x=1734455398; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YSIO3j2hiPETopMEBRaWEHGqqTjDWUJEuqsjdHcwMP4=; b=cxbZXqVDVI9fXT7ciV3H+WB/XdIXhk3rLBfbt0WDiWGv0jFYshevquMEnylG0OZRoO QrfEof9D9TOuW+g/8ME2SiYxzXVo1oEdBDKoBe6NO7yqwp2HAG1B+6Mw5/M8Q+WiqWOC HQuJ7yVGqND5UFuJdYYeiuq+WKb0YJMVCo/8/KgsIrp4jhC7YtVZnutCnagM9+0LD1pR 1Lg1wpwIXC9OUygZSREKZQshkxV/zF9UQxhPa32CioMNJgYvy0W9sdOX7gJ0dnrrHncS +Na4KXVQvOsyX3oUacy9IrD3tEArjqaz3Ae0eBhnJtOcF2GTqbnQdESToKwkJdD8xWo/ mn8A== X-Forwarded-Encrypted: i=1; AJvYcCWezipz2t43uSlItjZk5zsjXHf6ASpE4zh/jG4Mm+nJqe+fphKqZVwZhEqPgpMQuxjX4z8=@dpdk.org X-Gm-Message-State: AOJu0YxYPhsoD2kBdjmi0lS+Dd9h18r8C8TEoA4iFrwl+c5SmjfoxQWe rYEVfgL7QhCxbYeplyARB1XThRRweALtLOOaI7m+t1qcSSNmI5FNEB7fQcT8LRI= X-Gm-Gg: ASbGncug4KF1BOF6FCLEVWa7bsM9FaTysHGv9LNnyL82mLD7B/ar8KHDoi7IGU4e+0i mFtYCkG4HzI/ErNEA18+piVj1SEn0DpWUG2EHZHM3lnUvXuB6x8fh8KJ0ZHT/zZe8xr26Po19Nd NcrobaWVk3YGTphsSz31K1OvJhtI+9OuzxgFi50x6IWWggJfmHDo2hd3wTI2e/fPZXPLuWgqCL+ Kf+t33iFwud9TcRo6c/h1pdA8fHP8OK85WC9LffSgk08r363fOwsi0dFj41r8dFPN3rT7U+S0mM b7PBqx5we5zTvSSmNJP/b+bKF9eUJEE= X-Google-Smtp-Source: AGHT+IG1BVDEzdz9SY9IX0uOhN6uFKqodBvB/3iJTUgqw8YsNEzmxWFaGI22J4S71MDC2qqV+oEUcQ== X-Received: by 2002:a17:90b:51c5:b0:2ea:3f34:f18d with SMTP id 98e67ed59e1d1-2ef69e16c01mr25620326a91.10.1733850598086; Tue, 10 Dec 2024 09:09:58 -0800 (PST) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2efea4bdea6sm338681a91.10.2024.12.10.09.09.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 09:09:57 -0800 (PST) Date: Tue, 10 Dec 2024 09:09:56 -0800 From: Stephen Hemminger <stephen@networkplumber.org> To: Thomas Monjalon <thomas@monjalon.net> Cc: David Marchand <david.marchand@redhat.com>, Mattias =?UTF-8?B?UsO2bm5i?= =?UTF-8?B?bG9t?= <hofors@lysator.liu.se>, dev@dpdk.org, frode.nordahl@canonical.com, mattias.ronnblom@ericsson.com Subject: Re: [PATCH 0/3] Defer lcore variables allocation Message-ID: <20241210090956.53848f56@hermes.local> In-Reply-To: <6775037.31r3eYUQgx@thomas> References: <20241205175754.1673888-1-david.marchand@redhat.com> <fd8a71c1-d640-4984-8f94-81784d87a88a@lysator.liu.se> <6775037.31r3eYUQgx@thomas> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org On Fri, 06 Dec 2024 16:55:30 +0100 Thomas Monjalon <thomas@monjalon.net> wrote: > 06/12/2024 12:01, Mattias R=C3=B6nnblom: > > On 2024-12-05 18:57, David Marchand wrote: > > In retrospect, maybe the offset between lcore variable instances could= =20 > > have been encoded into the handle, and thus one could use=20 > > different-sized offset for different variables. =20 >=20 > Yes it would allow to allocate a minimum size, > instead of having a default which is also a maximum limit size of an obje= ct. >=20 > It is not too late to change the behavior as the API is experimental. >=20 > > > The general question on whether lcore variables in constructor should > > > be forbidden, is left to a later discussion. =20 > >=20 > > That discussion could be extended to cover the question if RTE_INIT()=20 > > type constructors should be used at all. Intuitively, it seems better i= f=20 > > all DPDK initialization, or at least all EAL init, happens at the time= =20 > > of rte_eal_init(), in some ordered/organized fashion. =20 >=20 > Yes we may avoid constructors and instead have callbacks called in rte_ea= l_init(). > In order to not break the RTE_INIT API, we could define some new macros > for registering such rte_eal_init callbacks. >=20 >=20 My intuition is that the OVS problem with using mlockall() is caused becaus= e when malloc is used, the malloc code will pre-allocate a new arena (memory area)= for use. If the malloc takes before the mlockall() it will then be pinned even if no= t used. If the malloc takes place later, perhaps that arena is coming from unpinned= area. Many more details on glibc malloc here: https://sourceware.org/glibc/wiki/MallocInternals Using blunt tool like mlockall() will have unintended side effects. The issue with constructors, is they look good when they are simple, statle= ss, and only a few of them. But they get to be a undebuggable mess when the con= structor does complex stuff; has dependencies; and there are lots of them. As a refinement, maybe having a way to register callback to be called in pa= rallel after EAL has started threads. But some things like random() need to be av= ailable early in startup.