From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 33CF642362; Wed, 11 Oct 2023 18:07:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C835A400EF; Wed, 11 Oct 2023 18:07:40 +0200 (CEST) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by mails.dpdk.org (Postfix) with ESMTP id 2B891400D7 for ; Wed, 11 Oct 2023 18:07:39 +0200 (CEST) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id BE4AF5C01F2; Wed, 11 Oct 2023 12:07:38 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Wed, 11 Oct 2023 12:07:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1697040458; x=1697126858; bh=pHUc0xLP9GUC3bxcO4RvWpBqZAi8bXabga+ WZ3Ytg4Q=; b=k9vTuROh+wBxOY9d6b+6RlbzzKJdjQyfhHkKWpHRiWfSLuh4UAr YVJxiFUko/F9UGMiwTE7uGNMuoxe1o1O0Whn/Janq91hlPSQ8XNtnrR4PcDGIoNI nq0HcqWTb/xlx0/RTOXzyCRpuIzOQiaOucu7KGevKIASlxjY0THxVQVzJSF6XEeO RkDNTNki4AUVg7Ak0hMu+OeX/vkpm3n4sq5s0sqw86njaW9mQbTz01tlWtr9+Mq9 NgMAeajEAZvlrjv1uOazNOjyM4OBZmBm2as+iMm/hsKWy2MzjfZdWtqqUvyMeQ+z bJ5lS3FzrIiwosDHEsAW6EViFVqakYSrrig== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t= 1697040458; x=1697126858; bh=pHUc0xLP9GUC3bxcO4RvWpBqZAi8bXabga+ WZ3Ytg4Q=; b=Wh2vpWt+/QwunaD32KIV/TSlzI0Cf11b3tr3RBSj+nbABquKnhI K0GYINH3trGHUMytgEK9UXOemz9T6+jzpxvez4nlS2cxOMrWpMstiSv5xSME2E1g PxV+F+zH4TstZ3LSWeM4QIjZNjr4PKnDc+IMAWYSIQrwDIb79x8Dvc6qvfeMIryO pbiGG3bQR4sLCU1jgQpA/ecO4p1f+UjdP4r8qz2HQgaKKvdfMLjXqKGoYZzAPpeC WUKO0TJcYbR9YLo8xUZVHBQSpZ0he+QTXTUO6fdi2nD5ZfuU2h/cwq/VS9MDGhb+ tEWogs+KRG0MS+mI45AW09NGXtZr4+vx4vQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrheekgdelgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvfevufffkfgjfhgggfgtsehtqhertddttddunecuhfhrohhmpefvhhhomhgr shcuofhonhhjrghlohhnuceothhhohhmrghssehmohhnjhgrlhhonhdrnhgvtheqnecugg ftrfgrthhtvghrnhepfefhjeeluedvvedtuddtuedtvefhieejtefhffeujefhteduudev tdektdeikeffnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrh homhepthhhohhmrghssehmohhnjhgrlhhonhdrnhgvth X-ME-Proxy: Feedback-ID: i47234305:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 11 Oct 2023 12:07:35 -0400 (EDT) From: Thomas Monjalon To: Mattias =?ISO-8859-1?Q?R=F6nnblom?= , Morten =?ISO-8859-1?Q?Br=F8rup?= , Stephen Hemminger Cc: dev@dpdk.org, david.marchand@redhat.com, mattias.ronnblom@ericsson.com, bruce.richardson@intel.com, olivier.matz@6wind.com, andrew.rybchenko@oktetlabs.ru, honnappa.nagarahalli@arm.com, konstantin.v.ananyev@yandex.ru Subject: Re: [PATCH] eal: add cache guard to per-lcore PRNG state Date: Wed, 11 Oct 2023 18:07:33 +0200 Message-ID: <4539298.cEBGB3zze1@thomas> In-Reply-To: <20230906092537.609d6462@hermes.local> References: <20230904092632.12675-1-mb@smartsharesystems.com> <86202387-4424-e4d8-64df-531a580bebd4@lysator.liu.se> <20230906092537.609d6462@hermes.local> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org TLS is an alternative solution proposed by Stephen. What do you think? 06/09/2023 18:25, Stephen Hemminger: > On Mon, 4 Sep 2023 13:57:19 +0200 > Mattias R=F6nnblom wrote: >=20 > > On 2023-09-04 11:26, Morten Br=F8rup wrote: > > > The per-lcore random state is frequently updated by their individual > > > lcores, so add a cache guard to prevent CPU cache thrashing. > > > =20 > >=20 > > "to prevent false sharing in case the CPU employs a next-N-lines (or=20 > > similar) hardware prefetcher" > >=20 > > In my world, cache trashing and cache line contention are two different= =20 > > things. > >=20 > > Other than that, > > Acked-by: Mattias R=F6nnblom >=20 > Could the per-lcore state be thread local? >=20 > Something like this: >=20 > From 3df5e28a7e5589d05e1eade62a0979e84697853d Mon Sep 17 00:00:00 2001 > From: Stephen Hemminger > Date: Wed, 6 Sep 2023 09:22:42 -0700 > Subject: [PATCH] random: use per lcore state >=20 > Move the random number state into thread local storage. > This has a several benefits. > - no false cache sharing from cpu prefetching > - fixes initialization of random state for non-DPDK threads > - fixes unsafe usage of random state by non-DPDK threads >=20 > The initialization of random number state is done by the > lcore (lazy initialization). >=20 > Signed-off-by: Stephen Hemminger > --- > lib/eal/common/rte_random.c | 35 +++++++++++++++++------------------ > 1 file changed, 17 insertions(+), 18 deletions(-) >=20 > diff --git a/lib/eal/common/rte_random.c b/lib/eal/common/rte_random.c > index 53636331a27b..62f36038ac52 100644 > --- a/lib/eal/common/rte_random.c > +++ b/lib/eal/common/rte_random.c > @@ -19,13 +19,14 @@ struct rte_rand_state { > uint64_t z3; > uint64_t z4; > uint64_t z5; > -} __rte_cache_aligned; > + uint64_t seed; > +}; > =20 > -/* One instance each for every lcore id-equipped thread, and one > - * additional instance to be shared by all others threads (i.e., all > - * unregistered non-EAL threads). > - */ > -static struct rte_rand_state rand_states[RTE_MAX_LCORE + 1]; > +/* Global random seed */ > +static uint64_t rte_rand_seed; > + > +/* Per lcore random state. */ > +static RTE_DEFINE_PER_LCORE(struct rte_rand_state, rte_rand_state); > =20 > static uint32_t > __rte_rand_lcg32(uint32_t *seed) > @@ -76,16 +77,14 @@ __rte_srand_lfsr258(uint64_t seed, struct rte_rand_st= ate *state) > state->z3 =3D __rte_rand_lfsr258_gen_seed(&lcg_seed, 4096UL); > state->z4 =3D __rte_rand_lfsr258_gen_seed(&lcg_seed, 131072UL); > state->z5 =3D __rte_rand_lfsr258_gen_seed(&lcg_seed, 8388608UL); > + > + state->seed =3D seed; > } > =20 > void > rte_srand(uint64_t seed) > { > - unsigned int lcore_id; > - > - /* add lcore_id to seed to avoid having the same sequence */ > - for (lcore_id =3D 0; lcore_id < RTE_MAX_LCORE; lcore_id++) > - __rte_srand_lfsr258(seed + lcore_id, &rand_states[lcore_id]); > + __atomic_store_n(&rte_rand_seed, seed, __ATOMIC_RELAXED); > } > =20 > static __rte_always_inline uint64_t > @@ -119,15 +118,15 @@ __rte_rand_lfsr258(struct rte_rand_state *state) > static __rte_always_inline > struct rte_rand_state *__rte_rand_get_state(void) > { > - unsigned int idx; > - > - idx =3D rte_lcore_id(); > + struct rte_rand_state *rand_state =3D &RTE_PER_LCORE(rte_rand_state); > + uint64_t seed; > =20 > - /* last instance reserved for unregistered non-EAL threads */ > - if (unlikely(idx =3D=3D LCORE_ID_ANY)) > - idx =3D RTE_MAX_LCORE; > + /* did seed change */ > + seed =3D __atomic_load_n(&rte_rand_seed, __ATOMIC_RELAXED); > + if (unlikely(seed !=3D rand_state->seed)) > + __rte_srand_lfsr258(seed, rand_state); > =20 > - return &rand_states[idx]; > + return rand_state; > } > =20 > uint64_t >=20