From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0B35143C27; Fri, 8 Mar 2024 10:44:26 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EC77742FC3; Fri, 8 Mar 2024 10:44:25 +0100 (CET) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id 24AC340274 for ; Fri, 8 Mar 2024 10:44:24 +0100 (CET) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id CAC601AE73 for ; Fri, 8 Mar 2024 10:44:23 +0100 (CET) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id BE4EA1ADF3; Fri, 8 Mar 2024 10:44:23 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on hermod.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=ALL_TRUSTED,AWL, T_SCC_BODY_TEXT_LINE autolearn=disabled version=4.0.0 X-Spam-Score: -1.4 Received: from [192.168.1.59] (h-62-63-215-114.A163.priv.bahnhof.se [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 4DF641AE72; Fri, 8 Mar 2024 10:44:22 +0100 (CET) Message-ID: Date: Fri, 8 Mar 2024 10:44:21 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: RTE lock To: Stephen Hemminger Cc: Tyler Retzlaff , "dev@dpdk.org" , =?UTF-8?Q?Mattias_R=C3=B6nnblom?= References: <0024db51-8b39-4aa7-969a-bde86fe1c764@lysator.liu.se> <20240305210225.GA16095@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> <20240307122730.35a07d8b@hermes.local> Content-Language: en-US From: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= In-Reply-To: <20240307122730.35a07d8b@hermes.local> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2024-03-07 21:27, Stephen Hemminger wrote: > On Thu, 7 Mar 2024 20:50:26 +0100 > Mattias Rönnblom wrote: > >> On 2024-03-05 22:02, Tyler Retzlaff wrote: >>> On Tue, Mar 05, 2024 at 09:18:20PM +0100, Mattias Rönnblom wrote: >>>> Shouldn't we have a DPDK-native mutex API, rather than using direct >>>> POSIX mutex lock calls? >>> >>> David raised this a while back and the consensus is yes. I admit it's >>> been on my radar for a long time for the obvious reasons you list below >>> but with other work hasn't been a priority (yet). >>> >>>> >>>> There are two reasons for this, as I see it >>>> 1) more cleanly support non-POSIX operating system (i.e., Microsoft >>>> Windows). >>>> 2) to discourage mechanical use of spinlocks in places where a >>>> regular mutex lock is more appropriate. >>>> >>>> I think (and hope) DPDK developers will tend to pick DPDK-native >>>> rather than other APIs as their first choice. >>> >>> I spent some time evaluating C11 mutex but it really didn't strike me as >>> being fit for purpose so I think DPDK-native is probably the only way to >>> go. If behind the scenes particular locks relied on something standard >>> for Windows perhaps it could be hidden as an implementation detail. >>> >> >> What was it you were missing? >> >> I'm not familiar with C11 mtx_*() >> >>>> >>>> For locks, they go for spinlocks, even in control (non-fast >>>> path/non-packet processing) code paths (e.g., calls made by the >>>> typical non-EAL thread). >>>> >>>> Using spinlocks to synchronize threads that may be preempted aren't >>>> great idea. >>> >>> If you're thinking of looking into this i'd be happy to see it solved. >>> >> >> I have no immediate plans, but this might be something I'll do in the >> future. >> >>> ty > > For Linux, what would be good is a simplified version of what pthread_mutex > does. The current code for pthread_mutex in glibc has a lot of indirection > and complexity which definately slows down the fast uncontended path. > Ideally, an inline version using futex. > > Glibc has so much indirection because it supports so many mutex variants > and operating systems like mach and hurd... The use case RTE lock would address could be A) Synchronizing different control plane threads, or data plane (EAL) threads when they haven't yet started processing packets. B) Synchronizing control plane and data plane threads during steady-state operation. It was A) I had in mind when I started this thread, and for this case, a little extra Hurd-induced lock overhead is not a huge deal. RTE lock here could be tiny wrapper, although some thought need to go into getting the API right obviously. With direct futex() calls one could attempt to address B as well. EAL threads could opt to spin, and control plane threads could be put to sleep (i.e., futex()). The uncontended and "trylock" paths could be very efficient. Etc.