From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4932CA00BE; Sun, 5 Jun 2022 18:05:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 04412400EF; Sun, 5 Jun 2022 18:05:26 +0200 (CEST) Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by mails.dpdk.org (Postfix) with ESMTP id DF53B40041 for ; Sun, 5 Jun 2022 18:05:24 +0200 (CEST) Received: by mail-pg1-f172.google.com with SMTP id s68so11070529pgs.10 for ; Sun, 05 Jun 2022 09:05:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=blIa9alLPKhXLsncT/E5T3XJgzIPblyffv26aM1TQ5c=; b=La+LYHPlLX6d064BR5uT+HODPI4+6VnuVsyfwU6TY2MoaZrq/tFSfFhHHuFCrTbPtS zy0IB+HshHxEAzhsw6s8yk4sO68AzEyD948SlHmO73f3Q5rO8Drv9Ebr2hrafSEERFmL tgMHKSKFudkDdhdYqa4aDdDQgsXRqU8oc/yIYKIR2Su03lRu4a9Ff5hTRpbu9nXp0GxW CaxpLwJ+h9dIInTfqS32dBtcPXDJ0bpWLcD/cwoY4h57x4VwgJMuy1aSqS9OgmL7wFIt XSB6Cmhaw0sNqUm474oI0G8O+Yo7haEhhj7ASNsk8ugavTHEIONQiaO4wFV4wKgFwPUI 96IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=blIa9alLPKhXLsncT/E5T3XJgzIPblyffv26aM1TQ5c=; b=NVXN42zfUWH/O+weRSZIhlMnUxfleNKJu/f++YmwJe56un/YjXUXI9ivx57ZsYY6Gw sH3/NuoBzsc1wD/3fzU93G+XFUB6vqhspGzbfgP2DzvSV1L8/9vEXkZn8j+tA7Isb0KB gEbTBiXJsYIc6c8p8uA23IqkCx/HPyj+P8S132azqxAhSrMJ4Py4c/Es6w34/lJXqZCT 4iDtG9NFCjR39W65fJ3ZiEIl7TsIKBg8BoBWk5m3bSgrC5FLfNAyr/HijxXjG8JMl96i +h96xLpGBmrXGmjXJ2RZZmRg5k0qUhfqmQSLQMKDymkabzM/111IAVd0nrwkLIn0k5Gc U9fw== X-Gm-Message-State: AOAM530V3vfyWZWBFFBAn3hhp28gOG7jMXETEX4Tgca9/YFPksc3IE4c BfQFfP2zLLHg9LHjbJD6UkfRuw== X-Google-Smtp-Source: ABdhPJzb9t/o8+X3tsqyJlU7UricFHcyWHs1ZaDZ4BbdXTi0aOCNyokC4OMG3cPsOotdFCRJSMP2ew== X-Received: by 2002:a63:894a:0:b0:3fc:a724:578c with SMTP id v71-20020a63894a000000b003fca724578cmr17548902pgd.499.1654445123982; Sun, 05 Jun 2022 09:05:23 -0700 (PDT) Received: from hermes.local (204-195-112-199.wavecable.com. [204.195.112.199]) by smtp.gmail.com with ESMTPSA id n4-20020aa79044000000b0051bc538baadsm7165110pfo.184.2022.06.05.09.05.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Jun 2022 09:05:23 -0700 (PDT) Date: Sun, 5 Jun 2022 09:05:21 -0700 From: Stephen Hemminger To: Andrew Rybchenko Cc: Morten =?UTF-8?B?QnLDuHJ1cA==?= , Jerin Jacob , dpdk-dev , techboard@dpdk.org Subject: Re: Optimizations are not features Message-ID: <20220605090521.5a3bdddf@hermes.local> In-Reply-To: References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk> <98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sat, 4 Jun 2022 15:51:58 +0300 Andrew Rybchenko wrote: > On 6/4/22 15:19, Morten Br=C3=B8rup wrote: > >> From: Jerin Jacob [mailto:jerinjacobk@gmail.com] > >> Sent: Saturday, 4 June 2022 13.10 > >> > >> On Sat, Jun 4, 2022 at 3:30 PM Andrew Rybchenko > >> wrote: =20 > >>> > >>> On 6/4/22 12:33, Jerin Jacob wrote: =20 > >>>> On Sat, Jun 4, 2022 at 2:39 PM Morten Br=C3=B8rup =20 > >> wrote: =20 > >>>>> > >>>>> I would like the DPDK community to change its view on compile time = =20 > >> options. Here is why: =20 > >>>>> > >>>>> > >>>>> > >>>>> Application specific performance micro-optimizations like =E2=80=9C= fast =20 > >> mbuf free=E2=80=9D and =E2=80=9Cmbuf direct re-arm=E2=80=9D are being = added to DPDK and > >> presented as features. =20 > >>>>> > >>>>> > >>>>> > >>>>> They are not features, but optimizations, and I don=E2=80=99t under= stand =20 > >> the need for them to be available at run-time! =20 > >>>>> > >>>>> > >>>>> > >>>>> Instead of adding a bunch of exotic exceptions to the fast path of = =20 > >> the PMDs, they should be compile time options. This will improve > >> performance by avoiding branches in the fast path, both for the > >> applications using them, and for generic applications (where the exotic > >> code is omitted). =20 > >>>> > >>>> Agree. I think, keeping the best of both worlds would be > >>>> > >>>> -Enable the feature/optimization as runtime > >>>> -Have a compile-time option to disable the feature/optimization as = =20 > >> an override. =20 > >>> > >>> It is hard to find the right balance, but in general compile > >>> time options are a nightmare for maintenance. Number of > >>> required builds will grow as an exponent. =20 > >=20 > > Test combinations are exponential for N features, regardless if N are r= untime or compile time options. =20 >=20 > But since I'm talking about build checks I don't care about exponential > grows in run time. Yes, testing should care, but it is a separate story. >=20 > > =20 > >>> Of course, we can > >>> limit number of checked combinations, but it will result in > >>> flow of patches to fix build in other cases. =20 > >> > >> The build breakage can be fixed if we use (2) vs (1) > >> > >> 1) > >> #ifdef ... > >> My feature > >> #endif > >> > >> 2) > >> static __rte_always_inline int > >> rte_has_xyz_feature(void) > >> { > >> #ifdef RTE_LIBRTE_XYZ_FEATURE > >> return RTE_LIBRTE_XYZ_FEATURE; > >> #else > >> return 0; > >> #endif > >> } > >> > >> if(rte_has_xyz_feature())) { > >> My feature code > >> > >> } > >> =20 >=20 > Jerin, thanks, very good example. >=20 > > I'm not sure all the features can be covered by that, e.g. added fields= in structures. =20 >=20 > +1 >=20 > >=20 > > Also, I would consider such features "opt in" at compile time only. As = such, they could be allowed to break the ABI/API. > > =20 > >> > >> =20 > >>> Also compile time options tend to make code less readable > >>> which makes all aspects of the development harder. > >>> > >>> Yes, compile time is nice for micro optimizations, but > >>> I have great concerns that it is a right way to go. > >>> =20 > >>>>> Please note that I am only talking about the performance =20 > >> optimizations that are limited to application specific use cases. I > >> think it makes sense to require that performance optimizing an > >> application also requires recompiling the performance critical > >> libraries used by it. =20 > >>>>> > >>>>> > >>>>> > >>>>> Allowing compile time options for application specific performance = =20 > >> optimizations in DPDK would also open a path for other optimizations, > >> which can only be achieved at compile time, such as =E2=80=9Cno fragme= nted > >> packets=E2=80=9D, =E2=80=9Cno attached mbufs=E2=80=9D and =E2=80=9Csin= gle mbuf pool=E2=80=9D. And even more > >> exotic optimizations, such as the =E2=80=9Cindexed mempool cache=E2=80= =9D, which was > >> rejected due to ABI violations =E2=80=93 they could be marked as =E2= =80=9Crisky and > >> untested=E2=80=9D or similar, but still be part of the DPDK main repos= itory. =20 There is a tradeoff that DPDK has had for several years. 1. For ease of use the DPDK should be available in Linux distributions in pre-built binary format. In that case any changes in behavior need to be done at runtime. 2. For performance and size, the DPDK should limit conditional branches and not include dead code. This is what embedded appliance developers want. 3. For flexibilty, the DPDK should allow every option at the smallest granu= larity (often per-packet or per-queue). This allows application to use feature if available but not be limited to only hardware that supports it. All of these do conflict. The big problem that I see is that when a feature that changes the semantic of mbuf is used (no-attach, single pool etc). It is opening other code to bugs. Therefore I am reluctant to use them; in real life production 1% performance gain is totally offset by the cost of .1% more bugs in code run by customers. It would make my life easier if DPDK supported one set of semantics and they worked everywhere. This is what every OS does (Linux, FreeBSD, Window= s). That would mean either all drivers support the feature in all cases or the feature is never introduced. What would help this would be a set of functions that help code do the right thing. Examples in Linux are skb_linearize(), skb_maypull(), etc.