From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 648C7A054F; Sat, 4 Jun 2022 11:09:23 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 01D7F4021E; Sat, 4 Jun 2022 11:09:23 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id E115F40041; Sat, 4 Jun 2022 11:09:21 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01D877F2.C72C2CFF" Subject: Optimizations are not features Date: Sat, 4 Jun 2022 11:09:20 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk> X-MimeOLE: Produced By Microsoft Exchange V6.5 X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Optimizations are not features Thread-Index: Adh38saBhCzWrq13QiKXVs+Vki9E2A== From: =?iso-8859-1?Q?Morten_Br=F8rup?= To: Cc: X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This is a multi-part message in MIME format. ------_=_NextPart_001_01D877F2.C72C2CFF Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I would like the DPDK community to change its view on compile time = options. Here is why: =20 Application specific performance micro-optimizations like "fast mbuf = free" and "mbuf direct re-arm" are being added to DPDK and presented as = features. =20 They are not features, but optimizations, and I don't understand the = need for them to be available at run-time! =20 Instead of adding a bunch of exotic exceptions to the fast path of the = PMDs, they should be compile time options. This will improve performance = by avoiding branches in the fast path, both for the applications using = them, and for generic applications (where the exotic code is omitted). =20 Please note that I am only talking about the performance optimizations = that are limited to application specific use cases. I think it makes = sense to require that performance optimizing an application also = requires recompiling the performance critical libraries used by it. =20 Allowing compile time options for application specific performance = optimizations in DPDK would also open a path for other optimizations, = which can only be achieved at compile time, such as "no fragmented = packets", "no attached mbufs" and "single mbuf pool". And even more = exotic optimizations, such as the "indexed mempool cache", which was = rejected due to ABI violations - they could be marked as "risky and = untested" or similar, but still be part of the DPDK main repository. =20 =20 Med venlig hilsen / Kind regards, -Morten Br=F8rup =20 ------_=_NextPart_001_01D877F2.C72C2CFF Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

I would = like the DPDK community to change its view on compile time options. Here = is why:

Application specific performance micro-optimizations = like “fast mbuf free” and “mbuf direct re-arm” = are being added to DPDK and presented as features.

They are not = features, but optimizations, and I don’t understand the need for = them to be available at run-time!

Instead of = adding a bunch of exotic exceptions to the fast path of the PMDs, they = should be compile time options. This will improve performance by = avoiding branches in the fast path, both for the applications using = them, and for generic applications (where the exotic code is = omitted).

Please note that I am only talking about the = performance optimizations that are limited to application specific use = cases. I think it makes sense to require that performance optimizing an = application also requires recompiling the performance critical libraries = used by it.

Allowing compile time options for application specific = performance optimizations in DPDK would also open a path for other = optimizations, which can only be achieved at compile time, such as = “no fragmented packets”, “no attached mbufs” and = “single mbuf pool”. And even more exotic optimizations, such = as the “indexed mempool cache”, which was rejected due to = ABI violations – they could be marked as “risky and = untested” or similar, but still be part of the DPDK main = repository.

Med venlig hilsen / Kind regards,

-Morten = Br=F8rup

------_=_NextPart_001_01D877F2.C72C2CFF-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5A767A0032; Sat, 4 Jun 2022 11:33:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3A7484021E; Sat, 4 Jun 2022 11:33:36 +0200 (CEST) Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by mails.dpdk.org (Postfix) with ESMTP id AA99B40041; Sat, 4 Jun 2022 11:33:34 +0200 (CEST) Received: by mail-qt1-f172.google.com with SMTP id 2so7344738qtw.0; Sat, 04 Jun 2022 02:33:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=Vgb2Y9oAFiaWdNDDA11aksI5LcHAnmxyfEQhm99JC2Y=; b=CmUOP7MOGqGVdswwpUyunFRwSnp1I1HnOtGaGOb1QsnapkmCkpz/iOv59tCsuLAm4R DNkQriGw3sVdGB+buZEZx6bUVPmIYR7/Q63UceHjcaPJUZ00Y94fymJzEgONtNB/IMIv MJHhv30XJktTcb9CfM4rjwNxMy2ystx9QKp7V+D0uKvy5tgqhb5mRYeA4voODHvj1M9f 3ze8qvdzCT8eVY4yuWu0RxmNwHkTQqoJ7UDE9pmlV36rhpCbfjHMOGFxb9/TFzHObEnZ domjHktN1SxaYwbNWoumpmROXcEcrIAd6ulBnZIbT0hwrcXhj734wMRdDCi0056Fm5ke a28w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=Vgb2Y9oAFiaWdNDDA11aksI5LcHAnmxyfEQhm99JC2Y=; b=i5aEoODV1o12vwhYBtYiJdVu8+2iBPe1zPB79MXDCce1ZZ8q7lwLq0cyPRETJIydER rN9S/IQNqKE4ENO5pyUCJDKg6iBp5aI2p74ylUK3/l/At+EI2BTrvFzVnLq0Sm7bPfoZ qVXKBR5SN+QqiRKrEdm34cqexWiEwCz9tb+1gXdOgQziTe8SekSspd7m97QShWClREsJ D5nADtw3GAk2YGaIiSQ7Vy1BmT3Nvi7QmRpCx/MeOFKG5ulzLyZJOEqDYMTikKsW1Bzl TEAOmb3EIt7zEhSCXEyVVjeaOQYhUr8LXdZinZgHSISJmJyW3kxD44ohLXZndnDF+pzG D7qw== X-Gm-Message-State: AOAM530j7XsjR0VY36uGbFvKoSfRm6jUn2AlKfbmvhTRzxsoHXYEyYAy b1wVztWiY+EQbxCq/7iK0rAXW4Bcn1vY1tAiE31mYAmv X-Google-Smtp-Source: ABdhPJxoqRfLcfbc+9jz1U6B2lLUG68szE1UAGUUwotknqnyKdty+erpg4vWNJ4g25ANng6hJeuZlPT3E8cEVCwze94= X-Received: by 2002:a05:622a:8c:b0:302:b38e:53e2 with SMTP id o12-20020a05622a008c00b00302b38e53e2mr11194926qtw.410.1654335214022; Sat, 04 Jun 2022 02:33:34 -0700 (PDT) MIME-Version: 1.0 References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk> From: Jerin Jacob Date: Sat, 4 Jun 2022 15:03:08 +0530 Message-ID: Subject: Re: Optimizations are not features To: =?UTF-8?Q?Morten_Br=C3=B8rup?= Cc: dpdk-dev , techboard@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sat, Jun 4, 2022 at 2:39 PM Morten Br=C3=B8rup wrote: > > I would like the DPDK community to change its view on compile time option= s. Here is why: > > > > Application specific performance micro-optimizations like =E2=80=9Cfast m= buf free=E2=80=9D and =E2=80=9Cmbuf direct re-arm=E2=80=9D are being added = to DPDK and presented as features. > > > > They are not features, but optimizations, and I don=E2=80=99t understand = the need for them to be available at run-time! > > > > Instead of adding a bunch of exotic exceptions to the fast path of the PM= Ds, they should be compile time options. This will improve performance by a= voiding branches in the fast path, both for the applications using them, an= d for generic applications (where the exotic code is omitted). Agree. I think, keeping the best of both worlds would be -Enable the feature/optimization as runtime -Have a compile-time option to disable the feature/optimization as an overr= ide. > > > > Please note that I am only talking about the performance optimizations th= at are limited to application specific use cases. I think it makes sense to= require that performance optimizing an application also requires recompili= ng the performance critical libraries used by it. > > > > Allowing compile time options for application specific performance optimi= zations in DPDK would also open a path for other optimizations, which can o= nly be achieved at compile time, such as =E2=80=9Cno fragmented packets=E2= =80=9D, =E2=80=9Cno attached mbufs=E2=80=9D and =E2=80=9Csingle mbuf pool= =E2=80=9D. And even more exotic optimizations, such as the =E2=80=9Cindexed= mempool cache=E2=80=9D, which was rejected due to ABI violations =E2=80=93= they could be marked as =E2=80=9Crisky and untested=E2=80=9D or similar, b= ut still be part of the DPDK main repository. > > > > > > Med venlig hilsen / Kind regards, > > -Morten Br=C3=B8rup > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BF42BA034C; Sat, 4 Jun 2022 12:00:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AACDE4021E; Sat, 4 Jun 2022 12:00:28 +0200 (CEST) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [91.220.146.113]) by mails.dpdk.org (Postfix) with ESMTP id 34F3840041; Sat, 4 Jun 2022 12:00:27 +0200 (CEST) Received: from [192.168.38.17] (aros.oktetlabs.ru [192.168.38.17]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 938F1148; Sat, 4 Jun 2022 13:00:26 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 shelob.oktetlabs.ru 938F1148 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1654336826; bh=7hir5Dn5pVwo+SUY7D6xSMdNo6rup1CAEKg6yK6Zy0w=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=BX0QSmZfCpaMqhbIXd8Vinqcaw7R3mrIzDvDt9Ooogtd7n4oubaO9zpYGEUitW9M7 KqQ7+/hChkuYYAxdBlle4+V5/XUTBTD5JlqusjgTYXhkL2/9OEnsBYffsDeVcX/Yz1 XaWLx0N6JYM/kVGdcSJ4stKGW0pPMZ3e1CEN5SlY= Message-ID: Date: Sat, 4 Jun 2022 13:00:26 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 Subject: Re: Optimizations are not features Content-Language: en-US To: Jerin Jacob , =?UTF-8?Q?Morten_Br=c3=b8rup?= Cc: dpdk-dev , techboard@dpdk.org References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk> From: Andrew Rybchenko Organization: OKTET Labs In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 6/4/22 12:33, Jerin Jacob wrote: > On Sat, Jun 4, 2022 at 2:39 PM Morten Brørup wrote: >> >> I would like the DPDK community to change its view on compile time options. Here is why: >> >> >> >> Application specific performance micro-optimizations like “fast mbuf free” and “mbuf direct re-arm” are being added to DPDK and presented as features. >> >> >> >> They are not features, but optimizations, and I don’t understand the need for them to be available at run-time! >> >> >> >> Instead of adding a bunch of exotic exceptions to the fast path of the PMDs, they should be compile time options. This will improve performance by avoiding branches in the fast path, both for the applications using them, and for generic applications (where the exotic code is omitted). > > Agree. I think, keeping the best of both worlds would be > > -Enable the feature/optimization as runtime > -Have a compile-time option to disable the feature/optimization as an override. It is hard to find the right balance, but in general compile time options are a nightmare for maintenance. Number of required builds will grow as an exponent. Of course, we can limit number of checked combinations, but it will result in flow of patches to fix build in other cases. Also compile time options tend to make code less readable which makes all aspects of the development harder. Yes, compile time is nice for micro optimizations, but I have great concerns that it is a right way to go. >> Please note that I am only talking about the performance optimizations that are limited to application specific use cases. I think it makes sense to require that performance optimizing an application also requires recompiling the performance critical libraries used by it. >> >> >> >> Allowing compile time options for application specific performance optimizations in DPDK would also open a path for other optimizations, which can only be achieved at compile time, such as “no fragmented packets”, “no attached mbufs” and “single mbuf pool”. And even more exotic optimizations, such as the “indexed mempool cache”, which was rejected due to ABI violations – they could be marked as “risky and untested” or similar, but still be part of the DPDK main repository. >> >> >> >> >> >> Med venlig hilsen / Kind regards, >> >> -Morten Brørup >> >> From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 78C1FA0554; Sat, 4 Jun 2022 13:10:51 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 285034021E; Sat, 4 Jun 2022 13:10:51 +0200 (CEST) Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by mails.dpdk.org (Postfix) with ESMTP id 5C9A140041; Sat, 4 Jun 2022 13:10:50 +0200 (CEST) Received: by mail-qt1-f173.google.com with SMTP id c8so7419821qtj.1; Sat, 04 Jun 2022 04:10:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=rHBnN7O/HpwBnizgBF+0Sh/AhexsCap+jZRsTB0cV6g=; b=VkC0D0XCRXJbuNRtYGBpReNw7JtFA3Wsj/Sk8qcPHAFKrwYm7G4WO2Iga1vwwyDbiK cg4jeGhWYe4PcEo7nuRbSDA5jSHLFhxLEJfwuJpsqvJ8fqEM/D/aHhZg6IFVnbxRRS0p 8bHWKhUW0nv1oPmiBwgZ2ZlbGBqHfwz0DHpEDDChoZMO/kcfKPqZZa2qhLruHaVqfm02 JUc8VhHstzjofKxn8NSxhsJEpSA2G+SO/FczOd+Pl6JPCg4Qpd+9VcaDB+7m5diQeDpv RfBmt8ESEbWyihfWQs/7oFPp3aHUIDAiGxH5YkWKfRa5gfZPZABGWktdBMuUgN+b2TdJ p8Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=rHBnN7O/HpwBnizgBF+0Sh/AhexsCap+jZRsTB0cV6g=; b=R1eKvOGfMoA08GoGoowqVj6+wsftnNzXY1HjAV7nUGaEHtqHXd4/hKDKY8B8VQJ0A8 MgpLEm+D2L/sEtvM62Xg1YtjH8Vj1LKqHym2WlGjLSgJlh+O5GfBsIaPaeBZdEOtQ0Lz CvdkAg4NXeYy5+LlN0VB0HkqdiMX28i6+B7w0qvvH0Hfj9OVrAA42btpjp14/cKIo43D t5k1m/IgBxzGLCOAp7UzP+JlGoQgy0Oqr/uRxhWTiniZ2OhTZZSGzT9+fJCCq/0mW+Bw t3jJ9lTmrxCQw4R5HTtmwEh7VQcT+q4UhCSbHUvDOgGuw/oJEjZu5oYlQFp3Yf/Bti1v QS9A== X-Gm-Message-State: AOAM533PQ+TrXP1qV/Wx+ZUEIywkNi1mrjB9SJ3Btf9swyvEExaoTkLi I0esgdAe0a1J8JlhiS5yLIGAYOiaaQxoa/gt+26T//jyBhs= X-Google-Smtp-Source: ABdhPJxg6uVnEZKcu5b+izK/gaXn8WVTJ9lv3pjS3VCefuyS3N8z+o/lIF2u+jgJNIv9jzOXbJlkBdGYYkEhhlreZcU= X-Received: by 2002:a05:622a:8f:b0:304:bbc4:7668 with SMTP id o15-20020a05622a008f00b00304bbc47668mr11398781qtw.172.1654341049618; Sat, 04 Jun 2022 04:10:49 -0700 (PDT) MIME-Version: 1.0 References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

In-Reply-To: From: Jerin Jacob Date: Sat, 4 Jun 2022 16:40:23 +0530 Message-ID: Subject: Re: Optimizations are not features To: Andrew Rybchenko Cc: =?UTF-8?Q?Morten_Br=C3=B8rup?= , dpdk-dev , techboard@dpdk.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sat, Jun 4, 2022 at 3:30 PM Andrew Rybchenko wrote: > > On 6/4/22 12:33, Jerin Jacob wrote: > > On Sat, Jun 4, 2022 at 2:39 PM Morten Br=C3=B8rup wrote: > >> > >> I would like the DPDK community to change its view on compile time opt= ions. Here is why: > >> > >> > >> > >> Application specific performance micro-optimizations like =E2=80=9Cfas= t mbuf free=E2=80=9D and =E2=80=9Cmbuf direct re-arm=E2=80=9D are being add= ed to DPDK and presented as features. > >> > >> > >> > >> They are not features, but optimizations, and I don=E2=80=99t understa= nd the need for them to be available at run-time! > >> > >> > >> > >> Instead of adding a bunch of exotic exceptions to the fast path of the= PMDs, they should be compile time options. This will improve performance b= y avoiding branches in the fast path, both for the applications using them,= and for generic applications (where the exotic code is omitted). > > > > Agree. I think, keeping the best of both worlds would be > > > > -Enable the feature/optimization as runtime > > -Have a compile-time option to disable the feature/optimization as an o= verride. > > It is hard to find the right balance, but in general compile > time options are a nightmare for maintenance. Number of > required builds will grow as an exponent. Of course, we can > limit number of checked combinations, but it will result in > flow of patches to fix build in other cases. The build breakage can be fixed if we use (2) vs (1) 1) #ifdef ... My feature #endif 2) static __rte_always_inline int rte_has_xyz_feature(void) { #ifdef RTE_LIBRTE_XYZ_FEATURE return RTE_LIBRTE_XYZ_FEATURE; #else return 0; #endif } if(rte_has_xyz_feature())) { My feature code } > Also compile time options tend to make code less readable > which makes all aspects of the development harder. > > Yes, compile time is nice for micro optimizations, but > I have great concerns that it is a right way to go. > > >> Please note that I am only talking about the performance optimizations= that are limited to application specific use cases. I think it makes sense= to require that performance optimizing an application also requires recomp= iling the performance critical libraries used by it. > >> > >> > >> > >> Allowing compile time options for application specific performance opt= imizations in DPDK would also open a path for other optimizations, which ca= n only be achieved at compile time, such as =E2=80=9Cno fragmented packets= =E2=80=9D, =E2=80=9Cno attached mbufs=E2=80=9D and =E2=80=9Csingle mbuf poo= l=E2=80=9D. And even more exotic optimizations, such as the =E2=80=9Cindexe= d mempool cache=E2=80=9D, which was rejected due to ABI violations =E2=80= =93 they could be marked as =E2=80=9Crisky and untested=E2=80=9D or similar= , but still be part of the DPDK main repository. > >> > >> > >> > >> > >> > >> Med venlig hilsen / Kind regards, > >> > >> -Morten Br=C3=B8rup > >> > >> > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 24D32A055A; Sat, 4 Jun 2022 14:19:41 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CD64A4021E; Sat, 4 Jun 2022 14:19:40 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id C4B0840041; Sat, 4 Jun 2022 14:19:39 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 Subject: RE: Optimizations are not features Date: Sat, 4 Jun 2022 14:19:38 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> X-MimeOLE: Produced By Microsoft Exchange V6.5 In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Optimizations are not features Thread-Index: Adh4A8Inm8eg46NcR+KuVUZrtcxKZQACIS8w References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

From: =?UTF-8?B?TW9ydGVuIEJyw7hydXA=?= To: "Jerin Jacob" , "Andrew Rybchenko" Cc: "dpdk-dev" , X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org PiBGcm9tOiBKZXJpbiBKYWNvYiBbbWFpbHRvOmplcmluamFjb2JrQGdtYWlsLmNvbV0NCj4gU2Vu dDogU2F0dXJkYXksIDQgSnVuZSAyMDIyIDEzLjEwDQo+IA0KPiBPbiBTYXQsIEp1biA0LCAyMDIy IGF0IDM6MzAgUE0gQW5kcmV3IFJ5YmNoZW5rbw0KPiA8YW5kcmV3LnJ5YmNoZW5rb0Bva3RldGxh YnMucnU+IHdyb3RlOg0KPiA+DQo+ID4gT24gNi80LzIyIDEyOjMzLCBKZXJpbiBKYWNvYiB3cm90 ZToNCj4gPiA+IE9uIFNhdCwgSnVuIDQsIDIwMjIgYXQgMjozOSBQTSBNb3J0ZW4gQnLDuHJ1cA0K PiA8bWJAc21hcnRzaGFyZXN5c3RlbXMuY29tPiB3cm90ZToNCj4gPiA+Pg0KPiA+ID4+IEkgd291 bGQgbGlrZSB0aGUgRFBESyBjb21tdW5pdHkgdG8gY2hhbmdlIGl0cyB2aWV3IG9uIGNvbXBpbGUg dGltZQ0KPiBvcHRpb25zLiBIZXJlIGlzIHdoeToNCj4gPiA+Pg0KPiA+ID4+DQo+ID4gPj4NCj4g PiA+PiBBcHBsaWNhdGlvbiBzcGVjaWZpYyBwZXJmb3JtYW5jZSBtaWNyby1vcHRpbWl6YXRpb25z IGxpa2Ug4oCcZmFzdA0KPiBtYnVmIGZyZWXigJ0gYW5kIOKAnG1idWYgZGlyZWN0IHJlLWFybeKA nSBhcmUgYmVpbmcgYWRkZWQgdG8gRFBESyBhbmQNCj4gcHJlc2VudGVkIGFzIGZlYXR1cmVzLg0K PiA+ID4+DQo+ID4gPj4NCj4gPiA+Pg0KPiA+ID4+IFRoZXkgYXJlIG5vdCBmZWF0dXJlcywgYnV0 IG9wdGltaXphdGlvbnMsIGFuZCBJIGRvbuKAmXQgdW5kZXJzdGFuZA0KPiB0aGUgbmVlZCBmb3Ig dGhlbSB0byBiZSBhdmFpbGFibGUgYXQgcnVuLXRpbWUhDQo+ID4gPj4NCj4gPiA+Pg0KPiA+ID4+ DQo+ID4gPj4gSW5zdGVhZCBvZiBhZGRpbmcgYSBidW5jaCBvZiBleG90aWMgZXhjZXB0aW9ucyB0 byB0aGUgZmFzdCBwYXRoIG9mDQo+IHRoZSBQTURzLCB0aGV5IHNob3VsZCBiZSBjb21waWxlIHRp bWUgb3B0aW9ucy4gVGhpcyB3aWxsIGltcHJvdmUNCj4gcGVyZm9ybWFuY2UgYnkgYXZvaWRpbmcg YnJhbmNoZXMgaW4gdGhlIGZhc3QgcGF0aCwgYm90aCBmb3IgdGhlDQo+IGFwcGxpY2F0aW9ucyB1 c2luZyB0aGVtLCBhbmQgZm9yIGdlbmVyaWMgYXBwbGljYXRpb25zICh3aGVyZSB0aGUgZXhvdGlj DQo+IGNvZGUgaXMgb21pdHRlZCkuDQo+ID4gPg0KPiA+ID4gQWdyZWUuIEkgdGhpbmssIGtlZXBp bmcgdGhlIGJlc3Qgb2YgYm90aCB3b3JsZHMgd291bGQgYmUNCj4gPiA+DQo+ID4gPiAtRW5hYmxl IHRoZSBmZWF0dXJlL29wdGltaXphdGlvbiBhcyBydW50aW1lDQo+ID4gPiAtSGF2ZSBhIGNvbXBp bGUtdGltZSBvcHRpb24gdG8gZGlzYWJsZSB0aGUgZmVhdHVyZS9vcHRpbWl6YXRpb24gYXMNCj4g YW4gb3ZlcnJpZGUuDQo+ID4NCj4gPiBJdCBpcyBoYXJkIHRvIGZpbmQgdGhlIHJpZ2h0IGJhbGFu Y2UsIGJ1dCBpbiBnZW5lcmFsIGNvbXBpbGUNCj4gPiB0aW1lIG9wdGlvbnMgYXJlIGEgbmlnaHRt YXJlIGZvciBtYWludGVuYW5jZS4gTnVtYmVyIG9mDQo+ID4gcmVxdWlyZWQgYnVpbGRzIHdpbGwg Z3JvdyBhcyBhbiBleHBvbmVudC4NCg0KVGVzdCBjb21iaW5hdGlvbnMgYXJlIGV4cG9uZW50aWFs IGZvciBOIGZlYXR1cmVzLCByZWdhcmRsZXNzIGlmIE4gYXJlIHJ1bnRpbWUgb3IgY29tcGlsZSB0 aW1lIG9wdGlvbnMuDQoNCj4gPiBPZiBjb3Vyc2UsIHdlIGNhbg0KPiA+IGxpbWl0IG51bWJlciBv ZiBjaGVja2VkIGNvbWJpbmF0aW9ucywgYnV0IGl0IHdpbGwgcmVzdWx0IGluDQo+ID4gZmxvdyBv ZiBwYXRjaGVzIHRvIGZpeCBidWlsZCBpbiBvdGhlciBjYXNlcy4NCj4gDQo+IFRoZSBidWlsZCBi cmVha2FnZSBjYW4gYmUgZml4ZWQgaWYgd2UgdXNlICgyKSB2cyAoMSkNCj4gDQo+IDEpDQo+ICNp ZmRlZiAuLi4NCj4gTXkgZmVhdHVyZQ0KPiAjZW5kaWYNCj4gDQo+IDIpDQo+IHN0YXRpYyBfX3J0 ZV9hbHdheXNfaW5saW5lIGludA0KPiBydGVfaGFzX3h5el9mZWF0dXJlKHZvaWQpDQo+IHsNCj4g I2lmZGVmIFJURV9MSUJSVEVfWFlaX0ZFQVRVUkUNCj4gICAgICAgICByZXR1cm4gUlRFX0xJQlJU RV9YWVpfRkVBVFVSRTsNCj4gI2Vsc2UNCj4gICAgICAgICByZXR1cm4gMDsNCj4gI2VuZGlmDQo+ IH0NCj4gDQo+IGlmKHJ0ZV9oYXNfeHl6X2ZlYXR1cmUoKSkpIHsNCj4gTXkgZmVhdHVyZSBjb2Rl DQo+IA0KPiB9DQo+IA0KDQpJJ20gbm90IHN1cmUgYWxsIHRoZSBmZWF0dXJlcyBjYW4gYmUgY292 ZXJlZCBieSB0aGF0LCBlLmcuIGFkZGVkIGZpZWxkcyBpbiBzdHJ1Y3R1cmVzLg0KDQpBbHNvLCBJ IHdvdWxkIGNvbnNpZGVyIHN1Y2ggZmVhdHVyZXMgIm9wdCBpbiIgYXQgY29tcGlsZSB0aW1lIG9u bHkuIEFzIHN1Y2gsIHRoZXkgY291bGQgYmUgYWxsb3dlZCB0byBicmVhayB0aGUgQUJJL0FQSS4N Cg0KPiANCj4gDQo+ID4gQWxzbyBjb21waWxlIHRpbWUgb3B0aW9ucyB0ZW5kIHRvIG1ha2UgY29k ZSBsZXNzIHJlYWRhYmxlDQo+ID4gd2hpY2ggbWFrZXMgYWxsIGFzcGVjdHMgb2YgdGhlIGRldmVs b3BtZW50IGhhcmRlci4NCj4gPg0KPiA+IFllcywgY29tcGlsZSB0aW1lIGlzIG5pY2UgZm9yIG1p Y3JvIG9wdGltaXphdGlvbnMsIGJ1dA0KPiA+IEkgaGF2ZSBncmVhdCBjb25jZXJucyB0aGF0IGl0 IGlzIGEgcmlnaHQgd2F5IHRvIGdvLg0KPiA+DQo+ID4gPj4gUGxlYXNlIG5vdGUgdGhhdCBJIGFt IG9ubHkgdGFsa2luZyBhYm91dCB0aGUgcGVyZm9ybWFuY2UNCj4gb3B0aW1pemF0aW9ucyB0aGF0 IGFyZSBsaW1pdGVkIHRvIGFwcGxpY2F0aW9uIHNwZWNpZmljIHVzZSBjYXNlcy4gSQ0KPiB0aGlu ayBpdCBtYWtlcyBzZW5zZSB0byByZXF1aXJlIHRoYXQgcGVyZm9ybWFuY2Ugb3B0aW1pemluZyBh bg0KPiBhcHBsaWNhdGlvbiBhbHNvIHJlcXVpcmVzIHJlY29tcGlsaW5nIHRoZSBwZXJmb3JtYW5j ZSBjcml0aWNhbA0KPiBsaWJyYXJpZXMgdXNlZCBieSBpdC4NCj4gPiA+Pg0KPiA+ID4+DQo+ID4g Pj4NCj4gPiA+PiBBbGxvd2luZyBjb21waWxlIHRpbWUgb3B0aW9ucyBmb3IgYXBwbGljYXRpb24g c3BlY2lmaWMgcGVyZm9ybWFuY2UNCj4gb3B0aW1pemF0aW9ucyBpbiBEUERLIHdvdWxkIGFsc28g b3BlbiBhIHBhdGggZm9yIG90aGVyIG9wdGltaXphdGlvbnMsDQo+IHdoaWNoIGNhbiBvbmx5IGJl IGFjaGlldmVkIGF0IGNvbXBpbGUgdGltZSwgc3VjaCBhcyDigJxubyBmcmFnbWVudGVkDQo+IHBh Y2tldHPigJ0sIOKAnG5vIGF0dGFjaGVkIG1idWZz4oCdIGFuZCDigJxzaW5nbGUgbWJ1ZiBwb29s 4oCdLiBBbmQgZXZlbiBtb3JlDQo+IGV4b3RpYyBvcHRpbWl6YXRpb25zLCBzdWNoIGFzIHRoZSDi gJxpbmRleGVkIG1lbXBvb2wgY2FjaGXigJ0sIHdoaWNoIHdhcw0KPiByZWplY3RlZCBkdWUgdG8g QUJJIHZpb2xhdGlvbnMg4oCTIHRoZXkgY291bGQgYmUgbWFya2VkIGFzIOKAnHJpc2t5IGFuZA0K PiB1bnRlc3RlZOKAnSBvciBzaW1pbGFyLCBidXQgc3RpbGwgYmUgcGFydCBvZiB0aGUgRFBESyBt YWluIHJlcG9zaXRvcnkuDQo+ID4gPj4NCj4gPiA+Pg0KPiA+ID4+DQo+ID4gPj4NCj4gPiA+Pg0K PiA+ID4+IE1lZCB2ZW5saWcgaGlsc2VuIC8gS2luZCByZWdhcmRzLA0KPiA+ID4+DQo+ID4gPj4g LU1vcnRlbiBCcsO4cnVwDQo+ID4gPj4NCj4gPiA+Pg0KPiA+DQoNCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 615ABA0545; Sat, 4 Jun 2022 14:52:01 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 08CA84021E; Sat, 4 Jun 2022 14:52:01 +0200 (CEST) Received: from shelob.oktetlabs.ru (shelob.oktetlabs.ru [91.220.146.113]) by mails.dpdk.org (Postfix) with ESMTP id B267240041; Sat, 4 Jun 2022 14:51:59 +0200 (CEST) Received: from [192.168.1.126] (unknown [188.242.181.57]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by shelob.oktetlabs.ru (Postfix) with ESMTPSA id 6617E13E; Sat, 4 Jun 2022 15:51:58 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 shelob.oktetlabs.ru 6617E13E DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=oktetlabs.ru; s=default; t=1654347118; bh=oq2UesGX2rZIDCLFLRnr7f/5xeqdJyZvhSMQih+McoA=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=L0KIqOfSGDKXDkHnLTBgP9yU+LMN8unKvl9S5CGsmB3V/DMHmtHETElIRrv59gaFr JtAthAz9yuGrrAIW4CjMOvFylD+BCbT+bZk0PHg4wZMC5BSe7ajuIsjzd1Q+2eVO7B KL+m4e2TyhmsgOdUE4JeOwTtXZhC+fFPbmQMj3MU= Message-ID: Date: Sat, 4 Jun 2022 15:51:58 +0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0 Subject: Re: Optimizations are not features Content-Language: en-US To: =?UTF-8?Q?Morten_Br=c3=b8rup?= , Jerin Jacob Cc: dpdk-dev , techboard@dpdk.org References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> From: Andrew Rybchenko In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 6/4/22 15:19, Morten Brørup wrote: >> From: Jerin Jacob [mailto:jerinjacobk@gmail.com] >> Sent: Saturday, 4 June 2022 13.10 >> >> On Sat, Jun 4, 2022 at 3:30 PM Andrew Rybchenko >> wrote: >>> >>> On 6/4/22 12:33, Jerin Jacob wrote: >>>> On Sat, Jun 4, 2022 at 2:39 PM Morten Brørup >> wrote: >>>>> >>>>> I would like the DPDK community to change its view on compile time >> options. Here is why: >>>>> >>>>> >>>>> >>>>> Application specific performance micro-optimizations like “fast >> mbuf free” and “mbuf direct re-arm” are being added to DPDK and >> presented as features. >>>>> >>>>> >>>>> >>>>> They are not features, but optimizations, and I don’t understand >> the need for them to be available at run-time! >>>>> >>>>> >>>>> >>>>> Instead of adding a bunch of exotic exceptions to the fast path of >> the PMDs, they should be compile time options. This will improve >> performance by avoiding branches in the fast path, both for the >> applications using them, and for generic applications (where the exotic >> code is omitted). >>>> >>>> Agree. I think, keeping the best of both worlds would be >>>> >>>> -Enable the feature/optimization as runtime >>>> -Have a compile-time option to disable the feature/optimization as >> an override. >>> >>> It is hard to find the right balance, but in general compile >>> time options are a nightmare for maintenance. Number of >>> required builds will grow as an exponent. > > Test combinations are exponential for N features, regardless if N are runtime or compile time options. But since I'm talking about build checks I don't care about exponential grows in run time. Yes, testing should care, but it is a separate story. > >>> Of course, we can >>> limit number of checked combinations, but it will result in >>> flow of patches to fix build in other cases. >> >> The build breakage can be fixed if we use (2) vs (1) >> >> 1) >> #ifdef ... >> My feature >> #endif >> >> 2) >> static __rte_always_inline int >> rte_has_xyz_feature(void) >> { >> #ifdef RTE_LIBRTE_XYZ_FEATURE >> return RTE_LIBRTE_XYZ_FEATURE; >> #else >> return 0; >> #endif >> } >> >> if(rte_has_xyz_feature())) { >> My feature code >> >> } >> Jerin, thanks, very good example. > I'm not sure all the features can be covered by that, e.g. added fields in structures. +1 > > Also, I would consider such features "opt in" at compile time only. As such, they could be allowed to break the ABI/API. > >> >> >>> Also compile time options tend to make code less readable >>> which makes all aspects of the development harder. >>> >>> Yes, compile time is nice for micro optimizations, but >>> I have great concerns that it is a right way to go. >>> >>>>> Please note that I am only talking about the performance >> optimizations that are limited to application specific use cases. I >> think it makes sense to require that performance optimizing an >> application also requires recompiling the performance critical >> libraries used by it. >>>>> >>>>> >>>>> >>>>> Allowing compile time options for application specific performance >> optimizations in DPDK would also open a path for other optimizations, >> which can only be achieved at compile time, such as “no fragmented >> packets”, “no attached mbufs” and “single mbuf pool”. And even more >> exotic optimizations, such as the “indexed mempool cache”, which was >> rejected due to ABI violations – they could be marked as “risky and >> untested” or similar, but still be part of the DPDK main repository. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Med venlig hilsen / Kind regards, >>>>> >>>>> -Morten Brørup >>>>> >>>>> >>> > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8FEE4A00C2; Sun, 5 Jun 2022 10:15:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2820C4021E; Sun, 5 Jun 2022 10:15:33 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id 303A640041; Sun, 5 Jun 2022 10:15:32 +0200 (CEST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 Subject: RE: Optimizations are not features Date: Sun, 5 Jun 2022 10:15:29 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D870F0@smartserver.smartshare.dk> X-MimeOLE: Produced By Microsoft Exchange V6.5 In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Optimizations are not features Thread-Index: Adh4EePLfF2Ii4k/SMGdAwCT5FxFFAAmwbaQ References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> From: =?UTF-8?B?TW9ydGVuIEJyw7hydXA=?= To: "Andrew Rybchenko" , "Jerin Jacob" Cc: "dpdk-dev" , X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org PiBGcm9tOiBBbmRyZXcgUnliY2hlbmtvIFttYWlsdG86YW5kcmV3LnJ5YmNoZW5rb0Bva3RldGxh YnMucnVdDQo+IFNlbnQ6IFNhdHVyZGF5LCA0IEp1bmUgMjAyMiAxNC41Mg0KPiANCj4gT24gNi80 LzIyIDE1OjE5LCBNb3J0ZW4gQnLDuHJ1cCB3cm90ZToNCj4gPj4gRnJvbTogSmVyaW4gSmFjb2Ig W21haWx0bzpqZXJpbmphY29ia0BnbWFpbC5jb21dDQo+ID4+IFNlbnQ6IFNhdHVyZGF5LCA0IEp1 bmUgMjAyMiAxMy4xMA0KPiA+Pg0KPiA+PiBPbiBTYXQsIEp1biA0LCAyMDIyIGF0IDM6MzAgUE0g QW5kcmV3IFJ5YmNoZW5rbw0KPiA+PiA8YW5kcmV3LnJ5YmNoZW5rb0Bva3RldGxhYnMucnU+IHdy b3RlOg0KPiA+Pj4NCj4gPj4+IE9uIDYvNC8yMiAxMjozMywgSmVyaW4gSmFjb2Igd3JvdGU6DQo+ ID4+Pj4gT24gU2F0LCBKdW4gNCwgMjAyMiBhdCAyOjM5IFBNIE1vcnRlbiBCcsO4cnVwDQo+ID4+ IDxtYkBzbWFydHNoYXJlc3lzdGVtcy5jb20+IHdyb3RlOg0KPiA+Pj4+Pg0KPiA+Pj4+PiBJIHdv dWxkIGxpa2UgdGhlIERQREsgY29tbXVuaXR5IHRvIGNoYW5nZSBpdHMgdmlldyBvbiBjb21waWxl DQo+IHRpbWUNCj4gPj4gb3B0aW9ucy4gSGVyZSBpcyB3aHk6DQo+ID4+Pj4+DQo+ID4+Pj4+DQo+ ID4+Pj4+DQo+ID4+Pj4+IEFwcGxpY2F0aW9uIHNwZWNpZmljIHBlcmZvcm1hbmNlIG1pY3JvLW9w dGltaXphdGlvbnMgbGlrZSDigJxmYXN0DQo+ID4+IG1idWYgZnJlZeKAnSBhbmQg4oCcbWJ1ZiBk aXJlY3QgcmUtYXJt4oCdIGFyZSBiZWluZyBhZGRlZCB0byBEUERLIGFuZA0KPiA+PiBwcmVzZW50 ZWQgYXMgZmVhdHVyZXMuDQo+ID4+Pj4+DQo+ID4+Pj4+DQo+ID4+Pj4+DQo+ID4+Pj4+IFRoZXkg YXJlIG5vdCBmZWF0dXJlcywgYnV0IG9wdGltaXphdGlvbnMsIGFuZCBJIGRvbuKAmXQgdW5kZXJz dGFuZA0KPiA+PiB0aGUgbmVlZCBmb3IgdGhlbSB0byBiZSBhdmFpbGFibGUgYXQgcnVuLXRpbWUh DQo+ID4+Pj4+DQo+ID4+Pj4+DQo+ID4+Pj4+DQo+ID4+Pj4+IEluc3RlYWQgb2YgYWRkaW5nIGEg YnVuY2ggb2YgZXhvdGljIGV4Y2VwdGlvbnMgdG8gdGhlIGZhc3QgcGF0aA0KPiBvZg0KPiA+PiB0 aGUgUE1EcywgdGhleSBzaG91bGQgYmUgY29tcGlsZSB0aW1lIG9wdGlvbnMuIFRoaXMgd2lsbCBp bXByb3ZlDQo+ID4+IHBlcmZvcm1hbmNlIGJ5IGF2b2lkaW5nIGJyYW5jaGVzIGluIHRoZSBmYXN0 IHBhdGgsIGJvdGggZm9yIHRoZQ0KPiA+PiBhcHBsaWNhdGlvbnMgdXNpbmcgdGhlbSwgYW5kIGZv ciBnZW5lcmljIGFwcGxpY2F0aW9ucyAod2hlcmUgdGhlDQo+IGV4b3RpYw0KPiA+PiBjb2RlIGlz IG9taXR0ZWQpLg0KPiA+Pj4+DQo+ID4+Pj4gQWdyZWUuIEkgdGhpbmssIGtlZXBpbmcgdGhlIGJl c3Qgb2YgYm90aCB3b3JsZHMgd291bGQgYmUNCj4gPj4+Pg0KPiA+Pj4+IC1FbmFibGUgdGhlIGZl YXR1cmUvb3B0aW1pemF0aW9uIGFzIHJ1bnRpbWUNCj4gPj4+PiAtSGF2ZSBhIGNvbXBpbGUtdGlt ZSBvcHRpb24gdG8gZGlzYWJsZSB0aGUgZmVhdHVyZS9vcHRpbWl6YXRpb24gYXMNCj4gPj4gYW4g b3ZlcnJpZGUuDQo+ID4+Pg0KPiA+Pj4gSXQgaXMgaGFyZCB0byBmaW5kIHRoZSByaWdodCBiYWxh bmNlLCBidXQgaW4gZ2VuZXJhbCBjb21waWxlDQo+ID4+PiB0aW1lIG9wdGlvbnMgYXJlIGEgbmln aHRtYXJlIGZvciBtYWludGVuYW5jZS4gTnVtYmVyIG9mDQo+ID4+PiByZXF1aXJlZCBidWlsZHMg d2lsbCBncm93IGFzIGFuIGV4cG9uZW50Lg0KPiA+DQo+ID4gVGVzdCBjb21iaW5hdGlvbnMgYXJl IGV4cG9uZW50aWFsIGZvciBOIGZlYXR1cmVzLCByZWdhcmRsZXNzIGlmIE4gYXJlDQo+IHJ1bnRp bWUgb3IgY29tcGlsZSB0aW1lIG9wdGlvbnMuDQo+IA0KPiBCdXQgc2luY2UgSSdtIHRhbGtpbmcg YWJvdXQgYnVpbGQgY2hlY2tzIEkgZG9uJ3QgY2FyZSBhYm91dCBleHBvbmVudGlhbA0KPiBncm93 cyBpbiBydW4gdGltZS4gWWVzLCB0ZXN0aW5nIHNob3VsZCBjYXJlLCBidXQgaXQgaXMgYSBzZXBh cmF0ZQ0KPiBzdG9yeS4NCg0KQnVpbGQgY2hlY2tzIGlzIGp1c3Qgb25lIG9mIG1hbnkgdGVzdCBt ZXRob2RzLiBBIHZlcnkgbG93IGNvc3QgYW5kIGVmZmljaWVudCB0ZXN0IG1ldGhvZCwgdGhvdWdo Lg0KDQpJIGFja25vd2xlZGdlIHRoYXQgYnVpbGQgY2hlY2tzIHdpbGwgYmUgbW9yZSBjb21wbGlj YXRlZCB3aXRoIGFkZGl0aW9uYWwgY29tcGlsZSB0aW1lIG9wdGlvbnMuIEFuZCBidWlsZCBjaGVj a3MgYXJlIGltcG9ydGFudCBmb3IgY29kZSBxdWFsaXR5LCBzbyB3ZSBzaG91bGQgZmluZCBhIHNv bHV0aW9uIGZvciB0aGF0IGNoYWxsZW5nZS4NCg0KVGhlIHByaW1hcnkgc2NvcGUgb2YgbXkgc3Vn Z2VzdGlvbiBpcyBhcHBsaWNhdGlvbiBzcGVjaWZpYyBwZXJmb3JtYW5jZSBvcHRpbWl6YXRpb24g ZmVhdHVyZXMgb25seSwgc28gbGV0J3MgY2FsbCB0aGVtICJleG90aWMgZmVhdHVyZXMiLiBBbHNv LCB0aGV5IHNob3VsZCBiZSAib3B0IGluIi4gS2VlcCBpbiBtaW5kOiBUaGV5IGFyZSBub3QgcmVh bGx5IGZlYXR1cmVzLCBzaW5jZSB0aGV5IGRvbid0IGFkZCBhbnl0aGluZyBuZXcsIHRoZXkgYXJl IG9ubHkgcGVyZm9ybWFuY2Ugb3B0aW1pemF0aW9ucyBiZW5lZmljaWFsIGZvciBzcGVjaWZpYyBh cHBsaWNhdGlvbiB1c2UgY2FzZXMuDQoNCldpdGggdGhpcyBpbiBtaW5kLCB3ZSBjb3VsZCBjb250 aW51ZSBvcmRpbmFyeSB0ZXN0aW5nIHdpdGggdGhlc2Ugb3B0aW9ucyBkaXNhYmxlZC4gSS5lLiBu byBhZGRpdGlvbmFsIHRlc3RpbmcuDQoNClJlZ2FyZGxlc3MgaG93IGV4b3RpYyB0aGUgZmVhdHVy ZXMgbWF5IGJlLCB3ZSBjZXJ0YWlubHkgd2FudCBzb21lIHRlc3Rpbmcgb2YgdGhlbSwgc28gd2Ug Y291bGQgZG8gc29tZSB0aGluZ3MgdG8gYXZvaWQgZXhwb25lbnRpYWwgdGVzdGluZzoNCg0KMS4g V2l0aCBlYWNoIG9yZGluYXJ5IGJ1aWxkIGNoZWNrICh3aXRoIGFsbCBvcHRpb25zIGRpc2FibGVk KSwgYW5vdGhlciBidWlsZCBjaGVjayBjb3VsZCBiZSBydW4gd2l0aCBhIHJhbmRvbSBzZXQgb2Yg b3B0aW9ucyBlbmFibGVkLiBPciBhZGRpdGlvbmFsIHRlc3RzIGNvdWxkIGJlIHJ1biBvbiBhIHdl ZWtseSBiYXNpcyB3aXRoIHJhbmRvbSBjb21iaW5hdGlvbnMgb2Ygb3B0aW9ucy4NCg0KMi4gQW55 IHBhdGNoIHJlbGF0ZWQgdG8gb25lIG9yIG1vcmUgb2YgdGhlc2UgZXhvdGljIGZlYXR1cmVzIGNv dWxkIGluY2x1ZGUgc29tZSBtYWdpYyBrZXl3b3JkcyB0byBpbmRpY2F0ZSB3aGljaCBjb21iaW5h dGlvbnMgb2Ygb3B0aW9ucyBtdXN0IGJlIGVuYWJsZWQgZHVyaW5nIGJ1aWxkIGNoZWNraW5nIGFu ZCB0ZXN0aW5nLg0KDQo+IA0KPiA+DQo+ID4+PiBPZiBjb3Vyc2UsIHdlIGNhbg0KPiA+Pj4gbGlt aXQgbnVtYmVyIG9mIGNoZWNrZWQgY29tYmluYXRpb25zLCBidXQgaXQgd2lsbCByZXN1bHQgaW4N Cj4gPj4+IGZsb3cgb2YgcGF0Y2hlcyB0byBmaXggYnVpbGQgaW4gb3RoZXIgY2FzZXMuDQo+ID4+ DQo+ID4+IFRoZSBidWlsZCBicmVha2FnZSBjYW4gYmUgZml4ZWQgaWYgd2UgdXNlICgyKSB2cyAo MSkNCj4gPj4NCj4gPj4gMSkNCj4gPj4gI2lmZGVmIC4uLg0KPiA+PiBNeSBmZWF0dXJlDQo+ID4+ ICNlbmRpZg0KPiA+Pg0KPiA+PiAyKQ0KPiA+PiBzdGF0aWMgX19ydGVfYWx3YXlzX2lubGluZSBp bnQNCj4gPj4gcnRlX2hhc194eXpfZmVhdHVyZSh2b2lkKQ0KPiA+PiB7DQo+ID4+ICNpZmRlZiBS VEVfTElCUlRFX1hZWl9GRUFUVVJFDQo+ID4+ICAgICAgICAgIHJldHVybiBSVEVfTElCUlRFX1hZ Wl9GRUFUVVJFOw0KPiA+PiAjZWxzZQ0KPiA+PiAgICAgICAgICByZXR1cm4gMDsNCj4gPj4gI2Vu ZGlmDQo+ID4+IH0NCj4gPj4NCj4gPj4gaWYocnRlX2hhc194eXpfZmVhdHVyZSgpKSkgew0KPiA+ PiBNeSBmZWF0dXJlIGNvZGUNCj4gPj4NCj4gPj4gfQ0KPiA+Pg0KPiANCj4gSmVyaW4sIHRoYW5r cywgdmVyeSBnb29kIGV4YW1wbGUuDQo+IA0KPiA+IEknbSBub3Qgc3VyZSBhbGwgdGhlIGZlYXR1 cmVzIGNhbiBiZSBjb3ZlcmVkIGJ5IHRoYXQsIGUuZy4gYWRkZWQNCj4gZmllbGRzIGluIHN0cnVj dHVyZXMuDQo+IA0KPiArMQ0KPiANCj4gPg0KPiA+IEFsc28sIEkgd291bGQgY29uc2lkZXIgc3Vj aCBmZWF0dXJlcyAib3B0IGluIiBhdCBjb21waWxlIHRpbWUgb25seS4NCj4gQXMgc3VjaCwgdGhl eSBjb3VsZCBiZSBhbGxvd2VkIHRvIGJyZWFrIHRoZSBBQkkvQVBJLg0KPiA+DQo+ID4+DQo+ID4+ DQo+ID4+PiBBbHNvIGNvbXBpbGUgdGltZSBvcHRpb25zIHRlbmQgdG8gbWFrZSBjb2RlIGxlc3Mg cmVhZGFibGUNCj4gPj4+IHdoaWNoIG1ha2VzIGFsbCBhc3BlY3RzIG9mIHRoZSBkZXZlbG9wbWVu dCBoYXJkZXIuDQo+ID4+Pg0KPiA+Pj4gWWVzLCBjb21waWxlIHRpbWUgaXMgbmljZSBmb3IgbWlj cm8gb3B0aW1pemF0aW9ucywgYnV0DQo+ID4+PiBJIGhhdmUgZ3JlYXQgY29uY2VybnMgdGhhdCBp dCBpcyBhIHJpZ2h0IHdheSB0byBnby4NCj4gPj4+DQo+ID4+Pj4+IFBsZWFzZSBub3RlIHRoYXQg SSBhbSBvbmx5IHRhbGtpbmcgYWJvdXQgdGhlIHBlcmZvcm1hbmNlDQo+ID4+IG9wdGltaXphdGlv bnMgdGhhdCBhcmUgbGltaXRlZCB0byBhcHBsaWNhdGlvbiBzcGVjaWZpYyB1c2UgY2FzZXMuIEkN Cj4gPj4gdGhpbmsgaXQgbWFrZXMgc2Vuc2UgdG8gcmVxdWlyZSB0aGF0IHBlcmZvcm1hbmNlIG9w dGltaXppbmcgYW4NCj4gPj4gYXBwbGljYXRpb24gYWxzbyByZXF1aXJlcyByZWNvbXBpbGluZyB0 aGUgcGVyZm9ybWFuY2UgY3JpdGljYWwNCj4gPj4gbGlicmFyaWVzIHVzZWQgYnkgaXQuDQo+ID4+ Pj4+DQo+ID4+Pj4+DQo+ID4+Pj4+DQo+ID4+Pj4+IEFsbG93aW5nIGNvbXBpbGUgdGltZSBvcHRp b25zIGZvciBhcHBsaWNhdGlvbiBzcGVjaWZpYw0KPiBwZXJmb3JtYW5jZQ0KPiA+PiBvcHRpbWl6 YXRpb25zIGluIERQREsgd291bGQgYWxzbyBvcGVuIGEgcGF0aCBmb3Igb3RoZXINCj4gb3B0aW1p emF0aW9ucywNCj4gPj4gd2hpY2ggY2FuIG9ubHkgYmUgYWNoaWV2ZWQgYXQgY29tcGlsZSB0aW1l LCBzdWNoIGFzIOKAnG5vIGZyYWdtZW50ZWQNCj4gPj4gcGFja2V0c+KAnSwg4oCcbm8gYXR0YWNo ZWQgbWJ1ZnPigJ0gYW5kIOKAnHNpbmdsZSBtYnVmIHBvb2zigJ0uIEFuZCBldmVuIG1vcmUNCj4g Pj4gZXhvdGljIG9wdGltaXphdGlvbnMsIHN1Y2ggYXMgdGhlIOKAnGluZGV4ZWQgbWVtcG9vbCBj YWNoZeKAnSwgd2hpY2ggd2FzDQo+ID4+IHJlamVjdGVkIGR1ZSB0byBBQkkgdmlvbGF0aW9ucyDi gJMgdGhleSBjb3VsZCBiZSBtYXJrZWQgYXMg4oCccmlza3kgYW5kDQo+ID4+IHVudGVzdGVk4oCd IG9yIHNpbWlsYXIsIGJ1dCBzdGlsbCBiZSBwYXJ0IG9mIHRoZSBEUERLIG1haW4gcmVwb3NpdG9y eS4NCj4gPj4+Pj4NCj4gPj4+Pj4NCj4gPj4+Pj4NCj4gPj4+Pj4NCj4gPj4+Pj4NCj4gPj4+Pj4g TWVkIHZlbmxpZyBoaWxzZW4gLyBLaW5kIHJlZ2FyZHMsDQo+ID4+Pj4+DQo+ID4+Pj4+IC1Nb3J0 ZW4gQnLDuHJ1cA0KPiA+Pj4+Pg0KPiA+Pj4+Pg0KPiA+Pj4NCj4gPg0KPiANCg0K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4932CA00BE; Sun, 5 Jun 2022 18:05:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 04412400EF; Sun, 5 Jun 2022 18:05:26 +0200 (CEST) Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by mails.dpdk.org (Postfix) with ESMTP id DF53B40041 for ; Sun, 5 Jun 2022 18:05:24 +0200 (CEST) Received: by mail-pg1-f172.google.com with SMTP id s68so11070529pgs.10 for ; Sun, 05 Jun 2022 09:05:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=blIa9alLPKhXLsncT/E5T3XJgzIPblyffv26aM1TQ5c=; b=La+LYHPlLX6d064BR5uT+HODPI4+6VnuVsyfwU6TY2MoaZrq/tFSfFhHHuFCrTbPtS zy0IB+HshHxEAzhsw6s8yk4sO68AzEyD948SlHmO73f3Q5rO8Drv9Ebr2hrafSEERFmL tgMHKSKFudkDdhdYqa4aDdDQgsXRqU8oc/yIYKIR2Su03lRu4a9Ff5hTRpbu9nXp0GxW CaxpLwJ+h9dIInTfqS32dBtcPXDJ0bpWLcD/cwoY4h57x4VwgJMuy1aSqS9OgmL7wFIt XSB6Cmhaw0sNqUm474oI0G8O+Yo7haEhhj7ASNsk8ugavTHEIONQiaO4wFV4wKgFwPUI 96IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=blIa9alLPKhXLsncT/E5T3XJgzIPblyffv26aM1TQ5c=; b=NVXN42zfUWH/O+weRSZIhlMnUxfleNKJu/f++YmwJe56un/YjXUXI9ivx57ZsYY6Gw sH3/NuoBzsc1wD/3fzU93G+XFUB6vqhspGzbfgP2DzvSV1L8/9vEXkZn8j+tA7Isb0KB gEbTBiXJsYIc6c8p8uA23IqkCx/HPyj+P8S132azqxAhSrMJ4Py4c/Es6w34/lJXqZCT 4iDtG9NFCjR39W65fJ3ZiEIl7TsIKBg8BoBWk5m3bSgrC5FLfNAyr/HijxXjG8JMl96i +h96xLpGBmrXGmjXJ2RZZmRg5k0qUhfqmQSLQMKDymkabzM/111IAVd0nrwkLIn0k5Gc U9fw== X-Gm-Message-State: AOAM530V3vfyWZWBFFBAn3hhp28gOG7jMXETEX4Tgca9/YFPksc3IE4c BfQFfP2zLLHg9LHjbJD6UkfRuw== X-Google-Smtp-Source: ABdhPJzb9t/o8+X3tsqyJlU7UricFHcyWHs1ZaDZ4BbdXTi0aOCNyokC4OMG3cPsOotdFCRJSMP2ew== X-Received: by 2002:a63:894a:0:b0:3fc:a724:578c with SMTP id v71-20020a63894a000000b003fca724578cmr17548902pgd.499.1654445123982; Sun, 05 Jun 2022 09:05:23 -0700 (PDT) Received: from hermes.local (204-195-112-199.wavecable.com. [204.195.112.199]) by smtp.gmail.com with ESMTPSA id n4-20020aa79044000000b0051bc538baadsm7165110pfo.184.2022.06.05.09.05.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Jun 2022 09:05:23 -0700 (PDT) Date: Sun, 5 Jun 2022 09:05:21 -0700 From: Stephen Hemminger To: Andrew Rybchenko Cc: Morten =?UTF-8?B?QnLDuHJ1cA==?= , Jerin Jacob , dpdk-dev , techboard@dpdk.org Subject: Re: Optimizations are not features Message-ID: <20220605090521.5a3bdddf@hermes.local> In-Reply-To: References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sat, 4 Jun 2022 15:51:58 +0300 Andrew Rybchenko wrote: > On 6/4/22 15:19, Morten Br=C3=B8rup wrote: > >> From: Jerin Jacob [mailto:jerinjacobk@gmail.com] > >> Sent: Saturday, 4 June 2022 13.10 > >> > >> On Sat, Jun 4, 2022 at 3:30 PM Andrew Rybchenko > >> wrote: =20 > >>> > >>> On 6/4/22 12:33, Jerin Jacob wrote: =20 > >>>> On Sat, Jun 4, 2022 at 2:39 PM Morten Br=C3=B8rup =20 > >> wrote: =20 > >>>>> > >>>>> I would like the DPDK community to change its view on compile time = =20 > >> options. Here is why: =20 > >>>>> > >>>>> > >>>>> > >>>>> Application specific performance micro-optimizations like =E2=80=9C= fast =20 > >> mbuf free=E2=80=9D and =E2=80=9Cmbuf direct re-arm=E2=80=9D are being = added to DPDK and > >> presented as features. =20 > >>>>> > >>>>> > >>>>> > >>>>> They are not features, but optimizations, and I don=E2=80=99t under= stand =20 > >> the need for them to be available at run-time! =20 > >>>>> > >>>>> > >>>>> > >>>>> Instead of adding a bunch of exotic exceptions to the fast path of = =20 > >> the PMDs, they should be compile time options. This will improve > >> performance by avoiding branches in the fast path, both for the > >> applications using them, and for generic applications (where the exotic > >> code is omitted). =20 > >>>> > >>>> Agree. I think, keeping the best of both worlds would be > >>>> > >>>> -Enable the feature/optimization as runtime > >>>> -Have a compile-time option to disable the feature/optimization as = =20 > >> an override. =20 > >>> > >>> It is hard to find the right balance, but in general compile > >>> time options are a nightmare for maintenance. Number of > >>> required builds will grow as an exponent. =20 > >=20 > > Test combinations are exponential for N features, regardless if N are r= untime or compile time options. =20 >=20 > But since I'm talking about build checks I don't care about exponential > grows in run time. Yes, testing should care, but it is a separate story. >=20 > > =20 > >>> Of course, we can > >>> limit number of checked combinations, but it will result in > >>> flow of patches to fix build in other cases. =20 > >> > >> The build breakage can be fixed if we use (2) vs (1) > >> > >> 1) > >> #ifdef ... > >> My feature > >> #endif > >> > >> 2) > >> static __rte_always_inline int > >> rte_has_xyz_feature(void) > >> { > >> #ifdef RTE_LIBRTE_XYZ_FEATURE > >> return RTE_LIBRTE_XYZ_FEATURE; > >> #else > >> return 0; > >> #endif > >> } > >> > >> if(rte_has_xyz_feature())) { > >> My feature code > >> > >> } > >> =20 >=20 > Jerin, thanks, very good example. >=20 > > I'm not sure all the features can be covered by that, e.g. added fields= in structures. =20 >=20 > +1 >=20 > >=20 > > Also, I would consider such features "opt in" at compile time only. As = such, they could be allowed to break the ABI/API. > > =20 > >> > >> =20 > >>> Also compile time options tend to make code less readable > >>> which makes all aspects of the development harder. > >>> > >>> Yes, compile time is nice for micro optimizations, but > >>> I have great concerns that it is a right way to go. > >>> =20 > >>>>> Please note that I am only talking about the performance =20 > >> optimizations that are limited to application specific use cases. I > >> think it makes sense to require that performance optimizing an > >> application also requires recompiling the performance critical > >> libraries used by it. =20 > >>>>> > >>>>> > >>>>> > >>>>> Allowing compile time options for application specific performance = =20 > >> optimizations in DPDK would also open a path for other optimizations, > >> which can only be achieved at compile time, such as =E2=80=9Cno fragme= nted > >> packets=E2=80=9D, =E2=80=9Cno attached mbufs=E2=80=9D and =E2=80=9Csin= gle mbuf pool=E2=80=9D. And even more > >> exotic optimizations, such as the =E2=80=9Cindexed mempool cache=E2=80= =9D, which was > >> rejected due to ABI violations =E2=80=93 they could be marked as =E2= =80=9Crisky and > >> untested=E2=80=9D or similar, but still be part of the DPDK main repos= itory. =20 There is a tradeoff that DPDK has had for several years. 1. For ease of use the DPDK should be available in Linux distributions in pre-built binary format. In that case any changes in behavior need to be done at runtime. 2. For performance and size, the DPDK should limit conditional branches and not include dead code. This is what embedded appliance developers want. 3. For flexibilty, the DPDK should allow every option at the smallest granu= larity (often per-packet or per-queue). This allows application to use feature if available but not be limited to only hardware that supports it. All of these do conflict. The big problem that I see is that when a feature that changes the semantic of mbuf is used (no-attach, single pool etc). It is opening other code to bugs. Therefore I am reluctant to use them; in real life production 1% performance gain is totally offset by the cost of .1% more bugs in code run by customers. It would make my life easier if DPDK supported one set of semantics and they worked everywhere. This is what every OS does (Linux, FreeBSD, Window= s). That would mean either all drivers support the feature in all cases or the feature is never introduced. What would help this would be a set of functions that help code do the right thing. Examples in Linux are skb_linearize(), skb_maypull(), etc. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 60190A0542; Mon, 6 Jun 2022 11:35:05 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F124F40A7F; Mon, 6 Jun 2022 11:35:04 +0200 (CEST) Received: from forward501p.mail.yandex.net (forward501p.mail.yandex.net [77.88.28.111]) by mails.dpdk.org (Postfix) with ESMTP id 2636240150; Mon, 6 Jun 2022 11:35:04 +0200 (CEST) Received: from sas1-5f1844c300c3.qloud-c.yandex.net (sas1-5f1844c300c3.qloud-c.yandex.net [IPv6:2a02:6b8:c14:5781:0:640:5f18:44c3]) by forward501p.mail.yandex.net (Yandex) with ESMTP id 8D5506212758; Mon, 6 Jun 2022 12:35:03 +0300 (MSK) Received: from sas1-0701b3ebb6ca.qloud-c.yandex.net (sas1-0701b3ebb6ca.qloud-c.yandex.net [2a02:6b8:c08:21a5:0:640:701:b3eb]) by sas1-5f1844c300c3.qloud-c.yandex.net (mxback/Yandex) with ESMTP id R3rAmID28N-Z3faCmd9; Mon, 06 Jun 2022 12:35:03 +0300 X-Yandex-Fwd: 2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1654508103; bh=yyZkUjjUW9CURdHj7NbC+e3pOf8bgV2tIv4w4CjFALY=; h=In-Reply-To:From:Subject:Cc:References:Date:Message-ID:To; b=Sx59gFBsZrU0VfQt7x1pMiPoRFGhwBfvaoJNIU91Y9EjJaje+odx+sxfyTeEEf82H mUeDg33yel1cktTYtrWVZv+XxqAbV7Y/eeD9QqAa4bd6AIml+/cEgNCylD0W4MrG9B vUyFLbMyt6fxfJkuIupUPg06oW3a1bzxue+7gxE4= Authentication-Results: sas1-5f1844c300c3.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-0701b3ebb6ca.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id N03bydqLWf-Z2JWFuOM; Mon, 06 Jun 2022 12:35:02 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <497ed526-2fb7-805a-f72c-5909b85a62c1@yandex.ru> Date: Mon, 6 Jun 2022 10:35:00 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: Optimizations are not features Content-Language: en-US To: Andrew Rybchenko , =?UTF-8?Q?Morten_Br=c3=b8rup?= , Jerin Jacob Cc: dpdk-dev , techboard@dpdk.org References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> From: Konstantin Ananyev In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 04/06/2022 13:51, Andrew Rybchenko пишет: > On 6/4/22 15:19, Morten Brørup wrote: >>> From: Jerin Jacob [mailto:jerinjacobk@gmail.com] >>> Sent: Saturday, 4 June 2022 13.10 >>> >>> On Sat, Jun 4, 2022 at 3:30 PM Andrew Rybchenko >>> wrote: >>>> >>>> On 6/4/22 12:33, Jerin Jacob wrote: >>>>> On Sat, Jun 4, 2022 at 2:39 PM Morten Brørup >>> wrote: >>>>>> >>>>>> I would like the DPDK community to change its view on compile time >>> options. Here is why: >>>>>> >>>>>> >>>>>> >>>>>> Application specific performance micro-optimizations like “fast >>> mbuf free” and “mbuf direct re-arm” are being added to DPDK and >>> presented as features. >>>>>> >>>>>> >>>>>> >>>>>> They are not features, but optimizations, and I don’t understand >>> the need for them to be available at run-time! >>>>>> >>>>>> >>>>>> >>>>>> Instead of adding a bunch of exotic exceptions to the fast path of >>> the PMDs, they should be compile time options. This will improve >>> performance by avoiding branches in the fast path, both for the >>> applications using them, and for generic applications (where the exotic >>> code is omitted). >>>>> >>>>> Agree. I think, keeping the best of both worlds would be >>>>> >>>>> -Enable the feature/optimization as runtime >>>>> -Have a compile-time option to disable the feature/optimization as >>> an override. >>>> >>>> It is hard to find the right balance, but in general compile >>>> time options are a nightmare for maintenance. Number of >>>> required builds will grow as an exponent. >> >> Test combinations are exponential for N features, regardless if N are >> runtime or compile time options. > > But since I'm talking about build checks I don't care about exponential > grows in run time. Yes, testing should care, but it is a separate story. > >> >>>> Of course, we can >>>> limit number of checked combinations, but it will result in >>>> flow of patches to fix build in other cases. >>> >>> The build breakage can be fixed if we use (2) vs (1) >>> >>> 1) >>> #ifdef ... >>> My feature >>> #endif >>> >>> 2) >>> static __rte_always_inline int >>> rte_has_xyz_feature(void) >>> { >>> #ifdef RTE_LIBRTE_XYZ_FEATURE >>> return RTE_LIBRTE_XYZ_FEATURE; >>> #else >>> return 0; >>> #endif >>> } >>> >>> if(rte_has_xyz_feature())) { >>> My feature code >>> >>> } >>> > > Jerin, thanks, very good example. > >> I'm not sure all the features can be covered by that, e.g. added >> fields in structures. > > +1 > >> >> Also, I would consider such features "opt in" at compile time only. As >> such, they could be allowed to break the ABI/API. >> >>> >>> >>>> Also compile time options tend to make code less readable >>>> which makes all aspects of the development harder. >>>> >>>> Yes, compile time is nice for micro optimizations, but >>>> I have great concerns that it is a right way to go. >>>> >>>>>> Please note that I am only talking about the performance >>> optimizations that are limited to application specific use cases. I >>> think it makes sense to require that performance optimizing an >>> application also requires recompiling the performance critical >>> libraries used by it. >>>>>>abandon some of existing functionality to create a 'short-cut' >>>>>> >>>>>> >>>>>> Allowing compile time options for application specific performance >>> optimizations in DPDK would also open a path for other optimizations, >>> which can only be achieved at compile time, such as “no fragmented >>> packets”, “no attached mbufs” and “single mbuf pool”. And even more >>> exotic optimizations, such as the “indexed mempool cache”, which was >>> rejected due to ABI violations – they could be marked as “risky and >>> untested” or similar, but still be part of the DPDK main repository. >>>>>> Thanks Morten for bringing it up, it is an interesting topic. Though I look at it from different angle. All optimizations you mentioned above introduce new limitations: MBUF_FAST_FREE - no indirect mbufs and multiple mempools, mempool object indexes - mempool size is limited to 4GB, direct rearm - drop ability to stop/reconfigure TX queue, while RX queue is still running, etc. Note that all these limitations are not forced by HW. All of them are pure SW limitations that developers forced in (or tried to) to get few extra performance. That's concerning tendency. As more and more such 'optimization via limitation' will come in: - DPDK feature list will become more and more fragmented. - Would cause more and more confusion for the users. - Unmet expectations - difference in performance between 'default' and 'optimized' version of DPDK will become bigger and bigger. - As Andrew already mentioned, maintaining all these 'sub-flavours' of DPDK will become more and more difficult. So, probably instead of making such changes easier, we need somehow to persuade developers to think more about optimizations that would be generic and transparent to the user. I do realize that it is not always possible due to various reasons (HW limitations, external dependencies, etc.) but that's another story. Let's take for example MBUF_FAST_FREE. In fact, I am not sure that we need it as tx offload flag at all. PMD TX-path has all necessary information to decide at run-time can it do fast_free() for not: At tx_burst() PMD can check are all mbufs satisfy these conditions (same mempool, refcnt==1) and update some fields and/or counters inside TXQ to reflect it. Then, at tx_free() we can use this info to decide between fast_free() and normal_free(). As at tx_burst() we read mbuf fields anyway, impact for this extra step I guess would be minimal. Yes, most likely, it wouldn't be as fast as with current TX offload flag, or conditional compilation approach. But it might be still significantly faster then normal_free(), plus such approach will be generic and transparent to the user. Konstantin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2EA60A034C; Wed, 29 Jun 2022 22:44:44 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D087B410D5; Wed, 29 Jun 2022 22:44:43 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2048.outbound.protection.outlook.com [40.107.22.48]) by mails.dpdk.org (Postfix) with ESMTP id E8FE140691; Wed, 29 Jun 2022 22:44:42 +0200 (CEST) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=idhZ2KlZciV8h0CzDg9DgI2vf53xJkE3vFPGeIg29uJRdxvtFsQtjsrUhXzivfHHvXe0cIu7LrU+8bplxJMIWqVZvfMOIl3hyxtd9wYfyHyxGU551P/NNl1i165KG0EmsX9toDYrygfPKD0s+Pp8zLAIkrI8/+5Shq6uWnqqxs40aXPEZecgEw79jKCRd4u8QMJAtFls8njdzYk+kSRUuFHLp3RRl1DlxBMhKYVuuVkqST761daxjB0sAMqK6n/4DCttYUDoZEhcb5KIdJfYHpi1P6UhzzE46+wvvZm4vDcotqThTAEK2s87yqgKrS4tfjEDFR5QhQM08U5XUHSJoQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mC3K6FChyLgbRfL2BWXf+Tz05qCZx5o3wDdX67n+Rt8=; b=IxU7KRFrX0CZWr4yW91LdY7hu6yggrLG74CMC3wU4992KsE0WWYYiQGA7ZXGZtJu7uAVwi6Jc+otSJupDOF4/3jHHsXvIc0hFGruw6cf1yhcxL1f2cPTIZx62l0MwAFoWSTt9lz20Zx+UK6JaALK4Kt96c+JOkt1n+Y93Cug/tykiLi628rDOO2tYasLrJmx/q07WPYwUXFbOvHnTbTYo5OFgHdKwM7crz6LZZ/f34eCuGJPlIaN8Fzn14WtzH5lLa5ngtcCeRhYsP+QhBCjpZ/5k3WjX3iFyCf8Q/6GvokKfwQyCe6L10fdwmFUvLH4Z8x8gvdmrSjHtH9SDVM55A== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mC3K6FChyLgbRfL2BWXf+Tz05qCZx5o3wDdX67n+Rt8=; b=utUpE+CzmbANd+rESA9OBlecWSAHMj5u+uj24NlA31QhoRq2sGIxSNy73SwVIGYGbeX5g5uyr7IrwuhNgU1d3Fo6MICTrt3O2vmtynizTU51S1pjhZ6gpbexwu+iK8nJA5loHcSshjaBf/4lJldj4+3/cA9xK6Y8DRobFTIgHTA= Received: from AM5PR0101CA0026.eurprd01.prod.exchangelabs.com (2603:10a6:206:16::39) by AS8PR08MB7283.eurprd08.prod.outlook.com (2603:10a6:20b:422::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.14; Wed, 29 Jun 2022 20:44:40 +0000 Received: from VE1EUR03FT050.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:16:cafe::28) by AM5PR0101CA0026.outlook.office365.com (2603:10a6:206:16::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.14 via Frontend Transport; Wed, 29 Jun 2022 20:44:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT050.mail.protection.outlook.com (10.152.19.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.15 via Frontend Transport; Wed, 29 Jun 2022 20:44:40 +0000 Received: ("Tessian outbound 3c5325c30453:v121"); Wed, 29 Jun 2022 20:44:39 +0000 X-CR-MTA-TID: 64aa7808 Received: from 534c86413748.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id C8B9B0D5-C7B1-40D6-BE4D-BBFEF5BD3C3D.1; Wed, 29 Jun 2022 20:44:33 +0000 Received: from EUR05-VI1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 534c86413748.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Wed, 29 Jun 2022 20:44:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c2hXGTk3nSlf9QWMF5aD0x66xQdFpK12TUbKfA4xRl4s+3PcoNZ//Lf4U07KdHW+IxC/2AZmU4frOEbzySSbnmbpBC7zUZPt7T4cOhgJhiaOixipoeXAFb5chyqzZRw50cJtT0CfV7r/y7vEsiiVYAMyPLvE6ivcQkPk6mkRB8vA2nSVs3dnyWTLBwh2/1ObiLl+jnKeEodkngcyaj7Gtq8SXArhxN/LSrkJwH+cwFO03HGfykVZpq20dmMXAQRF4kmMu3ezIc3IILOwTbGFdp4UXt0bvpKY49Plue8gnLtngZrtGc3tcwtRpQPDQuSs+5qPyTn/mWO+Vi3iXP79vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mC3K6FChyLgbRfL2BWXf+Tz05qCZx5o3wDdX67n+Rt8=; b=LJ1WuCH+WMfg20LpIHse/ffonJsOurpDzhjAuFA4FEKdlAfonIfs3aEdY5RSGZhz+Gm+04ZlM2MkrxVXqUozqhwT6wHBrVY1MTQ1SinRfg/VpuEUhcsJ0VgRmCqgacKEB1CjTQ4mnNmvIK2rwgt5dL+Md+k9NfWvlij2Bih4HL/B6CsdzML0gL/mVtbTPdPAq/a0EGdL31EsEKhUqqwUcB1N9zFWtgI0Gh5iHYvdnu5+76Mm/iddd+vucu3zqyN6qnwuJRnkcUx/a08NsvyF/871tAr+dJnIDsZToUkQRYCJLqBtJFEh9UsjPJr93dTcWQ45+F18VtdxfA2u3xMKjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mC3K6FChyLgbRfL2BWXf+Tz05qCZx5o3wDdX67n+Rt8=; b=utUpE+CzmbANd+rESA9OBlecWSAHMj5u+uj24NlA31QhoRq2sGIxSNy73SwVIGYGbeX5g5uyr7IrwuhNgU1d3Fo6MICTrt3O2vmtynizTU51S1pjhZ6gpbexwu+iK8nJA5loHcSshjaBf/4lJldj4+3/cA9xK6Y8DRobFTIgHTA= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by VE1PR08MB5055.eurprd08.prod.outlook.com (2603:10a6:803:115::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5373.17; Wed, 29 Jun 2022 20:44:29 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1c7f:6a8d:b518:f972]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1c7f:6a8d:b518:f972%4]) with mapi id 15.20.5395.014; Wed, 29 Jun 2022 20:44:29 +0000 From: Honnappa Nagarahalli To: Konstantin Ananyev , Andrew Rybchenko , =?utf-8?B?TW9ydGVuIEJyw7hydXA=?= , Jerin Jacob CC: dpdk-dev , "techboard@dpdk.org" , nd , nd Subject: RE: Optimizations are not features Thread-Topic: Optimizations are not features Thread-Index: Adh38saBhCzWrq13QiKXVs+Vki9E2AAA1LwAAAD0FQAAAnFmgAACayUAAAEhFQAAXbQwAARvikLw Date: Wed, 29 Jun 2022 20:44:29 +0000 Message-ID: References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> <497ed526-2fb7-805a-f72c-5909b85a62c1@yandex.ru> In-Reply-To: <497ed526-2fb7-805a-f72c-5909b85a62c1@yandex.ru> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: F74095BCBB02F54F99BCCB8968F8B22F.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: bab53036-9975-4881-4f5f-08da5a103054 x-ms-traffictypediagnostic: VE1PR08MB5055:EE_|VE1EUR03FT050:EE_|AS8PR08MB7283:EE_ x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: GQdtQNuZWauAnKS62RVC8wVeJKahjQLuyDwM2Yo5f/cFYpNyQ3R0VDsewtnorGUANVks8vVIEF43aFYaN9n+rUgwWlDTXTRVbcYwQMUj4DyXj76fArkKbXzz9G1iRyYcMlZr37srUeJJRSO47H3SNhvjHU+VgvldcmTgoYSLUEU3KywgPHN2iK7S8ZPOQOLnhTqNnxre8BZq0J8hglBExNw8OY4YAAsl0TwJwD1L5pvgkl/uKlh7hfLlZpyYeYhhPcyF5pX0ohc5MZCaZyp4PBFqgz4o3EKFROw6/3RhQ4KEkCqR36LV5VE4vyvFNyxpuy45UmPQn+EKIhKEsjyn170OrZm18sVbqYBzyUoSMPrjr7vOXcTJbjB/SdH1KLqZgmr6jrczfwQWdSgZDSw769g1qDFQX8GZicZ6+AwgU3MhdZn2++deS247WSWNDelwR1wGxmgGG9aTI/Q8LFlH+Qno5Ne0R435ylf0tomNLu6GiPG6G8AL5pJmb4WCUhRI/UVPMq6CbXHV4x08aE+piQ6HPnC8ZiirKiPU+vETxUaxjsWx1Dky5mfzIA1ctDTEhNDkePhyls2Inm/LxGhZqfFzJpeeqD9lI6vGW9kEAJQobPI2un3p9VBerBBH79R2PSUlPO/WUEK30sWVHR0169vRnaC/HcV7cB9mufoJWLPTPoKTIDVD2p/PAzOEXZHsvcbEefUI5XlaS8G92qMlFp590Ipv71voYAAN9wT18jexXE58B3CMitnc+kn64fb6GAEFg3hfkPxmqoTEM8HH7bUYZlHi9C6H5LfNL53RGPI= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(39860400002)(376002)(366004)(396003)(136003)(346002)(8676002)(3480700007)(66574015)(86362001)(186003)(122000001)(7696005)(38100700002)(2906002)(6506007)(8936002)(478600001)(71200400001)(52536014)(55016003)(53546011)(41300700001)(316002)(33656002)(5660300002)(4326008)(54906003)(83380400001)(66946007)(76116006)(110136005)(38070700005)(66556008)(66476007)(26005)(9686003)(66446008)(64756008); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5055 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT050.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: a21a0176-ce0a-4260-27ca-08da5a1029e4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yxZpf3C0Yt4jIuw0aBdR4b/u3UTSnUd5Q4/9F43K5ra6b79a//up4ltWM0+zqGcV0iY7JnLlNk96JzmiTTWoYl+e8nhYWaTgWGlum+D7b4ppHIsgZ/QLCko6HRIElKmAVCx4ok4gr0KSUdhixkihGq+grADQ95A2zVZ6ToK0BgwjNk3RkDmSHrAa00WERuMtUIpiNcSM9Q1iS1tkGQX3oPqkTgW6r+Dx18Gg2ELs0BiOtTd08HBJxjOb8DIKjkUqvoBh+VYYQq8iYBHXpIT+b5fp5GnhMium+Jv7IxB0e/i2mphafmm+WabidHAq2mu1/yf1suUhC6jPb7UU+wI7NmtL+KUD3bo1IxALQSzN63oLXyu3KH9XfX4su7iNAml9lgcccXnoqbTaaDTRPHfk6HDyge4wXGr1HvmPgDeV2E41xlhpK91JYukUG8F9SsUtR4tCFLE0W/yLh3UWRxkCIAqrswfeolz1ZOUHhl0SchzfbdyMZV7lef81usMYNmifnIiMEPC3Sb8kNW0Z24xDa7HvRXrk4NEtAkjfJqTa+0UIv6jgzMysrxinSDNyrV2Y8g3eLeVnwzgWsR/Md/I18jBnqJL9yhDw21l7UOSnxeHej+Zqngc+F4OgQ+ATq219kARRPyrbsJzUMnBZBSiibkGSaatsKInJQk+9ci26GflpSf3M6tBpBubTAVfJTlKhg7Uea/9uxQO6Zozx5OaNwY2j4ZX3e9zJb42WpC6bCK/QZQLJ2TNM03Vxb+Z1aZzkyl90jMbqlnNJCoVcmcWqaUjys6z07V+1AtYXxHaaTg6TuWspEJ1T8PTLRi6EKtcW X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230016)(4636009)(376002)(39860400002)(346002)(136003)(396003)(36840700001)(46966006)(40470700004)(82310400005)(4326008)(8676002)(52536014)(5660300002)(8936002)(40480700001)(70206006)(2906002)(40460700003)(70586007)(55016003)(36860700001)(83380400001)(86362001)(3480700007)(33656002)(82740400003)(356005)(81166007)(478600001)(336012)(110136005)(450100002)(41300700001)(316002)(54906003)(186003)(47076005)(66574015)(53546011)(6506007)(26005)(7696005)(9686003); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jun 2022 20:44:40.1279 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bab53036-9975-4881-4f5f-08da5a103054 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT050.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7283 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org PHNuaXA+DQoNCj4gDQo+IDA0LzA2LzIwMjIgMTM6NTEsIEFuZHJldyBSeWJjaGVua28g0L/QuNGI 0LXRgjoNCj4gPiBPbiA2LzQvMjIgMTU6MTksIE1vcnRlbiBCcsO4cnVwIHdyb3RlOg0KPiA+Pj4g RnJvbTogSmVyaW4gSmFjb2IgW21haWx0bzpqZXJpbmphY29ia0BnbWFpbC5jb21dDQo+ID4+PiBT ZW50OiBTYXR1cmRheSwgNCBKdW5lIDIwMjIgMTMuMTANCj4gPj4+DQo+ID4+PiBPbiBTYXQsIEp1 biA0LCAyMDIyIGF0IDM6MzAgUE0gQW5kcmV3IFJ5YmNoZW5rbw0KPiA+Pj4gPGFuZHJldy5yeWJj aGVua29Ab2t0ZXRsYWJzLnJ1PiB3cm90ZToNCj4gPj4+Pg0KPiA+Pj4+IE9uIDYvNC8yMiAxMjoz MywgSmVyaW4gSmFjb2Igd3JvdGU6DQo+ID4+Pj4+IE9uIFNhdCwgSnVuIDQsIDIwMjIgYXQgMjoz OSBQTSBNb3J0ZW4gQnLDuHJ1cA0KPiA+Pj4gPG1iQHNtYXJ0c2hhcmVzeXN0ZW1zLmNvbT4gd3Jv dGU6DQo+ID4+Pj4+Pg0KPiA+Pj4+Pj4gSSB3b3VsZCBsaWtlIHRoZSBEUERLIGNvbW11bml0eSB0 byBjaGFuZ2UgaXRzIHZpZXcgb24gY29tcGlsZQ0KPiA+Pj4+Pj4gdGltZQ0KPiA+Pj4gb3B0aW9u cy4gSGVyZSBpcyB3aHk6DQo+ID4+Pj4+Pg0KPiA+Pj4+Pj4NCj4gPj4+Pj4+DQo+ID4+Pj4+PiBB cHBsaWNhdGlvbiBzcGVjaWZpYyBwZXJmb3JtYW5jZSBtaWNyby1vcHRpbWl6YXRpb25zIGxpa2Ug 4oCcZmFzdA0KPiA+Pj4gbWJ1ZiBmcmVl4oCdIGFuZCDigJxtYnVmIGRpcmVjdCByZS1hcm3igJ0g YXJlIGJlaW5nIGFkZGVkIHRvIERQREsgYW5kDQo+ID4+PiBwcmVzZW50ZWQgYXMgZmVhdHVyZXMu DQo+ID4+Pj4+Pg0KPiA+Pj4+Pj4NCj4gPj4+Pj4+DQo+ID4+Pj4+PiBUaGV5IGFyZSBub3QgZmVh dHVyZXMsIGJ1dCBvcHRpbWl6YXRpb25zLCBhbmQgSSBkb27igJl0IHVuZGVyc3RhbmQNCj4gPj4+ IHRoZSBuZWVkIGZvciB0aGVtIHRvIGJlIGF2YWlsYWJsZSBhdCBydW4tdGltZSENCj4gPj4+Pj4+ DQo+ID4+Pj4+Pg0KPiA+Pj4+Pj4NCj4gPj4+Pj4+IEluc3RlYWQgb2YgYWRkaW5nIGEgYnVuY2gg b2YgZXhvdGljIGV4Y2VwdGlvbnMgdG8gdGhlIGZhc3QgcGF0aA0KPiA+Pj4+Pj4gb2YNCj4gPj4+ IHRoZSBQTURzLCB0aGV5IHNob3VsZCBiZSBjb21waWxlIHRpbWUgb3B0aW9ucy4gVGhpcyB3aWxs IGltcHJvdmUNCj4gPj4+IHBlcmZvcm1hbmNlIGJ5IGF2b2lkaW5nIGJyYW5jaGVzIGluIHRoZSBm YXN0IHBhdGgsIGJvdGggZm9yIHRoZQ0KPiA+Pj4gYXBwbGljYXRpb25zIHVzaW5nIHRoZW0sIGFu ZCBmb3IgZ2VuZXJpYyBhcHBsaWNhdGlvbnMgKHdoZXJlIHRoZQ0KPiA+Pj4gZXhvdGljIGNvZGUg aXMgb21pdHRlZCkuDQo+ID4+Pj4+DQo+ID4+Pj4+IEFncmVlLiBJIHRoaW5rLCBrZWVwaW5nIHRo ZSBiZXN0IG9mIGJvdGggd29ybGRzIHdvdWxkIGJlDQo+ID4+Pj4+DQo+ID4+Pj4+IC1FbmFibGUg dGhlIGZlYXR1cmUvb3B0aW1pemF0aW9uIGFzIHJ1bnRpbWUgLUhhdmUgYSBjb21waWxlLXRpbWUN Cj4gPj4+Pj4gb3B0aW9uIHRvIGRpc2FibGUgdGhlIGZlYXR1cmUvb3B0aW1pemF0aW9uIGFzDQo+ ID4+PiBhbiBvdmVycmlkZS4NCj4gPj4+Pg0KPiA+Pj4+IEl0IGlzIGhhcmQgdG8gZmluZCB0aGUg cmlnaHQgYmFsYW5jZSwgYnV0IGluIGdlbmVyYWwgY29tcGlsZSB0aW1lDQo+ID4+Pj4gb3B0aW9u cyBhcmUgYSBuaWdodG1hcmUgZm9yIG1haW50ZW5hbmNlLiBOdW1iZXIgb2YgcmVxdWlyZWQgYnVp bGRzDQo+ID4+Pj4gd2lsbCBncm93IGFzIGFuIGV4cG9uZW50Lg0KPiA+Pg0KPiA+PiBUZXN0IGNv bWJpbmF0aW9ucyBhcmUgZXhwb25lbnRpYWwgZm9yIE4gZmVhdHVyZXMsIHJlZ2FyZGxlc3MgaWYg TiBhcmUNCj4gPj4gcnVudGltZSBvciBjb21waWxlIHRpbWUgb3B0aW9ucy4NCj4gPg0KPiA+IEJ1 dCBzaW5jZSBJJ20gdGFsa2luZyBhYm91dCBidWlsZCBjaGVja3MgSSBkb24ndCBjYXJlIGFib3V0 DQo+ID4gZXhwb25lbnRpYWwgZ3Jvd3MgaW4gcnVuIHRpbWUuIFllcywgdGVzdGluZyBzaG91bGQg Y2FyZSwgYnV0IGl0IGlzIGEgc2VwYXJhdGUNCj4gc3RvcnkuDQo+ID4NCj4gPj4NCj4gPj4+PiBP ZiBjb3Vyc2UsIHdlIGNhbg0KPiA+Pj4+IGxpbWl0IG51bWJlciBvZiBjaGVja2VkIGNvbWJpbmF0 aW9ucywgYnV0IGl0IHdpbGwgcmVzdWx0IGluIGZsb3cgb2YNCj4gPj4+PiBwYXRjaGVzIHRvIGZp eCBidWlsZCBpbiBvdGhlciBjYXNlcy4NCj4gPj4+DQo+ID4+PiBUaGUgYnVpbGQgYnJlYWthZ2Ug Y2FuIGJlIGZpeGVkIGlmIHdlIHVzZSAoMikgdnMgKDEpDQo+ID4+Pg0KPiA+Pj4gMSkNCj4gPj4+ ICNpZmRlZiAuLi4NCj4gPj4+IE15IGZlYXR1cmUNCj4gPj4+ICNlbmRpZg0KPiA+Pj4NCj4gPj4+ IDIpDQo+ID4+PiBzdGF0aWMgX19ydGVfYWx3YXlzX2lubGluZSBpbnQNCj4gPj4+IHJ0ZV9oYXNf eHl6X2ZlYXR1cmUodm9pZCkNCj4gPj4+IHsNCj4gPj4+ICNpZmRlZiBSVEVfTElCUlRFX1hZWl9G RUFUVVJFDQo+ID4+PiDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiBSVEVfTElCUlRFX1hZWl9GRUFU VVJFOyAjZWxzZQ0KPiA+Pj4gwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gMDsNCj4gPj4+ICNlbmRp Zg0KPiA+Pj4gfQ0KPiA+Pj4NCj4gPj4+IGlmKHJ0ZV9oYXNfeHl6X2ZlYXR1cmUoKSkpIHsNCj4g Pj4+IE15IGZlYXR1cmUgY29kZQ0KPiA+Pj4NCj4gPj4+IH0NCj4gPj4+DQo+ID4NCj4gPiBKZXJp biwgdGhhbmtzLCB2ZXJ5IGdvb2QgZXhhbXBsZS4NCj4gPg0KPiA+PiBJJ20gbm90IHN1cmUgYWxs IHRoZSBmZWF0dXJlcyBjYW4gYmUgY292ZXJlZCBieSB0aGF0LCBlLmcuIGFkZGVkDQo+ID4+IGZp ZWxkcyBpbiBzdHJ1Y3R1cmVzLg0KPiA+DQo+ID4gKzENCj4gPg0KPiA+Pg0KPiA+PiBBbHNvLCBJ IHdvdWxkIGNvbnNpZGVyIHN1Y2ggZmVhdHVyZXMgIm9wdCBpbiIgYXQgY29tcGlsZSB0aW1lIG9u bHkuDQo+ID4+IEFzIHN1Y2gsIHRoZXkgY291bGQgYmUgYWxsb3dlZCB0byBicmVhayB0aGUgQUJJ L0FQSS4NCj4gPj4NCj4gPj4+DQo+ID4+Pg0KPiA+Pj4+IEFsc28gY29tcGlsZSB0aW1lIG9wdGlv bnMgdGVuZCB0byBtYWtlIGNvZGUgbGVzcyByZWFkYWJsZSB3aGljaA0KPiA+Pj4+IG1ha2VzIGFs bCBhc3BlY3RzIG9mIHRoZSBkZXZlbG9wbWVudCBoYXJkZXIuDQo+ID4+Pj4NCj4gPj4+PiBZZXMs IGNvbXBpbGUgdGltZSBpcyBuaWNlIGZvciBtaWNybyBvcHRpbWl6YXRpb25zLCBidXQgSSBoYXZl IGdyZWF0DQo+ID4+Pj4gY29uY2VybnMgdGhhdCBpdCBpcyBhIHJpZ2h0IHdheSB0byBnby4NCj4g Pj4+Pg0KPiA+Pj4+Pj4gUGxlYXNlIG5vdGUgdGhhdCBJIGFtIG9ubHkgdGFsa2luZyBhYm91dCB0 aGUgcGVyZm9ybWFuY2UNCj4gPj4+IG9wdGltaXphdGlvbnMgdGhhdCBhcmUgbGltaXRlZCB0byBh cHBsaWNhdGlvbiBzcGVjaWZpYyB1c2UgY2FzZXMuIEkNCj4gPj4+IHRoaW5rIGl0IG1ha2VzIHNl bnNlIHRvIHJlcXVpcmUgdGhhdCBwZXJmb3JtYW5jZSBvcHRpbWl6aW5nIGFuDQo+ID4+PiBhcHBs aWNhdGlvbiBhbHNvIHJlcXVpcmVzIHJlY29tcGlsaW5nIHRoZSBwZXJmb3JtYW5jZSBjcml0aWNh bA0KPiA+Pj4gbGlicmFyaWVzIHVzZWQgYnkgaXQuDQo+ID4+Pj4+PmFiYW5kb24gc29tZSBvZiBl eGlzdGluZyBmdW5jdGlvbmFsaXR5IHRvIGNyZWF0ZSBhICdzaG9ydC1jdXQnDQo+ID4+Pj4+Pg0K PiA+Pj4+Pj4NCj4gPj4+Pj4+IEFsbG93aW5nIGNvbXBpbGUgdGltZSBvcHRpb25zIGZvciBhcHBs aWNhdGlvbiBzcGVjaWZpYw0KPiA+Pj4+Pj4gcGVyZm9ybWFuY2UNCj4gPj4+IG9wdGltaXphdGlv bnMgaW4gRFBESyB3b3VsZCBhbHNvIG9wZW4gYSBwYXRoIGZvciBvdGhlcg0KPiA+Pj4gb3B0aW1p emF0aW9ucywgd2hpY2ggY2FuIG9ubHkgYmUgYWNoaWV2ZWQgYXQgY29tcGlsZSB0aW1lLCBzdWNo IGFzDQo+ID4+PiDigJxubyBmcmFnbWVudGVkIHBhY2tldHPigJ0sIOKAnG5vIGF0dGFjaGVkIG1i dWZz4oCdIGFuZCDigJxzaW5nbGUgbWJ1ZiBwb29s4oCdLg0KPiA+Pj4gQW5kIGV2ZW4gbW9yZSBl eG90aWMgb3B0aW1pemF0aW9ucywgc3VjaCBhcyB0aGUg4oCcaW5kZXhlZCBtZW1wb29sDQo+ID4+ PiBjYWNoZeKAnSwgd2hpY2ggd2FzIHJlamVjdGVkIGR1ZSB0byBBQkkgdmlvbGF0aW9ucyDigJMg dGhleSBjb3VsZCBiZQ0KPiA+Pj4gbWFya2VkIGFzIOKAnHJpc2t5IGFuZCB1bnRlc3RlZOKAnSBv ciBzaW1pbGFyLCBidXQgc3RpbGwgYmUgcGFydCBvZiB0aGUgRFBESyBtYWluDQo+IHJlcG9zaXRv cnkuDQo+ID4+Pj4+Pg0KPiANCj4gDQo+IFRoYW5rcyBNb3J0ZW4gZm9yIGJyaW5naW5nIGl0IHVw LCBpdCBpcyBhbiBpbnRlcmVzdGluZyB0b3BpYy4NCj4gVGhvdWdoIEkgbG9vayBhdCBpdCBmcm9t IGRpZmZlcmVudCBhbmdsZS4NCj4gQWxsIG9wdGltaXphdGlvbnMgeW91IG1lbnRpb25lZCBhYm92 ZSBpbnRyb2R1Y2UgbmV3IGxpbWl0YXRpb25zOg0KPiBNQlVGX0ZBU1RfRlJFRSAtIG5vIGluZGly ZWN0IG1idWZzIGFuZCBtdWx0aXBsZSBtZW1wb29scywgbWVtcG9vbCBvYmplY3QNCj4gaW5kZXhl cyAtIG1lbXBvb2wgc2l6ZSBpcyBsaW1pdGVkIHRvIDRHQiwgZGlyZWN0IHJlYXJtIC0gZHJvcCBh YmlsaXR5IHRvDQo+IHN0b3AvcmVjb25maWd1cmUgVFggcXVldWUsIHdoaWxlIFJYIHF1ZXVlIGlz IHN0aWxsIHJ1bm5pbmcsIGV0Yy4NCj4gTm90ZSB0aGF0IGFsbCB0aGVzZSBsaW1pdGF0aW9ucyBh cmUgbm90IGZvcmNlZCBieSBIVy4NCj4gQWxsIG9mIHRoZW0gYXJlIHB1cmUgU1cgbGltaXRhdGlv bnMgdGhhdCBkZXZlbG9wZXJzIGZvcmNlZCBpbiAob3IgdHJpZWQgdG8pIHRvIGdldA0KPiBmZXcg ZXh0cmEgcGVyZm9ybWFuY2UuDQo+IFRoYXQncyBjb25jZXJuaW5nIHRlbmRlbmN5Lg0KPiANCj4g QXMgbW9yZSBhbmQgbW9yZSBzdWNoICdvcHRpbWl6YXRpb24gdmlhIGxpbWl0YXRpb24nIHdpbGwg Y29tZSBpbjoNCj4gLSBEUERLIGZlYXR1cmUgbGlzdCB3aWxsIGJlY29tZSBtb3JlIGFuZCBtb3Jl IGZyYWdtZW50ZWQuDQo+IC0gV291bGQgY2F1c2UgbW9yZSBhbmQgbW9yZSBjb25mdXNpb24gZm9y IHRoZSB1c2Vycy4NCj4gLSBVbm1ldCBleHBlY3RhdGlvbnMgLSBkaWZmZXJlbmNlIGluIHBlcmZv cm1hbmNlIGJldHdlZW4gJ2RlZmF1bHQnDQo+ICAgIGFuZCAnb3B0aW1pemVkJyB2ZXJzaW9uIG9m IERQREsgd2lsbCBiZWNvbWUgYmlnZ2VyIGFuZCBiaWdnZXIuDQo+IC0gQXMgQW5kcmV3IGFscmVh ZHkgbWVudGlvbmVkLCBtYWludGFpbmluZyBhbGwgdGhlc2UgJ3N1Yi1mbGF2b3VycycNCj4gICAg b2YgRFBESyB3aWxsIGJlY29tZSBtb3JlIGFuZCBtb3JlIGRpZmZpY3VsdC4NClRoZSBwb2ludCB0 aGF0IHdlIG5lZWQgdG8gcmVtZW1iZXIgaXMsIHRoZXNlIGZlYXR1cmVzL29wdGltaXphdGlvbnMg YXJlIGludHJvZHVjZWQgYWZ0ZXIgc2VlaW5nIHBlcmZvcm1hbmNlIGlzc3VlcyBpbiBwcmFjdGlj YWwgdXNlIGNhc2VzLg0KRFBESyBpcyBub3QgYmVpbmcgdXNlZCBpbiBqdXN0IG9uZSB1c2UgY2Fz ZSwgaXQgaXMgYmVpbmcgdXNlZCBpbiBzZXZlcmFsIHVzZSBjYXNlcyB3aGljaCBoYXZlIHRoZWly IG93biB1bmlxdWUgcmVxdWlyZW1lbnRzLiBJcyA0R0IgZW5vdWdoIGZvciBwYWNrZXQgYnVmZmVy cyAtIHllcyBpdCBpcyBlbm91Z2ggaW4gY2VydGFpbiB1c2UgY2FzZXMuIEFyZSB0aGVpciBOSUNz IHdpdGggc2luZ2xlIHBvcnQgLSB5ZXMgdGhlcmUgYXJlLiBIVyBpcyBiZWluZyBjcmVhdGVkIGJl Y2F1c2UgdXNlIGNhc2VzIGFuZCBidXNpbmVzcyBjYXNlcyBleGlzdC4gSXQgaXMgb2J2aW91cyB0 aGF0IGFzIERQREsgZ2V0cyBhZG9wdGVkIG9uIG1vcmUgcGxhdGZvcm1zIHRoYXQgZGlmZmVyIGxh cmdlbHksIHRoZSBmZWF0dXJlcyB3aWxsIGluY3JlYXNlIGFuZCBpdCB3aWxsIGJlY29tZSBjb21w bGV4LiBDb21wbGV4aXR5IHNob3VsZCBub3QgYmUgdXNlZCBhcyBhIGNyaXRlcmlhIHRvIHJlamVj dCBwYXRjaGVzLg0KDQpUaGVyZSBpcyBkaWZmZXJlbnQgcGVyc3BlY3RpdmUgdG8gd2hhdCB5b3Ug YXJlIGNhbGxpbmcgYXMgJ2xpbWl0YXRpb25zJy4gSSBjYW4gYXJndWUgdGhhdCBtdWx0aXBsZSBt ZW1wb29scywgc3RvcC9yZWNvbmZpZ3VyZSBUWCBxdWV1ZSB3aGlsZSBSWCBxdWV1ZSBpcyBzdGls bCBydW5uaW5nIGFyZSBleG90aWMuIEp1c3QgYmVjYXVzZSB0aG9zZSBhcmUgYWxsb3dlZCBjdXJy ZW50bHkgKHByb2JhYmx5IGFjY2lkZW50bHkpIGRvZXMgbm90IG1lYW4gdGhleSBhcmUgYmVpbmcg dXNlZC4gQXJlIHRoZXJlIHVzZSBjYXNlcyB0aGF0IG1ha2UgdXNlIG9mIHRoZXNlIGZlYXR1cmVz Pw0KDQpUaGUgYmFzZS9leGlzdGluZyBkZXNpZ24gZm9yIERQREsgd2FzIGRvbmUgd2l0aCBvbmUg cGFydGljdWxhciBIVyBhcmNoaXRlY3R1cmUgaW4gbWluZCB3aGVyZSB0aGVyZSB3YXMgYW4gYWJ1 bmRhbmNlIG9mIHJlc291cmNlcy4gVW5mb3J0dW5hdGVseSwgdGhhdCBIVyBhcmNoaXRlY3R1cmUg aXMgZmFzdCBldm9sdmluZyBhbmQgRFBESyBpcyBhZG9wdGVkIGluIHVzZSBjYXNlcyB3aGVyZSB0 aGF0IGtpbmQgb2YgcmVzb3VyY2VzIGFyZSBub3QgYXZhaWxhYmxlLiBGb3IgZXg6IGVmZmljaWVu Y3kgY29yZXMgYXJlIGJlaW5nIGludHJvZHVjZWQgYnkgZXZlcnkgQ1BVIHZlbmRvciBub3cuIFNv b24gZW5vdWdoLCB3ZSB3aWxsIHNlZSBiaWctbGl0dGxlIGFyY2hpdGVjdHVyZSBpbiBuZXR3b3Jr aW5nIGFzIHdlbGwuIFRoZSBleGlzdGluZyBQTUQgZGVzaWduIGludHJvZHVjZXMgNTEyQiBvZiBz dG9yZXMgKDI1NkIgZm9yIGNvcHlpbmcgdG8gc3RhY2sgdmFyaWFibGUgYW5kIDI1NkIgdG8gc3Rv cmUgbGNvcmUgY2FjaGUpIGFuZCAyNTZCIGxvYWQvc3RvcmUgb24gUlggc2lkZSBldmVyeSAzMiBw YWNrZXRzIGJhY2sgdG8gYmFjay4gSXQgZG9lc24ndCBtYWtlIHNlbnNlIHRvIGhhdmUgdGhhdCBr aW5kIG9mIG1lbWNvcHkgZm9yIGxpdHRsZS9lZmZpY2llbmN5IGNvcmVzIGp1c3QgZm9yIHRoZSBk cml2ZXIgY29kZS4NCg0KPiANCj4gU28sIHByb2JhYmx5IGluc3RlYWQgb2YgbWFraW5nIHN1Y2gg Y2hhbmdlcyBlYXNpZXIsIHdlIG5lZWQgc29tZWhvdyB0bw0KPiBwZXJzdWFkZSBkZXZlbG9wZXJz IHRvIHRoaW5rIG1vcmUgYWJvdXQgb3B0aW1pemF0aW9ucyB0aGF0IHdvdWxkIGJlIGdlbmVyaWMN Cj4gYW5kIHRyYW5zcGFyZW50IHRvIHRoZSB1c2VyLg0KT3IgbWF5IGJlIHdlIG5lZWQgdG8gdGhp bmsgb2YgY3JlYXRpbmcgYWx0ZXJuYXRlIHdheXMgb2YgcHJvZ3JhbW1pbmcuDQoNCj4gSSBkbyBy ZWFsaXplIHRoYXQgaXQgaXMgbm90IGFsd2F5cyBwb3NzaWJsZSBkdWUgdG8gdmFyaW91cyByZWFz b25zIChIVyBsaW1pdGF0aW9ucywNCj4gZXh0ZXJuYWwgZGVwZW5kZW5jaWVzLCBldGMuKSBidXQg dGhhdCdzIGFub3RoZXIgc3RvcnkuDQo+IA0KPiBMZXQncyB0YWtlIGZvciBleGFtcGxlIE1CVUZf RkFTVF9GUkVFLg0KPiBJbiBmYWN0LCBJIGFtIG5vdCBzdXJlIHRoYXQgd2UgbmVlZCBpdCBhcyB0 eCBvZmZsb2FkIGZsYWcgYXQgYWxsLg0KPiBQTUQgVFgtcGF0aCBoYXMgYWxsIG5lY2Vzc2FyeSBp bmZvcm1hdGlvbiB0byBkZWNpZGUgYXQgcnVuLXRpbWUgY2FuIGl0IGRvDQo+IGZhc3RfZnJlZSgp IGZvciBub3Q6DQo+IEF0IHR4X2J1cnN0KCkgUE1EIGNhbiBjaGVjayBhcmUgYWxsIG1idWZzIHNh dGlzZnkgdGhlc2UgY29uZGl0aW9ucyAoc2FtZQ0KPiBtZW1wb29sLCByZWZjbnQ9PTEpIGFuZCB1 cGRhdGUgc29tZSBmaWVsZHMgYW5kL29yIGNvdW50ZXJzIGluc2lkZSBUWFEgdG8NCj4gcmVmbGVj dCBpdC4NCj4gVGhlbiwgYXQgdHhfZnJlZSgpIHdlIGNhbiB1c2UgdGhpcyBpbmZvIHRvIGRlY2lk ZSBiZXR3ZWVuIGZhc3RfZnJlZSgpIGFuZA0KPiBub3JtYWxfZnJlZSgpLg0KPiBBcyBhdCB0eF9i dXJzdCgpIHdlIHJlYWQgbWJ1ZiBmaWVsZHMgYW55d2F5LCBpbXBhY3QgZm9yIHRoaXMgZXh0cmEg c3RlcCBJIGd1ZXNzDQo+IHdvdWxkIGJlIG1pbmltYWwuDQo+IFllcywgbW9zdCBsaWtlbHksIGl0 IHdvdWxkbid0IGJlIGFzIGZhc3QgYXMgd2l0aCBjdXJyZW50IFRYIG9mZmxvYWQgZmxhZywgb3IN Cj4gY29uZGl0aW9uYWwgY29tcGlsYXRpb24gYXBwcm9hY2guDQo+IEJ1dCBpdCBtaWdodCBiZSBz dGlsbCBzaWduaWZpY2FudGx5IGZhc3RlciB0aGVuIG5vcm1hbF9mcmVlKCksIHBsdXMgc3VjaCBh cHByb2FjaA0KPiB3aWxsIGJlIGdlbmVyaWMgYW5kIHRyYW5zcGFyZW50IHRvIHRoZSB1c2VyLg0K SU1PLCB0aGlzIGRlcGVuZHMgb24gdGhlIHBoaWxvc29waHkgdGhhdCB3ZSB3YW50IHRvIGFkb3B0 LiBJIHdvdWxkIHByZWZlciB0byBtYWtlIGNvbnRyb2wgcGxhbmUgY29tcGxleCBmb3IgcGVyZm9y bWFuY2UgZ2FpbnMgb24gdGhlIGRhdGEgcGxhbmUuIFRoZSBwZXJmb3JtYW5jZSBvbiB0aGUgZGF0 YSBwbGFuZSBoYXMgYSBtdWx0aXBseWluZyBlZmZlY3QgZHVlIHRvIHRoZSByYXRpbyBvZiBudW1i ZXIgb2YgY29yZXMgYXNzaWduZWQgZm9yIGRhdGEgcGxhbmUgdnMgY29udHJvbCBwbGFuZS4NCg0K SSBhbSBub3QgYWdhaW5zdCBldmFsdWF0aW5nIGFsdGVybmF0aXZlcywgYnV0IHRoZSBhbHRlcm5h dGl2ZSBhcHByb2FjaGVzIG5lZWQgdG8gaGF2ZSBzaW1pbGFyIChub3QgdGhlIHNhbWUpIHBlcmZv cm1hbmNlLg0KDQo+IA0KPiBLb25zdGFudGluDQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EF3BEA00C4; Thu, 30 Jun 2022 17:40:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8CBF34069D; Thu, 30 Jun 2022 17:40:06 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id 53CAD40694; Thu, 30 Jun 2022 17:40:05 +0200 (CEST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 Subject: RE: Optimizations are not features Date: Thu, 30 Jun 2022 17:39:57 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D8719A@smartserver.smartshare.dk> In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Optimizations are not features Thread-Index: Adh38saBhCzWrq13QiKXVs+Vki9E2AAA1LwAAAD0FQAAAnFmgAACayUAAAEhFQAAXbQwAARvikLwAFOb5mA= References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> <497ed526-2fb7-805a-f72c-5909b85a62c1@yandex.ru> From: =?UTF-8?B?TW9ydGVuIEJyw7hydXA=?= To: "Honnappa Nagarahalli" , "Konstantin Ananyev" , "Andrew Rybchenko" , "Jerin Jacob" Cc: "dpdk-dev" , , "nd" , "nd" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org PiBGcm9tOiBIb25uYXBwYSBOYWdhcmFoYWxsaSBbbWFpbHRvOkhvbm5hcHBhLk5hZ2FyYWhhbGxp QGFybS5jb21dDQo+IFNlbnQ6IFdlZG5lc2RheSwgMjkgSnVuZSAyMDIyIDIyLjQ0DQo+IA0KPiA8 c25pcD4NCj4gDQo+ID4NCj4gPiAwNC8wNi8yMDIyIDEzOjUxLCBBbmRyZXcgUnliY2hlbmtvINC/ 0LjRiNC10YI6DQo+ID4gPiBPbiA2LzQvMjIgMTU6MTksIE1vcnRlbiBCcsO4cnVwIHdyb3RlOg0K PiA+ID4+PiBGcm9tOiBKZXJpbiBKYWNvYiBbbWFpbHRvOmplcmluamFjb2JrQGdtYWlsLmNvbV0N Cj4gPiA+Pj4gU2VudDogU2F0dXJkYXksIDQgSnVuZSAyMDIyIDEzLjEwDQo+ID4gPj4+DQo+ID4g Pj4+IE9uIFNhdCwgSnVuIDQsIDIwMjIgYXQgMzozMCBQTSBBbmRyZXcgUnliY2hlbmtvDQo+ID4g Pj4+IDxhbmRyZXcucnliY2hlbmtvQG9rdGV0bGFicy5ydT4gd3JvdGU6DQo+ID4gPj4+Pg0KPiA+ ID4+Pj4gT24gNi80LzIyIDEyOjMzLCBKZXJpbiBKYWNvYiB3cm90ZToNCj4gPiA+Pj4+PiBPbiBT YXQsIEp1biA0LCAyMDIyIGF0IDI6MzkgUE0gTW9ydGVuIEJyw7hydXANCj4gPiA+Pj4gPG1iQHNt YXJ0c2hhcmVzeXN0ZW1zLmNvbT4gd3JvdGU6DQo+ID4gPj4+Pj4+DQo+ID4gPj4+Pj4+IEkgd291 bGQgbGlrZSB0aGUgRFBESyBjb21tdW5pdHkgdG8gY2hhbmdlIGl0cyB2aWV3IG9uIGNvbXBpbGUN Cj4gPiA+Pj4+Pj4gdGltZQ0KPiA+ID4+PiBvcHRpb25zLiBIZXJlIGlzIHdoeToNCj4gPiA+Pj4+ Pj4NCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4gQXBwbGljYXRpb24gc3BlY2lm aWMgcGVyZm9ybWFuY2UgbWljcm8tb3B0aW1pemF0aW9ucyBsaWtlDQo+IOKAnGZhc3QNCj4gPiA+ Pj4gbWJ1ZiBmcmVl4oCdIGFuZCDigJxtYnVmIGRpcmVjdCByZS1hcm3igJ0gYXJlIGJlaW5nIGFk ZGVkIHRvIERQREsgYW5kDQo+ID4gPj4+IHByZXNlbnRlZCBhcyBmZWF0dXJlcy4NCj4gPiA+Pj4+ Pj4NCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4gVGhleSBhcmUgbm90IGZlYXR1 cmVzLCBidXQgb3B0aW1pemF0aW9ucywgYW5kIEkgZG9u4oCZdA0KPiB1bmRlcnN0YW5kDQo+ID4g Pj4+IHRoZSBuZWVkIGZvciB0aGVtIHRvIGJlIGF2YWlsYWJsZSBhdCBydW4tdGltZSENCj4gPiA+ Pj4+Pj4NCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4gSW5zdGVhZCBvZiBhZGRp bmcgYSBidW5jaCBvZiBleG90aWMgZXhjZXB0aW9ucyB0byB0aGUgZmFzdA0KPiBwYXRoDQo+ID4g Pj4+Pj4+IG9mDQo+ID4gPj4+IHRoZSBQTURzLCB0aGV5IHNob3VsZCBiZSBjb21waWxlIHRpbWUg b3B0aW9ucy4gVGhpcyB3aWxsIGltcHJvdmUNCj4gPiA+Pj4gcGVyZm9ybWFuY2UgYnkgYXZvaWRp bmcgYnJhbmNoZXMgaW4gdGhlIGZhc3QgcGF0aCwgYm90aCBmb3IgdGhlDQo+ID4gPj4+IGFwcGxp Y2F0aW9ucyB1c2luZyB0aGVtLCBhbmQgZm9yIGdlbmVyaWMgYXBwbGljYXRpb25zICh3aGVyZSB0 aGUNCj4gPiA+Pj4gZXhvdGljIGNvZGUgaXMgb21pdHRlZCkuDQo+ID4gPj4+Pj4NCj4gPiA+Pj4+ PiBBZ3JlZS4gSSB0aGluaywga2VlcGluZyB0aGUgYmVzdCBvZiBib3RoIHdvcmxkcyB3b3VsZCBi ZQ0KPiA+ID4+Pj4+DQo+ID4gPj4+Pj4gLUVuYWJsZSB0aGUgZmVhdHVyZS9vcHRpbWl6YXRpb24g YXMgcnVudGltZSAtSGF2ZSBhIGNvbXBpbGUtDQo+IHRpbWUNCj4gPiA+Pj4+PiBvcHRpb24gdG8g ZGlzYWJsZSB0aGUgZmVhdHVyZS9vcHRpbWl6YXRpb24gYXMNCj4gPiA+Pj4gYW4gb3ZlcnJpZGUu DQo+ID4gPj4+Pg0KPiA+ID4+Pj4gSXQgaXMgaGFyZCB0byBmaW5kIHRoZSByaWdodCBiYWxhbmNl LCBidXQgaW4gZ2VuZXJhbCBjb21waWxlDQo+IHRpbWUNCj4gPiA+Pj4+IG9wdGlvbnMgYXJlIGEg bmlnaHRtYXJlIGZvciBtYWludGVuYW5jZS4gTnVtYmVyIG9mIHJlcXVpcmVkDQo+IGJ1aWxkcw0K PiA+ID4+Pj4gd2lsbCBncm93IGFzIGFuIGV4cG9uZW50Lg0KPiA+ID4+DQo+ID4gPj4gVGVzdCBj b21iaW5hdGlvbnMgYXJlIGV4cG9uZW50aWFsIGZvciBOIGZlYXR1cmVzLCByZWdhcmRsZXNzIGlm IE4NCj4gYXJlDQo+ID4gPj4gcnVudGltZSBvciBjb21waWxlIHRpbWUgb3B0aW9ucy4NCj4gPiA+ DQo+ID4gPiBCdXQgc2luY2UgSSdtIHRhbGtpbmcgYWJvdXQgYnVpbGQgY2hlY2tzIEkgZG9uJ3Qg Y2FyZSBhYm91dA0KPiA+ID4gZXhwb25lbnRpYWwgZ3Jvd3MgaW4gcnVuIHRpbWUuIFllcywgdGVz dGluZyBzaG91bGQgY2FyZSwgYnV0IGl0IGlzDQo+IGEgc2VwYXJhdGUNCj4gPiBzdG9yeS4NCj4g PiA+DQo+ID4gPj4NCj4gPiA+Pj4+IE9mIGNvdXJzZSwgd2UgY2FuDQo+ID4gPj4+PiBsaW1pdCBu dW1iZXIgb2YgY2hlY2tlZCBjb21iaW5hdGlvbnMsIGJ1dCBpdCB3aWxsIHJlc3VsdCBpbiBmbG93 DQo+IG9mDQo+ID4gPj4+PiBwYXRjaGVzIHRvIGZpeCBidWlsZCBpbiBvdGhlciBjYXNlcy4NCj4g PiA+Pj4NCj4gPiA+Pj4gVGhlIGJ1aWxkIGJyZWFrYWdlIGNhbiBiZSBmaXhlZCBpZiB3ZSB1c2Ug KDIpIHZzICgxKQ0KPiA+ID4+Pg0KPiA+ID4+PiAxKQ0KPiA+ID4+PiAjaWZkZWYgLi4uDQo+ID4g Pj4+IE15IGZlYXR1cmUNCj4gPiA+Pj4gI2VuZGlmDQo+ID4gPj4+DQo+ID4gPj4+IDIpDQo+ID4g Pj4+IHN0YXRpYyBfX3J0ZV9hbHdheXNfaW5saW5lIGludA0KPiA+ID4+PiBydGVfaGFzX3h5el9m ZWF0dXJlKHZvaWQpDQo+ID4gPj4+IHsNCj4gPiA+Pj4gI2lmZGVmIFJURV9MSUJSVEVfWFlaX0ZF QVRVUkUNCj4gPiA+Pj4gwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gUlRFX0xJQlJURV9YWVpfRkVB VFVSRTsgI2Vsc2UNCj4gPiA+Pj4gwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gMDsNCj4gPiA+Pj4g I2VuZGlmDQo+ID4gPj4+IH0NCj4gPiA+Pj4NCj4gPiA+Pj4gaWYocnRlX2hhc194eXpfZmVhdHVy ZSgpKSkgew0KPiA+ID4+PiBNeSBmZWF0dXJlIGNvZGUNCj4gPiA+Pj4NCj4gPiA+Pj4gfQ0KPiA+ ID4+Pg0KPiA+ID4NCj4gPiA+IEplcmluLCB0aGFua3MsIHZlcnkgZ29vZCBleGFtcGxlLg0KPiA+ ID4NCj4gPiA+PiBJJ20gbm90IHN1cmUgYWxsIHRoZSBmZWF0dXJlcyBjYW4gYmUgY292ZXJlZCBi eSB0aGF0LCBlLmcuIGFkZGVkDQo+ID4gPj4gZmllbGRzIGluIHN0cnVjdHVyZXMuDQo+ID4gPg0K PiA+ID4gKzENCj4gPiA+DQo+ID4gPj4NCj4gPiA+PiBBbHNvLCBJIHdvdWxkIGNvbnNpZGVyIHN1 Y2ggZmVhdHVyZXMgIm9wdCBpbiIgYXQgY29tcGlsZSB0aW1lDQo+IG9ubHkuDQo+ID4gPj4gQXMg c3VjaCwgdGhleSBjb3VsZCBiZSBhbGxvd2VkIHRvIGJyZWFrIHRoZSBBQkkvQVBJLg0KPiA+ID4+ DQo+ID4gPj4+DQo+ID4gPj4+DQo+ID4gPj4+PiBBbHNvIGNvbXBpbGUgdGltZSBvcHRpb25zIHRl bmQgdG8gbWFrZSBjb2RlIGxlc3MgcmVhZGFibGUgd2hpY2gNCj4gPiA+Pj4+IG1ha2VzIGFsbCBh c3BlY3RzIG9mIHRoZSBkZXZlbG9wbWVudCBoYXJkZXIuDQo+ID4gPj4+Pg0KPiA+ID4+Pj4gWWVz LCBjb21waWxlIHRpbWUgaXMgbmljZSBmb3IgbWljcm8gb3B0aW1pemF0aW9ucywgYnV0IEkgaGF2 ZQ0KPiBncmVhdA0KPiA+ID4+Pj4gY29uY2VybnMgdGhhdCBpdCBpcyBhIHJpZ2h0IHdheSB0byBn by4NCj4gPiA+Pj4+DQo+ID4gPj4+Pj4+IFBsZWFzZSBub3RlIHRoYXQgSSBhbSBvbmx5IHRhbGtp bmcgYWJvdXQgdGhlIHBlcmZvcm1hbmNlDQo+ID4gPj4+IG9wdGltaXphdGlvbnMgdGhhdCBhcmUg bGltaXRlZCB0byBhcHBsaWNhdGlvbiBzcGVjaWZpYyB1c2UgY2FzZXMuDQo+IEkNCj4gPiA+Pj4g dGhpbmsgaXQgbWFrZXMgc2Vuc2UgdG8gcmVxdWlyZSB0aGF0IHBlcmZvcm1hbmNlIG9wdGltaXpp bmcgYW4NCj4gPiA+Pj4gYXBwbGljYXRpb24gYWxzbyByZXF1aXJlcyByZWNvbXBpbGluZyB0aGUg cGVyZm9ybWFuY2UgY3JpdGljYWwNCj4gPiA+Pj4gbGlicmFyaWVzIHVzZWQgYnkgaXQuDQo+ID4g Pj4+Pj4+YWJhbmRvbiBzb21lIG9mIGV4aXN0aW5nIGZ1bmN0aW9uYWxpdHkgdG8gY3JlYXRlIGEg J3Nob3J0LWN1dCcNCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4NCj4gPiA+Pj4+Pj4gQWxsb3dpbmcg Y29tcGlsZSB0aW1lIG9wdGlvbnMgZm9yIGFwcGxpY2F0aW9uIHNwZWNpZmljDQo+ID4gPj4+Pj4+ IHBlcmZvcm1hbmNlDQo+ID4gPj4+IG9wdGltaXphdGlvbnMgaW4gRFBESyB3b3VsZCBhbHNvIG9w ZW4gYSBwYXRoIGZvciBvdGhlcg0KPiA+ID4+PiBvcHRpbWl6YXRpb25zLCB3aGljaCBjYW4gb25s eSBiZSBhY2hpZXZlZCBhdCBjb21waWxlIHRpbWUsIHN1Y2gNCj4gYXMNCj4gPiA+Pj4g4oCcbm8g ZnJhZ21lbnRlZCBwYWNrZXRz4oCdLCDigJxubyBhdHRhY2hlZCBtYnVmc+KAnSBhbmQg4oCcc2lu Z2xlIG1idWYNCj4gcG9vbOKAnS4NCj4gPiA+Pj4gQW5kIGV2ZW4gbW9yZSBleG90aWMgb3B0aW1p emF0aW9ucywgc3VjaCBhcyB0aGUg4oCcaW5kZXhlZCBtZW1wb29sDQo+ID4gPj4+IGNhY2hl4oCd LCB3aGljaCB3YXMgcmVqZWN0ZWQgZHVlIHRvIEFCSSB2aW9sYXRpb25zIOKAkyB0aGV5IGNvdWxk IGJlDQo+ID4gPj4+IG1hcmtlZCBhcyDigJxyaXNreSBhbmQgdW50ZXN0ZWTigJ0gb3Igc2ltaWxh ciwgYnV0IHN0aWxsIGJlIHBhcnQgb2YNCj4gdGhlIERQREsgbWFpbg0KPiA+IHJlcG9zaXRvcnku DQo+ID4gPj4+Pj4+DQo+ID4NCj4gPg0KPiA+IFRoYW5rcyBNb3J0ZW4gZm9yIGJyaW5naW5nIGl0 IHVwLCBpdCBpcyBhbiBpbnRlcmVzdGluZyB0b3BpYy4NCj4gPiBUaG91Z2ggSSBsb29rIGF0IGl0 IGZyb20gZGlmZmVyZW50IGFuZ2xlLg0KPiA+IEFsbCBvcHRpbWl6YXRpb25zIHlvdSBtZW50aW9u ZWQgYWJvdmUgaW50cm9kdWNlIG5ldyBsaW1pdGF0aW9uczoNCj4gPiBNQlVGX0ZBU1RfRlJFRSAt IG5vIGluZGlyZWN0IG1idWZzIGFuZCBtdWx0aXBsZSBtZW1wb29scywgbWVtcG9vbA0KPiBvYmpl Y3QNCj4gPiBpbmRleGVzIC0gbWVtcG9vbCBzaXplIGlzIGxpbWl0ZWQgdG8gNEdCLCBkaXJlY3Qg cmVhcm0gLSBkcm9wIGFiaWxpdHkNCj4gdG8NCj4gPiBzdG9wL3JlY29uZmlndXJlIFRYIHF1ZXVl LCB3aGlsZSBSWCBxdWV1ZSBpcyBzdGlsbCBydW5uaW5nLCBldGMuDQo+ID4gTm90ZSB0aGF0IGFs bCB0aGVzZSBsaW1pdGF0aW9ucyBhcmUgbm90IGZvcmNlZCBieSBIVy4NCj4gPiBBbGwgb2YgdGhl bSBhcmUgcHVyZSBTVyBsaW1pdGF0aW9ucyB0aGF0IGRldmVsb3BlcnMgZm9yY2VkIGluIChvcg0K PiB0cmllZCB0bykgdG8gZ2V0DQo+ID4gZmV3IGV4dHJhIHBlcmZvcm1hbmNlLg0KPiA+IFRoYXQn cyBjb25jZXJuaW5nIHRlbmRlbmN5Lg0KPiA+DQo+ID4gQXMgbW9yZSBhbmQgbW9yZSBzdWNoICdv cHRpbWl6YXRpb24gdmlhIGxpbWl0YXRpb24nIHdpbGwgY29tZSBpbjoNCj4gPiAtIERQREsgZmVh dHVyZSBsaXN0IHdpbGwgYmVjb21lIG1vcmUgYW5kIG1vcmUgZnJhZ21lbnRlZC4NCj4gPiAtIFdv dWxkIGNhdXNlIG1vcmUgYW5kIG1vcmUgY29uZnVzaW9uIGZvciB0aGUgdXNlcnMuDQo+ID4gLSBV bm1ldCBleHBlY3RhdGlvbnMgLSBkaWZmZXJlbmNlIGluIHBlcmZvcm1hbmNlIGJldHdlZW4gJ2Rl ZmF1bHQnDQo+ID4gICAgYW5kICdvcHRpbWl6ZWQnIHZlcnNpb24gb2YgRFBESyB3aWxsIGJlY29t ZSBiaWdnZXIgYW5kIGJpZ2dlci4NCg0KSSBzdHJvbmdseSBkaXNhZ3JlZSB3aXRoIHRoaXMgYnVs bGV0IQ0KDQpXZSBzaG91bGQgbm90IGxpbWl0IHRoZSBwZXJmb3JtYW5jZSB0byBvbmx5IHdoYXQg aXMgcG9zc2libGUgd2l0aCBhbGwgZmVhdHVyZXMgZW5hYmxlZC4NCg0KQW4gYXBwbGljYXRpb24g ZGV2ZWxvcGVyIHNob3VsZCBoYXZlIHRoZSBhYmlsaXR5IHRvIGRpc2FibGUgcGVyZm9ybWFuY2Ut Y29zdGx5IGZlYXR1cmVzIG5vdCBiZWluZyB1c2VkLg0KDQo+ID4gLSBBcyBBbmRyZXcgYWxyZWFk eSBtZW50aW9uZWQsIG1haW50YWluaW5nIGFsbCB0aGVzZSAnc3ViLWZsYXZvdXJzJw0KPiA+ICAg IG9mIERQREsgd2lsbCBiZWNvbWUgbW9yZSBhbmQgbW9yZSBkaWZmaWN1bHQuDQo+IFRoZSBwb2lu dCB0aGF0IHdlIG5lZWQgdG8gcmVtZW1iZXIgaXMsIHRoZXNlIGZlYXR1cmVzL29wdGltaXphdGlv bnMgYXJlDQo+IGludHJvZHVjZWQgYWZ0ZXIgc2VlaW5nIHBlcmZvcm1hbmNlIGlzc3VlcyBpbiBw cmFjdGljYWwgdXNlIGNhc2VzLg0KPiBEUERLIGlzIG5vdCBiZWluZyB1c2VkIGluIGp1c3Qgb25l IHVzZSBjYXNlLCBpdCBpcyBiZWluZyB1c2VkIGluDQo+IHNldmVyYWwgdXNlIGNhc2VzIHdoaWNo IGhhdmUgdGhlaXIgb3duIHVuaXF1ZSByZXF1aXJlbWVudHMuIElzIDRHQg0KPiBlbm91Z2ggZm9y IHBhY2tldCBidWZmZXJzIC0geWVzIGl0IGlzIGVub3VnaCBpbiBjZXJ0YWluIHVzZSBjYXNlcy4g QXJlDQo+IHRoZWlyIE5JQ3Mgd2l0aCBzaW5nbGUgcG9ydCAtIHllcyB0aGVyZSBhcmUuIEhXIGlz IGJlaW5nIGNyZWF0ZWQNCj4gYmVjYXVzZSB1c2UgY2FzZXMgYW5kIGJ1c2luZXNzIGNhc2VzIGV4 aXN0LiBJdCBpcyBvYnZpb3VzIHRoYXQgYXMgRFBESw0KPiBnZXRzIGFkb3B0ZWQgb24gbW9yZSBw bGF0Zm9ybXMgdGhhdCBkaWZmZXIgbGFyZ2VseSwgdGhlIGZlYXR1cmVzIHdpbGwNCj4gaW5jcmVh c2UgYW5kIGl0IHdpbGwgYmVjb21lIGNvbXBsZXguIENvbXBsZXhpdHkgc2hvdWxkIG5vdCBiZSB1 c2VkIGFzIGENCj4gY3JpdGVyaWEgdG8gcmVqZWN0IHBhdGNoZXMuDQo+IA0KPiBUaGVyZSBpcyBk aWZmZXJlbnQgcGVyc3BlY3RpdmUgdG8gd2hhdCB5b3UgYXJlIGNhbGxpbmcgYXMNCj4gJ2xpbWl0 YXRpb25zJy4gSSBjYW4gYXJndWUgdGhhdCBtdWx0aXBsZSBtZW1wb29scywgc3RvcC9yZWNvbmZp Z3VyZSBUWA0KPiBxdWV1ZSB3aGlsZSBSWCBxdWV1ZSBpcyBzdGlsbCBydW5uaW5nIGFyZSBleG90 aWMuIEp1c3QgYmVjYXVzZSB0aG9zZQ0KPiBhcmUgYWxsb3dlZCBjdXJyZW50bHkgKHByb2JhYmx5 IGFjY2lkZW50bHkpIGRvZXMgbm90IG1lYW4gdGhleSBhcmUNCj4gYmVpbmcgdXNlZC4gQXJlIHRo ZXJlIHVzZSBjYXNlcyB0aGF0IG1ha2UgdXNlIG9mIHRoZXNlIGZlYXR1cmVzPw0KPiANCj4gVGhl IGJhc2UvZXhpc3RpbmcgZGVzaWduIGZvciBEUERLIHdhcyBkb25lIHdpdGggb25lIHBhcnRpY3Vs YXIgSFcNCj4gYXJjaGl0ZWN0dXJlIGluIG1pbmQgd2hlcmUgdGhlcmUgd2FzIGFuIGFidW5kYW5j ZSBvZiByZXNvdXJjZXMuDQo+IFVuZm9ydHVuYXRlbHksIHRoYXQgSFcgYXJjaGl0ZWN0dXJlIGlz IGZhc3QgZXZvbHZpbmcgYW5kIERQREsgaXMNCj4gYWRvcHRlZCBpbiB1c2UgY2FzZXMgd2hlcmUg dGhhdCBraW5kIG9mIHJlc291cmNlcyBhcmUgbm90IGF2YWlsYWJsZS4NCj4gRm9yIGV4OiBlZmZp Y2llbmN5IGNvcmVzIGFyZSBiZWluZyBpbnRyb2R1Y2VkIGJ5IGV2ZXJ5IENQVSB2ZW5kb3Igbm93 Lg0KPiBTb29uIGVub3VnaCwgd2Ugd2lsbCBzZWUgYmlnLWxpdHRsZSBhcmNoaXRlY3R1cmUgaW4g bmV0d29ya2luZyBhcyB3ZWxsLg0KPiBUaGUgZXhpc3RpbmcgUE1EIGRlc2lnbiBpbnRyb2R1Y2Vz IDUxMkIgb2Ygc3RvcmVzICgyNTZCIGZvciBjb3B5aW5nIHRvDQo+IHN0YWNrIHZhcmlhYmxlIGFu ZCAyNTZCIHRvIHN0b3JlIGxjb3JlIGNhY2hlKSBhbmQgMjU2QiBsb2FkL3N0b3JlIG9uIFJYDQo+ IHNpZGUgZXZlcnkgMzIgcGFja2V0cyBiYWNrIHRvIGJhY2suIEl0IGRvZXNuJ3QgbWFrZSBzZW5z ZSB0byBoYXZlIHRoYXQNCj4ga2luZCBvZiBtZW1jb3B5IGZvciBsaXR0bGUvZWZmaWNpZW5jeSBj b3JlcyBqdXN0IGZvciB0aGUgZHJpdmVyIGNvZGUuDQo+IA0KPiA+DQo+ID4gU28sIHByb2JhYmx5 IGluc3RlYWQgb2YgbWFraW5nIHN1Y2ggY2hhbmdlcyBlYXNpZXIsIHdlIG5lZWQgc29tZWhvdw0K PiB0bw0KPiA+IHBlcnN1YWRlIGRldmVsb3BlcnMgdG8gdGhpbmsgbW9yZSBhYm91dCBvcHRpbWl6 YXRpb25zIHRoYXQgd291bGQgYmUNCj4gZ2VuZXJpYw0KPiA+IGFuZCB0cmFuc3BhcmVudCB0byB0 aGUgdXNlci4NCj4gT3IgbWF5IGJlIHdlIG5lZWQgdG8gdGhpbmsgb2YgY3JlYXRpbmcgYWx0ZXJu YXRlIHdheXMgb2YgcHJvZ3JhbW1pbmcuDQoNCkV4YWN0bHkgd2hhdCBJIHdhcyBob3BpbmcgdG8g YWNoaWV2ZSB3aXRoIHRoaXMgZGlzY3Vzc2lvbi4NCg0KPiANCj4gPiBJIGRvIHJlYWxpemUgdGhh dCBpdCBpcyBub3QgYWx3YXlzIHBvc3NpYmxlIGR1ZSB0byB2YXJpb3VzIHJlYXNvbnMNCj4gKEhX IGxpbWl0YXRpb25zLA0KPiA+IGV4dGVybmFsIGRlcGVuZGVuY2llcywgZXRjLikgYnV0IHRoYXQn cyBhbm90aGVyIHN0b3J5Lg0KPiA+DQo+ID4gTGV0J3MgdGFrZSBmb3IgZXhhbXBsZSBNQlVGX0ZB U1RfRlJFRS4NCj4gPiBJbiBmYWN0LCBJIGFtIG5vdCBzdXJlIHRoYXQgd2UgbmVlZCBpdCBhcyB0 eCBvZmZsb2FkIGZsYWcgYXQgYWxsLg0KPiA+IFBNRCBUWC1wYXRoIGhhcyBhbGwgbmVjZXNzYXJ5 IGluZm9ybWF0aW9uIHRvIGRlY2lkZSBhdCBydW4tdGltZSBjYW4NCj4gaXQgZG8NCj4gPiBmYXN0 X2ZyZWUoKSBmb3Igbm90Og0KPiA+IEF0IHR4X2J1cnN0KCkgUE1EIGNhbiBjaGVjayBhcmUgYWxs IG1idWZzIHNhdGlzZnkgdGhlc2UgY29uZGl0aW9ucw0KPiAoc2FtZQ0KPiA+IG1lbXBvb2wsIHJl ZmNudD09MSkgYW5kIHVwZGF0ZSBzb21lIGZpZWxkcyBhbmQvb3IgY291bnRlcnMgaW5zaWRlIFRY UQ0KPiB0bw0KPiA+IHJlZmxlY3QgaXQuDQo+ID4gVGhlbiwgYXQgdHhfZnJlZSgpIHdlIGNhbiB1 c2UgdGhpcyBpbmZvIHRvIGRlY2lkZSBiZXR3ZWVuIGZhc3RfZnJlZSgpDQo+IGFuZA0KPiA+IG5v cm1hbF9mcmVlKCkuDQo+ID4gQXMgYXQgdHhfYnVyc3QoKSB3ZSByZWFkIG1idWYgZmllbGRzIGFu eXdheSwgaW1wYWN0IGZvciB0aGlzIGV4dHJhDQo+IHN0ZXAgSSBndWVzcw0KPiA+IHdvdWxkIGJl IG1pbmltYWwuDQo+ID4gWWVzLCBtb3N0IGxpa2VseSwgaXQgd291bGRuJ3QgYmUgYXMgZmFzdCBh cyB3aXRoIGN1cnJlbnQgVFggb2ZmbG9hZA0KPiBmbGFnLCBvcg0KPiA+IGNvbmRpdGlvbmFsIGNv bXBpbGF0aW9uIGFwcHJvYWNoLg0KPiA+IEJ1dCBpdCBtaWdodCBiZSBzdGlsbCBzaWduaWZpY2Fu dGx5IGZhc3RlciB0aGVuIG5vcm1hbF9mcmVlKCksIHBsdXMNCj4gc3VjaCBhcHByb2FjaA0KPiA+ IHdpbGwgYmUgZ2VuZXJpYyBhbmQgdHJhbnNwYXJlbnQgdG8gdGhlIHVzZXIuDQo+IElNTywgdGhp cyBkZXBlbmRzIG9uIHRoZSBwaGlsb3NvcGh5IHRoYXQgd2Ugd2FudCB0byBhZG9wdC4gSSB3b3Vs ZA0KPiBwcmVmZXIgdG8gbWFrZSBjb250cm9sIHBsYW5lIGNvbXBsZXggZm9yIHBlcmZvcm1hbmNl IGdhaW5zIG9uIHRoZSBkYXRhDQo+IHBsYW5lLiBUaGUgcGVyZm9ybWFuY2Ugb24gdGhlIGRhdGEg cGxhbmUgaGFzIGEgbXVsdGlwbHlpbmcgZWZmZWN0IGR1ZQ0KPiB0byB0aGUgcmF0aW8gb2YgbnVt YmVyIG9mIGNvcmVzIGFzc2lnbmVkIGZvciBkYXRhIHBsYW5lIHZzIGNvbnRyb2wNCj4gcGxhbmUu DQoNClllcy4gQW5kIGlmIHNvbWUgcGVyZm9ybWFuY2UtY29zdGluZyBmZWF0dXJlIGlzIG5vdCBw b3NzaWJsZSB0byBtb3ZlIG91dCBmcm9tIHRoZSBkYXRhIHBsYW5lIHRvIHRoZSBjb250cm9sIHBs YW5lLCBpdCBzaG91bGQgYmUgY29tcGlsZSB0aW1lIG9wdGlvbmFsLg0KDQpBbmQgcGxlYXNlIG5v dGUgdGhhdCBJIGRvbid0IGJ1eSB0aGUgYXJndW1lbnQgdGhhdCAiaXQgd2lsbCBiZSBjYXVnaHQg YnkgYnJhbmNoIHByZWRpY3Rpb24iLiBZb3UgYXJlIG5vdCBhbGxvd2VkIHRvIGZpbGwgdXAgbXkg YnJhbmNoIHByZWRpY3RvciB0YWJsZSB3aXRoIGNydWZ0IQ0KDQo+IA0KPiBJIGFtIG5vdCBhZ2Fp bnN0IGV2YWx1YXRpbmcgYWx0ZXJuYXRpdmVzLCBidXQgdGhlIGFsdGVybmF0aXZlDQo+IGFwcHJv YWNoZXMgbmVlZCB0byBoYXZlIHNpbWlsYXIgKG5vdCB0aGUgc2FtZSkgcGVyZm9ybWFuY2UuDQo+ IA0KPiA+DQo+ID4gS29uc3RhbnRpbg0K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7D5EEA04FD; Sun, 3 Jul 2022 21:38:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 27A1D41132; Sun, 3 Jul 2022 21:38:26 +0200 (CEST) Received: from forward501p.mail.yandex.net (forward501p.mail.yandex.net [77.88.28.111]) by mails.dpdk.org (Postfix) with ESMTP id 0D5AD4021F; Sun, 3 Jul 2022 21:38:25 +0200 (CEST) Received: from iva3-32ab41554fb4.qloud-c.yandex.net (iva3-32ab41554fb4.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:c82:0:640:32ab:4155]) by forward501p.mail.yandex.net (Yandex) with ESMTP id 5FE656212768; Sun, 3 Jul 2022 22:38:24 +0300 (MSK) Received: from iva1-dcde80888020.qloud-c.yandex.net (iva1-dcde80888020.qloud-c.yandex.net [2a02:6b8:c0c:7695:0:640:dcde:8088]) by iva3-32ab41554fb4.qloud-c.yandex.net (mxback/Yandex) with ESMTP id MMSWnyi7bJ-cOfiLwkY; Sun, 03 Jul 2022 22:38:24 +0300 X-Yandex-Fwd: 2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1656877104; bh=G7Y+RJ4hHXAF+9fccix1Q+znqkaf9sdhvLJ/YO4+5CU=; h=In-Reply-To:From:Subject:Cc:References:Date:Message-ID:To; b=a2OcSJz+ieckIYQX5dLFPKZaqIf3U8NTtB4LmQNWCk7YemOqMIwW0yNIqy4Lgn227 nUgYzmXGlOW9oeMYltDdXRzlf1xu8aDCsMuSv1sHTHuHn3bDSvWXFAi19GLwdTAzKa RDpUrRliXlbZFYvrrby0XLqQ6GVbildPWpem52nQ= Authentication-Results: iva3-32ab41554fb4.qloud-c.yandex.net; dkim=pass header.i=@yandex.ru Received: by iva1-dcde80888020.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id DoIIL9LIN4-cMMKe6FM; Sun, 03 Jul 2022 22:38:23 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Message-ID: <91f748cd-14c1-91ca-befe-64db36789346@yandex.ru> Date: Sun, 3 Jul 2022 20:38:21 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: Optimizations are not features Content-Language: en-US To: Honnappa Nagarahalli , Andrew Rybchenko , =?UTF-8?Q?Morten_Br=c3=b8rup?= , Jerin Jacob Cc: dpdk-dev , "techboard@dpdk.org" , nd References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> <497ed526-2fb7-805a-f72c-5909b85a62c1@yandex.ru> From: Konstantin Ananyev In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 29/06/2022 21:44, Honnappa Nagarahalli пишет: > > >> >> 04/06/2022 13:51, Andrew Rybchenko пишет: >>> On 6/4/22 15:19, Morten Brørup wrote: >>>>> From: Jerin Jacob [mailto:jerinjacobk@gmail.com] >>>>> Sent: Saturday, 4 June 2022 13.10 >>>>> >>>>> On Sat, Jun 4, 2022 at 3:30 PM Andrew Rybchenko >>>>> wrote: >>>>>> >>>>>> On 6/4/22 12:33, Jerin Jacob wrote: >>>>>>> On Sat, Jun 4, 2022 at 2:39 PM Morten Brørup >>>>> wrote: >>>>>>>> >>>>>>>> I would like the DPDK community to change its view on compile >>>>>>>> time >>>>> options. Here is why: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Application specific performance micro-optimizations like “fast >>>>> mbuf free” and “mbuf direct re-arm” are being added to DPDK and >>>>> presented as features. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> They are not features, but optimizations, and I don’t understand >>>>> the need for them to be available at run-time! >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Instead of adding a bunch of exotic exceptions to the fast path >>>>>>>> of >>>>> the PMDs, they should be compile time options. This will improve >>>>> performance by avoiding branches in the fast path, both for the >>>>> applications using them, and for generic applications (where the >>>>> exotic code is omitted). >>>>>>> >>>>>>> Agree. I think, keeping the best of both worlds would be >>>>>>> >>>>>>> -Enable the feature/optimization as runtime -Have a compile-time >>>>>>> option to disable the feature/optimization as >>>>> an override. >>>>>> >>>>>> It is hard to find the right balance, but in general compile time >>>>>> options are a nightmare for maintenance. Number of required builds >>>>>> will grow as an exponent. >>>> >>>> Test combinations are exponential for N features, regardless if N are >>>> runtime or compile time options. >>> >>> But since I'm talking about build checks I don't care about >>> exponential grows in run time. Yes, testing should care, but it is a separate >> story. >>> >>>> >>>>>> Of course, we can >>>>>> limit number of checked combinations, but it will result in flow of >>>>>> patches to fix build in other cases. >>>>> >>>>> The build breakage can be fixed if we use (2) vs (1) >>>>> >>>>> 1) >>>>> #ifdef ... >>>>> My feature >>>>> #endif >>>>> >>>>> 2) >>>>> static __rte_always_inline int >>>>> rte_has_xyz_feature(void) >>>>> { >>>>> #ifdef RTE_LIBRTE_XYZ_FEATURE >>>>> return RTE_LIBRTE_XYZ_FEATURE; #else >>>>> return 0; >>>>> #endif >>>>> } >>>>> >>>>> if(rte_has_xyz_feature())) { >>>>> My feature code >>>>> >>>>> } >>>>> >>> >>> Jerin, thanks, very good example. >>> >>>> I'm not sure all the features can be covered by that, e.g. added >>>> fields in structures. >>> >>> +1 >>> >>>> >>>> Also, I would consider such features "opt in" at compile time only. >>>> As such, they could be allowed to break the ABI/API. >>>> >>>>> >>>>> >>>>>> Also compile time options tend to make code less readable which >>>>>> makes all aspects of the development harder. >>>>>> >>>>>> Yes, compile time is nice for micro optimizations, but I have great >>>>>> concerns that it is a right way to go. >>>>>> >>>>>>>> Please note that I am only talking about the performance >>>>> optimizations that are limited to application specific use cases. I >>>>> think it makes sense to require that performance optimizing an >>>>> application also requires recompiling the performance critical >>>>> libraries used by it. >>>>>>>> abandon some of existing functionality to create a 'short-cut' >>>>>>>> >>>>>>>> >>>>>>>> Allowing compile time options for application specific >>>>>>>> performance >>>>> optimizations in DPDK would also open a path for other >>>>> optimizations, which can only be achieved at compile time, such as >>>>> “no fragmented packets”, “no attached mbufs” and “single mbuf pool”. >>>>> And even more exotic optimizations, such as the “indexed mempool >>>>> cache”, which was rejected due to ABI violations – they could be >>>>> marked as “risky and untested” or similar, but still be part of the DPDK main >> repository. >>>>>>>> >> >> >> Thanks Morten for bringing it up, it is an interesting topic. >> Though I look at it from different angle. >> All optimizations you mentioned above introduce new limitations: >> MBUF_FAST_FREE - no indirect mbufs and multiple mempools, mempool object >> indexes - mempool size is limited to 4GB, direct rearm - drop ability to >> stop/reconfigure TX queue, while RX queue is still running, etc. >> Note that all these limitations are not forced by HW. >> All of them are pure SW limitations that developers forced in (or tried to) to get >> few extra performance. >> That's concerning tendency. >> >> As more and more such 'optimization via limitation' will come in: >> - DPDK feature list will become more and more fragmented. >> - Would cause more and more confusion for the users. >> - Unmet expectations - difference in performance between 'default' >> and 'optimized' version of DPDK will become bigger and bigger. >> - As Andrew already mentioned, maintaining all these 'sub-flavours' >> of DPDK will become more and more difficult. > The point that we need to remember is, these features/optimizations are introduced after seeing performance issues in practical use cases. Sorry I didn't get it: what performance issues you are talking about? If let say our mempool code is sub-optimal in some place for some architecture due to bad design or bad implementation - please point to it and let's try to fix it, instead of avoiding using mempool API If you just saying that avoiding using mempool in some cases could buy us few extra performance (a short-cut), then yes it surely could. Another question - is it really worth it? Having all mbufs management covered by one SW abstraction helps a lot in terms of project maintainability, further extensions, introducing new common optimizations, etc. > DPDK is not being used in just one use case, it is being used in several use cases which have their own unique requirements. Is 4GB enough for packet buffers - yes it is enough in certain use cases. Are their NICs with single port - yes there are. Sure there are NICs with one port. But also there are NICs with 2 ports, 4 ports, etc. Should we maintain specific DPDK sub-versions for all these cases? From my perspective - no. It would be overwhelming effort for DPDK community, plus many customers use DPDK to build their own products that supposed to work seamlessly across multiple use-cases/platforms. HW is being created because use cases and business cases exist. It is obvious that as DPDK gets adopted on more platforms that differ largely, the features will increase and it will become complex. Complexity should not be used as a criteria to reject patches. Well, we do have plenty of HW specific optimizations inside DPDK and we put a lot of effort that all this HW specific staff be transparent to the user as much as possible. I don't see why for SW specific optimizations it should be different. > > There is different perspective to what you are calling as 'limitations'. By 'limitations' I mean situation when user has to cut off existing functionality to enable these 'optimizations'. I can argue that multiple mempools, stop/reconfigure TX queue while RX queue is still running are exotic. Just because those are allowed currently (probably accidently) does not mean they are being used. Are there use cases that make use of these features? If DPDK examples/l3fwd doesn't use these features, it doesn't mean they are useless :) I believe both multiple mempools (indirect-mbufs) and ability to start/stop queues separately are major DPDK features that are used across many real-world deployments. > > The base/existing design for DPDK was done with one particular HW architecture in mind where there was an abundance of resources. Unfortunately, that HW architecture is fast evolving and DPDK is adopted in use cases where that kind of resources are not available. For ex: efficiency cores are being introduced by every CPU vendor now. Soon enough, we will see big-little architecture in networking as well. The existing PMD design introduces 512B of stores (256B for copying to stack variable and 256B to store lcore cache) and 256B load/store on RX side every 32 packets back to back. It doesn't make sense to have that kind of memcopy for little/efficiency cores just for the driver code. I don't object about specific use-case optimizations. Specially if the use-case is a common one. But I think such changes has to be transparent to the user as much as possible and shouldn't cause further DPDK code fragmentation (new CONFIG options, etc.). I understand that it is not always possible, but for pure SW based optimizations, I think it is a reasonable expectation. >> >> So, probably instead of making such changes easier, we need somehow to >> persuade developers to think more about optimizations that would be generic >> and transparent to the user. > Or may be we need to think of creating alternate ways of programming. > >> I do realize that it is not always possible due to various reasons (HW limitations, >> external dependencies, etc.) but that's another story. >> >> Let's take for example MBUF_FAST_FREE. >> In fact, I am not sure that we need it as tx offload flag at all. >> PMD TX-path has all necessary information to decide at run-time can it do >> fast_free() for not: >> At tx_burst() PMD can check are all mbufs satisfy these conditions (same >> mempool, refcnt==1) and update some fields and/or counters inside TXQ to >> reflect it. >> Then, at tx_free() we can use this info to decide between fast_free() and >> normal_free(). >> As at tx_burst() we read mbuf fields anyway, impact for this extra step I guess >> would be minimal. >> Yes, most likely, it wouldn't be as fast as with current TX offload flag, or >> conditional compilation approach. >> But it might be still significantly faster then normal_free(), plus such approach >> will be generic and transparent to the user. > IMO, this depends on the philosophy that we want to adopt. I would prefer to make control plane complex for performance gains on the data plane. The performance on the data plane has a multiplying effect due to the ratio of number of cores assigned for data plane vs control plane. > > I am not against evaluating alternatives, but the alternative approaches need to have similar (not the same) performance. > >> >> Konstantin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CEB44A0540; Mon, 4 Jul 2022 18:33:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 572BA40E50; Mon, 4 Jul 2022 18:33:17 +0200 (CEST) Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by mails.dpdk.org (Postfix) with ESMTP id CC3C94021D for ; Mon, 4 Jul 2022 18:33:16 +0200 (CEST) Received: by mail-pj1-f51.google.com with SMTP id a15so3084085pjs.0 for ; Mon, 04 Jul 2022 09:33:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/i1j6HiQsr1l3IXEF9pj7eNKhtg/8hHyMsvIJEKMJfY=; b=Qy/ewOK9pJ+VBZBtpDFevmLw5s3cpIvEmnverDqF8UmL2CDSizi7Cj4O6c6kx8731W XQClXgtOwQJPwT6oInz/pAZOq/dxnC8D8m29/V9NIDfZa5xiAG1kAD1LqXvONOEKlOSa UfkPHKK1wDkZYqjGVcWrGRzy83zJlbzR+0q5XRym5aar8kCml+iLvV2xbwIufNjqQY+R TjF9gmKHwk56sIp5FFUSaFBZI4k5iBNGNg5BK4OVlpYOVCo7icK13C70p5me6fctz9tg Apk23iGk/bbMkRlPWpKQVJxWC6Hr8S3oNUYLUGBzx9H4EUb11bXWM31z98YwtOmPY50M b4ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/i1j6HiQsr1l3IXEF9pj7eNKhtg/8hHyMsvIJEKMJfY=; b=z5WVE5DRoy4O05b10hnKCX0D5PGYugKGB8R0hWGQfrpeyfv6rXRrHeOJhJ9S6R+2o9 O93nf+tLqDns5fvUZ4+e7mOrDNRvqw/6pQ1d9Eu49/M6woh6h4QsACayLDsgWuT7q580 dlxt5m8gEMOHln2DBm/3FdYGsjw8zDt5XvNkWfHt/Kmw8uDi2X3SD68CALMc3AwggXg2 HxctvArh3+Oan5Li2p0aY7qyJ+Buvqweree7fHcUhXXHZoLGwEWGtwXGxu9CB4+EyDud wKAFB++wHRJFtzy2+C90eGv4Fiyz1y6cETSX6KSQaBsVp9iECMwdFARQ6ZdVSWYflEiY rD9Q== X-Gm-Message-State: AJIora/Q//td6N+20OS6nutVMWLEH1EIArIpAQdtkrH1IhUvpUkLRAlA KRFypEri6zagh46pUs/MzNMNng== X-Google-Smtp-Source: AGRyM1sRjn0LmVasyNzEDZ5/MJKcEXkPlbYmG74iM4234bNuNgyEJyylXD70CID4MDHwB3hKaKCl2Q== X-Received: by 2002:a17:902:9f88:b0:16b:e3f3:cb6c with SMTP id g8-20020a1709029f8800b0016be3f3cb6cmr6004593plq.14.1656952395854; Mon, 04 Jul 2022 09:33:15 -0700 (PDT) Received: from hermes.local (204-195-112-199.wavecable.com. [204.195.112.199]) by smtp.gmail.com with ESMTPSA id s7-20020a17090302c700b00168e83eda56sm21505869plk.3.2022.07.04.09.33.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Jul 2022 09:33:14 -0700 (PDT) Date: Mon, 4 Jul 2022 09:33:11 -0700 From: Stephen Hemminger To: Konstantin Ananyev Cc: Honnappa Nagarahalli , Andrew Rybchenko , Morten =?UTF-8?B?QnLDuHJ1cA==?= , Jerin Jacob , dpdk-dev , "techboard@dpdk.org" , nd Subject: Re: Optimizations are not features Message-ID: <20220704093311.0582d592@hermes.local> In-Reply-To: <91f748cd-14c1-91ca-befe-64db36789346@yandex.ru> References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> <497ed526-2fb7-805a-f72c-5909b85a62c1@yandex.ru> <91f748cd-14c1-91ca-befe-64db36789346@yandex.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sun, 3 Jul 2022 20:38:21 +0100 Konstantin Ananyev wrote: > > > > The base/existing design for DPDK was done with one particular HW architecture in mind where there was an abundance of resources. Unfortunately, that HW architecture is fast evolving and DPDK is adopted in use cases where that kind of resources are not available. For ex: efficiency cores are being introduced by every CPU vendor now. Soon enough, we will see big-little architecture in networking as well. The existing PMD design introduces 512B of stores (256B for copying to stack variable and 256B to store lcore cache) and 256B load/store on RX side every 32 packets back to back. It doesn't make sense to have that kind of memcopy for little/efficiency cores just for the driver code. > > I don't object about specific use-case optimizations. > Specially if the use-case is a common one. > But I think such changes has to be transparent to the user as > much as possible and shouldn't cause further DPDK code fragmentation > (new CONFIG options, etc.). > I understand that it is not always possible, but for pure SW based > optimizations, I think it is a reasonable expectation. Great discussion. Also, if you look back at the mailing list history, you can see that lots of users just use DPDK because it is "go fast" secret sauce and have not understanding of the internals. My concern, is that if one untestable optimization goes in for one hardware platform then users will enable it all the time thinking it makes any and all uses cases faster. Try explaining to a Linux user that the real-time kernel is *not* faster than the normal kernel... From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CDD7DA0032; Tue, 5 Jul 2022 00:06:54 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 70A0D4021D; Tue, 5 Jul 2022 00:06:54 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id 99C00400D7; Tue, 5 Jul 2022 00:06:53 +0200 (CEST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 Subject: RE: Optimizations are not features Date: Tue, 5 Jul 2022 00:06:49 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35D871A8@smartserver.smartshare.dk> In-Reply-To: <20220704093311.0582d592@hermes.local> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Optimizations are not features Thread-Index: AdiPw8WzAM6ol+p8QRqPDIm+5xsiCAAIwusw References: <98CBD80474FA8B44BF855DF32C47DC35D870EB@smartserver.smartshare.dk>

<98CBD80474FA8B44BF855DF32C47DC35D870EF@smartserver.smartshare.dk> <497ed526-2fb7-805a-f72c-5909b85a62c1@yandex.ru> <91f748cd-14c1-91ca-befe-64db36789346@yandex.ru> <20220704093311.0582d592@hermes.local> From: =?UTF-8?B?TW9ydGVuIEJyw7hydXA=?= To: "Stephen Hemminger" , "Konstantin Ananyev" Cc: "Honnappa Nagarahalli" , "Andrew Rybchenko" , "Jerin Jacob" , "dpdk-dev" , , "nd" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org PiBGcm9tOiBTdGVwaGVuIEhlbW1pbmdlciBbbWFpbHRvOnN0ZXBoZW5AbmV0d29ya3BsdW1iZXIu b3JnXQ0KPiBTZW50OiBNb25kYXksIDQgSnVseSAyMDIyIDE4LjMzDQo+IA0KPiBPbiBTdW4sIDMg SnVsIDIwMjIgMjA6Mzg6MjEgKzAxMDANCj4gS29uc3RhbnRpbiBBbmFueWV2IDxrb25zdGFudGlu LnYuYW5hbnlldkB5YW5kZXgucnU+IHdyb3RlOg0KPiANCj4gPiA+DQo+ID4gPiBUaGUgYmFzZS9l eGlzdGluZyBkZXNpZ24gZm9yIERQREsgd2FzIGRvbmUgd2l0aCBvbmUgcGFydGljdWxhciBIVw0K PiBhcmNoaXRlY3R1cmUgaW4gbWluZCB3aGVyZSB0aGVyZSB3YXMgYW4gYWJ1bmRhbmNlIG9mIHJl c291cmNlcy4NCj4gVW5mb3J0dW5hdGVseSwgdGhhdCBIVyBhcmNoaXRlY3R1cmUgaXMgZmFzdCBl dm9sdmluZyBhbmQgRFBESyBpcw0KPiBhZG9wdGVkIGluIHVzZSBjYXNlcyB3aGVyZSB0aGF0IGtp bmQgb2YgcmVzb3VyY2VzIGFyZSBub3QgYXZhaWxhYmxlLg0KPiBGb3IgZXg6IGVmZmljaWVuY3kg Y29yZXMgYXJlIGJlaW5nIGludHJvZHVjZWQgYnkgZXZlcnkgQ1BVIHZlbmRvciBub3cuDQo+IFNv b24gZW5vdWdoLCB3ZSB3aWxsIHNlZSBiaWctbGl0dGxlIGFyY2hpdGVjdHVyZSBpbiBuZXR3b3Jr aW5nIGFzIHdlbGwuDQo+IFRoZSBleGlzdGluZyBQTUQgZGVzaWduIGludHJvZHVjZXMgNTEyQiBv ZiBzdG9yZXMgKDI1NkIgZm9yIGNvcHlpbmcgdG8NCj4gc3RhY2sgdmFyaWFibGUgYW5kIDI1NkIg dG8gc3RvcmUgbGNvcmUgY2FjaGUpIGFuZCAyNTZCIGxvYWQvc3RvcmUgb24gUlgNCj4gc2lkZSBl dmVyeSAzMiBwYWNrZXRzIGJhY2sgdG8gYmFjay4gSXQgZG9lc24ndCBtYWtlIHNlbnNlIHRvIGhh dmUgdGhhdA0KPiBraW5kIG9mIG1lbWNvcHkgZm9yIGxpdHRsZS9lZmZpY2llbmN5IGNvcmVzIGp1 c3QgZm9yIHRoZSBkcml2ZXIgY29kZS4NCj4gPg0KPiA+IEkgZG9uJ3Qgb2JqZWN0IGFib3V0IHNw ZWNpZmljIHVzZS1jYXNlIG9wdGltaXphdGlvbnMuDQo+ID4gU3BlY2lhbGx5IGlmIHRoZSB1c2Ut Y2FzZSBpcyBhIGNvbW1vbiBvbmUuDQoNCk9yIGV4b3RpYywgYnV0IGhpZ2gtdm9sdW1lLCB1c2Ug Y2FzZXMhIFRob3NlIHVzdWFsbHkgZ2V0IGEgbG90IG9mIGF0dGVudGlvbiBmcm9tIHNhbGVzIGFu ZCBwcm9kdWN0IG1hbmFnZW1lbnQgcGVvcGxlLiA6LSkNCg0KRFBESyBuZWVkcyB0byBzdXBwb3J0 IHRob3NlIGluIHRoZSBtYWlubGluZSwgb3Igd2Ugd2lsbCBlbmQgdXAgd2l0aCBmb3JrcyBsaWtl IFF1YWxjb21tJ3MgUVNESyBmb3JrIG9mIHRoZSBMaW51eCBrZXJuZWwuIChUaGUgUVNESyBmb3Jr IGZyb20gUXVhbGNvbW0sIGEgbGVhZGluZyBXaS1GaSBjaGlwIHNldCB2ZW5kb3IsIGJ5cGFzc2Vz IGEgbG90IG9mIHRoZSBMaW51eCBrZXJuZWwncyBJUCBzdGFjayB0byBwcm92aWRlIG11Y2ggaGln aGVyIHRocm91Z2hwdXQgZm9yIG9uZSB1c2Ugc3BlY2lmaWMgY2FzZSwgd2hpY2ggaXMgYSBxdWl0 ZSBoaWdoIHZvbHVtZSB1c2UgY2FzZTogYSBXaS1GaSBBY2Nlc3MgUG9pbnQuKQ0KDQo+ID4gQnV0 IEkgdGhpbmsgc3VjaCBjaGFuZ2VzIGhhcyB0byBiZSB0cmFuc3BhcmVudCB0byB0aGUgdXNlciBh cw0KPiA+IG11Y2ggYXMgcG9zc2libGUgYW5kIHNob3VsZG4ndCBjYXVzZSBmdXJ0aGVyIERQREsg Y29kZSBmcmFnbWVudGF0aW9uDQo+ID4gKG5ldyBDT05GSUcgb3B0aW9ucywgZXRjLikuDQo+ID4g SSB1bmRlcnN0YW5kIHRoYXQgaXQgaXMgbm90IGFsd2F5cyBwb3NzaWJsZSwgYnV0IGZvciBwdXJl IFNXIGJhc2VkDQo+ID4gb3B0aW1pemF0aW9ucywgSSB0aGluayBpdCBpcyBhIHJlYXNvbmFibGUg ZXhwZWN0YXRpb24uDQo+IA0KPiBHcmVhdCBkaXNjdXNzaW9uLg0KPiANCj4gQWxzbywgaWYgeW91 IGxvb2sgYmFjayBhdCB0aGUgbWFpbGluZyBsaXN0IGhpc3RvcnksIHlvdSBjYW4gc2VlIHRoYXQN Cj4gbG90cyBvZiB1c2VycyBqdXN0DQo+IHVzZSBEUERLIGJlY2F1c2UgaXQgaXMgImdvIGZhc3Qi IHNlY3JldCBzYXVjZSBhbmQgaGF2ZSBub3QNCj4gdW5kZXJzdGFuZGluZyBvZiB0aGUgaW50ZXJu YWxzLg0KDQpDZXJ0YWlubHksIERQREsgc2hvdWxkIHN0aWxsIGRvIHRoYXQhDQoNCkkganVzdCB3 YW50IERQREsgdG8gYmUgYWJsZSB0byBnbyBmYXN0ZXIgZm9yIGV4cGVydHMuDQoNCkNhciBhbmFs b2d5OiBJZiB5b3UgYnV5IGEgZmFzdCBjYXIsIGl0IHdpbGwgZ28gZmFzdC4gSWYgeW91IGJyaW5n IGl0IHRvIGEgdHVuaW5nIHNwZWNpYWxpc3QsIGl0IHdpbGwgZ28gZmFzdGVyLiBTaW1pbGFybHks IERQREsgc2hvdWxkIGdvICJmYXN0IiwgYnV0IGFsc28gYWNjZXB0IHRoYXQgc3BlY2lhbGlzdHMg Y2FuIG1ha2UgaXQgZ28gImZhc3RlciIuDQoNCj4gDQo+IE15IGNvbmNlcm4sIGlzIHRoYXQgaWYg b25lIHVudGVzdGFibGUgb3B0aW1pemF0aW9uIGdvZXMgaW4gZm9yIG9uZQ0KPiBoYXJkd2FyZSBw bGF0Zm9ybSB0aGVuDQo+IHVzZXJzIHdpbGwgZW5hYmxlIGl0IGFsbCB0aGUgdGltZSB0aGlua2lu ZyBpdCBtYWtlcyBhbnkgYW5kIGFsbCB1c2VzDQo+IGNhc2VzIGZhc3Rlci4NCj4gVHJ5IGV4cGxh aW5pbmcgdG8gYSBMaW51eCB1c2VyIHRoYXQgdGhlIHJlYWwtdGltZSBrZXJuZWwgaXMgKm5vdCoN Cj4gZmFzdGVyIHRoYW4NCj4gdGhlIG5vcm1hbCBrZXJuZWwuLi4NCg0KWWVzLCBiZWNhdXNlIG9m IHRoZSBjb21tb24gbWlzY29uY2VwdGlvbiB0aGF0IGZhc3RlciBlcXVhbHMgdG8gaGlnaGVyIGJh bmR3aWR0aC4gQnV0IHRoZSByZWFsLXRpbWUga2VybmVsIGRvZXMgcHJvdmlkZSBsb3dlciBsYXRl bmN5ICh1bmRlciBjZXJ0YWluIGNvbmRpdGlvbnMpLCB3aGljaCBtZWFucyBmYXN0ZXIgdG8gc29t ZSBvZiB1cy4gSSdtIHNvcnJ5Li4uIHdvcmtpbmcgd2l0aCBsYXRlbmN5IGFzIG9uZSBvZiBvdXIg S1BJcywgSSBqdXN0IGNvdWxkbid0IHJlc2lzdCBpdCEgOy0pDQoNClNlcmlvdXNseSwgRFBESyBj YW5ub3QgYmUgbGltaXRlZCB0byBjYXRlciB0byBldmVyeW9uZSBvbiBTdGFjayBPdmVyZmxvdyEN Cg0KSm9rZXMgYXNpZGUuLi4NCg0KV2hlbiB3ZSBzdGFydGVkIHVzaW5nIERQREsgYXQgU21hcnRT aGFyZSBTeXN0ZW1zLCBEUERLIHdhcyBhIGhpZ2hseSBvcHRpbWl6ZWQgZGV2ZWxvcG1lbnQga2l0 IGZvciBlbWJlZGRlZCBuZXR3b3JrIGFwcGxpYW5jZXMsIHBlcmZlY3QgZm9yIG91ciBTbWFydFNo YXJlIFN0cmFpZ2h0U2hhcGVyIFdBTiBvcHRpbWl6YXRpb24gYXBwbGlhbmNlcyBhbmQgZnV0dXJl IHJvYWRtYXAuIE92ZXIgdGltZSwgRFBESyBoYXMgbW9ycGhlZCBpbnRvIGEgcGFja2V0IHByb2Nl c3NpbmcgbGlicmFyeSBmb3IgVWJ1bnR1IGFuZCBSZWQgSGF0LCB3aXRoIGEgbG90IG9mIGFkZGVk IGZlYXR1cmVzIHdlIGRvbid0IHVzZSwgYW5kIG5vIGFiaWxpdHkgdG8gcmVtb3ZlIHRob3NlIGFk ZGVkIGZlYXR1cmVzLiBUaG9zZSBhZGRlZCBmZWF0dXJlcyBwb3RlbnRpYWxseSBkZWdyYWRlIHRo ZSBmYXN0IHBhdGggcGVyZm9ybWFuY2UsIGFuZCBpbmNyZWFzZSB0aGUgcmlzayBvZiBidWdzIGF0 IHN5c3RlbSBsZXZlbC4NCg0KU29tZSBzb2Z0d2FyZSBvcHRpbWl6YXRpb25zIGhhdmUgYmVlbiBw cm9wb3NlZCB0byBEUERLLCB0byBzdXBwb3J0IHNvbWUgc3BlY2lmaWMgaGlnaC12b2x1bWUgdXNl IGNhc2VzLiAibWJ1ZiBmYXN0IGZyZWUiIGdvdCBhY2NlcHRlZCwgYnV0ICJkaXJlY3QgcmUtYXJt IiBpcyBnZXR0aW5nIGEgbG90IG9mIHB1c2gtYmFjaywgYW5kIHRoZSBtb3N0IHJlY2VudCAiSU9W QSBWQSBvbmx5IG1vZGUiIGlzIGFub3RoZXIgbmV3IG9wdGltaXphdGlvbiBzdWdnZXN0aW9uIGJl aW5nIGRpc2N1c3NlZC4NCg0KSW4gdGhlb3J5LCBpdCB3b3VsZCBiZSBuaWNlIGlmIGFsbCBzb2Z0 d2FyZSBvcHRpbWl6YXRpb25zIGNvdWxkIGJlIHN1cHBvcnRlZCBhdCBydW4tdGltZSwgYnV0IGl0 IGFkZHMgYXQgbGVhc3Qgb25lIGJyYW5jaCB0byB0aGUgZmFzdCBwYXRoIGZvciBldmVyeSBvcHRp bWl6YXRpb24sIGV2ZW50dWFsbHkgc2xvd2luZyBkb3duIHRoZSBmYXN0IHBhdGggc2lnbmlmaWNh bnRseS4gQW5kIHNvbWUgb2YgdGhlIG9wdGltaXphdGlvbnMganVzdCBtYWtlIHNvIG11Y2ggYmV0 dGVyIHNlbnNlIGF0IGNvbXBpbGUgdGltZSB0aGFuIGF0IHJ1bnRpbWUsIGUuZy4gdGhlICJJT1ZB IFZBIG1vZGUiLg0KDQpTbywgSSB0aGluayB3ZSBzaG91bGQgc3RhcnQgdGhpbmtpbmcgYWJvdXQg c3VjaCBvcHRpbWl6YXRpb25zIGRpZmZlcmVudGx5OiBJZiBzb21lb25lIG5lZWRzIHRvIG9wdGlt aXplIHNvbWV0aGluZyBmb3IgYSBzcGVjaWZpYyB1c2UgY2FzZSwgaXQgY2FuIGJlIGRvbmUgYXQg Y29tcGlsZSB0aW1lOyB0aGVyZSBpcyBubyBuZWVkIHRvIGRvIGl0IGF0IHJ1bnRpbWUuIFdoaWNo IGlzIHdoYXQgSSBtZWFudCBieSB0aGUgc3ViamVjdCBvZiBteSBlbWFpbDogRG9uJ3Qgb2ZmZXIg b3B0aW1pemF0aW9ucyBhcyBydW50aW1lIGZlYXR1cmVzOyB0aGV5IGFyZSB1c2UgY2FzZSBzcGVj aWZpYywgYW5kIHNob3VsZCBiZSBjaG9zZW4gYXQgY29tcGlsZSB0aW1lIG9ubHkuDQoNClJlZmVy cmluZyB0byB0aGUgTGludXgga2VybmVsIGFzIHRoZSBnb2xkZW4gc3RhbmRhcmQsIGl0IGV2ZW4g aGFzICJtYWtlIG1lbnVjb25maWciLi4uIGEgbWVudSBkcml2ZW4gY29uZmlndXJhdGlvbiBpbnRl cmZhY2UgZm9yIGNvbXBpbGUgdGltZSBjb25maWd1cmF0aW9uLiBXaHkgbXVzdCBEUERLIGhhdmUg ZXZlcnkgZXhvdGljIG9wdGlvbiBhdmFpbGFibGUgYXQgcnVudGltZSwgd2hlbiB0aGUgTGludXgg a2VybmVsIGNvbnNpZGVycyBpdCBwZXJmZWN0bHkgYWNjZXB0YWJsZSB0byBoYXZlIHNvbWUgdGhp bmdzIGNvbmZpZ3VyYWJsZSBhdCBjb21waWxlIHRpbWUgb25seT8NCg0KV2l0aCB0aGlzIGRpc2N1 c3Npb24sIEkgYW0gb25seSBhc2tpbmcgZm9yIHNvZnR3YXJlIG9wdGltaXphdGlvbnMgKHdoaWNo IHVzdWFsbHkgYWxzbyBpbXBseSBzb21lIG90aGVyIGxpbWl0YXRpb25zKSB0byBiZSBjb21waWxl IHRpbWUgb3B0aW9ucywgcmF0aGVyIHRoYW4gY29tcGlsZSB0aW1lIG9wdGlvbnMuIEFueSBhcHBs aWNhdGlvbiBjYW4gYWNoaWV2ZSBleGFjdGx5IHRoZSBzYW1lIHdpdGhvdXQgdGhvc2Ugb3B0aW1p emF0aW9ucyBlbmFibGVkLCBidXQgaXQgd2lsbCBiZSBmYXN0ZXIgd2l0aCB0aGUgb3B0aW1pemF0 aW9uIGVuYWJsZWQuDQoNCkkgd291bGQgbG92ZSB0byBnbyBiYWNrIHRvIHRoZSBnb29kIG9sZCBk YXlzLCB3aGVyZSBEUERLIGhhZCBhIGxvdCBvZiBjb21waWxlIHRpbWUgb3B0aW9ucyB0byBkaXNh YmxlIGNydWZ0IHdlJ3JlIG5vdCB1c2luZywgYnV0IEkga25vdyB0aGF0IGdhbWUgd2FzIGxvc3Qg YSBsb25nIHRpbWUgYWdvISBTbyBJJ20gdHJ5aW5nIHRvIGZpbmQgc29tZSBtaWRkbGUgZ3JvdW5k IHRoYXQga2VlcHMgYWxsIGZlYXR1cmVzIGluIHRoZSAiRFBESyBsaWJyYXJ5IGZvciBkaXN0cm9z IiwgYnV0IGFsc28gYWxsb3dzIGhhcmQgY29yZSBkZXZlbG9wZXJzIHRvIHR1bmUgdGhlIHBlcmZv cm1hbmNlIGZvciB0aGVpciBpbmRpdmlkdWFsIHVzZSBjYXNlcy4NCg0KT2ZmZXJpbmcgc29mdHdh cmUgb3B0aW1pemF0aW9ucyBhcyBjb21waWxlIHRpbWUgb3B0aW9ucyBvbmx5LCBzaG91bGQgYWxz byByZWR1Y2UgdGhlIGFtb3VudCBvZiBwdXNoLWJhY2sgZm9yIHN1Y2ggc29mdHdhcmUgb3B0aW1p emF0aW9ucy4NCg0KUmVhZGluZyBhbGwgdGhlIGZlZWRiYWNrIGZyb20gdGhlIHRocmVhZCwgaXQg c2VlbXMgdGhhdCB0aGUgbWFqb3IgY29uY2VybiBpcyB0ZXN0aW5nLiBBbmQgZm9yIHNvbWUgbXlz dGVyaW91cyByZWFzb24sIGNvbXBpbGluZyAyXk4gZmVhdHVyZXMgY2F1c2VzIG1vcmUgY29uY2Vy biB0aGFuIHJ1bi10aW1lIHRlc3RpbmcgMl5OIGZlYXR1cmVzLiBJIGdldCB0aGUgc2Vuc2UgdGhh dCBydW4tdGltZSB0ZXN0aW5nIHRoZSB2YXJpb3VzIGZlYXR1cmUgY29tYmluYXRpb25zIGlzIG5v dCBoYXBwZW5pbmcgdG9kYXkuIDotKA0KDQo=