From: Ivan Malov <ivan.malov@arknetworks.am>
To: Ferruh Yigit <ferruh.yigit@amd.com>
Cc: Jerin Jacob <jerinjacobk@gmail.com>,
Thomas Monjalon <thomas@monjalon.net>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
Ori Kam <orika@nvidia.com>,
Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>,
Aman Singh <aman.deep.singh@intel.com>,
Yuying Zhang <yuying.zhang@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
Hanumanth Reddy Pothula <hpothula@marvell.com>,
Slava Ovsiienko <viacheslavo@nvidia.com>,
Jerin Jacob Kollanukkaran <jerinj@marvell.com>,
"david.marchand@redhat.com" <david.marchand@redhat.com>
Subject: Re: [EXT] Re: [PATCH v5 2/2] app/testpmd: add command to process Rx metadata negotiation
Date: Thu, 2 Feb 2023 14:48:44 +0400 (+04) [thread overview]
Message-ID: <7157f5a0-313e-f937-36b3-13ee50f794be@arknetworks.am> (raw)
In-Reply-To: <da48595-f8ba-dcb8-9ce4-a66ee87274c@arknetworks.am>
I apologise, there was a typo in the previous mail: "PMD does
not need this API". Should read as "TESTPMD does not need it".
Thank you.
On Thu, 2 Feb 2023, Ivan Malov wrote:
> On Thu, 2 Feb 2023, Ferruh Yigit wrote:
>
>> On 2/2/2023 8:50 AM, Ivan Malov wrote:
>>> On Thu, 2 Feb 2023, Ferruh Yigit wrote:
>>>
>>>> On 2/1/2023 3:22 PM, Jerin Jacob wrote:
>>>>> On Wed, Feb 1, 2023 at 8:20 PM Ferruh Yigit <ferruh.yigit@amd.com>
>>>>> wrote:
>>>>>>
>>>>>> On 2/1/2023 1:48 PM, Jerin Jacob wrote:
>>>>>>> On Wed, Feb 1, 2023 at 5:06 PM Ferruh Yigit <ferruh.yigit@amd.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 2/1/2023 11:15 AM, Jerin Jacob wrote:
>>>>>>>>> On Wed, Feb 1, 2023 at 4:35 PM Thomas Monjalon
>>>>>>>>> <thomas@monjalon.net> wrote:
>>>>>>>>>>
>>>>>>>>>> 01/02/2023 11:58, Andrew Rybchenko:
>>>>>>>>>>> On 2/1/23 13:48, Jerin Jacob wrote:
>>>>>>>>>>>> On Wed, Feb 1, 2023 at 2:59 PM Andrew Rybchenko
>>>>>>>>>>>> <andrew.rybchenko@oktetlabs.ru> wrote:
>>>>>>>>>>>>> Frankly speaking I don't understand why default value is so
>>>>>>>>>>>>> important if we have a way to change it. Reasons should be
>>>>>>>>>>>>> really strong to change existing defaults.
>>>>>>>>>>>>
>>>>>>>>>>>> The only reason is, typically testpmd will be used performance
>>>>>>>>>>>> benchmarking as an industry standard. It is difficult to
>>>>>>>>>>>> tell/educate
>>>>>>>>>>>> the QA or customers
>>>>>>>>>>>> that, "BTW if you need to get better performance add more flag to
>>>>>>>>>>>> testpmd command line".
>>>>>>>>>>
>>>>>>>>>> I disagree.
>>>>>>>>>> When you do performance benchmark, you tune settings accordingly.
>>>>>>>>>
>>>>>>>>> IMO, We tune the system resources like queue depth not the disabling
>>>>>>>>> features for raw performance.
>>>>>>>>> queue depth etc people know to tune so it is obvious. What is not
>>>>>>>>> obvious is, testpmd only
>>>>>>>>> negotiated some features by default.I am not using that feature,
>>>>>>>>> hence
>>>>>>>>> I need to explicitly
>>>>>>>>> disable it.
>>>>>>>>>
>>>>>>>>
>>>>>>>> When 'rte_eth_rx_metadata_negotiate()' API is NOT used at all, and I
>>>>>>>> believe that is the case for almost all applications since API is a
>>>>>>>> relatively new one, PMD default behavior should be to enable Rx
>>>>>>>> metadata
>>>>>>>> flow rules, in case user requests them later.
>>>>>>>>
>>>>>>>> So, enabling all in application is same with not calling the API
>>>>>>>> at all.
>>>>>>>>
>>>>>>>> In this perspective, disabling Rx metadata is additional
>>>>>>>> optimization/tuning that application can do if it is sure that Rx
>>>>>>>> metadata flow rules won't be used at all.
>>>>>>>> And API is more meaningful when it is used to disable Rx metadata.
>>>>>>>>
>>>>>>>> I think it is reasonable to enable all Rx metadata by default in
>>>>>>>> testpmd
>>>>>>>> with a capability to disable it when wanted.
>>>>>>>>
>>>>>>>> OR
>>>>>>>>
>>>>>>>> May be we don't call 'rte_eth_rx_metadata_negotiate()' API by
>>>>>>>> default in
>>>>>>>> testpmd, it is only called when it is requested explicitly from user,
>>>>>>>> enable or disable.
>>>>>>>
>>>>>>> Second option looks good to me.
>>>>>>> When
>>>>>>> 1) user request for action which is needed negotiate(),
>>>>>>> AND
>>>>>>> 2) rte_eth_rx_metadata_negotiate() != ENOSUP
>>>>>>> then, testpmd print a warning that need to enable
>>>>>>> rte_eth_rx_metadata_negotiate().
>>>>>>>
>>>>>>
>>>>>> We are not suggesting same thing.
>>>>>>
>>>>>> What you described above assumes PMD disabled Rx metadata flow rule
>>>>>> support by default, and it needs to be enabled explicitly by
>>>>>> 'rte_eth_rx_metadata_negotiate()' API. This API becomes mandatory for
>>>>>> functionality.
>>>>>>
>>>>>> As far as I understand PMD wants to disable this flow rule by default
>>>>>> because of performance concerns. But this creates inconsistency between
>>>>>> PMDs, because rest of them will enable this flow rule by default (if it
>>>>>> is supported) and be ready to use it when proper flow rule created.
>>>>>>
>>>>>> With this approach some PMDs will need
>>>>>> 'rte_eth_rx_metadata_negotiate()'
>>>>>> to enable Rx metadata flow rules, some won't. This can be confusing for
>>>>>> applications that *some* PMDs require double enabling with specific API
>>>>>> call.
>>>>>>
>>>>>>
>>>>>> Instead what I was trying to suggest is reverse,
>>>>>> all PMDs enable the Rx metadata flow rule by default, and don't require
>>>>>> double enabling.
>>>>>> But if application knows that it won't use Rx metadata flow rule, it
>>>>>> can
>>>>>> disable it to optimize the performance.
>>>>>> This makes 'rte_eth_rx_metadata_negotiate()' functionally optional, and
>>>>>> for testpmd context it can be called via a command on demand by user
>>>>>> for
>>>>>> optimization purpose.
>>>>>
>>>>> This won't solve concern I have outlined earlier[1].
>>>>>
>>>>
>>>> Yes, it won't.
>>>>
>>>>> I think, The part of the problem there is no enough adaption of
>>>>> rte_eth_rx_metadata_negotiate(),
>>>>>
>>>>> The view is total different from PMD maintainer PoV vs testpmd
>>>>> application PoV.
>>>>>
>>>>
>>>> Agree,
>>>> and I assume it is different for user application too, which may
>>>> prioritize consistency and portability.
>>>>
>>>> Overall, I am not fan of the 'rte_eth_rx_metadata_negotiate()' API, I
>>>> think it is confusing.
>>>
>>> Forgive me, in which way is it confusing?
>>>
>>
>> All other flow rules enabled by creating flow rule, but this one
>
> Hold on.. Did you just say "flow rules"? But this API is *not* about
> flow rules. I suggest you please re-read description of the API.
> It reads: "Negotiate the NIC's ability to deliver specific
> kinds of metadata to the PMD". Nothing about flows there.
>
> Why is it decoupled from flow library this way? Because there is
> a drastic difference between generating and plumbing metadata
> INSIDE the NIC's flow engine, on the one hand, and delivering
> these data from the NIC to the host driver, on the other.
>
> Let me explain. Say, one creates the following flow rules:
>
> (a) flow create 0 transfer group 0 pattern [A] / end \
> actions mark id XXX jump group 1 / end
> (b) flow create 0 transfer group 0 pattern [B] / end \
> actions mark id YYY jump group 1 / end
>
> (c) flow create 0 transfer group 1 pattern mark id is XXX / end \
> actions represented_port ethdev_id 1 / end
> (d) flow create 0 transfer group 1 pattern mark id is YYY / end \
> actions drop / end
>
> In this example, metadata of type "MARK" is used to partition
> flow group (table) 1 into multiple lookup sections. So the
> mark value is GENERATED by the flow engine and then it is
> CONSUMED by this very flow engine. The application may
> NOT necessarily want to receive the mark with mbufs...
>
> And it is only when the application wants these metadata
> DELIVERED to it that it has to call the negotiate API.
>
> The short of it, nothing prevents the driver from accepting
> flow create requests that leverage some meta items/actions.
> Drivers do not need the negotiate API to *configure flow*.
> They only need this API in order to let the application
> choose whether metadata will be DELIVERED (!) or not.
>
>> requires an API to enable it, so I believe it is inconsistent in that way.
>
> Please see above. Everything is consistent as *flow library*
> and *negotiate metadata delivery* API are totally decoupled.
>
>>
>> From application perspective, assume that it doesn't know NIC details,
>> should it call this API or not? Without API call should application
>> assume Rx metadata flow rules are enabled or disabled?
>
> Frankly, applications like testpmd need never call this API.
> Simply because seeing, for example, MARK values in mbufs is
> useless to it. This API is needed by other applications.
> For example, OvS has partial MARK+RSS offload. It adds
> flows that distribute packets across multiple queues
> and set some MARK values for them. OvS is interested
> in getting this values with mbufs since they affect
> further lookups in software... So, OvS, knowing it
> wants these metadata DELIVERED (!), should invoke
> this metadata negotiate API to ensure that.
>
>>
>>
>> As I understand intention is to get hint from application if it will
>> require Rx metadata flow rules so that PMD can optimize better, but if
>
> No, nothing about flow rules. Just delivery of metadata with mbufs.
>
>> PMD doesn't enable Rx metadata flow rules when this API call is missing
>
> Again, PMD shall not make decisions on whether to enable or disable
> support for some FLOW primitives based on interactions via this API.
> This API exists solely to let PMD configure delivery of metadata,
> i.e. not the way it is generated in the first instance.
>
>> than it becomes a mandatory API to configure the device. But I think it
>> should be optional for optimization.
>>
>> Also if application not sure to use this flow rule or not, it will by
>> default enable all in any case, this will reduce the benefit. As done in
>> testpmd.
>
> Please see above. PMD does not need this API, I take it.
>
>>
>>
>> API works in both ways, it request to enable some Rx metadata flow rule,
>> but based on what PMD returns application can know what device supports,
>> this also inconsistent with how other flow rules work, we don't have API
>> to get capability but detect them via flow create/validate.
>> Can there be a case API returns a flow rule negotiated, but it fails
>> when tried to create the flow rule, isn't this confusing for application
>>
>>
>> I think if we continue this approach there can be multiple enable and
>> capability learning APIs for various flow rules or flow rule groups, and
>> this can make flow API much more harder to use for applications.
>
> See my explanations above. This API is not about flows. Period.
>
>>
>>
>>
>>>>
>>>>> Just to avoid back and forth. We will call off this patch and remove
>>>>> rte_eth_rx_metadata_negotiate()
>>>>> PMD callback from cnxk driver. Keep it as old behavior, so we don't
>>>>> need to care
>>>>> about rte_eth_rx_metadata_negotiate().
>>>>>
>>>>
>>>> When you remove 'rx_metadata_negotiate' callback, what will be the PMD
>>>> behavior? I assume PMD will do the required preparations as if all Rx
>>>> metadata is enabled.
>>>> And what is the performance impact, is removing callback improve the
>>>> performance?
>>>>
>>>>
>>>>> [1]
>>>>> The only reason is, typically testpmd will be used performance
>>>>> benchmarking as an industry standard. It is difficult to tell/educate
>>>>> the QA or customers
>>>>> that, "BTW if you need to get better performance add more flag to
>>>>> testpmd command line".
>>>>> To make that worst, only some PMD needs to give the additional
>>>>> parameter to get better number.
>>>>> And also, testpmd usage will be treated as application modeling.
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> To make that worst, only some PMD needs to give the additional
>>>>>>>>>>>> parameter to get better number.
>>>>>>>>>>>> And also, testpmd usage will be treated as application modeling.
>>>>>>>>>>>>
>>>>>>>>>>>> Since this feature only used on sfc and cnxk driver, What is the
>>>>>>>>>>>> situation with sfc driver?
>>>>>>>>>>>> Keeping it as negotiated and not use the feature, will impact
>>>>>>>>>>>> the per
>>>>>>>>>>>> core performance of sfc or
>>>>>>>>>>>> is it just PCI bandwidth thing which really dont show any
>>>>>>>>>>>> difference in testpmd?
>>>>>>>>>>>
>>>>>>>>>>> Yes, sfc could run faster if no Rx metadata are negotiated. So,
>>>>>>>>>>> it is better to negotiate nothing by default. But it is always
>>>>>>>>>>> painful to change defaults. You need to explain that now you
>>>>>>>>>>> need to negotiate Rx metadata to use mark, flag and tunnel
>>>>>>>>>>> offloads.
>>>>>>>>>>> Yes, it will be required on sfc and cnxk only.
>>>>>>>>>>> As an sfc maintainer I don't mind to change testpmd defaults.
>>>>>>>>>>
>>>>>>>>>> If we change testpmd defaults to "do nothing",
>>>>>>>>>> then we should disable MBUF_FAST_FREE as well.
>>>>>>>>>
>>>>>>>>> if you see MBUF_FAST_FREE, it does nothing. Actually,
>>>>>>>>> !MBUF_FAST_FREE is doing more work.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
>
next prev parent reply other threads:[~2023-02-02 10:48 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-01 4:41 [PATCH] app/testpmd: add command line argument 'rx-metadata' Hanumanth Pothula
2022-08-01 13:11 ` Hanumanth Pothula
2022-08-01 13:13 ` Hanumanth Pothula
2022-08-01 19:41 ` Ivan Malov
2022-08-02 16:45 ` [PATCH] app/testpmd: add command line argument 'nic-to-pmd-rx-metadata' Hanumanth Pothula
2022-08-02 16:45 ` [PATCH v2 1/2] version: 22.11-rc0 Hanumanth Pothula
2022-08-02 16:45 ` [PATCH v2 2/2] app/testpmd: add command line argument 'nic-to-pmd-rx-metadata' Hanumanth Pothula
2022-08-02 17:51 ` [PATCH v2 1/1] " Hanumanth Pothula
2022-08-30 12:36 ` Hanumanth Reddy Pothula
2022-09-01 8:03 ` Singh, Aman Deep
2022-10-04 14:48 ` Andrew Rybchenko
2022-10-06 18:35 ` [PATCH v3 1/1] app/testpmd: control passing Rx metadata to PMD Hanumanth Pothula
2022-10-17 8:32 ` Andrew Rybchenko
2022-10-27 7:34 ` Thomas Monjalon
2022-10-27 12:54 ` Thomas Monjalon
2022-12-02 16:14 ` [EXT] " Hanumanth Reddy Pothula
2022-12-02 19:41 ` Thomas Monjalon
2022-12-05 7:59 ` Hanumanth Reddy Pothula
2022-12-05 8:28 ` Thomas Monjalon
2022-12-05 9:43 ` Slava Ovsiienko
2022-12-20 20:02 ` [PATCH v4 1/2] ethdev: control Rx metadata negotiation Hanumanth Pothula
2022-12-20 20:02 ` [PATCH v4 2/2] app/testpmd: add command to process " Hanumanth Pothula
2022-12-20 21:23 ` Stephen Hemminger
2022-12-21 2:07 ` [PATCH v5 1/2] ethdev: fix ethdev configuration state on reset Hanumanth Pothula
2022-12-21 2:07 ` [PATCH v5 2/2] app/testpmd: add command to process Rx metadata negotiation Hanumanth Pothula
2023-01-18 10:32 ` Thomas Monjalon
2023-01-19 10:33 ` [EXT] " Hanumanth Reddy Pothula
2023-01-25 12:51 ` Thomas Monjalon
2023-01-24 18:04 ` Ferruh Yigit
2023-01-25 9:30 ` [EXT] " Hanumanth Reddy Pothula
2023-01-25 12:55 ` Thomas Monjalon
2023-01-25 13:55 ` Ferruh Yigit
2023-01-25 13:59 ` Thomas Monjalon
2023-01-25 14:42 ` Nithin Kumar Dabilpuram
2023-01-26 11:03 ` Thomas Monjalon
2023-01-27 5:02 ` Nithin Kumar Dabilpuram
2023-01-27 8:54 ` Thomas Monjalon
2023-01-27 10:42 ` Nithin Kumar Dabilpuram
2023-01-27 15:01 ` Thomas Monjalon
2023-01-31 16:17 ` Jerin Jacob
2023-01-31 23:03 ` Thomas Monjalon
2023-02-01 6:10 ` Ivan Malov
2023-02-01 7:16 ` Andrew Rybchenko
2023-02-01 8:53 ` Jerin Jacob
2023-02-01 9:00 ` Ori Kam
2023-02-01 9:05 ` Thomas Monjalon
2023-02-01 9:07 ` Andrew Rybchenko
2023-02-01 9:14 ` Jerin Jacob
2023-02-01 9:29 ` Andrew Rybchenko
2023-02-01 10:48 ` Jerin Jacob
2023-02-01 10:58 ` Andrew Rybchenko
2023-02-01 11:04 ` Thomas Monjalon
2023-02-01 11:15 ` Jerin Jacob
2023-02-01 11:35 ` Ferruh Yigit
2023-02-01 13:48 ` Jerin Jacob
2023-02-01 14:50 ` Ferruh Yigit
2023-02-01 15:22 ` Jerin Jacob
2023-02-02 8:43 ` Ferruh Yigit
2023-02-02 8:50 ` Ivan Malov
2023-02-02 9:17 ` Ferruh Yigit
2023-02-02 10:41 ` Ivan Malov
2023-02-02 10:48 ` Ivan Malov [this message]
2023-02-02 11:41 ` Thomas Monjalon
2023-02-02 11:55 ` Ivan Malov
2023-02-02 12:03 ` Thomas Monjalon
2023-02-02 12:21 ` Andrew Rybchenko
2023-02-01 11:20 ` Ivan Malov
2023-01-25 13:17 ` Ferruh Yigit
2023-01-25 13:21 ` Ferruh Yigit
2023-01-25 13:21 ` Ferruh Yigit
2023-01-16 10:43 ` [PATCH v5 1/2] ethdev: fix ethdev configuration state on reset Hanumanth Reddy Pothula
2023-01-18 10:29 ` Thomas Monjalon
2023-01-24 18:14 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7157f5a0-313e-f937-36b3-13ee50f794be@arknetworks.am \
--to=ivan.malov@arknetworks.am \
--cc=aman.deep.singh@intel.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=hpothula@marvell.com \
--cc=jerinj@marvell.com \
--cc=jerinjacobk@gmail.com \
--cc=ndabilpuram@marvell.com \
--cc=orika@nvidia.com \
--cc=thomas@monjalon.net \
--cc=viacheslavo@nvidia.com \
--cc=yuying.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).