From: Ivan Malov <ivan.malov@arknetworks.am>
To: Ferruh Yigit <ferruh.yigit@amd.com>
Cc: Jerin Jacob <jerinjacobk@gmail.com>,
Thomas Monjalon <thomas@monjalon.net>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
Ori Kam <orika@nvidia.com>,
Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>,
Aman Singh <aman.deep.singh@intel.com>,
Yuying Zhang <yuying.zhang@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
Hanumanth Reddy Pothula <hpothula@marvell.com>,
Slava Ovsiienko <viacheslavo@nvidia.com>,
Jerin Jacob Kollanukkaran <jerinj@marvell.com>,
"david.marchand@redhat.com" <david.marchand@redhat.com>
Subject: Re: [EXT] Re: [PATCH v5 2/2] app/testpmd: add command to process Rx metadata negotiation
Date: Thu, 2 Feb 2023 14:41:20 +0400 (+04) [thread overview]
Message-ID: <da48595-f8ba-dcb8-9ce4-a66ee87274c@arknetworks.am> (raw)
In-Reply-To: <a95161fe-e468-8c05-ef02-2bda4d7269a2@amd.com>
On Thu, 2 Feb 2023, Ferruh Yigit wrote:
> On 2/2/2023 8:50 AM, Ivan Malov wrote:
>> On Thu, 2 Feb 2023, Ferruh Yigit wrote:
>>
>>> On 2/1/2023 3:22 PM, Jerin Jacob wrote:
>>>> On Wed, Feb 1, 2023 at 8:20 PM Ferruh Yigit <ferruh.yigit@amd.com>
>>>> wrote:
>>>>>
>>>>> On 2/1/2023 1:48 PM, Jerin Jacob wrote:
>>>>>> On Wed, Feb 1, 2023 at 5:06 PM Ferruh Yigit <ferruh.yigit@amd.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> On 2/1/2023 11:15 AM, Jerin Jacob wrote:
>>>>>>>> On Wed, Feb 1, 2023 at 4:35 PM Thomas Monjalon
>>>>>>>> <thomas@monjalon.net> wrote:
>>>>>>>>>
>>>>>>>>> 01/02/2023 11:58, Andrew Rybchenko:
>>>>>>>>>> On 2/1/23 13:48, Jerin Jacob wrote:
>>>>>>>>>>> On Wed, Feb 1, 2023 at 2:59 PM Andrew Rybchenko
>>>>>>>>>>> <andrew.rybchenko@oktetlabs.ru> wrote:
>>>>>>>>>>>> Frankly speaking I don't understand why default value is so
>>>>>>>>>>>> important if we have a way to change it. Reasons should be
>>>>>>>>>>>> really strong to change existing defaults.
>>>>>>>>>>>
>>>>>>>>>>> The only reason is, typically testpmd will be used performance
>>>>>>>>>>> benchmarking as an industry standard. It is difficult to
>>>>>>>>>>> tell/educate
>>>>>>>>>>> the QA or customers
>>>>>>>>>>> that, "BTW if you need to get better performance add more flag to
>>>>>>>>>>> testpmd command line".
>>>>>>>>>
>>>>>>>>> I disagree.
>>>>>>>>> When you do performance benchmark, you tune settings accordingly.
>>>>>>>>
>>>>>>>> IMO, We tune the system resources like queue depth not the disabling
>>>>>>>> features for raw performance.
>>>>>>>> queue depth etc people know to tune so it is obvious. What is not
>>>>>>>> obvious is, testpmd only
>>>>>>>> negotiated some features by default.I am not using that feature,
>>>>>>>> hence
>>>>>>>> I need to explicitly
>>>>>>>> disable it.
>>>>>>>>
>>>>>>>
>>>>>>> When 'rte_eth_rx_metadata_negotiate()' API is NOT used at all, and I
>>>>>>> believe that is the case for almost all applications since API is a
>>>>>>> relatively new one, PMD default behavior should be to enable Rx
>>>>>>> metadata
>>>>>>> flow rules, in case user requests them later.
>>>>>>>
>>>>>>> So, enabling all in application is same with not calling the API
>>>>>>> at all.
>>>>>>>
>>>>>>> In this perspective, disabling Rx metadata is additional
>>>>>>> optimization/tuning that application can do if it is sure that Rx
>>>>>>> metadata flow rules won't be used at all.
>>>>>>> And API is more meaningful when it is used to disable Rx metadata.
>>>>>>>
>>>>>>> I think it is reasonable to enable all Rx metadata by default in
>>>>>>> testpmd
>>>>>>> with a capability to disable it when wanted.
>>>>>>>
>>>>>>> OR
>>>>>>>
>>>>>>> May be we don't call 'rte_eth_rx_metadata_negotiate()' API by
>>>>>>> default in
>>>>>>> testpmd, it is only called when it is requested explicitly from user,
>>>>>>> enable or disable.
>>>>>>
>>>>>> Second option looks good to me.
>>>>>> When
>>>>>> 1) user request for action which is needed negotiate(),
>>>>>> AND
>>>>>> 2) rte_eth_rx_metadata_negotiate() != ENOSUP
>>>>>> then, testpmd print a warning that need to enable
>>>>>> rte_eth_rx_metadata_negotiate().
>>>>>>
>>>>>
>>>>> We are not suggesting same thing.
>>>>>
>>>>> What you described above assumes PMD disabled Rx metadata flow rule
>>>>> support by default, and it needs to be enabled explicitly by
>>>>> 'rte_eth_rx_metadata_negotiate()' API. This API becomes mandatory for
>>>>> functionality.
>>>>>
>>>>> As far as I understand PMD wants to disable this flow rule by default
>>>>> because of performance concerns. But this creates inconsistency between
>>>>> PMDs, because rest of them will enable this flow rule by default (if it
>>>>> is supported) and be ready to use it when proper flow rule created.
>>>>>
>>>>> With this approach some PMDs will need
>>>>> 'rte_eth_rx_metadata_negotiate()'
>>>>> to enable Rx metadata flow rules, some won't. This can be confusing for
>>>>> applications that *some* PMDs require double enabling with specific API
>>>>> call.
>>>>>
>>>>>
>>>>> Instead what I was trying to suggest is reverse,
>>>>> all PMDs enable the Rx metadata flow rule by default, and don't require
>>>>> double enabling.
>>>>> But if application knows that it won't use Rx metadata flow rule, it
>>>>> can
>>>>> disable it to optimize the performance.
>>>>> This makes 'rte_eth_rx_metadata_negotiate()' functionally optional, and
>>>>> for testpmd context it can be called via a command on demand by user
>>>>> for
>>>>> optimization purpose.
>>>>
>>>> This won't solve concern I have outlined earlier[1].
>>>>
>>>
>>> Yes, it won't.
>>>
>>>> I think, The part of the problem there is no enough adaption of
>>>> rte_eth_rx_metadata_negotiate(),
>>>>
>>>> The view is total different from PMD maintainer PoV vs testpmd
>>>> application PoV.
>>>>
>>>
>>> Agree,
>>> and I assume it is different for user application too, which may
>>> prioritize consistency and portability.
>>>
>>> Overall, I am not fan of the 'rte_eth_rx_metadata_negotiate()' API, I
>>> think it is confusing.
>>
>> Forgive me, in which way is it confusing?
>>
>
> All other flow rules enabled by creating flow rule, but this one
Hold on.. Did you just say "flow rules"? But this API is *not* about
flow rules. I suggest you please re-read description of the API.
It reads: "Negotiate the NIC's ability to deliver specific
kinds of metadata to the PMD". Nothing about flows there.
Why is it decoupled from flow library this way? Because there is
a drastic difference between generating and plumbing metadata
INSIDE the NIC's flow engine, on the one hand, and delivering
these data from the NIC to the host driver, on the other.
Let me explain. Say, one creates the following flow rules:
(a) flow create 0 transfer group 0 pattern [A] / end \
actions mark id XXX jump group 1 / end
(b) flow create 0 transfer group 0 pattern [B] / end \
actions mark id YYY jump group 1 / end
(c) flow create 0 transfer group 1 pattern mark id is XXX / end \
actions represented_port ethdev_id 1 / end
(d) flow create 0 transfer group 1 pattern mark id is YYY / end \
actions drop / end
In this example, metadata of type "MARK" is used to partition
flow group (table) 1 into multiple lookup sections. So the
mark value is GENERATED by the flow engine and then it is
CONSUMED by this very flow engine. The application may
NOT necessarily want to receive the mark with mbufs...
And it is only when the application wants these metadata
DELIVERED to it that it has to call the negotiate API.
The short of it, nothing prevents the driver from accepting
flow create requests that leverage some meta items/actions.
Drivers do not need the negotiate API to *configure flow*.
They only need this API in order to let the application
choose whether metadata will be DELIVERED (!) or not.
> requires an API to enable it, so I believe it is inconsistent in that way.
Please see above. Everything is consistent as *flow library*
and *negotiate metadata delivery* API are totally decoupled.
>
> From application perspective, assume that it doesn't know NIC details,
> should it call this API or not? Without API call should application
> assume Rx metadata flow rules are enabled or disabled?
Frankly, applications like testpmd need never call this API.
Simply because seeing, for example, MARK values in mbufs is
useless to it. This API is needed by other applications.
For example, OvS has partial MARK+RSS offload. It adds
flows that distribute packets across multiple queues
and set some MARK values for them. OvS is interested
in getting this values with mbufs since they affect
further lookups in software... So, OvS, knowing it
wants these metadata DELIVERED (!), should invoke
this metadata negotiate API to ensure that.
>
>
> As I understand intention is to get hint from application if it will
> require Rx metadata flow rules so that PMD can optimize better, but if
No, nothing about flow rules. Just delivery of metadata with mbufs.
> PMD doesn't enable Rx metadata flow rules when this API call is missing
Again, PMD shall not make decisions on whether to enable or disable
support for some FLOW primitives based on interactions via this API.
This API exists solely to let PMD configure delivery of metadata,
i.e. not the way it is generated in the first instance.
> than it becomes a mandatory API to configure the device. But I think it
> should be optional for optimization.
>
> Also if application not sure to use this flow rule or not, it will by
> default enable all in any case, this will reduce the benefit. As done in
> testpmd.
Please see above. PMD does not need this API, I take it.
>
>
> API works in both ways, it request to enable some Rx metadata flow rule,
> but based on what PMD returns application can know what device supports,
> this also inconsistent with how other flow rules work, we don't have API
> to get capability but detect them via flow create/validate.
> Can there be a case API returns a flow rule negotiated, but it fails
> when tried to create the flow rule, isn't this confusing for application
>
>
> I think if we continue this approach there can be multiple enable and
> capability learning APIs for various flow rules or flow rule groups, and
> this can make flow API much more harder to use for applications.
See my explanations above. This API is not about flows. Period.
>
>
>
>>>
>>>> Just to avoid back and forth. We will call off this patch and remove
>>>> rte_eth_rx_metadata_negotiate()
>>>> PMD callback from cnxk driver. Keep it as old behavior, so we don't
>>>> need to care
>>>> about rte_eth_rx_metadata_negotiate().
>>>>
>>>
>>> When you remove 'rx_metadata_negotiate' callback, what will be the PMD
>>> behavior? I assume PMD will do the required preparations as if all Rx
>>> metadata is enabled.
>>> And what is the performance impact, is removing callback improve the
>>> performance?
>>>
>>>
>>>> [1]
>>>> The only reason is, typically testpmd will be used performance
>>>> benchmarking as an industry standard. It is difficult to tell/educate
>>>> the QA or customers
>>>> that, "BTW if you need to get better performance add more flag to
>>>> testpmd command line".
>>>> To make that worst, only some PMD needs to give the additional
>>>> parameter to get better number.
>>>> And also, testpmd usage will be treated as application modeling.
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> To make that worst, only some PMD needs to give the additional
>>>>>>>>>>> parameter to get better number.
>>>>>>>>>>> And also, testpmd usage will be treated as application modeling.
>>>>>>>>>>>
>>>>>>>>>>> Since this feature only used on sfc and cnxk driver, What is the
>>>>>>>>>>> situation with sfc driver?
>>>>>>>>>>> Keeping it as negotiated and not use the feature, will impact
>>>>>>>>>>> the per
>>>>>>>>>>> core performance of sfc or
>>>>>>>>>>> is it just PCI bandwidth thing which really dont show any
>>>>>>>>>>> difference in testpmd?
>>>>>>>>>>
>>>>>>>>>> Yes, sfc could run faster if no Rx metadata are negotiated. So,
>>>>>>>>>> it is better to negotiate nothing by default. But it is always
>>>>>>>>>> painful to change defaults. You need to explain that now you
>>>>>>>>>> need to negotiate Rx metadata to use mark, flag and tunnel
>>>>>>>>>> offloads.
>>>>>>>>>> Yes, it will be required on sfc and cnxk only.
>>>>>>>>>> As an sfc maintainer I don't mind to change testpmd defaults.
>>>>>>>>>
>>>>>>>>> If we change testpmd defaults to "do nothing",
>>>>>>>>> then we should disable MBUF_FAST_FREE as well.
>>>>>>>>
>>>>>>>> if you see MBUF_FAST_FREE, it does nothing. Actually,
>>>>>>>> !MBUF_FAST_FREE is doing more work.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>>>
>
>
next prev parent reply other threads:[~2023-02-02 10:41 UTC|newest]
Thread overview: 73+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-01 4:41 [PATCH] app/testpmd: add command line argument 'rx-metadata' Hanumanth Pothula
2022-08-01 13:11 ` Hanumanth Pothula
2022-08-01 13:13 ` Hanumanth Pothula
2022-08-01 19:41 ` Ivan Malov
2022-08-02 16:45 ` [PATCH] app/testpmd: add command line argument 'nic-to-pmd-rx-metadata' Hanumanth Pothula
2022-08-02 16:45 ` [PATCH v2 1/2] version: 22.11-rc0 Hanumanth Pothula
2022-08-02 16:45 ` [PATCH v2 2/2] app/testpmd: add command line argument 'nic-to-pmd-rx-metadata' Hanumanth Pothula
2022-08-02 17:51 ` [PATCH v2 1/1] " Hanumanth Pothula
2022-08-30 12:36 ` Hanumanth Reddy Pothula
2022-09-01 8:03 ` Singh, Aman Deep
2022-10-04 14:48 ` Andrew Rybchenko
2022-10-06 18:35 ` [PATCH v3 1/1] app/testpmd: control passing Rx metadata to PMD Hanumanth Pothula
2022-10-17 8:32 ` Andrew Rybchenko
2022-10-27 7:34 ` Thomas Monjalon
2022-10-27 12:54 ` Thomas Monjalon
2022-12-02 16:14 ` [EXT] " Hanumanth Reddy Pothula
2022-12-02 19:41 ` Thomas Monjalon
2022-12-05 7:59 ` Hanumanth Reddy Pothula
2022-12-05 8:28 ` Thomas Monjalon
2022-12-05 9:43 ` Slava Ovsiienko
2022-12-20 20:02 ` [PATCH v4 1/2] ethdev: control Rx metadata negotiation Hanumanth Pothula
2022-12-20 20:02 ` [PATCH v4 2/2] app/testpmd: add command to process " Hanumanth Pothula
2022-12-20 21:23 ` Stephen Hemminger
2022-12-21 2:07 ` [PATCH v5 1/2] ethdev: fix ethdev configuration state on reset Hanumanth Pothula
2022-12-21 2:07 ` [PATCH v5 2/2] app/testpmd: add command to process Rx metadata negotiation Hanumanth Pothula
2023-01-18 10:32 ` Thomas Monjalon
2023-01-19 10:33 ` [EXT] " Hanumanth Reddy Pothula
2023-01-25 12:51 ` Thomas Monjalon
2023-01-24 18:04 ` Ferruh Yigit
2023-01-25 9:30 ` [EXT] " Hanumanth Reddy Pothula
2023-01-25 12:55 ` Thomas Monjalon
2023-01-25 13:55 ` Ferruh Yigit
2023-01-25 13:59 ` Thomas Monjalon
2023-01-25 14:42 ` Nithin Kumar Dabilpuram
2023-01-26 11:03 ` Thomas Monjalon
2023-01-27 5:02 ` Nithin Kumar Dabilpuram
2023-01-27 8:54 ` Thomas Monjalon
2023-01-27 10:42 ` Nithin Kumar Dabilpuram
2023-01-27 15:01 ` Thomas Monjalon
2023-01-31 16:17 ` Jerin Jacob
2023-01-31 23:03 ` Thomas Monjalon
2023-02-01 6:10 ` Ivan Malov
2023-02-01 7:16 ` Andrew Rybchenko
2023-02-01 8:53 ` Jerin Jacob
2023-02-01 9:00 ` Ori Kam
2023-02-01 9:05 ` Thomas Monjalon
2023-02-01 9:07 ` Andrew Rybchenko
2023-02-01 9:14 ` Jerin Jacob
2023-02-01 9:29 ` Andrew Rybchenko
2023-02-01 10:48 ` Jerin Jacob
2023-02-01 10:58 ` Andrew Rybchenko
2023-02-01 11:04 ` Thomas Monjalon
2023-02-01 11:15 ` Jerin Jacob
2023-02-01 11:35 ` Ferruh Yigit
2023-02-01 13:48 ` Jerin Jacob
2023-02-01 14:50 ` Ferruh Yigit
2023-02-01 15:22 ` Jerin Jacob
2023-02-02 8:43 ` Ferruh Yigit
2023-02-02 8:50 ` Ivan Malov
2023-02-02 9:17 ` Ferruh Yigit
2023-02-02 10:41 ` Ivan Malov [this message]
2023-02-02 10:48 ` Ivan Malov
2023-02-02 11:41 ` Thomas Monjalon
2023-02-02 11:55 ` Ivan Malov
2023-02-02 12:03 ` Thomas Monjalon
2023-02-02 12:21 ` Andrew Rybchenko
2023-02-01 11:20 ` Ivan Malov
2023-01-25 13:17 ` Ferruh Yigit
2023-01-25 13:21 ` Ferruh Yigit
2023-01-25 13:21 ` Ferruh Yigit
2023-01-16 10:43 ` [PATCH v5 1/2] ethdev: fix ethdev configuration state on reset Hanumanth Reddy Pothula
2023-01-18 10:29 ` Thomas Monjalon
2023-01-24 18:14 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=da48595-f8ba-dcb8-9ce4-a66ee87274c@arknetworks.am \
--to=ivan.malov@arknetworks.am \
--cc=aman.deep.singh@intel.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=hpothula@marvell.com \
--cc=jerinj@marvell.com \
--cc=jerinjacobk@gmail.com \
--cc=ndabilpuram@marvell.com \
--cc=orika@nvidia.com \
--cc=thomas@monjalon.net \
--cc=viacheslavo@nvidia.com \
--cc=yuying.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).