From: Spike Du <spiked@nvidia.com>
To: Jerin Jacob <jerinjacobk@gmail.com>
Cc: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
Cristian Dumitrescu <cristian.dumitrescu@intel.com>,
Ferruh Yigit <ferruh.yigit@intel.com>,
"techboard@dpdk.org" <techboard@dpdk.org>,
Matan Azrad <matan@nvidia.com>,
Slava Ovsiienko <viacheslavo@nvidia.com>,
Ori Kam <orika@nvidia.com>,
"NBU-Contact-Thomas Monjalon (EXTERNAL)" <thomas@monjalon.net>,
dpdk-dev <dev@dpdk.org>, Raslan Darawsheh <rasland@nvidia.com>
Subject: RE: [RFC 0/6] net/mlx5: introduce limit watermark and host shaper
Date: Mon, 2 May 2022 03:58:36 +0000 [thread overview]
Message-ID: <MN2PR12MB3647460CFC2981D6231AB387A8C19@MN2PR12MB3647.namprd12.prod.outlook.com> (raw)
In-Reply-To: <CALBAE1N6cyghQ0hTv_za6GHxqZsjBzJFee3+CNoc-m+p6Df4KA@mail.gmail.com>
Hi Jerin,
> > For case two(host shaper), I think we can't use RX meter, because it's
> actually TX shaper on a remote system. It's quite specific to Mellanox/Nvidia
> BlueField 2(BF2 for short) NIC. The NIC contains an ARM system. We have
> two terms here: Host-system stands for the system the BF2 NIC is inserted;
> ARM-system stands for the embedded ARM in BF2. ARM-system is doing the
> forwarding. This is the way host shaper works: we configure the register on
> ARM-system, but it affects Host-system's TX shaper, which means the
> shaper is working on the remote port, it's not a RX meter concept, hence we
> can't use DPDK RX meter framework. I'd suggest to still use private API.
>
> OK. If the host is using the DPDK application then rte_tm can be used on the
> egress side to enable the same. If it is not DPDK, then yes, we need private
> APIs.
I see your point. The RX drop happens on ARM-system, it'll be too late to notify Host-system to reduce traffic rate. To achieve dropless, MLX developed
this feature to configure host shaper on remote port. The Host-system is flexible, it may use DPDK or not.
Regards,
Spike.
> -----Original Message-----
> From: Jerin Jacob <jerinjacobk@gmail.com>
> Sent: Sunday, May 1, 2022 8:51 PM
> To: Spike Du <spiked@nvidia.com>
> Cc: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>; Cristian
> Dumitrescu <cristian.dumitrescu@intel.com>; Ferruh Yigit
> <ferruh.yigit@intel.com>; techboard@dpdk.org; Matan Azrad
> <matan@nvidia.com>; Slava Ovsiienko <viacheslavo@nvidia.com>; Ori Kam
> <orika@nvidia.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
> <thomas@monjalon.net>; dpdk-dev <dev@dpdk.org>; Raslan Darawsheh
> <rasland@nvidia.com>
> Subject: Re: [RFC 0/6] net/mlx5: introduce limit watermark and host shaper
>
> External email: Use caution opening links or attachments
>
>
> On Tue, Apr 26, 2022 at 8:12 AM Spike Du <spiked@nvidia.com> wrote:
> >
> > Hi Jerin,
>
> Hi Spike,
>
> > Thanks for your comments and sorry for the late response.
> >
> > For case one, I think I can refine the design and add LWM(limit
> watermark) in rte_eth_rxconf, and add a new rte_eth_event_type event.
>
> OK.
>
> >
> > For case two(host shaper), I think we can't use RX meter, because it's
> actually TX shaper on a remote system. It's quite specific to Mellanox/Nvidia
> BlueField 2(BF2 for short) NIC. The NIC contains an ARM system. We have
> two terms here: Host-system stands for the system the BF2 NIC is inserted;
> ARM-system stands for the embedded ARM in BF2. ARM-system is doing the
> forwarding. This is the way host shaper works: we configure the register on
> ARM-system, but it affects Host-system's TX shaper, which means the
> shaper is working on the remote port, it's not a RX meter concept, hence we
> can't use DPDK RX meter framework. I'd suggest to still use private API.
>
> OK. If the host is using the DPDK application then rte_tm can be used on the
> egress side to enable the same. If it is not DPDK, then yes, we need private
> APIs.
>
> >
> > For testpmd part, I understand your concern. Because we need one
> private API for host shaper, and we need testpmd's forwarding code to show
> how it works to user, we need to call the private API in testpmd. If current
> patch is not acceptable, what's the correct way to do it? Any framework to
> isolate the PMD private logic from testpmd common code, but still give a
> chance to call private APIs in testpmd?
>
> Please check "PMD API" item in
> http://mails.dpdk.org/archives/dev/2022-April/239191.html
>
> >
> >
> > Regards,
> > Spike.
> >
> >
> >
> > > -----Original Message-----
> > > From: Jerin Jacob <jerinjacobk@gmail.com>
> > > Sent: Tuesday, April 5, 2022 4:59 PM
> > > To: Spike Du <spiked@nvidia.com>; Andrew Rybchenko
> > > <andrew.rybchenko@oktetlabs.ru>; Cristian Dumitrescu
> > > <cristian.dumitrescu@intel.com>; Ferruh Yigit
> > > <ferruh.yigit@intel.com>; techboard@dpdk.org
> > > Cc: Matan Azrad <matan@nvidia.com>; Slava Ovsiienko
> > > <viacheslavo@nvidia.com>; Ori Kam <orika@nvidia.com>; NBU-Contact-
> > > Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; dpdk-dev
> > > <dev@dpdk.org>; Raslan Darawsheh <rasland@nvidia.com>
> > > Subject: Re: [RFC 0/6] net/mlx5: introduce limit watermark and host
> > > shaper
> > >
> > > External email: Use caution opening links or attachments
> > >
> > >
> > > On Fri, Apr 1, 2022 at 8:53 AM Spike Du <spiked@nvidia.com> wrote:
> > > >
> > > > LWM(limit watermark) is per RX queue attribute, when RX queue
> > > > fullness reach the LWM limit, HW sends an event to dpdk application.
> > > > Host shaper can configure shaper rate and lwm-triggered for a host port.
> > > > The shaper limits the rate of traffic from host port to wire port.
> > > > If lwm-triggered is enabled, a 100Mbps shaper is enabled
> > > > automatically when one of the host port's Rx queues receives LWM
> event.
> > > >
> > > > These two features can combine to control traffic from host port
> > > > to wire
> > > port.
> > > > The work flow is configure LWM to RX queue and enable
> > > > lwm-triggered flag in host shaper, after receiving LWM event,
> > > > delay a while until RX queue is empty , then disable the shaper.
> > > > We recycle this work flow to
> > > reduce RX queue drops.
> > > >
> > > > Spike Du (6):
> > > > net/mlx5: add LWM support for Rxq
> > > > common/mlx5: share interrupt management
> > > > net/mlx5: add LWM event handling support
> > > > net/mlx5: add private API to configure Rxq LWM
> > > > net/mlx5: add private API to config host port shaper
> > > > app/testpmd: add LWM and Host Shaper command
> > >
> > > + @Andrew Rybchenko @Ferruh Yigit cristian.dumitrescu@intel.com
> > >
> > > I think, case one, can be easily abstracted via adding new
> > > rte_eth_event_type event and case two can be abstracted via the
> > > existing Rx meter framework in ethdev.
> > >
> > > Also, Updating generic testpmd to support PMD specific API should be
> > > avoided, I know there is existing stuff in testpmd, I think, we
> > > should have the policy to add PMD specific commands to testpmd.
> > >
> > > There are around 56PMDs in ethdev now, If PMDs try to add PMD
> > > specific API in testpmd it will be bloated or at minimum, it should
> > > a separate file in testpmd if we choose to take that path.
> > >
> > > + @techboard@dpdk.org
next prev parent reply other threads:[~2022-05-02 3:58 UTC|newest]
Thread overview: 131+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-01 3:22 Spike Du
2022-04-01 3:22 ` [RFC 1/6] net/mlx5: add LWM support for Rxq Spike Du
2022-05-06 3:56 ` [RFC v1 0/7] net/mlx5: introduce limit watermark and host shaper Spike Du
2022-05-06 3:56 ` [RFC v1 1/7] net/mlx5: add LWM support for Rxq Spike Du
2022-05-06 3:56 ` [RFC v1 2/7] common/mlx5: share interrupt management Spike Du
2022-05-06 3:56 ` [RFC v1 3/7] ethdev: introduce Rx queue based limit watermark Spike Du
2022-05-19 9:37 ` Andrew Rybchenko
2022-05-06 3:56 ` [RFC v1 4/7] net/mlx5: add LWM event handling support Spike Du
2022-05-06 3:56 ` [RFC v1 5/7] net/mlx5: support Rx queue based limit watermark Spike Du
2022-05-06 3:56 ` [RFC v1 6/7] net/mlx5: add private API to config host port shaper Spike Du
2022-05-06 3:56 ` [RFC v1 7/7] app/testpmd: add LWM and Host Shaper command Spike Du
2022-05-22 5:58 ` [RFC v2 0/7] introduce per-queue limit watermark and host shaper Spike Du
2022-05-22 5:58 ` [RFC v2 1/7] net/mlx5: add LWM support for Rxq Spike Du
2022-05-22 5:58 ` [RFC v2 2/7] common/mlx5: share interrupt management Spike Du
2022-05-22 5:58 ` [RFC v2 3/7] ethdev: introduce Rx queue based limit watermark Spike Du
2022-05-22 15:23 ` Stephen Hemminger
2022-05-23 3:01 ` Spike Du
2022-05-23 21:45 ` Thomas Monjalon
2022-05-24 2:50 ` Spike Du
2022-05-24 8:18 ` Thomas Monjalon
2022-05-25 12:59 ` Andrew Rybchenko
2022-05-25 13:58 ` Thomas Monjalon
2022-05-25 14:23 ` Andrew Rybchenko
2022-05-23 22:54 ` Stephen Hemminger
2022-05-24 3:46 ` Spike Du
2022-05-22 15:24 ` Stephen Hemminger
2022-05-23 2:18 ` Spike Du
2022-05-23 6:07 ` Morten Brørup
2022-05-23 10:58 ` Thomas Monjalon
2022-05-23 14:10 ` Spike Du
2022-05-23 14:39 ` Thomas Monjalon
2022-05-24 6:35 ` Andrew Rybchenko
2022-05-24 9:40 ` Morten Brørup
2022-05-22 5:58 ` [RFC v2 4/7] net/mlx5: add LWM event handling support Spike Du
2022-05-22 5:58 ` [RFC v2 5/7] net/mlx5: support Rx queue based limit watermark Spike Du
2022-05-22 5:58 ` [RFC v2 6/7] net/mlx5: add private API to config host port shaper Spike Du
2022-05-22 5:59 ` [RFC v2 7/7] app/testpmd: add LWM and Host Shaper command Spike Du
2022-05-24 15:20 ` [PATCH v3 0/7] introduce per-queue limit watermark and host shaper Spike Du
2022-05-24 15:20 ` [PATCH v3 1/7] net/mlx5: add LWM support for Rxq Spike Du
2022-05-24 15:20 ` [PATCH v3 2/7] common/mlx5: share interrupt management Spike Du
2022-05-24 15:20 ` [PATCH v3 3/7] ethdev: introduce Rx queue based limit watermark Spike Du
2022-05-24 15:20 ` [PATCH v3 4/7] net/mlx5: add LWM event handling support Spike Du
2022-05-24 15:20 ` [PATCH v3 5/7] net/mlx5: support Rx queue based limit watermark Spike Du
2022-05-24 15:20 ` [PATCH v3 6/7] net/mlx5: add private API to config host port shaper Spike Du
2022-05-24 15:20 ` [PATCH v3 7/7] app/testpmd: add LWM and Host Shaper command Spike Du
2022-05-24 15:59 ` [PATCH v3 0/7] introduce per-queue limit watermark and host shaper Thomas Monjalon
2022-05-24 19:00 ` Morten Brørup
2022-05-24 19:22 ` Thomas Monjalon
2022-05-25 14:11 ` Andrew Rybchenko
2022-05-25 13:14 ` Spike Du
2022-05-25 13:40 ` Morten Brørup
2022-05-25 13:59 ` Spike Du
2022-05-25 14:16 ` Morten Brørup
2022-05-25 14:30 ` Andrew Rybchenko
2022-06-03 12:48 ` [PATCH v4 0/7] introduce per-queue fill threshold " Spike Du
2022-06-03 12:48 ` [PATCH v4 1/7] net/mlx5: add LWM support for Rxq Spike Du
2022-06-03 12:48 ` [PATCH v4 2/7] common/mlx5: share interrupt management Spike Du
2022-06-03 14:30 ` Ray Kinsella
2022-06-03 12:48 ` [PATCH v4 3/7] ethdev: introduce Rx queue based fill threshold Spike Du
2022-06-03 14:30 ` Ray Kinsella
2022-06-04 12:46 ` Andrew Rybchenko
2022-06-06 13:16 ` Spike Du
2022-06-06 17:15 ` Andrew Rybchenko
2022-06-06 21:30 ` Thomas Monjalon
2022-06-07 8:02 ` Andrew Rybchenko
2022-06-07 6:00 ` Spike Du
2022-06-06 15:49 ` Stephen Hemminger
2022-06-03 12:48 ` [PATCH v4 4/7] net/mlx5: add LWM event handling support Spike Du
2022-06-03 12:48 ` [PATCH v4 5/7] net/mlx5: support Rx queue based fill threshold Spike Du
2022-06-03 12:48 ` [PATCH v4 6/7] net/mlx5: add private API to config host port shaper Spike Du
2022-06-03 14:55 ` Ray Kinsella
2022-06-03 12:48 ` [PATCH v4 7/7] app/testpmd: add Host Shaper command Spike Du
2022-06-07 12:59 ` [PATCH v5 0/7] introduce per-queue available descriptor threshold and host shaper Spike Du
2022-06-07 12:59 ` [PATCH v5 1/7] net/mlx5: add LWM support for Rxq Spike Du
2022-06-08 20:10 ` Matan Azrad
2022-06-07 12:59 ` [PATCH v5 2/7] common/mlx5: share interrupt management Spike Du
2022-06-07 12:59 ` [PATCH v5 3/7] ethdev: introduce Rx queue based available descriptor threshold Spike Du
2022-06-07 12:59 ` [PATCH v5 4/7] net/mlx5: add LWM event handling support Spike Du
2022-06-07 12:59 ` [PATCH v5 5/7] net/mlx5: support Rx queue based available descriptor threshold Spike Du
2022-06-07 12:59 ` [PATCH v5 6/7] net/mlx5: add private API to config host port shaper Spike Du
2022-06-07 12:59 ` [PATCH v5 7/7] app/testpmd: add Host Shaper command Spike Du
2022-06-09 7:55 ` Andrew Rybchenko
2022-06-10 2:22 ` Spike Du
2022-06-13 2:50 ` [PATCH v6] " Spike Du
2022-06-13 2:50 ` Spike Du
2022-06-14 9:43 ` Singh, Aman Deep
2022-06-14 9:54 ` Spike Du
2022-06-14 12:01 ` [PATCH v7] " Spike Du
2022-06-14 12:01 ` Spike Du
2022-06-15 7:51 ` Matan Azrad
2022-06-15 11:08 ` Thomas Monjalon
2022-06-15 12:58 ` [PATCH v8 0/6] introduce per-queue available descriptor threshold and host shaper Spike Du
2022-06-15 12:58 ` [PATCH v8 1/6] net/mlx5: add LWM support for Rxq Spike Du
2022-06-15 14:43 ` [PATCH v9 0/6] introduce per-queue available descriptor threshold and host shaper Spike Du
2022-06-15 14:43 ` [PATCH v9 1/6] net/mlx5: add LWM support for Rxq Spike Du
2022-06-16 8:41 ` [PATCH v10 0/6] introduce per-queue available descriptor threshold and host shaper Spike Du
2022-06-16 8:41 ` [PATCH v10 1/6] net/mlx5: add LWM support for Rxq Spike Du
2022-06-16 8:41 ` [PATCH v10 2/6] common/mlx5: share interrupt management Spike Du
2022-06-23 16:05 ` Ray Kinsella
2022-06-16 8:41 ` [PATCH v10 3/6] net/mlx5: add LWM event handling support Spike Du
2022-06-16 8:41 ` [PATCH v10 4/6] net/mlx5: support Rx queue based available descriptor threshold Spike Du
2022-06-16 8:41 ` [PATCH v10 5/6] net/mlx5: add private API to config host port shaper Spike Du
2022-06-16 8:41 ` [PATCH v10 6/6] app/testpmd: add Host Shaper command Spike Du
2022-06-19 8:14 ` [PATCH v10 0/6] introduce per-queue available descriptor threshold and host shaper Raslan Darawsheh
2022-06-15 14:43 ` [PATCH v9 2/6] common/mlx5: share interrupt management Spike Du
2022-06-15 14:43 ` [PATCH v9 3/6] net/mlx5: add LWM event handling support Spike Du
2022-06-15 14:43 ` [PATCH v9 4/6] net/mlx5: support Rx queue based available descriptor threshold Spike Du
2022-06-15 14:43 ` [PATCH v9 5/6] net/mlx5: add private API to config host port shaper Spike Du
2022-06-15 14:43 ` [PATCH v9 6/6] app/testpmd: add Host Shaper command Spike Du
2022-06-15 12:58 ` [PATCH v8 2/6] common/mlx5: share interrupt management Spike Du
2022-06-15 12:58 ` [PATCH v8 3/6] net/mlx5: add LWM event handling support Spike Du
2022-06-15 12:58 ` [PATCH v8 4/6] net/mlx5: support Rx queue based available descriptor threshold Spike Du
2022-06-15 12:58 ` [PATCH v8 5/6] net/mlx5: add private API to config host port shaper Spike Du
2022-06-15 12:58 ` [PATCH v8 6/6] app/testpmd: add Host Shaper command Spike Du
2022-06-08 9:43 ` [PATCH v5 0/7] introduce per-queue available descriptor threshold and host shaper Andrew Rybchenko
2022-06-08 16:35 ` [PATCH v6] ethdev: introduce available Rx descriptors threshold Andrew Rybchenko
2022-06-08 17:22 ` Thomas Monjalon
2022-06-08 17:46 ` Thomas Monjalon
2022-06-09 0:17 ` fengchengwen
2022-06-09 7:05 ` Thomas Monjalon
2022-06-10 0:01 ` fengchengwen
2022-04-01 3:22 ` [RFC 2/6] common/mlx5: share interrupt management Spike Du
2022-04-01 3:22 ` [RFC 3/6] net/mlx5: add LWM event handling support Spike Du
2022-04-01 3:22 ` [RFC 4/6] net/mlx5: add private API to configure Rxq LWM Spike Du
2022-04-01 3:22 ` [RFC 5/6] net/mlx5: add private API to config host port shaper Spike Du
2022-04-01 3:22 ` [RFC 6/6] app/testpmd: add LWM and Host Shaper command Spike Du
2022-04-05 8:58 ` [RFC 0/6] net/mlx5: introduce limit watermark and host shaper Jerin Jacob
2022-04-26 2:42 ` Spike Du
2022-05-01 12:50 ` Jerin Jacob
2022-05-02 3:58 ` Spike Du [this message]
2022-04-29 5:48 ` Spike Du
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=MN2PR12MB3647460CFC2981D6231AB387A8C19@MN2PR12MB3647.namprd12.prod.outlook.com \
--to=spiked@nvidia.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=cristian.dumitrescu@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=jerinjacobk@gmail.com \
--cc=matan@nvidia.com \
--cc=orika@nvidia.com \
--cc=rasland@nvidia.com \
--cc=techboard@dpdk.org \
--cc=thomas@monjalon.net \
--cc=viacheslavo@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).