From: Avi Kivity <avi@cloudius-systems.com>
To: Konstantin Ananyev <konstantin.ananyev@intel.com>
Cc: "<dev@dpdk.org>" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH v1] ixgbe_pmd: forbid tx_rs_thresh above 1 for all NICs but 82598
Date: Sun, 13 Sep 2015 19:01:56 +0300 [thread overview]
Message-ID: <CAF950WL+oy__E=UwU9ZQnimbJ1_-MrE2fBcwhGMko=pT9riFzQ@mail.gmail.com> (raw)
In-Reply-To: <2601191342CEEE43887BDE71AB97725836A85FDD@irsmsx105.ger.corp.intel.com>
On Sep 13, 2015 6:54 PM, "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
wrote:
>
>
>
> > -----Original Message-----
> > From: Avi Kivity [mailto:avi@cloudius-systems.com]
> > Sent: Sunday, September 13, 2015 1:33 PM
> > To: Ananyev, Konstantin; Thomas Monjalon; Vladislav Zolotarov;
didier.pallard
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v1] ixgbe_pmd: forbid tx_rs_thresh above
1 for all NICs but 82598
> >
> > On 09/13/2015 02:47 PM, Ananyev, Konstantin wrote:
> > >
> > >> -----Original Message-----
> > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Avi Kivity
> > >> Sent: Friday, September 11, 2015 6:48 PM
> > >> To: Thomas Monjalon; Vladislav Zolotarov; didier.pallard
> > >> Cc: dev@dpdk.org
> > >> Subject: Re: [dpdk-dev] [PATCH v1] ixgbe_pmd: forbid tx_rs_thresh
above 1 for all NICs but 82598
> > >>
> > >> On 09/11/2015 07:08 PM, Thomas Monjalon wrote:
> > >>> 2015-09-11 18:43, Avi Kivity:
> > >>>> On 09/11/2015 06:12 PM, Vladislav Zolotarov wrote:
> > >>>>> On Sep 11, 2015 5:55 PM, "Thomas Monjalon" <
thomas.monjalon@6wind.com
> > >>>>> <mailto:thomas.monjalon@6wind.com>> wrote:
> > >>>>>> 2015-09-11 17:47, Avi Kivity:
> > >>>>>>> On 09/11/2015 05:25 PM, didier.pallard wrote:
> > >>>>>>>> Hi vlad,
> > >>>>>>>>
> > >>>>>>>> Documentation states that a packet (or multiple packets in
transmit
> > >>>>>>>> segmentation) can span any number of
> > >>>>>>>> buffers (and their descriptors) up to a limit of 40 minus
WTHRESH
> > >>>>>>>> minus 2.
> > >>>>>>>>
> > >>>>>>>> Shouldn't there be a test in transmit function that drops
> > >>>>> properly the
> > >>>>>>>> mbufs with a too large number of
> > >>>>>>>> segments, while incrementing a statistic; otherwise transmit
> > >>>>> function
> > >>>>>>>> may be locked by the faulty packet without
> > >>>>>>>> notification.
> > >>>>>>>>
> > >>>>>>> What we proposed is that the pmd expose to dpdk, and dpdk expose
> > >>>>> to the
> > >>>>>>> application, an mbuf check function. This way applications
that can
> > >>>>>>> generate complex packets can verify that the device will be
able to
> > >>>>>>> process them, and applications that only generate simple mbufs
can
> > >>>>> avoid
> > >>>>>>> the overhead by not calling the function.
> > >>>>>> More than a check, it should be exposed as a capability of the
port.
> > >>>>>> Anyway, if the application sends too much segments, the driver
must
> > >>>>>> drop it to avoid hang, and maintain a dedicated statistic
counter to
> > >>>>>> allow easy debugging.
> > >>>>> I agree with Thomas - this should not be optional. Malformed
packets
> > >>>>> should be dropped. In the icgbe case it's a very simple test -
it's a
> > >>>>> single branch per packet so i doubt that it could impose any
> > >>>>> measurable performance degradation.
> > >>>> A drop allows the application no chance to recover. The driver
must
> > >>>> either provide the ability for the application to know that it
cannot
> > >>>> accept the packet, or it must fix it up itself.
> > >>> I have the feeling that everybody agrees on the same thing:
> > >>> the application must be able to make a well formed packet by
checking
> > >>> limitations of the port. What about a field
rte_eth_dev_info.max_tx_segs?
> > >> It is not generic enough. i40e has a limit that it imposes post-TSO.
> > >>
> > >>
> > >>> In case the application fails in its checks, the driver must drop
it and
> > >>> notify the user via a stat counter.
> > >>> The driver can also remove the hardware limitation by gathering the
segments
> > >>> but it may be hard to implement and would be a slow operation.
> > >> I think that to satisfy both the 64b full line rate applications and
the
> > >> more complicated full stack applications, this must be made optional.
> > >> In particular, and application that only forwards packets will never
hit
> > >> a NIC's limits, so it need not take any action. That's why I think a
> > >> verification function is ideal; a forwarding application can ignore
it,
> > >> and a complex application can call it, and if it fails the packet, it
> > >> can linearize it itself, removing complexity from dpdk itself.
> > > I think that's a good approach to that problem.
> > > As I remember we discussed something similar a while ago -
> > > A function (tx_prep() or something) that would check nb_segs and
probably some other HW specific restrictions,
> > > calculate pseudo-header checksum, reset ip header len, etc.
> > >
> > > From other hand we also can add two more fields into
rte_eth_dev_info:
> > > 1) Max num of segs per TSO packet (tx_max_seg ?).
> > > 2) Max num of segs per single packet/TSO segment (tx_max_mtu_seg ?).
> > > So for ixgbe both will have value 40 - wthresh,
> > > while for i40e 1) would be UINT8_MAX and 2) will be 8.
> > > Then upper layer can use that information to select an optimal size
for its TX buffers.
> > >
> > >
> >
> > This will break whenever the fevered imagination of hardware designers
> > comes up with a new limit.
> >
> > We can have an internal function that accepts these two parameters, and
> > then the driver-specific function can call this internal function:
> >
> > static bool i40e_validate_packet(mbuf* m) {
> > return rte_generic_validate_packet(m, 0, 8);
> > }
> >
> > static bool ixgbe_validate_packet(mbuf* m) {
> > return rte_generic_validate_packet(m, 40, 2);
> > }
> >
> > this way, the application is isolated from changes in how invalid
> > packets are detected.
> >
> >
>
> I am not saying we shouldn't have tx_prep (tx_validate?) function per PMD.
> As I said before I like that approach.
> I think we should have tx_prep (as you suggested) that most people using
full-path TX would call,
> *plus* these extra fields in re_eth_dev_conf, so if someone needs that
information - it would be there.
I think this is reasonable. Having those values can allow the application
to avoid generating bad packets in the first place.
> Konstantin
>
next prev parent reply other threads:[~2015-09-13 16:01 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-08-13 18:06 Vlad Zolotarov
2015-08-13 20:28 ` Zhang, Helin
2015-08-14 5:37 ` Vlad Zolotarov
2015-08-19 0:42 ` Lu, Wenzhuo
2015-08-19 4:55 ` Vladislav Zolotarov
2015-08-19 7:43 ` Ananyev, Konstantin
2015-08-19 10:02 ` Vlad Zolotarov
2015-08-20 8:41 ` Ananyev, Konstantin
2015-08-20 8:56 ` Vlad Zolotarov
2015-08-20 9:05 ` Vlad Zolotarov
2015-08-20 9:06 ` Vlad Zolotarov
2015-08-25 17:33 ` Ananyev, Konstantin
2015-08-25 17:39 ` Avi Kivity
2015-08-19 17:29 ` Zhang, Helin
2015-08-25 18:13 ` Zhang, Helin
2015-08-25 18:33 ` Vladislav Zolotarov
2015-08-25 18:43 ` Zhang, Helin
2015-08-25 18:52 ` Vlad Zolotarov
2015-08-25 19:16 ` Zhang, Helin
2015-08-25 19:23 ` Avi Kivity
2015-08-25 19:30 ` Vladislav Zolotarov
2015-08-25 20:07 ` Vlad Zolotarov
2015-08-25 20:13 ` Zhang, Helin
2015-09-09 12:18 ` Thomas Monjalon
2015-09-09 13:19 ` Ananyev, Konstantin
2015-09-11 15:17 ` Vladislav Zolotarov
2015-09-11 14:25 ` didier.pallard
2015-09-11 14:47 ` Avi Kivity
2015-09-11 14:55 ` Thomas Monjalon
2015-09-11 15:12 ` Vladislav Zolotarov
2015-09-11 15:43 ` Avi Kivity
2015-09-11 16:04 ` Vladislav Zolotarov
2015-09-11 16:07 ` Richardson, Bruce
2015-09-11 16:14 ` Vladislav Zolotarov
2015-09-11 17:44 ` Avi Kivity
2015-09-11 16:08 ` Thomas Monjalon
2015-09-11 16:18 ` Vladislav Zolotarov
2015-09-11 17:17 ` Matthew Hall
2015-09-11 17:42 ` Ananyev, Konstantin
2015-09-11 17:58 ` Matthew Hall
2015-09-11 17:48 ` Avi Kivity
2015-09-13 11:47 ` Ananyev, Konstantin
2015-09-13 12:24 ` Vlad Zolotarov
2015-09-13 12:32 ` Avi Kivity
2015-09-13 15:54 ` Ananyev, Konstantin
2015-09-13 16:01 ` Avi Kivity [this message]
2015-09-11 16:00 ` Richardson, Bruce
2015-09-11 16:13 ` Vladislav Zolotarov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAF950WL+oy__E=UwU9ZQnimbJ1_-MrE2fBcwhGMko=pT9riFzQ@mail.gmail.com' \
--to=avi@cloudius-systems.com \
--cc=dev@dpdk.org \
--cc=konstantin.ananyev@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).