Re: [dpdk-dev] ixgbe: account more Rx errors Issue

[Kyle Larose <eomereadig@gmail.com>]

On Mon, Sep 7, 2015 at 7:44 AM, Tahhan, Maryam <maryam.tahhan@intel.com>
wrote:

> > From: Olivier MATZ [mailto:olivier.matz@6wind.com]
> > Sent: Monday, September 7, 2015 9:30 AM
> > To: Tahhan, Maryam; Andriy Berestovskyy
> > Cc: dev@dpdk.org
> > Subject: Re: ixgbe: account more Rx errors Issue
> >
> > Hi,
> >
> > On 09/06/2015 07:15 PM, Tahhan, Maryam wrote:
> > >> From: Andriy Berestovskyy [mailto:aber@semihalf.com]
> > >> Sent: Friday, September 4, 2015 5:59 PM
> > >> To: Tahhan, Maryam
> > >> Cc: dev@dpdk.org; Olivier MATZ
> > >> Subject: Re: ixgbe: account more Rx errors Issue
> > >>
> > >> Hi Maryam,
> > >> Please see below.
> > >>
> > >>> XEC counts the Number of receive IPv4, TCP, UDP or SCTP XSUM errors
> > >>
> > >> Please note than UDP checksum is optional for IPv4, but UDP packets
> > >> with zero checksum hit XEC.
> > >>
> > >
> > > I understand, but this is what the hardware register is picking up and
> what I
> > included previously is the definitions of the registers from the
> datasheet.
> > >
> > >>> And general crc errors counts Counts the number of receive packets
> > >>> with
> > >> CRC errors.
> > >>
> > >> Let me explain you with an example.
> > >>
> > >> DPDK 2.0 behavior:
> > >> host A sends 10M IPv4 UDP packets (no checksum) to host B host B
> > >> stats: 9M ipackets + 1M ierrors (missed) = 10M
> > >>
> > >> DPDK 2.1 behavior:
> > >> host A sends 10M IPv4 UDP packets (no checksum) to host B host B
> > >> stats: 9M ipackets + 11M in ierrors (1M missed + 10M XEC) = 20M?
> > >
> > > Because it's hitting the 2 error registers. If you had packets with
> multiple
> > errors that are added up as part of ierrors you'll still be getting more
> than
> > 10M errors which is why I asked for feedback on the 3 suggestions below.
> > What I'm saying is the number of errors being > the number of received
> > packets will be seen if you hit multiple error registers on the NIC.
> > >
> > >>
> > >>> So our options are we can:
> > >>> 1. Add only one of these into the error stats.
> > >>> 2. We can introduce some cooking of stats in this scenario, so only
> > >>> add
> > >> either or if they are equal or one is higher than the other.
> > >>> 3. Add them all which means you can have more errors than the number
> > >>> of
> > >> received packets, but TBH this is going to be the case if your
> > >> packets have multiple errors anyway.
> > >>
> > >> 4. ierrors should reflect NIC drops only.
> > >
> > > I may have misinterpreted this, but ierrors in rte_ethdev.h ierrors is
> defined
> > as the Total number of erroneous received packets.
> > > Maybe we need a clear definition or a separate drop counter as I see
> > uint64_t q_errors defined as: Total number of queue packets received that
> > are dropped.
> > >
> > >> XEC does not count drops, so IMO it should be removed from ierrors.
> > >
> > > While it's picking up the 0 checksum as an error (which it shouldn't
> > > necessarily be doing), removing it could mean missing other valid
> > > L3/L4 checksum errors... Let me experiment some more with L3/L4
> > > checksum errors and crcerrs to see if we can cook the stats around
> > > this register in particular. I would hate to remove it and miss
> > > genuine errors
> >
> > For me, the definition that looks the most straightforward is:
> >
> > ipackets = packets successfully received by hardware imissed = packets
> > dropped by hardware because the software does
> >   not poll fast enough (= queue full)
> > ierrors = packets dropped by hardware (malformed packets, ...)
> >
> > These 3 stats never count twice the same packet.
> >
> > If we want more statistics, they could go in xstats. For instance, a
> counter for
> > invalid checksum. The definition of these stats would be pmd-specific.
> >
> > I agree we should clarify and have a consensus on the definitions before
> going
> > further.
> >
> >
> > Regards,
> > Olivier
>
> Hi Olivier
> I think it's important to distinguish between errors and drops and provide
> a statistics API that exposes both. This way people have access to as much
> information as possible when things do go wrong and nothing is missed in
> terms of errors.
>
> My suggestion for the high level registers would be:
> ipackets = Total number of packets successfully received by hardware
> imissed = Total number of  packets dropped by hardware because the
> software does not poll fast enough (= queue full)
> idrops = Total number of packets dropped by hardware (malformed packets,
> ...) Where the # of drops can ONLY be <=  the packets received (without
> overlap between registers).
> ierrors = Total number of erroneous received packets. Where the # of
> errors can be >= the packets received (without overlap between registers),
> this is because there may be multiple errors associated with a packet.
>
> This way people can see how many packets were dropped and why at a high
> level as well as through the extended stats API rather than using one API
> or the other. What do you think?
>
> Best Regards
> Maryam
> >
> >
> >
> > >
> > >>
> > >> Please note that we still can access the XEC using
> > >> rte_eth_xstats_get()
> > >>
> > >>
> > >> Regards,
> > >> Andriy
>

Hi Maryam,

If we look to the if-mib (from http://www.ietf.org/rfc/rfc2233.txt), we can
see that their definition of in errors aligns more closely with Olivier's.

There they say (>>> <<< mine):

   ifInErrors OBJECT-TYPE
       SYNTAX      Counter32
       MAX-ACCESS  read-only
       STATUS      current
       DESCRIPTION
               "For packet-oriented interfaces, >>> the number of inbound
               packets that contained errors preventing them from
               being deliverable to a higher-layer protocol <<<.  For
               character-oriented or fixed-length interfaces, the
               number of inbound transmission units that contained
               errors preventing them from being deliverable to a
               higher-layer protocol.

               Discontinuities in the value of this counter can occur
               at re-initialization of the management system, and at
               other times as indicated by the value of
               ifCounterDiscontinuityTime."
       ::= { ifEntry 14 }

They count it as the number of packets, not the number of errors. So, if a
packet contains two errors, it is only counted once.

I'm not sure what the intention of the ierrors stat is. Do we intend to use
it to feed into MIBs/standards such as the above? Or do we intend to make
it something different? If the former, I think we should conform to the
meaning suggested by rfc2233.
Thanks,

Kyle