Re: [dpdk-dev] Surprisingly high TCP ACK packets drop counter

[Prashant Upadhyaya <prashant.upadhyaya@aricent.com>]

Hi Alexander,

Please confirm if the patch works for you.

@Wang, are you saying that without the patch the NIC does not fan out the messages properly on all the receive queues ?
So what exactly happens ?

Regards
-Prashant


-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alexander Belyakov
Sent: Monday, November 04, 2013 1:51 AM
To: Wang, Shawn
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Surprisingly high TCP ACK packets drop counter

Hi,

thanks for the patch and explanation. We have tried DPDK 1.3 and 1.5 - both have the same issue.

Regards,
Alexander


On Fri, Nov 1, 2013 at 6:54 PM, Wang, Shawn <xingbow@amazon.com> wrote:

> Hi:
>
> We had the same problem before. It turned out that RSC (receive side
> coalescing) is enabled by default in DPDK. So we write this naïve
> patch to disable it. This patch is based on DPDK 1.3. Not sure 1.5 has
> changed it or not.
> After this patch, ACK rate should go back to 14.5Mpps. For details,
> you can refer to Intel® 82599 10 GbE Controller Datasheet. (7.11
> Receive Side Coalescing).
>
> From: xingbow <xingbow@amazon.com>
> Date: Wed, 21 Aug 2013 11:35:23 -0700
> Subject: [PATCH] Disable RSC in ixgbe_dev_rx_init function in file
>
>  ixgbe_rxtx.c
>
> ---
>
>  DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h | 2 +-
>  DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c       | 7 +++++++
>  2 files changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h
> b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h
> index 7fffd60..f03046f 100644
>
> --- a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h
>
> +++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h
>
> @@ -1930,7 +1930,7 @@ enum {
>
>  #define IXGBE_RFCTL_ISCSI_DIS          0x00000001
>  #define IXGBE_RFCTL_ISCSI_DWC_MASK     0x0000003E
>  #define IXGBE_RFCTL_ISCSI_DWC_SHIFT    1
> -#define IXGBE_RFCTL_RSC_DIS            0x00000010
>
> +#define IXGBE_RFCTL_RSC_DIS            0x00000020
>
>  #define IXGBE_RFCTL_NFSW_DIS           0x00000040
>  #define IXGBE_RFCTL_NFSR_DIS           0x00000080
>  #define IXGBE_RFCTL_NFS_VER_MASK       0x00000300
> diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
> index 07830b7..ba6e05d 100755
>
> --- a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
>
> +++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
>
> @@ -3007,6 +3007,7 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev)
>
>         uint64_t bus_addr;
>         uint32_t rxctrl;
>         uint32_t fctrl;
> +       uint32_t rfctl;
>
>         uint32_t hlreg0;
>         uint32_t maxfrs;
>         uint32_t srrctl;
> @@ -3033,6 +3034,12 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev)
>
>         fctrl |= IXGBE_FCTRL_PMCF;
>         IXGBE_WRITE_REG(hw, IXGBE_FCTRL, fctrl);
>
> +       /* Disable RSC */
> +       RTE_LOG(INFO, PMD, "Disable RSC\n");
> +       rfctl = IXGBE_READ_REG(hw, IXGBE_RFCTL);
> +       rfctl |= IXGBE_RFCTL_RSC_DIS;
> +       IXGBE_WRITE_REG(hw, IXGBE_RFCTL, rfctl);
> +
>
>         /*
>          * Configure CRC stripping, if any.
>          */
> --
>
>
> Thanks.
> Wang, Xingbo
>
>
>
>
> On 11/1/13 6:43 AM, "Alexander Belyakov" <abelyako@gmail.com> wrote:
>
> >Hello,
> >
> >we have simple test application on top of DPDK which sole purpose is
> >to forward as much packets as possible. Generally we easily achieve
> >14.5Mpps with two 82599EB (one as input and one as output). The only
> >suprising exception is forwarding pure TCP ACK flood when performace
> >always drops to approximately 7Mpps.
> >
> >For simplicity consider two different types of traffic:
> >1) TCP SYN flood is forwarded at 14.5Mpps rate,
> >2) pure TCP ACK flood is forwarded only at 7Mpps rate.
> >
> >Both SYN and ACK packets have exactly the same length.
> >
> >It is worth to mention, this forwarding application looks at Ethernet
> >and IP headers, but never deals with L4 headers.
> >
> >We tracked down issue to RX circuit. To be specific, there are 4 RX
> >queues initialized on input port and rte_eth_stats_get() shows
> >uniform packet distribution (q_ipackets) among them, while q_errors
> >remain zero for all queues. The only drop counter quickly increasing
> >in the case of pure ACK flood is ierrors, while rx_nombuf remains zero.
> >
> >We tried different kinds of traffic generators, but always got the
> >same
> >result: 7Mpps (instead of expected 14Mpps) for TCP packets with ACK
> >flag bit set while all other flag bits dropped. Source IPs and ports
> >are selected randomly.
> >
> >Please let us know if anyone is aware of such strange behavior and
> >where should we look at to narrow down the problem.
> >
> >Thanks in advance,
> >Alexander Belyakov
>
>




===============================================================================
Please refer to http://www.aricent.com/legal/email_disclaimer.html
for important disclosures regarding this electronic communication.
===============================================================================