From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qc0-x22d.google.com (mail-qc0-x22d.google.com [IPv6:2607:f8b0:400d:c01::22d]) by dpdk.org (Postfix) with ESMTP id 3CF1C156 for ; Sun, 3 Nov 2013 21:20:12 +0100 (CET) Received: by mail-qc0-f173.google.com with SMTP id l13so3547760qcy.18 for ; Sun, 03 Nov 2013 12:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=F5pbOZS/xtQyaL403+jn4ejrQCDF4zwSuhRwBHlWgoc=; b=CruJ2S+sQ31tHXpBt1mO5x5UU3d2xSO/wVhfAMVWXysZ3+9mgyjMCSkSSyUWxQrSc8 LayVU5G0GFl+j0l1qlBP6l03vF6Rs1vtXUsOMSYFpPM6+FnJpHg7/dE9OG2iZ6DbWxVW E6k2yw7Q+wfYNBHj2ak3qXuB10VF/miEmupKOq47Dxi2wNEzBib4ZbZNqo+QaJ+fX9/X 6wXKAG9LWZm8D1zYgFQRD46ID9CMYTwPQmjjb/AvcF/WDWEcpXaw79Hr8g3po8X0PyFS h+3l3JHbc80xW6pGqqibzY3sEms3U4NR4jjqUSU6L4j2R6fLfj4eYiVUbtFY2qQG8lC6 9Qeg== MIME-Version: 1.0 X-Received: by 10.49.35.144 with SMTP id h16mr18284969qej.35.1383510059559; Sun, 03 Nov 2013 12:20:59 -0800 (PST) Received: by 10.140.27.4 with HTTP; Sun, 3 Nov 2013 12:20:59 -0800 (PST) In-Reply-To: References: Date: Mon, 4 Nov 2013 00:20:59 +0400 Message-ID: From: Alexander Belyakov To: "Wang, Shawn" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Surprisingly high TCP ACK packets drop counter X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Nov 2013 20:20:12 -0000 Hi, thanks for the patch and explanation. We have tried DPDK 1.3 and 1.5 - both have the same issue. Regards, Alexander On Fri, Nov 1, 2013 at 6:54 PM, Wang, Shawn wrote: > Hi: > > We had the same problem before. It turned out that RSC (receive side > coalescing) is enabled by default in DPDK. So we write this na=C3=AFve pa= tch to > disable it. This patch is based on DPDK 1.3. Not sure 1.5 has changed it > or not. > After this patch, ACK rate should go back to 14.5Mpps. For details, you > can refer to Intel=C2=AE 82599 10 GbE Controller Datasheet. (7.11 Receive= Side > Coalescing). > > From: xingbow > Date: Wed, 21 Aug 2013 11:35:23 -0700 > Subject: [PATCH] Disable RSC in ixgbe_dev_rx_init function in file > > ixgbe_rxtx.c > > --- > > DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h | 2 +- > DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 7 +++++++ > 2 files changed, 8 insertions(+), 1 deletion(-) > > diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > index 7fffd60..f03046f 100644 > > --- a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > > +++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h > > @@ -1930,7 +1930,7 @@ enum { > > #define IXGBE_RFCTL_ISCSI_DIS 0x00000001 > #define IXGBE_RFCTL_ISCSI_DWC_MASK 0x0000003E > #define IXGBE_RFCTL_ISCSI_DWC_SHIFT 1 > -#define IXGBE_RFCTL_RSC_DIS 0x00000010 > > +#define IXGBE_RFCTL_RSC_DIS 0x00000020 > > #define IXGBE_RFCTL_NFSW_DIS 0x00000040 > #define IXGBE_RFCTL_NFSR_DIS 0x00000080 > #define IXGBE_RFCTL_NFS_VER_MASK 0x00000300 > diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > index 07830b7..ba6e05d 100755 > > --- a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > > +++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > > @@ -3007,6 +3007,7 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev) > > uint64_t bus_addr; > uint32_t rxctrl; > uint32_t fctrl; > + uint32_t rfctl; > > uint32_t hlreg0; > uint32_t maxfrs; > uint32_t srrctl; > @@ -3033,6 +3034,12 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev) > > fctrl |=3D IXGBE_FCTRL_PMCF; > IXGBE_WRITE_REG(hw, IXGBE_FCTRL, fctrl); > > + /* Disable RSC */ > + RTE_LOG(INFO, PMD, "Disable RSC\n"); > + rfctl =3D IXGBE_READ_REG(hw, IXGBE_RFCTL); > + rfctl |=3D IXGBE_RFCTL_RSC_DIS; > + IXGBE_WRITE_REG(hw, IXGBE_RFCTL, rfctl); > + > > /* > * Configure CRC stripping, if any. > */ > -- > > > Thanks. > Wang, Xingbo > > > > > On 11/1/13 6:43 AM, "Alexander Belyakov" wrote: > > >Hello, > > > >we have simple test application on top of DPDK which sole purpose is to > >forward as much packets as possible. Generally we easily achieve 14.5Mpp= s > >with two 82599EB (one as input and one as output). The only suprising > >exception is forwarding pure TCP ACK flood when performace always drops = to > >approximately 7Mpps. > > > >For simplicity consider two different types of traffic: > >1) TCP SYN flood is forwarded at 14.5Mpps rate, > >2) pure TCP ACK flood is forwarded only at 7Mpps rate. > > > >Both SYN and ACK packets have exactly the same length. > > > >It is worth to mention, this forwarding application looks at Ethernet an= d > >IP headers, but never deals with L4 headers. > > > >We tracked down issue to RX circuit. To be specific, there are 4 RX queu= es > >initialized on input port and rte_eth_stats_get() shows uniform packet > >distribution (q_ipackets) among them, while q_errors remain zero for all > >queues. The only drop counter quickly increasing in the case of pure ACK > >flood is ierrors, while rx_nombuf remains zero. > > > >We tried different kinds of traffic generators, but always got the same > >result: 7Mpps (instead of expected 14Mpps) for TCP packets with ACK flag > >bit set while all other flag bits dropped. Source IPs and ports are > >selected randomly. > > > >Please let us know if anyone is aware of such strange behavior and where > >should we look at to narrow down the problem. > > > >Thanks in advance, > >Alexander Belyakov > >