From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f178.google.com (mail-io0-f178.google.com [209.85.223.178]) by dpdk.org (Postfix) with ESMTP id C53F25AB0 for ; Wed, 11 May 2016 17:32:49 +0200 (CEST) Received: by mail-io0-f178.google.com with SMTP id d62so58921991iof.2 for ; Wed, 11 May 2016 08:32:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=yKAaLIWLi1LgjGYnbcidg0opmKfz7l91/3gThKxx7rc=; b=0tIqZOK63vTOXAPhAKDdXtKbEzHkxduDYleXChFwF+U94t5MCFIg/HcTgIEVnEVTWp dePlSl2B0BTTx37TmBIEUJiCJ7aXzkhYfm3/xNA7UKPktOaYrgHp5NXwJtLk5ghHCgwP CRTQDQbiQ/1p5c19x7EZc0Qo2HvooXWAm1rSJGQA8phfLJOS4zpcqKHRVvZcQxzaaHzu ST+LJxPGWkJJ7TPx/gVcX0K+TwJC91TItvyYOU729hXLI8PML3SWs95z18xRIHN9Ihhp AYXeu32TfpBjyajRmZn9trpFSve825WCfaIjyYoTzfhRMCl/AA2lcZ9l98s2HJarruml oK8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=yKAaLIWLi1LgjGYnbcidg0opmKfz7l91/3gThKxx7rc=; b=mAzWj5f18DyOV3TvOwcb3wL7oKMF3vp8pPjfsYI4dqZLJh4/FCK381bClbcgDLomrR ZRG20E7t1P/SAqaKH1jWCRbhxlkZqgoSZ04btDwIy46uO9M2pkYoD2IP7187FBsw1V0W IbuaiSOlyWmAe7Qf1gcxD/q6kF0sCg4WE+p5jVSILjsYUSVmeTEcMFfasnCxCa0Eb4Uv 65toK1nE6miJjox4e+JU1Wq102tcEg93HBJsMjLGbX7n8INxbC5TEpXfEWs6r4faWp7C ODr0dOTK5SAtTL7j6EVDDZ1qxlMhChXL9dSeevDr0CxXeGNROK11agcrNDZtpSteNyo/ wrNQ== X-Gm-Message-State: AOPr4FUKMIwh3VRViXJyTEi21Qem4LNwkbYDvNOyjRhRb5m2rkkEyJLM15J0nR3nBI9tePqvvvOPmtxVGZSvUA== X-Received: by 10.36.254.136 with SMTP id w130mr2815493ith.49.1462980769166; Wed, 11 May 2016 08:32:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.4.130 with HTTP; Wed, 11 May 2016 08:32:29 -0700 (PDT) In-Reply-To: <72819DC9-3FAB-47A6-A91C-590EE6910959@akamai.com> References: <3B4F586F-A21F-4EDB-84FB-A5CAB253C476@tencara.com> <72819DC9-3FAB-47A6-A91C-590EE6910959@akamai.com> From: Andriy Berestovskyy Date: Wed, 11 May 2016 17:32:29 +0200 Message-ID: To: "Pavey, Nicholas" Cc: Shawn Lewis , "users@dpdk.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-users] Unable to see incoming packets with example KNI application X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2016 15:32:50 -0000 I think you should check the ARP table on the other side. The destination MAC of the ICMP packets should be the same as your vEth0_0 MAC. Could you please check the tcpdump -e -i vEth0_0 with the pings and then ifconfig vEth0_0? Andriy On Wed, May 11, 2016 at 5:00 PM, Pavey, Nicholas wrote: > Hi Shawn, > > I don=E2=80=99t _think_ that=E2=80=99s the problem, but I=E2=80=99m not a= n expert in low level networking so I could be mistaken. > > If I shut down the KNI application, then the vEth* interfaces shut down a= nd the arp table entries related to those interfaces are cleared. > > Here=E2=80=99s the arp table before I start the KNI app: > > arp > Address HWtype HWaddress Flags Mask = Iface > 172.25.48.100 ether a0:42:3f:25:0b:a7 C = eth2 > 172.25.48.1 ether 18:e7:28:96:83:01 C = eth2 > > 172.25.48.1 is the router=E2=80=99s IP. =E2=80=98eth2=E2=80=99 is my 1Gbp= s copper management interface. > > I start the KNI app: > > ./build/kni -c 0xf0 -n 4 -- -P -p 0x3 --config=3D"(0,4,6,8),(1,5,7,9)= =E2=80=9D > > The arp table is unchanged from above. > > Now I set up the IP addresses for the KNI virtual interfaces: > > ifconfig vEth0_0 172.25.48.200 > ifconfig vEth1_0 10.25.48.200 > > > Arp is still unchanged. > > Now, I ping the router using the =E2=80=98vEth0_0=E2=80=99 inteface, whic= h will require an ARP to start with: > > ping -I vEth0_0 172.25.48.1 > > My DPDK machine=E2=80=99s MAC appears in the router=E2=80=99s arp table. > > The arp table on the DPDK machine is now: > > Address HWtype HWaddress Flags Mask = Iface > >>>> 172.25.48.1 ether 18:e7:28:96:83:01 C = vEth0_0 > 172.25.48.100 ether a0:42:3f:25:0b:a7 C = eth2 > 172.25.48.1 ether 18:e7:28:96:83:01 C = eth2 > > > The 1st line is the new entry. > > This looks to me like the ARP process is working correctly, and it seems = that =E2=80=98tcpdump=E2=80=99 is also working as expected. > > What do you think? > > > Thanks, > > > Nick > > > > > From: Shawn Lewis > Date: Wednesday, May 11, 2016 at 10:36 AM > To: "Pavey, Nicholas" > Subject: Re: [dpdk-users] Unable to see incoming packets with example KN= I application > > > > >> On May 11, 2016, at 10:31 AM, Pavey, Nicholas wrote: >> >> Hi Sakthivel, Andriy, >> >> Thanks for the email. >> >> I=E2=80=99m still having trouble, although things do seem to be working = better than before. >> >> It seems that the MAC address suggestion and also the IPs on the same cl= ass B network aren=E2=80=99t the root cause. >> >> >> Current symptoms >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> I rebooted my machine, and took a note of the MAC addresses as the machi= ne started up. It turns out that the KNI virtual interfaces actually did in= itialize with the correct MAC addresses. >> >> I also reconfigured the virtual interfaces to be using different network= classes, so that should no longer be a problem. >> >> Something has improved because I=E2=80=99m able to see traffic on the KN= I virtual interface with tcpdump, which I was not able to do previously. >> >> Unfortunately, even though tcpdump is able to see the traffic correctly,= it seems that the networking stack isn=E2=80=99t working as I would expect= . >> >> >> >> Outbound >> =3D=3D=3D=3D=3D=3D=3D=3D >> >> Ping >> ---- >> >> I can see outbound =E2=80=98pings=E2=80=99 (with initial ARP requests) w= ith =E2=80=98tcpdump', and I can see a response coming back. However, the = =E2=80=98ping=E2=80=99 application reports 100% packet loss. >> >> The ARP traffic is definitely getting out, because I see the IP address = registered in the router=E2=80=99s ARP cache. Likewise, I see a response to= the original =E2=80=98ping=E2=80=99 packet, so the outbound direction seem= s to be working. >> >> >> Inbound >> =3D=3D=3D=3D=3D=3D=3D >> >> The problems seem to be on the inbound side. As we saw above, the outbou= nd side appears to be working reasonably, but I don=E2=80=99t appear to be = able to capture inbound packets. >> >> TCP >> --- >> >> For example, if I set up a simple =E2=80=98netcat=E2=80=99 listener (usi= ng TCP for transport) on the target server: >> >> nc =E2=80=93l 172.25.48.200 9876 >> >> And then attempt to connect to it from another machine, as follows: >> >> nc 172.25.48.200 9876 >> >> >> =E2=80=98tcpdump=E2=80=99 on the target server will show me the incoming= =E2=80=98syn=E2=80=99 and a =E2=80=98syn=E2=80=99 retransmission, but ther= e are no outbound =E2=80=98ack=E2=80=99 packets. >> >> >> UDP >> --- >> >> Similarly, inbound UDP traffic never appears to be routed to the user sp= ace application. >> >> I can counters incrementing on the virtual interface with =E2=80=98sar = =E2=80=93n DEV 1 100=E2=80=99. =E2=80=98tcpdump=E2=80=99 also shows me the = incoming data. >> >> However, if I look at the UDP stats with =E2=80=98sar =E2=80=93n UDP 1 1= 00=E2=80=99, I=E2=80=99m not seeing any packets arriving, even with =E2=80= =98no port=E2=80=99 or =E2=80=98idgmerr=E2=80=99, which I=E2=80=99d normall= y expect if there=E2=80=99s no listening application bound to the IP addres= s. >> >> >> Next steps >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> It almost seems as if the receive side of the network stack simply isn= =E2=80=99t seeing the inbound data (regardless of whether it=E2=80=99s ICMP= , TCP or UDP) and therefore isn=E2=80=99t sending responses. >> >> >> The thing I=E2=80=99m confused about here is how =E2=80=98tcpdump=E2=80= =99 is able to see the traffic - after all, if it=E2=80=99s able to see the= inbound traffic, then a good part of the RX side of the stack must be work= ing. I=E2=80=99d have thought that if =E2=80=98tcpdump=E2=80=99 can see the= traffic, then the rest of the stack should be working too. >> >> It makes me wondering whether perhaps I=E2=80=99m misunderstanding the p= urpose of the KNI system? >> >> My interpretation is that it=E2=80=99s supposed to route traffic from th= e DPDK into the regular Linux network stack, where it can be used as if it = were regular traffic. Do I have that right? >> >> >> >> Do you have any ideas? >> >> Thanks, >> >> >> Nick >> >> >> From: SAKTHIVEL ANAND S >> Date: Wednesday, May 11, 2016 at 3:30 AM >> To: "Pavey, Nicholas" >> Subject: Re: [dpdk-users] Unable to see incoming packets with example K= NI application >> >> >> Hi >> >> When you try to send echo packets(outbound), your PC will try to resolve= ARP, which it could not complete properly .. due to random mac generation = for KNI interface(your KNI interface having different MAC than actual inter= face). >> >> >> After starting KNI app write your hardware address on KNI by doing, "ifc= onfig hw ether " and try ping. Let me know the = results. >> >> >> use "tcpdump -n -i vEth** -e | grep " >> >> Regards >> >> Sakthivel S OM >> >> > > In many cases I have seen where the ether Mac header is 0s for arp packet= s. So when pushing the packets to the kernel you may have to repopulate the= Mac header. This way the kernel can build its internal arp cache. > > Took me a week to find that !! > > After doing that all worked fine > > > --=20 Andriy Berestovskyy