From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E1F9A0093 for ; Wed, 9 Nov 2022 09:50:03 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 45627400D7; Wed, 9 Nov 2022 09:50:03 +0100 (CET) Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by mails.dpdk.org (Postfix) with ESMTP id 3F103400D4 for ; Wed, 9 Nov 2022 09:50:01 +0100 (CET) Received: by mail-pl1-f176.google.com with SMTP id p12so10987235plq.4 for ; Wed, 09 Nov 2022 00:50:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=HuNnhZLMaTHJsG//sI6NJNZNm4tDx4YW0RsqpNkLy3E=; b=Q5qJd26gwP19CQXpdi3pwUa4MO4JHi/sExAtW+GorR38gQEopAuOlbUcHgBGGy/sN+ N/GGJN8MrxbFT/7df4qSSGyFJq03xpuEkhNKW5VxNYydwYe7d432pZUwwh7mb1ZIzJXz ZQdbj0Y0pvc0K6IE535TJAMg99MH4Fc8W2mCm1t8QBdjKJUBaIsHYLvYpF+Dp4R1Mq5T 455lU90+zn05sZZGQ+3YETbOxg2wZH47bhwtYzSkBPoz4gvUyKz4ttI2rgLaNiNnLf3o 1G5cNJo5aD3zFm35BLBbfpXXkupDqKGFEYLHWC8hYSpImzN1sH/jRm9X5ae1Jobv3936 3DPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HuNnhZLMaTHJsG//sI6NJNZNm4tDx4YW0RsqpNkLy3E=; b=Z5Eyxq4OY4FEYbV9gfSPENPDjgzXVf/s4t0fOBnRkhP2AZx9cSLiS7hMh+s0PteS8H OP4gMxPR7hJvJcVw1S1eF9coLS386LN+0xXRXBJSKGJqy59fRuLDqdSoBDBPcmgv2PT0 8SZiYv155EPSlTgJYOI7qdLt4S/yt1oBeQG3oQu4+7Uf4UHxXWsQtRxmq6M9xwhai2YR +yKaP3kNqxdmhqYp33bZcWgExGhXgb/eKHrAywUPMulEb5pNoCEJ915nb7YD7WmIInlt AXOuOk3m1XB5ha1KEuSgPKpQakdJNRbi55meHHHfkV+c7X4itUcuXXibl4/M/+OpsLJR IzoA== X-Gm-Message-State: ACrzQf2IN+o9NthmCTzAmTd4UocjU8zCqqVs58067vgptFxDmrbu/ji+ ObPcGDS9M4i1lQdwm5OApVMiDjxqRJaEaybr31CvC/viC9E= X-Google-Smtp-Source: AMsMyM7LoyczsL29YwAu2rVhDzi3a3m57eoLBvC6E1DeeQ0oQ/7tcUBixwJWYY42OfSlXabD/ri00vRhLzn4xNZMSdo= X-Received: by 2002:a17:90a:9dc5:b0:213:c885:bc06 with SMTP id x5-20020a17090a9dc500b00213c885bc06mr56571845pjv.223.1667983800336; Wed, 09 Nov 2022 00:50:00 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: "J.J. Mars" Date: Wed, 9 Nov 2022 16:48:57 +0800 Message-ID: Subject: Re: DPDK server packets with AF_PACKET driver are not received by the client connected to the same virtual bridge. To: Boris Ouretskey Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="000000000000e9052005ed05bd5d" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000e9052005ed05bd5d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable If the nic on docker 1 of echo client can receive replies, the reason should not be PMD I think. Did you make sure every field in every layer is completely the same between eth0 in docker 1 and eth0 in docker 2, and all of them are expected? Boris Ouretskey =E4=BA=8E2022=E5=B9=B411=E6=9C=889=E6= =97=A5=E5=91=A8=E4=B8=89 03:51=E5=86=99=E9=81=93=EF=BC=9A > > > Hi > > I am trying to run a docker based system in one VM, with some servers > using the DPDK and facing a very strange issue. > > The relevant part of the system is the echo server which is built with > DPDK library and runs in docker, connected to the virtual bridge (with > AF_PACKET driver), and echo client built with regular sockets (java > application). See the drawing below for visual details. > > The strange thing is that echo replies sent from DPDK echo server docker > (Docker2), can be seen in tcpdump in the docker of echo client (Docker1) > however the application (java client) does not receive it on socket level= . > When the echo server runs as a socket application everything works. Seams > like OS (linux) decide not to route the packet to the application for > some reason. > > Some quick important notes. > > - DPDK layer starts ok and no EAL errors are reported. Its echo replies a= s > mentioned before can clearly be seen when capturing traffic on eth0 in > client docker. > - It is not the issue of parsing the echo replies on application level. I > checked the strace logs and cannot see any traffic delivered to the > application > - I have compared the captures from working socket based echo server and > MACs seam to be ok, also the packets sent from DPDK server are formed wel= l. > - No logs for martian packets when enabled. > > I would like to focus the attention of dear DPDK experts on the following > question: is it possible that DPDK AF_PACKET driver intercepts both packe= ts > destined to the virtual nic eth0 of Docker1 AND eth0 of Docker2, > preventing it from being received by application? > > DPDK Version 18 > > VIRTUAL BOX VM (Alma Linux) > > Docker1 Docker2 > +---------------+ +------------+ > | | Request | | > | | --------------> | DPDK | > | Socket Echo | | Echo Server > | Client | Reply | Server | > | | <------------- | | > --vdev=3Deth_af_packet0,iface=3Deth0,qdisc_bypass=3D1 > | | | | > | tcpdump -i eth0 +------+-----+ > +-------+-------+ eth0 | > | eth0 | > | No IP | > +------v--------------------------------------v-----+ > | Virtual Docker Bridge | > +---------------------------------------------------+ > > > Thanks > --000000000000e9052005ed05bd5d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
If the nic on docker 1 of echo client can receive rep= lies, the reason should not be PMD I think.
Did you make sure eve= ry field in every layer is completely the same between eth0 in docker 1 and= eth0 in docker 2, and all of them are expected?

Boris Ouretskey &= lt;borisusun@gmail.com> =E4= =BA=8E2022=E5=B9=B411=E6=9C=889=E6=97=A5=E5=91=A8=E4=B8=89 03:51=E5=86=99= =E9=81=93=EF=BC=9A


H= i

I am trying to run a docker based system in one VM, with some serv= ers using the DPDK and facing a very strange issue.

The relevant par= t of the system is the echo server which is built with DPDK library and run= s in docker, connected to the virtual bridge (with AF_PACKET driver), and e= cho client built with regular sockets (java application). See the drawing b= elow for visual details.

The strange thing is that echo replies sent= from DPDK echo server docker (Docker2), can be seen in tcpdump in the dock= er of echo client (Docker1) however the application (java client) does not = receive it on socket level. When the echo server runs as a socket applicati= on everything works. Seams like OS (linux) decide not to route the packet t= o the application for some=C2=A0reason.

Some quick important notes.=

- DPDK layer starts ok and no EAL errors are reported. Its echo=C2= =A0replies as mentioned before can clearly be seen when capturing traffic o= n eth0 in client docker.
- It is not th= e issue of parsing the echo replies on application level. I checked the str= ace logs and cannot see any traffic delivered to the application
- I hav= e compared the captures from working socket based echo server and MACs seam= to be ok, also the packets sent from DPDK server are formed well.
- No = logs for martian packets when enabled.

I would like to focus the att= ention of dear DPDK experts on the following question: is it possible that = DPDK AF_PACKET driver intercepts both packets destined to =C2=A0the virtual= nic eth0 of Docker1 AND eth0 of Docker2, preventing it from being received= by application?

DPDK Version 18

VIRTUAL BOX VM (Alma Linux)<= br>
=C2=A0 =C2=A0 Docker1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Docker2 =
+---------------+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 +------------+
| =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0Request =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 -------------->=C2=A0 =C2=A0= =C2=A0| =C2=A0DPDK =C2=A0 =C2=A0 =C2=A0|
| Socket Echo =C2=A0 | =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2= =A0Echo Server
| =C2=A0 Client =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Reply =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0Server =C2=A0 =C2=A0|<= br>| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 <--------= ----- =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| --vd= ev=3Deth_af_packet0,iface=3Deth0,qdisc_bypass=3D1
| =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|
| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tcpdump -i eth0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0+------+-----+
+-------+-------+ = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 eth0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 | eth0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 No IP =C2=A0|
=C2=A0+------v------------------------------= --------v-----+
=C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Virtua= l Docker Bridge =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0+---------------------------------------------------+
=C2=A0
Thanks
--000000000000e9052005ed05bd5d--