From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D20C9429E2 for ; Tue, 25 Apr 2023 02:43:17 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48A3340A7E; Tue, 25 Apr 2023 02:43:17 +0200 (CEST) Received: from mail-vk1-f172.google.com (mail-vk1-f172.google.com [209.85.221.172]) by mails.dpdk.org (Postfix) with ESMTP id 995D6400D7 for ; Tue, 25 Apr 2023 02:43:15 +0200 (CEST) Received: by mail-vk1-f172.google.com with SMTP id 71dfb90a1353d-4404c9d9fceso1809625e0c.2 for ; Mon, 24 Apr 2023 17:43:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=utexas-edu.20221208.gappssmtp.com; s=20221208; t=1682383395; x=1684975395; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=/KeHrAaSoL7td0k/1cEHXy6iNyFxiZFoTIm5ORrkMds=; b=v++EqW4KGI21X6OxeAbbtMNj4YZTh0QxJPUHrKH4QEFJMlo3u9bYdJZKtoL3L8v/jY A3dBlN5MbwRDsVwMFgSA0Yoe+/cKQySicOObNoTOnHt/D/qJhPe5ztxG+1FjXUTXteUL pm302EdfKfPK6jB/9kpn2sj0ajA/r0qk0cm3XTwdm8+X/XkoQjfHtg0P0qJTw99HWgAQ y/RbGv5Z4v3a9UZPOLf4Z62IQPHATZMosbf40AFoe+kgxnEI+OYDOnljFAdq6SEIclLD r1ia2klhXfFTJrrSaxi02v/Y5p4DtMfcqzDWIx976B5McjIOX5VJo22FkvKBqYtbF9Gw RzGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682383395; x=1684975395; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/KeHrAaSoL7td0k/1cEHXy6iNyFxiZFoTIm5ORrkMds=; b=SambIhtygc2ZvfNLPbxCJ6bxPHDGAZT3GLWDn2mjTgH/jhTlrIIA8beHuSTIwZTtZ0 5JjSVsDs3GWQCkjBD6pcx9xj1vWEUwB+iG7E1cdPpPWK40RaQSxTem5tSMVu5i8tqbJs 8pcrTsg7CwZVL9MOe45bxJf/kYDSF9v0EWR9Wo1SgL2AB5yQ1KLaJHtSwbKYMcnMtcOZ vtGhqPSMgDphrcrqw0rl7BF+6W8RyJsAdRX76hiiB6qz7Ho5yGvnj2WhKn7qPsvhQPrg 6qq6psUay7IayXR4/LZ6c+vzSQJaQcVPkpVRhXq4L+l3G9zm7EqlI+cRi9AXtrSFe0Df kvGQ== X-Gm-Message-State: AAQBX9csr9uAqnW3WV0VdMFpnH9Ngf3RABZSBQ06Nx1AORCOkjA9hopE jE2ussA8i0o6jYkysr6tW1KaV0mxG7Dnp6tOq+z7InBRpvUdxkwXR5oV2w== X-Google-Smtp-Source: AKy350aorkgrYK3hIOr+3PXbWbKbdVNBwA9DcnE4/T9Gn3AMgRoXoYqa0S4NZ1pxps04JbRP1PSQRcFpO1TGhk9i3j4= X-Received: by 2002:a1f:3d93:0:b0:43f:b968:b19b with SMTP id k141-20020a1f3d93000000b0043fb968b19bmr3875879vka.13.1682383394691; Mon, 24 Apr 2023 17:43:14 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Dinesh Kumar Date: Mon, 24 Apr 2023 20:43:03 -0400 Message-ID: Subject: Re: Use case understanding for l3fwd To: "J.J. Martzki" Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="0000000000009df82b05fa1e6838" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000009df82b05fa1e6838 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi JJ, Sorry for the late reply. Here are the answers to your questions. Still having the same issue. Nothing is reaching to DPDK application after ping . I tried to run l2fwd and found the same issue on VM2 after running l2fwd I have two Mac addresses for port 0 : FA:16:3E:EF:2C:26 and Port 1, MAC address: FA:16:3E:0D:39:EC then I added entry in to ARP table to associate IP address with these mac address on VM2 like : arp -s x.y.t.z FA:16:3E:EF:2C:26 dev ens3(ens3 is bind with kernel module) arp -s x.y.t.r FA:16:3E:0D:39:EC dev ens3 After it I tried to ping x.y.t.z assuming that this message will go to port 0 RX queue but not see any message in l2fwd . Here is a log snapshot: "iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 14 done: Port 0, MAC address: FA:16:3E:EF:2C:26 Initializing port 1... iavf_dev_init_vlan(): Failed to update vlan offload iavf_dev_configure(): configure VLAN failed: -95 iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 24 iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 23 iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 25 iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 26 iavf_dev_rx_queue_setup(): >> iavf_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port=3D1, queue=3D0. check_rx_vec_allow(): Vector Rx can be enabled on this rxq. iavf_dev_tx_queue_setup(): >> check_tx_vec_allow(): Vector tx can be enabled on this txq. iavf_dev_start(): >> iavf_set_rx_function(): request RXDID[22] in Queue[0] is not supported, set rx_pkt_burst as legacy for all queues iavf_set_rx_function(): Using avx2 Vector Rx (port 1). iavf_set_tx_function(): Using avx2 Vector Tx (port 1). iavf_configure_queues(): RXDID[22] is not supported, request default RXDID[1] in Queue[0] iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 6 iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 7 iavf_add_del_all_mac_addr(): add/rm mac:FA:16:3E:0D:39:EC iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 10 iavf_dev_tx_queue_start(): >> iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 8 iavf_handle_pf_event_msg(): VIRTCHNL_EVENT_LINK_CHANGE event iavf_dev_rx_queue_start(): >> iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 8 iavf_handle_pf_event_msg(): VIRTCHNL_EVENT_LINK_CHANGE event iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 14 iavf_dev_alarm_handler(): ICR01_ADMINQ is reported iavf_handle_virtchnl_msg(): adminq response is received, opcode =3D 14 done: Port 1, MAC address: FA:16:3E:0D:39:EC Skipping disabled port 2 Checking link statusdone Port 0 Link up at 25 Gbps FDX Autoneg Port 1 Link up at 25 Gbps FDX Autoneg L2FWD: entering main loop on lcore 2 L2FWD: -- lcoreid=3D2 portid=3D1 L2FWD: entering main loop on lcore 1 L2FWD: -- lcoreid=3D1 portid=3D0" and then log I added at RX queue to dump source and dest in Rx buffer: L2FWD: ITE one ----nb_rx=3D1 portid=3D0 L2FWD: ITR ---2----src=3D134610945 dest=3D134219268 I added this login /* main processing loop */ static void l2fwd_main_loop(void) { in /* Read packet from RX queues. 8< */ second loop for (i =3D 0; i < qconf->n_rx_port; i++) { portid =3D qconf->rx_port_list[i]; nb_rx =3D rte_eth_rx_burst(portid, 0, pkts_burst, MAX_PKT_BURST)= ; port_statistics[portid].rx +=3D nb_rx; for (j =3D 0; j < nb_rx; j++) { m =3D pkts_burst[j]; uint32_t ip_dst,ip_src; struct rte_ipv4_hdr *ip_hdr; rte_prefetch0(rte_pktmbuf_mtod(m, void *)); ip_hdr =3D rte_pktmbuf_mtod(m, struct rte_ipv4_hdr *); ip_dst =3D rte_be_to_cpu_32(ip_hdr->dst_add= r); ip_src =3D rte_be_to_cpu_32(ip_hdr->src_add= r); RTE_LOG(INFO, L2FWD, "ITE one ----nb_rx=3D%= u portid=3D%u\n", nb_rx,portid); RTE_LOG(INFO, L2FWD, "ITR ---2----src=3D%u dest=3D%u\n",ip_src,ip_dst); l2fwd_simple_forward(m, portid); } } Please let me know why the ping message is not reaching dpdk even though on the source side Thanks, Dinesh On Tue, Apr 18, 2023 at 3:51=E2=80=AFAM J.J. Martzki = wrote: > From your picture, l3fwd is running in VM2, with a tap vdev. But > what's the relationship between VM1 and VM2? Are they running on the > same host? > Yes, They are running on the same host with 3 NICs two are bind with dpdk and one is with kernel module, These NICs are on separate subnets > > I assume: > 1. VM1 and VM2 are running on the same host. > 2. ens05 is a net device in VM1. > 3. VM1 has its own net device, let's call it ens06, and IP2 is on it. > Yes: > 0000:00:05.0 'Ethernet Virtual Function 700 Series 154c' drv=3Duio_pci_generic unused=3Diavf,vfio-pci 0000:00:06.0 'Ethernet Virtual Function 700 Series 154c' drv=3Duio_pci_generic unused=3Diavf,vfio-pci Network devices using kernel driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:00:03.0 'Virtio network device 1000' if=3Dens3 drv=3Dvirtio-pci unused=3Dvfio-pci,uio_pci_generic *Active* > Based on the assumptions above, you should make sure VM1 and VM2 can > communicate with each other without l3fwd, just through the linux > network stack. > Yes . I tested with ens05 and ens03 and ping is working fine without dpdk. > This ensures the link between these 2 VMs is good (ens05 in VM2 and > ens06 in VM1 can reach each other directly). Maybe they are connected > by a linux bridge on the host or something like that. > > And you l3fwd should attach to ens05, by vfio or uio. Just use > dpdk-devbind.py to do this. > After this, ens05 is no longer available in the kernel, and l3fwd can > use it. As your command line '-p 0x3' described, l3fwd will use port 0 > and port 1. > You can use l3fwd init log to make sure there are 2 ports available > for l3fwd as we expected. > > BTW, I suggest you start from test-pmd first. test-pmd can connect > ports ethier. But it supports interactive mode and many command lines. > Such as display of port info like MAC you mentioned before. > > > Please share some link that has details descriptions about testing this > flow : > I didn't find a suitable article to describe this, maybe you can try: > > https://www.redhat.com/en/blog/how-vhost-user-came-being-virtio-networkin= g-and-dpdk > In this article, the DPDK application is ovs, just like your l3fwd. > And it can attach many ports not only vhost ports but also physical > ports, vdev ports like tap and so on. > > J.J. Martzki, > Best wishes. > > > Dinesh Kumar =E4=BA=8E2023=E5=B9=B44=E6=9C=8817=E6= =97=A5=E5=91=A8=E4=B8=80 23:47=E5=86=99=E9=81=93=EF=BC=9A > > > > Hi JJ, > > Thank you for the clarification. I am new to this so trying to > understand how it will work. > > > > Please share some link that has details descriptions about testing this > flow : > > > > | VM1 | HOST > > | VM2 | > > app <- device -> qemu <- pmd -> l3fwd <- pmd -> qemu <- device -> app > > > > Sorry for my misunderstanding. > > > > here is the correct flow I am working on now : > > After running the below command from VM2, I am pinging IP address of > VM1 vi network namespace ns0, but not able to see any message in l3fwd an= d > ping is unreachable. > > Thanks for your time and any help will be really appreciated. > > > > > > > > On Fri, Apr 14, 2023 at 10:46=E2=80=AFPM J.J. Martzki > wrote: > >> > >> > Is this process correct to assign IP and then device to DPDK? > >> No, after binding the driver to vfio/uio, the IP configuration is no > >> longer available. > >> The IP is something belonging to an application, not a driver (DPDK). > >> > >> > Is there any way to see ip and ethernet address after binding to DPD= K? > >> MAC is available for the rte_eth_dev, there's some API like > >> rte_eth_macaddr_get() to get MAC > >> from a specific port. l3fwd should have some CLI commands for getting > >> MAC ethier, maybe > >> you can read it's documents. > >> IP is not available as I said before. > >> > >> BTW, your topo is confusing. Are you running 2 l3fwd in 2 separate > >> VMs? It's not gonna work. > >> AFAK, l3fwd switches packets from ports which attached to it. You > >> can't use 2 separate l3fwd > >> to connect 2 VMs. > >> > >> If you are tring to connect 2 VMs, the classic topo is something like: > >> | VM1 | HOST > >> | VM2 | > >> app <- device -> qemu <- pmd -> l3fwd <- pmd -> qemu <- device -> app > >> > >> J.J. Martzki, > >> Best wishes. > >> > >> Dinesh Kumar =E4=BA=8E2023=E5=B9=B44=E6=9C=8814= =E6=97=A5=E5=91=A8=E4=BA=94 01:32=E5=86=99=E9=81=93=EF=BC=9A > >> > > >> > Hello Experts, > >> > > >> > I am trying to run the following scenario using l3fwd example in > dpdk-stable-21.11.2 release . > >> > > >> > The flow is : > >> > > >> > VM1 > VM2 > >> > ns0->tap0->DPDK <---------------------> DPDK-> bind = a > device with IP > >> > > >> > > >> > The first question I have for VM2: > >> > I assigned IP to the device and can see eth and static ip address > via 'ip add ', then after binding it to dpdk, I can not see this > information via ip add because it is not controlled by kernel module. > >> > Is there any way to see ip and ethernet address after binding to DPD= K? > >> > Is this process correct to assign IP and then device to DPDK? > >> > > >> > > >> > on VM1: > >> > After starting the DPDK application, I can see a tap0 device create= d > in kernel mode via 'ip add show'.Then I attached this interface with > network namespace via: > >> > > >> > ip netns add ns0 > >> > ip link set tap0 netns ns0 > >> > ip netns exec ns0 ip addr add dev tap0 > >> > ip netns exec ns0 ip link set tap0 up > >> > ip netns exec ns0 ip link set lo up > >> > > >> > Still, I am not able to ping anything from this ns0. > >> > > >> > Do I need any other configuration to ping via ns0 to VM2? > >> > > >> > Sorry, in case I am asking any dumb questions, I am new to DPDK and > trying to make the use case work. > >> > Any help/pointer is really appreciated. > >> > > >> > > --0000000000009df82b05fa1e6838 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi JJ,
Sorry for the late reply.
Here are the answers to your questions.
Still having the same issu= e.
Nothing is reaching to DPDK application after ping .
I trie= d to run l2fwd and found the same issue
on VM2 after running l2fwd I hav= e two Mac addresses for port 0 :=C2=A0FA:16:3E:EF:2C:26=C2=A0 and=C2=A0 Por= t 1, MAC address: FA:16:3E:0D:39:EC

then I added entry in to ARP tab= le to associate=C2=A0IP address with these mac address on VM2 like :
<= div>arp -s x.y.t.z FA:16:3E:EF:2C:26 dev ens3(ens3 is bind with=C2=A0 kerne= l module)
arp -s=C2=A0 x.y.t.r=C2=A0=C2=A0FA:16:3E:0D:39:EC dev ens3
After it I tried to ping= =C2=A0 =C2=A0x.y.t.z assuming that this message=C2=A0will go to port 0 RX queue bu= t not see any message=C2=A0in l2fwd .

Here is a lo= g snapshot:
"iavf_handle_virtchnl_msg(): adminq response is receive= d, opcode =3D 14
done:
Port 0, MAC address: FA:16:3E:EF:2C:26

= Initializing port 1... iavf_dev_init_vlan(): Failed to update vlan offload<= br>iavf_dev_configure(): configure VLAN failed: -95
iavf_dev_alarm_handl= er(): ICR01_ADMINQ is reported
iavf_handle_virtchnl_msg(): adminq respon= se is received, opcode =3D 24
iavf_dev_alarm_handler(): ICR01_ADMINQ is = reported
iavf_handle_virtchnl_msg(): adminq response is received, opcode= =3D 23
iavf_dev_alarm_handler(): ICR01_ADMINQ is reported
iavf_handl= e_virtchnl_msg(): adminq response is received, opcode =3D 25
iavf_dev_al= arm_handler(): ICR01_ADMINQ is reported
iavf_handle_virtchnl_msg(): admi= nq response is received, opcode =3D 26
iavf_dev_rx_queue_setup(): =C2=A0= >>
iavf_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions ar= e satisfied. Rx Burst Bulk Alloc function will be used on port=3D1, queue= =3D0.
check_rx_vec_allow(): Vector Rx can be enabled on this rxq.
iav= f_dev_tx_queue_setup(): =C2=A0>>
check_tx_vec_allow(): Vector tx c= an be enabled on this txq.
iavf_dev_start(): =C2=A0>>
iavf_set_= rx_function(): request RXDID[22] in Queue[0] is not supported, set rx_pkt_b= urst as legacy for all queues
iavf_set_rx_function(): Using avx2 Vector = Rx (port 1).
iavf_set_tx_function(): Using avx2 Vector Tx (port 1).
i= avf_configure_queues(): RXDID[22] is not supported, request default RXDID[1= ] in Queue[0]
iavf_dev_alarm_handler(): ICR01_ADMINQ is reported
iavf= _handle_virtchnl_msg(): adminq response is received, opcode =3D 6
iavf_d= ev_alarm_handler(): ICR01_ADMINQ is reported
iavf_handle_virtchnl_msg():= adminq response is received, opcode =3D 7
iavf_add_del_all_mac_addr(): = add/rm mac:FA:16:3E:0D:39:EC
iavf_dev_alarm_handler(): ICR01_ADMINQ is r= eported
iavf_handle_virtchnl_msg(): adminq response is received, opcode = =3D 10
iavf_dev_tx_queue_start(): =C2=A0>>
iavf_dev_alarm_handl= er(): ICR01_ADMINQ is reported
iavf_handle_virtchnl_msg(): adminq respon= se is received, opcode =3D 8
iavf_handle_pf_event_msg(): VIRTCHNL_EVENT_= LINK_CHANGE event
iavf_dev_rx_queue_start(): =C2=A0>>
iavf_dev_= alarm_handler(): ICR01_ADMINQ is reported
iavf_handle_virtchnl_msg(): ad= minq response is received, opcode =3D 8
iavf_handle_pf_event_msg(): VIRT= CHNL_EVENT_LINK_CHANGE event
iavf_dev_alarm_handler(): ICR01_ADMINQ is r= eported
iavf_handle_virtchnl_msg(): adminq response is received, opcode = =3D 14
iavf_dev_alarm_handler(): ICR01_ADMINQ is reported
iavf_handle= _virtchnl_msg(): adminq response is received, opcode =3D 14
done:
Por= t 1, MAC address: FA:16:3E:0D:39:EC

Skipping disabled port 2

= Checking link statusdone
Port 0 Link up at 25 Gbps FDX Autoneg
Port 1= Link up at 25 Gbps FDX Autoneg
L2FWD: entering main loop on lcore 2
= L2FWD: =C2=A0-- lcoreid=3D2 portid=3D1
L2FWD: entering main loop on lcor= e 1
L2FWD: =C2=A0-- lcoreid=3D1 portid=3D0"=C2=A0

and then l= og I added at RX queue to dump source and dest in Rx buffer:
L2FWD: ITE = one ----nb_rx=3D1 portid=3D0
L2FWD: ITR ---2----src=3D134610945 dest=3D1= 34219268
I added this login=C2=A0=C2=A0
/* main processing loop */
static = void
l2fwd_main_loo= p(void)
{

=C2=A0in=C2=A0
/* Read packet from RX queues. 8< */<= /span>=C2=A0 second loop=C2=A0

=C2=A0for (i =3D 0; i &l= t; qconf->n_rx_port; i++) {
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 portid =3D qconf->rx_port_lis= t[i];
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 nb_rx =3D rte_eth_rx_burst(portid, 0,
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0pkts_burst, MAX_PKT_BURST);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 port_statistics[portid= ].rx +=3D nb_rx;

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 for (j =3D 0; j < nb_rx; j++) {
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 m =3D pkts_burst[j];
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 uint32_t ip_dst,ip_src;
=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 struct rte_ipv4_hdr *ip_hdr;
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 rte_prefetch0(rte_pktmbuf_mtod(m, void *));
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 ip_hdr =3D rte_pktmbuf_mtod(m, struct rte_ipv4_hdr= *);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ip_dst =3D rte_be_to_cpu_32(i= p_hdr->dst_addr);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ip_src =3D r= te_be_to_cpu_32(ip_hdr->src_addr);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 RTE_LOG(INFO, L2FWD, "ITE one ----nb_rx=3D%u portid=3D%u\n", = nb_rx,portid);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 RTE_LOG(INFO, L2FWD= , "ITR ---2----src=3D%u dest=3D%u\n",ip_src,ip_dst);
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 l2fwd_simple_forward(m, portid);
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 }
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }
Please let me know why the ping message is not reaching dpdk even= though on the source side

Thanks,
Dines= h=C2=A0

On Tue, Apr 18, 2023 at 3:51=E2=80=AFAM J.J. Martzki <mars14850@gmail.com> wrote:
From your picture, l3fw= d is running in VM2, with a tap vdev. But
what's the relationship between VM1 and VM2? Are they running on the same host?
Yes, They are runni= ng on the same host with=C2=A0 3 NICs two are bind with dpdk and one is wit= h kernel=C2=A0module, These NICs are on separate subnets=C2=A0
=

I assume:
1. VM1 and VM2 are running on the same host.
2. ens05 is a net device in VM1.
3. VM1 has its own net device, let's call it ens06, and IP2 is on it. Yes:
0000:00:05.0 'Ether= net Virtual Function 700 Series 154c' drv=3Duio_pci_generic unused=3Dia= vf,vfio-pci
0000:00:06.0 'Ethernet Virtual Function 700 Series 154c&= #39; drv=3Duio_pci_generic unused=3Diavf,vfio-pci

Network devices us= ing kernel driver
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
0000:00:03.0 'Virtio network device 1000' if=3Dens3= drv=3Dvirtio-pci unused=3Dvfio-pci,uio_pci_generic *Active*
=C2=A0
Based on the assumptions above, you should make sure VM1 and VM2 can
communicate with each other without l3fwd, just through the linux
network stack.
Yes=C2=A0 . I tested with ens05 and ens= 03 and ping is working fine without dpdk.
This ensures the link between these 2 VMs is good (ens05 in VM2=C2=A0 and ens06 in VM1 can reach each other directly). Maybe they are connected
by a linux bridge on the host or something like that.

And you l3fwd should attach to ens05, by vfio or uio. Just use
dpdk-devbind.py to do this.
After this, ens05 is no longer available in the kernel, and l3fwd can
use it. As your command line '-p 0x3' described, l3fwd will use por= t 0
and port 1.
You can use l3fwd init log to make sure there are 2 ports available
for l3fwd as we expected.

BTW, I suggest you start from test-pmd first. test-pmd can connect
ports ethier. But it supports interactive mode and many command lines.
Such as display of port info like MAC you mentioned before.

> Please share some link that has details descriptions about testing thi= s flow :
I didn't find a suitable article to describe this, maybe you can try: https://www.redha= t.com/en/blog/how-vhost-user-came-being-virtio-networking-and-dpdk
In this article, the DPDK application is ovs, just like your l3fwd.
And it can attach many ports not only vhost ports but also physical
ports, vdev ports like tap and so on.

J.J. Martzki,
Best wishes.


Dinesh Kumar <= raidinesh@utexas.edu> =E4=BA=8E2023=E5=B9=B44=E6=9C=8817=E6=97=A5=E5= =91=A8=E4=B8=80 23:47=E5=86=99=E9=81=93=EF=BC=9A
>
> Hi JJ,
> Thank you for the clarification. I am new to this so trying to underst= and how it will work.
>
> Please share some link that has details descriptions about testing thi= s flow :
>
> |=C2=A0 =C2=A0 =C2=A0 =C2=A0VM1=C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 HOST
>=C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 VM2=C2=A0 =C2=A0 =C2= =A0 =C2=A0|
> app <- device -> qemu <- pmd -> l3fwd <- pmd -> qemu= <- device -> app
>
> Sorry for my misunderstanding.
>
> here is the correct flow I am working on now=C2=A0 :
> After running the below command from VM2,=C2=A0 I am pinging IP addres= s of VM1 vi network namespace ns0, but not able to see any message in l3fwd= and ping is unreachable.
> Thanks for your time and any help will be really appreciated.
>
>
>
> On Fri, Apr 14, 2023 at 10:46=E2=80=AFPM J.J. Martzki <mars14850@gmail.com> wr= ote:
>>
>> > Is this process correct to assign IP=C2=A0 and then device to= DPDK?
>> No, after binding the driver to vfio/uio, the IP configuration is = no
>> longer available.
>> The IP is something belonging to an application, not a driver (DPD= K).
>>
>> > Is there any way to see ip and ethernet address after binding= to DPDK?
>> MAC is available for the rte_eth_dev, there's some API like >> rte_eth_macaddr_get() to get MAC
>> from a specific port. l3fwd should have some CLI commands for gett= ing
>> MAC ethier, maybe
>> you can read it's documents.
>> IP is not available as I said before.
>>
>> BTW, your topo is confusing. Are you running 2 l3fwd in 2 separate=
>> VMs? It's not gonna work.
>> AFAK, l3fwd switches packets from ports which attached to it. You<= br> >> can't use 2 separate l3fwd
>> to connect 2 VMs.
>>
>> If you are tring to connect 2 VMs, the classic topo is something l= ike:
>> |=C2=A0 =C2=A0 =C2=A0 =C2=A0VM1=C2=A0 =C2=A0 =C2=A0 =C2=A0|=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 HOST
>>=C2=A0 =C2=A0 =C2=A0 |=C2=A0 =C2=A0 =C2=A0 =C2=A0 VM2=C2=A0 =C2=A0 = =C2=A0 =C2=A0|
>> app <- device -> qemu <- pmd -> l3fwd <- pmd -> = qemu <- device -> app
>>
>> J.J. Martzki,
>> Best wishes.
>>
>> Dinesh Kumar <raidinesh@utexas.edu> =E4=BA=8E2023=E5=B9=B44=E6=9C=8814=E6= =97=A5=E5=91=A8=E4=BA=94 01:32=E5=86=99=E9=81=93=EF=BC=9A
>> >
>> > Hello Experts,
>> >
>> > I am trying to run the following scenario using l3fwd example= in dpdk-stable-21.11.2 release .
>> >
>> > The flow is :
>> >
>> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 VM1=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0VM2
>> > ns0->tap0<IPV4 IP>->DPDK=C2=A0 =C2=A0 =C2=A0 =C2= =A0<--------------------->=C2=A0 DPDK-> bind a device=C2=A0 with I= P <IPV4 IP >
>> >
>> >
>> > The first question I have for VM2:
>> > I assigned IP to the device and can see eth and static ip add= ress via=C2=A0 'ip add ', then after binding it to dpdk, I can not = see this information via ip add because it is not controlled by kernel modu= le.
>> > Is there any way to see ip and ethernet address after binding= to DPDK?
>> > Is this process correct to assign IP=C2=A0 and then device to= DPDK?
>> >
>> >
>> > on VM1:
>> >=C2=A0 After starting the DPDK application, I can see a tap0 d= evice created in kernel mode via 'ip add show'.Then I attached this= interface with network namespace via:
>> >
>> > ip netns add ns0
>> > ip link set tap0 netns ns0
>> > ip netns exec ns0 ip addr add <IPV4 Address> dev tap0 >> > ip netns exec ns0 ip link set tap0 up
>> > ip netns exec ns0 ip link set lo up
>> >
>> > Still, I am not able to ping anything from this ns0.
>> >
>> > Do I need any other configuration to ping via ns0 to VM2?
>> >
>> > Sorry, in case I am asking any dumb questions, I am new to DP= DK and trying to make the use case work.
>> > Any help/pointer is really appreciated.
>> >
>> >
--0000000000009df82b05fa1e6838--