DPDK usage discussions
 help / color / Atom feed
* [dpdk-users] l2fwd application are not sending continuous packets .
@ 2020-01-04  7:24 Rohit kamble
       [not found] ` <CAMJ3rejbOZ5mzOYyNPFqVk-Scs7CTaVZLUQOd-GMnVpaYf+pmg@mail.gmail.com>
  0 siblings, 1 reply; 6+ messages in thread
From: Rohit kamble @ 2020-01-04  7:24 UTC (permalink / raw)
  To: users

Hello Dpdk Team,

I'm facing issue while forwarding packets in DPDK's l2fwd application.
While sending 1 Lac packets from Scapy, I could see sometimes packets are
sending from one VM to another VM.
Before explaining issue let me explain topology.



*Topology :-=======*1) I am having 4 VMs(Virtual Machines) in same host.
All these VM are running on Ubuntu 16.04.1.
2) VM-1 is used as Scapy to forward packets (Scapy version 2.4.3) . While
creating packets I am giving destination mac(d-mac) address of VM-1.
3) In VM-2 am running L2 forwarding application.
    In this l2fwd application, I am doing simple packet forwarding by
statically keeping mac address of VM-3.
Code :-
     l2fwd_mac_updating () {
   ...
   ......
   .......
      *((uint64_t *)tmp) = 0xddccbbaa/*VM-3 mac address*/ +
((uint64_t)dest_portid << 40);
     }
4) Also in VM-3, I am doing same like VM-2, but I kept mac address of VM-4
5) In VM-4, I am using wireshark to see packets coming from VM-3.
6) In VM-2 and VM-3, I kept promiscuous mode off by commenting out
rte_eth_promiscuous_enable().

   ---------------                                   ---------------------
                         ------------------
------------------------
   |    VM-1  |      ---------->                |       VM-2     |
---------------->   |     VM-3    |       ------------------> |        VM-4
     |
   ---------------                                   ---------------------
                         ------------------
------------------------
        Scapy                              Simple L2 forwarding
      Simple L2 forwarding                        Wireshark
used for sending packets            Sending all packets to
 Sending all packets to
                                                            VM-3
                        VM-4
                                                          (DPDK)
                     (DPDK)


*Problem :- ======*==
From scapy VM, I am sending 1 lac packets with rate of 100 packets per
second. *During problematic condition I could see packets are not getting
forward from VM-3.**Problematic state is happening anytime after sending 1k
packets.*
This issue is not consistence but I could see this issue 8 out of 10 times.


* Debugging and Observations :-=========================*
When I tried to debug this issue , I could see that
1) In problematic state, rx queue of VM-3 is not getting packets, but VM-2
is sending packets properly. I checked this by using pdump of rx queue.
2) Just before problematic state in VM-3, I could see that  previous
packet(only one packet) instead of going to VM-4, it is coming back to
again in rx queue of VM-3 and after wards I did not get any packets in rx
queue.
3) I have changed rate of packet forward 10packets per second. but still
see the issue.


*Can anyone please help to solve this problem ? I need it urgently .*
Thanks,
Rohit

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-users] Fwd: l2fwd application are not sending continuous packets .
       [not found] ` <CAMJ3rejbOZ5mzOYyNPFqVk-Scs7CTaVZLUQOd-GMnVpaYf+pmg@mail.gmail.com>
@ 2020-01-06  5:55   ` " satyavalli rama
  2020-01-06  7:07     ` [dpdk-users] " satyavalli rama
  2020-01-09  1:55     ` satyavalli rama
  0 siblings, 2 replies; 6+ messages in thread
From: satyavalli rama @ 2020-01-06  5:55 UTC (permalink / raw)
  To: dev, users

Hello Dpdk Team,

I'm facing issue while forwarding packets in DPDK's l2fwd application.
While sending 1 Lac packets from Scapy, I could see sometimes packets are
sending from one VM to another VM.
Before explaining issue let me explain topology.



*Topology :-=======*1) I am having 4 VMs(Virtual Machines) in same host.
All these VM are running on Ubuntu 16.04.1.
2) VM-1 is used as Scapy to forward packets (Scapy version 2.4.3) . While
creating packets I am giving destination mac(d-mac) address of VM-1.
3) In VM-2 am running L2 forwarding application.
    In this l2fwd application, I am doing simple packet forwarding by
statically keeping mac address of VM-3.
Code :-
     l2fwd_mac_updating () {
   ...
   ......
   .......
      *((uint64_t *)tmp) = 0xddccbbaa/*VM-3 mac address*/ +
((uint64_t)dest_portid << 40);
     }
4) Also in VM-3, I am doing same like VM-2, but I kept mac address of VM-4
5) In VM-4, I am using wireshark to see packets coming from VM-3.
6) In VM-2 and VM-3, I kept promiscuous mode off by commenting out
rte_eth_promiscuous_enable().

   ---------------                                   ---------------------
                         ------------------
------------------------
   |    VM-1  |      ---------->                |       VM-2     |
---------------->   |     VM-3    |       ------------------> |        VM-4
     |
   ---------------                                   ---------------------
                         ------------------
------------------------
        Scapy                              Simple L2 forwarding
      Simple L2 forwarding                        Wireshark
used for sending packets            Sending all packets to
 Sending all packets to
                                                            VM-3
                        VM-4
                                                          (DPDK)
                     (DPDK)


*Problem :- ======*==
From scapy VM, I am sending 1 lac packets with rate of 100 packets per
second. *During problematic condition I could see packets are not getting
forward from VM-3.**Problematic state is happening anytime after sending 1k
packets.*
This issue is not consistence but I could see this issue 8 out of 10 times.


* Debugging and Observations :-=========================*
When I tried to debug this issue , I could see that
1) In problematic state, rx queue of VM-3 is not getting packets, but VM-2
is sending packets properly. I checked this by using pdump of rx queue.
2) Just before problematic state in VM-3, I could see that  previous
packet(only one packet) instead of going to VM-4, it is coming back to
again in rx queue of VM-3 and after wards I did not get any packets in rx
queue.
3) I have changed rate of packet forward 10packets per second. but still
see the issue.


*Can anyone please help to solve this problem ? I need it urgently .*
Thanks,
Satya

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [dpdk-users] l2fwd application are not sending continuous packets .
  2020-01-06  5:55   ` [dpdk-users] Fwd: " satyavalli rama
@ 2020-01-06  7:07     ` " satyavalli rama
  2020-01-09  1:55     ` satyavalli rama
  1 sibling, 0 replies; 6+ messages in thread
From: satyavalli rama @ 2020-01-06  7:07 UTC (permalink / raw)
  To: dev, stable, users

Hello Dpdk Team,

I'm facing issue while forwarding packets in DPDK's l2fwd application.
While sending 1 Lac packets from Scapy, I could see sometimes packets are
sending from one VM to another VM.
Before explaining issue let me explain topology.



*Topology :-=======*1) I am having 4 VMs(Virtual Machines) in same host.
All these VM are running on Ubuntu 16.04.1.
2) VM-1 is used as Scapy to forward packets (Scapy version 2.4.3) . While
creating packets I am giving destination mac(d-mac) address of VM-1.
3) In VM-2 am running L2 forwarding application.
    In this l2fwd application, I am doing simple packet forwarding by
statically keeping mac address of VM-3.
Code :-
     l2fwd_mac_updating () {
   ...
   ......
   .......
      *((uint64_t *)tmp) = 0xddccbbaa/*VM-3 mac address*/ +
((uint64_t)dest_portid << 40);
     }
4) Also in VM-3, I am doing same like VM-2, but I kept mac address of VM-4
5) In VM-4, I am using wireshark to see packets coming from VM-3.
6) In VM-2 and VM-3, I kept promiscuous mode off by commenting out
rte_eth_promiscuous_enable().

   ---------------                                   ---------------------
                         ------------------
------------------------
   |    VM-1  |      ---------->                |       VM-2     |
---------------->   |     VM-3    |       ------------------> |        VM-4
     |
   ---------------                                   ---------------------
                         ------------------
------------------------
        Scapy                              Simple L2 forwarding
      Simple L2 forwarding                        Wireshark
used for sending packets            Sending all packets to
 Sending all packets to
                                                            VM-3
                        VM-4
                                                          (DPDK)
                     (DPDK)


*Problem :- ======*==
From scapy VM, I am sending 1 lac packets with rate of 100 packets per
second. *During problematic condition I could see packets are not getting
forward from VM-3.**Problematic state is happening anytime after sending 1k
packets.*
This issue is not consistence but I could see this issue 8 out of 10 times.


* Debugging and Observations :-=========================*
When I tried to debug this issue , I could see that
1) In problematic state, rx queue of VM-3 is not getting packets, but VM-2
is sending packets properly. I checked this by using pdump of rx queue.
2) Just before problematic state in VM-3, I could see that  previous
packet(only one packet) instead of going to VM-4, it is coming back to
again in rx queue of VM-3 and after wards I did not get any packets in rx
queue.
3) I have changed rate of packet forward 10packets per second. but still
see the issue.


*Can anyone please help to solve this problem ? I need it urgently .*
Thanks,
Satya

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-users] l2fwd application are not sending continuous packets .
  2020-01-06  5:55   ` [dpdk-users] Fwd: " satyavalli rama
  2020-01-06  7:07     ` [dpdk-users] " satyavalli rama
@ 2020-01-09  1:55     ` satyavalli rama
  2020-01-09  6:12       ` [dpdk-users] [dpdk-dev] " Stephen Hemminger
  1 sibling, 1 reply; 6+ messages in thread
From: satyavalli rama @ 2020-01-09  1:55 UTC (permalink / raw)
  To: dev, users

Can anyone please help us with this?


On Mon, Jan 6, 2020, 11:25 satyavalli rama <satyavalli.rama@gmail.com>
wrote:

>
>
>
> Hello Dpdk Team,
>
> I'm facing issue while forwarding packets in DPDK's l2fwd application.
> While sending 1 Lac packets from Scapy, I could see sometimes packets are
> sending from one VM to another VM.
> Before explaining issue let me explain topology.
>
>
>
> *Topology :-=======*1) I am having 4 VMs(Virtual Machines) in same host.
> All these VM are running on Ubuntu 16.04.1.
> 2) VM-1 is used as Scapy to forward packets (Scapy version 2.4.3) . While
> creating packets I am giving destination mac(d-mac) address of VM-1.
> 3) In VM-2 am running L2 forwarding application.
>     In this l2fwd application, I am doing simple packet forwarding by
> statically keeping mac address of VM-3.
> Code :-
>      l2fwd_mac_updating () {
>    ...
>    ......
>    .......
>       *((uint64_t *)tmp) = 0xddccbbaa/*VM-3 mac address*/ +
> ((uint64_t)dest_portid << 40);
>      }
> 4) Also in VM-3, I am doing same like VM-2, but I kept mac address of VM-4
> 5) In VM-4, I am using wireshark to see packets coming from VM-3.
> 6) In VM-2 and VM-3, I kept promiscuous mode off by commenting out
> rte_eth_promiscuous_enable().
>
>    ---------------                                   ---------------------
>                          ------------------
> ------------------------
>    |    VM-1  |      ---------->                |       VM-2     |
> ---------------->   |     VM-3    |       ------------------> |        VM-4
>      |
>    ---------------                                   ---------------------
>                          ------------------
> ------------------------
>         Scapy                              Simple L2 forwarding
>         Simple L2 forwarding                        Wireshark
> used for sending packets            Sending all packets to
>  Sending all packets to
>                                                             VM-3
>                         VM-4
>                                                           (DPDK)
>                      (DPDK)
>
>
> *Problem :- ======*==
> From scapy VM, I am sending 1 lac packets with rate of 100 packets per
> second. *During problematic condition I could see packets are not getting
> forward from VM-3.**Problematic state is happening anytime after sending
> 1k packets.*
> This issue is not consistence but I could see this issue 8 out of 10 times.
>
>
> * Debugging and Observations :-=========================*
> When I tried to debug this issue , I could see that
> 1) In problematic state, rx queue of VM-3 is not getting packets, but VM-2
> is sending packets properly. I checked this by using pdump of rx queue.
> 2) Just before problematic state in VM-3, I could see that  previous
> packet(only one packet) instead of going to VM-4, it is coming back to
> again in rx queue of VM-3 and after wards I did not get any packets in rx
> queue.
> 3) I have changed rate of packet forward 10packets per second. but still
> see the issue.
>
>
> *Can anyone please help to solve this problem ? I need it urgently .*
> Thanks,
> Satya
>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-users] [dpdk-dev] l2fwd application are not sending continuous packets .
  2020-01-09  1:55     ` satyavalli rama
@ 2020-01-09  6:12       ` " Stephen Hemminger
  2020-01-09 10:34         ` satyavalli rama
  0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2020-01-09  6:12 UTC (permalink / raw)
  To: satyavalli rama; +Cc: dev, users

On Thu, 9 Jan 2020 07:25:45 +0530
satyavalli rama <satyavalli.rama@gmail.com> wrote:

> Can anyone please help us with this?
> 
> 
> On Mon, Jan 6, 2020, 11:25 satyavalli rama <satyavalli.rama@gmail.com>
> wrote:
> 
> >
> >
> >
> > Hello Dpdk Team,
> >
> > I'm facing issue while forwarding packets in DPDK's l2fwd application.
> > While sending 1 Lac packets from Scapy, I could see sometimes packets are
> > sending from one VM to another VM.
> > Before explaining issue let me explain topology.
> >
> >
> >
> > *Topology :-=======*1) I am having 4 VMs(Virtual Machines) in same host.
> > All these VM are running on Ubuntu 16.04.1.
> > 2) VM-1 is used as Scapy to forward packets (Scapy version 2.4.3) . While
> > creating packets I am giving destination mac(d-mac) address of VM-1.
> > 3) In VM-2 am running L2 forwarding application.
> >     In this l2fwd application, I am doing simple packet forwarding by
> > statically keeping mac address of VM-3.
> > Code :-
> >      l2fwd_mac_updating () {
> >    ...
> >    ......
> >    .......
> >       *((uint64_t *)tmp) = 0xddccbbaa/*VM-3 mac address*/ +
> > ((uint64_t)dest_portid << 40);
> >      }
> > 4) Also in VM-3, I am doing same like VM-2, but I kept mac address of VM-4
> > 5) In VM-4, I am using wireshark to see packets coming from VM-3.
> > 6) In VM-2 and VM-3, I kept promiscuous mode off by commenting out
> > rte_eth_promiscuous_enable().
> >
> >    ---------------                                   ---------------------
> >                          ------------------
> > ------------------------
> >    |    VM-1  |      ---------->                |       VM-2     |  
> > ---------------->   |     VM-3    |       ------------------> |        VM-4  
> >      |
> >    ---------------                                   ---------------------
> >                          ------------------
> > ------------------------
> >         Scapy                              Simple L2 forwarding
> >         Simple L2 forwarding                        Wireshark
> > used for sending packets            Sending all packets to
> >  Sending all packets to
> >                                                             VM-3
> >                         VM-4
> >                                                           (DPDK)
> >                      (DPDK)
> >
> >
> > *Problem :- ======*==
> > From scapy VM, I am sending 1 lac packets with rate of 100 packets per
> > second. *During problematic condition I could see packets are not getting
> > forward from VM-3.**Problematic state is happening anytime after sending
> > 1k packets.*
> > This issue is not consistence but I could see this issue 8 out of 10 times.
> >
> >
> > * Debugging and Observations :-=========================*
> > When I tried to debug this issue , I could see that
> > 1) In problematic state, rx queue of VM-3 is not getting packets, but VM-2
> > is sending packets properly. I checked this by using pdump of rx queue.
> > 2) Just before problematic state in VM-3, I could see that  previous
> > packet(only one packet) instead of going to VM-4, it is coming back to
> > again in rx queue of VM-3 and after wards I did not get any packets in rx
> > queue.
> > 3) I have changed rate of packet forward 10packets per second. but still
> > see the issue.
> >
> >
> > *Can anyone please help to solve this problem ? I need it urgently .*
> > Thanks,
> > Satya
> >  

Sorry, you need to dig inside the forwarding application and instrument
what is coming in and how packets are being processed. You usually can't
treat DPDK as a black box.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-users] [dpdk-dev] l2fwd application are not sending continuous packets .
  2020-01-09  6:12       ` [dpdk-users] [dpdk-dev] " Stephen Hemminger
@ 2020-01-09 10:34         ` satyavalli rama
  0 siblings, 0 replies; 6+ messages in thread
From: satyavalli rama @ 2020-01-09 10:34 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev, users

Thanks Stephen,
We've tried to debug issue in couple of ways, please correct us whether
these approaches or right not wrong?
1. We have increased the MAX_PKT_BURST from 32 to 64 and
MEMEPOOL_CAHCE_SIZE 256 to 512
2. Keeping IGB_UIO as the only driver module.
3. Resetting the RX queues.
But still we are seeing the issue, as mentioned above========
> > * Debugging and Observations :-=========================*
> > When I tried to debug this issue , I could see that
> > 1) In problematic state, rx queue of VM-3 is not getting packets, but
VM-2
> > is sending packets properly. I checked this by using pdump of rx queue.
> > 2) Just before problematic state in VM-3, I could see that  previous
> > packet(only one packet) instead of going to VM-4, it is coming back to
> > again in rx queue of VM-3 and after wards I did not get any packets in
rx
> > queue.


On Thu, Jan 9, 2020, 11:42 Stephen Hemminger <stephen@networkplumber.org>
wrote:

> On Thu, 9 Jan 2020 07:25:45 +0530
> satyavalli rama <satyavalli.rama@gmail.com> wrote:
>
> > Can anyone please help us with this?
> >
> >
> > On Mon, Jan 6, 2020, 11:25 satyavalli rama <satyavalli.rama@gmail.com>
> > wrote:
> >
> > >
> > >
> > >
> > > Hello Dpdk Team,
> > >
> > > I'm facing issue while forwarding packets in DPDK's l2fwd application.
> > > While sending 1 Lac packets from Scapy, I could see sometimes packets
> are
> > > sending from one VM to another VM.
> > > Before explaining issue let me explain topology.
> > >
> > >
> > >
> > > *Topology :-=======*1) I am having 4 VMs(Virtual Machines) in same
> host.
> > > All these VM are running on Ubuntu 16.04.1.
> > > 2) VM-1 is used as Scapy to forward packets (Scapy version 2.4.3) .
> While
> > > creating packets I am giving destination mac(d-mac) address of VM-1.
> > > 3) In VM-2 am running L2 forwarding application.
> > >     In this l2fwd application, I am doing simple packet forwarding by
> > > statically keeping mac address of VM-3.
> > > Code :-
> > >      l2fwd_mac_updating () {
> > >    ...
> > >    ......
> > >    .......
> > >       *((uint64_t *)tmp) = 0xddccbbaa/*VM-3 mac address*/ +
> > > ((uint64_t)dest_portid << 40);
> > >      }
> > > 4) Also in VM-3, I am doing same like VM-2, but I kept mac address of
> VM-4
> > > 5) In VM-4, I am using wireshark to see packets coming from VM-3.
> > > 6) In VM-2 and VM-3, I kept promiscuous mode off by commenting out
> > > rte_eth_promiscuous_enable().
> > >
> > >    ---------------
>  ---------------------
> > >                          ------------------
> > > ------------------------
> > >    |    VM-1  |      ---------->                |       VM-2     |
> > > ---------------->   |     VM-3    |       ------------------> |
> VM-4
> > >      |
> > >    ---------------
>  ---------------------
> > >                          ------------------
> > > ------------------------
> > >         Scapy                              Simple L2 forwarding
> > >         Simple L2 forwarding                        Wireshark
> > > used for sending packets            Sending all packets to
> > >  Sending all packets to
> > >                                                             VM-3
> > >                         VM-4
> > >                                                           (DPDK)
> > >                      (DPDK)
> > >
> > >
> > > *Problem :- ======*==
> > > From scapy VM, I am sending 1 lac packets with rate of 100 packets per
> > > second. *During problematic condition I could see packets are not
> getting
> > > forward from VM-3.**Problematic state is happening anytime after
> sending
> > > 1k packets.*
> > > This issue is not consistence but I could see this issue 8 out of 10
> times.
> > >
> > >
> > > * Debugging and Observations :-=========================*
> > > When I tried to debug this issue , I could see that
> > > 1) In problematic state, rx queue of VM-3 is not getting packets, but
> VM-2
> > > is sending packets properly. I checked this by using pdump of rx queue.
> > > 2) Just before problematic state in VM-3, I could see that  previous
> > > packet(only one packet) instead of going to VM-4, it is coming back to
> > > again in rx queue of VM-3 and after wards I did not get any packets in
> rx
> > > queue.
> > > 3) I have changed rate of packet forward 10packets per second. but
> still
> > > see the issue.
> > >
> > >
> > > *Can anyone please help to solve this problem ? I need it urgently .*
> > > Thanks,
> > > Satya
> > >
>
> Sorry, you need to dig inside the forwarding application and instrument
> what is coming in and how packets are being processed. You usually can't
> treat DPDK as a black box.
>

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, back to index

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-04  7:24 [dpdk-users] l2fwd application are not sending continuous packets Rohit kamble
     [not found] ` <CAMJ3rejbOZ5mzOYyNPFqVk-Scs7CTaVZLUQOd-GMnVpaYf+pmg@mail.gmail.com>
2020-01-06  5:55   ` [dpdk-users] Fwd: " satyavalli rama
2020-01-06  7:07     ` [dpdk-users] " satyavalli rama
2020-01-09  1:55     ` satyavalli rama
2020-01-09  6:12       ` [dpdk-users] [dpdk-dev] " Stephen Hemminger
2020-01-09 10:34         ` satyavalli rama

DPDK usage discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/ public-inbox