[dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Alex Kiselev]

Hi.

It seems that xl710 NIC doesn't receive 1518 (maximum MTU size 1500 +
dot1q ethernet header
18 bytes) bytes packets. And there is no such problem when I use x520 NIC.

I am sending ipv4 ICMP ping packets and I am sure that packets go out
from the port
on the switch to which my application is connected. This is the
tcpdump output from the mirror port:

00:17:52.644171 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
802.1Q (0x8100), length 1518: vlan 10, p 0, ethertype IPv4, (tos 0x0,
ttl 64, id 630, offset 0, flags [DF], proto ICMP (1), length 1500)
    1.0.0.2 > 1.0.0.1: ICMP echo request, id 5386, seq 7, length 1480

And I if send 1514 bytes packets everything is also ok

h7 ~ # ip netns exec bs1 ping -s 1468 1.0.0.1
PING 1.0.0.1 (1.0.0.1) 1468(1496) bytes of data.
1476 bytes from 1.0.0.1: icmp_seq=1 ttl=64 time=0.935 ms

tcpdump on mirror switch port for 1514 bytes (ping 1468):

00:28:15.890543 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
    1.0.0.2 > 1.0.0.1: ICMP echo request, id 5389, seq 1, length 1476
00:28:15.890548 68:05:ca:30:61:80 > f8:32:e4:72:61:1b, ethertype
802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
    1.0.0.1 > 1.0.0.2: ICMP echo reply, id 5389, seq 1, length 1476

I’ve tried to use rte_eth_dev_set_mtu() with different parameters
but nothing has changed.

Have I missed something to configure?

Thanks.

--
Alex Kiselev

Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Alex Kiselev]

The issue with receiving 1518 byte packets is half resolved. It was an
MTU problem. I moved rte_eth_dev_set_mtu() before the
rte_dev_dev_start and it helped. Thanks to Alex _
<asb.tyum@gmail.com>.
So, 1518 packets now are successfully sent and received.

But, now I've got the same problem with the bonding driver. I created
a bond port with four i40e slave ports and place it in the LACP mode.
And the bond port doesn't receive 1518 bytes packets.

Please, help me to resolve the issue.

2018-01-24 0:44 GMT+03:00 Alex Kiselev <kiselev99@gmail.com>:
> Hi.
>
> It seems that xl710 NIC doesn't receive 1518 (maximum MTU size 1500 +
> dot1q ethernet header
> 18 bytes) bytes packets. And there is no such problem when I use x520 NIC.
>
> I am sending ipv4 ICMP ping packets and I am sure that packets go out
> from the port
> on the switch to which my application is connected. This is the
> tcpdump output from the mirror port:
>
> 00:17:52.644171 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
> 802.1Q (0x8100), length 1518: vlan 10, p 0, ethertype IPv4, (tos 0x0,
> ttl 64, id 630, offset 0, flags [DF], proto ICMP (1), length 1500)
>     1.0.0.2 > 1.0.0.1: ICMP echo request, id 5386, seq 7, length 1480
>
> And I if send 1514 bytes packets everything is also ok
>
> h7 ~ # ip netns exec bs1 ping -s 1468 1.0.0.1
> PING 1.0.0.1 (1.0.0.1) 1468(1496) bytes of data.
> 1476 bytes from 1.0.0.1: icmp_seq=1 ttl=64 time=0.935 ms
>
> tcpdump on mirror switch port for 1514 bytes (ping 1468):
>
> 00:28:15.890543 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
> 802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
> ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
>     1.0.0.2 > 1.0.0.1: ICMP echo request, id 5389, seq 1, length 1476
> 00:28:15.890548 68:05:ca:30:61:80 > f8:32:e4:72:61:1b, ethertype
> 802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
> ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
>     1.0.0.1 > 1.0.0.2: ICMP echo reply, id 5389, seq 1, length 1476
>
> I’ve tried to use rte_eth_dev_set_mtu() with different parameters
> but nothing has changed.
>
> Have I missed something to configure?
>
> Thanks.
>
> --
> Alex Kiselev



-- 
--
Kiselev Alexander

Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Kyle Larose]

Did you set the mtu on the bond port? It has separate configuration IIRC.

> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Alex Kiselev
> Sent: Wednesday, January 24, 2018 8:51 AM
> To: users
> Subject: Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets
> 
> The issue with receiving 1518 byte packets is half resolved. It was an MTU
> problem. I moved rte_eth_dev_set_mtu() before the rte_dev_dev_start and it
> helped. Thanks to Alex _ <asb.tyum@gmail.com>.
> So, 1518 packets now are successfully sent and received.
> 
> But, now I've got the same problem with the bonding driver. I created a bond
> port with four i40e slave ports and place it in the LACP mode.
> And the bond port doesn't receive 1518 bytes packets.
> 
> Please, help me to resolve the issue.
> 
> 2018-01-24 0:44 GMT+03:00 Alex Kiselev <kiselev99@gmail.com>:
> > Hi.
> >
> > It seems that xl710 NIC doesn't receive 1518 (maximum MTU size 1500 +
> > dot1q ethernet header
> > 18 bytes) bytes packets. And there is no such problem when I use x520 NIC.
> >
> > I am sending ipv4 ICMP ping packets and I am sure that packets go out
> > from the port on the switch to which my application is connected. This
> > is the tcpdump output from the mirror port:
> >
> > 00:17:52.644171 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
> > 802.1Q (0x8100), length 1518: vlan 10, p 0, ethertype IPv4, (tos 0x0,
> > ttl 64, id 630, offset 0, flags [DF], proto ICMP (1), length 1500)
> >     1.0.0.2 > 1.0.0.1: ICMP echo request, id 5386, seq 7, length 1480
> >
> > And I if send 1514 bytes packets everything is also ok
> >
> > h7 ~ # ip netns exec bs1 ping -s 1468 1.0.0.1 PING 1.0.0.1 (1.0.0.1)
> > 1468(1496) bytes of data.
> > 1476 bytes from 1.0.0.1: icmp_seq=1 ttl=64 time=0.935 ms
> >
> > tcpdump on mirror switch port for 1514 bytes (ping 1468):
> >
> > 00:28:15.890543 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
> > 802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
> > ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
> >     1.0.0.2 > 1.0.0.1: ICMP echo request, id 5389, seq 1, length 1476
> > 00:28:15.890548 68:05:ca:30:61:80 > f8:32:e4:72:61:1b, ethertype
> > 802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
> > ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
> >     1.0.0.1 > 1.0.0.2: ICMP echo reply, id 5389, seq 1, length 1476
> >
> > I’ve tried to use rte_eth_dev_set_mtu() with different parameters but
> > nothing has changed.
> >
> > Have I missed something to configure?
> >
> > Thanks.
> >
> > --
> > Alex Kiselev
> 
> 
> 
> --
> --
> Kiselev Alexander

Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Alex Kiselev]

Hi Kyle.

2018-01-24 17:01 GMT+03:00 Kyle Larose <klarose@sandvine.com>:
> Did you set the mtu on the bond port? It has separate configuration IIRC.

Yes, I did. I create bond ports via command line parameters and work
with them the same way as with usual ports
(except few things regarding the requirement to send LACP packets every N ms)

>
>> -----Original Message-----
>> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Alex Kiselev
>> Sent: Wednesday, January 24, 2018 8:51 AM
>> To: users
>> Subject: Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets
>>
>> The issue with receiving 1518 byte packets is half resolved. It was an MTU
>> problem. I moved rte_eth_dev_set_mtu() before the rte_dev_dev_start and it
>> helped. Thanks to Alex _ <asb.tyum@gmail.com>.
>> So, 1518 packets now are successfully sent and received.
>>
>> But, now I've got the same problem with the bonding driver. I created a bond
>> port with four i40e slave ports and place it in the LACP mode.
>> And the bond port doesn't receive 1518 bytes packets.
>>
>> Please, help me to resolve the issue.
>>
>> 2018-01-24 0:44 GMT+03:00 Alex Kiselev <kiselev99@gmail.com>:
>> > Hi.
>> >
>> > It seems that xl710 NIC doesn't receive 1518 (maximum MTU size 1500 +
>> > dot1q ethernet header
>> > 18 bytes) bytes packets. And there is no such problem when I use x520 NIC.
>> >
>> > I am sending ipv4 ICMP ping packets and I am sure that packets go out
>> > from the port on the switch to which my application is connected. This
>> > is the tcpdump output from the mirror port:
>> >
>> > 00:17:52.644171 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
>> > 802.1Q (0x8100), length 1518: vlan 10, p 0, ethertype IPv4, (tos 0x0,
>> > ttl 64, id 630, offset 0, flags [DF], proto ICMP (1), length 1500)
>> >     1.0.0.2 > 1.0.0.1: ICMP echo request, id 5386, seq 7, length 1480
>> >
>> > And I if send 1514 bytes packets everything is also ok
>> >
>> > h7 ~ # ip netns exec bs1 ping -s 1468 1.0.0.1 PING 1.0.0.1 (1.0.0.1)
>> > 1468(1496) bytes of data.
>> > 1476 bytes from 1.0.0.1: icmp_seq=1 ttl=64 time=0.935 ms
>> >
>> > tcpdump on mirror switch port for 1514 bytes (ping 1468):
>> >
>> > 00:28:15.890543 f8:32:e4:72:61:1b > 68:05:ca:30:61:80, ethertype
>> > 802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
>> > ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
>> >     1.0.0.2 > 1.0.0.1: ICMP echo request, id 5389, seq 1, length 1476
>> > 00:28:15.890548 68:05:ca:30:61:80 > f8:32:e4:72:61:1b, ethertype
>> > 802.1Q (0x8100), length 1514: vlan 10, p 0, ethertype IPv4, (tos 0x0,
>> > ttl 64, id 22101, offset 0, flags [DF], proto ICMP (1), length 1496)
>> >     1.0.0.1 > 1.0.0.2: ICMP echo reply, id 5389, seq 1, length 1476
>> >
>> > I’ve tried to use rte_eth_dev_set_mtu() with different parameters but
>> > nothing has changed.
>> >
>> > Have I missed something to configure?
>> >
>> > Thanks.
>> >
>> > --
>> > Alex Kiselev
>>
>>
>>
>> --
>> --
>> Kiselev Alexander



-- 
--
Kiselev Alexander

Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Alex Kiselev]

2018-01-24 17:14 GMT+03:00 Alex Kiselev <kiselev99@gmail.com>:
> Hi Kyle.
>
> 2018-01-24 17:01 GMT+03:00 Kyle Larose <klarose@sandvine.com>:
>> Did you set the MTU on the bond port? It has separate configuration IIRC.

I don't see any special API functions for setting MTU of a bonding
port neither in 17.08.1
that I am currently using nor in the latest DPDK branch.

So, I use regular MTU setting function rte_eth_dev_set_mtu() for my
bond port created via command line arguments.
And It doesn't work. 1518 bytes packets are being filtered.

I tried to hardcode setting MTU for each slave port and it solved the
problem. So, it looks like
the problem is the rte_eth_dev_set_mtu() call that doesn't change MTU
of slave ports when it is called
for a bonding port.

The described problem exists at least with i40e driver and doesn't
show up when I use x520 NICS.

-- 
Alex Kiselev

Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Kyle Larose]

I meant to suggest that, sorry. :( 

That said, I think a patch was submitted recently to set the mtu of slave ports when the bond's mtu is set.

http://dpdk.org/browse/dpdk/commit/drivers/net/bonding/rte_eth_bond_pmd.c?id=55b58a7374554cd1c86f4a13a0e2f54e9ba6fe4d

Are you running with that patch?

> -----Original Message-----
> From: Alex Kiselev [mailto:kiselev99@gmail.com]
> Sent: Friday, January 26, 2018 8:44 AM
> To: Kyle Larose
> Cc: users
> Subject: Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets
> 
> 2018-01-24 17:14 GMT+03:00 Alex Kiselev <kiselev99@gmail.com>:
> > Hi Kyle.
> >
> > 2018-01-24 17:01 GMT+03:00 Kyle Larose <klarose@sandvine.com>:
> >> Did you set the MTU on the bond port? It has separate configuration IIRC.
> 
> I don't see any special API functions for setting MTU of a bonding port
> neither in 17.08.1 that I am currently using nor in the latest DPDK branch.
> 
> So, I use regular MTU setting function rte_eth_dev_set_mtu() for my bond
> port created via command line arguments.
> And It doesn't work. 1518 bytes packets are being filtered.
> 
> I tried to hardcode setting MTU for each slave port and it solved the
> problem. So, it looks like the problem is the rte_eth_dev_set_mtu() call
> that doesn't change MTU of slave ports when it is called for a bonding port.
> 
> The described problem exists at least with i40e driver and doesn't show up
> when I use x520 NICS.
> 
> --
> Alex Kiselev

Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Alex Kiselev]

Hi Kyle.

2018-01-26 16:47 GMT+03:00 Kyle Larose <klarose@sandvine.com>:
> I meant to suggest that, sorry. :(
>
> That said, I think a patch was submitted recently to set the mtu of slave ports when the bond's mtu is set.
>
> http://dpdk.org/browse/dpdk/commit/drivers/net/bonding/rte_eth_bond_pmd.c?id=55b58a7374554cd1c86f4a13a0e2f54e9ba6fe4d
>
> Are you running with that patch?

No. I wasn't aware of this patch.
I'll try it. Thanks.

>
>> -----Original Message-----
>> From: Alex Kiselev [mailto:kiselev99@gmail.com]
>> Sent: Friday, January 26, 2018 8:44 AM
>> To: Kyle Larose
>> Cc: users
>> Subject: Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets
>>
>> 2018-01-24 17:14 GMT+03:00 Alex Kiselev <kiselev99@gmail.com>:
>> > Hi Kyle.
>> >
>> > 2018-01-24 17:01 GMT+03:00 Kyle Larose <klarose@sandvine.com>:
>> >> Did you set the MTU on the bond port? It has separate configuration IIRC.
>>
>> I don't see any special API functions for setting MTU of a bonding port
>> neither in 17.08.1 that I am currently using nor in the latest DPDK branch.
>>
>> So, I use regular MTU setting function rte_eth_dev_set_mtu() for my bond
>> port created via command line arguments.
>> And It doesn't work. 1518 bytes packets are being filtered.
>>
>> I tried to hardcode setting MTU for each slave port and it solved the
>> problem. So, it looks like the problem is the rte_eth_dev_set_mtu() call
>> that doesn't change MTU of slave ports when it is called for a bonding port.
>>
>> The described problem exists at least with i40e driver and doesn't show up
>> when I use x520 NICS.
>>
>> --
>> Alex Kiselev



-- 
--
Kiselev Alexander

Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets

[Don maillist]

Interesting we are using OVS+DPDK with DPDKBOND and LACP with an MTU of
9000. We have not seen any issues except when mixing DPDK and non DPDK
ports on the same NIC. Intel is fixing that but otherwise, we are good. I
do know that Redhat uses 16.02.1 DPDK.

Best Regards,
Don

On Fri, Jan 26, 2018 at 7:52 AM, Alex Kiselev <kiselev99@gmail.com> wrote:

> Hi Kyle.
>
> 2018-01-26 16:47 GMT+03:00 Kyle Larose <klarose@sandvine.com>:
> > I meant to suggest that, sorry. :(
> >
> > That said, I think a patch was submitted recently to set the mtu of
> slave ports when the bond's mtu is set.
> >
> > http://dpdk.org/browse/dpdk/commit/drivers/net/bonding/
> rte_eth_bond_pmd.c?id=55b58a7374554cd1c86f4a13a0e2f54e9ba6fe4d
> >
> > Are you running with that patch?
>
> No. I wasn't aware of this patch.
> I'll try it. Thanks.
>
> >
> >> -----Original Message-----
> >> From: Alex Kiselev [mailto:kiselev99@gmail.com]
> >> Sent: Friday, January 26, 2018 8:44 AM
> >> To: Kyle Larose
> >> Cc: users
> >> Subject: Re: [dpdk-users] xl710 NIC doesn't receive 1518 bytes packets
> >>
> >> 2018-01-24 17:14 GMT+03:00 Alex Kiselev <kiselev99@gmail.com>:
> >> > Hi Kyle.
> >> >
> >> > 2018-01-24 17:01 GMT+03:00 Kyle Larose <klarose@sandvine.com>:
> >> >> Did you set the MTU on the bond port? It has separate configuration
> IIRC.
> >>
> >> I don't see any special API functions for setting MTU of a bonding port
> >> neither in 17.08.1 that I am currently using nor in the latest DPDK
> branch.
> >>
> >> So, I use regular MTU setting function rte_eth_dev_set_mtu() for my bond
> >> port created via command line arguments.
> >> And It doesn't work. 1518 bytes packets are being filtered.
> >>
> >> I tried to hardcode setting MTU for each slave port and it solved the
> >> problem. So, it looks like the problem is the rte_eth_dev_set_mtu() call
> >> that doesn't change MTU of slave ports when it is called for a bonding
> port.
> >>
> >> The described problem exists at least with i40e driver and doesn't show
> up
> >> when I use x520 NICS.
> >>
> >> --
> >> Alex Kiselev
>
>
>
> --
> --
> Kiselev Alexander
>