Hi All, I am trying to build a multi-RX-queue dpdk program, using RSS to split the incoming traffic into RX queues on a single port. Mellanox ConnectX-5 and DPDK Version 19.11 is used for this purpose. It works fine when I use IP over Ethernet packets as input. However when the packet contains IP over MPLS over Ethernet, RSS does not seem to work. As a result, all packets belonging to various flows (with different src & dst IPs, ports over MPLS) are all sent into the same RX queue. My queries are 1. Is there any parameter/techniques in DPDK to distribute MPLS packets to multiple RX queues? 2. Is there any way to strip off MPLS tags (between Eth and IP) in hardware, something like hw_vlan_strip? My Port configuration is const struct rte_eth_conf default_port_conf = { .rxmode = { .hw_vlan_strip = 0, /* VLAN strip enabled. */ .header_split = 0, /* Header Split disabled. */ .hw_ip_checksum = 0, /* IP checksum offload disabled. */ .hw_strip_crc = 0, /* CRC stripping by hardware disabled. */ }, .rx_adv_conf = { .rss_conf = { .rss_key = NULL, .rss_key_len = 0, .rss_hf = ETH_RSS_IP, }, } }; _______________________ Thanks and Regards Raktim Bhatt
++, Hi Raktim, > -----Original Message----- > From: users <users-bounces@dpdk.org> On Behalf Of raktim bhatt > Sent: Saturday, January 16, 2021 10:46 AM > To: users@dpdk.org > Subject: [dpdk-users] DPDK: MPLS packet processing > > Hi All, > > I am trying to build a multi-RX-queue dpdk program, using RSS to split the > incoming traffic into RX queues on a single port. Mellanox ConnectX-5 and > DPDK Version 19.11 is used for this purpose. It works fine when I use IP > over Ethernet packets as input. However when the packet contains IP over > MPLS over Ethernet, RSS does not seem to work. As a result, all packets > belonging to various flows (with different src & dst IPs, ports over MPLS) > are all sent into the same RX queue. > > > My queries are > > 1. Is there any parameter/techniques in DPDK to distribute MPLS packets to > multiple RX queues? > I've tried it over my setup with testpmd: ./build/app/dpdk-testpmd -n 4 -w 0000:08:00.0 -- --mbcache=512 -i --nb-cores=27 --rxq=4 --txq=4 --rss-ip testpmd> set verbose 1 testpmd> start then tried to send two MPLS packets with different src IP: packet1 = Ether()/MPLS()/IP(src='1.1.1.1') packet2 = Ether()/MPLS()/IP(src='1.1.1.2') and I see that both packets are being spread over the queues, see the bellow testpmd dump output: testpmd> port 0/queue 3: received 1 packets src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - nb_segs=1 - RSS hash=0x43781943 - RSS queue=0x3 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_NONFRAG - sw ptype: L2_ETHER - l2_len=14 - Receive queue=0x3 ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN port 0/queue 1: received 1 packets src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - nb_segs=1 - RSS hash=0xb8631e05 - RSS queue=0x1 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_NONFRAG - sw ptype: L2_ETHER - l2_len=14 - Receive queue=0x1 ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN first packet was received on queue 3 and the second one was received over queue 1, by the way, this is with both 19.11.0 and v19.11.6 > 2. Is there any way to strip off MPLS tags (between Eth and IP) in > hardware, something like hw_vlan_strip? > For this I'm not sure we have such thing in dpdk maybe Thomas can confirm this here? > > My Port configuration is > > > const struct rte_eth_conf default_port_conf = { > > .rxmode = { > > .hw_vlan_strip = 0, /* VLAN strip enabled. */ > > .header_split = 0, /* Header Split disabled. */ > > .hw_ip_checksum = 0, /* IP checksum offload disabled. */ > > .hw_strip_crc = 0, /* CRC stripping by hardware disabled. > */ > > }, > > .rx_adv_conf = { > > .rss_conf = { > > .rss_key = NULL, > > .rss_key_len = 0, > > .rss_hf = ETH_RSS_IP, > > }, > > } }; > > _______________________ > > > Thanks and Regards > > Raktim Bhatt
18/01/2021 09:46, Raslan Darawsheh:
> From: raktim bhatt
>
> > Hi All,
> >
> > I am trying to build a multi-RX-queue dpdk program, using RSS to split the
> > incoming traffic into RX queues on a single port. Mellanox ConnectX-5 and
> > DPDK Version 19.11 is used for this purpose. It works fine when I use IP
> > over Ethernet packets as input. However when the packet contains IP over
> > MPLS over Ethernet, RSS does not seem to work. As a result, all packets
> > belonging to various flows (with different src & dst IPs, ports over MPLS)
> > are all sent into the same RX queue.
> >
> >
> > My queries are
> >
> > 1. Is there any parameter/techniques in DPDK to distribute MPLS packets to
> > multiple RX queues?
> >
> I've tried it over my setup with testpmd:
> ./build/app/dpdk-testpmd -n 4 -w 0000:08:00.0 -- --mbcache=512 -i --nb-cores=27 --rxq=4 --txq=4 --rss-ip
> testpmd> set verbose 1
> testpmd> start
>
> then tried to send two MPLS packets with different src IP:
> packet1 = Ether()/MPLS()/IP(src='1.1.1.1')
> packet2 = Ether()/MPLS()/IP(src='1.1.1.2')
>
> and I see that both packets are being spread over the queues, see the bellow testpmd dump output:
> testpmd> port 0/queue 3: received 1 packets
> src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - nb_segs=1 - RSS hash=0x43781943 - RSS queue=0x3 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_NONFRAG - sw ptype: L2_ETHER - l2_len=14 - Receive queue=0x3
> ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> port 0/queue 1: received 1 packets
> src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 - nb_segs=1 - RSS hash=0xb8631e05 - RSS queue=0x1 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_NONFRAG - sw ptype: L2_ETHER - l2_len=14 - Receive queue=0x1
> ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN
>
> first packet was received on queue 3 and the second one was received over queue 1,
> by the way, this is with both 19.11.0 and v19.11.6
>
> > 2. Is there any way to strip off MPLS tags (between Eth and IP) in
> > hardware, something like hw_vlan_strip?
> >
> For this I'm not sure we have such thing in dpdk maybe Thomas can confirm this here?
Look for "POP_MPLS" in rte_flow.
> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, January 18, 2021 12:08 PM
> To: raktim bhatt <rocky.cair@gmail.com>; Raslan Darawsheh
> <rasland@nvidia.com>
> Cc: users@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; Asaf Penso
> <asafp@nvidia.com>
> Subject: Re: [dpdk-users] DPDK: MPLS packet processing
>
> 18/01/2021 09:46, Raslan Darawsheh:
> > From: raktim bhatt
> >
> > > Hi All,
> > >
> > > I am trying to build a multi-RX-queue dpdk program, using RSS to split the
> > > incoming traffic into RX queues on a single port. Mellanox ConnectX-5
> and
> > > DPDK Version 19.11 is used for this purpose. It works fine when I use IP
> > > over Ethernet packets as input. However when the packet contains IP
> over
> > > MPLS over Ethernet, RSS does not seem to work. As a result, all packets
> > > belonging to various flows (with different src & dst IPs, ports over MPLS)
> > > are all sent into the same RX queue.
> > >
> > >
> > > My queries are
> > >
> > > 1. Is there any parameter/techniques in DPDK to distribute MPLS packets
> to
> > > multiple RX queues?
> > >
> > I've tried it over my setup with testpmd:
> > ./build/app/dpdk-testpmd -n 4 -w 0000:08:00.0 -- --mbcache=512 -i --nb-
> cores=27 --rxq=4 --txq=4 --rss-ip
> > testpmd> set verbose 1
> > testpmd> start
> >
> > then tried to send two MPLS packets with different src IP:
> > packet1 = Ether()/MPLS()/IP(src='1.1.1.1')
> > packet2 = Ether()/MPLS()/IP(src='1.1.1.2')
> >
> > and I see that both packets are being spread over the queues, see the
> bellow testpmd dump output:
> > testpmd> port 0/queue 3: received 1 packets
> > src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 -
> nb_segs=1 - RSS hash=0x43781943 - RSS queue=0x3 - hw ptype: L2_ETHER
> L3_IPV4_EXT_UNKNOWN L4_NONFRAG - sw ptype: L2_ETHER - l2_len=14 -
> Receive queue=0x3
> > ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN
> PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> > port 0/queue 1: received 1 packets
> > src=00:00:00:00:00:00 - dst=FF:FF:FF:FF:FF:FF - type=0x8847 - length=60 -
> nb_segs=1 - RSS hash=0xb8631e05 - RSS queue=0x1 - hw ptype: L2_ETHER
> L3_IPV4_EXT_UNKNOWN L4_NONFRAG - sw ptype: L2_ETHER - l2_len=14 -
> Receive queue=0x1
> > ol_flags: PKT_RX_RSS_HASH PKT_RX_L4_CKSUM_UNKNOWN
> PKT_RX_IP_CKSUM_GOOD PKT_RX_OUTER_L4_CKSUM_UNKNOWN
> >
> > first packet was received on queue 3 and the second one was received
> over queue 1,
> > by the way, this is with both 19.11.0 and v19.11.6
> >
> > > 2. Is there any way to strip off MPLS tags (between Eth and IP) in
> > > hardware, something like hw_vlan_strip?
> > >
> > For this I'm not sure we have such thing in dpdk maybe Thomas can confirm
> this here?
>
> Look for "POP_MPLS" in rte_flow.
>
Thanks, for pointing to it, I've just noticed it.
But unfortunately, we don't have support for this action in MLX5 PMD.
Kindest regards
Raslan Darawsheh