DPDK usage discussions
 help / color / Atom feed
* [dpdk-users] Round-robin packet distribution
@ 2020-07-24 10:05 Filip Janiszewski
  2020-07-25  8:35 ` Tom Barbette
       [not found] ` <8c8034d8-4cd5-bd0b-0958-14d08f5ae6e3@kth.se>
  0 siblings, 2 replies; 8+ messages in thread
From: Filip Janiszewski @ 2020-07-24 10:05 UTC (permalink / raw)
  To: users

Hi,

Is there a way in DPDK to configure the NIC to distribute the incoming
packets to multiple queues in a round robin fashion? Without taking into
account the payload/headers or type of packet, just plain round robin
distribution to multiple queues.

I'm struggling to obtain a fair mechanism using RSS, perhaps the
rte_flow API can do the trick? Any other suggestion?

Thanks

-- 
BR, Filip
+48 666 369 823

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-users] Round-robin packet distribution
  2020-07-24 10:05 [dpdk-users] Round-robin packet distribution Filip Janiszewski
@ 2020-07-25  8:35 ` Tom Barbette
       [not found] ` <8c8034d8-4cd5-bd0b-0958-14d08f5ae6e3@kth.se>
  1 sibling, 0 replies; 8+ messages in thread
From: Tom Barbette @ 2020-07-25  8:35 UTC (permalink / raw)
  To: Filip Janiszewski, users

Hi Filip,

This is not possible, but you may use the idea of Sprayer 
(http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch 
packets randomly (use RSS on the checksum).

However, in our paper RSS++ 
(https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we 
show it's nearly always a bad idea because you'll have to share state, 
and even for "stateless" function, that leads to a very bad locality in 
a firewall as the same rules have to be fetched to L1 to all cores at 
the same time when you receive a burst of similar packets.

Tom

Le 24/07/2020 à 12:05, Filip Janiszewski a écrit :
> Hi,
> 
> Is there a way in DPDK to configure the NIC to distribute the incoming
> packets to multiple queues in a round robin fashion? Without taking into
> account the payload/headers or type of packet, just plain round robin
> distribution to multiple queues.
> 
> I'm struggling to obtain a fair mechanism using RSS, perhaps the
> rte_flow API can do the trick? Any other suggestion?
> 
> Thanks
> 


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-users] Round-robin packet distribution
       [not found] ` <8c8034d8-4cd5-bd0b-0958-14d08f5ae6e3@kth.se>
@ 2020-08-18 12:40   ` Filip Janiszewski
  2020-08-18 13:50     ` Pawel Wodkowski
  0 siblings, 1 reply; 8+ messages in thread
From: Filip Janiszewski @ 2020-08-18 12:40 UTC (permalink / raw)
  To: Tom Barbette, users

Hi,

We had a look at that, and decided that it might be a bit too
complicated to implement in our SW and will not work in a performant way
as we might wish, ideally we're looking for a simple approach even if
not ideal..

So, I was wondering if we can get at least a "fair" distribution (no
round robin) using rte flow? And BTW, is it even possible to access the
checksum from this API using for example the pattern matching with an
offset that points to the proper byte location in the packet? (It seems
it can't be done without modification to the driver..)

Thanks

Il 7/25/20 10:28 AM, Tom Barbette ha scritto:
> Hi Filip,
> 
> This is not possible, but you may use the idea of Sprayer
> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch
> packets randomly (use RSS on the checksum).
> 
> However, in our paper RSS++
> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we
> show it's nearly always a bad idea because you'll have to share state,
> and even for "stateless" function, that leads to a very bad locality in
> a firewall as the same rules have to be fetched to L1 to all cores at
> the same time when you receive a burst of similar packets.
> 
> Tom
> 
> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit :
>> Hi,
>>
>> Is there a way in DPDK to configure the NIC to distribute the incoming
>> packets to multiple queues in a round robin fashion? Without taking into
>> account the payload/headers or type of packet, just plain round robin
>> distribution to multiple queues.
>>
>> I'm struggling to obtain a fair mechanism using RSS, perhaps the
>> rte_flow API can do the trick? Any other suggestion?
>>
>> Thanks
>>

-- 
BR, Filip
+48 666 369 823

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-users] Round-robin packet distribution
  2020-08-18 12:40   ` Filip Janiszewski
@ 2020-08-18 13:50     ` Pawel Wodkowski
  2020-08-18 14:27       ` Filip Janiszewski
  0 siblings, 1 reply; 8+ messages in thread
From: Pawel Wodkowski @ 2020-08-18 13:50 UTC (permalink / raw)
  To: Filip Janiszewski, Tom Barbette, users

Flexible payload matching (aka RAW in flow API) works up to first 64b of the
packet - at least in e1000, ixgbe and i40e.

It will be easier if you can provide some details about you network
traffic.

Paweł

On 18.08.2020 14:40, Filip Janiszewski wrote:
> Hi,
>
> We had a look at that, and decided that it might be a bit too
> complicated to implement in our SW and will not work in a performant way
> as we might wish, ideally we're looking for a simple approach even if
> not ideal..
>
> So, I was wondering if we can get at least a "fair" distribution (no
> round robin) using rte flow? And BTW, is it even possible to access the
> checksum from this API using for example the pattern matching with an
> offset that points to the proper byte location in the packet? (It seems
> it can't be done without modification to the driver..)
>
> Thanks
>
> Il 7/25/20 10:28 AM, Tom Barbette ha scritto:
>> Hi Filip,
>>
>> This is not possible, but you may use the idea of Sprayer
>> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch
>> packets randomly (use RSS on the checksum).
>>
>> However, in our paper RSS++
>> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we
>> show it's nearly always a bad idea because you'll have to share state,
>> and even for "stateless" function, that leads to a very bad locality in
>> a firewall as the same rules have to be fetched to L1 to all cores at
>> the same time when you receive a burst of similar packets.
>>
>> Tom
>>
>> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit :
>>> Hi,
>>>
>>> Is there a way in DPDK to configure the NIC to distribute the incoming
>>> packets to multiple queues in a round robin fashion? Without taking into
>>> account the payload/headers or type of packet, just plain round robin
>>> distribution to multiple queues.
>>>
>>> I'm struggling to obtain a fair mechanism using RSS, perhaps the
>>> rte_flow API can do the trick? Any other suggestion?
>>>
>>> Thanks
>>>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-users] Round-robin packet distribution
  2020-08-18 13:50     ` Pawel Wodkowski
@ 2020-08-18 14:27       ` Filip Janiszewski
  2020-08-24 10:08         ` Tom Barbette
       [not found]         ` <09498ac1-672d-a1e5-f917-d5720e1c90a7@kth.se>
  0 siblings, 2 replies; 8+ messages in thread
From: Filip Janiszewski @ 2020-08-18 14:27 UTC (permalink / raw)
  To: Pawel Wodkowski; +Cc: Tom Barbette, users

Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As
for the network traffic type, we're capturing financial market traffic
over TCP/UDP, nothing really fancy.

Thanks

Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto:
> Flexible payload matching (aka RAW in flow API) works up to first 64b of
> the
> packet - at least in e1000, ixgbe and i40e.
> 
> It will be easier if you can provide some details about you network
> traffic.
> 
> Paweł
> 
> On 18.08.2020 14:40, Filip Janiszewski wrote:
>> Hi,
>>
>> We had a look at that, and decided that it might be a bit too
>> complicated to implement in our SW and will not work in a performant way
>> as we might wish, ideally we're looking for a simple approach even if
>> not ideal..
>>
>> So, I was wondering if we can get at least a "fair" distribution (no
>> round robin) using rte flow? And BTW, is it even possible to access the
>> checksum from this API using for example the pattern matching with an
>> offset that points to the proper byte location in the packet? (It seems
>> it can't be done without modification to the driver..)
>>
>> Thanks
>>
>> Il 7/25/20 10:28 AM, Tom Barbette ha scritto:
>>> Hi Filip,
>>>
>>> This is not possible, but you may use the idea of Sprayer
>>> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch
>>> packets randomly (use RSS on the checksum).
>>>
>>> However, in our paper RSS++
>>> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we
>>> show it's nearly always a bad idea because you'll have to share state,
>>> and even for "stateless" function, that leads to a very bad locality in
>>> a firewall as the same rules have to be fetched to L1 to all cores at
>>> the same time when you receive a burst of similar packets.
>>>
>>> Tom
>>>
>>> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit :
>>>> Hi,
>>>>
>>>> Is there a way in DPDK to configure the NIC to distribute the incoming
>>>> packets to multiple queues in a round robin fashion? Without taking
>>>> into
>>>> account the payload/headers or type of packet, just plain round robin
>>>> distribution to multiple queues.
>>>>
>>>> I'm struggling to obtain a fair mechanism using RSS, perhaps the
>>>> rte_flow API can do the trick? Any other suggestion?
>>>>
>>>> Thanks
>>>>
> 

-- 
BR, Filip
+48 666 369 823

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-users] Round-robin packet distribution
  2020-08-18 14:27       ` Filip Janiszewski
@ 2020-08-24 10:08         ` Tom Barbette
       [not found]         ` <09498ac1-672d-a1e5-f917-d5720e1c90a7@kth.se>
  1 sibling, 0 replies; 8+ messages in thread
From: Tom Barbette @ 2020-08-24 10:08 UTC (permalink / raw)
  To: Filip Janiszewski, Pawel Wodkowski; +Cc: users

He means bytes.

I think some drivers only allow the flexible bytes to start after the 
actual matching though, eg TCP header. So check for that. I'd contact 
Sprayer authors to ask how they did it if I were you.

Do you have a specific NIC in mind? Mellanox's ones are pretty powerful, 
it may be worth it to have a PM-to-PM meeting asking about feature, or 
CC one of the maintainer. Devs do not always look at all mails.

Cheers,

Tom

Le 18/08/2020 à 16:27, Filip Janiszewski a écrit :
> Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As
> for the network traffic type, we're capturing financial market traffic
> over TCP/UDP, nothing really fancy.
>
> Thanks
>
> Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto:
>> Flexible payload matching (aka RAW in flow API) works up to first 64b of
>> the
>> packet - at least in e1000, ixgbe and i40e.
>>
>> It will be easier if you can provide some details about you network
>> traffic.
>>
>> Paweł
>>
>> On 18.08.2020 14:40, Filip Janiszewski wrote:
>>> Hi,
>>>
>>> We had a look at that, and decided that it might be a bit too
>>> complicated to implement in our SW and will not work in a performant way
>>> as we might wish, ideally we're looking for a simple approach even if
>>> not ideal..
>>>
>>> So, I was wondering if we can get at least a "fair" distribution (no
>>> round robin) using rte flow? And BTW, is it even possible to access the
>>> checksum from this API using for example the pattern matching with an
>>> offset that points to the proper byte location in the packet? (It seems
>>> it can't be done without modification to the driver..)
>>>
>>> Thanks
>>>
>>> Il 7/25/20 10:28 AM, Tom Barbette ha scritto:
>>>> Hi Filip,
>>>>
>>>> This is not possible, but you may use the idea of Sprayer
>>>> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch
>>>> packets randomly (use RSS on the checksum).
>>>>
>>>> However, in our paper RSS++
>>>> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we
>>>> show it's nearly always a bad idea because you'll have to share state,
>>>> and even for "stateless" function, that leads to a very bad locality in
>>>> a firewall as the same rules have to be fetched to L1 to all cores at
>>>> the same time when you receive a burst of similar packets.
>>>>
>>>> Tom
>>>>
>>>> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit :
>>>>> Hi,
>>>>>
>>>>> Is there a way in DPDK to configure the NIC to distribute the incoming
>>>>> packets to multiple queues in a round robin fashion? Without taking
>>>>> into
>>>>> account the payload/headers or type of packet, just plain round robin
>>>>> distribution to multiple queues.
>>>>>
>>>>> I'm struggling to obtain a fair mechanism using RSS, perhaps the
>>>>> rte_flow API can do the trick? Any other suggestion?
>>>>>
>>>>> Thanks
>>>>>

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-users] Round-robin packet distribution
       [not found]         ` <09498ac1-672d-a1e5-f917-d5720e1c90a7@kth.se>
@ 2020-08-25 12:17           ` Pawel Wodkowski
  2020-08-31 14:56             ` Filip Janiszewski
  0 siblings, 1 reply; 8+ messages in thread
From: Pawel Wodkowski @ 2020-08-25 12:17 UTC (permalink / raw)
  To: Tom Barbette, Filip Janiszewski; +Cc: users

Yes, I meant bytes. Thanks Tom.

If you deal with encrypted traffic you can use the payload as random 
data and spread it using flow director.
Constructing flow rules is painful process and be prepared that you case 
might not work if it is not coverd by test scenarios in DTS.
Also you will have to read documentation of your NIC to see what kind of 
flow type it might support.

Start from reading https://doc.dpdk.org/guides/prog_guide/rte_flow.html 
eg Table 12.22

But in my mind, as already mentioned by someone else, I think that 
randomly spreading packets across different cores
might not be a good idea.

If you want packets to be spread across all cores in deterministic but 
even way then easiest would be to use
RSS over IP / UDP. As a second iteration I would use Flow director API + 
IP / UDP rules *without* RAW items.

Paweł

On 24.08.2020 12:07, Tom Barbette wrote:
> He means bytes.
>
> I think some drivers only allow the flexible bytes to start after the 
> actual matching though, eg TCP header. So check for that. I'd contact 
> Sprayer authors to ask how they did it if I were you.
>
> Do you have a specific NIC in mind? Mellanox's ones are pretty 
> powerful, it may be worth it to have a PM-to-PM meeting asking about 
> feature, or CC one of the maintainer. Devs do not always look at all 
> mails.
>
> Cheers,
>
> Tom
>
> Le 18/08/2020 à 16:27, Filip Janiszewski a écrit :
>> Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As
>> for the network traffic type, we're capturing financial market traffic
>> over TCP/UDP, nothing really fancy.
>>
>> Thanks
>>
>> Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto:
>>> Flexible payload matching (aka RAW in flow API) works up to first 
>>> 64b of
>>> the
>>> packet - at least in e1000, ixgbe and i40e.
>>>
>>> It will be easier if you can provide some details about you network
>>> traffic.
>>>
>>> Paweł
>>>
>>> On 18.08.2020 14:40, Filip Janiszewski wrote:
>>>> Hi,
>>>>
>>>> We had a look at that, and decided that it might be a bit too
>>>> complicated to implement in our SW and will not work in a 
>>>> performant way
>>>> as we might wish, ideally we're looking for a simple approach even if
>>>> not ideal..
>>>>
>>>> So, I was wondering if we can get at least a "fair" distribution (no
>>>> round robin) using rte flow? And BTW, is it even possible to access 
>>>> the
>>>> checksum from this API using for example the pattern matching with an
>>>> offset that points to the proper byte location in the packet? (It 
>>>> seems
>>>> it can't be done without modification to the driver..)
>>>>
>>>> Thanks
>>>>
>>>> Il 7/25/20 10:28 AM, Tom Barbette ha scritto:
>>>>> Hi Filip,
>>>>>
>>>>> This is not possible, but you may use the idea of Sprayer
>>>>> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch
>>>>> packets randomly (use RSS on the checksum).
>>>>>
>>>>> However, in our paper RSS++
>>>>> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) 
>>>>> we
>>>>> show it's nearly always a bad idea because you'll have to share 
>>>>> state,
>>>>> and even for "stateless" function, that leads to a very bad 
>>>>> locality in
>>>>> a firewall as the same rules have to be fetched to L1 to all cores at
>>>>> the same time when you receive a burst of similar packets.
>>>>>
>>>>> Tom
>>>>>
>>>>> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit :
>>>>>> Hi,
>>>>>>
>>>>>> Is there a way in DPDK to configure the NIC to distribute the 
>>>>>> incoming
>>>>>> packets to multiple queues in a round robin fashion? Without taking
>>>>>> into
>>>>>> account the payload/headers or type of packet, just plain round 
>>>>>> robin
>>>>>> distribution to multiple queues.
>>>>>>
>>>>>> I'm struggling to obtain a fair mechanism using RSS, perhaps the
>>>>>> rte_flow API can do the trick? Any other suggestion?
>>>>>>
>>>>>> Thanks
>>>>>>
>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-users] Round-robin packet distribution
  2020-08-25 12:17           ` Pawel Wodkowski
@ 2020-08-31 14:56             ` Filip Janiszewski
  0 siblings, 0 replies; 8+ messages in thread
From: Filip Janiszewski @ 2020-08-31 14:56 UTC (permalink / raw)
  To: Pawel Wodkowski; +Cc: Tom Barbette, users

Hi,


> Start from reading https://doc.dpdk.org/guides/prog_guide/rte_flow.html
> eg Table 12.22
> 

And here's the painful news, the Mellanox cards we're using do not
support RAW flow items, which means I can't make use of the features
explained in the table 12.22


> But in my mind, as already mentioned by someone else, I think that
> randomly spreading packets across different cores
> might not be a good idea.
> 
> If you want packets to be spread across all cores in deterministic but
> even way then easiest would be to use
> RSS over IP / UDP. As a second iteration I would use Flow director API +
> IP / UDP rules *without* RAW items.
> 

Can you elaborate a bit? Perhaps there's some example somewhere on git
we can use as reference?

Thanks

> Paweł
> 
> On 24.08.2020 12:07, Tom Barbette wrote:
>> He means bytes.
>>
>> I think some drivers only allow the flexible bytes to start after the
>> actual matching though, eg TCP header. So check for that. I'd contact
>> Sprayer authors to ask how they did it if I were you.
>>
>> Do you have a specific NIC in mind? Mellanox's ones are pretty
>> powerful, it may be worth it to have a PM-to-PM meeting asking about
>> feature, or CC one of the maintainer. Devs do not always look at all
>> mails.
>>
>> Cheers,
>>
>> Tom
>>
>> Le 18/08/2020 à 16:27, Filip Janiszewski a écrit :
>>> Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As
>>> for the network traffic type, we're capturing financial market traffic
>>> over TCP/UDP, nothing really fancy.
>>>
>>> Thanks
>>>
>>> Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto:
>>>> Flexible payload matching (aka RAW in flow API) works up to first
>>>> 64b of
>>>> the
>>>> packet - at least in e1000, ixgbe and i40e.
>>>>
>>>> It will be easier if you can provide some details about you network
>>>> traffic.
>>>>
>>>> Paweł
>>>>
>>>> On 18.08.2020 14:40, Filip Janiszewski wrote:
>>>>> Hi,
>>>>>
>>>>> We had a look at that, and decided that it might be a bit too
>>>>> complicated to implement in our SW and will not work in a
>>>>> performant way
>>>>> as we might wish, ideally we're looking for a simple approach even if
>>>>> not ideal..
>>>>>
>>>>> So, I was wondering if we can get at least a "fair" distribution (no
>>>>> round robin) using rte flow? And BTW, is it even possible to access
>>>>> the
>>>>> checksum from this API using for example the pattern matching with an
>>>>> offset that points to the proper byte location in the packet? (It
>>>>> seems
>>>>> it can't be done without modification to the driver..)
>>>>>
>>>>> Thanks
>>>>>
>>>>> Il 7/25/20 10:28 AM, Tom Barbette ha scritto:
>>>>>> Hi Filip,
>>>>>>
>>>>>> This is not possible, but you may use the idea of Sprayer
>>>>>> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch
>>>>>> packets randomly (use RSS on the checksum).
>>>>>>
>>>>>> However, in our paper RSS++
>>>>>> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf)
>>>>>> we
>>>>>> show it's nearly always a bad idea because you'll have to share
>>>>>> state,
>>>>>> and even for "stateless" function, that leads to a very bad
>>>>>> locality in
>>>>>> a firewall as the same rules have to be fetched to L1 to all cores at
>>>>>> the same time when you receive a burst of similar packets.
>>>>>>
>>>>>> Tom
>>>>>>
>>>>>> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit :
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there a way in DPDK to configure the NIC to distribute the
>>>>>>> incoming
>>>>>>> packets to multiple queues in a round robin fashion? Without taking
>>>>>>> into
>>>>>>> account the payload/headers or type of packet, just plain round
>>>>>>> robin
>>>>>>> distribution to multiple queues.
>>>>>>>
>>>>>>> I'm struggling to obtain a fair mechanism using RSS, perhaps the
>>>>>>> rte_flow API can do the trick? Any other suggestion?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>
> 

-- 
BR, Filip
+48 666 369 823

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, back to index

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-24 10:05 [dpdk-users] Round-robin packet distribution Filip Janiszewski
2020-07-25  8:35 ` Tom Barbette
     [not found] ` <8c8034d8-4cd5-bd0b-0958-14d08f5ae6e3@kth.se>
2020-08-18 12:40   ` Filip Janiszewski
2020-08-18 13:50     ` Pawel Wodkowski
2020-08-18 14:27       ` Filip Janiszewski
2020-08-24 10:08         ` Tom Barbette
     [not found]         ` <09498ac1-672d-a1e5-f917-d5720e1c90a7@kth.se>
2020-08-25 12:17           ` Pawel Wodkowski
2020-08-31 14:56             ` Filip Janiszewski

DPDK usage discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/ public-inbox