DPDK usage discussions
 help / color / mirror / Atom feed
* [dpdk-users] Dpdk multi core packet acquisition.
@ 2021-07-14  8:32 Nirmal R
  2021-07-14 15:15 ` Stephen Hemminger
  0 siblings, 1 reply; 5+ messages in thread
From: Nirmal R @ 2021-07-14  8:32 UTC (permalink / raw)
  To: users

Hello All,

I need to acquire packets from a source at more than 40Gbps. I was thinking
to use rx_burst in a function with more than one core and load it into a
ring. My question is, will the packets received will be in order between
the cores say core 1 receives packet 1,2,3 and core 2 receives packet 4,5,6
or will it be in out of order(Random).  Also can we combine 2 cores to
launch one function.

Thank you,

With Regards,
Nirmalraj R

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-users] Dpdk multi core packet acquisition.
  2021-07-14  8:32 [dpdk-users] Dpdk multi core packet acquisition Nirmal R
@ 2021-07-14 15:15 ` Stephen Hemminger
  2021-07-14 16:32   ` Gábor LENCSE
  0 siblings, 1 reply; 5+ messages in thread
From: Stephen Hemminger @ 2021-07-14 15:15 UTC (permalink / raw)
  To: Nirmal R; +Cc: users

On Wed, 14 Jul 2021 10:32:56 +0200
Nirmal R <raja.nirmalraj@gmail.com> wrote:

> Hello All,
> 
> I need to acquire packets from a source at more than 40Gbps. I was thinking
> to use rx_burst in a function with more than one core and load it into a
> ring. My question is, will the packets received will be in order between
> the cores say core 1 receives packet 1,2,3 and core 2 receives packet 4,5,6
> or will it be in out of order(Random).  Also can we combine 2 cores to
> launch one function.
> 
> Thank you,
> 
> With Regards,
> Nirmalraj R

DPDK drivers assume a single core at a time is reading a single queue.
If you have multiple cores reading a single queue then you need to do
some form of mutual exclusion in your application.

This is in the documentation

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-users] Dpdk multi core packet acquisition.
  2021-07-14 15:15 ` Stephen Hemminger
@ 2021-07-14 16:32   ` Gábor LENCSE
  2021-07-14 17:05     ` Van Haaren, Harry
  0 siblings, 1 reply; 5+ messages in thread
From: Gábor LENCSE @ 2021-07-14 16:32 UTC (permalink / raw)
  To: users

I recommend you to use multi queue receiving (RSS). Thus you do not need 
to use mutual exclusion. (Performance!)
Of course, you need some entropy for the hash function to distribute the 
packets approximately evenly among the queues.

Gábor

2021.07.14. 17:15 keltezéssel, Stephen Hemminger írta:
> On Wed, 14 Jul 2021 10:32:56 +0200
> Nirmal R <raja.nirmalraj@gmail.com> wrote:
>
>> Hello All,
>>
>> I need to acquire packets from a source at more than 40Gbps. I was thinking
>> to use rx_burst in a function with more than one core and load it into a
>> ring. My question is, will the packets received will be in order between
>> the cores say core 1 receives packet 1,2,3 and core 2 receives packet 4,5,6
>> or will it be in out of order(Random).  Also can we combine 2 cores to
>> launch one function.
>>
>> Thank you,
>>
>> With Regards,
>> Nirmalraj R
> DPDK drivers assume a single core at a time is reading a single queue.
> If you have multiple cores reading a single queue then you need to do
> some form of mutual exclusion in your application.
>
> This is in the documentation


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-users] Dpdk multi core packet acquisition.
  2021-07-14 16:32   ` Gábor LENCSE
@ 2021-07-14 17:05     ` Van Haaren, Harry
       [not found]       ` <CAPk29mLgMP3EaKjLZvAkgz4aUj0U0obQ5+Y60kK7FsUPLy5WaQ@mail.gmail.com>
  0 siblings, 1 reply; 5+ messages in thread
From: Van Haaren, Harry @ 2021-07-14 17:05 UTC (permalink / raw)
  To: Gábor LENCSE, users; +Cc: raja.nirmalraj, Stephen Hemminger

(+CC Raja and Stephen)

> -----Original Message-----
> From: users <users-bounces@dpdk.org> On Behalf Of Gábor LENCSE
> Sent: Wednesday, July 14, 2021 5:32 PM
> To: users@dpdk.org
> Subject: Re: [dpdk-users] Dpdk multi core packet acquisition.
> 
> I recommend you to use multi queue receiving (RSS). Thus you do not need
> to use mutual exclusion. (Performance!)
> Of course, you need some entropy for the hash function to distribute the
> packets approximately evenly among the queues.
> 
> Gábor

Good input above!

Just for future reference, please don't top-post on mailing lists, and
always use the "reply all" to keep everybody on CC.
Top posting detail: https://www.mediawiki.org/wiki/Mailing_list_etiquette

> 2021.07.14. 17:15 keltezéssel, Stephen Hemminger írta:
> > On Wed, 14 Jul 2021 10:32:56 +0200
> > Nirmal R <raja.nirmalraj@gmail.com> wrote:
> >
> >> Hello All,
> >>
> >> I need to acquire packets from a source at more than 40Gbps. I was thinking
> >> to use rx_burst in a function with more than one core and load it into a
> >> ring. My question is, will the packets received will be in order between
> >> the cores say core 1 receives packet 1,2,3 and core 2 receives packet 4,5,6
> >> or will it be in out of order(Random).  Also can we combine 2 cores to
> >> launch one function.

As an alternative, you could have one lcore (dataplane thread) running the
rte_eth_rx_burst() function in a loop (without locks), enqueueing all mbufs
into a rte_ring ringbuffer. That allows capture of single-flow at very high MPPS
rates, depending on packet size, this likely covers 40Gbps.

Later, multiple cores can consume from the ringbuffer (in Multi-Consumer mode)
and each can write its own packet data to storage.

If you need to be able to restore full ordering, the RX lcore could increment
a counter for each packet received. By storing that metadata with the packet
absolute packet ordering could be restored for the entire packet stream, even
if multiple cores are actually writing each packet to separate storage files/devices.

I'd likely take this design approach if your avg packet size is ~128 or higher.

If its 64 byte packet line-rate you want, that's 59.x mpps, which is also achievable on one lcore,
but perhaps using RSS with multiple RX lcores is a better approach (if entropy available).
This becomes more difficult with absolute packet ordering, as multiple RSS queues
will ultimately be "out of absolute order" by nature of being in different HW queues.
(Restoring per-RSS-queue order is possible using the above counter approach though :)


> >> Thank you,
> >>
> >> With Regards,
> >> Nirmalraj R

Hope that helps! Regards, -Harry


> > DPDK drivers assume a single core at a time is reading a single queue.
> > If you have multiple cores reading a single queue then you need to do
> > some form of mutual exclusion in your application.
> >
> > This is in the documentation


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-users] Dpdk multi core packet acquisition.
       [not found]       ` <CAPk29mLgMP3EaKjLZvAkgz4aUj0U0obQ5+Y60kK7FsUPLy5WaQ@mail.gmail.com>
@ 2021-07-15  8:07         ` Nirmal R
  0 siblings, 0 replies; 5+ messages in thread
From: Nirmal R @ 2021-07-15  8:07 UTC (permalink / raw)
  To: users

Thank you so much for the input..

>
I tried with Harry solution and it worked well.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-07-15  8:08 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-14  8:32 [dpdk-users] Dpdk multi core packet acquisition Nirmal R
2021-07-14 15:15 ` Stephen Hemminger
2021-07-14 16:32   ` Gábor LENCSE
2021-07-14 17:05     ` Van Haaren, Harry
     [not found]       ` <CAPk29mLgMP3EaKjLZvAkgz4aUj0U0obQ5+Y60kK7FsUPLy5WaQ@mail.gmail.com>
2021-07-15  8:07         ` Nirmal R

DPDK usage discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror http://inbox.dpdk.org/users/0 users/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 users users/ http://inbox.dpdk.org/users \
		users@dpdk.org
	public-inbox-index users

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.users


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git