DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] DPDK RSS support for ixgbevf PMD
@ 2014-12-15 19:11 Vladislav Zolotarov
  2014-12-15 20:33 ` Thomas Monjalon
  0 siblings, 1 reply; 11+ messages in thread
From: Vladislav Zolotarov @ 2014-12-15 19:11 UTC (permalink / raw)
  To: dev; +Cc: Dor Laor, Avi Kivity, Gleb Natapov

Hi,
I'm running an ixgbevf PMD on an AWS guests with extended networking
(SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
current git tree there is no support for a multi-queue in this PMD: reta
size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues and
max_tx_queues are both 4.

Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
RSS queues: for some reason it always limits the number of RSS queues by 2.

ixgbevf_main.c: line 2539
u16 rss = min_t(u16, num_online_cpus(), 2);

The above is strange since if MRQE is set to 1010b there are 4 RSS queues
available which seems to be the case in my AWS Guest.

However, let's get back to DPDK. As I've mentioned above the SR-IOV
function i have is RSS capable (to be 100% sure I've verified both queues
are receiving packets in a multi-socket TCP test). And it's a shame I can't
utilize it with a DPDK.

I wonder if there are any blockers to add this capability to the ixgbevf
PMD and if not is it scheduled to some time soon?

Thanks in advance,
vlad

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-15 19:11 [dpdk-dev] DPDK RSS support for ixgbevf PMD Vladislav Zolotarov
@ 2014-12-15 20:33 ` Thomas Monjalon
  2014-12-15 20:45   ` Dor Laor
  2014-12-16 15:36   ` [dpdk-dev] DPDK RSS support for ixgbevf PMD Vlad Zolotarov
  0 siblings, 2 replies; 11+ messages in thread
From: Thomas Monjalon @ 2014-12-15 20:33 UTC (permalink / raw)
  To: Vladislav Zolotarov; +Cc: dev, Dor Laor, Avi Kivity, Gleb Natapov

2014-12-15 21:11, Vladislav Zolotarov:
> Hi,
> I'm running an ixgbevf PMD on an AWS guests with extended networking
> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
> current git tree there is no support for a multi-queue in this PMD: reta
> size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues and
> max_tx_queues are both 4.
> 
> Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
> RSS queues: for some reason it always limits the number of RSS queues by 2.
> 
> ixgbevf_main.c: line 2539
> u16 rss = min_t(u16, num_online_cpus(), 2);
> 
> The above is strange since if MRQE is set to 1010b there are 4 RSS queues
> available which seems to be the case in my AWS Guest.
> 
> However, let's get back to DPDK. As I've mentioned above the SR-IOV
> function i have is RSS capable (to be 100% sure I've verified both queues
> are receiving packets in a multi-socket TCP test). And it's a shame I can't
> utilize it with a DPDK.

Yes, it is not yet supported.
But a patch was recently sent:
	http://dpdk.org/ml/archives/dev/2014-December/010028.html

> I wonder if there are any blockers to add this capability to the ixgbevf
> PMD and if not is it scheduled to some time soon?

Please jump in the above thread for your questions.

My turn to ask a question:
Does it mean DPDK is going to be supported in your system (OSv)?
I've seen it in the OSv roadmap.
Could we have more details about how it would be integrated?

Thanks
-- 
Thomas

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-15 20:33 ` Thomas Monjalon
@ 2014-12-15 20:45   ` Dor Laor
  2014-12-15 21:04     ` [dpdk-dev] DPDK on OSv (was: RSS for ixgbevf) Thomas Monjalon
  2014-12-16 15:36   ` [dpdk-dev] DPDK RSS support for ixgbevf PMD Vlad Zolotarov
  1 sibling, 1 reply; 11+ messages in thread
From: Dor Laor @ 2014-12-15 20:45 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: dev, Vladislav Zolotarov, Avi Kivity, Gleb Natapov

On Mon, Dec 15, 2014 at 10:33 PM, Thomas Monjalon <thomas.monjalon@6wind.com
> wrote:
>
> 2014-12-15 21:11, Vladislav Zolotarov:
> > Hi,
> > I'm running an ixgbevf PMD on an AWS guests with extended networking
> > (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
> > current git tree there is no support for a multi-queue in this PMD: reta
> > size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues
> and
> > max_tx_queues are both 4.
> >
> > Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
> > RSS queues: for some reason it always limits the number of RSS queues by
> 2.
> >
> > ixgbevf_main.c: line 2539
> > u16 rss = min_t(u16, num_online_cpus(), 2);
> >
> > The above is strange since if MRQE is set to 1010b there are 4 RSS queues
> > available which seems to be the case in my AWS Guest.
> >
> > However, let's get back to DPDK. As I've mentioned above the SR-IOV
> > function i have is RSS capable (to be 100% sure I've verified both queues
> > are receiving packets in a multi-socket TCP test). And it's a shame I
> can't
> > utilize it with a DPDK.
>
> Yes, it is not yet supported.
> But a patch was recently sent:
>         http://dpdk.org/ml/archives/dev/2014-December/010028.html


Good to know and nice timing

>
> t
> > I wonder if there are any blockers to add this capability to the ixgbevf
> > PMD and if not is it scheduled to some time soon?
>
> Please jump in the above thread for your questions.
>
> My turn to ask a question:
> Does it mean DPDK is going to be supported in your system (OSv)?
>

In general yes. It's premature to ask now since there are components
that already implemented in OSv and won't be needed (efficient
producer-consumer, huge pages, etc). We sure like to reuse the driver code,
it's a pity not to contribute to the same code base. We'll certainly like
to let unmodified dpdk apps run as well

Initially we're playing w/ the Linux dpdk on the host and AWS to gain
familiarity w/ it


> I've seen it in the OSv roadmap.
> Could we have more details about how it would be integrated?
>
> Thanks
> --
> Thomas
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK on OSv (was: RSS for ixgbevf)
  2014-12-15 20:45   ` Dor Laor
@ 2014-12-15 21:04     ` Thomas Monjalon
  2014-12-15 23:04       ` Dor Laor
  0 siblings, 1 reply; 11+ messages in thread
From: Thomas Monjalon @ 2014-12-15 21:04 UTC (permalink / raw)
  To: Dor Laor; +Cc: dev, Vladislav Zolotarov, Avi Kivity, Gleb Natapov

2014-12-15 22:45, Dor Laor:
> On Mon, Dec 15, 2014 at 10:33 PM, Thomas Monjalon <thomas.monjalon@6wind.com
> > wrote:
> > Does it mean DPDK is going to be supported in your system (OSv)?
> 
> In general yes. It's premature to ask now since there are components
> that already implemented in OSv and won't be needed (efficient
> producer-consumer, huge pages, etc). We sure like to reuse the driver code,
> it's a pity not to contribute to the same code base. We'll certainly like
> to let unmodified dpdk apps run as well

I'm not familiar with OSv but perhaps it has a sense to do an OSv integration
in DPDK.
We already have directories linuxapp and bsdapp for kernel-specific stuff like
memory management, PCI access, etc.

> Initially we're playing w/ the Linux dpdk on the host and AWS to gain
> familiarity w/ it
> 
> > I've seen it in the OSv roadmap.
> > Could we have more details about how it would be integrated?

-- 
Thomas

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK on OSv (was: RSS for ixgbevf)
  2014-12-15 21:04     ` [dpdk-dev] DPDK on OSv (was: RSS for ixgbevf) Thomas Monjalon
@ 2014-12-15 23:04       ` Dor Laor
  0 siblings, 0 replies; 11+ messages in thread
From: Dor Laor @ 2014-12-15 23:04 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: dev, Vladislav Zolotarov, Avi Kivity, Gleb Natapov

On Mon, Dec 15, 2014 at 11:04 PM, Thomas Monjalon <thomas.monjalon@6wind.com
> wrote:
>
> 2014-12-15 22:45, Dor Laor:
> > On Mon, Dec 15, 2014 at 10:33 PM, Thomas Monjalon <
> thomas.monjalon@6wind.com
> > > wrote:
> > > Does it mean DPDK is going to be supported in your system (OSv)?
> >
> > In general yes. It's premature to ask now since there are components
> > that already implemented in OSv and won't be needed (efficient
> > producer-consumer, huge pages, etc). We sure like to reuse the driver
> code,
> > it's a pity not to contribute to the same code base. We'll certainly like
> > to let unmodified dpdk apps run as well
>
> I'm not familiar with OSv but perhaps it has a sense to do an OSv
> integration
> in DPDK.
> We already have directories linuxapp and bsdapp for kernel-specific stuff
> like
> memory management, PCI access, etc.
>

Thanks, it will definetly ease the porting


>
> > Initially we're playing w/ the Linux dpdk on the host and AWS to gain
> > familiarity w/ it
> >
> > > I've seen it in the OSv roadmap.
> > > Could we have more details about how it would be integrated?
>
> --
> Thomas
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-15 20:33 ` Thomas Monjalon
  2014-12-15 20:45   ` Dor Laor
@ 2014-12-16 15:36   ` Vlad Zolotarov
  2014-12-17  1:03     ` Ouyang, Changchun
  1 sibling, 1 reply; 11+ messages in thread
From: Vlad Zolotarov @ 2014-12-16 15:36 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: dev, Avi Kivity, Gleb Natapov


On 12/15/14 22:33, Thomas Monjalon wrote:
> 2014-12-15 21:11, Vladislav Zolotarov:
>> Hi,
>> I'm running an ixgbevf PMD on an AWS guests with extended networking
>> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
>> current git tree there is no support for a multi-queue in this PMD: reta
>> size returned by rte_eth_dev_info_get() call is 0, while max_rx_queues and
>> max_tx_queues are both 4.
>>
>> Linux ixgbevf-2.15.3 driver on the other hand successfully initializes 2
>> RSS queues: for some reason it always limits the number of RSS queues by 2.
>>
>> ixgbevf_main.c: line 2539
>> u16 rss = min_t(u16, num_online_cpus(), 2);
>>
>> The above is strange since if MRQE is set to 1010b there are 4 RSS queues
>> available which seems to be the case in my AWS Guest.
>>
>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
>> function i have is RSS capable (to be 100% sure I've verified both queues
>> are receiving packets in a multi-socket TCP test). And it's a shame I can't
>> utilize it with a DPDK.
> Yes, it is not yet supported.
> But a patch was recently sent:
> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html

Applying this patchset seems to break the NIC fast path functionality of 
a AWS Guest NIC.
I'm still debugging it - will update u as soon as I have more specific info.

vlad

>
>> I wonder if there are any blockers to add this capability to the ixgbevf
>> PMD and if not is it scheduled to some time soon?
> Please jump in the above thread for your questions.
>
> My turn to ask a question:
> Does it mean DPDK is going to be supported in your system (OSv)?
> I've seen it in the OSv roadmap.
> Could we have more details about how it would be integrated?
>
> Thanks

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-16 15:36   ` [dpdk-dev] DPDK RSS support for ixgbevf PMD Vlad Zolotarov
@ 2014-12-17  1:03     ` Ouyang, Changchun
  2014-12-17  8:46       ` Vlad Zolotarov
  0 siblings, 1 reply; 11+ messages in thread
From: Ouyang, Changchun @ 2014-12-17  1:03 UTC (permalink / raw)
  To: Vlad Zolotarov, Thomas Monjalon; +Cc: dev, Avi Kivity, Gleb Natapov

Hi ,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vlad Zolotarov
> Sent: Tuesday, December 16, 2014 11:36 PM
> To: Thomas Monjalon
> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> 
> 
> On 12/15/14 22:33, Thomas Monjalon wrote:
> > 2014-12-15 21:11, Vladislav Zolotarov:
> >> Hi,
> >> I'm running an ixgbevf PMD on an AWS guests with extended networking
> >> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
> >> current git tree there is no support for a multi-queue in this PMD:
> >> reta size returned by rte_eth_dev_info_get() call is 0, while
> >> max_rx_queues and max_tx_queues are both 4.
> >>
> >> Linux ixgbevf-2.15.3 driver on the other hand successfully
> >> initializes 2 RSS queues: for some reason it always limits the number of
> RSS queues by 2.
> >>
> >> ixgbevf_main.c: line 2539
> >> u16 rss = min_t(u16, num_online_cpus(), 2);
> >>
> >> The above is strange since if MRQE is set to 1010b there are 4 RSS
> >> queues available which seems to be the case in my AWS Guest.
> >>
> >> However, let's get back to DPDK. As I've mentioned above the SR-IOV
> >> function i have is RSS capable (to be 100% sure I've verified both
> >> queues are receiving packets in a multi-socket TCP test). And it's a
> >> shame I can't utilize it with a DPDK.
> > Yes, it is not yet supported.
> > But a patch was recently sent:
> > 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
> 
> Applying this patchset seems to break the NIC fast path functionality of a
> AWS Guest NIC.
> I'm still debugging it - will update u as soon as I have more specific info.
> 
Pls make sure enabling and using 4 queues on guest if you enable vf rss on host and if each vf has 4 queues,
The packet could be received and put in any of 4 queues on guest depend on hash value of the packet.
You could let packet IP address increased continuously and make those packets distribute to different queues.

Thanks
Changchun

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-17  1:03     ` Ouyang, Changchun
@ 2014-12-17  8:46       ` Vlad Zolotarov
  2014-12-18  1:32         ` Ouyang, Changchun
  0 siblings, 1 reply; 11+ messages in thread
From: Vlad Zolotarov @ 2014-12-17  8:46 UTC (permalink / raw)
  To: Ouyang, Changchun, Thomas Monjalon; +Cc: dev, Avi Kivity, Gleb Natapov


On 12/17/14 03:03, Ouyang, Changchun wrote:
> Hi ,
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vlad Zolotarov
>> Sent: Tuesday, December 16, 2014 11:36 PM
>> To: Thomas Monjalon
>> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
>>
>>
>> On 12/15/14 22:33, Thomas Monjalon wrote:
>>> 2014-12-15 21:11, Vladislav Zolotarov:
>>>> Hi,
>>>> I'm running an ixgbevf PMD on an AWS guests with extended networking
>>>> (SR-IOV functions of 82599 Intel's NIC) and noticed that even in the
>>>> current git tree there is no support for a multi-queue in this PMD:
>>>> reta size returned by rte_eth_dev_info_get() call is 0, while
>>>> max_rx_queues and max_tx_queues are both 4.
>>>>
>>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
>>>> initializes 2 RSS queues: for some reason it always limits the number of
>> RSS queues by 2.
>>>> ixgbevf_main.c: line 2539
>>>> u16 rss = min_t(u16, num_online_cpus(), 2);
>>>>
>>>> The above is strange since if MRQE is set to 1010b there are 4 RSS
>>>> queues available which seems to be the case in my AWS Guest.
>>>>
>>>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
>>>> function i have is RSS capable (to be 100% sure I've verified both
>>>> queues are receiving packets in a multi-socket TCP test). And it's a
>>>> shame I can't utilize it with a DPDK.
>>> Yes, it is not yet supported.
>>> But a patch was recently sent:
>>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
>> Applying this patchset seems to break the NIC fast path functionality of a
>> AWS Guest NIC.
>> I'm still debugging it - will update u as soon as I have more specific info.

Hi, thanks for tips but I have a question below.

> Pls make sure enabling and using 4 queues on guest

Why should I enable all 4? Won't any number below or equal 4 work if (!) 
4 queues are available?
Note that there is a 82599 mode when only 2 RSS queues are available per 
VF: MRQE=1011b.

>   if you enable vf rss on host and if each vf has 4 queues,
> The packet could be received and put in any of 4 queues on guest depend on hash value of the packet.
> You could let packet IP address increased continuously and make those packets distribute to different queues.
>
> Thanks
> Changchun

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-17  8:46       ` Vlad Zolotarov
@ 2014-12-18  1:32         ` Ouyang, Changchun
  2014-12-18 10:33           ` Vlad Zolotarov
  0 siblings, 1 reply; 11+ messages in thread
From: Ouyang, Changchun @ 2014-12-18  1:32 UTC (permalink / raw)
  To: Vlad Zolotarov, Thomas Monjalon; +Cc: dev, Avi Kivity, Gleb Natapov

Hi

> -----Original Message-----
> From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com]
> Sent: Wednesday, December 17, 2014 4:47 PM
> To: Ouyang, Changchun; Thomas Monjalon
> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> 
> 
> On 12/17/14 03:03, Ouyang, Changchun wrote:
> > Hi ,
> >
> >> -----Original Message-----
> >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vlad Zolotarov
> >> Sent: Tuesday, December 16, 2014 11:36 PM
> >> To: Thomas Monjalon
> >> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
> >> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> >>
> >>
> >> On 12/15/14 22:33, Thomas Monjalon wrote:
> >>> 2014-12-15 21:11, Vladislav Zolotarov:
> >>>> Hi,
> >>>> I'm running an ixgbevf PMD on an AWS guests with extended
> >>>> networking (SR-IOV functions of 82599 Intel's NIC) and noticed that
> >>>> even in the current git tree there is no support for a multi-queue in this
> PMD:
> >>>> reta size returned by rte_eth_dev_info_get() call is 0, while
> >>>> max_rx_queues and max_tx_queues are both 4.
> >>>>
> >>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
> >>>> initializes 2 RSS queues: for some reason it always limits the
> >>>> number of
> >> RSS queues by 2.
> >>>> ixgbevf_main.c: line 2539
> >>>> u16 rss = min_t(u16, num_online_cpus(), 2);
> >>>>
> >>>> The above is strange since if MRQE is set to 1010b there are 4 RSS
> >>>> queues available which seems to be the case in my AWS Guest.
> >>>>
> >>>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
> >>>> function i have is RSS capable (to be 100% sure I've verified both
> >>>> queues are receiving packets in a multi-socket TCP test). And it's
> >>>> a shame I can't utilize it with a DPDK.
> >>> Yes, it is not yet supported.
> >>> But a patch was recently sent:
> >>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
> >> Applying this patchset seems to break the NIC fast path functionality
> >> of a AWS Guest NIC.
> >> I'm still debugging it - will update u as soon as I have more specific info.
> 
> Hi, thanks for tips but I have a question below.
> 
> > Pls make sure enabling and using 4 queues on guest
> 
> Why should I enable all 4? Won't any number below or equal 4 work if (!)
> 4 queues are available?
> Note that there is a 82599 mode when only 2 RSS queues are available per
> VF: MRQE=1011b.
> 

Yes you are right, it could be 2 queues, the actual queue number per VF depends on the number of VF:
VF number from 1~32:     4 queues per VF;
VF number from 33~64:   2 queues per VF;
The queue number is determined by the vf number at the pf init stage,
If pf setup 4 queues for each vf and distribute packets to 4 queues, but vf only poll 1 or 2 queue,
Then packets in other queues will be lost.
You can try to poll 4 queues firstly to debug your issue.
Thanks
Changchun

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-18  1:32         ` Ouyang, Changchun
@ 2014-12-18 10:33           ` Vlad Zolotarov
  2014-12-19  1:17             ` Ouyang, Changchun
  0 siblings, 1 reply; 11+ messages in thread
From: Vlad Zolotarov @ 2014-12-18 10:33 UTC (permalink / raw)
  To: Ouyang, Changchun, Thomas Monjalon; +Cc: dev, Avi Kivity, Gleb Natapov


On 12/18/14 03:32, Ouyang, Changchun wrote:
> Hi
>
>> -----Original Message-----
>> From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com]
>> Sent: Wednesday, December 17, 2014 4:47 PM
>> To: Ouyang, Changchun; Thomas Monjalon
>> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
>>
>>
>> On 12/17/14 03:03, Ouyang, Changchun wrote:
>>> Hi ,
>>>
>>>> -----Original Message-----
>>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vlad Zolotarov
>>>> Sent: Tuesday, December 16, 2014 11:36 PM
>>>> To: Thomas Monjalon
>>>> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
>>>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
>>>>
>>>>
>>>> On 12/15/14 22:33, Thomas Monjalon wrote:
>>>>> 2014-12-15 21:11, Vladislav Zolotarov:
>>>>>> Hi,
>>>>>> I'm running an ixgbevf PMD on an AWS guests with extended
>>>>>> networking (SR-IOV functions of 82599 Intel's NIC) and noticed that
>>>>>> even in the current git tree there is no support for a multi-queue in this
>> PMD:
>>>>>> reta size returned by rte_eth_dev_info_get() call is 0, while
>>>>>> max_rx_queues and max_tx_queues are both 4.
>>>>>>
>>>>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
>>>>>> initializes 2 RSS queues: for some reason it always limits the
>>>>>> number of
>>>> RSS queues by 2.
>>>>>> ixgbevf_main.c: line 2539
>>>>>> u16 rss = min_t(u16, num_online_cpus(), 2);
>>>>>>
>>>>>> The above is strange since if MRQE is set to 1010b there are 4 RSS
>>>>>> queues available which seems to be the case in my AWS Guest.
>>>>>>
>>>>>> However, let's get back to DPDK. As I've mentioned above the SR-IOV
>>>>>> function i have is RSS capable (to be 100% sure I've verified both
>>>>>> queues are receiving packets in a multi-socket TCP test). And it's
>>>>>> a shame I can't utilize it with a DPDK.
>>>>> Yes, it is not yet supported.
>>>>> But a patch was recently sent:
>>>>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
>>>> Applying this patchset seems to break the NIC fast path functionality
>>>> of a AWS Guest NIC.
>>>> I'm still debugging it - will update u as soon as I have more specific info.
>> Hi, thanks for tips but I have a question below.
>>
>>> Pls make sure enabling and using 4 queues on guest
>> Why should I enable all 4? Won't any number below or equal 4 work if (!)
>> 4 queues are available?
>> Note that there is a 82599 mode when only 2 RSS queues are available per
>> VF: MRQE=1011b.
>>
> Yes you are right, it could be 2 queues, the actual queue number per VF depends on the number of VF:
> VF number from 1~32:     4 queues per VF;
> VF number from 33~64:   2 queues per VF;
> The queue number is determined by the vf number at the pf init stage,
> If pf setup 4 queues for each vf and distribute packets to 4 queues, but vf only poll 1 or 2 queue,
> Then packets in other queues will be lost.
> You can try to poll 4 queues firstly to debug your issue.

Sure. When I hardcoded the queues number to 4 all works like a charm. 
The problem is that we can't leave it this way. I've posted some 
comments to your patches (0 and 5). Since I've joined the list after the 
patches have been sent I had to paste the patches bodies into my email 
and they will most like not appear in the thread of the original series. 
I hope this won't be a big problem... ;)

vlad

> Thanks
> Changchun
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
  2014-12-18 10:33           ` Vlad Zolotarov
@ 2014-12-19  1:17             ` Ouyang, Changchun
  0 siblings, 0 replies; 11+ messages in thread
From: Ouyang, Changchun @ 2014-12-19  1:17 UTC (permalink / raw)
  To: Vlad Zolotarov, Thomas Monjalon; +Cc: dev, Avi Kivity, Gleb Natapov

Hi ,

> -----Original Message-----
> From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com]
> Sent: Thursday, December 18, 2014 6:34 PM
> To: Ouyang, Changchun; Thomas Monjalon
> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> 
> 
> On 12/18/14 03:32, Ouyang, Changchun wrote:
> > Hi
> >
> >> -----Original Message-----
> >> From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com]
> >> Sent: Wednesday, December 17, 2014 4:47 PM
> >> To: Ouyang, Changchun; Thomas Monjalon
> >> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
> >> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> >>
> >>
> >> On 12/17/14 03:03, Ouyang, Changchun wrote:
> >>> Hi ,
> >>>
> >>>> -----Original Message-----
> >>>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Vlad
> Zolotarov
> >>>> Sent: Tuesday, December 16, 2014 11:36 PM
> >>>> To: Thomas Monjalon
> >>>> Cc: dev@dpdk.org; Avi Kivity; Gleb Natapov
> >>>> Subject: Re: [dpdk-dev] DPDK RSS support for ixgbevf PMD
> >>>>
> >>>>
> >>>> On 12/15/14 22:33, Thomas Monjalon wrote:
> >>>>> 2014-12-15 21:11, Vladislav Zolotarov:
> >>>>>> Hi,
> >>>>>> I'm running an ixgbevf PMD on an AWS guests with extended
> >>>>>> networking (SR-IOV functions of 82599 Intel's NIC) and noticed
> >>>>>> that even in the current git tree there is no support for a
> >>>>>> multi-queue in this
> >> PMD:
> >>>>>> reta size returned by rte_eth_dev_info_get() call is 0, while
> >>>>>> max_rx_queues and max_tx_queues are both 4.
> >>>>>>
> >>>>>> Linux ixgbevf-2.15.3 driver on the other hand successfully
> >>>>>> initializes 2 RSS queues: for some reason it always limits the
> >>>>>> number of
> >>>> RSS queues by 2.
> >>>>>> ixgbevf_main.c: line 2539
> >>>>>> u16 rss = min_t(u16, num_online_cpus(), 2);
> >>>>>>
> >>>>>> The above is strange since if MRQE is set to 1010b there are 4
> >>>>>> RSS queues available which seems to be the case in my AWS Guest.
> >>>>>>
> >>>>>> However, let's get back to DPDK. As I've mentioned above the
> >>>>>> SR-IOV function i have is RSS capable (to be 100% sure I've
> >>>>>> verified both queues are receiving packets in a multi-socket TCP
> >>>>>> test). And it's a shame I can't utilize it with a DPDK.
> >>>>> Yes, it is not yet supported.
> >>>>> But a patch was recently sent:
> >>>>> 	http://dpdk.org/ml/archives/dev/2014-December/010028.html
> >>>> Applying this patchset seems to break the NIC fast path
> >>>> functionality of a AWS Guest NIC.
> >>>> I'm still debugging it - will update u as soon as I have more specific info.
> >> Hi, thanks for tips but I have a question below.
> >>
> >>> Pls make sure enabling and using 4 queues on guest
> >> Why should I enable all 4? Won't any number below or equal 4 work if
> >> (!)
> >> 4 queues are available?
> >> Note that there is a 82599 mode when only 2 RSS queues are available
> >> per
> >> VF: MRQE=1011b.
> >>
> > Yes you are right, it could be 2 queues, the actual queue number per VF
> depends on the number of VF:
> > VF number from 1~32:     4 queues per VF;
> > VF number from 33~64:   2 queues per VF;
> > The queue number is determined by the vf number at the pf init stage,
> > If pf setup 4 queues for each vf and distribute packets to 4 queues,
> > but vf only poll 1 or 2 queue, Then packets in other queues will be lost.
> > You can try to poll 4 queues firstly to debug your issue.
> 
> Sure. When I hardcoded the queues number to 4 all works like a charm.
> The problem is that we can't leave it this way. I've posted some comments to
> your patches (0 and 5). Since I've joined the list after the patches have been
> sent I had to paste the patches bodies into my email and they will most like
> not appear in the thread of the original series.
> I hope this won't be a big problem... ;)
> 
Yes, I have seen your comments there, and respond it already, let's use that thread to discuss from now on.

Thanks
Changchun

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2014-12-19  1:18 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-12-15 19:11 [dpdk-dev] DPDK RSS support for ixgbevf PMD Vladislav Zolotarov
2014-12-15 20:33 ` Thomas Monjalon
2014-12-15 20:45   ` Dor Laor
2014-12-15 21:04     ` [dpdk-dev] DPDK on OSv (was: RSS for ixgbevf) Thomas Monjalon
2014-12-15 23:04       ` Dor Laor
2014-12-16 15:36   ` [dpdk-dev] DPDK RSS support for ixgbevf PMD Vlad Zolotarov
2014-12-17  1:03     ` Ouyang, Changchun
2014-12-17  8:46       ` Vlad Zolotarov
2014-12-18  1:32         ` Ouyang, Changchun
2014-12-18 10:33           ` Vlad Zolotarov
2014-12-19  1:17             ` Ouyang, Changchun

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).