DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
@ 2015-12-10 10:22 Dey, Souvik
  2015-12-10 17:24 ` Stephen Hemminger
  0 siblings, 1 reply; 8+ messages in thread
From: Dey, Souvik @ 2015-12-10 10:22 UTC (permalink / raw)
  To: dev

Hi,
                In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.

status = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); return a non zero status. Though the normal vmxnet3.ko works fine in the same system. Any idea if anyone has faced this type of issue.

--
Regards,
Souvik

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
  2015-12-10 10:22 [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7 Dey, Souvik
@ 2015-12-10 17:24 ` Stephen Hemminger
  2015-12-14  4:52   ` Dey, Souvik
  0 siblings, 1 reply; 8+ messages in thread
From: Stephen Hemminger @ 2015-12-10 17:24 UTC (permalink / raw)
  To: Dey, Souvik; +Cc: dev

On Thu, 10 Dec 2015 10:22:30 +0000
"Dey, Souvik" <sodey@sonusnet.com> wrote:

> Hi,
>                 In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.

Please try something more recent. DPDK 1.7 had a version of vmxnet3
which was brand new at the time, and had lots of bugs. A lot of collabrative
changes went into later versions. I would recommend 2.0 or later
if possible.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
  2015-12-10 17:24 ` Stephen Hemminger
@ 2015-12-14  4:52   ` Dey, Souvik
  0 siblings, 0 replies; 8+ messages in thread
From: Dey, Souvik @ 2015-12-14  4:52 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: dev

I have also tried with DPDK 2.1 but the result is same. 

-----Original Message-----
From: Stephen Hemminger [mailto:stephen@networkplumber.org] 
Sent: Thursday, December 10, 2015 10:54 PM
To: Dey, Souvik <sodey@sonusnet.com>
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7

On Thu, 10 Dec 2015 10:22:30 +0000
"Dey, Souvik" <sodey@sonusnet.com> wrote:

> Hi,
>                 In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.

Please try something more recent. DPDK 1.7 had a version of vmxnet3 which was brand new at the time, and had lots of bugs. A lot of collabrative changes went into later versions. I would recommend 2.0 or later if possible.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
  2015-12-14  7:06     ` Dey, Souvik
@ 2015-12-22 20:36       ` Yong Wang
  0 siblings, 0 replies; 8+ messages in thread
From: Yong Wang @ 2015-12-22 20:36 UTC (permalink / raw)
  To: Dey, Souvik, dev

I checked with our engineers internally and it turns out that the changes to lower the max ring2 size never got checked-in in 6.0. This should explain why it works on ESXi6.0.  And for 5.5, the plan is to revert the changes in 55p08 patch to bring it back to 4096.




On 12/13/15, 11:06 PM, "Dey, Souvik" <sodey@sonusnet.com> wrote:
>Thanks for the update. Yes I tried with both 1024 and 2048 it worked fine. But interestingly 4096 is working fine in ESXi6.0. Are you aware of some configuration in ESXi which can solve it issue or bring back nb_rx_desc to max of 4096 ?. Or any idea of any fixes from vmware which fixes this issue ?
>
>-----Original Message-----
>From: Yong Wang [mailto:yongwang@vmware.com] 
>Sent: Monday, December 14, 2015 11:57 AM
>To: Dey, Souvik <sodey@sonusnet.com>; dev@dpdk.org
>Subject: Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
>
>nb_rx_desc should be less or equal to 2048 in update 3 due to a change in vmxnet3 backend. More details can be found at http://kb.vmware.com/kb/2136932.
>
>
>On 12/13/15, 9:01 PM, "Dey, Souvik" <sodey@sonusnet.com> wrote:
>
>>Not sure about but it definitely worked in ESXi5.5 Update1, and also prior versions. Also on ESXi6.0 it works fine. 
>>We are using 1rx queue and 8 tx queues per port. The nb_rx_desc  is set to 4096  and the each mbuf size is set to 2048. The rx_conf struct has the following values during the init time .
>>
>>static const struct rte_eth_rxconf rx_conf = {
>>   .rx_thresh = {
>>      .pthresh = RX_PTHRESH,
>>      .hthresh = RX_HTHRESH,
>>      .wthresh = RX_WTHRESH,
>>   },
>>};
>>
>>Do you suspect anything wrong in this ?
>>
>>-----Original Message-----
>>From: Yong Wang [mailto:yongwang@vmware.com] 
>>Sent: Friday, December 11, 2015 12:07 AM
>>To: Dey, Souvik <sodey@sonusnet.com>; dev@dpdk.org
>>Subject: Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
>>
>>On 12/10/15, 2:22 AM, "dev on behalf of Dey, Souvik" <dev-bounces@dpdk.org on behalf of sodey@sonusnet.com> wrote:
>>
>>
>>
>>>Hi,
>>>                In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.
>>>
>>>status = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); return a non zero status. Though the normal vmxnet3.ko works fine in the same system. Any idea if anyone has faced this type of issue.
>>>
>>>--
>>>Regards,
>>>Souvik
>>
>>Did it work for ESXi 5.5 update 2 or some earlier version?
>>
>>Can you also post the rx ring size you used to config the rx queue?
>>In update 3, there are some changes to the max allowed rx ring size for ring1. Even ring1 is not used yet in the pmd, the setup code sets ring1’s size the same as ring0’s.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
  2015-12-14  6:27   ` Yong Wang
@ 2015-12-14  7:06     ` Dey, Souvik
  2015-12-22 20:36       ` Yong Wang
  0 siblings, 1 reply; 8+ messages in thread
From: Dey, Souvik @ 2015-12-14  7:06 UTC (permalink / raw)
  To: Yong Wang, dev

Thanks for the update. Yes I tried with both 1024 and 2048 it worked fine. But interestingly 4096 is working fine in ESXi6.0. Are you aware of some configuration in ESXi which can solve it issue or bring back nb_rx_desc to max of 4096 ?. Or any idea of any fixes from vmware which fixes this issue ?

-----Original Message-----
From: Yong Wang [mailto:yongwang@vmware.com] 
Sent: Monday, December 14, 2015 11:57 AM
To: Dey, Souvik <sodey@sonusnet.com>; dev@dpdk.org
Subject: Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7

nb_rx_desc should be less or equal to 2048 in update 3 due to a change in vmxnet3 backend. More details can be found at http://kb.vmware.com/kb/2136932.


On 12/13/15, 9:01 PM, "Dey, Souvik" <sodey@sonusnet.com> wrote:

>Not sure about but it definitely worked in ESXi5.5 Update1, and also prior versions. Also on ESXi6.0 it works fine. 
>We are using 1rx queue and 8 tx queues per port. The nb_rx_desc  is set to 4096  and the each mbuf size is set to 2048. The rx_conf struct has the following values during the init time .
>
>static const struct rte_eth_rxconf rx_conf = {
>   .rx_thresh = {
>      .pthresh = RX_PTHRESH,
>      .hthresh = RX_HTHRESH,
>      .wthresh = RX_WTHRESH,
>   },
>};
>
>Do you suspect anything wrong in this ?
>
>-----Original Message-----
>From: Yong Wang [mailto:yongwang@vmware.com] 
>Sent: Friday, December 11, 2015 12:07 AM
>To: Dey, Souvik <sodey@sonusnet.com>; dev@dpdk.org
>Subject: Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
>
>On 12/10/15, 2:22 AM, "dev on behalf of Dey, Souvik" <dev-bounces@dpdk.org on behalf of sodey@sonusnet.com> wrote:
>
>
>
>>Hi,
>>                In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.
>>
>>status = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); return a non zero status. Though the normal vmxnet3.ko works fine in the same system. Any idea if anyone has faced this type of issue.
>>
>>--
>>Regards,
>>Souvik
>
>Did it work for ESXi 5.5 update 2 or some earlier version?
>
>Can you also post the rx ring size you used to config the rx queue?
>In update 3, there are some changes to the max allowed rx ring size for ring1. Even ring1 is not used yet in the pmd, the setup code sets ring1’s size the same as ring0’s.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
  2015-12-14  5:01 ` Dey, Souvik
@ 2015-12-14  6:27   ` Yong Wang
  2015-12-14  7:06     ` Dey, Souvik
  0 siblings, 1 reply; 8+ messages in thread
From: Yong Wang @ 2015-12-14  6:27 UTC (permalink / raw)
  To: Dey, Souvik, dev

nb_rx_desc should be less or equal to 2048 in update 3 due to a change in vmxnet3 backend. More details can be found at http://kb.vmware.com/kb/2136932.


On 12/13/15, 9:01 PM, "Dey, Souvik" <sodey@sonusnet.com> wrote:

>Not sure about but it definitely worked in ESXi5.5 Update1, and also prior versions. Also on ESXi6.0 it works fine. 
>We are using 1rx queue and 8 tx queues per port. The nb_rx_desc  is set to 4096  and the each mbuf size is set to 2048. The rx_conf struct has the following values during the init time .
>
>static const struct rte_eth_rxconf rx_conf = {
>   .rx_thresh = {
>      .pthresh = RX_PTHRESH,
>      .hthresh = RX_HTHRESH,
>      .wthresh = RX_WTHRESH,
>   },
>};
>
>Do you suspect anything wrong in this ?
>
>-----Original Message-----
>From: Yong Wang [mailto:yongwang@vmware.com] 
>Sent: Friday, December 11, 2015 12:07 AM
>To: Dey, Souvik <sodey@sonusnet.com>; dev@dpdk.org
>Subject: Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
>
>On 12/10/15, 2:22 AM, "dev on behalf of Dey, Souvik" <dev-bounces@dpdk.org on behalf of sodey@sonusnet.com> wrote:
>
>
>
>>Hi,
>>                In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.
>>
>>status = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); return a non zero status. Though the normal vmxnet3.ko works fine in the same system. Any idea if anyone has faced this type of issue.
>>
>>--
>>Regards,
>>Souvik
>
>Did it work for ESXi 5.5 update 2 or some earlier version?
>
>Can you also post the rx ring size you used to config the rx queue?
>In update 3, there are some changes to the max allowed rx ring size for ring1. Even ring1 is not used yet in the pmd, the setup code sets ring1’s size the same as ring0’s.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
  2015-12-10 18:36 Yong Wang
@ 2015-12-14  5:01 ` Dey, Souvik
  2015-12-14  6:27   ` Yong Wang
  0 siblings, 1 reply; 8+ messages in thread
From: Dey, Souvik @ 2015-12-14  5:01 UTC (permalink / raw)
  To: Yong Wang, dev

Not sure about but it definitely worked in ESXi5.5 Update1, and also prior versions. Also on ESXi6.0 it works fine. 
We are using 1rx queue and 8 tx queues per port. The nb_rx_desc  is set to 4096  and the each mbuf size is set to 2048. The rx_conf struct has the following values during the init time .

static const struct rte_eth_rxconf rx_conf = {
   .rx_thresh = {
      .pthresh = RX_PTHRESH,
      .hthresh = RX_HTHRESH,
      .wthresh = RX_WTHRESH,
   },
};

Do you suspect anything wrong in this ?

-----Original Message-----
From: Yong Wang [mailto:yongwang@vmware.com] 
Sent: Friday, December 11, 2015 12:07 AM
To: Dey, Souvik <sodey@sonusnet.com>; dev@dpdk.org
Subject: Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7

On 12/10/15, 2:22 AM, "dev on behalf of Dey, Souvik" <dev-bounces@dpdk.org on behalf of sodey@sonusnet.com> wrote:



>Hi,
>                In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.
>
>status = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); return a non zero status. Though the normal vmxnet3.ko works fine in the same system. Any idea if anyone has faced this type of issue.
>
>--
>Regards,
>Souvik

Did it work for ESXi 5.5 update 2 or some earlier version?

Can you also post the rx ring size you used to config the rx queue?
In update 3, there are some changes to the max allowed rx ring size for ring1. Even ring1 is not used yet in the pmd, the setup code sets ring1’s size the same as ring0’s.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7
@ 2015-12-10 18:36 Yong Wang
  2015-12-14  5:01 ` Dey, Souvik
  0 siblings, 1 reply; 8+ messages in thread
From: Yong Wang @ 2015-12-10 18:36 UTC (permalink / raw)
  To: Dey, Souvik, dev

On 12/10/15, 2:22 AM, "dev on behalf of Dey, Souvik" <dev-bounces@dpdk.org on behalf of sodey@sonusnet.com> wrote:



>Hi,
>                In DPDK 1.7 , while using the vmxnet3 pmd on vmware Esxi 5.5 update 3 we are seeing that activation of the device fails.
>
>status = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD); return a non zero status. Though the normal vmxnet3.ko works fine in the same system. Any idea if anyone has faced this type of issue.
>
>--
>Regards,
>Souvik

Did it work for ESXi 5.5 update 2 or some earlier version?

Can you also post the rx ring size you used to config the rx queue?
In update 3, there are some changes to the max allowed rx ring size
for ring1. Even ring1 is not used yet in the pmd, the setup code
sets ring1’s size the same as ring0’s.

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2015-12-22 20:36 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-12-10 10:22 [dpdk-dev] Vmxnet3 activation of device fails in DPDK1.7 Dey, Souvik
2015-12-10 17:24 ` Stephen Hemminger
2015-12-14  4:52   ` Dey, Souvik
2015-12-10 18:36 Yong Wang
2015-12-14  5:01 ` Dey, Souvik
2015-12-14  6:27   ` Yong Wang
2015-12-14  7:06     ` Dey, Souvik
2015-12-22 20:36       ` Yong Wang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).