Re: [dpdk-dev] vmxnet3 pmd dev restart

[Navakanth M <navakanthdev@gmail.com>]

Thanks a lot guys for your support.
I have tried the patch by Yong Wang, stop and start functions are
working fine with out any crash.
I have noticed that tx/rx stats are being reset to 0. I am not sure if
this is valid behavior as we do not reset stats to 0 when we do
ifconfig down and ifconfig up.

Thanks
Navakanth


On Mon, Oct 13, 2014 at 11:42 PM, Patel, Rashmin N
<rashmin.n.patel@intel.com> wrote:
> Waterman/Navakanth, we've got some patches posted for the same by Yong Wang@VMware. I haven't got the chance to look at it but if you can validate it, it'd be great.
>
> Thanks,
> Rashmin
>
> -----Original Message-----
> From: Navakanth M [mailto:navakanthdev@gmail.com]
> Sent: Sunday, October 12, 2014 8:07 PM
> To: Patel, Rashmin N; Cao, Waterman
> Cc: stephen@networkplumber.org; dev@dpdk.org; Jiajia, SunX
> Subject: Re: vmxnet3 pmd dev restart
>
> Hi Rashmin
>
> I have tried the memset change but still I am facing the problem which I pointed out earlier. After restart, packets are not being received in vmxnet3_recv_pkts().
> I have also observed PANIC in vmxnet3_tq_tx_complete() after couple of stop and start operations.
>
> PANIC in vmxnet3_tq_tx_complete():
> EOP desc does not point to a valid mbuf15:
> [/lib64/libc.so.6(clone+0x6d) [0x7fd60354c52d]]
> 1: [/mswitch/bin/sos.shumway.elf(rte_dump_stack+0x23) [0x463313]]
> 2: [/mswitch/bin/sos.shumway.elf(__rte_panic+0xc1) [0x447ae8]]
> 3: [/mswitch/bin/sos.shumway.elf(vmxnet3_xmit_pkts+0x382) [0x4f4f22]]
>
> Thanks
> Navakanth
>
> On Fri, Oct 10, 2014 at 8:39 AM, Cao, Waterman <waterman.cao@intel.com> wrote:
>> Hi Rashmin,
>>
>>  We found similar issue when we start/stop vmnet dev several time. (>
>> 3 times)  It happens kernel panic, and sometimes kernel will occur core dump.
>>  Let me know if you want to submit patch to fix it.
>>
>> Thanks
>> Waterman
>>
>> -----Original Message-----
>>>From: Patel, Rashmin N
>>>Sent: Friday, October 10, 2014 6:07 AM
>>>To: Navakanth M; stephen@networkplumber.org; Cao, Waterman
>>>Cc: dev@dpdk.org
>>>Subject: RE: vmxnet3 pmd dev restart
>>>
>>>I just quickly looked into the code and instead of releasing memory or simply set it to NULL (patch:
>>> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/4683), you can zero it out and it should work perfectly, you can give it a quick try.
>>>
>>>//rte_free(ring->buf_info);
>>>memset(ring->buf_info, 0x0, ring->size*sizeof(vmxnet3_buf_info_t));
>>>
>>>This will not free the memory from heap but just wipe it out to 0x0, provided that we freed all the mbuf(s) pointed by each buf_info->m pointers. Hence you won't need to reallocate it when you start device after this stop.
>>>
>>>Thanks,
>>>Rashmin
>>>
>>>-----Original Message-----
>>>From: Navakanth M [mailto:navakanthdev@gmail.com]
>>>Sent: Wednesday, October 08, 2014 10:11 PM
>>>To: stephen@networkplumber.org; Patel, Rashmin N; Cao, Waterman
>>>Cc: dev@dpdk.org
>>>Subject: Re: vmxnet3 pmd dev restart
>>>
>>>I had tried with Stephen's patch but after stop is done and when we
>>>call start it crashed at vmxnet3_dev_start()->
>>>vmxnet3_dev_rxtx_init()->vmxnet3_post_rx_bufs() as buf_info is freed
>>>and is not allocated again. buf_info is allocated in
>>>vmxnet3_dev_rx_queue_setup() which would be called once at the initialization only.
>>>I also tried not freeing buf_info in stop but then i see different
>>>issue after start, packets are not received due to check while
>>>(rcd->gen == rxq->comp_ring.gen) { in vmxnet3_recv_pkts()
>>>
>>>Waterman, Have you got chance to test stop and start of vmnet dev if so did you notice any issue similar to this?
>>>
>>>Thanks
>>>Navakanth
>>>
>>>On Thu, Oct 9, 2014 at 12:46 AM, Patel, Rashmin N <rashmin.n.patel@intel.com> wrote:
>>>> Yes I had a local copy working with couple of lines fix. But someone else, I think Stephen added a fix patch for the same, and I assume if it's been merged, should be working, so did not follow up later.
>>>>
>>>> I don't have a VMware setup handy at moment but I think Waterman would have more information about testing that patch if he has found any issue with it.
>>>>
>>>> Thanks,
>>>> Rashmin
>>>>
>>>> -----Original Message-----
>>>> From: Navakanth M [mailto:navakanthdev@gmail.com]
>>>> Sent: Wednesday, October 08, 2014 4:14 AM
>>>> To: dev@dpdk.org; Patel, Rashmin N
>>>> Subject: Re: vmxnet3 pmd dev restart
>>>>
>>>> Hi Rashmin
>>>>
>>>> I have come across your reply in following post that you have worked on this problem and would submit the patch for it.
>>>> Can you please share information on the changes you worked on or patch log if you had submitted any for it?
>>>> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/4683
>>>>
>>>> Thanks
>>>> Navakanth
>>>>
>>>> On Tue, Sep 30, 2014 at 1:44 PM, Navakanth M <navakanthdev@gmail.com> wrote:
>>>>> Hi
>>>>>
>>>>> I am using DPDKv1.7.0 running on Vmware Esxi 5.1 and am trying to
>>>>> reset the port which uses pmd_vmnet3 library functions from below
>>>>> function calls.
>>>>> rte_eth_dev_stop
>>>>> rte_eth_dev_start
>>>>>
>>>>> Doing this, i face panic while rte_free(ring->buf_info) in
>>>>> Vmxnet3_cmd_ring_release().
>>>>> I have gone through following thread but the patch mentioned didn't
>>>>> help rather it crashed in start function while accessing buf_info
>>>>> in vmxnet3_post_rx_bufs. I see this buf_info is allocated in queue
>>>>> setup functions which are called at initialization.
>>>>> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/4683
>>>>>
>>>>> I tried not freeing it and then rx packets are not received due to
>>>>> mismatch in while (rcd->gen == rxq->comp_ring.gen) in
>>>>> vmxnet3_recv_pkts()
>>>>>
>>>>> To reset the device port, is this the right way what i am doing?
>>>>> Or do I have to call vmxnet3_dev_tx_queue_setup()
>>>>> vmxnet3_dev_rx_queue_setup() once stop is called? I have checked
>>>>> recent patches and threads but did not get much information on this.
>>>>>
>>>>> Thanks
>>>>> Navakanth