Re: [dpdk-dev] [PATCH] cast used->idx to volatile

["Xie, Huawei" <huawei.xie@intel.com>]

On 3/24/2015 9:00 AM, Linhaifeng wrote:
>
> On 2015/3/23 20:54, Xie, Huawei wrote:
>>
>>> -----Original Message-----
>>> From: Linhaifeng [mailto:haifeng.lin@huawei.com]
>>> Sent: Monday, March 23, 2015 8:24 PM
>>> To: dev@dpdk.org
>>> Cc: Ouyang, Changchun; Xie, Huawei
>>> Subject: Re: [dpdk-dev] [PATCH] cast used->idx to volatile
>>>
>>>
>>>
>>> On 2015/3/21 16:07, linhaifeng wrote:
>>>> From: Linhaifeng <haifeng.lin@huawei.com>
>>>>
>>>> Same as rte_vhost_enqueue_burst we should cast used->idx
>>>> to volatile before notify guest.
>>>>
>>>> Signed-off-by: Linhaifeng <haifeng.lin@huawei.com>
>>>> ---
>>>>  lib/librte_vhost/vhost_rxtx.c | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/lib/librte_vhost/vhost_rxtx.c b/lib/librte_vhost/vhost_rxtx.c
>>>> index 535c7a1..8d674d1 100644
>>>> --- a/lib/librte_vhost/vhost_rxtx.c
>>>> +++ b/lib/librte_vhost/vhost_rxtx.c
>>>> @@ -722,7 +722,7 @@ rte_vhost_dequeue_burst(struct virtio_net *dev,
>>> uint16_t queue_id,
>>>>  	}
>>>>
>>>>  	rte_compiler_barrier();
>>>> -	vq->used->idx += entry_success;
>>>> +	*(volatile uint16_t *)&vq->used->idx += entry_success;
>>
>> Haifeng:
>> We have compiler barrier before and an external function call behind, so we don't need volatile  here.
>> Do you meet issue?
>>
> Tx_q is sometimes stopped when we use virtio_net. Because vhost thought there are no buffers in tx_q and virtio_net
> though vhost haven't handle all packets so we have to restart VM to restore work.
>
> The status in VM is:
> Mar 18 17:11:10 linux-b2ij kernel: [46337.246687] net eth7: virtnet_poll
> Mar 18 17:11:10 linux-b2ij kernel: [46337.246690] net eth7: receive_buf
> Mar 18 17:11:10 linux-b2ij kernel: [46337.246693] net eth7: vi->num=239
> Mar 18 17:11:10 linux-b2ij kernel: [46337.246695] net eth7: svq:avail->idx=52939 used->idx=52939 num_free=18 num_added=0 svq->last_used_idx=52820
> Mar 18 17:11:10 linux-b2ij kernel: [46337.246699] net eth7: rvq:avail->idx=36215 used->idx=35977 num_free=18 num_added=0 rvq->last_used_idx=35977
> Mar 18 17:11:11 linux-b2ij kernel: [46337.901038] net eth7: dev_queue_xmit, qdisc->flags=4, qdisc->state deactiveed=0
> Mar 18 17:11:12 linux-b2ij kernel: [46337.901042] net eth7: dev_queue_xmit, txq->state=1, stopped=1
>
> Why compiler barrier not take effect in our case? Is compiler barrier depended on -O3 option? We use -O2 option.
compiler barrier always works regardless of the optimization option.
I don't get your story, but the key thing is, do you check the asm code?
If called from outside as an API, how is it possible it is optimized?
there is only one update to used->idx in that function.
>
>>>>  	/* Kick guest if required. */
>>>>  	if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT))
>>>>  		eventfd_write((int)vq->callfd, 1);
>>>>
>>> --
>>> Regards,
>>> Haifeng
>>
>>