DPDK patches and discussions
 help / color / mirror / Atom feed
* Re: [dpdk-dev] [PATCH] vhost: fix vhost user virtqueue not accessable
  2019-10-25 16:20 [dpdk-dev] [PATCH] vhost: fix vhost user virtqueue not accessable Marvin Liu
@ 2019-10-25 12:20 ` Adrian Moreno
  2019-10-28  2:13   ` Liu, Yong
  2019-10-30 11:07 ` [dpdk-dev] [PATCH v2] vhost: fix vhost user virtqueue not accessible Marvin Liu
  1 sibling, 1 reply; 16+ messages in thread
From: Adrian Moreno @ 2019-10-25 12:20 UTC (permalink / raw)
  To: Marvin Liu, maxime.coquelin, tiwei.bie, zhihong.wang; +Cc: dev

Hi Marvin,

On 10/25/19 6:20 PM, Marvin Liu wrote:
> Log feature is disabled in vhost user, so that log address was invalid
> when checking. Add feature bit check can skip useless address check.
>
Just so I understand, what conditions is the log address invalid?

> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> 
> Signed-off-by: Marvin Liu <yong.liu@intel.com>
> ---
>  lib/librte_vhost/vhost_user.c | 16 +++++++++-------
>  1 file changed, 9 insertions(+), 7 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 61ef699ac..0407fdc29 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -741,13 +741,15 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  		vq->last_avail_idx = vq->used->idx;
>  	}
>  
> -	vq->log_guest_addr =
> -		translate_log_addr(dev, vq, addr->log_guest_addr);
> -	if (vq->log_guest_addr == 0) {
> -		RTE_LOG(DEBUG, VHOST_CONFIG,
> -			"(%d) failed to map log_guest_addr .\n",
> -			dev->vid);
> -		return dev;
> +	if (dev->features & (1ULL << VHOST_F_LOG_ALL)) {
> +		vq->log_guest_addr =
> +			translate_log_addr(dev, vq, addr->log_guest_addr);

VHOST_F_LOG_ALL is only negotiated once the migration has started (at least from
qemu's perspective).
That means that we will postponing the translation  of the log address to the
vhost_user_set_vring_addr() call that follows the VHOST_F_LOG_ALL enabling. In
that call there are (at least) two things that could go wrong and lead to a
migration failure:
- If VHOST_USER_F_PROTOCOL_FEATURES is not enabled, the address won't be translated:

vhost_user:795
	if ((vq->enabled && (dev->features &
				(1ULL << VHOST_USER_F_PROTOCOL_FEATURES))) ||
			access_ok) {
		dev = translate_ring_addresses(dev, msg->payload.addr.index);
		if (!dev)
			return RTE_VHOST_MSG_RESULT_ERR;

		*pdev = dev;
	}

- If the IOMMU is enabled and there's a miss, we would have to wait for the
IOTLB_UPDATE and during that time, there would be failed accesses to the (still
untranslated) log address.



> +		if (vq->log_guest_addr == 0) {
> +			RTE_LOG(DEBUG, VHOST_CONFIG,
> +				"(%d) failed to map log_guest_addr .\n",
> +				dev->vid);
> +			return dev;
> +		}
>  	}
>  	vq->access_ok = 1;
>  
> 
Thanks,
Adrian

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dpdk-dev] [PATCH] vhost: fix vhost user virtqueue not accessable
@ 2019-10-25 16:20 Marvin Liu
  2019-10-25 12:20 ` Adrian Moreno
  2019-10-30 11:07 ` [dpdk-dev] [PATCH v2] vhost: fix vhost user virtqueue not accessible Marvin Liu
  0 siblings, 2 replies; 16+ messages in thread
From: Marvin Liu @ 2019-10-25 16:20 UTC (permalink / raw)
  To: maxime.coquelin, tiwei.bie, zhihong.wang, amorenoz; +Cc: dev, Marvin Liu

Log feature is disabled in vhost user, so that log address was invalid
when checking. Add feature bit check can skip useless address check.

Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")

Signed-off-by: Marvin Liu <yong.liu@intel.com>
---
 lib/librte_vhost/vhost_user.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 61ef699ac..0407fdc29 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -741,13 +741,15 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 		vq->last_avail_idx = vq->used->idx;
 	}
 
-	vq->log_guest_addr =
-		translate_log_addr(dev, vq, addr->log_guest_addr);
-	if (vq->log_guest_addr == 0) {
-		RTE_LOG(DEBUG, VHOST_CONFIG,
-			"(%d) failed to map log_guest_addr .\n",
-			dev->vid);
-		return dev;
+	if (dev->features & (1ULL << VHOST_F_LOG_ALL)) {
+		vq->log_guest_addr =
+			translate_log_addr(dev, vq, addr->log_guest_addr);
+		if (vq->log_guest_addr == 0) {
+			RTE_LOG(DEBUG, VHOST_CONFIG,
+				"(%d) failed to map log_guest_addr .\n",
+				dev->vid);
+			return dev;
+		}
 	}
 	vq->access_ok = 1;
 
-- 
2.17.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH] vhost: fix vhost user virtqueue not accessable
  2019-10-25 12:20 ` Adrian Moreno
@ 2019-10-28  2:13   ` Liu, Yong
  2019-10-29 10:02     ` Adrian Moreno
  0 siblings, 1 reply; 16+ messages in thread
From: Liu, Yong @ 2019-10-28  2:13 UTC (permalink / raw)
  To: Adrian Moreno; +Cc: dev, maxime.coquelin, Bie, Tiwei, Wang, Zhihong



> -----Original Message-----
> From: Adrian Moreno [mailto:amorenoz@redhat.com]
> Sent: Friday, October 25, 2019 8:21 PM
> To: Liu, Yong <yong.liu@intel.com>; maxime.coquelin@redhat.com; Bie, Tiwei
> <tiwei.bie@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [PATCH] vhost: fix vhost user virtqueue not accessable
> 
> Hi Marvin,
> 
> On 10/25/19 6:20 PM, Marvin Liu wrote:
> > Log feature is disabled in vhost user, so that log address was invalid
> > when checking. Add feature bit check can skip useless address check.
> >
> Just so I understand, what conditions is the log address invalid?
> 
> > Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> >
> > Signed-off-by: Marvin Liu <yong.liu@intel.com>
> > ---
> >  lib/librte_vhost/vhost_user.c | 16 +++++++++-------
> >  1 file changed, 9 insertions(+), 7 deletions(-)
> >
> > diff --git a/lib/librte_vhost/vhost_user.c
> b/lib/librte_vhost/vhost_user.c
> > index 61ef699ac..0407fdc29 100644
> > --- a/lib/librte_vhost/vhost_user.c
> > +++ b/lib/librte_vhost/vhost_user.c
> > @@ -741,13 +741,15 @@ translate_ring_addresses(struct virtio_net *dev,
> int vq_index)
> >  		vq->last_avail_idx = vq->used->idx;
> >  	}
> >
> > -	vq->log_guest_addr =
> > -		translate_log_addr(dev, vq, addr->log_guest_addr);
> > -	if (vq->log_guest_addr == 0) {
> > -		RTE_LOG(DEBUG, VHOST_CONFIG,
> > -			"(%d) failed to map log_guest_addr .\n",
> > -			dev->vid);
> > -		return dev;
> > +	if (dev->features & (1ULL << VHOST_F_LOG_ALL)) {
> > +		vq->log_guest_addr =
> > +			translate_log_addr(dev, vq, addr->log_guest_addr);
> 
> VHOST_F_LOG_ALL is only negotiated once the migration has started (at least
> from
> qemu's perspective).
> That means that we will postponing the translation  of the log address to
> the
> vhost_user_set_vring_addr() call that follows the VHOST_F_LOG_ALL enabling.
> In
> that call there are (at least) two things that could go wrong and lead to a
> migration failure:
> - If VHOST_USER_F_PROTOCOL_FEATURES is not enabled, the address won't be
> translated:
> 
> vhost_user:795
> 	if ((vq->enabled && (dev->features &
> 				(1ULL << VHOST_USER_F_PROTOCOL_FEATURES))) ||
> 			access_ok) {
> 		dev = translate_ring_addresses(dev, msg->payload.addr.index);
> 		if (!dev)
> 			return RTE_VHOST_MSG_RESULT_ERR;
> 
> 		*pdev = dev;
> 	}
> 
> - If the IOMMU is enabled and there's a miss, we would have to wait for the
> IOTLB_UPDATE and during that time, there would be failed accesses to the
> (still
> untranslated) log address.
> 
> 

Thanks, Adrian. 
Log address can be zero when logging is not enabled.
How about add other criteria after translation? Log address will be translated anyway and not affect vq status.

        vq->log_guest_addr =
                translate_log_addr(dev, vq, addr->log_guest_addr);
-       if (vq->log_guest_addr == 0) {
+       if (vq->log_guest_addr == 0 && addr->flags) {
                RTE_LOG(DEBUG, VHOST_CONFIG,
                        "(%d) failed to map log_guest_addr .\n",
                        dev->vid);
                return dev;
        }

Meanwhile, log address of packed ring is fixed to zero. Is any special reason to do that? 

Regards,
Marvin

> 
> > +		if (vq->log_guest_addr == 0) {
> > +			RTE_LOG(DEBUG, VHOST_CONFIG,
> > +				"(%d) failed to map log_guest_addr .\n",
> > +				dev->vid);
> > +			return dev;
> > +		}
> >  	}
> >  	vq->access_ok = 1;
> >
> >
> Thanks,
> Adrian

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH] vhost: fix vhost user virtqueue not accessable
  2019-10-28  2:13   ` Liu, Yong
@ 2019-10-29 10:02     ` Adrian Moreno
  0 siblings, 0 replies; 16+ messages in thread
From: Adrian Moreno @ 2019-10-29 10:02 UTC (permalink / raw)
  To: Liu, Yong; +Cc: dev, maxime.coquelin, Bie, Tiwei, Wang, Zhihong

On 10/28/19 3:13 AM, Liu, Yong wrote:
> 
> 
>> -----Original Message-----
>> From: Adrian Moreno [mailto:amorenoz@redhat.com]
>> Sent: Friday, October 25, 2019 8:21 PM
>> To: Liu, Yong <yong.liu@intel.com>; maxime.coquelin@redhat.com; Bie, Tiwei
>> <tiwei.bie@intel.com>; Wang, Zhihong <zhihong.wang@intel.com>
>> Cc: dev@dpdk.org
>> Subject: Re: [PATCH] vhost: fix vhost user virtqueue not accessable
>>
>> Hi Marvin,
>>
>> On 10/25/19 6:20 PM, Marvin Liu wrote:
>>> Log feature is disabled in vhost user, so that log address was invalid
>>> when checking. Add feature bit check can skip useless address check.
>>>
>> Just so I understand, what conditions is the log address invalid?
>>
>>> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
>>>
>>> Signed-off-by: Marvin Liu <yong.liu@intel.com>
>>> ---
>>>  lib/librte_vhost/vhost_user.c | 16 +++++++++-------
>>>  1 file changed, 9 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/lib/librte_vhost/vhost_user.c
>> b/lib/librte_vhost/vhost_user.c
>>> index 61ef699ac..0407fdc29 100644
>>> --- a/lib/librte_vhost/vhost_user.c
>>> +++ b/lib/librte_vhost/vhost_user.c
>>> @@ -741,13 +741,15 @@ translate_ring_addresses(struct virtio_net *dev,
>> int vq_index)
>>>  		vq->last_avail_idx = vq->used->idx;
>>>  	}
>>>
>>> -	vq->log_guest_addr =
>>> -		translate_log_addr(dev, vq, addr->log_guest_addr);
>>> -	if (vq->log_guest_addr == 0) {
>>> -		RTE_LOG(DEBUG, VHOST_CONFIG,
>>> -			"(%d) failed to map log_guest_addr .\n",
>>> -			dev->vid);
>>> -		return dev;
>>> +	if (dev->features & (1ULL << VHOST_F_LOG_ALL)) {
>>> +		vq->log_guest_addr =
>>> +			translate_log_addr(dev, vq, addr->log_guest_addr);
>>
>> VHOST_F_LOG_ALL is only negotiated once the migration has started (at least
>> from
>> qemu's perspective).
>> That means that we will postponing the translation  of the log address to
>> the
>> vhost_user_set_vring_addr() call that follows the VHOST_F_LOG_ALL enabling.
>> In
>> that call there are (at least) two things that could go wrong and lead to a
>> migration failure:
>> - If VHOST_USER_F_PROTOCOL_FEATURES is not enabled, the address won't be
>> translated:
>>
>> vhost_user:795
>> 	if ((vq->enabled && (dev->features &
>> 				(1ULL << VHOST_USER_F_PROTOCOL_FEATURES))) ||
>> 			access_ok) {
>> 		dev = translate_ring_addresses(dev, msg->payload.addr.index);
>> 		if (!dev)
>> 			return RTE_VHOST_MSG_RESULT_ERR;
>>
>> 		*pdev = dev;
>> 	}
>>
>> - If the IOMMU is enabled and there's a miss, we would have to wait for the
>> IOTLB_UPDATE and during that time, there would be failed accesses to the
>> (still
>> untranslated) log address.
>>
>>
> 
> Thanks, Adrian. 
> Log address can be zero when logging is not enabled.
> How about add other criteria after translation? Log address will be translated anyway and not affect vq status.
> 
>         vq->log_guest_addr =
>                 translate_log_addr(dev, vq, addr->log_guest_addr);
> -       if (vq->log_guest_addr == 0) {
> +       if (vq->log_guest_addr == 0 && addr->flags) {
>                 RTE_LOG(DEBUG, VHOST_CONFIG,
>                         "(%d) failed to map log_guest_addr .\n",
>                         dev->vid);
>                 return dev;
>         }
> 
That makes perfect sense IMHO. Thank you.

> Meanwhile, log address of packed ring is fixed to zero. Is any special reason to do that? 
> 
No idea to be honest.

Thanks.
Adrian
> Regards,
> Marvin
> 
>>
>>> +		if (vq->log_guest_addr == 0) {
>>> +			RTE_LOG(DEBUG, VHOST_CONFIG,
>>> +				"(%d) failed to map log_guest_addr .\n",
>>> +				dev->vid);
>>> +			return dev;
>>> +		}
>>>  	}
>>>  	vq->access_ok = 1;
>>>
>>>
>> Thanks,
>> Adrian


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v2] vhost: fix vhost user virtqueue not accessible
  2019-10-30 11:07 ` [dpdk-dev] [PATCH v2] vhost: fix vhost user virtqueue not accessible Marvin Liu
@ 2019-10-30  6:58   ` Tiwei Bie
  2019-10-30 14:56   ` [dpdk-dev] [PATCH v3] " Marvin Liu
  1 sibling, 0 replies; 16+ messages in thread
From: Tiwei Bie @ 2019-10-30  6:58 UTC (permalink / raw)
  To: Marvin Liu; +Cc: maxime.coquelin, zhihong.wang, amorenoz, dev

On Wed, Oct 30, 2019 at 07:07:23PM +0800, Marvin Liu wrote:
> Log feature is disabled in vhost user, so that log address was invalid
> when checking. Check whether log address is valid can workaround it.
> Also log address should be translated in packed ring virtqueue.
> 
> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> 
> Signed-off-by: Marvin Liu <yong.liu@intel.com>
> ---
>  lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
>  1 file changed, 13 insertions(+), 17 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 61ef699ac..759cc795c 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  	struct vhost_vring_addr *addr = &vq->ring_addrs;
>  	uint64_t len, expected_len;
>  
> +	dev = numa_realloc(dev, vq_index);
> +	vq = dev->virtqueue[vq_index];
> +	if (addr->log_guest_addr) {

Please use VHOST_VRING_F_LOG to check whether log address
is valid or not.

Thanks,
Tiwei

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dpdk-dev] [PATCH v2] vhost: fix vhost user virtqueue not accessible
  2019-10-25 16:20 [dpdk-dev] [PATCH] vhost: fix vhost user virtqueue not accessable Marvin Liu
  2019-10-25 12:20 ` Adrian Moreno
@ 2019-10-30 11:07 ` Marvin Liu
  2019-10-30  6:58   ` Tiwei Bie
  2019-10-30 14:56   ` [dpdk-dev] [PATCH v3] " Marvin Liu
  1 sibling, 2 replies; 16+ messages in thread
From: Marvin Liu @ 2019-10-30 11:07 UTC (permalink / raw)
  To: maxime.coquelin, tiwei.bie, zhihong.wang, amorenoz; +Cc: dev, Marvin Liu

Log feature is disabled in vhost user, so that log address was invalid
when checking. Check whether log address is valid can workaround it.
Also log address should be translated in packed ring virtqueue.

Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")

Signed-off-by: Marvin Liu <yong.liu@intel.com>
---
 lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
 1 file changed, 13 insertions(+), 17 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 61ef699ac..759cc795c 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 	struct vhost_vring_addr *addr = &vq->ring_addrs;
 	uint64_t len, expected_len;
 
+	dev = numa_realloc(dev, vq_index);
+	vq = dev->virtqueue[vq_index];
+	if (addr->log_guest_addr) {
+		vq->log_guest_addr =
+			translate_log_addr(dev, vq, addr->log_guest_addr);
+		if (vq->log_guest_addr == 0) {
+			RTE_LOG(DEBUG, VHOST_CONFIG,
+					"(%d) failed to map log_guest_addr.\n",
+					dev->vid);
+			return dev;
+		}
+	}
+
 	if (vq_is_packed(dev)) {
 		len = sizeof(struct vring_packed_desc) * vq->size;
 		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
 			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
-		vq->log_guest_addr = 0;
 		if (vq->desc_packed == NULL ||
 				len != sizeof(struct vring_packed_desc) *
 				vq->size) {
@@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 			return dev;
 		}
 
-		dev = numa_realloc(dev, vq_index);
-		vq = dev->virtqueue[vq_index];
-		addr = &vq->ring_addrs;
-
 		len = sizeof(struct vring_packed_desc_event);
 		vq->driver_event = (struct vring_packed_desc_event *)
 					(uintptr_t)ring_addr_to_vva(dev,
@@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 		return dev;
 	}
 
-	dev = numa_realloc(dev, vq_index);
-	vq = dev->virtqueue[vq_index];
-	addr = &vq->ring_addrs;
-
 	len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
 	if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
 		len += sizeof(uint16_t);
@@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 		vq->last_avail_idx = vq->used->idx;
 	}
 
-	vq->log_guest_addr =
-		translate_log_addr(dev, vq, addr->log_guest_addr);
-	if (vq->log_guest_addr == 0) {
-		RTE_LOG(DEBUG, VHOST_CONFIG,
-			"(%d) failed to map log_guest_addr .\n",
-			dev->vid);
-		return dev;
-	}
 	vq->access_ok = 1;
 
 	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
-- 
2.17.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dpdk-dev] [PATCH v3] vhost: fix vhost user virtqueue not accessible
  2019-10-30 11:07 ` [dpdk-dev] [PATCH v2] vhost: fix vhost user virtqueue not accessible Marvin Liu
  2019-10-30  6:58   ` Tiwei Bie
@ 2019-10-30 14:56   ` Marvin Liu
  2019-10-31 10:42     ` Tiwei Bie
  2019-11-04 10:13     ` [dpdk-dev] [PATCH v4] " Marvin Liu
  1 sibling, 2 replies; 16+ messages in thread
From: Marvin Liu @ 2019-10-30 14:56 UTC (permalink / raw)
  To: maxime.coquelin, tiwei.bie, zhihong.wang, amorenoz; +Cc: dev, Marvin Liu

Log feature is disabled in vhost user, so that log address was invalid
when checking. Check whether log address is valid can workaround it.
Also log address should be translated in packed ring virtqueue.

Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")

Signed-off-by: Marvin Liu <yong.liu@intel.com>
---
 lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
 1 file changed, 13 insertions(+), 17 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 61ef699ac..7754d2467 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 	struct vhost_vring_addr *addr = &vq->ring_addrs;
 	uint64_t len, expected_len;
 
+	dev = numa_realloc(dev, vq_index);
+	vq = dev->virtqueue[vq_index];
+	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
+		vq->log_guest_addr =
+			translate_log_addr(dev, vq, addr->log_guest_addr);
+		if (vq->log_guest_addr == 0) {
+			RTE_LOG(DEBUG, VHOST_CONFIG,
+					"(%d) failed to map log_guest_addr.\n",
+					dev->vid);
+			return dev;
+		}
+	}
+
 	if (vq_is_packed(dev)) {
 		len = sizeof(struct vring_packed_desc) * vq->size;
 		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
 			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
-		vq->log_guest_addr = 0;
 		if (vq->desc_packed == NULL ||
 				len != sizeof(struct vring_packed_desc) *
 				vq->size) {
@@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 			return dev;
 		}
 
-		dev = numa_realloc(dev, vq_index);
-		vq = dev->virtqueue[vq_index];
-		addr = &vq->ring_addrs;
-
 		len = sizeof(struct vring_packed_desc_event);
 		vq->driver_event = (struct vring_packed_desc_event *)
 					(uintptr_t)ring_addr_to_vva(dev,
@@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 		return dev;
 	}
 
-	dev = numa_realloc(dev, vq_index);
-	vq = dev->virtqueue[vq_index];
-	addr = &vq->ring_addrs;
-
 	len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
 	if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
 		len += sizeof(uint16_t);
@@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 		vq->last_avail_idx = vq->used->idx;
 	}
 
-	vq->log_guest_addr =
-		translate_log_addr(dev, vq, addr->log_guest_addr);
-	if (vq->log_guest_addr == 0) {
-		RTE_LOG(DEBUG, VHOST_CONFIG,
-			"(%d) failed to map log_guest_addr .\n",
-			dev->vid);
-		return dev;
-	}
 	vq->access_ok = 1;
 
 	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
-- 
2.17.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v3] vhost: fix vhost user virtqueue not accessible
  2019-10-30 14:56   ` [dpdk-dev] [PATCH v3] " Marvin Liu
@ 2019-10-31 10:42     ` Tiwei Bie
  2019-10-31 14:54       ` Liu, Yong
  2019-11-04 10:13     ` [dpdk-dev] [PATCH v4] " Marvin Liu
  1 sibling, 1 reply; 16+ messages in thread
From: Tiwei Bie @ 2019-10-31 10:42 UTC (permalink / raw)
  To: Marvin Liu; +Cc: maxime.coquelin, zhihong.wang, amorenoz, dev

On Wed, Oct 30, 2019 at 10:56:02PM +0800, Marvin Liu wrote:
> Log feature is disabled in vhost user, so that log address was invalid
> when checking. Check whether log address is valid can workaround it.
> Also log address should be translated in packed ring virtqueue.
> 
> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> 
> Signed-off-by: Marvin Liu <yong.liu@intel.com>
> ---
>  lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
>  1 file changed, 13 insertions(+), 17 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 61ef699ac..7754d2467 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  	struct vhost_vring_addr *addr = &vq->ring_addrs;
>  	uint64_t len, expected_len;
>  
> +	dev = numa_realloc(dev, vq_index);

We need to update `vq->desc` first before doing numa_realloc.
https://github.com/DPDK/dpdk/blob/19397c7bf2545e6adab41b657a1f1da3c7344e7b/lib/librte_vhost/vhost_user.c#L445

> +	vq = dev->virtqueue[vq_index];
> +	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {

`vq` can be reallocated by numa_realloc.
We need to update the `addr` pointer before using it.

Thanks,
Tiwei


> +		vq->log_guest_addr =
> +			translate_log_addr(dev, vq, addr->log_guest_addr);
> +		if (vq->log_guest_addr == 0) {
> +			RTE_LOG(DEBUG, VHOST_CONFIG,
> +					"(%d) failed to map log_guest_addr.\n",
> +					dev->vid);
> +			return dev;
> +		}
> +	}
> +
>  	if (vq_is_packed(dev)) {
>  		len = sizeof(struct vring_packed_desc) * vq->size;
>  		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
>  			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
> -		vq->log_guest_addr = 0;
>  		if (vq->desc_packed == NULL ||
>  				len != sizeof(struct vring_packed_desc) *
>  				vq->size) {
> @@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  			return dev;
>  		}
>  
> -		dev = numa_realloc(dev, vq_index);
> -		vq = dev->virtqueue[vq_index];
> -		addr = &vq->ring_addrs;
> -
>  		len = sizeof(struct vring_packed_desc_event);
>  		vq->driver_event = (struct vring_packed_desc_event *)
>  					(uintptr_t)ring_addr_to_vva(dev,
> @@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  		return dev;
>  	}
>  
> -	dev = numa_realloc(dev, vq_index);
> -	vq = dev->virtqueue[vq_index];
> -	addr = &vq->ring_addrs;
> -
>  	len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
>  	if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
>  		len += sizeof(uint16_t);
> @@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  		vq->last_avail_idx = vq->used->idx;
>  	}
>  
> -	vq->log_guest_addr =
> -		translate_log_addr(dev, vq, addr->log_guest_addr);
> -	if (vq->log_guest_addr == 0) {
> -		RTE_LOG(DEBUG, VHOST_CONFIG,
> -			"(%d) failed to map log_guest_addr .\n",
> -			dev->vid);
> -		return dev;
> -	}
>  	vq->access_ok = 1;
>  
>  	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v3] vhost: fix vhost user virtqueue not accessible
  2019-10-31 10:42     ` Tiwei Bie
@ 2019-10-31 14:54       ` Liu, Yong
  2019-10-31 17:47         ` Adrian Moreno
  0 siblings, 1 reply; 16+ messages in thread
From: Liu, Yong @ 2019-10-31 14:54 UTC (permalink / raw)
  To: Bie, Tiwei; +Cc: maxime.coquelin, Wang, Zhihong, amorenoz, dev



> -----Original Message-----
> From: Bie, Tiwei
> Sent: Thursday, October 31, 2019 6:42 PM
> To: Liu, Yong <yong.liu@intel.com>
> Cc: maxime.coquelin@redhat.com; Wang, Zhihong <zhihong.wang@intel.com>;
> amorenoz@redhat.com; dev@dpdk.org
> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
> 
> On Wed, Oct 30, 2019 at 10:56:02PM +0800, Marvin Liu wrote:
> > Log feature is disabled in vhost user, so that log address was invalid
> > when checking. Check whether log address is valid can workaround it.
> > Also log address should be translated in packed ring virtqueue.
> >
> > Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> >
> > Signed-off-by: Marvin Liu <yong.liu@intel.com>
> > ---
> >  lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
> >  1 file changed, 13 insertions(+), 17 deletions(-)
> >
> > diff --git a/lib/librte_vhost/vhost_user.c
> b/lib/librte_vhost/vhost_user.c
> > index 61ef699ac..7754d2467 100644
> > --- a/lib/librte_vhost/vhost_user.c
> > +++ b/lib/librte_vhost/vhost_user.c
> > @@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev,
> int vq_index)
> >  	struct vhost_vring_addr *addr = &vq->ring_addrs;
> >  	uint64_t len, expected_len;
> >
> > +	dev = numa_realloc(dev, vq_index);
> 
> We need to update `vq->desc` first before doing numa_realloc.
> https://github.com/DPDK/dpdk/blob/19397c7bf2545e6adab41b657a1f1da3c7344e7b/
> lib/librte_vhost/vhost_user.c#L445
> 
> > +	vq = dev->virtqueue[vq_index];
> > +	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
> 
> `vq` can be reallocated by numa_realloc.
> We need to update the `addr` pointer before using it.
> 

Hi Tiwei,
Numa_realloc function will copy data from original vq structure to new vq when reallocating.
The content of vhost_ring_addr will be the same in new and old vqs, it may not be necessary to update pointer.

Regards,
Marvin

> Thanks,
> Tiwei
> 
> 
> > +		vq->log_guest_addr =
> > +			translate_log_addr(dev, vq, addr->log_guest_addr);
> > +		if (vq->log_guest_addr == 0) {
> > +			RTE_LOG(DEBUG, VHOST_CONFIG,
> > +					"(%d) failed to map log_guest_addr.\n",
> > +					dev->vid);
> > +			return dev;
> > +		}
> > +	}
> > +
> >  	if (vq_is_packed(dev)) {
> >  		len = sizeof(struct vring_packed_desc) * vq->size;
> >  		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
> >  			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
> > -		vq->log_guest_addr = 0;
> >  		if (vq->desc_packed == NULL ||
> >  				len != sizeof(struct vring_packed_desc) *
> >  				vq->size) {
> > @@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev, int
> vq_index)
> >  			return dev;
> >  		}
> >
> > -		dev = numa_realloc(dev, vq_index);
> > -		vq = dev->virtqueue[vq_index];
> > -		addr = &vq->ring_addrs;
> > -
> >  		len = sizeof(struct vring_packed_desc_event);
> >  		vq->driver_event = (struct vring_packed_desc_event *)
> >  					(uintptr_t)ring_addr_to_vva(dev,
> > @@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev, int
> vq_index)
> >  		return dev;
> >  	}
> >
> > -	dev = numa_realloc(dev, vq_index);
> > -	vq = dev->virtqueue[vq_index];
> > -	addr = &vq->ring_addrs;
> > -
> >  	len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
> >  	if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
> >  		len += sizeof(uint16_t);
> > @@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev, int
> vq_index)
> >  		vq->last_avail_idx = vq->used->idx;
> >  	}
> >
> > -	vq->log_guest_addr =
> > -		translate_log_addr(dev, vq, addr->log_guest_addr);
> > -	if (vq->log_guest_addr == 0) {
> > -		RTE_LOG(DEBUG, VHOST_CONFIG,
> > -			"(%d) failed to map log_guest_addr .\n",
> > -			dev->vid);
> > -		return dev;
> > -	}
> >  	vq->access_ok = 1;
> >
> >  	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
> > --
> > 2.17.1
> >

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v3] vhost: fix vhost user virtqueue not accessible
  2019-10-31 14:54       ` Liu, Yong
@ 2019-10-31 17:47         ` Adrian Moreno
  2019-11-01  7:16           ` Liu, Yong
  0 siblings, 1 reply; 16+ messages in thread
From: Adrian Moreno @ 2019-10-31 17:47 UTC (permalink / raw)
  To: Liu, Yong, Bie, Tiwei; +Cc: maxime.coquelin, Wang, Zhihong, dev

Hi Marvin,

On 10/31/19 3:54 PM, Liu, Yong wrote:
> 
> 
>> -----Original Message-----
>> From: Bie, Tiwei
>> Sent: Thursday, October 31, 2019 6:42 PM
>> To: Liu, Yong <yong.liu@intel.com>
>> Cc: maxime.coquelin@redhat.com; Wang, Zhihong <zhihong.wang@intel.com>;
>> amorenoz@redhat.com; dev@dpdk.org
>> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
>>
>> On Wed, Oct 30, 2019 at 10:56:02PM +0800, Marvin Liu wrote:
>>> Log feature is disabled in vhost user, so that log address was invalid
>>> when checking. Check whether log address is valid can workaround it.
>>> Also log address should be translated in packed ring virtqueue.
>>>
>>> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
>>>
>>> Signed-off-by: Marvin Liu <yong.liu@intel.com>
>>> ---
>>>  lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
>>>  1 file changed, 13 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/lib/librte_vhost/vhost_user.c
>> b/lib/librte_vhost/vhost_user.c
>>> index 61ef699ac..7754d2467 100644
>>> --- a/lib/librte_vhost/vhost_user.c
>>> +++ b/lib/librte_vhost/vhost_user.c
>>> @@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev,
>> int vq_index)
>>>  	struct vhost_vring_addr *addr = &vq->ring_addrs;
>>>  	uint64_t len, expected_len;
>>>
>>> +	dev = numa_realloc(dev, vq_index);
>>
>> We need to update `vq->desc` first before doing numa_realloc.
>> https://github.com/DPDK/dpdk/blob/19397c7bf2545e6adab41b657a1f1da3c7344e7b/
>> lib/librte_vhost/vhost_user.c#L445
>>
>>> +	vq = dev->virtqueue[vq_index];
>>> +	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
>>
I fear the possible consequences of this change.
Before 04cfc7fdbfca the approach was "best-effort". The log address would be
assigned without further checks:

	vq->log_guest_addr = addr->log_guest_addr;

Then, the behavior changed and an error was generated if the log address was
invalid, which I guess is the problem you have hit:

	vq->log_guest_addr =
		translate_log_addr(dev, vq, addr->log_guest_addr);
	if (vq->log_guest_addr == 0) {
		RTE_LOG(DEBUG, VHOST_CONFIG,
			"(%d) failed to map log_guest_addr .\n",
			dev->vid);
		return dev;
	}

In the tests I ran I always saw valid log addresses being sent at ring
initialization phase, but if, as you claim, it's possible that invalid addresses
are given at initialization phase, maybe we should go back to "best-effort"
(i.e: remove the return statement)

But it's unlikely that qemu has enabled logging at ring initialization so this
would effectively disable the translation at the initialization phase. I cannot
forecast the consequences of this change without deeper analysis.

>> `vq` can be reallocated by numa_realloc.
>> We need to update the `addr` pointer before using it.
>>
> 
> Hi Tiwei,
> Numa_realloc function will copy data from original vq structure to new vq when reallocating.
> The content of vhost_ring_addr will be the same in new and old vqs, it may not be necessary to update pointer.
That's true but 'addr' still holds a pointer to the old structure, assigned at
line 641.

Also, note Tiwei's comment regarding updating 'vq->desc'. The idea behind
numa_realloc is to reallocate the vhost_virtqueue structure to the same numa
node as the descriptor ring. This function is updating the descriptor rings, so
I think the idea is to update the ring addresses and then reallocate the
virtqueue structure if needed.

Thanks,
Adrian

> Regards,
> Marvin
> 
>> Thanks,
>> Tiwei
>>
>>
>>> +		vq->log_guest_addr =
>>> +			translate_log_addr(dev, vq, addr->log_guest_addr);
>>> +		if (vq->log_guest_addr == 0) {
>>> +			RTE_LOG(DEBUG, VHOST_CONFIG,
>>> +					"(%d) failed to map log_guest_addr.\n",
>>> +					dev->vid);
>>> +			return dev;
>>> +		}
>>> +	}
>>> +
>>>  	if (vq_is_packed(dev)) {
>>>  		len = sizeof(struct vring_packed_desc) * vq->size;
>>>  		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
>>>  			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
>>> -		vq->log_guest_addr = 0;
>>>  		if (vq->desc_packed == NULL ||
>>>  				len != sizeof(struct vring_packed_desc) *
>>>  				vq->size) {
>>> @@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev, int
>> vq_index)
>>>  			return dev;
>>>  		}
>>>
>>> -		dev = numa_realloc(dev, vq_index);
>>> -		vq = dev->virtqueue[vq_index];
>>> -		addr = &vq->ring_addrs;
>>> -
>>>  		len = sizeof(struct vring_packed_desc_event);
>>>  		vq->driver_event = (struct vring_packed_desc_event *)
>>>  					(uintptr_t)ring_addr_to_vva(dev,
>>> @@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev, int
>> vq_index)
>>>  		return dev;
>>>  	}
>>>
>>> -	dev = numa_realloc(dev, vq_index);
>>> -	vq = dev->virtqueue[vq_index];
>>> -	addr = &vq->ring_addrs;
>>> -
>>>  	len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
>>>  	if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
>>>  		len += sizeof(uint16_t);
>>> @@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev, int
>> vq_index)
>>>  		vq->last_avail_idx = vq->used->idx;
>>>  	}
>>>
>>> -	vq->log_guest_addr =
>>> -		translate_log_addr(dev, vq, addr->log_guest_addr);
>>> -	if (vq->log_guest_addr == 0) {
>>> -		RTE_LOG(DEBUG, VHOST_CONFIG,
>>> -			"(%d) failed to map log_guest_addr .\n",
>>> -			dev->vid);
>>> -		return dev;
>>> -	}
>>>  	vq->access_ok = 1;
>>>
>>>  	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
>>> --
>>> 2.17.1
>>>


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v3] vhost: fix vhost user virtqueue not accessible
  2019-10-31 17:47         ` Adrian Moreno
@ 2019-11-01  7:16           ` Liu, Yong
  2019-11-04  8:44             ` Adrian Moreno
  0 siblings, 1 reply; 16+ messages in thread
From: Liu, Yong @ 2019-11-01  7:16 UTC (permalink / raw)
  To: Adrian Moreno, Bie, Tiwei; +Cc: maxime.coquelin, Wang, Zhihong, dev



> -----Original Message-----
> From: Adrian Moreno [mailto:amorenoz@redhat.com]
> Sent: Friday, November 01, 2019 1:48 AM
> To: Liu, Yong <yong.liu@intel.com>; Bie, Tiwei <tiwei.bie@intel.com>
> Cc: maxime.coquelin@redhat.com; Wang, Zhihong <zhihong.wang@intel.com>;
> dev@dpdk.org
> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
> 
> Hi Marvin,
> 
> On 10/31/19 3:54 PM, Liu, Yong wrote:
> >
> >
> >> -----Original Message-----
> >> From: Bie, Tiwei
> >> Sent: Thursday, October 31, 2019 6:42 PM
> >> To: Liu, Yong <yong.liu@intel.com>
> >> Cc: maxime.coquelin@redhat.com; Wang, Zhihong <zhihong.wang@intel.com>;
> >> amorenoz@redhat.com; dev@dpdk.org
> >> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
> >>
> >> On Wed, Oct 30, 2019 at 10:56:02PM +0800, Marvin Liu wrote:
> >>> Log feature is disabled in vhost user, so that log address was invalid
> >>> when checking. Check whether log address is valid can workaround it.
> >>> Also log address should be translated in packed ring virtqueue.
> >>>
> >>> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> >>>
> >>> Signed-off-by: Marvin Liu <yong.liu@intel.com>
> >>> ---
> >>>  lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
> >>>  1 file changed, 13 insertions(+), 17 deletions(-)
> >>>
> >>> diff --git a/lib/librte_vhost/vhost_user.c
> >> b/lib/librte_vhost/vhost_user.c
> >>> index 61ef699ac..7754d2467 100644
> >>> --- a/lib/librte_vhost/vhost_user.c
> >>> +++ b/lib/librte_vhost/vhost_user.c
> >>> @@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev,
> >> int vq_index)
> >>>  	struct vhost_vring_addr *addr = &vq->ring_addrs;
> >>>  	uint64_t len, expected_len;
> >>>
> >>> +	dev = numa_realloc(dev, vq_index);
> >>
> >> We need to update `vq->desc` first before doing numa_realloc.
> >>
> https://github.com/DPDK/dpdk/blob/19397c7bf2545e6adab41b657a1f1da3c7344e7b/
> >> lib/librte_vhost/vhost_user.c#L445
> >>
> >>> +	vq = dev->virtqueue[vq_index];
> >>> +	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
> >>
> I fear the possible consequences of this change.
> Before 04cfc7fdbfca the approach was "best-effort". The log address would
> be
> assigned without further checks:
> 
> 	vq->log_guest_addr = addr->log_guest_addr;
> 
> Then, the behavior changed and an error was generated if the log address
> was
> invalid, which I guess is the problem you have hit:
> 
> 	vq->log_guest_addr =
> 		translate_log_addr(dev, vq, addr->log_guest_addr);
> 	if (vq->log_guest_addr == 0) {
> 		RTE_LOG(DEBUG, VHOST_CONFIG,
> 			"(%d) failed to map log_guest_addr .\n",
> 			dev->vid);
> 		return dev;
> 	}
> 
> In the tests I ran I always saw valid log addresses being sent at ring
> initialization phase, but if, as you claim, it's possible that invalid
> addresses
> are given at initialization phase, maybe we should go back to "best-effort"
> (i.e: remove the return statement)
> 
> But it's unlikely that qemu has enabled logging at ring initialization so
> this
> would effectively disable the translation at the initialization phase. I
> cannot
> forecast the consequences of this change without deeper analysis.

That's fine, Adrian. This issue only occurred when using dpdk virtio user device.
Since address logging is disabled in virtio user, simple flag check fix it.

> 
> >> `vq` can be reallocated by numa_realloc.
> >> We need to update the `addr` pointer before using it.
> >>
> >
> > Hi Tiwei,
> > Numa_realloc function will copy data from original vq structure to new vq
> when reallocating.
> > The content of vhost_ring_addr will be the same in new and old vqs, it
> may not be necessary to update pointer.
> That's true but 'addr' still holds a pointer to the old structure, assigned
> at
> line 641.
> 
> Also, note Tiwei's comment regarding updating 'vq->desc'. The idea behind
> numa_realloc is to reallocate the vhost_virtqueue structure to the same
> numa
> node as the descriptor ring. This function is updating the descriptor rings,
> so
> I think the idea is to update the ring addresses and then reallocate the
> virtqueue structure if needed.
> 
You are right, I misunderstood tiwei's comment. The ring address is useful for 
checking numa id, numa reallocation should be after address translation.

Regards,
Marvin

> Thanks,
> Adrian
> 
> > Regards,
> > Marvin
> >
> >> Thanks,
> >> Tiwei
> >>
> >>
> >>> +		vq->log_guest_addr =
> >>> +			translate_log_addr(dev, vq, addr->log_guest_addr);
> >>> +		if (vq->log_guest_addr == 0) {
> >>> +			RTE_LOG(DEBUG, VHOST_CONFIG,
> >>> +					"(%d) failed to map log_guest_addr.\n",
> >>> +					dev->vid);
> >>> +			return dev;
> >>> +		}
> >>> +	}
> >>> +
> >>>  	if (vq_is_packed(dev)) {
> >>>  		len = sizeof(struct vring_packed_desc) * vq->size;
> >>>  		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
> >>>  			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
> >>> -		vq->log_guest_addr = 0;
> >>>  		if (vq->desc_packed == NULL ||
> >>>  				len != sizeof(struct vring_packed_desc) *
> >>>  				vq->size) {
> >>> @@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev,
> int
> >> vq_index)
> >>>  			return dev;
> >>>  		}
> >>>
> >>> -		dev = numa_realloc(dev, vq_index);
> >>> -		vq = dev->virtqueue[vq_index];
> >>> -		addr = &vq->ring_addrs;
> >>> -
> >>>  		len = sizeof(struct vring_packed_desc_event);
> >>>  		vq->driver_event = (struct vring_packed_desc_event *)
> >>>  					(uintptr_t)ring_addr_to_vva(dev,
> >>> @@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev,
> int
> >> vq_index)
> >>>  		return dev;
> >>>  	}
> >>>
> >>> -	dev = numa_realloc(dev, vq_index);
> >>> -	vq = dev->virtqueue[vq_index];
> >>> -	addr = &vq->ring_addrs;
> >>> -
> >>>  	len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
> >>>  	if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
> >>>  		len += sizeof(uint16_t);
> >>> @@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev,
> int
> >> vq_index)
> >>>  		vq->last_avail_idx = vq->used->idx;
> >>>  	}
> >>>
> >>> -	vq->log_guest_addr =
> >>> -		translate_log_addr(dev, vq, addr->log_guest_addr);
> >>> -	if (vq->log_guest_addr == 0) {
> >>> -		RTE_LOG(DEBUG, VHOST_CONFIG,
> >>> -			"(%d) failed to map log_guest_addr .\n",
> >>> -			dev->vid);
> >>> -		return dev;
> >>> -	}
> >>>  	vq->access_ok = 1;
> >>>
> >>>  	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
> >>> --
> >>> 2.17.1
> >>>


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v3] vhost: fix vhost user virtqueue not accessible
  2019-11-01  7:16           ` Liu, Yong
@ 2019-11-04  8:44             ` Adrian Moreno
  0 siblings, 0 replies; 16+ messages in thread
From: Adrian Moreno @ 2019-11-04  8:44 UTC (permalink / raw)
  To: Liu, Yong, Bie, Tiwei; +Cc: maxime.coquelin, Wang, Zhihong, dev

On 11/1/19 8:16 AM, Liu, Yong wrote:
> 
> 
>> -----Original Message-----
>> From: Adrian Moreno [mailto:amorenoz@redhat.com]
>> Sent: Friday, November 01, 2019 1:48 AM
>> To: Liu, Yong <yong.liu@intel.com>; Bie, Tiwei <tiwei.bie@intel.com>
>> Cc: maxime.coquelin@redhat.com; Wang, Zhihong <zhihong.wang@intel.com>;
>> dev@dpdk.org
>> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
>>
>> Hi Marvin,
>>
>> On 10/31/19 3:54 PM, Liu, Yong wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Bie, Tiwei
>>>> Sent: Thursday, October 31, 2019 6:42 PM
>>>> To: Liu, Yong <yong.liu@intel.com>
>>>> Cc: maxime.coquelin@redhat.com; Wang, Zhihong <zhihong.wang@intel.com>;
>>>> amorenoz@redhat.com; dev@dpdk.org
>>>> Subject: Re: [PATCH v3] vhost: fix vhost user virtqueue not accessible
>>>>
>>>> On Wed, Oct 30, 2019 at 10:56:02PM +0800, Marvin Liu wrote:
>>>>> Log feature is disabled in vhost user, so that log address was invalid
>>>>> when checking. Check whether log address is valid can workaround it.
>>>>> Also log address should be translated in packed ring virtqueue.
>>>>>
>>>>> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
>>>>>
>>>>> Signed-off-by: Marvin Liu <yong.liu@intel.com>
>>>>> ---
>>>>>  lib/librte_vhost/vhost_user.c | 30 +++++++++++++-----------------
>>>>>  1 file changed, 13 insertions(+), 17 deletions(-)
>>>>>
>>>>> diff --git a/lib/librte_vhost/vhost_user.c
>>>> b/lib/librte_vhost/vhost_user.c
>>>>> index 61ef699ac..7754d2467 100644
>>>>> --- a/lib/librte_vhost/vhost_user.c
>>>>> +++ b/lib/librte_vhost/vhost_user.c
>>>>> @@ -641,11 +641,23 @@ translate_ring_addresses(struct virtio_net *dev,
>>>> int vq_index)
>>>>>  	struct vhost_vring_addr *addr = &vq->ring_addrs;
>>>>>  	uint64_t len, expected_len;
>>>>>
>>>>> +	dev = numa_realloc(dev, vq_index);
>>>>
>>>> We need to update `vq->desc` first before doing numa_realloc.
>>>>
>> https://github.com/DPDK/dpdk/blob/19397c7bf2545e6adab41b657a1f1da3c7344e7b/
>>>> lib/librte_vhost/vhost_user.c#L445
>>>>
>>>>> +	vq = dev->virtqueue[vq_index];
>>>>> +	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
>>>>
>> I fear the possible consequences of this change.
>> Before 04cfc7fdbfca the approach was "best-effort". The log address would
>> be
>> assigned without further checks:
>>
>> 	vq->log_guest_addr = addr->log_guest_addr;
>>
>> Then, the behavior changed and an error was generated if the log address
>> was
>> invalid, which I guess is the problem you have hit:
>>
>> 	vq->log_guest_addr =
>> 		translate_log_addr(dev, vq, addr->log_guest_addr);
>> 	if (vq->log_guest_addr == 0) {
>> 		RTE_LOG(DEBUG, VHOST_CONFIG,
>> 			"(%d) failed to map log_guest_addr .\n",
>> 			dev->vid);
>> 		return dev;
>> 	}
>>
>> In the tests I ran I always saw valid log addresses being sent at ring
>> initialization phase, but if, as you claim, it's possible that invalid
>> addresses
>> are given at initialization phase, maybe we should go back to "best-effort"
>> (i.e: remove the return statement)
>>
>> But it's unlikely that qemu has enabled logging at ring initialization so
>> this
>> would effectively disable the translation at the initialization phase. I
>> cannot
>> forecast the consequences of this change without deeper analysis.
> 
> That's fine, Adrian. This issue only occurred when using dpdk virtio user device.
> Since address logging is disabled in virtio user, simple flag check fix it.
> 
What I meant is that we would be changing the behavior for the vhost-user case
in which qemu sends a valid log address even when logging is still not enabled.
In that case we would be effectively postponing the translation.

In any case, I've run a quick test and that case still works fine.

Thanks,
Adrian
>>
>>>> `vq` can be reallocated by numa_realloc.
>>>> We need to update the `addr` pointer before using it.
>>>>
>>>
>>> Hi Tiwei,
>>> Numa_realloc function will copy data from original vq structure to new vq
>> when reallocating.
>>> The content of vhost_ring_addr will be the same in new and old vqs, it
>> may not be necessary to update pointer.
>> That's true but 'addr' still holds a pointer to the old structure, assigned
>> at
>> line 641.
>>
>> Also, note Tiwei's comment regarding updating 'vq->desc'. The idea behind
>> numa_realloc is to reallocate the vhost_virtqueue structure to the same
>> numa
>> node as the descriptor ring. This function is updating the descriptor rings,
>> so
>> I think the idea is to update the ring addresses and then reallocate the
>> virtqueue structure if needed.
>>
> You are right, I misunderstood tiwei's comment. The ring address is useful for 
> checking numa id, numa reallocation should be after address translation.
> 
> Regards,
> Marvin
> 
>> Thanks,
>> Adrian
>>
>>> Regards,
>>> Marvin
>>>
>>>> Thanks,
>>>> Tiwei
>>>>
>>>>
>>>>> +		vq->log_guest_addr =
>>>>> +			translate_log_addr(dev, vq, addr->log_guest_addr);
>>>>> +		if (vq->log_guest_addr == 0) {
>>>>> +			RTE_LOG(DEBUG, VHOST_CONFIG,
>>>>> +					"(%d) failed to map log_guest_addr.\n",
>>>>> +					dev->vid);
>>>>> +			return dev;
>>>>> +		}
>>>>> +	}
>>>>> +
>>>>>  	if (vq_is_packed(dev)) {
>>>>>  		len = sizeof(struct vring_packed_desc) * vq->size;
>>>>>  		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
>>>>>  			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
>>>>> -		vq->log_guest_addr = 0;
>>>>>  		if (vq->desc_packed == NULL ||
>>>>>  				len != sizeof(struct vring_packed_desc) *
>>>>>  				vq->size) {
>>>>> @@ -655,10 +667,6 @@ translate_ring_addresses(struct virtio_net *dev,
>> int
>>>> vq_index)
>>>>>  			return dev;
>>>>>  		}
>>>>>
>>>>> -		dev = numa_realloc(dev, vq_index);
>>>>> -		vq = dev->virtqueue[vq_index];
>>>>> -		addr = &vq->ring_addrs;
>>>>> -
>>>>>  		len = sizeof(struct vring_packed_desc_event);
>>>>>  		vq->driver_event = (struct vring_packed_desc_event *)
>>>>>  					(uintptr_t)ring_addr_to_vva(dev,
>>>>> @@ -701,10 +709,6 @@ translate_ring_addresses(struct virtio_net *dev,
>> int
>>>> vq_index)
>>>>>  		return dev;
>>>>>  	}
>>>>>
>>>>> -	dev = numa_realloc(dev, vq_index);
>>>>> -	vq = dev->virtqueue[vq_index];
>>>>> -	addr = &vq->ring_addrs;
>>>>> -
>>>>>  	len = sizeof(struct vring_avail) + sizeof(uint16_t) * vq->size;
>>>>>  	if (dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))
>>>>>  		len += sizeof(uint16_t);
>>>>> @@ -741,14 +745,6 @@ translate_ring_addresses(struct virtio_net *dev,
>> int
>>>> vq_index)
>>>>>  		vq->last_avail_idx = vq->used->idx;
>>>>>  	}
>>>>>
>>>>> -	vq->log_guest_addr =
>>>>> -		translate_log_addr(dev, vq, addr->log_guest_addr);
>>>>> -	if (vq->log_guest_addr == 0) {
>>>>> -		RTE_LOG(DEBUG, VHOST_CONFIG,
>>>>> -			"(%d) failed to map log_guest_addr .\n",
>>>>> -			dev->vid);
>>>>> -		return dev;
>>>>> -	}
>>>>>  	vq->access_ok = 1;
>>>>>
>>>>>  	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
>>>>> --
>>>>> 2.17.1
>>>>>
> 


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v4] vhost: fix vhost user virtqueue not accessible
  2019-11-04 10:13     ` [dpdk-dev] [PATCH v4] " Marvin Liu
@ 2019-11-04  8:47       ` Adrian Moreno
  2019-11-06 20:16         ` Maxime Coquelin
  2019-11-06 20:59       ` Maxime Coquelin
  1 sibling, 1 reply; 16+ messages in thread
From: Adrian Moreno @ 2019-11-04  8:47 UTC (permalink / raw)
  To: Marvin Liu, maxime.coquelin, tiwei.bie, zhihong.wang; +Cc: dev

On 11/4/19 11:13 AM, Marvin Liu wrote:
> Log feature is disabled in vhost user, so that log address was invalid
> when checking. Check whether log address is valid can work around it.
> Log address should also be translated in packed ring virtqueue.
> 
> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> 
> Signed-off-by: Marvin Liu <yong.liu@intel.com>
> ---
>  lib/librte_vhost/vhost_user.c | 20 +++++++++++---------
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
> index 61ef699ac..09e241305 100644
> --- a/lib/librte_vhost/vhost_user.c
> +++ b/lib/librte_vhost/vhost_user.c
> @@ -641,11 +641,21 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  	struct vhost_vring_addr *addr = &vq->ring_addrs;
>  	uint64_t len, expected_len;
>  
> +	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
> +		vq->log_guest_addr =
> +			translate_log_addr(dev, vq, addr->log_guest_addr);
> +		if (vq->log_guest_addr == 0) {
> +			RTE_LOG(DEBUG, VHOST_CONFIG,
> +				"(%d) failed to map log_guest_addr.\n",
> +				dev->vid);
> +			return dev;
> +		}
> +	}
> +
>  	if (vq_is_packed(dev)) {
>  		len = sizeof(struct vring_packed_desc) * vq->size;
>  		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
>  			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
> -		vq->log_guest_addr = 0;
>  		if (vq->desc_packed == NULL ||
>  				len != sizeof(struct vring_packed_desc) *
>  				vq->size) {
> @@ -741,14 +751,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>  		vq->last_avail_idx = vq->used->idx;
>  	}
>  
> -	vq->log_guest_addr =
> -		translate_log_addr(dev, vq, addr->log_guest_addr);
> -	if (vq->log_guest_addr == 0) {
> -		RTE_LOG(DEBUG, VHOST_CONFIG,
> -			"(%d) failed to map log_guest_addr .\n",
> -			dev->vid);
> -		return dev;
> -	}
>  	vq->access_ok = 1;
>  
>  	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
> 

Reviewed-by: Adrian Moreno <amorenoz@redhat.com>


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [dpdk-dev] [PATCH v4] vhost: fix vhost user virtqueue not accessible
  2019-10-30 14:56   ` [dpdk-dev] [PATCH v3] " Marvin Liu
  2019-10-31 10:42     ` Tiwei Bie
@ 2019-11-04 10:13     ` Marvin Liu
  2019-11-04  8:47       ` Adrian Moreno
  2019-11-06 20:59       ` Maxime Coquelin
  1 sibling, 2 replies; 16+ messages in thread
From: Marvin Liu @ 2019-11-04 10:13 UTC (permalink / raw)
  To: maxime.coquelin, tiwei.bie, zhihong.wang, amorenoz; +Cc: dev, Marvin Liu

Log feature is disabled in vhost user, so that log address was invalid
when checking. Check whether log address is valid can work around it.
Log address should also be translated in packed ring virtqueue.

Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")

Signed-off-by: Marvin Liu <yong.liu@intel.com>
---
 lib/librte_vhost/vhost_user.c | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
index 61ef699ac..09e241305 100644
--- a/lib/librte_vhost/vhost_user.c
+++ b/lib/librte_vhost/vhost_user.c
@@ -641,11 +641,21 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 	struct vhost_vring_addr *addr = &vq->ring_addrs;
 	uint64_t len, expected_len;
 
+	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
+		vq->log_guest_addr =
+			translate_log_addr(dev, vq, addr->log_guest_addr);
+		if (vq->log_guest_addr == 0) {
+			RTE_LOG(DEBUG, VHOST_CONFIG,
+				"(%d) failed to map log_guest_addr.\n",
+				dev->vid);
+			return dev;
+		}
+	}
+
 	if (vq_is_packed(dev)) {
 		len = sizeof(struct vring_packed_desc) * vq->size;
 		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
 			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
-		vq->log_guest_addr = 0;
 		if (vq->desc_packed == NULL ||
 				len != sizeof(struct vring_packed_desc) *
 				vq->size) {
@@ -741,14 +751,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
 		vq->last_avail_idx = vq->used->idx;
 	}
 
-	vq->log_guest_addr =
-		translate_log_addr(dev, vq, addr->log_guest_addr);
-	if (vq->log_guest_addr == 0) {
-		RTE_LOG(DEBUG, VHOST_CONFIG,
-			"(%d) failed to map log_guest_addr .\n",
-			dev->vid);
-		return dev;
-	}
 	vq->access_ok = 1;
 
 	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
-- 
2.17.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v4] vhost: fix vhost user virtqueue not accessible
  2019-11-04  8:47       ` Adrian Moreno
@ 2019-11-06 20:16         ` Maxime Coquelin
  0 siblings, 0 replies; 16+ messages in thread
From: Maxime Coquelin @ 2019-11-06 20:16 UTC (permalink / raw)
  To: Adrian Moreno, Marvin Liu, tiwei.bie, zhihong.wang; +Cc: dev



On 11/4/19 9:47 AM, Adrian Moreno wrote:
> On 11/4/19 11:13 AM, Marvin Liu wrote:
>> Log feature is disabled in vhost user, so that log address was invalid
>> when checking. Check whether log address is valid can work around it.
>> Log address should also be translated in packed ring virtqueue.
>>
>> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
>>
>> Signed-off-by: Marvin Liu <yong.liu@intel.com>
>> ---
>>  lib/librte_vhost/vhost_user.c | 20 +++++++++++---------
>>  1 file changed, 11 insertions(+), 9 deletions(-)
>>
>> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c
>> index 61ef699ac..09e241305 100644
>> --- a/lib/librte_vhost/vhost_user.c
>> +++ b/lib/librte_vhost/vhost_user.c
>> @@ -641,11 +641,21 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>>  	struct vhost_vring_addr *addr = &vq->ring_addrs;
>>  	uint64_t len, expected_len;
>>  
>> +	if (addr->flags & (1 << VHOST_VRING_F_LOG)) {
>> +		vq->log_guest_addr =
>> +			translate_log_addr(dev, vq, addr->log_guest_addr);
>> +		if (vq->log_guest_addr == 0) {
>> +			RTE_LOG(DEBUG, VHOST_CONFIG,
>> +				"(%d) failed to map log_guest_addr.\n",
>> +				dev->vid);
>> +			return dev;
>> +		}
>> +	}
>> +
>>  	if (vq_is_packed(dev)) {
>>  		len = sizeof(struct vring_packed_desc) * vq->size;
>>  		vq->desc_packed = (struct vring_packed_desc *)(uintptr_t)
>>  			ring_addr_to_vva(dev, vq, addr->desc_user_addr, &len);
>> -		vq->log_guest_addr = 0;
>>  		if (vq->desc_packed == NULL ||
>>  				len != sizeof(struct vring_packed_desc) *
>>  				vq->size) {
>> @@ -741,14 +751,6 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index)
>>  		vq->last_avail_idx = vq->used->idx;
>>  	}
>>  
>> -	vq->log_guest_addr =
>> -		translate_log_addr(dev, vq, addr->log_guest_addr);
>> -	if (vq->log_guest_addr == 0) {
>> -		RTE_LOG(DEBUG, VHOST_CONFIG,
>> -			"(%d) failed to map log_guest_addr .\n",
>> -			dev->vid);
>> -		return dev;
>> -	}
>>  	vq->access_ok = 1;
>>  
>>  	VHOST_LOG_DEBUG(VHOST_CONFIG, "(%d) mapped address desc: %p\n",
>>
> 
> Reviewed-by: Adrian Moreno <amorenoz@redhat.com>
> 

Thanks Marvin for the fix, and Adrian for the review.

It looks good to me too now:
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Maxime


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [dpdk-dev] [PATCH v4] vhost: fix vhost user virtqueue not accessible
  2019-11-04 10:13     ` [dpdk-dev] [PATCH v4] " Marvin Liu
  2019-11-04  8:47       ` Adrian Moreno
@ 2019-11-06 20:59       ` Maxime Coquelin
  1 sibling, 0 replies; 16+ messages in thread
From: Maxime Coquelin @ 2019-11-06 20:59 UTC (permalink / raw)
  To: Marvin Liu, tiwei.bie, zhihong.wang, amorenoz; +Cc: dev



On 11/4/19 11:13 AM, Marvin Liu wrote:
> Log feature is disabled in vhost user, so that log address was invalid
> when checking. Check whether log address is valid can work around it.
> Log address should also be translated in packed ring virtqueue.
> 
> Fixes: 04cfc7fdbfca ("vhost: translate incoming log address to gpa")
> 
> Signed-off-by: Marvin Liu <yong.liu@intel.com>
> ---
>  lib/librte_vhost/vhost_user.c | 20 +++++++++++---------
>  1 file changed, 11 insertions(+), 9 deletions(-)

Applied to dpdk-next-virtio/master.

Thanks,
Maxime


^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2019-11-06 20:59 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-25 16:20 [dpdk-dev] [PATCH] vhost: fix vhost user virtqueue not accessable Marvin Liu
2019-10-25 12:20 ` Adrian Moreno
2019-10-28  2:13   ` Liu, Yong
2019-10-29 10:02     ` Adrian Moreno
2019-10-30 11:07 ` [dpdk-dev] [PATCH v2] vhost: fix vhost user virtqueue not accessible Marvin Liu
2019-10-30  6:58   ` Tiwei Bie
2019-10-30 14:56   ` [dpdk-dev] [PATCH v3] " Marvin Liu
2019-10-31 10:42     ` Tiwei Bie
2019-10-31 14:54       ` Liu, Yong
2019-10-31 17:47         ` Adrian Moreno
2019-11-01  7:16           ` Liu, Yong
2019-11-04  8:44             ` Adrian Moreno
2019-11-04 10:13     ` [dpdk-dev] [PATCH v4] " Marvin Liu
2019-11-04  8:47       ` Adrian Moreno
2019-11-06 20:16         ` Maxime Coquelin
2019-11-06 20:59       ` Maxime Coquelin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).