[dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[Wang, Wei 5. (NSB - CN/Hangzhou)]

Hi, all

In DPDK document, it it described that Virtio in containers Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) hugepages. In another word, do not use 2MB hugepage so far. Do you know the reason of this limitation?

In my envirionment, the pdpe1gb is not set in cpu flag, so hugepage can't be set to 1GB size. The hugepage number shall be more than 8


So is there any solution or workaround to fix this limitation? Or change dpdk code to fix this limitation?



Vivian
seat:      19009
phone: 13738006921

Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[Yuanhan Liu]

On Tue, Dec 05, 2017 at 06:58:53AM +0000, Wang, Wei 5. (NSB - CN/Hangzhou) wrote:
> 
> 
> Hi, all
> 
> In DPDK document, it it described that Virtio in containers Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) hugepages. In another word, do not use 2MB hugepage so far. Do you know the reason of this limitation?

It comes from the vhost-user spec.

> In my envirionment, the pdpe1gb is not set in cpu flag, so hugepage can't be set to 1GB size. The hugepage number shall be more than 8
> 
> 
> So is there any solution or workaround to fix this limitation? Or change dpdk code to fix this limitation?
> 

AFAIK, there are no workarounds. And I think the DPDK EAL memory code need be
refactored a bit to fix this limitation. Unforunately, seems there are no work
on that recently.

	--yliu

Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[Tan, Jianfeng]

> -----Original Message-----
> From: Yuanhan Liu [mailto:yliu@fridaylinux.org]
> Sent: Tuesday, December 5, 2017 10:42 PM
> To: Wang, Wei 5. (NSB - CN/Hangzhou)
> Cc: dev@dpdk.org; Tan, Jianfeng
> Subject: Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS
> in virtio-user with vhost-user backend
> 
> On Tue, Dec 05, 2017 at 06:58:53AM +0000, Wang, Wei 5. (NSB -
> CN/Hangzhou) wrote:
> >
> >
> > Hi, all
> >
> > In DPDK document, it it described that Virtio in containers Cannot work
> when there are more than VHOST_MEMORY_MAX_NREGIONS(8)
> hugepages. In another word, do not use 2MB hugepage so far. Do you know
> the reason of this limitation?
> 
> It comes from the vhost-user spec.
> 
> > In my envirionment, the pdpe1gb is not set in cpu flag, so hugepage can't
> be set to 1GB size. The hugepage number shall be more than 8
> >
> >
> > So is there any solution or workaround to fix this limitation? Or change
> dpdk code to fix this limitation?
> >
> 
> AFAIK, there are no workarounds. And I think the DPDK EAL memory code
> need be
> refactored a bit to fix this limitation. Unforunately, seems there are no work
> on that recently.

For more info, you can refer to: http://dpdk.org/dev/patchwork/patch/10406/

Thanks,
Jianfeng

> 
> 	--yliu

Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[Wang, Wei 5. (NSB - CN/Hangzhou)]

Hi, 
Thank you so much for your information. 


Vivian
seat:      19009
phone: 13738006921

-----Original Message-----
From: Tan, Jianfeng [mailto:jianfeng.tan@intel.com] 
Sent: Wednesday, December 06, 2017 10:56 AM
To: Yuanhan Liu <yliu@fridaylinux.org>; Wang, Wei 5. (NSB - CN/Hangzhou) <wei.5.wang@nokia-sbell.com>
Cc: dev@dpdk.org
Subject: RE: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend



> -----Original Message-----
> From: Yuanhan Liu [mailto:yliu@fridaylinux.org]
> Sent: Tuesday, December 5, 2017 10:42 PM
> To: Wang, Wei 5. (NSB - CN/Hangzhou)
> Cc: dev@dpdk.org; Tan, Jianfeng
> Subject: Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS
> in virtio-user with vhost-user backend
> 
> On Tue, Dec 05, 2017 at 06:58:53AM +0000, Wang, Wei 5. (NSB -
> CN/Hangzhou) wrote:
> >
> >
> > Hi, all
> >
> > In DPDK document, it it described that Virtio in containers Cannot work
> when there are more than VHOST_MEMORY_MAX_NREGIONS(8)
> hugepages. In another word, do not use 2MB hugepage so far. Do you know
> the reason of this limitation?
> 
> It comes from the vhost-user spec.
> 
> > In my envirionment, the pdpe1gb is not set in cpu flag, so hugepage can't
> be set to 1GB size. The hugepage number shall be more than 8
> >
> >
> > So is there any solution or workaround to fix this limitation? Or change
> dpdk code to fix this limitation?
> >
> 
> AFAIK, there are no workarounds. And I think the DPDK EAL memory code
> need be
> refactored a bit to fix this limitation. Unforunately, seems there are no work
> on that recently.

For more info, you can refer to: http://dpdk.org/dev/patchwork/patch/10406/

Thanks,
Jianfeng

> 
> 	--yliu

Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[Maxime Coquelin]

Hi,

On 12/05/2017 07:58 AM, Wang, Wei 5. (NSB - CN/Hangzhou) wrote:
> 
> 
> Hi, all
> 
> In DPDK document, it it described that Virtio in containers Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) hugepages. In another word, do not use 2MB hugepage so far. Do you know the reason of this limitation?

What document are you referring to?
Actually, a single region is composed of several hugepages, be it 1G or
2M. I use 2M hugepages daily without issues.

The limitation comes from the Vhost-user protocol spec. I know there are
discussions to enlarge it, but it would require a new protocol flag not
to break existing implementations, so no workaround possible.

> In my envirionment, the pdpe1gb is not set in cpu flag, so hugepage can't be set to 1GB size. The hugepage number shall be more than 8
> 
> 
> So is there any solution or workaround to fix this limitation? Or change dpdk code to fix this limitation?

Have you tried to use 2MB pages and faced the NREGIONS > 8 case?
If so, could you tell us more about your setup (QEMU version, VM RAM
size, etc...)?

Cheers,
Maxime
> 
> 
> Vivian
> seat:      19009
> phone: 13738006921
> 
> 
> 

Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[Yuanhan Liu]

On Wed, Dec 06, 2017 at 12:23:19PM +0100, Maxime Coquelin wrote:
> Hi,
> 
> On 12/05/2017 07:58 AM, Wang, Wei 5. (NSB - CN/Hangzhou) wrote:
> >
> >
> >Hi, all
> >
> >In DPDK document, it it described that Virtio in containers Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) hugepages. In another word, do not use 2MB hugepage so far. Do you know the reason of this limitation?
> 
> What document are you referring to?
> Actually, a single region is composed of several hugepages, be it 1G or
> 2M. I use 2M hugepages daily without issues.

He was talking about virtio-user.

	--yliu
> 
> The limitation comes from the Vhost-user protocol spec. I know there are
> discussions to enlarge it, but it would require a new protocol flag not
> to break existing implementations, so no workaround possible.
> 
> >In my envirionment, the pdpe1gb is not set in cpu flag, so hugepage can't be set to 1GB size. The hugepage number shall be more than 8
> >
> >
> >So is there any solution or workaround to fix this limitation? Or change dpdk code to fix this limitation?
> 
> Have you tried to use 2MB pages and faced the NREGIONS > 8 case?
> If so, could you tell us more about your setup (QEMU version, VM RAM
> size, etc...)?
> 
> Cheers,
> Maxime
> >
> >
> >Vivian
> >seat:      19009
> >phone: 13738006921
> >
> >
> >

Re: [dpdk-dev] The limitation of VHOST_MEMORY_MAX_NREGIONS in virtio-user with vhost-user backend

[Maxime Coquelin]

On 12/06/2017 03:06 PM, Yuanhan Liu wrote:
> On Wed, Dec 06, 2017 at 12:23:19PM +0100, Maxime Coquelin wrote:
>> Hi,
>>
>> On 12/05/2017 07:58 AM, Wang, Wei 5. (NSB - CN/Hangzhou) wrote:
>>>
>>>
>>> Hi, all
>>>
>>> In DPDK document, it it described that Virtio in containers Cannot work when there are more than VHOST_MEMORY_MAX_NREGIONS(8) hugepages. In another word, do not use 2MB hugepage so far. Do you know the reason of this limitation?
>>
>> What document are you referring to?
>> Actually, a single region is composed of several hugepages, be it 1G or
>> 2M. I use 2M hugepages daily without issues.
> 
> He was talking about virtio-user.

Ok, thanks for the correction.

The lookup time for every translation will be very costly if we have one
region per 2MB page.

> 
> 	--yliu
>>
>> The limitation comes from the Vhost-user protocol spec. I know there are
>> discussions to enlarge it, but it would require a new protocol flag not
>> to break existing implementations, so no workaround possible.
>>
>>> In my envirionment, the pdpe1gb is not set in cpu flag, so hugepage can't be set to 1GB size. The hugepage number shall be more than 8
>>>
>>>
>>> So is there any solution or workaround to fix this limitation? Or change dpdk code to fix this limitation?
>>
>> Have you tried to use 2MB pages and faced the NREGIONS > 8 case?
>> If so, could you tell us more about your setup (QEMU version, VM RAM
>> size, etc...)?
>>
>> Cheers,
>> Maxime
>>>
>>>
>>> Vivian
>>> seat:      19009
>>> phone: 13738006921
>>>
>>>
>>>