[dpdk-stable] [PATCH 1/1] eal/linux: do not create user mem map repeatedly when it exists

[dpdk-stable] [PATCH 1/1] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

From: Yunjian Wang <wangyunjian@huawei.com>

Currently, we will create new user mem map entry for the same memory
segment, but in fact it has already been added to the user mem maps.
It's not necessary to create it twice.

Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
Cc: stable@dpdk.org

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
---
 lib/librte_eal/linux/eal_vfio.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
index abb12a354..d8a8c39ab 100644
--- a/lib/librte_eal/linux/eal_vfio.c
+++ b/lib/librte_eal/linux/eal_vfio.c
@@ -1828,6 +1828,13 @@ container_dma_map(struct vfio_config *vfio_cfg, uint64_t vaddr, uint64_t iova,
 		ret = -1;
 		goto out;
 	}
+
+	/* we don't need create new user mem map entry
+	 * for the same memory segment.
+	 */
+	if (errno == EBUSY || errno == EEXIST)
+		goto out;
+
 	/* create new user mem map entry */
 	new_map = &user_mem_maps->maps[user_mem_maps->n_maps++];
 	new_map->addr = vaddr;
-- 
2.23.0

Re: [dpdk-stable] [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 16-Jul-20 2:38 PM, wangyunjian wrote:
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> Currently, we will create new user mem map entry for the same memory
> segment, but in fact it has already been added to the user mem maps.
> It's not necessary to create it twice.
> 
> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> ---
>   lib/librte_eal/linux/eal_vfio.c | 7 +++++++
>   1 file changed, 7 insertions(+)
> 
> diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
> index abb12a354..d8a8c39ab 100644
> --- a/lib/librte_eal/linux/eal_vfio.c
> +++ b/lib/librte_eal/linux/eal_vfio.c
> @@ -1828,6 +1828,13 @@ container_dma_map(struct vfio_config *vfio_cfg, uint64_t vaddr, uint64_t iova,
>   		ret = -1;
>   		goto out;
>   	}
> +
> +	/* we don't need create new user mem map entry
> +	 * for the same memory segment.
> +	 */
> +	if (errno == EBUSY || errno == EEXIST)
> +		goto out;
> +

I'm not sure i understand this patch. If we get errno, the call has 
failed, which means we're doing "goto out" from a few lines above. Am i 
missing something here?

>   	/* create new user mem map entry */
>   	new_map = &user_mem_maps->maps[user_mem_maps->n_maps++];
>   	new_map->addr = vaddr;
> 


-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 17-Jul-20 3:19 PM, Burakov, Anatoly wrote:
> On 16-Jul-20 2:38 PM, wangyunjian wrote:
>> From: Yunjian Wang <wangyunjian@huawei.com>
>>
>> Currently, we will create new user mem map entry for the same memory
>> segment, but in fact it has already been added to the user mem maps.
>> It's not necessary to create it twice.
>>
>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
>> ---
>>   lib/librte_eal/linux/eal_vfio.c | 7 +++++++
>>   1 file changed, 7 insertions(+)
>>
>> diff --git a/lib/librte_eal/linux/eal_vfio.c 
>> b/lib/librte_eal/linux/eal_vfio.c
>> index abb12a354..d8a8c39ab 100644
>> --- a/lib/librte_eal/linux/eal_vfio.c
>> +++ b/lib/librte_eal/linux/eal_vfio.c
>> @@ -1828,6 +1828,13 @@ container_dma_map(struct vfio_config *vfio_cfg, 
>> uint64_t vaddr, uint64_t iova,
>>           ret = -1;
>>           goto out;
>>       }
>> +
>> +    /* we don't need create new user mem map entry
>> +     * for the same memory segment.
>> +     */
>> +    if (errno == EBUSY || errno == EEXIST)
>> +        goto out;
>> +
> 
> I'm not sure i understand this patch. If we get errno, the call has 
> failed, which means we're doing "goto out" from a few lines above. Am i 
> missing something here?
> 
>>       /* create new user mem map entry */
>>       new_map = &user_mem_maps->maps[user_mem_maps->n_maps++];
>>       new_map->addr = vaddr;
>>
> 
> 

Oh, i see, the actual functions will set errno and return 0.

I don't think it's an actual issue as compacting will presumably remove 
the extra user mem map anyway. What exactly is being fixed here? Does 
compacting user mem maps not remove the extra entry?

-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

> -----Original Message-----
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Friday, July 17, 2020 10:24 PM
> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> david.marchand@redhat.com
> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> <xudingke@huawei.com>; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map
> repeatedly when it exists
> 
> On 17-Jul-20 3:19 PM, Burakov, Anatoly wrote:
> > On 16-Jul-20 2:38 PM, wangyunjian wrote:
> >> From: Yunjian Wang <wangyunjian@huawei.com>
> >>
> >> Currently, we will create new user mem map entry for the same memory
> >> segment, but in fact it has already been added to the user mem maps.
> >> It's not necessary to create it twice.
> >>
> >> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> >> Cc: stable@dpdk.org
> >>
> >> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> >> ---
> >>   lib/librte_eal/linux/eal_vfio.c | 7 +++++++
> >>   1 file changed, 7 insertions(+)
> >>
> >> diff --git a/lib/librte_eal/linux/eal_vfio.c
> >> b/lib/librte_eal/linux/eal_vfio.c index abb12a354..d8a8c39ab 100644
> >> --- a/lib/librte_eal/linux/eal_vfio.c
> >> +++ b/lib/librte_eal/linux/eal_vfio.c
> >> @@ -1828,6 +1828,13 @@ container_dma_map(struct vfio_config
> >> *vfio_cfg, uint64_t vaddr, uint64_t iova,
> >>           ret = -1;
> >>           goto out;
> >>       }
> >> +
> >> +    /* we don't need create new user mem map entry
> >> +     * for the same memory segment.
> >> +     */
> >> +    if (errno == EBUSY || errno == EEXIST)
> >> +        goto out;
> >> +
> >
> > I'm not sure i understand this patch. If we get errno, the call has
> > failed, which means we're doing "goto out" from a few lines above. Am
> > i missing something here?
> >
> >>       /* create new user mem map entry */
> >>       new_map =
> &user_mem_maps->maps[user_mem_maps->n_maps++];
> >>       new_map->addr = vaddr;
> >>
> >
> >
> 
> Oh, i see, the actual functions will set errno and return 0.
> 
> I don't think it's an actual issue as compacting will presumably remove the
> extra user mem map anyway. What exactly is being fixed here? Does
> compacting user mem maps not remove the extra entry?

I read the codes about compacting user mem maps. Currently, the function
only merges adjacent user mem maps and does not remove the same entry.

How about removing the same entry in the fuction?

Thanks
Yunjian

> 
> --
> Thanks,
> Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 20-Jul-20 3:00 AM, wangyunjian wrote:
>> -----Original Message-----
>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>> Sent: Friday, July 17, 2020 10:24 PM
>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
>> david.marchand@redhat.com
>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
>> <xudingke@huawei.com>; stable@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map
>> repeatedly when it exists
>>
>> On 17-Jul-20 3:19 PM, Burakov, Anatoly wrote:
>>> On 16-Jul-20 2:38 PM, wangyunjian wrote:
>>>> From: Yunjian Wang <wangyunjian@huawei.com>
>>>>
>>>> Currently, we will create new user mem map entry for the same memory
>>>> segment, but in fact it has already been added to the user mem maps.
>>>> It's not necessary to create it twice.
>>>>
>>>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
>>>> Cc: stable@dpdk.org
>>>>
>>>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
>>>> ---
>>>>    lib/librte_eal/linux/eal_vfio.c | 7 +++++++
>>>>    1 file changed, 7 insertions(+)
>>>>
>>>> diff --git a/lib/librte_eal/linux/eal_vfio.c
>>>> b/lib/librte_eal/linux/eal_vfio.c index abb12a354..d8a8c39ab 100644
>>>> --- a/lib/librte_eal/linux/eal_vfio.c
>>>> +++ b/lib/librte_eal/linux/eal_vfio.c
>>>> @@ -1828,6 +1828,13 @@ container_dma_map(struct vfio_config
>>>> *vfio_cfg, uint64_t vaddr, uint64_t iova,
>>>>            ret = -1;
>>>>            goto out;
>>>>        }
>>>> +
>>>> +    /* we don't need create new user mem map entry
>>>> +     * for the same memory segment.
>>>> +     */
>>>> +    if (errno == EBUSY || errno == EEXIST)
>>>> +        goto out;
>>>> +
>>>
>>> I'm not sure i understand this patch. If we get errno, the call has
>>> failed, which means we're doing "goto out" from a few lines above. Am
>>> i missing something here?
>>>
>>>>        /* create new user mem map entry */
>>>>        new_map =
>> &user_mem_maps->maps[user_mem_maps->n_maps++];
>>>>        new_map->addr = vaddr;
>>>>
>>>
>>>
>>
>> Oh, i see, the actual functions will set errno and return 0.
>>
>> I don't think it's an actual issue as compacting will presumably remove the
>> extra user mem map anyway. What exactly is being fixed here? Does
>> compacting user mem maps not remove the extra entry?
> 
> I read the codes about compacting user mem maps. Currently, the function
> only merges adjacent user mem maps and does not remove the same entry.
> 
> How about removing the same entry in the fuction?

I would've expected "the same" to be within the definition of 
"adjacent". Can you confirm that this actually doesn't happen? If so, 
then yes, probably compacting should do that, instead of relying on an 
artifact of implementation.

> 
> Thanks
> Yunjian
> 
>>
>> --
>> Thanks,
>> Anatoly


-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

> -----Original Message-----
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Monday, July 20, 2020 7:46 PM
> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> david.marchand@redhat.com
> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> <xudingke@huawei.com>; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem map
> repeatedly when it exists
> 
> On 20-Jul-20 3:00 AM, wangyunjian wrote:
> >> -----Original Message-----
> >> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> >> Sent: Friday, July 17, 2020 10:24 PM
> >> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> >> david.marchand@redhat.com
> >> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> >> <xudingke@huawei.com>; stable@dpdk.org
> >> Subject: Re: [dpdk-dev] [PATCH 1/1] eal/linux: do not create user mem
> >> map repeatedly when it exists
> >>
> >> On 17-Jul-20 3:19 PM, Burakov, Anatoly wrote:
> >>> On 16-Jul-20 2:38 PM, wangyunjian wrote:
> >>>> From: Yunjian Wang <wangyunjian@huawei.com>
> >>>>
> >>>> Currently, we will create new user mem map entry for the same
> >>>> memory segment, but in fact it has already been added to the user mem
> maps.
> >>>> It's not necessary to create it twice.
> >>>>
> >>>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already
> >>>> mapped")
> >>>> Cc: stable@dpdk.org
> >>>>
> >>>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> >>>> ---
> >>>>    lib/librte_eal/linux/eal_vfio.c | 7 +++++++
> >>>>    1 file changed, 7 insertions(+)
> >>>>
> >>>> diff --git a/lib/librte_eal/linux/eal_vfio.c
> >>>> b/lib/librte_eal/linux/eal_vfio.c index abb12a354..d8a8c39ab 100644
> >>>> --- a/lib/librte_eal/linux/eal_vfio.c
> >>>> +++ b/lib/librte_eal/linux/eal_vfio.c
> >>>> @@ -1828,6 +1828,13 @@ container_dma_map(struct vfio_config
> >>>> *vfio_cfg, uint64_t vaddr, uint64_t iova,
> >>>>            ret = -1;
> >>>>            goto out;
> >>>>        }
> >>>> +
> >>>> +    /* we don't need create new user mem map entry
> >>>> +     * for the same memory segment.
> >>>> +     */
> >>>> +    if (errno == EBUSY || errno == EEXIST)
> >>>> +        goto out;
> >>>> +
> >>>
> >>> I'm not sure i understand this patch. If we get errno, the call has
> >>> failed, which means we're doing "goto out" from a few lines above.
> >>> Am i missing something here?
> >>>
> >>>>        /* create new user mem map entry */
> >>>>        new_map =
> >> &user_mem_maps->maps[user_mem_maps->n_maps++];
> >>>>        new_map->addr = vaddr;
> >>>>
> >>>
> >>>
> >>
> >> Oh, i see, the actual functions will set errno and return 0.
> >>
> >> I don't think it's an actual issue as compacting will presumably
> >> remove the extra user mem map anyway. What exactly is being fixed
> >> here? Does compacting user mem maps not remove the extra entry?
> >
> > I read the codes about compacting user mem maps. Currently, the
> > function only merges adjacent user mem maps and does not remove the
> same entry.
> >
> > How about removing the same entry in the fuction?
> 
> I would've expected "the same" to be within the definition of "adjacent". Can
> you confirm that this actually doesn't happen? If so, then yes, probably
> compacting should do that, instead of relying on an artifact of implementation.

OK, I will do that, will send the v2 later.

Thanks
Yunjian

> 
> >
> > Thanks
> > Yunjian
> >
> >>
> >> --
> >> Thanks,
> >> Anatoly
> 
> 
> --
> Thanks,
> Anatoly

[dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

From: Yunjian Wang <wangyunjian@huawei.com>

Currently, we will create new user mem map entry for the same memory
segment, but in fact it has already been added to the user mem maps.
It's not necessary to create it twice.

To resolve the issue, add support to remove the same entry in the
function compact_user_maps().

Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
Cc: stable@dpdk.org

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
---
v2:
* Remove the same entry in the function compact_user_maps()
---
 lib/librte_eal/linux/eal_vfio.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
index abb12a354..df99307b7 100644
--- a/lib/librte_eal/linux/eal_vfio.c
+++ b/lib/librte_eal/linux/eal_vfio.c
@@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src, struct user_mem_map *end,
 static int
 merge_map(struct user_mem_map *left, struct user_mem_map *right)
 {
+	/* merge the same maps into one */
+	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
+		goto out;
+
 	if (left->addr + left->len != right->addr)
 		return 0;
 	if (left->iova + left->len != right->iova)
@@ -174,6 +178,7 @@ merge_map(struct user_mem_map *left, struct user_mem_map *right)
 
 	left->len += right->len;
 
+out:
 	memset(right, 0, sizeof(*right));
 
 	return 1;
-- 
2.23.0

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 23-Jul-20 3:48 PM, wangyunjian wrote:
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> Currently, we will create new user mem map entry for the same memory
> segment, but in fact it has already been added to the user mem maps.
> It's not necessary to create it twice.
> 
> To resolve the issue, add support to remove the same entry in the
> function compact_user_maps().
> 
> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> ---
> v2:
> * Remove the same entry in the function compact_user_maps()
> ---
>   lib/librte_eal/linux/eal_vfio.c | 5 +++++
>   1 file changed, 5 insertions(+)
> 
> diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
> index abb12a354..df99307b7 100644
> --- a/lib/librte_eal/linux/eal_vfio.c
> +++ b/lib/librte_eal/linux/eal_vfio.c
> @@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src, struct user_mem_map *end,
>   static int
>   merge_map(struct user_mem_map *left, struct user_mem_map *right)
>   {
> +	/* merge the same maps into one */
> +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
> +		goto out;
> +

merge_map looks for adjacent maps only, but does not handle maps that 
are wholly contained within one another ("the same map" also matches 
this definition). wouldn't it be better to check for that instead of 
*just* handling identical maps?

>   	if (left->addr + left->len != right->addr)
>   		return 0;
>   	if (left->iova + left->len != right->iova)
> @@ -174,6 +178,7 @@ merge_map(struct user_mem_map *left, struct user_mem_map *right)
>   
>   	left->len += right->len;
>   
> +out:
>   	memset(right, 0, sizeof(*right));
>   
>   	return 1;
> 


-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

> -----Original Message-----
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Friday, July 24, 2020 9:25 PM
> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> david.marchand@redhat.com
> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> <xudingke@huawei.com>; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
> repeatedly when it exists
> 
> On 23-Jul-20 3:48 PM, wangyunjian wrote:
> > From: Yunjian Wang <wangyunjian@huawei.com>
> >
> > Currently, we will create new user mem map entry for the same memory
> > segment, but in fact it has already been added to the user mem maps.
> > It's not necessary to create it twice.
> >
> > To resolve the issue, add support to remove the same entry in the
> > function compact_user_maps().
> >
> > Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> > ---
> > v2:
> > * Remove the same entry in the function compact_user_maps()
> > ---
> >   lib/librte_eal/linux/eal_vfio.c | 5 +++++
> >   1 file changed, 5 insertions(+)
> >
> > diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
> > index abb12a354..df99307b7 100644
> > --- a/lib/librte_eal/linux/eal_vfio.c
> > +++ b/lib/librte_eal/linux/eal_vfio.c
> > @@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src, struct
> user_mem_map *end,
> >   static int
> >   merge_map(struct user_mem_map *left, struct user_mem_map *right)
> >   {
> > +	/* merge the same maps into one */
> > +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
> > +		goto out;
> > +
> 
> merge_map looks for adjacent maps only, but does not handle maps that
> are wholly contained within one another ("the same map" also matches
> this definition). wouldn't it be better to check for that instead of
> *just* handling identical maps?

What about using the initial implementation?
We don't create new user mem map entry for the same memory segment.

@@ -1828,6 +1828,13 @@  container_dma_map(struct vfio_config *vfio_cfg, uint64_t vaddr, uint64_t iova,
 		ret = -1;
 		goto out;
 	}
+
+	/* we don't need create new user mem map entry
+	 * for the same memory segment.
+	 */
+	if (errno == EBUSY || errno == EEXIST)
+		goto out;
+
 	/* create new user mem map entry */
 	new_map = &user_mem_maps->maps[user_mem_maps->n_maps++];
 	new_map->addr = vaddr;

Thanks,
Yunjian
> 
> >   	if (left->addr + left->len != right->addr)
> >   		return 0;
> >   	if (left->iova + left->len != right->iova)
> > @@ -174,6 +178,7 @@ merge_map(struct user_mem_map *left, struct
> user_mem_map *right)
> >
> >   	left->len += right->len;
> >
> > +out:
> >   	memset(right, 0, sizeof(*right));
> >
> >   	return 1;
> >
> 
> 
> --
> Thanks,
> Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 25-Jul-20 10:59 AM, wangyunjian wrote:
>> -----Original Message-----
>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>> Sent: Friday, July 24, 2020 9:25 PM
>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
>> david.marchand@redhat.com
>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
>> <xudingke@huawei.com>; stable@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
>> repeatedly when it exists
>>
>> On 23-Jul-20 3:48 PM, wangyunjian wrote:
>>> From: Yunjian Wang <wangyunjian@huawei.com>
>>>
>>> Currently, we will create new user mem map entry for the same memory
>>> segment, but in fact it has already been added to the user mem maps.
>>> It's not necessary to create it twice.
>>>
>>> To resolve the issue, add support to remove the same entry in the
>>> function compact_user_maps().
>>>
>>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
>>> ---
>>> v2:
>>> * Remove the same entry in the function compact_user_maps()
>>> ---
>>>    lib/librte_eal/linux/eal_vfio.c | 5 +++++
>>>    1 file changed, 5 insertions(+)
>>>
>>> diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
>>> index abb12a354..df99307b7 100644
>>> --- a/lib/librte_eal/linux/eal_vfio.c
>>> +++ b/lib/librte_eal/linux/eal_vfio.c
>>> @@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src, struct
>> user_mem_map *end,
>>>    static int
>>>    merge_map(struct user_mem_map *left, struct user_mem_map *right)
>>>    {
>>> +	/* merge the same maps into one */
>>> +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
>>> +		goto out;
>>> +
>>
>> merge_map looks for adjacent maps only, but does not handle maps that
>> are wholly contained within one another ("the same map" also matches
>> this definition). wouldn't it be better to check for that instead of
>> *just* handling identical maps?
> 
> What about using the initial implementation?
> We don't create new user mem map entry for the same memory segment.

I don't like this implementation because it relies on particulars of how 
VFIO mapping work without explicitly specifying them. I.e. it's prone to 
breaking when changing code. That's not even mentioning that we have no 
guarantees on kernel behavior in that particular case being identical on 
all supported platforms.

I would honestly prefer an explicit compaction over implicit one.

> 
> @@ -1828,6 +1828,13 @@  container_dma_map(struct vfio_config *vfio_cfg, uint64_t vaddr, uint64_t iova,
>   		ret = -1;
>   		goto out;
>   	}
> +
> +	/* we don't need create new user mem map entry
> +	 * for the same memory segment.
> +	 */
> +	if (errno == EBUSY || errno == EEXIST)
> +		goto out;
> +
>   	/* create new user mem map entry */
>   	new_map = &user_mem_maps->maps[user_mem_maps->n_maps++];
>   	new_map->addr = vaddr;
> 
> Thanks,
> Yunjian
>>
>>>    	if (left->addr + left->len != right->addr)
>>>    		return 0;
>>>    	if (left->iova + left->len != right->iova)
>>> @@ -174,6 +178,7 @@ merge_map(struct user_mem_map *left, struct
>> user_mem_map *right)
>>>
>>>    	left->len += right->len;
>>>
>>> +out:
>>>    	memset(right, 0, sizeof(*right));
>>>
>>>    	return 1;
>>>
>>
>>
>> --
>> Thanks,
>> Anatoly


-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

> -----Original Message-----
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Monday, July 27, 2020 5:24 PM
> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> david.marchand@redhat.com
> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> <xudingke@huawei.com>; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
> repeatedly when it exists
> 
> On 25-Jul-20 10:59 AM, wangyunjian wrote:
> >> -----Original Message-----
> >> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> >> Sent: Friday, July 24, 2020 9:25 PM
> >> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> >> david.marchand@redhat.com
> >> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> >> <xudingke@huawei.com>; stable@dpdk.org
> >> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem
> >> map repeatedly when it exists
> >>
> >> On 23-Jul-20 3:48 PM, wangyunjian wrote:
> >>> From: Yunjian Wang <wangyunjian@huawei.com>
> >>>
> >>> Currently, we will create new user mem map entry for the same memory
> >>> segment, but in fact it has already been added to the user mem maps.
> >>> It's not necessary to create it twice.
> >>>
> >>> To resolve the issue, add support to remove the same entry in the
> >>> function compact_user_maps().
> >>>
> >>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> >>> Cc: stable@dpdk.org
> >>>
> >>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> >>> ---
> >>> v2:
> >>> * Remove the same entry in the function compact_user_maps()
> >>> ---
> >>>    lib/librte_eal/linux/eal_vfio.c | 5 +++++
> >>>    1 file changed, 5 insertions(+)
> >>>
> >>> diff --git a/lib/librte_eal/linux/eal_vfio.c
> >>> b/lib/librte_eal/linux/eal_vfio.c index abb12a354..df99307b7 100644
> >>> --- a/lib/librte_eal/linux/eal_vfio.c
> >>> +++ b/lib/librte_eal/linux/eal_vfio.c
> >>> @@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src, struct
> >> user_mem_map *end,
> >>>    static int
> >>>    merge_map(struct user_mem_map *left, struct user_mem_map
> *right)
> >>>    {
> >>> +	/* merge the same maps into one */
> >>> +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
> >>> +		goto out;
> >>> +
> >>
> >> merge_map looks for adjacent maps only, but does not handle maps that
> >> are wholly contained within one another ("the same map" also matches
> >> this definition). wouldn't it be better to check for that instead of
> >> *just* handling identical maps?
> >
> > What about using the initial implementation?
> > We don't create new user mem map entry for the same memory segment.
> 
> I don't like this implementation because it relies on particulars of how VFIO
> mapping work without explicitly specifying them. I.e. it's prone to breaking
> when changing code. That's not even mentioning that we have no guarantees
> on kernel behavior in that particular case being identical on all supported
> platforms.
> 
> I would honestly prefer an explicit compaction over implicit one.

What about this implementation?

diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
index e07979936..8dcb04cd9 100644
--- a/lib/librte_eal/linux/eal_vfio.c
+++ b/lib/librte_eal/linux/eal_vfio.c
@@ -179,6 +179,19 @@ merge_map(struct user_mem_map *left, struct user_mem_map *right)
 	return 1;
 }
 
+/* try merging two same maps into one, return 1 if succeeded */
+static int
+merge_same_map(struct user_mem_map *left, struct user_mem_map *right)
+{
+	if (memcmp(left, right, sizeof(struct user_mem_map)) != 0) {
+		return 0;
+	}
+
+	memset(right, 0, sizeof(*right));
+
+	return 1;
+}
+
 static struct user_mem_map *
 find_user_mem_map(struct user_mem_maps *user_mem_maps, uint64_t addr,
 		uint64_t iova, uint64_t len)
@@ -232,7 +245,7 @@ compact_user_maps(struct user_mem_maps *user_mem_maps)
 		if (is_null_map(l) || is_null_map(r))
 			continue;
 
-		if (merge_map(l, r))
+		if (merge_map(l, r) || merge_same_map(l, r))
 			n_merged++;
 	}

Thanks,
Yunjian

> 
> >
> > @@ -1828,6 +1828,13 @@  container_dma_map(struct vfio_config
> *vfio_cfg, uint64_t vaddr, uint64_t iova,
> >   		ret = -1;
> >   		goto out;
> >   	}
> > +
> > +	/* we don't need create new user mem map entry
> > +	 * for the same memory segment.
> > +	 */
> > +	if (errno == EBUSY || errno == EEXIST)
> > +		goto out;
> > +
> >   	/* create new user mem map entry */
> >   	new_map =
> &user_mem_maps->maps[user_mem_maps->n_maps++];
> >   	new_map->addr = vaddr;
> >
> > Thanks,
> > Yunjian
> >>
> >>>    	if (left->addr + left->len != right->addr)
> >>>    		return 0;
> >>>    	if (left->iova + left->len != right->iova) @@ -174,6 +178,7 @@
> >>> merge_map(struct user_mem_map *left, struct
> >> user_mem_map *right)
> >>>
> >>>    	left->len += right->len;
> >>>
> >>> +out:
> >>>    	memset(right, 0, sizeof(*right));
> >>>
> >>>    	return 1;
> >>>
> >>
> >>
> >> --
> >> Thanks,
> >> Anatoly
> 
> 
> --
> Thanks,
> Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 30-Jul-20 2:16 PM, wangyunjian wrote:
>> -----Original Message-----
>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>> Sent: Monday, July 27, 2020 5:24 PM
>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
>> david.marchand@redhat.com
>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
>> <xudingke@huawei.com>; stable@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
>> repeatedly when it exists
>>
>> On 25-Jul-20 10:59 AM, wangyunjian wrote:
>>>> -----Original Message-----
>>>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>>>> Sent: Friday, July 24, 2020 9:25 PM
>>>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
>>>> david.marchand@redhat.com
>>>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
>>>> <xudingke@huawei.com>; stable@dpdk.org
>>>> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem
>>>> map repeatedly when it exists
>>>>
>>>> On 23-Jul-20 3:48 PM, wangyunjian wrote:
>>>>> From: Yunjian Wang <wangyunjian@huawei.com>
>>>>>
>>>>> Currently, we will create new user mem map entry for the same memory
>>>>> segment, but in fact it has already been added to the user mem maps.
>>>>> It's not necessary to create it twice.
>>>>>
>>>>> To resolve the issue, add support to remove the same entry in the
>>>>> function compact_user_maps().
>>>>>
>>>>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
>>>>> Cc: stable@dpdk.org
>>>>>
>>>>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
>>>>> ---
>>>>> v2:
>>>>> * Remove the same entry in the function compact_user_maps()
>>>>> ---
>>>>>     lib/librte_eal/linux/eal_vfio.c | 5 +++++
>>>>>     1 file changed, 5 insertions(+)
>>>>>
>>>>> diff --git a/lib/librte_eal/linux/eal_vfio.c
>>>>> b/lib/librte_eal/linux/eal_vfio.c index abb12a354..df99307b7 100644
>>>>> --- a/lib/librte_eal/linux/eal_vfio.c
>>>>> +++ b/lib/librte_eal/linux/eal_vfio.c
>>>>> @@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src, struct
>>>> user_mem_map *end,
>>>>>     static int
>>>>>     merge_map(struct user_mem_map *left, struct user_mem_map
>> *right)
>>>>>     {
>>>>> +	/* merge the same maps into one */
>>>>> +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
>>>>> +		goto out;
>>>>> +
>>>>
>>>> merge_map looks for adjacent maps only, but does not handle maps that
>>>> are wholly contained within one another ("the same map" also matches
>>>> this definition). wouldn't it be better to check for that instead of
>>>> *just* handling identical maps?
>>>
>>> What about using the initial implementation?
>>> We don't create new user mem map entry for the same memory segment.
>>
>> I don't like this implementation because it relies on particulars of how VFIO
>> mapping work without explicitly specifying them. I.e. it's prone to breaking
>> when changing code. That's not even mentioning that we have no guarantees
>> on kernel behavior in that particular case being identical on all supported
>> platforms.
>>
>> I would honestly prefer an explicit compaction over implicit one.
> 
> What about this implementation?

Again, this works, but i feel like specializing it to just merge the 
exact same maps is missing an opportunity to provide a more general 
solution that merges same *and* subset maps.

> 
> diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
> index e07979936..8dcb04cd9 100644
> --- a/lib/librte_eal/linux/eal_vfio.c
> +++ b/lib/librte_eal/linux/eal_vfio.c
> @@ -179,6 +179,19 @@ merge_map(struct user_mem_map *left, struct user_mem_map *right)
>   	return 1;
>   }
>   
> +/* try merging two same maps into one, return 1 if succeeded */
> +static int
> +merge_same_map(struct user_mem_map *left, struct user_mem_map *right)
> +{
> +	if (memcmp(left, right, sizeof(struct user_mem_map)) != 0) {
> +		return 0;
> +	}
> +
> +	memset(right, 0, sizeof(*right));
> +
> +	return 1;
> +}
> +
>   static struct user_mem_map *
>   find_user_mem_map(struct user_mem_maps *user_mem_maps, uint64_t addr,
>   		uint64_t iova, uint64_t len)
> @@ -232,7 +245,7 @@ compact_user_maps(struct user_mem_maps *user_mem_maps)
>   		if (is_null_map(l) || is_null_map(r))
>   			continue;
>   
> -		if (merge_map(l, r))
> +		if (merge_map(l, r) || merge_same_map(l, r))
>   			n_merged++;
>   	}
> 
> Thanks,
> Yunjian
> 
>>
>>>
>>> @@ -1828,6 +1828,13 @@  container_dma_map(struct vfio_config
>> *vfio_cfg, uint64_t vaddr, uint64_t iova,
>>>    		ret = -1;
>>>    		goto out;
>>>    	}
>>> +
>>> +	/* we don't need create new user mem map entry
>>> +	 * for the same memory segment.
>>> +	 */
>>> +	if (errno == EBUSY || errno == EEXIST)
>>> +		goto out;
>>> +
>>>    	/* create new user mem map entry */
>>>    	new_map =
>> &user_mem_maps->maps[user_mem_maps->n_maps++];
>>>    	new_map->addr = vaddr;
>>>
>>> Thanks,
>>> Yunjian
>>>>
>>>>>     	if (left->addr + left->len != right->addr)
>>>>>     		return 0;
>>>>>     	if (left->iova + left->len != right->iova) @@ -174,6 +178,7 @@
>>>>> merge_map(struct user_mem_map *left, struct
>>>> user_mem_map *right)
>>>>>
>>>>>     	left->len += right->len;
>>>>>
>>>>> +out:
>>>>>     	memset(right, 0, sizeof(*right));
>>>>>
>>>>>     	return 1;
>>>>>
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Anatoly
>>
>>
>> --
>> Thanks,
>> Anatoly


-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

> -----Original Message-----
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Friday, July 31, 2020 7:55 PM
> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> david.marchand@redhat.com
> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> <xudingke@huawei.com>; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
> repeatedly when it exists
> 
> On 30-Jul-20 2:16 PM, wangyunjian wrote:
> >> -----Original Message-----
> >> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> >> Sent: Monday, July 27, 2020 5:24 PM
> >> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> >> david.marchand@redhat.com
> >> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> >> <xudingke@huawei.com>; stable@dpdk.org
> >> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
> >> repeatedly when it exists
> >>
> >> On 25-Jul-20 10:59 AM, wangyunjian wrote:
> >>>> -----Original Message-----
> >>>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> >>>> Sent: Friday, July 24, 2020 9:25 PM
> >>>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> >>>> david.marchand@redhat.com
> >>>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> >>>> <xudingke@huawei.com>; stable@dpdk.org
> >>>> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem
> >>>> map repeatedly when it exists
> >>>>
> >>>> On 23-Jul-20 3:48 PM, wangyunjian wrote:
> >>>>> From: Yunjian Wang <wangyunjian@huawei.com>
> >>>>>
> >>>>> Currently, we will create new user mem map entry for the same memory
> >>>>> segment, but in fact it has already been added to the user mem maps.
> >>>>> It's not necessary to create it twice.
> >>>>>
> >>>>> To resolve the issue, add support to remove the same entry in the
> >>>>> function compact_user_maps().
> >>>>>
> >>>>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> >>>>> Cc: stable@dpdk.org
> >>>>>
> >>>>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> >>>>> ---
> >>>>> v2:
> >>>>> * Remove the same entry in the function compact_user_maps()
> >>>>> ---
> >>>>>     lib/librte_eal/linux/eal_vfio.c | 5 +++++
> >>>>>     1 file changed, 5 insertions(+)
> >>>>>
> >>>>> diff --git a/lib/librte_eal/linux/eal_vfio.c
> >>>>> b/lib/librte_eal/linux/eal_vfio.c index abb12a354..df99307b7 100644
> >>>>> --- a/lib/librte_eal/linux/eal_vfio.c
> >>>>> +++ b/lib/librte_eal/linux/eal_vfio.c
> >>>>> @@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src,
> struct
> >>>> user_mem_map *end,
> >>>>>     static int
> >>>>>     merge_map(struct user_mem_map *left, struct user_mem_map
> >> *right)
> >>>>>     {
> >>>>> +	/* merge the same maps into one */
> >>>>> +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
> >>>>> +		goto out;
> >>>>> +
> >>>>
> >>>> merge_map looks for adjacent maps only, but does not handle maps that
> >>>> are wholly contained within one another ("the same map" also matches
> >>>> this definition). wouldn't it be better to check for that instead of
> >>>> *just* handling identical maps?
> >>>
> >>> What about using the initial implementation?
> >>> We don't create new user mem map entry for the same memory segment.
> >>
> >> I don't like this implementation because it relies on particulars of how VFIO
> >> mapping work without explicitly specifying them. I.e. it's prone to breaking
> >> when changing code. That's not even mentioning that we have no
> guarantees
> >> on kernel behavior in that particular case being identical on all supported
> >> platforms.
> >>
> >> I would honestly prefer an explicit compaction over implicit one.
> >
> > What about this implementation?
> 
> Again, this works, but i feel like specializing it to just merge the
> exact same maps is missing an opportunity to provide a more general
> solution that merges same *and* subset maps.

Currently, the problem that I encounter is that a container has many
devices and the application will map the same memory many times.
The kernel driver returns EEXIST as long as there are overlapping memory
areas. Therefore, the application needs to ensure that the memory blocks
of the DMA do not overlap. Otherwise, it will not work normally.

Could you offer me some ideas or advise to fix it？

Thanks,
Yunjian
> 
> >
> > diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
> > index e07979936..8dcb04cd9 100644
> > --- a/lib/librte_eal/linux/eal_vfio.c
> > +++ b/lib/librte_eal/linux/eal_vfio.c
> > @@ -179,6 +179,19 @@ merge_map(struct user_mem_map *left, struct
> user_mem_map *right)
> >   	return 1;
> >   }
> >
> > +/* try merging two same maps into one, return 1 if succeeded */
> > +static int
> > +merge_same_map(struct user_mem_map *left, struct user_mem_map
> *right)
> > +{
> > +	if (memcmp(left, right, sizeof(struct user_mem_map)) != 0) {
> > +		return 0;
> > +	}
> > +
> > +	memset(right, 0, sizeof(*right));
> > +
> > +	return 1;
> > +}
> > +
> >   static struct user_mem_map *
> >   find_user_mem_map(struct user_mem_maps *user_mem_maps,
> uint64_t addr,
> >   		uint64_t iova, uint64_t len)
> > @@ -232,7 +245,7 @@ compact_user_maps(struct user_mem_maps
> *user_mem_maps)
> >   		if (is_null_map(l) || is_null_map(r))
> >   			continue;
> >
> > -		if (merge_map(l, r))
> > +		if (merge_map(l, r) || merge_same_map(l, r))
> >   			n_merged++;
> >   	}
> >
> > Thanks,
> > Yunjian
> >
> >>
> >>>
> >>> @@ -1828,6 +1828,13 @@  container_dma_map(struct vfio_config
> >> *vfio_cfg, uint64_t vaddr, uint64_t iova,
> >>>    		ret = -1;
> >>>    		goto out;
> >>>    	}
> >>> +
> >>> +	/* we don't need create new user mem map entry
> >>> +	 * for the same memory segment.
> >>> +	 */
> >>> +	if (errno == EBUSY || errno == EEXIST)
> >>> +		goto out;
> >>> +
> >>>    	/* create new user mem map entry */
> >>>    	new_map =
> >> &user_mem_maps->maps[user_mem_maps->n_maps++];
> >>>    	new_map->addr = vaddr;
> >>>
> >>> Thanks,
> >>> Yunjian
> >>>>
> >>>>>     	if (left->addr + left->len != right->addr)
> >>>>>     		return 0;
> >>>>>     	if (left->iova + left->len != right->iova) @@ -174,6 +178,7 @@
> >>>>> merge_map(struct user_mem_map *left, struct
> >>>> user_mem_map *right)
> >>>>>
> >>>>>     	left->len += right->len;
> >>>>>
> >>>>> +out:
> >>>>>     	memset(right, 0, sizeof(*right));
> >>>>>
> >>>>>     	return 1;
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Thanks,
> >>>> Anatoly
> >>
> >>
> >> --
> >> Thanks,
> >> Anatoly
> 
> 
> --
> Thanks,
> Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 05-Aug-20 1:58 PM, wangyunjian wrote:
>> -----Original Message-----
>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>> Sent: Friday, July 31, 2020 7:55 PM
>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
>> david.marchand@redhat.com
>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
>> <xudingke@huawei.com>; stable@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
>> repeatedly when it exists
>>
>> On 30-Jul-20 2:16 PM, wangyunjian wrote:
>>>> -----Original Message-----
>>>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>>>> Sent: Monday, July 27, 2020 5:24 PM
>>>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
>>>> david.marchand@redhat.com
>>>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
>>>> <xudingke@huawei.com>; stable@dpdk.org
>>>> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
>>>> repeatedly when it exists
>>>>
>>>> On 25-Jul-20 10:59 AM, wangyunjian wrote:
>>>>>> -----Original Message-----
>>>>>> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
>>>>>> Sent: Friday, July 24, 2020 9:25 PM
>>>>>> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
>>>>>> david.marchand@redhat.com
>>>>>> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
>>>>>> <xudingke@huawei.com>; stable@dpdk.org
>>>>>> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem
>>>>>> map repeatedly when it exists
>>>>>>
>>>>>> On 23-Jul-20 3:48 PM, wangyunjian wrote:
>>>>>>> From: Yunjian Wang <wangyunjian@huawei.com>
>>>>>>>
>>>>>>> Currently, we will create new user mem map entry for the same memory
>>>>>>> segment, but in fact it has already been added to the user mem maps.
>>>>>>> It's not necessary to create it twice.
>>>>>>>
>>>>>>> To resolve the issue, add support to remove the same entry in the
>>>>>>> function compact_user_maps().
>>>>>>>
>>>>>>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
>>>>>>> Cc: stable@dpdk.org
>>>>>>>
>>>>>>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
>>>>>>> ---
>>>>>>> v2:
>>>>>>> * Remove the same entry in the function compact_user_maps()
>>>>>>> ---
>>>>>>>      lib/librte_eal/linux/eal_vfio.c | 5 +++++
>>>>>>>      1 file changed, 5 insertions(+)
>>>>>>>
>>>>>>> diff --git a/lib/librte_eal/linux/eal_vfio.c
>>>>>>> b/lib/librte_eal/linux/eal_vfio.c index abb12a354..df99307b7 100644
>>>>>>> --- a/lib/librte_eal/linux/eal_vfio.c
>>>>>>> +++ b/lib/librte_eal/linux/eal_vfio.c
>>>>>>> @@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src,
>> struct
>>>>>> user_mem_map *end,
>>>>>>>      static int
>>>>>>>      merge_map(struct user_mem_map *left, struct user_mem_map
>>>> *right)
>>>>>>>      {
>>>>>>> +	/* merge the same maps into one */
>>>>>>> +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
>>>>>>> +		goto out;
>>>>>>> +
>>>>>>
>>>>>> merge_map looks for adjacent maps only, but does not handle maps that
>>>>>> are wholly contained within one another ("the same map" also matches
>>>>>> this definition). wouldn't it be better to check for that instead of
>>>>>> *just* handling identical maps?
>>>>>
>>>>> What about using the initial implementation?
>>>>> We don't create new user mem map entry for the same memory segment.
>>>>
>>>> I don't like this implementation because it relies on particulars of how VFIO
>>>> mapping work without explicitly specifying them. I.e. it's prone to breaking
>>>> when changing code. That's not even mentioning that we have no
>> guarantees
>>>> on kernel behavior in that particular case being identical on all supported
>>>> platforms.
>>>>
>>>> I would honestly prefer an explicit compaction over implicit one.
>>>
>>> What about this implementation?
>>
>> Again, this works, but i feel like specializing it to just merge the
>> exact same maps is missing an opportunity to provide a more general
>> solution that merges same *and* subset maps.
> 
> Currently, the problem that I encounter is that a container has many
> devices and the application will map the same memory many times.
> The kernel driver returns EEXIST as long as there are overlapping memory
> areas. Therefore, the application needs to ensure that the memory blocks
> of the DMA do not overlap. Otherwise, it will not work normally.
> 
> Could you offer me some ideas or advise to fix it？
> 

It sounds like your approach is better if that is indeed the case.

-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 23-Jul-20 3:48 PM, wangyunjian wrote:
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> Currently, we will create new user mem map entry for the same memory
> segment, but in fact it has already been added to the user mem maps.
> It's not necessary to create it twice.
> 
> To resolve the issue, add support to remove the same entry in the
> function compact_user_maps().
> 
> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

Hi David,

Can this patch be merged before RC1?

Thanks,
Yunjian

> -----Original Message-----
> From: Burakov, Anatoly [mailto:anatoly.burakov@intel.com]
> Sent: Thursday, September 17, 2020 7:35 PM
> To: wangyunjian <wangyunjian@huawei.com>; dev@dpdk.org;
> david.marchand@redhat.com
> Cc: Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> <xudingke@huawei.com>; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
> repeatedly when it exists
> 
> On 23-Jul-20 3:48 PM, wangyunjian wrote:
> > From: Yunjian Wang <wangyunjian@huawei.com>
> >
> > Currently, we will create new user mem map entry for the same memory
> > segment, but in fact it has already been added to the user mem maps.
> > It's not necessary to create it twice.
> >
> > To resolve the issue, add support to remove the same entry in the
> > function compact_user_maps().
> >
> > Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> > ---
> 
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> 
> --
> Thanks,
> Anatoly

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[David Marchand]

On Thu, Oct 15, 2020 at 2:46 PM wangyunjian <wangyunjian@huawei.com> wrote:
>
> Hi David,
>
> Can this patch be merged before RC1?

Trying to understand what this is about.
What is the impact from the issue you fix from an application point of view?

Getting a good title / description is important to help stable
maintainers later when judging whether they should pick it or not.
Thanks.

-- 
David Marchand

[dpdk-stable] [dpdk-dev] [PATCH v3] eal: fix create user mem map repeatedly when it exists

[wangyunjian]

From: Yunjian Wang <wangyunjian@huawei.com>

Currently, a issue that a container has many devices and the
application will map the same memory many times. The kernel
driver returns EEXIST as long as there are overlapping memory
areas. As a result, we repeatedly create new user mem map entry
for the same memory segment and this will lead to no more space
for other user mem maps.

To resolve the issue, add support to remove the same entry in
the function compact_user_maps().

Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
Cc: stable@dpdk.org

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
v3:
   Only update commit log and title
---
 lib/librte_eal/linux/eal_vfio.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
index 380f2f44a..7cff51e16 100644
--- a/lib/librte_eal/linux/eal_vfio.c
+++ b/lib/librte_eal/linux/eal_vfio.c
@@ -167,6 +167,10 @@ adjust_map(struct user_mem_map *src, struct user_mem_map *end,
 static int
 merge_map(struct user_mem_map *left, struct user_mem_map *right)
 {
+	/* merge the same maps into one */
+	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
+		goto out;
+
 	if (left->addr + left->len != right->addr)
 		return 0;
 	if (left->iova + left->len != right->iova)
@@ -174,6 +178,7 @@ merge_map(struct user_mem_map *left, struct user_mem_map *right)
 
 	left->len += right->len;
 
+out:
 	memset(right, 0, sizeof(*right));
 
 	return 1;
-- 
2.23.0

Re: [dpdk-stable] [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map repeatedly when it exists

[wangyunjian]

> -----Original Message-----
> From: David Marchand [mailto:david.marchand@redhat.com]
> Sent: Thursday, October 15, 2020 8:54 PM
> To: wangyunjian <wangyunjian@huawei.com>
> Cc: dev@dpdk.org; Burakov, Anatoly <anatoly.burakov@intel.com>; Lilijun
> (Jerry) <jerry.lilijun@huawei.com>; xudingke <xudingke@huawei.com>;
> stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] eal/linux: do not create user mem map
> repeatedly when it exists
> 
> On Thu, Oct 15, 2020 at 2:46 PM wangyunjian <wangyunjian@huawei.com>
> wrote:
> >
> > Hi David,
> >
> > Can this patch be merged before RC1?
> 
> Trying to understand what this is about.
> What is the impact from the issue you fix from an application point of view?
> 
> Getting a good title / description is important to help stable maintainers later
> when judging whether they should pick it or not.
> Thanks.

Thanks for your suggestion, I have updated commit log and title in new version.

https://patchwork.dpdk.org/patch/81069/

Yunjian
> 
> --
> David Marchand

Re: [dpdk-stable] [dpdk-dev] [PATCH v3] eal: fix create user mem map repeatedly when it exists

[Thomas Monjalon]

16/10/2020 11:28, wangyunjian:
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> Currently, a issue that a container has many devices and the
> application will map the same memory many times. The kernel
> driver returns EEXIST as long as there are overlapping memory
> areas. As a result, we repeatedly create new user mem map entry
> for the same memory segment and this will lead to no more space
> for other user mem maps.
> 
> To resolve the issue, add support to remove the same entry in
> the function compact_user_maps().

Sorry I don't understand the explanations above.
Anatoly, please could you help in rewording?

Re: [dpdk-stable] [dpdk-dev] [PATCH v3] eal: fix create user mem map repeatedly when it exists

[Thomas Monjalon]

20/10/2020 16:09, Thomas Monjalon:
> 16/10/2020 11:28, wangyunjian:
> > From: Yunjian Wang <wangyunjian@huawei.com>
> > 
> > Currently, a issue that a container has many devices and the
> > application will map the same memory many times. The kernel
> > driver returns EEXIST as long as there are overlapping memory
> > areas. As a result, we repeatedly create new user mem map entry
> > for the same memory segment and this will lead to no more space
> > for other user mem maps.
> > 
> > To resolve the issue, add support to remove the same entry in
> > the function compact_user_maps().
> 
> Sorry I don't understand the explanations above.
> Anatoly, please could you help in rewording?

Ping for rewording please.

Re: [dpdk-stable] [dpdk-dev] [PATCH v3] eal: fix create user mem map repeatedly when it exists

[Thomas Monjalon]

15/11/2020 15:23, Thomas Monjalon:
> 20/10/2020 16:09, Thomas Monjalon:
> > 16/10/2020 11:28, wangyunjian:
> > > From: Yunjian Wang <wangyunjian@huawei.com>
> > > 
> > > Currently, a issue that a container has many devices and the
> > > application will map the same memory many times. The kernel
> > > driver returns EEXIST as long as there are overlapping memory
> > > areas. As a result, we repeatedly create new user mem map entry
> > > for the same memory segment and this will lead to no more space
> > > for other user mem maps.
> > > 
> > > To resolve the issue, add support to remove the same entry in
> > > the function compact_user_maps().
> > 
> > Sorry I don't understand the explanations above.
> > Anatoly, please could you help in rewording?
> 
> Ping for rewording please.

What is the conclusion? This fix is not worth the effort?

Re: [dpdk-stable] [dpdk-dev] [PATCH v3] eal: fix create user mem map repeatedly when it exists

[wangyunjian]

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas@monjalon.net]
> Sent: Monday, November 23, 2020 2:20 AM
> To: anatoly.burakov@intel.com; wangyunjian <wangyunjian@huawei.com>
> Cc: stable@dpdk.org; dev@dpdk.org; david.marchand@redhat.com; Lilijun
> (Jerry) <jerry.lilijun@huawei.com>; xudingke <xudingke@huawei.com>;
> bruce.richardson@intel.com; john.mcnamara@intel.com; asafp@nvidia.com
> Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH v3] eal: fix create user mem map
> repeatedly when it exists
> 
> 15/11/2020 15:23, Thomas Monjalon:
> > 20/10/2020 16:09, Thomas Monjalon:
> > > 16/10/2020 11:28, wangyunjian:
> > > > From: Yunjian Wang <wangyunjian@huawei.com>
> > > >
> > > > Currently, a issue that a container has many devices and the
> > > > application will map the same memory many times. The kernel driver
> > > > returns EEXIST as long as there are overlapping memory areas. As a
> > > > result, we repeatedly create new user mem map entry for the same
> > > > memory segment and this will lead to no more space for other user
> > > > mem maps.
> > > >
> > > > To resolve the issue, add support to remove the same entry in the
> > > > function compact_user_maps().
> > >
> > > Sorry I don't understand the explanations above.
> > > Anatoly, please could you help in rewording?
> >
> > Ping for rewording please.
> 
> What is the conclusion? This fix is not worth the effort?
> 

In my opinion, this issue needs to be fixed. Currently, the 'user_mem_maps->maps[]'
may store many same user mem maps but the maps array is limited, which will lead
to other mem maps cannot be created because of no more space left.

Thanks,
Yunjian

Re: [dpdk-stable] [dpdk-dev] [PATCH v3] eal: fix create user mem map repeatedly when it exists

[Burakov, Anatoly]

On 20-Oct-20 3:09 PM, Thomas Monjalon wrote:
> 16/10/2020 11:28, wangyunjian:
>> From: Yunjian Wang <wangyunjian@huawei.com>
>>
>> Currently, a issue that a container has many devices and the
>> application will map the same memory many times. The kernel
>> driver returns EEXIST as long as there are overlapping memory
>> areas. As a result, we repeatedly create new user mem map entry
>> for the same memory segment and this will lead to no more space
>> for other user mem maps.
>>
>> To resolve the issue, add support to remove the same entry in
>> the function compact_user_maps().
> 
> Sorry I don't understand the explanations above.
> Anatoly, please could you help in rewording?
> 
> 
> 

Apologies for delay, fell through the cracks.

Suggested rewording:

Currently, user mem maps will check if the newly mapped area is adjacent 
to any existing mapping, but will not check if the mapping is identical 
because it assumes that the API will never get called with the same 
mapping twice. This will result in duplicate entries in the user mem 
maps list.

Fix it by also checking for duplicate mappings, and skipping them if 
they are found.

-- 
Thanks,
Anatoly

[dpdk-stable] [dpdk-dev] [PATCH v4] eal: fix create user mem map repeatedly when it exists

[wangyunjian]

From: Yunjian Wang <wangyunjian@huawei.com>

Currently, user mem maps will check if the newly mapped area is adjacent
to any existing mapping, but will not check if the mapping is identical
because it assumes that the API will never get called with the same
mapping twice. This will result in duplicate entries in the user mem
maps list.

Fix it by also checking for duplicate mappings, and skipping them if
they are found.

Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
Cc: stable@dpdk.org

Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
v4:
  Update commit log suggested by Anatoly Burakov
---
 lib/librte_eal/linux/eal_vfio.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
index 050082444e..0967215783 100644
--- a/lib/librte_eal/linux/eal_vfio.c
+++ b/lib/librte_eal/linux/eal_vfio.c
@@ -168,6 +168,10 @@ adjust_map(struct user_mem_map *src, struct user_mem_map *end,
 static int
 merge_map(struct user_mem_map *left, struct user_mem_map *right)
 {
+	/* merge the same maps into one */
+	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
+		goto out;
+
 	if (left->addr + left->len != right->addr)
 		return 0;
 	if (left->iova + left->len != right->iova)
@@ -175,6 +179,7 @@ merge_map(struct user_mem_map *left, struct user_mem_map *right)
 
 	left->len += right->len;
 
+out:
 	memset(right, 0, sizeof(*right));
 
 	return 1;
-- 
2.23.0

Re: [dpdk-stable] [dpdk-dev] [PATCH v4] eal: fix create user mem map repeatedly when it exists

[wangyunjian]

Friendly ping.

> -----Original Message-----
> From: wangyunjian
> Sent: Monday, December 7, 2020 7:09 PM
> To: dev@dpdk.org
> Cc: david.marchand@redhat.com; thomas@monjalon.net;
> anatoly.burakov@intel.com; Lilijun (Jerry) <jerry.lilijun@huawei.com>; xudingke
> <xudingke@huawei.com>; wangyunjian <wangyunjian@huawei.com>;
> stable@dpdk.org
> Subject: [dpdk-dev] [PATCH v4] eal: fix create user mem map repeatedly when it
> exists
> 
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> Currently, user mem maps will check if the newly mapped area is adjacent to
> any existing mapping, but will not check if the mapping is identical because it
> assumes that the API will never get called with the same mapping twice. This
> will result in duplicate entries in the user mem maps list.
> 
> Fix it by also checking for duplicate mappings, and skipping them if they are
> found.
> 
> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> v4:
>   Update commit log suggested by Anatoly Burakov
> ---
>  lib/librte_eal/linux/eal_vfio.c | 5 +++++
>  1 file changed, 5 insertions(+)
> 
> diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c index
> 050082444e..0967215783 100644
> --- a/lib/librte_eal/linux/eal_vfio.c
> +++ b/lib/librte_eal/linux/eal_vfio.c
> @@ -168,6 +168,10 @@ adjust_map(struct user_mem_map *src, struct
> user_mem_map *end,  static int  merge_map(struct user_mem_map *left,
> struct user_mem_map *right)  {
> +	/* merge the same maps into one */
> +	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
> +		goto out;
> +
>  	if (left->addr + left->len != right->addr)
>  		return 0;
>  	if (left->iova + left->len != right->iova) @@ -175,6 +179,7 @@
> merge_map(struct user_mem_map *left, struct user_mem_map *right)
> 
>  	left->len += right->len;
> 
> +out:
>  	memset(right, 0, sizeof(*right));
> 
>  	return 1;
> --
> 2.23.0

Re: [dpdk-stable] [dpdk-dev] [PATCH v4] eal: fix create user mem map repeatedly when it exists

[Thomas Monjalon]

07/12/2020 12:08, wangyunjian:
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> Currently, user mem maps will check if the newly mapped area is adjacent
> to any existing mapping, but will not check if the mapping is identical
> because it assumes that the API will never get called with the same
> mapping twice. This will result in duplicate entries in the user mem
> maps list.
> 
> Fix it by also checking for duplicate mappings, and skipping them if
> they are found.

Sorry, that's still difficult to read,
and it is not clear what is the impact of the bug.

+Cc some english native speakers for help.

> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

Re: [dpdk-stable] [dpdk-dev] [PATCH v4] eal: fix create user mem map repeatedly when it exists

[Kevin Traynor]

On 25/03/2021 14:30, Thomas Monjalon wrote:
> 07/12/2020 12:08, wangyunjian:
>> From: Yunjian Wang <wangyunjian@huawei.com>
>>
>> Currently, user mem maps will check if the newly mapped area is adjacent
>> to any existing mapping, but will not check if the mapping is identical
>> because it assumes that the API will never get called with the same
>> mapping twice. This will result in duplicate entries in the user mem
>> maps list.
>>
>> Fix it by also checking for duplicate mappings, and skipping them if
>> they are found.
> 
> Sorry, that's still difficult to read,
> and it is not clear what is the impact of the bug.
> 

I agree the impact of the bug is not clear from the description. It
seems to be explained at a low level in
http://inbox.dpdk.org/dev/34EFBCA9F01B0748BEB6B629CE643AE60DB32BD6@DGGEMM533-MBX.china.huawei.com/
that the array size is 256 (VFIO_MAX_USER_MEM_MAPS) and it may fill up
due to duplicate mem maps.

How about something like:
--
New user mem maps are checked if they are adjacent to an existing mem
map and if so, the mem map entries are merged.

It did not check for duplicate mem maps, so if the API is called with
the same mem map multiple times, they will occupy multiple mem map
entries. This will reduce the amount of entries available for unique mem
maps.

Check for duplicate mem maps and merge them into one mem map entry if
any found.
--

You might want to add something about the possible impact for
applications that is being fixed here too.

> +Cc some english native speakers for help.

(Probably the worst people to ask)

> 
>> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> 
> 
> 

[dpdk-stable] [dpdk-dev] [PATCH v5] eal: fix create user mem map repeatedly when it exists

[wangyunjian]

From: Yunjian Wang <wangyunjian@huawei.com>

Currently, new user mem maps are checked if they are adjacent to
an existing mem map and if so, the mem map entries are merged.

It doesn't check for duplicate mem maps, so if the API is called
with the same mem map multiple times, they will occupy multiple
mem map entries. This will reduce the amount of entries available
for unique mem maps.

So check for duplicate mem maps and merge them into one mem map
entry if any found.

Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
Cc: stable@dpdk.org

Suggested-by: Kevin Traynor <ktraynor@redhat.com>
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
v5:
   Update commit log suggested by Kevin Traynor
---
 lib/librte_eal/linux/eal_vfio.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c
index fe25719991..25add2fa5d 100644
--- a/lib/librte_eal/linux/eal_vfio.c
+++ b/lib/librte_eal/linux/eal_vfio.c
@@ -171,6 +171,10 @@ adjust_map(struct user_mem_map *src, struct user_mem_map *end,
 static int
 merge_map(struct user_mem_map *left, struct user_mem_map *right)
 {
+	/* merge the same maps into one */
+	if (memcmp(left, right, sizeof(struct user_mem_map)) == 0)
+		goto out;
+
 	if (left->addr + left->len != right->addr)
 		return 0;
 	if (left->iova + left->len != right->iova)
@@ -178,6 +182,7 @@ merge_map(struct user_mem_map *left, struct user_mem_map *right)
 
 	left->len += right->len;
 
+out:
 	memset(right, 0, sizeof(*right));
 
 	return 1;
-- 
2.23.0

Re: [dpdk-stable] [dpdk-dev] [PATCH v5] eal: fix create user mem map repeatedly when it exists

[Thomas Monjalon]

10/04/2021 11:37, wangyunjian:
> From: Yunjian Wang <wangyunjian@huawei.com>
> 
> Currently, new user mem maps are checked if they are adjacent to
> an existing mem map and if so, the mem map entries are merged.
> 
> It doesn't check for duplicate mem maps, so if the API is called

As Kevin suggested, it should be in the past tense: "didn't".

> with the same mem map multiple times, they will occupy multiple
> mem map entries. This will reduce the amount of entries available
> for unique mem maps.
> 
> So check for duplicate mem maps and merge them into one mem map
> entry if any found.
> 
> Fixes: 0cbce3a167f1 ("vfio: skip DMA map failure if already mapped")
> Cc: stable@dpdk.org
> 
> Suggested-by: Kevin Traynor <ktraynor@redhat.com>
> Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

Applied, thanks.