DPDK patches and discussions
 help / color / mirror / Atom feed
* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-18 19:53 [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes Dariusz Stojaczyk
@ 2018-06-18 17:21 ` Alejandro Lucero
  2018-06-18 19:03   ` Stojaczyk, DariuszX
  2018-06-19  9:21 ` Burakov, Anatoly
  1 sibling, 1 reply; 11+ messages in thread
From: Alejandro Lucero @ 2018-06-18 17:21 UTC (permalink / raw)
  To: Dariusz Stojaczyk; +Cc: dev, Anatoly Burakov, stable

On Mon, Jun 18, 2018 at 8:53 PM, Dariusz Stojaczyk <
dariuszx.stojaczyk@intel.com> wrote:

> Since secondary process' address space is highly dictated
> by the primary process' mappings, it doesn't make much
> sense to use base-virtaddr for secondary processes.
>
> This patch is intended to fix PCI resource mapping
> in secondary processes using the same base-virtaddr
> as their primary processes. PCI uses the end of the hugepage
> memory area to map all resources. [pci_find_max_end_va()]
> It works for primary processes, but can't be mapped 1:1
> by secondary ones, as the same addresses are currently always
> occupied by shadow memseg lists, which were created with
> eal_get_virtual_area(NULL, ...).
>
>
Should not be better to handle these allocations being aware about the
problem for secondary processes?

I do not know exactly what are the (other) reasons behind base-virtaddr,
but it turns out NFP requires this to be used when DPDK apps executed by
non-root users.

I'm working on a RFC for handling our specific case, that could also be
required for other devices, and this change would make the NFP unusable for
the secondary processes.



> ```
> PRIMARY PROCESS
> 0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
> 0x6e01000000 16777216K r----   [ anon ]
> 0x7201000000     16K rw-s- resource0
>
> SECONDARY PROCESS
> 0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
> 0x6e01000000 16777216K r----   [ anon ]
> 0x7201000000      4K rw-s- fbarray_memseg-1048576k-0-0_203213
> ```
>
> Fixes: 524e43c2ad9a ("mem: prepare memseg lists for multiprocess sync")
> Cc: anatoly.burakov@intel.com
> Cc: stable@dpdk.org
>
> Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
> ---
>  lib/librte_eal/common/eal_common_memory.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_eal/common/eal_common_memory.c
> b/lib/librte_eal/common/eal_common_memory.c
> index 53d9b51..6c47e11 100644
> --- a/lib/librte_eal/common/eal_common_memory.c
> +++ b/lib/librte_eal/common/eal_common_memory.c
> @@ -56,7 +56,8 @@ eal_get_virtual_area(void *requested_addr, size_t *size,
>         allow_shrink = (flags & EAL_VIRTUAL_AREA_ALLOW_SHRINK) > 0;
>         unmap = (flags & EAL_VIRTUAL_AREA_UNMAP) > 0;
>
> -       if (next_baseaddr == NULL && internal_config.base_virtaddr != 0)
> +       if (next_baseaddr == NULL && internal_config.base_virtaddr != 0 &&
> +                       rte_eal_process_type() == RTE_PROC_PRIMARY)
>                 next_baseaddr = (void *) internal_config.base_virtaddr;
>
>         if (requested_addr == NULL && next_baseaddr) {
> --
> 2.7.4
>
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-18 17:21 ` Alejandro Lucero
@ 2018-06-18 19:03   ` Stojaczyk, DariuszX
  2018-06-18 19:33     ` Alejandro Lucero
  0 siblings, 1 reply; 11+ messages in thread
From: Stojaczyk, DariuszX @ 2018-06-18 19:03 UTC (permalink / raw)
  To: Alejandro Lucero; +Cc: dev, Burakov, Anatoly, stable


> -----Original Message-----
> From: Alejandro Lucero [mailto:alejandro.lucero@netronome.com]
> Sent: Monday, June 18, 2018 7:22 PM
> 
> Should not be better to handle these allocations being aware about the
> problem for secondary processes?
> 
> I do not know exactly what are the (other) reasons behind base-virtaddr,
> but it turns out NFP requires this to be used when DPDK apps executed
> by non-root users.
> 
> I'm working on a RFC for handling our specific case, that could also be
> required for other devices, and this change would make the NFP unusable
> for the secondary processes.
> 

The only place base-virtaddr is used in secondary processes in DPDK 18.05 is this shadow memseg mapping, which shouldn't really need to be accessed by anyone else than DPDK EAL. Can you point me out to an NFP guide or some code that describes this in more detail?

If we're talking about base-virtaddr for hugepages, then that's always inherited from the primary process, regardless of what base-virtaddr is supplied to the secondary.

Regards!
D.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-18 19:03   ` Stojaczyk, DariuszX
@ 2018-06-18 19:33     ` Alejandro Lucero
  2018-06-18 20:12       ` Stojaczyk, DariuszX
  0 siblings, 1 reply; 11+ messages in thread
From: Alejandro Lucero @ 2018-06-18 19:33 UTC (permalink / raw)
  To: Stojaczyk, DariuszX; +Cc: dev, Burakov, Anatoly, stable

On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX <
dariuszx.stojaczyk@intel.com> wrote:

>
> > -----Original Message-----
> > From: Alejandro Lucero [mailto:alejandro.lucero@netronome.com]
> > Sent: Monday, June 18, 2018 7:22 PM
> >
> > Should not be better to handle these allocations being aware about the
> > problem for secondary processes?
> >
> > I do not know exactly what are the (other) reasons behind base-virtaddr,
> > but it turns out NFP requires this to be used when DPDK apps executed
> > by non-root users.
> >
> > I'm working on a RFC for handling our specific case, that could also be
> > required for other devices, and this change would make the NFP unusable
> > for the secondary processes.
> >
>
> The only place base-virtaddr is used in secondary processes in DPDK 18.05
> is this shadow memseg mapping, which shouldn't really need to be accessed
> by anyone else than DPDK EAL.


Yes, I'm aware this is EAL code.

Can you point me out to an NFP guide or some code that describes this in
> more detail?
>

As I said, I'm working on a RFC. I will send something shortly. But I could
give you an advance: the hugepages needs to be mapped below certain virtual
address, 1TB, and I'm afraid that includes the primary and also the
secondary processes. At least if any process can send or receive packets
to/from a NFP.


>
> If we're talking about base-virtaddr for hugepages, then that's always
> inherited from the primary process, regardless of what base-virtaddr is
> supplied to the secondary.
>
>
But, is not your patch avoiding to use that base-virtaddr for secondary
processes?


> Regards!
> D.
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
@ 2018-06-18 19:53 Dariusz Stojaczyk
  2018-06-18 17:21 ` Alejandro Lucero
  2018-06-19  9:21 ` Burakov, Anatoly
  0 siblings, 2 replies; 11+ messages in thread
From: Dariusz Stojaczyk @ 2018-06-18 19:53 UTC (permalink / raw)
  To: dev, Anatoly Burakov; +Cc: Dariusz Stojaczyk, stable

Since secondary process' address space is highly dictated
by the primary process' mappings, it doesn't make much
sense to use base-virtaddr for secondary processes.

This patch is intended to fix PCI resource mapping
in secondary processes using the same base-virtaddr
as their primary processes. PCI uses the end of the hugepage
memory area to map all resources. [pci_find_max_end_va()]
It works for primary processes, but can't be mapped 1:1
by secondary ones, as the same addresses are currently always
occupied by shadow memseg lists, which were created with
eal_get_virtual_area(NULL, ...).

```
PRIMARY PROCESS
0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
0x6e01000000 16777216K r----   [ anon ]
0x7201000000     16K rw-s- resource0

SECONDARY PROCESS
0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
0x6e01000000 16777216K r----   [ anon ]
0x7201000000      4K rw-s- fbarray_memseg-1048576k-0-0_203213
```

Fixes: 524e43c2ad9a ("mem: prepare memseg lists for multiprocess sync")
Cc: anatoly.burakov@intel.com
Cc: stable@dpdk.org

Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
---
 lib/librte_eal/common/eal_common_memory.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eal/common/eal_common_memory.c b/lib/librte_eal/common/eal_common_memory.c
index 53d9b51..6c47e11 100644
--- a/lib/librte_eal/common/eal_common_memory.c
+++ b/lib/librte_eal/common/eal_common_memory.c
@@ -56,7 +56,8 @@ eal_get_virtual_area(void *requested_addr, size_t *size,
 	allow_shrink = (flags & EAL_VIRTUAL_AREA_ALLOW_SHRINK) > 0;
 	unmap = (flags & EAL_VIRTUAL_AREA_UNMAP) > 0;
 
-	if (next_baseaddr == NULL && internal_config.base_virtaddr != 0)
+	if (next_baseaddr == NULL && internal_config.base_virtaddr != 0 &&
+			rte_eal_process_type() == RTE_PROC_PRIMARY)
 		next_baseaddr = (void *) internal_config.base_virtaddr;
 
 	if (requested_addr == NULL && next_baseaddr) {
-- 
2.7.4

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-18 19:33     ` Alejandro Lucero
@ 2018-06-18 20:12       ` Stojaczyk, DariuszX
  2018-06-19  9:24         ` Burakov, Anatoly
  0 siblings, 1 reply; 11+ messages in thread
From: Stojaczyk, DariuszX @ 2018-06-18 20:12 UTC (permalink / raw)
  To: Alejandro Lucero; +Cc: dev, Burakov, Anatoly, stable



> -----Original Message-----
> From: Alejandro Lucero [mailto:alejandro.lucero@netronome.com]
> Sent: Monday, June 18, 2018 9:33 PM
> 
> On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX
> <dariuszx.stojaczyk@intel.com <mailto:dariuszx.stojaczyk@intel.com> >
> wrote:
> 
> 	Can you point me out to an NFP guide or some code that describes
> this in more detail?
> 
> 
> 
> As I said, I'm working on a RFC. I will send something shortly. But I could give
> you an advance: the hugepages needs to be mapped below certain virtual
> address, 1TB, and I'm afraid that includes the primary and also the
> secondary processes. At least if any process can send or receive packets
> to/from a NFP.
> 
> 

Thanks, I'm pretty sure we're safe, then.

> 
> 	If we're talking about base-virtaddr for hugepages, then that's always
> inherited from the primary process, regardless of what base-virtaddr is
> supplied to the secondary.
> 
> 
> 
> 
> But, is not your patch avoiding to use that base-virtaddr for secondary
> processes?

I see now that the patch name is slightly misleading. Maybe I shouldn’t pick such a catchy title. Let me clarify: As of DPDK 18.05, --base-virtaddr param for secondary process applications only affects that shadow memseg metadata that's not useful for anyone, but can still do a lot of harm. Hugepage memory in secondary processes is always mapped to the same addresses the primary process uses.

D.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-18 19:53 [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes Dariusz Stojaczyk
  2018-06-18 17:21 ` Alejandro Lucero
@ 2018-06-19  9:21 ` Burakov, Anatoly
  2018-07-12 22:08   ` Thomas Monjalon
  1 sibling, 1 reply; 11+ messages in thread
From: Burakov, Anatoly @ 2018-06-19  9:21 UTC (permalink / raw)
  To: Dariusz Stojaczyk, dev; +Cc: stable

On 18-Jun-18 8:53 PM, Dariusz Stojaczyk wrote:
> Since secondary process' address space is highly dictated
> by the primary process' mappings, it doesn't make much
> sense to use base-virtaddr for secondary processes.
> 
> This patch is intended to fix PCI resource mapping
> in secondary processes using the same base-virtaddr
> as their primary processes. PCI uses the end of the hugepage
> memory area to map all resources. [pci_find_max_end_va()]
> It works for primary processes, but can't be mapped 1:1
> by secondary ones, as the same addresses are currently always
> occupied by shadow memseg lists, which were created with
> eal_get_virtual_area(NULL, ...).
> 
> ```
> PRIMARY PROCESS
> 0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
> 0x6e01000000 16777216K r----   [ anon ]
> 0x7201000000     16K rw-s- resource0
> 
> SECONDARY PROCESS
> 0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
> 0x6e01000000 16777216K r----   [ anon ]
> 0x7201000000      4K rw-s- fbarray_memseg-1048576k-0-0_203213
> ```
> 
> Fixes: 524e43c2ad9a ("mem: prepare memseg lists for multiprocess sync")
> Cc: anatoly.burakov@intel.com
> Cc: stable@dpdk.org
> 
> Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
> ---

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-18 20:12       ` Stojaczyk, DariuszX
@ 2018-06-19  9:24         ` Burakov, Anatoly
  2018-06-19 10:23           ` Alejandro Lucero
  0 siblings, 1 reply; 11+ messages in thread
From: Burakov, Anatoly @ 2018-06-19  9:24 UTC (permalink / raw)
  To: Stojaczyk, DariuszX, Alejandro Lucero; +Cc: dev, stable

On 18-Jun-18 9:12 PM, Stojaczyk, DariuszX wrote:
> 
> 
>> -----Original Message-----
>> From: Alejandro Lucero [mailto:alejandro.lucero@netronome.com]
>> Sent: Monday, June 18, 2018 9:33 PM
>>
>> On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX
>> <dariuszx.stojaczyk@intel.com <mailto:dariuszx.stojaczyk@intel.com> >
>> wrote:
>>
>> 	Can you point me out to an NFP guide or some code that describes
>> this in more detail?
>>
>>
>>
>> As I said, I'm working on a RFC. I will send something shortly. But I could give
>> you an advance: the hugepages needs to be mapped below certain virtual
>> address, 1TB, and I'm afraid that includes the primary and also the
>> secondary processes. At least if any process can send or receive packets
>> to/from a NFP.
>>
>>
> 
> Thanks, I'm pretty sure we're safe, then.
> 
>>
>> 	If we're talking about base-virtaddr for hugepages, then that's always
>> inherited from the primary process, regardless of what base-virtaddr is
>> supplied to the secondary.
>>
>>
>>
>>
>> But, is not your patch avoiding to use that base-virtaddr for secondary
>> processes?
> 
> I see now that the patch name is slightly misleading. Maybe I shouldn’t pick such a catchy title. Let me clarify: As of DPDK 18.05, --base-virtaddr param for secondary process applications only affects that shadow memseg metadata that's not useful for anyone, but can still do a lot of harm. Hugepage memory in secondary processes is always mapped to the same addresses the primary process uses.
> 
> D.
> 

Hi Alejandro,

To solve this problem, one possible approach would be to have maximum VA 
address, and allocate memory downwards, rather than upwards. Is that by 
any chance approximate contents of your RFC? :)

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-19  9:24         ` Burakov, Anatoly
@ 2018-06-19 10:23           ` Alejandro Lucero
  2018-06-19 10:27             ` Burakov, Anatoly
  0 siblings, 1 reply; 11+ messages in thread
From: Alejandro Lucero @ 2018-06-19 10:23 UTC (permalink / raw)
  To: Burakov, Anatoly; +Cc: Stojaczyk, DariuszX, dev, stable

On Tue, Jun 19, 2018 at 10:24 AM, Burakov, Anatoly <
anatoly.burakov@intel.com> wrote:

> On 18-Jun-18 9:12 PM, Stojaczyk, DariuszX wrote:
>
>>
>>
>> -----Original Message-----
>>> From: Alejandro Lucero [mailto:alejandro.lucero@netronome.com]
>>> Sent: Monday, June 18, 2018 9:33 PM
>>>
>>> On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX
>>> <dariuszx.stojaczyk@intel.com <mailto:dariuszx.stojaczyk@intel.com> >
>>> wrote:
>>>
>>>         Can you point me out to an NFP guide or some code that describes
>>> this in more detail?
>>>
>>>
>>>
>>> As I said, I'm working on a RFC. I will send something shortly. But I
>>> could give
>>> you an advance: the hugepages needs to be mapped below certain virtual
>>> address, 1TB, and I'm afraid that includes the primary and also the
>>> secondary processes. At least if any process can send or receive packets
>>> to/from a NFP.
>>>
>>>
>>>
>> Thanks, I'm pretty sure we're safe, then.
>>
>>
>>>         If we're talking about base-virtaddr for hugepages, then that's
>>> always
>>> inherited from the primary process, regardless of what base-virtaddr is
>>> supplied to the secondary.
>>>
>>>
>>>
>>>
>>> But, is not your patch avoiding to use that base-virtaddr for secondary
>>> processes?
>>>
>>
>> I see now that the patch name is slightly misleading. Maybe I shouldn’t
>> pick such a catchy title. Let me clarify: As of DPDK 18.05, --base-virtaddr
>> param for secondary process applications only affects that shadow memseg
>> metadata that's not useful for anyone, but can still do a lot of harm.
>> Hugepage memory in secondary processes is always mapped to the same
>> addresses the primary process uses.
>>
>> D.
>>
>>
> Hi Alejandro,
>
> To solve this problem, one possible approach would be to have maximum VA
> address, and allocate memory downwards, rather than upwards. Is that by any
> chance approximate contents of your RFC? :)
>
>
Hi Anatoly,

There's a lot of space below 1TB, but this specific upper limit is just in
the NFP case. My RFC will propose a generic solution assuming there could
be other devices in the future, and not just NIC devices, which could have
also some sort of limitation. The problem is, those devices limitations can
not be known at memory initialization time, so some sort of check is
required afterwards, once the devices have been proved and initialised.


> --
> Thanks,
> Anatoly
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-19 10:23           ` Alejandro Lucero
@ 2018-06-19 10:27             ` Burakov, Anatoly
  2018-06-19 11:48               ` Alejandro Lucero
  0 siblings, 1 reply; 11+ messages in thread
From: Burakov, Anatoly @ 2018-06-19 10:27 UTC (permalink / raw)
  To: Alejandro Lucero; +Cc: Stojaczyk, DariuszX, dev, stable

On 19-Jun-18 11:23 AM, Alejandro Lucero wrote:
> 
> 
> On Tue, Jun 19, 2018 at 10:24 AM, Burakov, Anatoly 
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> 
>     On 18-Jun-18 9:12 PM, Stojaczyk, DariuszX wrote:
> 
> 
> 
>             -----Original Message-----
>             From: Alejandro Lucero
>             [mailto:alejandro.lucero@netronome.com
>             <mailto:alejandro.lucero@netronome.com>]
>             Sent: Monday, June 18, 2018 9:33 PM
> 
>             On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX
>             <dariuszx.stojaczyk@intel.com
>             <mailto:dariuszx.stojaczyk@intel.com>
>             <mailto:dariuszx.stojaczyk@intel.com
>             <mailto:dariuszx.stojaczyk@intel.com>> >
>             wrote:
> 
>                      Can you point me out to an NFP guide or some code
>             that describes
>             this in more detail?
> 
> 
> 
>             As I said, I'm working on a RFC. I will send something
>             shortly. But I could give
>             you an advance: the hugepages needs to be mapped below
>             certain virtual
>             address, 1TB, and I'm afraid that includes the primary and
>             also the
>             secondary processes. At least if any process can send or
>             receive packets
>             to/from a NFP.
> 
> 
> 
>         Thanks, I'm pretty sure we're safe, then.
> 
> 
>                      If we're talking about base-virtaddr for hugepages,
>             then that's always
>             inherited from the primary process, regardless of what
>             base-virtaddr is
>             supplied to the secondary.
> 
> 
> 
> 
>             But, is not your patch avoiding to use that base-virtaddr
>             for secondary
>             processes?
> 
> 
>         I see now that the patch name is slightly misleading. Maybe I
>         shouldn’t pick such a catchy title. Let me clarify: As of DPDK
>         18.05, --base-virtaddr param for secondary process applications
>         only affects that shadow memseg metadata that's not useful for
>         anyone, but can still do a lot of harm. Hugepage memory in
>         secondary processes is always mapped to the same addresses the
>         primary process uses.
> 
>         D.
> 
> 
>     Hi Alejandro,
> 
>     To solve this problem, one possible approach would be to have
>     maximum VA address, and allocate memory downwards, rather than
>     upwards. Is that by any chance approximate contents of your RFC? :)
> 
> 
> Hi Anatoly,
> 
> There's a lot of space below 1TB, but this specific upper limit is just 
> in the NFP case. My RFC will propose a generic solution assuming there 
> could be other devices in the future, and not just NIC devices, which 
> could have also some sort of limitation.

OK, looking forward to it.

> The problem is, those devices 
> limitations can not be known at memory initialization time, so some sort 
> of check is required afterwards, once the devices have been proved and 
> initialised.
> 

Presumably, device driver would be in the best position to know things 
like that, no? Maybe a new device flag and an API to query such things 
from all devices?

-- 
Thanks,
Anatoly

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-19 10:27             ` Burakov, Anatoly
@ 2018-06-19 11:48               ` Alejandro Lucero
  0 siblings, 0 replies; 11+ messages in thread
From: Alejandro Lucero @ 2018-06-19 11:48 UTC (permalink / raw)
  To: Burakov, Anatoly; +Cc: Stojaczyk, DariuszX, dev, stable

On Tue, Jun 19, 2018 at 11:27 AM, Burakov, Anatoly <
anatoly.burakov@intel.com> wrote:

> On 19-Jun-18 11:23 AM, Alejandro Lucero wrote:
>
>>
>>
>> On Tue, Jun 19, 2018 at 10:24 AM, Burakov, Anatoly <
>> anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
>>
>>     On 18-Jun-18 9:12 PM, Stojaczyk, DariuszX wrote:
>>
>>
>>
>>             -----Original Message-----
>>             From: Alejandro Lucero
>>             [mailto:alejandro.lucero@netronome.com
>>             <mailto:alejandro.lucero@netronome.com>]
>>             Sent: Monday, June 18, 2018 9:33 PM
>>
>>             On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX
>>             <dariuszx.stojaczyk@intel.com
>>             <mailto:dariuszx.stojaczyk@intel.com>
>>             <mailto:dariuszx.stojaczyk@intel.com
>>
>>             <mailto:dariuszx.stojaczyk@intel.com>> >
>>             wrote:
>>
>>                      Can you point me out to an NFP guide or some code
>>             that describes
>>             this in more detail?
>>
>>
>>
>>             As I said, I'm working on a RFC. I will send something
>>             shortly. But I could give
>>             you an advance: the hugepages needs to be mapped below
>>             certain virtual
>>             address, 1TB, and I'm afraid that includes the primary and
>>             also the
>>             secondary processes. At least if any process can send or
>>             receive packets
>>             to/from a NFP.
>>
>>
>>
>>         Thanks, I'm pretty sure we're safe, then.
>>
>>
>>                      If we're talking about base-virtaddr for hugepages,
>>             then that's always
>>             inherited from the primary process, regardless of what
>>             base-virtaddr is
>>             supplied to the secondary.
>>
>>
>>
>>
>>             But, is not your patch avoiding to use that base-virtaddr
>>             for secondary
>>             processes?
>>
>>
>>         I see now that the patch name is slightly misleading. Maybe I
>>         shouldn’t pick such a catchy title. Let me clarify: As of DPDK
>>         18.05, --base-virtaddr param for secondary process applications
>>         only affects that shadow memseg metadata that's not useful for
>>         anyone, but can still do a lot of harm. Hugepage memory in
>>         secondary processes is always mapped to the same addresses the
>>         primary process uses.
>>
>>         D.
>>
>>
>>     Hi Alejandro,
>>
>>     To solve this problem, one possible approach would be to have
>>     maximum VA address, and allocate memory downwards, rather than
>>     upwards. Is that by any chance approximate contents of your RFC? :)
>>
>>
>> Hi Anatoly,
>>
>> There's a lot of space below 1TB, but this specific upper limit is just
>> in the NFP case. My RFC will propose a generic solution assuming there
>> could be other devices in the future, and not just NIC devices, which could
>> have also some sort of limitation.
>>
>
> OK, looking forward to it.
>
> The problem is, those devices limitations can not be known at memory
>> initialization time, so some sort of check is required afterwards, once the
>> devices have been proved and initialised.
>>
>>
> Presumably, device driver would be in the best position to know things
> like that, no? Maybe a new device flag and an API to query such things from
> all devices?
>
>
Right. I plan to add a per-device DMA mask, just mimicking what the kernel
has.


> --
> Thanks,
> Anatoly
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes
  2018-06-19  9:21 ` Burakov, Anatoly
@ 2018-07-12 22:08   ` Thomas Monjalon
  0 siblings, 0 replies; 11+ messages in thread
From: Thomas Monjalon @ 2018-07-12 22:08 UTC (permalink / raw)
  To: Dariusz Stojaczyk; +Cc: dev, Burakov, Anatoly, stable

19/06/2018 11:21, Burakov, Anatoly:
> On 18-Jun-18 8:53 PM, Dariusz Stojaczyk wrote:
> > Since secondary process' address space is highly dictated
> > by the primary process' mappings, it doesn't make much
> > sense to use base-virtaddr for secondary processes.
> > 
> > This patch is intended to fix PCI resource mapping
> > in secondary processes using the same base-virtaddr
> > as their primary processes. PCI uses the end of the hugepage
> > memory area to map all resources. [pci_find_max_end_va()]
> > It works for primary processes, but can't be mapped 1:1
> > by secondary ones, as the same addresses are currently always
> > occupied by shadow memseg lists, which were created with
> > eal_get_virtual_area(NULL, ...).
> > 
> > ```
> > PRIMARY PROCESS
> > 0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
> > 0x6e01000000 16777216K r----   [ anon ]
> > 0x7201000000     16K rw-s- resource0
> > 
> > SECONDARY PROCESS
> > 0x6e00e00000    388K rw-s- fbarray_memseg-2048k-1-3
> > 0x6e01000000 16777216K r----   [ anon ]
> > 0x7201000000      4K rw-s- fbarray_memseg-1048576k-0-0_203213
> > ```
> > 
> > Fixes: 524e43c2ad9a ("mem: prepare memseg lists for multiprocess sync")
> > Cc: anatoly.burakov@intel.com
> > Cc: stable@dpdk.org
> > 
> > Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
> > ---
> 
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

Applied, thanks

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2018-07-12 22:08 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-06-18 19:53 [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes Dariusz Stojaczyk
2018-06-18 17:21 ` Alejandro Lucero
2018-06-18 19:03   ` Stojaczyk, DariuszX
2018-06-18 19:33     ` Alejandro Lucero
2018-06-18 20:12       ` Stojaczyk, DariuszX
2018-06-19  9:24         ` Burakov, Anatoly
2018-06-19 10:23           ` Alejandro Lucero
2018-06-19 10:27             ` Burakov, Anatoly
2018-06-19 11:48               ` Alejandro Lucero
2018-06-19  9:21 ` Burakov, Anatoly
2018-07-12 22:08   ` Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).