Re: [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes

[Alejandro Lucero <alejandro.lucero@netronome.com>]

On Tue, Jun 19, 2018 at 11:27 AM, Burakov, Anatoly <
anatoly.burakov@intel.com> wrote:

> On 19-Jun-18 11:23 AM, Alejandro Lucero wrote:
>
>>
>>
>> On Tue, Jun 19, 2018 at 10:24 AM, Burakov, Anatoly <
>> anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
>>
>>     On 18-Jun-18 9:12 PM, Stojaczyk, DariuszX wrote:
>>
>>
>>
>>             -----Original Message-----
>>             From: Alejandro Lucero
>>             [mailto:alejandro.lucero@netronome.com
>>             <mailto:alejandro.lucero@netronome.com>]
>>             Sent: Monday, June 18, 2018 9:33 PM
>>
>>             On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX
>>             <dariuszx.stojaczyk@intel.com
>>             <mailto:dariuszx.stojaczyk@intel.com>
>>             <mailto:dariuszx.stojaczyk@intel.com
>>
>>             <mailto:dariuszx.stojaczyk@intel.com>> >
>>             wrote:
>>
>>                      Can you point me out to an NFP guide or some code
>>             that describes
>>             this in more detail?
>>
>>
>>
>>             As I said, I'm working on a RFC. I will send something
>>             shortly. But I could give
>>             you an advance: the hugepages needs to be mapped below
>>             certain virtual
>>             address, 1TB, and I'm afraid that includes the primary and
>>             also the
>>             secondary processes. At least if any process can send or
>>             receive packets
>>             to/from a NFP.
>>
>>
>>
>>         Thanks, I'm pretty sure we're safe, then.
>>
>>
>>                      If we're talking about base-virtaddr for hugepages,
>>             then that's always
>>             inherited from the primary process, regardless of what
>>             base-virtaddr is
>>             supplied to the secondary.
>>
>>
>>
>>
>>             But, is not your patch avoiding to use that base-virtaddr
>>             for secondary
>>             processes?
>>
>>
>>         I see now that the patch name is slightly misleading. Maybe I
>>         shouldn’t pick such a catchy title. Let me clarify: As of DPDK
>>         18.05, --base-virtaddr param for secondary process applications
>>         only affects that shadow memseg metadata that's not useful for
>>         anyone, but can still do a lot of harm. Hugepage memory in
>>         secondary processes is always mapped to the same addresses the
>>         primary process uses.
>>
>>         D.
>>
>>
>>     Hi Alejandro,
>>
>>     To solve this problem, one possible approach would be to have
>>     maximum VA address, and allocate memory downwards, rather than
>>     upwards. Is that by any chance approximate contents of your RFC? :)
>>
>>
>> Hi Anatoly,
>>
>> There's a lot of space below 1TB, but this specific upper limit is just
>> in the NFP case. My RFC will propose a generic solution assuming there
>> could be other devices in the future, and not just NIC devices, which could
>> have also some sort of limitation.
>>
>
> OK, looking forward to it.
>
> The problem is, those devices limitations can not be known at memory
>> initialization time, so some sort of check is required afterwards, once the
>> devices have been proved and initialised.
>>
>>
> Presumably, device driver would be in the best position to know things
> like that, no? Maybe a new device flag and an API to query such things from
> all devices?
>
>
Right. I plan to add a per-device DMA mask, just mimicking what the kernel
has.


> --
> Thanks,
> Anatoly
>