Only a single DPDK process on the system can be using the /dev/contigmem mappings at a time, but this was never explicitly enforced, e.g. when using --in-memory flag on two processes. To prevent possible conflict issues, we lock the dev node when it's in use, preventing other DPDK processes from starting up and causing problems for us. Fixes: 764bf26873b9 ("add FreeBSD support") Cc: stable@dpdk.org Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> --- lib/eal/freebsd/eal_hugepage_info.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c index 408f054f7a..4a8d87c23e 100644 --- a/lib/eal/freebsd/eal_hugepage_info.c +++ b/lib/eal/freebsd/eal_hugepage_info.c @@ -90,6 +90,10 @@ eal_hugepage_info_init(void) RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n"); return -1; } + if (flock(fd, LOCK_EX) < 0) { + RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n"); + return -1; + } if (buffer_size >= 1<<30) RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n", -- 2.30.2
On 13-Sep-21 12:06 PM, Bruce Richardson wrote:
> Only a single DPDK process on the system can be using the /dev/contigmem
> mappings at a time, but this was never explicitly enforced, e.g. when
> using --in-memory flag on two processes. To prevent possible conflict
> issues, we lock the dev node when it's in use, preventing other DPDK
> processes from starting up and causing problems for us.
>
> Fixes: 764bf26873b9 ("add FreeBSD support")
> Cc: stable@dpdk.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> ---
> lib/eal/freebsd/eal_hugepage_info.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c
> index 408f054f7a..4a8d87c23e 100644
> --- a/lib/eal/freebsd/eal_hugepage_info.c
> +++ b/lib/eal/freebsd/eal_hugepage_info.c
> @@ -90,6 +90,10 @@ eal_hugepage_info_init(void)
> RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n");
> return -1;
> }
> + if (flock(fd, LOCK_EX) < 0) {
> + RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n");
> + return -1;
> + }
>
> if (buffer_size >= 1<<30)
> RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n",
>
This only gets triggered when regular init path is chosen, i.e.
--no-huge still works. I'm a bit uneasy with --in-memory mode pretending
to work on FreeBSD and Windows, but that's a separate problem :) As far
as the patch goes, the problem it addresses does get fixed.
Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
--
Thanks,
Anatoly
On Mon, Sep 13, 2021 at 02:14:55PM +0100, Burakov, Anatoly wrote: > On 13-Sep-21 12:06 PM, Bruce Richardson wrote: > > Only a single DPDK process on the system can be using the /dev/contigmem > > mappings at a time, but this was never explicitly enforced, e.g. when > > using --in-memory flag on two processes. To prevent possible conflict > > issues, we lock the dev node when it's in use, preventing other DPDK > > processes from starting up and causing problems for us. > > > > Fixes: 764bf26873b9 ("add FreeBSD support") > > Cc: stable@dpdk.org > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > > --- > > lib/eal/freebsd/eal_hugepage_info.c | 4 ++++ > > 1 file changed, 4 insertions(+) > > > > diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c > > index 408f054f7a..4a8d87c23e 100644 > > --- a/lib/eal/freebsd/eal_hugepage_info.c > > +++ b/lib/eal/freebsd/eal_hugepage_info.c > > @@ -90,6 +90,10 @@ eal_hugepage_info_init(void) > > RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n"); > > return -1; > > } > > + if (flock(fd, LOCK_EX) < 0) { > > + RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n"); > > + return -1; > > + } > > if (buffer_size >= 1<<30) > > RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n", > > > > This only gets triggered when regular init path is chosen, i.e. --no-huge > still works. Yes, but that is ok, I think, since no-huge doesn't use these resources or suffer from this problem. On the other hand, except for running unit tests, no-huge mode is pretty useless on FreeBSD as we don't have any vfio-equivalent support, so all HW access has to use physical addresses which can only be got using contigmem. > I'm a bit uneasy with --in-memory mode pretending to work on > FreeBSD and Windows, but that's a separate problem :) Yes, it is, though one that does belong is the same area as this one. The "fix" is probably to just print a warning when --in-memory is used, informing the user that the flag is ignored and then continue. Alternatively we can error out, but I think the warn+continue is better, myself. > As far as the patch > goes, the problem it addresses does get fixed. > > Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com> > Thanks. /Bruce
2021-09-13 14:14 (UTC+0100), Burakov, Anatoly:
> [...]
> I'm a bit uneasy with --in-memory mode pretending
> to work on FreeBSD and Windows, but that's a separate problem :)
On Windows, --in-memory does not pretend to work, just the opposite,
it is enabled implicitly as it's the only working mode.
Only a single DPDK process on the system can be using the /dev/contigmem mappings at a time, but this was never explicitly enforced, e.g. when using --in-memory flag on two processes. To prevent possible conflict issues, we lock the dev node when it's in use, preventing other DPDK processes from starting up and causing problems for us. Fixes: 764bf26873b9 ("add FreeBSD support") Cc: stable@dpdk.org Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com> --- V2: Adding missing LOCK_NB flag to make sure the process doesn't sit waiting for the lock to be released lib/eal/freebsd/eal_hugepage_info.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c index 408f054f7a..9dbe375bd3 100644 --- a/lib/eal/freebsd/eal_hugepage_info.c +++ b/lib/eal/freebsd/eal_hugepage_info.c @@ -90,6 +90,10 @@ eal_hugepage_info_init(void) RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n"); return -1; } + if (flock(fd, LOCK_EX | LOCK_NB) < 0) { + RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n"); + return -1; + } if (buffer_size >= 1<<30) RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n", -- 2.30.2
On 13-Sep-21 2:36 PM, Bruce Richardson wrote: > On Mon, Sep 13, 2021 at 02:14:55PM +0100, Burakov, Anatoly wrote: >> On 13-Sep-21 12:06 PM, Bruce Richardson wrote: >>> Only a single DPDK process on the system can be using the /dev/contigmem >>> mappings at a time, but this was never explicitly enforced, e.g. when >>> using --in-memory flag on two processes. To prevent possible conflict >>> issues, we lock the dev node when it's in use, preventing other DPDK >>> processes from starting up and causing problems for us. >>> >>> Fixes: 764bf26873b9 ("add FreeBSD support") >>> Cc: stable@dpdk.org >>> >>> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> >>> --- >>> lib/eal/freebsd/eal_hugepage_info.c | 4 ++++ >>> 1 file changed, 4 insertions(+) >>> >>> diff --git a/lib/eal/freebsd/eal_hugepage_info.c b/lib/eal/freebsd/eal_hugepage_info.c >>> index 408f054f7a..4a8d87c23e 100644 >>> --- a/lib/eal/freebsd/eal_hugepage_info.c >>> +++ b/lib/eal/freebsd/eal_hugepage_info.c >>> @@ -90,6 +90,10 @@ eal_hugepage_info_init(void) >>> RTE_LOG(ERR, EAL, "could not open "CONTIGMEM_DEV"\n"); >>> return -1; >>> } >>> + if (flock(fd, LOCK_EX) < 0) { >>> + RTE_LOG(ERR, EAL, "could not lock memory. Is another DPDK process running?\n"); >>> + return -1; >>> + } >>> if (buffer_size >= 1<<30) >>> RTE_LOG(INFO, EAL, "Contigmem driver has %d buffers, each of size %dGB\n", >>> >> >> This only gets triggered when regular init path is chosen, i.e. --no-huge >> still works. > > Yes, but that is ok, I think, since no-huge doesn't use these resources or > suffer from this problem. On the other hand, except for running unit tests, > no-huge mode is pretty useless on FreeBSD as we don't have any > vfio-equivalent support, so all HW access has to use physical addresses > which can only be got using contigmem. What i meant to say was, i've checked this against '--no-huge' which *should* still work with this patch, and it does :) So, the phrasing was unfortunate, but we agree! > >> I'm a bit uneasy with --in-memory mode pretending to work on >> FreeBSD and Windows, but that's a separate problem :) > > Yes, it is, though one that does belong is the same area as this one. The > "fix" is probably to just print a warning when --in-memory is used, > informing the user that the flag is ignored and then continue. > Alternatively we can error out, but I think the warn+continue is better, > myself. I think erroring out is better. The feature is intended to work a certain way, so if we can't guarantee that it does, we can't pretend it is "supported" or "is working". But again, irrelevant to this patch :) > >> As far as the patch >> goes, the problem it addresses does get fixed. >> >> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com> >> > Thanks. > > /Bruce > -- Thanks, Anatoly
On Mon, Sep 13, 2021 at 3:15 PM Burakov, Anatoly
<anatoly.burakov@intel.com> wrote:
> On 13-Sep-21 12:06 PM, Bruce Richardson wrote:
> > Only a single DPDK process on the system can be using the /dev/contigmem
> > mappings at a time, but this was never explicitly enforced, e.g. when
> > using --in-memory flag on two processes. To prevent possible conflict
> > issues, we lock the dev node when it's in use, preventing other DPDK
> > processes from starting up and causing problems for us.
> >
> > Fixes: 764bf26873b9 ("add FreeBSD support")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
Applied, thanks.
--
David Marchand
On Mon, Sep 13, 2021 at 4:40 PM Burakov, Anatoly <anatoly.burakov@intel.com> wrote: > >> I'm a bit uneasy with --in-memory mode pretending to work on > >> FreeBSD and Windows, but that's a separate problem :) > > > > Yes, it is, though one that does belong is the same area as this one. The > > "fix" is probably to just print a warning when --in-memory is used, > > informing the user that the flag is ignored and then continue. > > Alternatively we can error out, but I think the warn+continue is better, > > myself. > > I think erroring out is better. The feature is intended to work a > certain way, so if we can't guarantee that it does, we can't pretend it > is "supported" or "is working". But again, irrelevant to this patch :) > Please, can you review https://patchwork.dpdk.org/project/dpdk/patch/20210913140848.184928-1-bruce.richardson@intel.com/ ? Thanks. -- David Marchand
13/09/2021 16:08, Bruce Richardson:
> Only a single DPDK process on the system can be using the /dev/contigmem
> mappings at a time, but this was never explicitly enforced, e.g. when
> using --in-memory flag on two processes. To prevent possible conflict
> issues, we lock the dev node when it's in use, preventing other DPDK
> processes from starting up and causing problems for us.
>
> Fixes: 764bf26873b9 ("add FreeBSD support")
> Cc: stable@dpdk.org
>
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
> Reviewed-by: Anatoly Burakov <anatoly.burakov@intel.com>
Looks to be applied by David.