DPDK patches and discussions
 help / color / mirror / Atom feed
From: Anatoly Burakov <anatoly.burakov@intel.com>
To: dev@dpdk.org
Cc: bruce.richardson@intel.com, thomas@monjalon.net
Subject: [dpdk-dev] [PATCH v2 0/2] Fix file locking in EAL memory
Date: Tue, 24 Apr 2018 16:54:21 +0100	[thread overview]
Message-ID: <cover.1524585160.git.anatoly.burakov@intel.com> (raw)
In-Reply-To: <cover.1524140413.git.anatoly.burakov@intel.com>

This patchset replaces the fcntl()-based locking used in
the original DPDK memory hotplug patchset, to an flock()-
and lockfile-based implementation, due to numerous (well,
one, really) problems with how fcntl() locks work.

Long story short, fcntl() locks will be dropped if any
fd referring to locked file, is closed - even if it's not
the last fd, even if it wasn't even the fd that was used
to lock the file in the first place, even if it wasn't
you who closed that fd, but some other library.

This patchset corrects this saddening design defect in the
original implementation.

One of the ways to work around this was using OFD locks,
but they are only supported on kernels 3.15+, so we cannot
rely on them if we want to support old kernels. Hence, we
use per-segment lockfiles. The number of file descriptors
we open does not end up more than in non-single file
segments case - we still open the same amount of files (a
file per page), plus a file per memseg list.

Additionally, since flock() is not atomic, we also lock the
hugepage dir to prevent multiple processes from concurrently
performing operations on hugetlbfs mounts.

If you know of a more enlightened way of fixing this
limitation, you are certainly welcome to comment :)

v2:
- Fixes as per review comments
- Make lockfiles hidden by default

Anatoly Burakov (2):
  mem: add memalloc init stage
  mem: revert to using flock() and add per-segment lockfiles

 lib/librte_eal/bsdapp/eal/eal_memalloc.c        |   6 +
 lib/librte_eal/common/eal_common_memory.c       |   3 +
 lib/librte_eal/common/eal_filesystem.h          |  18 +
 lib/librte_eal/common/eal_memalloc.h            |   3 +
 lib/librte_eal/linuxapp/eal/eal_hugepage_info.c |  28 +-
 lib/librte_eal/linuxapp/eal/eal_memalloc.c      | 604 +++++++++++++++---------
 lib/librte_eal/linuxapp/eal/eal_memory.c        |  22 +-
 7 files changed, 420 insertions(+), 264 deletions(-)

-- 
2.7.4

  parent reply	other threads:[~2018-04-24 15:54 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-19 12:26 [dpdk-dev] [PATCH " Anatoly Burakov
2018-04-19 12:26 ` [dpdk-dev] [PATCH 1/2] mem: add memalloc init stage Anatoly Burakov
2018-04-24 14:06   ` Bruce Richardson
2018-04-19 12:26 ` [dpdk-dev] [PATCH 2/2] mem: revert to using flock() and add per-segment lockfiles Anatoly Burakov
2018-04-24 13:57   ` Bruce Richardson
2018-04-24 14:07   ` Bruce Richardson
2018-04-24 15:54 ` Anatoly Burakov [this message]
2018-04-24 16:32   ` [dpdk-dev] [PATCH v2 0/2] Fix file locking in EAL memory Stephen Hemminger
2018-04-24 17:25     ` Burakov, Anatoly
2018-04-24 20:05       ` Thomas Monjalon
2018-04-24 20:34         ` Stephen Hemminger
2018-04-25 10:36   ` [dpdk-dev] [PATCH v3 " Anatoly Burakov
2018-04-27 21:50     ` Thomas Monjalon
2018-04-25 10:36   ` [dpdk-dev] [PATCH v3 1/2] mem: add memalloc init stage Anatoly Burakov
2018-04-25 10:36   ` [dpdk-dev] [PATCH v3 2/2] mem: revert to using flock() and add per-segment lockfiles Anatoly Burakov
2018-04-28  9:38     ` Andrew Rybchenko
2018-04-30  8:29       ` Burakov, Anatoly
2018-04-30 11:31       ` Burakov, Anatoly
2018-04-30 11:51         ` Maxime Coquelin
2018-04-30 13:08         ` Andrew Rybchenko
2018-04-24 15:54 ` [dpdk-dev] [PATCH v2 1/2] mem: add memalloc init stage Anatoly Burakov
2018-04-24 15:54 ` [dpdk-dev] [PATCH v2 2/2] mem: revert to using flock() and add per-segment lockfiles Anatoly Burakov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1524585160.git.anatoly.burakov@intel.com \
    --to=anatoly.burakov@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).