[dpdk-dev] [RFC 0/3] Make device mapping more reliable

[Anatoly Burakov <anatoly.burakov@intel.com>]

Currently, memory for device maps is allocated ad-hoc, by calculating
end of VA space allocated for hugepages and crossing fingers in hopes that
those addresses will be free in primary and secondary processes. This leads
to situations such as this:

EAL: Detected 88 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket_178323_8af2229603de4
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: PCI device 0000:81:00.0 on NUMA socket 1
EAL:   probe driver: 8086:1563 net_ixgbe
EAL: Cannot mmap device resource file /sys/bus/pci/devices/0000:81:00.0/resource0 to address: 0x7ff7f5800000
EAL: Requested device 0000:81:00.0 cannot be used
EAL: Error - exiting with code: 1
  Cause: No Ethernet ports - bye

As can be seen from the above log, secondary process has initialized
successfully, but device BAR mapping has failed, which resulted in missing ports
in the secondary process.

This patchset is an attempt to fix this problem once and for all, by using
the same method we use for memory to do device mappings as well. That is,
by preallocating all of the device memory in advance, so that initialization
either succeeds and allows for device mappings, or it fails outright (whereas
currently we may be in an in-between kind of situation, where init has
succeeded but device mappings have failed).

This change breaks the ABI, so it is not for this release. However, i'd like
to hear feedback on the approach and whether there are potential problems with
other buses/use cases that i didn't think of.

Anatoly Burakov (3):
  fbarray: allow zero-sized elements
  mem: add device memory reserve/free API
  bus/pci: use the new device memory API for BAR mapping

 drivers/bus/pci/linux/pci_init.h              |   1 -
 drivers/bus/pci/linux/pci_uio.c               |  11 +-
 drivers/bus/pci/linux/pci_vfio.c              |  27 +-
 lib/librte_eal/common/eal_common_fbarray.c    |  10 +-
 lib/librte_eal/common/eal_common_memory.c     | 270 ++++++++++++++++--
 .../common/include/rte_eal_memconfig.h        |  18 ++
 lib/librte_eal/common/include/rte_memory.h    |  40 +++
 lib/librte_pci/Makefile                       |   1 +
 lib/librte_pci/rte_pci.c                      |  20 +-
 9 files changed, 350 insertions(+), 48 deletions(-)

-- 
2.17.0