DPDK patches and discussions
 help / color / mirror / Atom feed
From: Olivier Matz <olivier.matz@6wind.com>
To: dev@dpdk.org
Subject: [dpdk-dev] [PATCH 00/11] eal: allow virtual pmd drivers as shared lib
Date: Fri, 28 Feb 2014 18:25:39 +0100	[thread overview]
Message-ID: <1393608350-4431-1-git-send-email-olivier.matz@6wind.com> (raw)

The main goal of the following commits is to be able to load a PMD
virtual device driver as a shared library (DPDK extension). Today it is
already possible to load an external PMD PCI driver (ex: memnic,
virtio-net-pmd, ...), but the DPDK framework does not allow to load a
virtual device driver in this way. For instance, adding the support of
host-side in memnic would require these patches.

How it worked before the patch
------------------------------

Example of what occurs when we started testpmd:

  testpmd -c 0x15 -n 3 \
    --proc-type=primary --huge-dir=/mnt/huge --use-device=eth_ring0 \
    --use-device=eth_ring1 \
    --use-device=02:00.0 \
    -- -i --port-topology=chained

For each "--use-device" option, we call eal_dev_whitelist_add_entry():

  main()
    rte_eal_init()
      eal_parse_args()
        eal_dev_whitelist_add_entry()

Each device is appended in a global string. After parsing all the
"--use-device" argumentss, we have dev_list_str="eth_ring0,eth_ring1,".

Then, after all eal arguments are parsed, eal_dev_whitelist_parse() is
called:

  main()
    rte_eal_init()
      eal_parse_args()
        eal_dev_whitelist_parse()
          is_valid_wl_entry()

It removes the extra ',' at the end of dev_list_str. Then, it splits
each device name and its argument (separated by ';'). Each device is
checked by is_valid_wl_entry(). It checks that the PCI identifier is
correct or that the name of the device starts with a known prefix
("eth_ring", "eth_pcap" or "eth_xenvirt", defined statically in eal code).

Then, rte_eal_pci_init() is called, it scans the PCI bus:

  main()
    rte_eal_init()
      rte_eal_pci_init()

After that, rte_eal_non_pci_ethdev_init() tries all combination:
"eth_ring0", "eth_ring1", ..., "eth_ring31", "eth_pcap0", ...,
"eth_pcap31", ..., "-nodev-0", ..., "-nodev-31". For each, test if it
is in whitelist or not:

  main()
    rte_eal_init()
      rte_eal_non_pci_ethdev_init()
        eal_dev_is_whitelisted(name, &params) # fills params
        dev_types[i].init_fn(name, params) [ex: rte_pmd_ring_init()]
          ...
            rte_eth_dev_allocate()

To check that, eal_dev_is_whitelisted() browse the list of devices in
the whitelist. Then, the init function of the device allocates the
ethernet device structure (attributing a port_id).

The main application then calls rte_pmd_init_all(). It initializes all
poll-mode drivers. Each driver calls rte_eth_driver_register(), like in
this example:

  main()
    rte_pmd_init_all()
      rte_igb_pmd_init()
        rte_eth_driver_register()

The probing of the PCI bus is done using rte_eal_pci_probe(), which
calls pci_probe_all_drivers(dev) for each PCI device:

  main()
    rte_eal_pci_probe()
      pcidev_is_whitelisted(dev)
      pci_probe_all_drivers(dev)
        rte_eal_pci_probe_one_driver(dev, driver)
          driver->devinit(driver, dev) [rte_eth_dev_init()]
            rte_eth_dev_allocate()
            eth_drv->eth_dev_init(eth_drv, eth_dev) [ex: eth_em_dev_init()]

List of problems
----------------

- pmd_ring, pmd_pcap and pmd_xenvirt are referenced in eal code:
  - it's not possible to add a new virtual pmd dynamically
  - eal compilation depends on these pmd... but the pmd depends on eal.
  - it's referenced twice with duplicated values: eal_common_nonpci_devs.c
    and eal_common_whitelist.c

- the parsing of virtual devices arguments is complex:
  - they are all appended in a global string then split again

- there are 2 ways to do the same thing, like in the following example:
  - --use-device="eth_ring0,eth_pcap0;iface=ixgbe0"
  - --use-device="eth_ring0" --use-device="eth_pcap0;iface=ixgbe0"

- the same --use-device option is used for both pci whitelist and
  virtual devices

- files and functions related to virtual devices are called 'non_pci',
  'vdev' would be clearer (all non_pci devices aren't virtual devices).

- using ";" to separate a device and its argument is not a good idea in
  a command line argument as it can be used in shell to separate
  commands.

- it is not possible to use the blacklist mode (bind all devices) while
  we add virtual devices.

Summary of the changes introduced by the patchset
-------------------------------------------------

- allow to register a virtual device driver from a dpdk extension
  provided as a shared library

- embed all library symbols in dpdk library.

- remove references to rte_pmd_ring, rte_pmd_pcap and rte_pmd_xenvirt in
  eal code

- add a new rte_devargs file in eal that unifies the code storing the
  user arguments pci-blacklist, pci-whitelist, and virtual devices
  arguments in one file

- rework eal user arguments
  - "--use-device" becomes "--pci-whitelist" and "--vdev"
  - replace ";" by "," when parsing device args

- support start-up arguments for PCI devices


Olivier Matz (11):
  mk: use whole-archive option when creating dpdk binaries
  devices-args: introduce rte_devargs in eal
  devices-args: use rte_devargs and remove old whitelist code
  devices-args: add a dump_devargs command in basic test application
  pci: rename device_list as pci_device_list
  vdev: rename eal_common_nonpci_devs.c as eal_common_vdev.c
  vdev: allow external registration of virtual device drivers
  device-args: use a comma instead of semicolon to separate key/values
  device-args: replace use-device eal option by pci-whitelist and vdev
  device-args: allow to provide per pci device command line arguments
  testpmd: add several dump commands, useful for debug

 app/test-pmd/cmdline.c                         | 114 +++++++++++++
 app/test/Makefile                              |   1 +
 app/test/commands.c                            |   9 +-
 app/test/test.h                                |   1 +
 app/test/test_devargs.c                        | 132 ++++++++++++++
 app/test/test_eal_flags.c                      |  70 ++------
 app/test/test_kvargs.c                         |  14 +-
 app/test/test_pci.c                            |  55 +++---
 app/test/test_pmd_ring.c                       |   6 +-
 lib/librte_eal/common/Makefile                 |   2 +-
 lib/librte_eal/common/eal_common_devargs.c     | 153 +++++++++++++++++
 lib/librte_eal/common/eal_common_nonpci_devs.c |  93 ----------
 lib/librte_eal/common/eal_common_pci.c         |  98 +++++------
 lib/librte_eal/common/eal_common_vdev.c        | 107 ++++++++++++
 lib/librte_eal/common/eal_common_whitelist.c   | 227 -------------------------
 lib/librte_eal/common/include/eal_private.h    |  42 +----
 lib/librte_eal/common/include/rte_devargs.h    | 140 +++++++++++++++
 lib/librte_eal/common/include/rte_pci.h        |  20 +--
 lib/librte_eal/common/include/rte_vdev.h       |  90 ++++++++++
 lib/librte_eal/linuxapp/eal/Makefile           |   4 +-
 lib/librte_eal/linuxapp/eal/eal.c              | 114 +++++++------
 lib/librte_eal/linuxapp/eal/eal_ivshmem.c      |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci.c          |  16 +-
 lib/librte_kvargs/rte_kvargs.h                 |   6 +-
 lib/librte_pmd_pcap/rte_eth_pcap.c             |  16 +-
 lib/librte_pmd_pcap/rte_eth_pcap.h             |   8 -
 lib/librte_pmd_ring/rte_eth_ring.c             |  17 +-
 lib/librte_pmd_ring/rte_eth_ring.h             |   6 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.c       |  14 +-
 lib/librte_pmd_xenvirt/rte_eth_xenvirt.h       |   4 +-
 mk/rte.app.mk                                  |   5 +
 31 files changed, 987 insertions(+), 599 deletions(-)
 create mode 100644 app/test/test_devargs.c
 create mode 100644 lib/librte_eal/common/eal_common_devargs.c
 delete mode 100644 lib/librte_eal/common/eal_common_nonpci_devs.c
 create mode 100644 lib/librte_eal/common/eal_common_vdev.c
 delete mode 100644 lib/librte_eal/common/eal_common_whitelist.c
 create mode 100644 lib/librte_eal/common/include/rte_devargs.h
 create mode 100644 lib/librte_eal/common/include/rte_vdev.h

-- 
1.8.5.3

             reply	other threads:[~2014-02-28 17:24 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-02-28 17:25 Olivier Matz [this message]
2014-02-28 17:25 ` [dpdk-dev] [PATCH 01/11] mk: use whole-archive option when creating dpdk binaries Olivier Matz
2014-04-10 13:58   ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 02/11] devices-args: introduce rte_devargs in eal Olivier Matz
2014-02-28 21:39   ` Stephen Hemminger
2014-03-01 12:02     ` Olivier MATZ
2014-03-01 12:14       ` [dpdk-dev] [PATCH v2 " Olivier Matz
2014-04-10 13:59         ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 03/11] devices-args: use rte_devargs and remove old whitelist code Olivier Matz
2014-03-01 12:14   ` [dpdk-dev] [PATCH v2 " Olivier Matz
2014-04-10 14:01     ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 04/11] devices-args: add a dump_devargs command in basic test application Olivier Matz
2014-04-10 14:02   ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 05/11] pci: rename device_list as pci_device_list Olivier Matz
2014-04-10 14:03   ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 06/11] vdev: rename eal_common_nonpci_devs.c as eal_common_vdev.c Olivier Matz
2014-04-10 14:39   ` Thomas Monjalon
2014-04-11  7:36     ` [dpdk-dev] [PATCH v2 06/11] vdev: rename nonpci_devs as vdev Olivier Matz
2014-04-11 11:25       ` Thomas Monjalon
2014-04-11 11:45         ` [dpdk-dev] [PATCH v3 " Olivier Matz
2014-04-11 12:37           ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 07/11] vdev: allow external registration of virtual device drivers Olivier Matz
2014-04-10 14:55   ` Thomas Monjalon
2014-04-11  7:36     ` [dpdk-dev] [PATCH v2 07/11 1/2] vdev: new registration API Olivier Matz
2014-04-11  7:36       ` [dpdk-dev] [PATCH v2 07/11 2/2] vdev: allow external registration of virtual device drivers Olivier Matz
2014-04-11 14:31         ` Thomas Monjalon
2014-04-11 10:49       ` [dpdk-dev] [PATCH v2 07/11 1/2] vdev: new registration API Neil Horman
2014-04-11 13:11         ` Thomas Monjalon
2014-04-11 15:50           ` Neil Horman
2014-04-11 16:18             ` Thomas Monjalon
2014-04-11 17:44               ` Neil Horman
2014-04-11 20:08                 ` Richardson, Bruce
2014-04-12  6:05                   ` Thomas Monjalon
2014-04-12 11:03                     ` Neil Horman
2014-04-12 11:23                       ` Richardson, Bruce
2014-04-12 14:06                         ` Neil Horman
2014-04-14 13:20                     ` John W. Linville
2014-04-14 13:45                       ` Thomas Monjalon
2014-04-14 13:54                         ` Neil Horman
2014-04-14 14:10                         ` John W. Linville
2014-04-14 14:39                           ` Thomas Monjalon
2014-04-11 14:31       ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 08/11] device-args: use a comma instead of semicolon to separate key/values Olivier Matz
2014-04-10 14:05   ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 09/11] device-args: replace use-device eal option by pci-whitelist and vdev Olivier Matz
2014-03-01 12:14   ` [dpdk-dev] [PATCH v2 " Olivier Matz
2014-04-10 14:06     ` Thomas Monjalon
2014-03-03 17:14   ` [dpdk-dev] [PATCH " Richardson, Bruce
2014-03-04 13:09     ` Olivier MATZ
2014-03-04 13:14       ` Richardson, Bruce
2014-03-24 22:39         ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 10/11] device-args: allow to provide per pci device command line arguments Olivier Matz
2014-03-01 12:14   ` [dpdk-dev] [PATCH v2 " Olivier Matz
2014-04-10 14:06     ` Thomas Monjalon
2014-02-28 17:25 ` [dpdk-dev] [PATCH 11/11] testpmd: add several dump commands, useful for debug Olivier Matz
2014-03-01 12:15   ` [dpdk-dev] [PATCH v2 " Olivier Matz
2014-04-10 14:08     ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1393608350-4431-1-git-send-email-olivier.matz@6wind.com \
    --to=olivier.matz@6wind.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).