Soft Patch Panel
 help / color / mirror / Atom feed
From: Yasufumi Ogawa <yasufum.o@gmail.com>
To: spp@dpdk.org, ferruh.yigit@intel.com, yasufum.o@gmail.com
Subject: [spp] [PATCH 01/29] tools/sppc: update options for assigning devices
Date: Tue, 25 Feb 2020 19:34:18 +0900
Message-ID: <20200225103446.8243-2-yasufum.o@gmail.com> (raw)
In-Reply-To: <20200225103446.8243-1-yasufum.o@gmail.com>

For non SPP container apps, such as l2fwd or testpmd, it is connected
with SPP via vhost. This update is to add other PMDs to the interface.
There are two types of PMDs, vhost and memif, supported SPP container.

This patch includes two updates, one is for revising `-d` option, and
another is for adding `-v` of docker and `--vdev` of DPDK to make givin
devices directly.

Option `-d` for device ID is changed from getting a list of numbers to a
list of resouce UIDs because it should take not only vhost, but also
other types. Devices are described as a comma separated list of resource
UIDs. Full name of the option is also changed from `--dev-ids` to
`--dev-uids` because it is not just ID but res UIDs.

  $ python3 app/l2fwd.py -l 5-6 ... -d memif:0,memif:1 ...

If you use `-d`, app container launcher generates `-v` for docker and
`--vdev` for DPDK options with appropriate parameters for the container
app. However, you might want to customize the params sometimes. You can
assign devices directly with `-v` and `--vdev` in this case, but the
launcher does not validate the options.

Signed-off-by: Yasufumi Ogawa <yasufum.o@gmail.com>
---
 tools/sppc/lib/app_helper.py | 262 +++++++++++++++++++++--------------
 1 file changed, 159 insertions(+), 103 deletions(-)

diff --git a/tools/sppc/lib/app_helper.py b/tools/sppc/lib/app_helper.py
index 7ab1982..b5669da 100644
--- a/tools/sppc/lib/app_helper.py
+++ b/tools/sppc/lib/app_helper.py
@@ -3,10 +3,17 @@
 
 from . import common
 import os
+import secrets
 import sys
 
 
+# Supported vdev types of SPP Container.
+VDEV_TYPES = ['vhost', 'memif']
+
+
 def add_eal_args(parser, mem_size=1024, mem_channel=4):
+    """Add EAL args to app."""
+
     parser.add_argument(
         '-l', '--core-list',
         type=str,
@@ -20,6 +27,10 @@ def add_eal_args(parser, mem_size=1024, mem_channel=4):
         type=int,
         default=mem_size,
         help='Memory size (default is %s)' % mem_size)
+    parser.add_argument(
+        '--vdev',
+        nargs='*', type=str,
+        help='Virtual device in the format of DPDK')
     parser.add_argument(
         '--socket-mem',
         type=str,
@@ -44,6 +55,80 @@ def add_eal_args(parser, mem_size=1024, mem_channel=4):
     return parser
 
 
+def add_sppc_args(parser):
+    """Add args of SPP Container to app."""
+
+    parser.add_argument(
+        '--dist-name',
+        type=str,
+        default='ubuntu',
+        help="Name of Linux distribution")
+    parser.add_argument(
+        '--dist-ver',
+        type=str,
+        default='latest',
+        help="Version of Linux distribution")
+    parser.add_argument(
+        '--workdir',
+        type=str,
+        help="Path of directory in which the command is launched")
+    parser.add_argument(
+        '-ci', '--container-image',
+        type=str,
+        help="Name of container image")
+    parser.add_argument(
+        '-fg', '--foreground',
+        action='store_true',
+        help="Run container as foreground mode")
+    parser.add_argument(
+        '--dry-run',
+        action='store_true',
+        help="Only print matrix, do not run, and exit")
+    return parser
+
+
+def add_appc_args(parser):
+    """Add docker options and other common args."""
+
+    parser.add_argument(
+        '-d', '--dev-uids',
+        type=str,
+        help='Virtual devices of SPP in resource UID format')
+    parser.add_argument(
+        '-v', '--volume',
+        nargs='*', type=str,
+        help='Bind mount a volume (for docker)')
+    parser.add_argument(
+        '-nq', '--nof-queues',
+        type=int,
+        default=1,
+        help="Number of queues of virtio (default is 1)")
+    parser.add_argument(
+        '--no-privileged',
+        action='store_true',
+        help="Disable docker's privileged mode if it's needed")
+    return parser
+
+
+def is_valid_dev_uids(dev_uids):
+    """Return True if value of --dev-uids is valid.
+
+    dev_uids should be a list of resource UIDs separated with ',', for example
+    'vhost:0,vhost:1'.
+
+    If given port type is not supported in SPP Container, it returns False.
+    """
+
+    if dev_uids is None:
+        return False
+
+    for dev_uid in dev_uids.split(','):
+        if dev_uid.split(':')[0] not in VDEV_TYPES:
+            return False
+
+    return True
+
+
 def get_core_opt(args):
     # Check core_mask or core_list is defined.
     if args.core_mask is not None:
@@ -74,21 +159,22 @@ def setup_eal_opts(args, file_prefix, proc_type='auto', hugedir=None):
         mem_opt['attr'], mem_opt['val'], '\\',
         '--proc-type', proc_type, '\\']
 
-    if args.dev_ids is None:
-        common.error_exit('--dev-ids')
-    else:
-        dev_ids = dev_ids_to_list(args.dev_ids)
+    if args.dev_uids is not None:
+        dev_uids_list = args.dev_uids.split(',')
 
-    socks = []
-    for dev_id in dev_ids:
-        socks.append({
-            'host': '/tmp/sock%d' % dev_id,
-            'guest': '/var/run/usvhost%d' % dev_id})
+        socks = sock_files(dev_uids_list)
 
-    for i in range(len(dev_ids)):
-        eal_opts += [
-            '--vdev', 'virtio_user%d,queues=%d,path=%s' % (
-                dev_ids[i], args.nof_queues, socks[i]['guest']), '\\']
+        # Configure '--vdev' options
+        for i in range(len(dev_uids_list)):
+            dev_uid = dev_uids_list[i].split(':')
+            if dev_uid[0] == 'vhost':
+                eal_opts += [
+                    '--vdev', 'virtio_user{},queues={},path={}'.format(
+                        dev_uid[1], args.nof_queues, socks[i]['guest']), '\\']
+            elif dev_uid[0] == 'memif':  # Only 'slave' role is supported.
+                eal_opts += [
+                    '--vdev', 'net_memif{0},id={0},socket={1}'.format(
+                        dev_uid[1], socks[0]['guest']), '\\']
 
     if (args.pci_blacklist is not None) and (args.pci_whitelist is not None):
         common.error_exit("Cannot use both of '-b' and '-w' at once")
@@ -109,36 +195,6 @@ def setup_eal_opts(args, file_prefix, proc_type='auto', hugedir=None):
     return eal_opts
 
 
-def add_sppc_args(parser):
-    parser.add_argument(
-        '--dist-name',
-        type=str,
-        default='ubuntu',
-        help="Name of Linux distribution")
-    parser.add_argument(
-        '--dist-ver',
-        type=str,
-        default='latest',
-        help="Version of Linux distribution")
-    parser.add_argument(
-        '--workdir',
-        type=str,
-        help="Path of directory in which the command is launched")
-    parser.add_argument(
-        '-ci', '--container-image',
-        type=str,
-        help="Name of container image")
-    parser.add_argument(
-        '-fg', '--foreground',
-        action='store_true',
-        help="Run container as foreground mode")
-    parser.add_argument(
-        '--dry-run',
-        action='store_true',
-        help="Only print matrix, do not run, and exit")
-    return parser
-
-
 def setup_docker_opts(args, container_image, sock_files, workdir=None):
     docker_opts = []
 
@@ -153,9 +209,10 @@ def setup_docker_opts(args, container_image, sock_files, workdir=None):
     if args.no_privileged is not True:
         docker_opts += ['--privileged', '\\']
 
-    for sock in sock_files:
-        docker_opts += [
-            '-v', '%s:%s' % (sock['host'], sock['guest']), '\\']
+    if sock_files is not None:
+        for sock in sock_files:
+            docker_opts += [
+                '-v', '%s:%s' % (sock['host'], sock['guest']), '\\']
 
     docker_opts += [
         '-v', '/dev/hugepages:/dev/hugepages', '\\',
@@ -164,73 +221,57 @@ def setup_docker_opts(args, container_image, sock_files, workdir=None):
     return docker_opts
 
 
-def add_appc_args(parser):
-    parser.add_argument(
-        '-d', '--dev-ids',
-        type=str,
-        help='two or more even vhost device IDs')
-    parser.add_argument(
-        '-nq', '--nof-queues',
-        type=int,
-        default=1,
-        help="Number of queues of virtio (default is 1)")
-    parser.add_argument(
-        '--no-privileged',
-        action='store_true',
-        help="Disable docker's privileged mode if it's needed")
-    return parser
-
-
-def uniq(dup_list):
-    """Remove duplicated elements in a list and return a unique list
-
-    Example: [1,1,2,2,3,3] #=> [1,2,3]
-    """
-
-    return list(set(dup_list))
-
-
-def dev_ids_to_list(dev_ids):
-    """Parse vhost device IDs and return as a list.
-
-    Example:
-    '1,3-5' #=> [1,3,4,5]
-    """
-
-    res = []
-    for dev_id_part in dev_ids.split(','):
-        if '-' in dev_id_part:
-            cl = dev_id_part.split('-')
-            res = res + list(range(int(cl[0]), int(cl[1])+1))
-        else:
-            res.append(int(dev_id_part))
-    return res
-
-
-def is_sufficient_dev_ids(dev_ids, port_mask):
-    """Check if ports can be reserved for dev_ids
+def is_sufficient_ports(args):
+    """Check if ports can be reserved.
 
-    Return true if the number of dev IDs equals or more than given ports.
-    'dev_ids' is a value of '-d' or '--dev-ids' such as '1,2'.
+    Return True if the number of vdevs is enogh for given ports.
     """
 
-    dev_ids_list = dev_ids_to_list(dev_ids)
-    if not ('0x' in port_mask):  # invalid port mask
+    # TODO(yasufum): It doesn't check if no given portmask and dev_uids, so
+    # add this additional check.
+    if (args.port_mask is None) or (args.dev_uids is None):
+        return False
+    elif not ('0x' in args.port_mask):  # invalid port mask
         return False
 
-    ports_in_binary = format(int(port_mask, 16), 'b')
-    if len(dev_ids_list) >= len(ports_in_binary):
+    dev_uids_list = args.dev_uids.split(',')
+
+    ports_in_binary = format(int(args.port_mask, 16), 'b')
+    if len(dev_uids_list) >= len(ports_in_binary):
         return True
     else:
         return False
 
 
-def sock_files(dev_ids_list):
+def sock_files(dev_uids_list):
+    """Return list of socket files on host and containers.
+
+    The name of socket files is defined with a conventional ones described
+    in DPDK doc, though you can use any name actually.
+
+    Here is an example of two vhost devices.
+        [vhost:0, vhost:1]
+        => [
+              {'host': '/tmp/sock0, 'guest': '/var/run/usvhost0'},
+              {'host': '/tmp/sock1, 'guest': '/var/run/usvhost1'}
+            ]
+    """
     socks = []
-    for dev_id in dev_ids_list:
-        socks.append({
-            'host': '/tmp/sock%d' % dev_id,
-            'guest': '/var/run/usvhost%d' % dev_id})
+    for dev_uid in dev_uids_list:
+        dev_uid = dev_uid.split(':')
+        if dev_uid[0] == 'vhost':
+            socks.append({
+                'host': '/tmp/sock{}'.format(dev_uid[1]),
+                'guest': '/var/run/usvhost{}'.format(dev_uid[1])})
+        elif dev_uid[0] == 'memif':
+            # Single sock file is enough for memif because it is just used for
+            # negotiation between master and slaves processes.
+            socks.append({
+                'host': '/tmp/spp-memif.sock',
+                'guest': '/var/run/spp-memif.sock'})
+            break
+        else:
+            break
     return socks
 
 
@@ -278,6 +319,21 @@ def cores_to_list(core_opt):
                 res.append(int(core_part))
     else:
         pass
-    res = uniq(res)
+    res = _uniq(res)
     res.sort()
     return res
+
+
+def gen_sppc_file_prefix(app_name):
+    """Generate a unique file prefix of DPDK for SPP Container app."""
+
+    return 'sppc-{}-{}'.format(app_name, secrets.token_hex(8))
+
+
+def _uniq(dup_list):
+    """Remove duplicated elements in a list and return a unique list.
+
+    Example: [1,1,2,2,3,3] #=> [1,2,3]
+    """
+
+    return list(set(dup_list))
-- 
2.17.1


  reply	other threads:[~2020-02-25 10:34 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-25 10:34 [spp] [PATCH 00/29] Update SPP Container tools Yasufumi Ogawa
2020-02-25 10:34 ` Yasufumi Ogawa [this message]
2020-02-25 10:34 ` [spp] [PATCH 02/29] tools/sppc: update dev options of l2fwd Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 03/29] tools/sppc: add container name option Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 04/29] tools/sppc: update l2fwd app for " Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 05/29] tools/sppc: update dev options of l3fwd Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 06/29] tools/sppc: update dev options of l3fwd-acl Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 07/29] tools/sppc: update dev options of testpmd Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 08/29] tools/sppc: update dev options of pktgen Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 09/29] tools/sppc: update dev options of load-balancer Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 10/29] tools/sppc: version checker for container DPDK ver Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 11/29] tools/sppc: check DPDK ver in load-balancer Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 12/29] tools/sppc: setup spp_pri opts in app_helper Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 13/29] tools/sppc: define file prefix for SPP Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 14/29] tools/sppc: update dev options of spp_primary Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 15/29] tools/sppc: setup with docker opts in SPP pri Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 16/29] tools/sppc: update calling setup_docker_opts() Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 17/29] tools/sppc: update dev options of helloworld Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 18/29] tools/sppc: update dev options of suricata Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 19/29] tools/sppc: update dev options of spp_nfv Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 20/29] tools/sppc: change to gen EAL opts with app name Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 21/29] tools/sppc: remove nouse variable Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 22/29] bin: remove sock files created by docker Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 23/29] tools/sppc: skip checking rule file if dry run Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 24/29] docs: revise examples in sppc Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 25/29] docs: update versions in " Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 26/29] docs: update old example in spp_primary container Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 27/29] tools/sppc: python3 support for sppc build tool Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 28/29] docs: update app container help msg Yasufumi Ogawa
2020-02-25 10:34 ` [spp] [PATCH 29/29] docs: update howto define app container guide Yasufumi Ogawa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200225103446.8243-2-yasufum.o@gmail.com \
    --to=yasufum.o@gmail.com \
    --cc=ferruh.yigit@intel.com \
    --cc=spp@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Soft Patch Panel

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/spp/0 spp/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 spp spp/ https://inbox.dpdk.org/spp \
		spp@dpdk.org
	public-inbox-index spp

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.spp


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git