From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by dpdk.org (Postfix) with ESMTP id 9E42823C for ; Fri, 20 Jul 2018 12:57:50 +0200 (CEST) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 30B3521B6A; Fri, 20 Jul 2018 06:57:50 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 20 Jul 2018 06:57:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:date:from:in-reply-to:message-id:references:subject:to :x-me-sender:x-me-sender:x-sasl-enc; s=mesmtp; bh=AAtUI9VDRKNzHH 0tZvG8LXxDAA+9unu0SYpxgdptftU=; b=rMMvgzNp4buhBqVHADjffEA9artY8t HNT0938+4Xspmp/Jm97T+kUZG4mSNzJ0OvztOFWaCTkU7MtBxMIxQW+Io6qzvW2m 2Mqr5kr8/bTwLAGSb0qwF0mNLM30errSwxFi6a807f8nnf9ZlU7KS4D1OwW6zxtZ f1MLt41nUaWpc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:date:from:in-reply-to:message-id :references:subject:to:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; bh=AAtUI9VDRKNzHH0tZvG8LXxDAA+9unu0SYpxgdptftU=; b=b71i2EVD DwtHEXrrJvofcWbpXdWEZ0n1aWYRCJT0hYdfeCT9ZU/KWayK3E5M7bLqH8OBDVJU G3x1T/qvXyFcyMskG0Zl2uWV53fzS2f0dsTe50qxay6aD+SrA/2uOnBMZTPzYFvx SJdwNvlVgJgP6CFIJMqf23exQ08q5Mh2N94mghRzCTRPENA8OaDsJ4JUq4pseE3V XZya5qgaXNfM1nlvhPUtJfpprjx+JCtCgndt1EC3awwBtlWOU3rSjxML4BlQdTmd zG0bLt/KIM5v1bDj1EyR1KbAIEv1eoA37Mk92+kra9Vo9KYjJlJVSb4GH6CVbXK3 Ts/zvTenYZEAMw== X-ME-Proxy: X-ME-Sender: Received: from xps.monjalon.net (unknown [207.232.18.42]) by mail.messagingengine.com (Postfix) with ESMTPA id 21223E4040; Fri, 20 Jul 2018 06:57:47 -0400 (EDT) From: Thomas Monjalon To: keith.wiles@intel.com Cc: rasland@mellanox.com, ophirmu@mellanox.com, dev@dpdk.org Date: Fri, 20 Jul 2018 12:57:42 +0200 Message-Id: <20180720105742.12669-1-thomas@monjalon.net> X-Mailer: git-send-email 2.17.1 In-Reply-To: <1528374591-26126-1-git-send-email-rasland@mellanox.com> References: <1528374591-26126-1-git-send-email-rasland@mellanox.com> Subject: [dpdk-dev] [PATCH v2] net/tap: add queues when attaching from secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Jul 2018 10:57:50 -0000 From: Raslan Darawsheh In the case the device is created by the primary process, the secondary must request some file descriptors to attach the queues. The file descriptors are shared via IPC Unix socket. Thanks to the IPC synchronization, the secondary process is now able to do Rx/Tx on a TAP created by the primary process. Signed-off-by: Raslan Darawsheh Signed-off-by: Thomas Monjalon --- Note: there is a bug in EAL IPC regarding fd translation. A fix will be sent later for EAL. v2: - translate file descriptors via IPC API - add documentation --- doc/guides/nics/tap.rst | 16 ++++ doc/guides/rel_notes/release_18_08.rst | 5 + drivers/net/tap/Makefile | 1 + drivers/net/tap/rte_eth_tap.c | 127 ++++++++++++++++++++++++- 4 files changed, 148 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/tap.rst b/doc/guides/nics/tap.rst index 27148681c..d1f3e1c24 100644 --- a/doc/guides/nics/tap.rst +++ b/doc/guides/nics/tap.rst @@ -152,6 +152,22 @@ Distribute IPv4 TCP packets using RSS to a given MAC address over queues 0-3:: testpmd> flow create 0 priority 4 ingress pattern eth dst is 0a:0b:0c:0d:0e:0f \ / ipv4 / tcp / end actions rss queues 0 1 2 3 end / end +Multi-process sharing +--------------------- + +It is possible to attach an existing TAP device in a secondary process, +by declaring it as a vdev with the same name as in the primary process, +and without any parameter. + +The port attached in a secondary process will give access to the +statistics and the queues. +Therefore it can be used for monitoring or Rx/Tx processing. + +The IPC synchronization of Rx/Tx queues is currently limited: + + - Only 8 queues + - Synchronized on probing, but not on later port update + Example ------- diff --git a/doc/guides/rel_notes/release_18_08.rst b/doc/guides/rel_notes/release_18_08.rst index df0ad61f7..771befecb 100644 --- a/doc/guides/rel_notes/release_18_08.rst +++ b/doc/guides/rel_notes/release_18_08.rst @@ -77,6 +77,11 @@ New Features * Add handlers to add/delete VxLAN port number. * Add devarg to specify ingress VLAN rewrite mode. +* **Added TAP Rx/Tx queues sharing with a secondary process.** + + A secondary process can attach a TAP device created in the primary process, + probe the queues, and process Rx/Tx in a secondary process. + API Changes ----------- diff --git a/drivers/net/tap/Makefile b/drivers/net/tap/Makefile index 324336535..3dcf05a72 100644 --- a/drivers/net/tap/Makefile +++ b/drivers/net/tap/Makefile @@ -27,6 +27,7 @@ LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs -lrte_hash LDLIBS += -lrte_bus_vdev -lrte_gso CFLAGS += -DTAP_MAX_QUEUES=$(TAP_MAX_QUEUES) +CFLAGS += -DALLOW_EXPERIMENTAL_API # # all source are stored in SRCS-y diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.c index 22ba872ed..8d70126c0 100644 --- a/drivers/net/tap/rte_eth_tap.c +++ b/drivers/net/tap/rte_eth_tap.c @@ -16,6 +16,8 @@ #include #include #include +#include +#include #include #include @@ -62,6 +64,9 @@ #define TAP_GSO_MBUFS_NUM \ (TAP_GSO_MBUFS_PER_CORE * TAP_GSO_MBUF_CACHE_SIZE) +/* IPC key for queue fds sync */ +#define TAP_MP_KEY "tap_mp_sync_queues" + static struct rte_vdev_driver pmd_tap_drv; static struct rte_vdev_driver pmd_tun_drv; @@ -100,6 +105,17 @@ enum ioctl_mode { REMOTE_ONLY, }; +/* Message header to synchronize queues via IPC */ +struct ipc_queues { + char port_name[RTE_DEV_NAME_MAX_LEN]; + int rxq_count; + int txq_count; + /* + * The file descriptors are in the dedicated part + * of the Unix message to be translated by the kernel. + */ +}; + static int tap_intr_handle_set(struct rte_eth_dev *dev, int set); /** @@ -1972,6 +1988,96 @@ rte_pmd_tun_probe(struct rte_vdev_device *dev) return ret; } +/* Request queue file descriptors from secondary to primary. */ +static int +tap_mp_attach_queues(const char *port_name, struct rte_eth_dev *dev) +{ + int ret; + struct pmd_internals *devpriv; + struct timespec timeout = {.tv_sec = 1, .tv_nsec = 0}; + struct rte_mp_msg request, *reply; + struct rte_mp_reply replies; + struct ipc_queues *request_param = (struct ipc_queues *)request.param; + struct ipc_queues *reply_param; + int queue, fd_iterator; + + /* Prepare the request */ + strcpy(request.name, TAP_MP_KEY); + strcpy(request_param->port_name, port_name); + request.len_param = sizeof(*request_param); + + /* Send request and receive reply */ + ret = rte_mp_request_sync(&request, &replies, &timeout); + if (ret < 0) { + TAP_LOG(ERR, "Failed to request queues from primary: %d", rte_errno); + return -1; + } + /* FIXME: handle replies.nb_received > 1 */ + reply = &replies.msgs[0]; + reply_param = (struct ipc_queues *)reply->param; + TAP_LOG(DEBUG, "Received IPC reply for %s", reply_param->port_name); + + /* Attach the queues from received file descriptors */ + devpriv = dev->data->dev_private; + fd_iterator = 0; + for (queue = 0; queue < reply_param->rxq_count; queue++) + devpriv->rxq[queue].fd = reply->fds[fd_iterator++]; + for (queue = 0; queue < reply_param->txq_count; queue++) + devpriv->txq[queue].fd = reply->fds[fd_iterator++]; + + return 0; +} + +/* Send the queue file descriptors from the primary process to secondary. */ +static int +tap_mp_sync_queues(const struct rte_mp_msg *request, const void *peer) +{ + struct rte_eth_dev *dev; + struct pmd_internals *devpriv; + struct rte_mp_msg reply; + const struct ipc_queues *request_param = (const struct ipc_queues *)request->param; + struct ipc_queues *reply_param = (struct ipc_queues *)reply.param; + uint16_t port_id; + int queue; + int ret; + + /* Get requested port */ + TAP_LOG(DEBUG, "Received IPC request for %s", request_param->port_name); + ret = rte_eth_dev_get_port_by_name(request_param->port_name, &port_id); + if (ret) { + TAP_LOG(ERR, "Failed to get port id for %s", request_param->port_name); + return -1; + } + dev = &rte_eth_devices[port_id]; + devpriv = dev->data->dev_private; + + /* Fill file descriptors for all queues */ + reply.num_fds = 0; + reply_param->rxq_count = 0; + for (queue = 0; queue < dev->data->nb_rx_queues; queue++) { + reply.fds[reply.num_fds++] = devpriv->rxq[queue].fd; + reply_param->rxq_count++; + } + reply_param->txq_count = 0; + for (queue = 0; queue < dev->data->nb_tx_queues; queue++) { + reply.fds[reply.num_fds++] = devpriv->txq[queue].fd; + reply_param->txq_count++; + } + /* FIXME: split message if more queues than RTE_MP_MAX_FD_NUM */ + RTE_ASSERT(reply.num_fds <= RTE_MP_MAX_FD_NUM); + + /* Send reply */ + strcpy(reply.name, request->name); + strcpy(reply_param->port_name, request_param->port_name); + reply.len_param = sizeof(*reply_param); + if (rte_mp_reply(&reply, peer) < 0) { + TAP_LOG(ERR, "Failed to reply an IPC request to sync queues"); + return -1; + } + + return 0; +} + /* Open a TAP interface device. */ static int @@ -1998,8 +2104,18 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) TAP_LOG(ERR, "Failed to probe %s", name); return -1; } - /* TODO: request info from primary to set up Rx and Tx */ eth_dev->dev_ops = &ops; + eth_dev->rx_pkt_burst = pmd_rx_burst; + eth_dev->tx_pkt_burst = pmd_tx_burst; + + if (!rte_eal_primary_proc_alive(NULL)) { + TAP_LOG(ERR, "Primary process is missing"); + return -1; + } + ret = tap_mp_attach_queues(name, eth_dev); + if (ret != 0) + return -1; + rte_eth_dev_probing_finish(eth_dev); return 0; } @@ -2050,10 +2166,19 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) ret = eth_dev_tap_create(dev, tap_name, remote_iface, &user_mac, ETH_TUNTAP_TYPE_TAP); + /* Register IPC feed callback */ + ret = rte_mp_action_register(TAP_MP_KEY, tap_mp_sync_queues); + if (ret < 0 && rte_errno != EEXIST) { + TAP_LOG(ERR, "%s: Failed to register IPC callback: %s", + tuntap_name, strerror(rte_errno)); + goto leave; + } + leave: if (ret == -1) { TAP_LOG(ERR, "Failed to create pmd for %s as %s", name, tap_name); + rte_mp_action_unregister(TAP_MP_KEY); tap_unit--; /* Restore the unit number */ } rte_kvargs_free(kvlist); -- 2.17.1