From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60080.outbound.protection.outlook.com [40.107.6.80]) by dpdk.org (Postfix) with ESMTP id D869972FA for ; Fri, 8 Jun 2018 01:24:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3PTuVBk/3qWBUQKEqHQxmdiFNAaT8yXcn+hHmlAUWnU=; b=UTcff5JFGcuyo+15LNGAHlS4IllKj27v5aY7QpYf0oprAaLc9kE0T8AZ/SKkUf1G66SFjW6hQIUU+W2M5w6eLPn8hP6Www1fIhgFIaEbVAEEtnrKgJY2gYGujIJ1rIbap/ATelIkkkXBnvjBrL0QONl9yYLyLrLuOi8qrjDIQLg= Received: from DB5PR05MB1254.eurprd05.prod.outlook.com (10.162.157.140) by DB5PR05MB1877.eurprd05.prod.outlook.com (10.166.173.28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.841.14; Thu, 7 Jun 2018 23:24:01 +0000 Received: from DB5PR05MB1254.eurprd05.prod.outlook.com ([fe80::40c5:ca67:87b6:fc5e]) by DB5PR05MB1254.eurprd05.prod.outlook.com ([fe80::40c5:ca67:87b6:fc5e%4]) with mapi id 15.20.0841.011; Thu, 7 Jun 2018 23:24:01 +0000 From: Raslan Darawsheh To: "Wiles, Keith" CC: Thomas Monjalon , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [RFC] net/tap: add queues when attaching from secondary process Thread-Index: AQHT/pMfsazoxn4/UUWcNwN629lPw6RVbtsw Date: Thu, 7 Jun 2018 23:24:01 +0000 Message-ID: References: <1528374591-26126-1-git-send-email-rasland@mellanox.com> <077BA557-4ED0-4BAF-AFD7-8F54116229DB@intel.com> In-Reply-To: <077BA557-4ED0-4BAF-AFD7-8F54116229DB@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rasland@mellanox.com; x-originating-ip: [188.161.229.198] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB5PR05MB1877; 7:oNz/j52E/pIW8+TOQl/WIRgZzRXIkdDIHeoBBCAsKGfU2fG4sReONFTRkCeG0327YfyOjbrz7oRNE2r8YMxcqF4fi6ko5KADR5OvwUHCRc/IRfiefCQJ2Yg1N8IvarGVmI8okgR9sXpHd6rSRGPH5KAiUlUidYSysp0CA5ZPS3kiPOA4RtN/Dm4RfnRJeKZuxUIO0RYuqnyyxbrKE0ZPl5iScsXpstB06CMTmWsaHyi73fHOEv/1HSFhWYwx6ql5 x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(48565401081)(2017052603328)(7153060)(7193020); SRVR:DB5PR05MB1877; x-ms-traffictypediagnostic: DB5PR05MB1877: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:DB5PR05MB1877; BCL:0; PCL:0; RULEID:; SRVR:DB5PR05MB1877; x-forefront-prvs: 06968FD8C4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(979002)(366004)(346002)(39860400002)(396003)(39380400002)(376002)(13464003)(199004)(189003)(74316002)(2900100001)(305945005)(5250100002)(14454004)(105586002)(26005)(186003)(106356001)(5890100001)(7736002)(86362001)(476003)(486006)(229853002)(6436002)(3280700002)(102836004)(446003)(66066001)(6916009)(81166006)(81156014)(97736004)(76176011)(316002)(55016002)(99286004)(478600001)(54906003)(25786009)(11346002)(8936002)(3660700001)(7696005)(33656002)(68736007)(6116002)(53936002)(2906002)(4326008)(6246003)(9686003)(5660300001)(3846002)(53546011)(6506007)(59450400001)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR05MB1877; H:DB5PR05MB1254.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: M/M2qlSfPUPFfZL5YSRiNCYT61atQOh6U2HD7TF6nCMs9azi9xiHL5LvhbMQpzexUM64Bbzi6WiZsyf4KhmFBhDovLMFBOKxhaI05Lv2CqFbnmfJfzlQxm7N9GifWT5w6DxaDV+X5YiU9Pd+0ye9/hUtRVzG8YNor12/QPpmMUNtYFQYs47HE/Dlya683oxb spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 118c9f88-d575-493d-62f9-08d5cccdc0b1 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 118c9f88-d575-493d-62f9-08d5cccdc0b1 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jun 2018 23:24:01.4995 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1877 Subject: Re: [dpdk-dev] [RFC] net/tap: add queues when attaching from secondary process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2018 23:24:05 -0000 Hi, As you know that currently TAP pmd support attaching a secondary process to= a primary process.=20 But, it's still lacking the ability to do Rx/Tx burst since it's lacking th= e necessary fds for RX/TX queues, And the setting of Rx/Tx burst function. This patch the main purpose is to exchange the fds between the processes th= row the IPC massages And to set the Rx/Tx functions for the secondary. I hope I explained it properly, please let me know if you still didn't get = it.=20 Kindest regards, Raslan Darawsheh -----Original Message----- From: Wiles, Keith [mailto:keith.wiles@intel.com]=20 Sent: Thursday, June 7, 2018 10:10 PM To: Raslan Darawsheh Cc: Thomas Monjalon ; dev@dpdk.org Subject: Re: [dpdk-dev] [RFC] net/tap: add queues when attaching from secon= dary process > On Jun 7, 2018, at 5:29 AM, Raslan Darawsheh wrote= : >=20 > In the case where the device is created by the primary process. > Currently, secondary process only contains the eth dev without being > able to do any Rx/Tx. >=20 > When attaching the device from secondary process this patch adds queues > info got from IPC massaging. Can you explain this details a bit more here, not sure I follow the real pr= oblem and the solution? >=20 > Signed-off-by: Raslan Darawsheh > --- > drivers/net/tap/Makefile | 2 + > drivers/net/tap/rte_eth_tap.c | 105 +++++++++++++++++++++++++++++++++++++= ++++- > 2 files changed, 106 insertions(+), 1 deletion(-) >=20 > diff --git a/drivers/net/tap/Makefile b/drivers/net/tap/Makefile > index ccc5c5f..913423c 100644 > --- a/drivers/net/tap/Makefile > +++ b/drivers/net/tap/Makefile > @@ -27,6 +27,8 @@ LDLIBS +=3D -lrte_ethdev -lrte_net -lrte_kvargs -lrte_h= ash > LDLIBS +=3D -lrte_bus_vdev >=20 > CFLAGS +=3D -DTAP_MAX_QUEUES=3D$(TAP_MAX_QUEUES) > +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API > + >=20 > # > # all source are stored in SRCS-y > diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.= c > index 5531fe9..0f4c8d9 100644 > --- a/drivers/net/tap/rte_eth_tap.c > +++ b/drivers/net/tap/rte_eth_tap.c > @@ -16,6 +16,8 @@ > #include > #include > #include > +#include > +#include >=20 > #include > #include > @@ -55,6 +57,9 @@ > #define ETH_TAP_CMP_MAC_FMT "0123456789ABCDEFabcdef" > #define ETH_TAP_MAC_ARG_FMT ETH_TAP_MAC_FIXED "|" ETH_TAP_USR_MAC_FMT >=20 > +/* IPC key for communication of queue fds between processes. */ > +#define TAP_MP_KEY "tap_mp_exchange_fds" > + > static struct rte_vdev_driver pmd_tap_drv; > static struct rte_vdev_driver pmd_tun_drv; >=20 > @@ -93,6 +98,15 @@ enum ioctl_mode { > REMOTE_ONLY, > }; >=20 > +/* To communicate queue infos between processes */ > +struct queues_fds { > + char name[RTE_DEV_NAME_MAX_LEN]; > + int num_rxq_fds; > + int num_txq_fds; > + int rxq_fds[RTE_PMD_TAP_MAX_QUEUES]; > + int txq_fds[RTE_PMD_TAP_MAX_QUEUES]; > +}; > + > static int tap_intr_handle_set(struct rte_eth_dev *dev, int set); >=20 > /** > @@ -1731,6 +1745,47 @@ rte_pmd_tun_probe(struct rte_vdev_device *dev) > return ret; > } >=20 > +/* > + * Send the queues fds from the primary process to secondary. > + */ > +static int > +tap_exchange_queues_fds(const struct rte_mp_msg *mp_msg, const void *pee= r) > +{ > + struct rte_eth_dev *eth_dev; > + struct rte_mp_msg mp_resp; > + struct queues_fds *out =3D (struct queues_fds *)&mp_resp.param; > + const struct queues_fds *in =3D (const struct queues_fds *)mp_msg->para= m; > + uint16_t port_id; > + int i, ret; > + > + TAP_LOG(DEBUG, "received request"); > + strlcpy(out->name, in->name, sizeof(out->name)); > + ret =3D rte_eth_dev_get_port_by_name(in->name, &port_id); > + if (ret) { > + TAP_LOG(ERR, "Failed to get dev %s", in->name); > + return -1; > + } > + eth_dev =3D &rte_eth_devices[port_id]; > + struct pmd_internals *pmd =3D eth_dev->data->dev_private; > + > + /* fill the queues fds data in the reply msg */ > + strlcpy(mp_resp.name, TAP_MP_KEY, sizeof(mp_resp.name)); > + out->num_rxq_fds =3D eth_dev->data->nb_rx_queues; > + for (i =3D 0; i < eth_dev->data->nb_rx_queues; i++) > + out->rxq_fds[i] =3D pmd->rxq[i].fd; > + out->num_txq_fds =3D eth_dev->data->nb_tx_queues; > + for (i =3D 0; i < eth_dev->data->nb_tx_queues; i++) > + out->txq_fds[i] =3D pmd->txq[i].fd; > + mp_resp.len_param =3D sizeof(*out); > + mp_resp.num_fds =3D 0; > + if (rte_mp_reply(&mp_resp, peer) < 0) { > + TAP_LOG(ERR, "Failed to reply a fds request"); > + return -1; > + } > + > + return 0; > +} > + > /* Open a TAP interface device. > */ > static int > @@ -1744,6 +1799,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) > char remote_iface[RTE_ETH_NAME_MAX_LEN]; > struct ether_addr user_mac =3D { .addr_bytes =3D {0} }; > struct rte_eth_dev *eth_dev; > + int queue_id; >=20 > strcpy(tuntap_name, "TAP"); >=20 > @@ -1757,8 +1813,46 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) > TAP_LOG(ERR, "Failed to probe %s", name); > return -1; > } > - /* TODO: request info from primary to set up Rx and Tx */ > eth_dev->dev_ops =3D &ops; > + /* request a sync from the primary process to get queues fds */ > + eth_dev->rx_pkt_burst =3D pmd_rx_burst; > + eth_dev->tx_pkt_burst =3D pmd_tx_burst; > + if (!rte_eal_primary_proc_alive(NULL)) { > + TAP_LOG(ERR, "cannot initialize secondary process " > + "without a primary one"); > + return -1; > + } > + struct rte_mp_msg mp_req, *mp_rep; > + struct rte_mp_reply mp_reply; > + struct timespec ts =3D {.tv_sec =3D 5, .tv_nsec =3D 0}; > + struct queues_fds *req =3D (struct queues_fds *)mp_req.param; > + struct queues_fds *resp; What is the rule for DPDK coding style of having declares in the middle of = a block, I thought we only wanted them at the beginning of block of code. > + > + strlcpy(req->name, name, sizeof(mp_req.name)); > + strlcpy(mp_req.name, TAP_MP_KEY, sizeof(mp_req.name)); > + mp_req.len_param =3D sizeof(*req); > + /* request for sync from primary process to get queues fds. */ > + if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) =3D=3D 0 && > + mp_reply.nb_received =3D=3D 1) { > + mp_rep =3D &mp_reply.msgs[0]; > + resp =3D (struct queues_fds *)mp_rep->param; > + TAP_LOG(INFO, "Received fds for %d rx_queues and " > + "%d tx_queues", resp->num_rxq_fds, > + resp->num_txq_fds); > + } else { > + TAP_LOG(ERR, "Failed to request queues from primary"); > + return -1; > + } I thought passing a FD from process to process you had to have the kernel c= onvert the FD to the local process value. At least that was the way it was = done in mmap memory FD. Is this the same problem and needs to be send in a = message using a control structure with the correct flags set? > + > + struct pmd_internals *pmd =3D eth_dev->data->dev_private; > + for (queue_id =3D 0; queue_id < resp->num_rxq_fds; queue_id++) > + pmd->rxq[queue_id].fd =3D resp->rxq_fds[queue_id]; > + > + for (queue_id =3D 0; queue_id < resp->num_txq_fds; queue_id++) > + pmd->txq[queue_id].fd =3D resp->txq_fds[queue_id]; > + > + TAP_LOG(NOTICE, "Initializing secondary process pmd_tap for %s", > + name); > rte_eth_dev_probing_finish(eth_dev); > return 0; > } > @@ -1806,6 +1900,14 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) > TAP_LOG(NOTICE, "Initializing pmd_tap for %s as %s", > name, tap_name); >=20 > + /* register for mp communication between secondary and primary */ > + if (rte_mp_action_register(TAP_MP_KEY, tap_exchange_queues_fds) && > + rte_errno !=3D EEXIST) { > + TAP_LOG(ERR, "%s : %s fail to register MP action : %s", > + tuntap_name, name, strerror(errno)); > + return -1; > + } > + > ret =3D eth_dev_tap_create(dev, tap_name, remote_iface, &user_mac, > ETH_TUNTAP_TYPE_TAP); >=20 > @@ -1813,6 +1915,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev) > if (ret =3D=3D -1) { > TAP_LOG(ERR, "Failed to create pmd for %s as %s", > name, tap_name); > + rte_mp_action_unregister(TAP_MP_KEY); > tap_unit--; /* Restore the unit number */ > } > rte_kvargs_free(kvlist); > --=20 > 2.7.4 >=20 Regards, Keith