From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <rasland@mellanox.com>
Received: from EUR04-DB3-obe.outbound.protection.outlook.com
(mail-eopbgr60080.outbound.protection.outlook.com [40.107.6.80])
by dpdk.org (Postfix) with ESMTP id D869972FA
for <dev@dpdk.org>; Fri, 8 Jun 2018 01:24:04 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com;
s=selector1;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=3PTuVBk/3qWBUQKEqHQxmdiFNAaT8yXcn+hHmlAUWnU=;
b=UTcff5JFGcuyo+15LNGAHlS4IllKj27v5aY7QpYf0oprAaLc9kE0T8AZ/SKkUf1G66SFjW6hQIUU+W2M5w6eLPn8hP6Www1fIhgFIaEbVAEEtnrKgJY2gYGujIJ1rIbap/ATelIkkkXBnvjBrL0QONl9yYLyLrLuOi8qrjDIQLg=
Received: from DB5PR05MB1254.eurprd05.prod.outlook.com (10.162.157.140) by
DB5PR05MB1877.eurprd05.prod.outlook.com (10.166.173.28) with Microsoft SMTP
Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.20.841.14; Thu, 7 Jun 2018 23:24:01 +0000
Received: from DB5PR05MB1254.eurprd05.prod.outlook.com
([fe80::40c5:ca67:87b6:fc5e]) by DB5PR05MB1254.eurprd05.prod.outlook.com
([fe80::40c5:ca67:87b6:fc5e%4]) with mapi id 15.20.0841.011; Thu, 7 Jun 2018
23:24:01 +0000
From: Raslan Darawsheh <rasland@mellanox.com>
To: "Wiles, Keith" <keith.wiles@intel.com>
CC: Thomas Monjalon <thomas@monjalon.net>, "dev@dpdk.org" <dev@dpdk.org>
Thread-Topic: [dpdk-dev] [RFC] net/tap: add queues when attaching from
secondary process
Thread-Index: AQHT/pMfsazoxn4/UUWcNwN629lPw6RVbtsw
Date: Thu, 7 Jun 2018 23:24:01 +0000
Message-ID: <DB5PR05MB12545AB6F7AF3E4A049E1031C2640@DB5PR05MB1254.eurprd05.prod.outlook.com>
References: <1528374591-26126-1-git-send-email-rasland@mellanox.com>
<077BA557-4ED0-4BAF-AFD7-8F54116229DB@intel.com>
In-Reply-To: <077BA557-4ED0-4BAF-AFD7-8F54116229DB@intel.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is )
smtp.mailfrom=rasland@mellanox.com;
x-originating-ip: [188.161.229.198]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; DB5PR05MB1877;
7:oNz/j52E/pIW8+TOQl/WIRgZzRXIkdDIHeoBBCAsKGfU2fG4sReONFTRkCeG0327YfyOjbrz7oRNE2r8YMxcqF4fi6ko5KADR5OvwUHCRc/IRfiefCQJ2Yg1N8IvarGVmI8okgR9sXpHd6rSRGPH5KAiUlUidYSysp0CA5ZPS3kiPOA4RtN/Dm4RfnRJeKZuxUIO0RYuqnyyxbrKE0ZPl5iScsXpstB06CMTmWsaHyi73fHOEv/1HSFhWYwx6ql5
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0;
RULEID:(7020095)(4652020)(4534165)(4627221)(201703031133081)(201702281549075)(5600026)(48565401081)(2017052603328)(7153060)(7193020);
SRVR:DB5PR05MB1877;
x-ms-traffictypediagnostic: DB5PR05MB1877:
x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr
x-microsoft-antispam-prvs: <DB5PR05MB1877DC489D0B2E58D5659B49C2640@DB5PR05MB1877.eurprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(228905959029699);
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0;
RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(93006095)(93001095)(3002001)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016);
SRVR:DB5PR05MB1877; BCL:0; PCL:0; RULEID:; SRVR:DB5PR05MB1877;
x-forefront-prvs: 06968FD8C4
x-forefront-antispam-report: SFV:NSPM;
SFS:(10009020)(979002)(366004)(346002)(39860400002)(396003)(39380400002)(376002)(13464003)(199004)(189003)(74316002)(2900100001)(305945005)(5250100002)(14454004)(105586002)(26005)(186003)(106356001)(5890100001)(7736002)(86362001)(476003)(486006)(229853002)(6436002)(3280700002)(102836004)(446003)(66066001)(6916009)(81166006)(81156014)(97736004)(76176011)(316002)(55016002)(99286004)(478600001)(54906003)(25786009)(11346002)(8936002)(3660700001)(7696005)(33656002)(68736007)(6116002)(53936002)(2906002)(4326008)(6246003)(9686003)(5660300001)(3846002)(53546011)(6506007)(59450400001)(969003)(989001)(999001)(1009001)(1019001);
DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR05MB1877;
H:DB5PR05MB1254.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en;
PTR:InfoNoRecords; A:1; MX:1;
received-spf: None (protection.outlook.com: mellanox.com does not designate
permitted sender hosts)
x-microsoft-antispam-message-info: M/M2qlSfPUPFfZL5YSRiNCYT61atQOh6U2HD7TF6nCMs9azi9xiHL5LvhbMQpzexUM64Bbzi6WiZsyf4KhmFBhDovLMFBOKxhaI05Lv2CqFbnmfJfzlQxm7N9GifWT5w6DxaDV+X5YiU9Pd+0ye9/hUtRVzG8YNor12/QPpmMUNtYFQYs47HE/Dlya683oxb
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Office365-Filtering-Correlation-Id: 118c9f88-d575-493d-62f9-08d5cccdc0b1
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 118c9f88-d575-493d-62f9-08d5cccdc0b1
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jun 2018 23:24:01.4995 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1877
Subject: Re: [dpdk-dev] [RFC] net/tap: add queues when attaching from
secondary process
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Jun 2018 23:24:05 -0000
Hi,
As you know that currently TAP pmd support attaching a secondary process to=
a primary process.=20
But, it's still lacking the ability to do Rx/Tx burst since it's lacking th=
e necessary fds for RX/TX queues,
And the setting of Rx/Tx burst function.
This patch the main purpose is to exchange the fds between the processes th=
row the IPC massages
And to set the Rx/Tx functions for the secondary.
I hope I explained it properly, please let me know if you still didn't get =
it.=20
Kindest regards,
Raslan Darawsheh
-----Original Message-----
From: Wiles, Keith [mailto:keith.wiles@intel.com]=20
Sent: Thursday, June 7, 2018 10:10 PM
To: Raslan Darawsheh <rasland@mellanox.com>
Cc: Thomas Monjalon <thomas@monjalon.net>; dev@dpdk.org
Subject: Re: [dpdk-dev] [RFC] net/tap: add queues when attaching from secon=
dary process
> On Jun 7, 2018, at 5:29 AM, Raslan Darawsheh <rasland@mellanox.com> wrote=
:
>=20
> In the case where the device is created by the primary process.
> Currently, secondary process only contains the eth dev without being
> able to do any Rx/Tx.
>=20
> When attaching the device from secondary process this patch adds queues
> info got from IPC massaging.
Can you explain this details a bit more here, not sure I follow the real pr=
oblem and the solution?
>=20
> Signed-off-by: Raslan Darawsheh <rasland@mellanox.com>
> ---
> drivers/net/tap/Makefile | 2 +
> drivers/net/tap/rte_eth_tap.c | 105 +++++++++++++++++++++++++++++++++++++=
++++-
> 2 files changed, 106 insertions(+), 1 deletion(-)
>=20
> diff --git a/drivers/net/tap/Makefile b/drivers/net/tap/Makefile
> index ccc5c5f..913423c 100644
> --- a/drivers/net/tap/Makefile
> +++ b/drivers/net/tap/Makefile
> @@ -27,6 +27,8 @@ LDLIBS +=3D -lrte_ethdev -lrte_net -lrte_kvargs -lrte_h=
ash
> LDLIBS +=3D -lrte_bus_vdev
>=20
> CFLAGS +=3D -DTAP_MAX_QUEUES=3D$(TAP_MAX_QUEUES)
> +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API
> +
>=20
> #
> # all source are stored in SRCS-y
> diff --git a/drivers/net/tap/rte_eth_tap.c b/drivers/net/tap/rte_eth_tap.=
c
> index 5531fe9..0f4c8d9 100644
> --- a/drivers/net/tap/rte_eth_tap.c
> +++ b/drivers/net/tap/rte_eth_tap.c
> @@ -16,6 +16,8 @@
> #include <rte_debug.h>
> #include <rte_ip.h>
> #include <rte_string_fns.h>
> +#include <rte_ethdev.h>
> +#include <rte_errno.h>
>=20
> #include <sys/types.h>
> #include <sys/stat.h>
> @@ -55,6 +57,9 @@
> #define ETH_TAP_CMP_MAC_FMT "0123456789ABCDEFabcdef"
> #define ETH_TAP_MAC_ARG_FMT ETH_TAP_MAC_FIXED "|" ETH_TAP_USR_MAC_FMT
>=20
> +/* IPC key for communication of queue fds between processes. */
> +#define TAP_MP_KEY "tap_mp_exchange_fds"
> +
> static struct rte_vdev_driver pmd_tap_drv;
> static struct rte_vdev_driver pmd_tun_drv;
>=20
> @@ -93,6 +98,15 @@ enum ioctl_mode {
> REMOTE_ONLY,
> };
>=20
> +/* To communicate queue infos between processes */
> +struct queues_fds {
> + char name[RTE_DEV_NAME_MAX_LEN];
> + int num_rxq_fds;
> + int num_txq_fds;
> + int rxq_fds[RTE_PMD_TAP_MAX_QUEUES];
> + int txq_fds[RTE_PMD_TAP_MAX_QUEUES];
> +};
> +
> static int tap_intr_handle_set(struct rte_eth_dev *dev, int set);
>=20
> /**
> @@ -1731,6 +1745,47 @@ rte_pmd_tun_probe(struct rte_vdev_device *dev)
> return ret;
> }
>=20
> +/*
> + * Send the queues fds from the primary process to secondary.
> + */
> +static int
> +tap_exchange_queues_fds(const struct rte_mp_msg *mp_msg, const void *pee=
r)
> +{
> + struct rte_eth_dev *eth_dev;
> + struct rte_mp_msg mp_resp;
> + struct queues_fds *out =3D (struct queues_fds *)&mp_resp.param;
> + const struct queues_fds *in =3D (const struct queues_fds *)mp_msg->para=
m;
> + uint16_t port_id;
> + int i, ret;
> +
> + TAP_LOG(DEBUG, "received request");
> + strlcpy(out->name, in->name, sizeof(out->name));
> + ret =3D rte_eth_dev_get_port_by_name(in->name, &port_id);
> + if (ret) {
> + TAP_LOG(ERR, "Failed to get dev %s", in->name);
> + return -1;
> + }
> + eth_dev =3D &rte_eth_devices[port_id];
> + struct pmd_internals *pmd =3D eth_dev->data->dev_private;
> +
> + /* fill the queues fds data in the reply msg */
> + strlcpy(mp_resp.name, TAP_MP_KEY, sizeof(mp_resp.name));
> + out->num_rxq_fds =3D eth_dev->data->nb_rx_queues;
> + for (i =3D 0; i < eth_dev->data->nb_rx_queues; i++)
> + out->rxq_fds[i] =3D pmd->rxq[i].fd;
> + out->num_txq_fds =3D eth_dev->data->nb_tx_queues;
> + for (i =3D 0; i < eth_dev->data->nb_tx_queues; i++)
> + out->txq_fds[i] =3D pmd->txq[i].fd;
> + mp_resp.len_param =3D sizeof(*out);
> + mp_resp.num_fds =3D 0;
> + if (rte_mp_reply(&mp_resp, peer) < 0) {
> + TAP_LOG(ERR, "Failed to reply a fds request");
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> /* Open a TAP interface device.
> */
> static int
> @@ -1744,6 +1799,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev)
> char remote_iface[RTE_ETH_NAME_MAX_LEN];
> struct ether_addr user_mac =3D { .addr_bytes =3D {0} };
> struct rte_eth_dev *eth_dev;
> + int queue_id;
>=20
> strcpy(tuntap_name, "TAP");
>=20
> @@ -1757,8 +1813,46 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev)
> TAP_LOG(ERR, "Failed to probe %s", name);
> return -1;
> }
> - /* TODO: request info from primary to set up Rx and Tx */
> eth_dev->dev_ops =3D &ops;
> + /* request a sync from the primary process to get queues fds */
> + eth_dev->rx_pkt_burst =3D pmd_rx_burst;
> + eth_dev->tx_pkt_burst =3D pmd_tx_burst;
> + if (!rte_eal_primary_proc_alive(NULL)) {
> + TAP_LOG(ERR, "cannot initialize secondary process "
> + "without a primary one");
> + return -1;
> + }
> + struct rte_mp_msg mp_req, *mp_rep;
> + struct rte_mp_reply mp_reply;
> + struct timespec ts =3D {.tv_sec =3D 5, .tv_nsec =3D 0};
> + struct queues_fds *req =3D (struct queues_fds *)mp_req.param;
> + struct queues_fds *resp;
What is the rule for DPDK coding style of having declares in the middle of =
a block, I thought we only wanted them at the beginning of block of code.
> +
> + strlcpy(req->name, name, sizeof(mp_req.name));
> + strlcpy(mp_req.name, TAP_MP_KEY, sizeof(mp_req.name));
> + mp_req.len_param =3D sizeof(*req);
> + /* request for sync from primary process to get queues fds. */
> + if (rte_mp_request_sync(&mp_req, &mp_reply, &ts) =3D=3D 0 &&
> + mp_reply.nb_received =3D=3D 1) {
> + mp_rep =3D &mp_reply.msgs[0];
> + resp =3D (struct queues_fds *)mp_rep->param;
> + TAP_LOG(INFO, "Received fds for %d rx_queues and "
> + "%d tx_queues", resp->num_rxq_fds,
> + resp->num_txq_fds);
> + } else {
> + TAP_LOG(ERR, "Failed to request queues from primary");
> + return -1;
> + }
I thought passing a FD from process to process you had to have the kernel c=
onvert the FD to the local process value. At least that was the way it was =
done in mmap memory FD. Is this the same problem and needs to be send in a =
message using a control structure with the correct flags set?
> +
> + struct pmd_internals *pmd =3D eth_dev->data->dev_private;
> + for (queue_id =3D 0; queue_id < resp->num_rxq_fds; queue_id++)
> + pmd->rxq[queue_id].fd =3D resp->rxq_fds[queue_id];
> +
> + for (queue_id =3D 0; queue_id < resp->num_txq_fds; queue_id++)
> + pmd->txq[queue_id].fd =3D resp->txq_fds[queue_id];
> +
> + TAP_LOG(NOTICE, "Initializing secondary process pmd_tap for %s",
> + name);
> rte_eth_dev_probing_finish(eth_dev);
> return 0;
> }
> @@ -1806,6 +1900,14 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev)
> TAP_LOG(NOTICE, "Initializing pmd_tap for %s as %s",
> name, tap_name);
>=20
> + /* register for mp communication between secondary and primary */
> + if (rte_mp_action_register(TAP_MP_KEY, tap_exchange_queues_fds) &&
> + rte_errno !=3D EEXIST) {
> + TAP_LOG(ERR, "%s : %s fail to register MP action : %s",
> + tuntap_name, name, strerror(errno));
> + return -1;
> + }
> +
> ret =3D eth_dev_tap_create(dev, tap_name, remote_iface, &user_mac,
> ETH_TUNTAP_TYPE_TAP);
>=20
> @@ -1813,6 +1915,7 @@ rte_pmd_tap_probe(struct rte_vdev_device *dev)
> if (ret =3D=3D -1) {
> TAP_LOG(ERR, "Failed to create pmd for %s as %s",
> name, tap_name);
> + rte_mp_action_unregister(TAP_MP_KEY);
> tap_unit--; /* Restore the unit number */
> }
> rte_kvargs_free(kvlist);
> --=20
> 2.7.4
>=20
Regards,
Keith