From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00053.outbound.protection.outlook.com [40.107.0.53]) by dpdk.org (Postfix) with ESMTP id EDA1F1B29A for ; Thu, 18 Jan 2018 11:02:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=sj1sfOkHSnabXiWRwHkETv01QtbEB5YtE2EVse5Qwfs=; b=as6A7S0w+cDA7Y0XUssgUsp/YUKsgkRVhhYNuaKS0BnGD3/rUH5EiKVMMr8JkFQtbgRzTJC0JQ4DY1WGAmegYFjFeTI3mzPoxHtGwSxGUoAEgZMctyPNcuCgZ/7k9rg/jgYSiBLY/OjF7i62t9jD/nbmZOFjE5nma2j80YPUwEQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; Received: from mellanox.com (37.142.13.130) by AM6PR0502MB3797.eurprd05.prod.outlook.com (2603:10a6:209:a::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.428.17; Thu, 18 Jan 2018 10:02:06 +0000 From: Matan Azrad To: Ferruh Yigit Cc: Thomas Monjalon , dev@dpdk.org, stephen@networkplumber.org, Adrien Mazarguil Date: Thu, 18 Jan 2018 10:01:46 +0000 Message-Id: <1516269709-15252-6-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1516269709-15252-1-git-send-email-matan@mellanox.com> References: <1516265026-6469-1-git-send-email-matan@mellanox.com> <1516269709-15252-1-git-send-email-matan@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: VI1PR0202CA0033.eurprd02.prod.outlook.com (2603:10a6:803:14::46) To AM6PR0502MB3797.eurprd05.prod.outlook.com (2603:10a6:209:a::26) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f1eb1260-cedc-4540-1e22-08d55e5a8883 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(48565401081)(2017052603307)(7153060)(7193020); SRVR:AM6PR0502MB3797; X-Microsoft-Exchange-Diagnostics: 1; AM6PR0502MB3797; 3:65RoeFau9dfxdRoJu0k5VSbGwLynWSsux8JFhTkXIe93ppspj01PKa5JFEt/Xiws3QKyf6orttWj1Zzf4p0WQOToXz0NaZYyysVzgJTdGJKFU59uJPoQZBaR/EJbKdDqG4rgE4ejGoy1fDJwwkZRgrB784U963NZuFjS6pjelDyp2hVQaAFLBUuH5BnxtdZJnqvd+KPP1unwuedzoAQEyLo3YRE0ckRIU0ikQLgOwoGSvob+Tqpmepjyx9JqGc2+; 25:wZTBN4t2VbwpBFQiv2rpvzzMoIjB7jeqrqQtHkeyTwh9ngfVHSzjtqUTMTruqnvPIs7e3/0BJrSb8JNk9Wp1rti4ryo/TTKs3Hf3FseNWAvN2hO06EDehOu0sDbLAeeDGoyverTJ9cSJtlofcbcCDDYvh63foJn9/wXFObb8u4SNJamg6KiH4LjReXDlJ9jAoTas8hT3dtIZ6r7cJ4UBhuMLAjTtOYz9EwBfto8n/V5wKjsvkuhLxEw5C83gwjXzcZqXCH3cCDhL/dE6y3ffwFcaOh5FHm5V3+6nie5SLnEaD+9o0IlH7hIIp6xXhWwtryEeCxJDv5J7kKw9AzI9uw==; 31:oimFv5RvqDjL/uLxsZpX/Qb+vteIepMFJ5CUDgwZaj3DXBilaeZUyVOLgIKRC41Zs0kKXVGM9e2mLrFx7FY7Elxj3LqoF8pkaw95ym0WwYVXkTeBKSA4Yc4uDW83qe0+KWfHV7t+rXPKtQs8kgRMMIxrbvB9CNu/3bseqBIifuOZpjUdgfO33+yZA/bp3mehQVfiPrh4hJBPMu8XVsyyAPR+wdsQYgmfb4j3U0nL4YA= X-MS-TrafficTypeDiagnostic: AM6PR0502MB3797: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; AM6PR0502MB3797; 20:cpY6n4F4kJPLupZ+WuFw4Tze33/82mMj9Rk4QOONdNrEF42zut+1+jVfuFBjxGlxDZxaNvT8l236EhQvYdiVw0xKUcmOLoQp/US7613jCtCXUvN+HnLNCh9wZ/w5NOYI2ndko82muCe5BBxs4aly1FZY0u4Ys5WsOgfecrVB6Lhy+lz7K/XH/brvUiP00IadULGq4J/Z++c7mAfhFFR4ROMCqWgUpXkaarfECUhPr28Nmf9xwUkeVry0qBq9rntxdrwJMQmGE5GMQUY3mvpPKaF8n+yiG1a/zY5vQ4cdBeAH7PBQGZEFDOAh/bBQcUE8gyDMeEBgkLnJM2Bs/g4OBIPagR9w8H+dJxCdEtf9nUsjYo5JNYoopPWSKip4ZpaGbzGxkBnuBj55eBxd/0zZ3njnBiaGPj0bFplxQTnFWqvh7SgP/rS9splGy6T32YvJAU0L8/+O/0ZtP6S9ND7VMcQzkMXxM5VxVqotjT19mCXIsBydpJXwkOu9l37g7f2W; 4:+sNSWSNjFcYCR5kYDQKPQhOk1h2cJfElc4ABd38aOcofxYVPqZkhnY4xnmryxGzF7MwLyMLIGoPVhbIQPYkRUCkGkynzT54LtL0BtOH5FyCvveK5Dq5HLspw9Ud10Pkf9RdkfxzyVEmwCGMYt1ptyUKN3K1BYWeVBLK8JBOfDer6Bj5+a0tuJ8arW/GwkyjFLBSPiSDudBK9F+hNos1TgCxUnVFPJYVXT+l/5rf6FTvmdCaidfl6YnMdbkw4nny+IQby4j1z14UhHMD5hyJ5cqreYyXmetofNzYxZeSrYSDoefPDmTp7QQciBu1owRRWuqJOXYpKosGRamDRtAmeaZxjagCJBqJDVPb3hhe0hW4= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(28532068793085)(278428928389397); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(5005006)(8121501046)(10201501046)(3231023)(2400060)(944501161)(3002001)(93006095)(93001095)(6055026)(6041268)(20161123558120)(20161123564045)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:AM6PR0502MB3797; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:AM6PR0502MB3797; X-Forefront-PRVS: 05568D1FF7 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39860400002)(366004)(39380400002)(346002)(396003)(376002)(199004)(189003)(6116002)(53936002)(76176011)(386003)(4720700003)(59450400001)(26005)(7736002)(305945005)(50466002)(66066001)(2906002)(47776003)(966005)(16586007)(54906003)(5890100001)(8676002)(478600001)(316002)(21086003)(2950100002)(25786009)(48376002)(52116002)(6666003)(51416003)(7696005)(6916009)(81166006)(81156014)(8936002)(4326008)(86362001)(575784001)(16526018)(6306002)(97736004)(68736007)(36756003)(50226002)(5660300001)(105586002)(3846002)(69596002)(106356001)(33026002)(55016002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR0502MB3797; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM6PR0502MB3797; 23:/nwtaXBZxjU8PXMRGPj+1hLOYQDcfLeMA2lpShR?= =?us-ascii?Q?6dTXhNfHDe9zFuqCwCFgxe6p9FCHUty9AmGWWy4TaD0xH3L6Ii6sAUiX998j?= =?us-ascii?Q?KOsOCFggb3EehAFHWzWFljRrv6CLnXBq3k4qommo4QL5nl91EbdZtsB9S87h?= =?us-ascii?Q?GHaL8mAh1icmJUHEBwHgHK687Q/eAPd74z21rQ+8KGVqJXSON9YFSXoQOgef?= =?us-ascii?Q?+VoRQqF0swwRNYHda1D2p5VBMzeKMYPWPJMaXLXO22UD935AbZaxO3RjQrdi?= =?us-ascii?Q?xEDftyzwtp4xPQ4wk8XarBfEyvoWg9qdmiNaudxeh3T2Vl1+Hirk/txtpPlz?= =?us-ascii?Q?qmNT9bVmNJ7phsPpz0v/3ENXEKcejM1Qc8y+xXh6BXQnuvM/j30sE8Y/KaBI?= =?us-ascii?Q?z+7f91wdlvuALg2mcW6dj99f5BP0JmH0f+vKjOhgz2134sIsUTZQ/C3FCCLY?= =?us-ascii?Q?fImitE9AEvcPZKUq62qb8F5mDjPz1DPZCKW1T+n9bXFtpqpofZVWKFHmeJYj?= =?us-ascii?Q?DqU8RiSwTY5TTb5sHT9cN/1vk5JtLp/9KKncNmLKAe/QtV2z3OQQ1N+a/nXT?= =?us-ascii?Q?rAG625KoamvTWasjL0IhZE4GZMgX2Ka+WIP4DPtcB/2jGfKQshhwz1yRic6J?= =?us-ascii?Q?grMLxJR5jhOQ7i2jv/7wySR4O6YH+0B/W5jYLZoqK2viZsUNIc5YfopPT4bi?= =?us-ascii?Q?ShZoHpdP5yf5Tylr9x+NJNcKwS3L/j/xzbc2xsKvyvUeP7ZqqDPDICNmHVQw?= =?us-ascii?Q?BUrf9kJdYISLGf3MDwLu2fmBDzQeVvseDKogCNw2cUb8g8qAi37yCyPpFZgB?= =?us-ascii?Q?+UOCCWLnC0tzZSexH7ABjyV1nvzKq8loqdOJbY4PjDKwvJJzWOxnT4LFAO+j?= =?us-ascii?Q?Pid6EnHCrxuEJZ0wTJvSO0IilkW3kMhQBX8m/WSbPqXqvW5WOSBowXjfLi5P?= =?us-ascii?Q?xAKR7/4XddM5YsKqovkF/iMxrXojWJLYHg6MUg478+AFVrneKsBASjLenGCg?= =?us-ascii?Q?p57H4Em2ecFVc0lRi15IAS7SDFjSLjedzSFJ7bebmTZulN3U1MXrVpvxzSnL?= =?us-ascii?Q?3eAVUTGUrppod9XzTGrYOHvOoGCTLjYpt8101pFmP1vBsujhRFJwxcuonp99?= =?us-ascii?Q?lVfdfFODyGvD9MxVCFUHkncx3dicp9PAAY5JzcchTUbG6cv6jMqcH/yy5TA5?= =?us-ascii?Q?9hA1P1fEMFhL0I7vKUk9G9Hb6zsjX5ga9Z4yw2dZrQiye31DDmelKFRiHEI4?= =?us-ascii?Q?IIy3xUobcRnNW1RvyAvNAwyxjo/fIsMhef71JF8tWuC+/TaagoHWxC2axblc?= =?us-ascii?Q?V8Q=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; AM6PR0502MB3797; 6:+aV8FpgxKZW5JRytAxARGWuABkn26phr4tFRYTGkfcbtKOJYQQIAw8A+Ws/Ul6sWgg6t9zS6y3Yglz3MRKOmOu4ApNwSXHsaj1/pg/e2gsMm0BfT+nKAj9/VjYCndsn/qr9j5m9NhQuP1H3QOOwrlY5eMFcRQCs3k0VhAhTtFKkkkadicMjTKqzVmT6dyfo2R0NdcdqkzWLr2wjXIxLJf3D6TrJez7IJoAaHhqZV4/lhvxbstO/jcxl6gon9fzLWtTb0Pa6E0lobzHAO/+zHmkpBr/BaK+Gjkezmubp3c2pbcoKkRLKp8+ewS/tH4sJ9RIHQrn4wTmCQfABxd0l2XEhGx53lJkUEuUGgwrP75qU=; 5:ClQaJEP1MZT9XS3gg8jT41FTX+pqErkL0I7TGKwfUXFEJ/bLt21OB0iRGckOZe4aW/Rmm37X2ii6Mv5wxjEtgjFItScRbmwLYcw3aALmp/BHLU6BY5aj96M+EEnRmXLNzZ7m7nL6banyYKqSGYJjuaN7JXKWzUIDYDdRlOSl12g=; 24:AsnsiOuR/ykvLfYApTRL8QPS/JHgo4IZDtPiaWAebcvHvLSwwhNIupRRu/zCYsXl19DHUvABfxXfPaTGQVKrRr1MHYCCOPXBzlSfBqgMLtc=; 7:5cNg8bCIxckOtnhaWa4Qu6ZUxE5RhKN1ci2uHVKm9+J9VgdReN2EkDM68UllBtZuFPxWU+fMYdewc9BSIiT3cRHpBE4ge+Ac1W92vhPbHLx5A9eohfeC359W3SM6v+W3ayJ4Ry/A+V+naxsDgrIXd3OxxGo6XDosO8APjO4AfGGtHvNX1iMuWDRoOwU+pDwFj007pnMK3zw5Lq9ifaZC9UO53WFt3y/P4cdgW4a5pcB0SIHgCJZkojv3nk0k4mDY SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2018 10:02:06.5811 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f1eb1260-cedc-4540-1e22-08d55e5a8883 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0502MB3797 Subject: [dpdk-dev] [PATCH v5 5/8] net/vdev_netvsc: implement core functionality X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jan 2018 10:02:09 -0000 As described in more details in the attached documentation (see patch contents), this virtual device driver manages NetVSC interfaces in virtual machines hosted by Hyper-V/Azure platforms. This driver does not manage traffic nor Ethernet devices directly; it acts as a thin configuration layer that automatically instantiates and controls fail-safe PMD instances combining tap and PCI sub-devices, so that each NetVSC interface is exposed as a single consolidated port to DPDK applications. PCI sub-devices being hot-pluggable (e.g. during VM migration), applications automatically benefit from increased throughput when present and automatic fallback on NetVSC otherwise without interruption thanks to fail-safe's hot-plug handling. Once initialized, the sole job of the vdev_netvsc driver is to regularly scan for PCI devices to associate with NetVSC interfaces and feed their addresses to corresponding fail-safe instances. Signed-off-by: Adrien Mazarguil Signed-off-by: Matan Azrad --- doc/guides/nics/vdev_netvsc.rst | 70 +++++ drivers/net/vdev_netvsc/Makefile | 4 + drivers/net/vdev_netvsc/vdev_netvsc.c | 550 +++++++++++++++++++++++++++++++++- 3 files changed, 623 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/vdev_netvsc.rst b/doc/guides/nics/vdev_netvsc.rst index a952908..fde1fb8 100644 --- a/doc/guides/nics/vdev_netvsc.rst +++ b/doc/guides/nics/vdev_netvsc.rst @@ -12,9 +12,79 @@ platforms. .. _Hyper-V: https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-hyper-v +Implementation details +---------------------- + +Each instance of this driver effectively needs to drive two devices: the +NetVSC interface proper and its SR-IOV VF (referred to as "physical" from +this point on) counterpart sharing the same MAC address. + +Physical devices are part of the host system and cannot be maintained during +VM migration. From a VM standpoint they appear as hot-plug devices that come +and go without prior notice. + +When the physical device is present, egress and most of the ingress traffic +flows through it; only multicasts and other hypervisor control still flow +through NetVSC. Otherwise, NetVSC acts as a fallback for all traffic. + +To avoid unnecessary code duplication and ensure maximum performance, +handling of physical devices is left to their original PMDs; this virtual +device driver (also known as *vdev*) manages other PMDs as summarized by the +following block diagram:: + + .------------------. + | DPDK application | + `--------+---------' + | + .------+------. + | DPDK ethdev | + `------+------' Control + | | + .------------+------------. v .--------------------. + | failsafe PMD +---------+ vdev_netvsc driver | + `--+-------------------+--' `--------------------' + | | + | .........|......... + | : | : + .----+----. : .----+----. : + | tap PMD | : | any PMD | : + `----+----' : `----+----' : <-- Hot-pluggable + | : | : + .------+-------. : .-----+-----. : + | NetVSC-based | : | SR-IOV VF | : + | netdevice | : | device | : + `--------------' : `-----------' : + :.................: + + +This driver implementation may be temporary and should be improved or removed +either when hot-plug will be fully supported in EAL and bus drivers or when +a new NetVSC driver will be integrated. + Build options ------------- - ``CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD`` (default ``y``) Toggle compilation of this driver. + +Run-time parameters +------------------- + +To invoke this driver, applications have to explicitly provide the +``--vdev=net_vdev_netvsc`` EAL option. + +The following device parameters are supported: + +- ``iface`` [string] + + Provide a specific NetVSC interface (netdevice) name to attach this driver + to. Can be provided multiple times for additional instances. + +- ``mac`` [string] + + Same as ``iface`` except a suitable NetVSC interface is located using its + MAC address. + +Not specifying either ``iface`` or ``mac`` makes this driver attach itself to +all NetVSC interfaces found on the system. diff --git a/drivers/net/vdev_netvsc/Makefile b/drivers/net/vdev_netvsc/Makefile index 2fb059d..f2b2ac5 100644 --- a/drivers/net/vdev_netvsc/Makefile +++ b/drivers/net/vdev_netvsc/Makefile @@ -13,6 +13,9 @@ EXPORT_MAP := rte_pmd_vdev_netvsc_version.map CFLAGS += -O3 CFLAGS += -g CFLAGS += -std=c11 -pedantic -Wall -Wextra +CFLAGS += -D_XOPEN_SOURCE=600 +CFLAGS += -D_BSD_SOURCE +CFLAGS += -D_DEFAULT_SOURCE CFLAGS += $(WERROR_FLAGS) # Dependencies. @@ -20,6 +23,7 @@ LDLIBS += -lrte_bus_vdev LDLIBS += -lrte_eal LDLIBS += -lrte_ethdev LDLIBS += -lrte_kvargs +LDLIBS += -lrte_net # Source files. SRCS-$(CONFIG_RTE_LIBRTE_VDEV_NETVSC_PMD) += vdev_netvsc.c diff --git a/drivers/net/vdev_netvsc/vdev_netvsc.c b/drivers/net/vdev_netvsc/vdev_netvsc.c index e895b32..21c3265 100644 --- a/drivers/net/vdev_netvsc/vdev_netvsc.c +++ b/drivers/net/vdev_netvsc/vdev_netvsc.c @@ -3,17 +3,42 @@ * Copyright 2017 Mellanox Technologies, Ltd. */ +#include +#include +#include +#include +#include +#include +#include +#include #include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include #include #include #include +#include +#include +#include +#include #include #include #define VDEV_NETVSC_DRIVER net_vdev_netvsc #define VDEV_NETVSC_ARG_IFACE "iface" #define VDEV_NETVSC_ARG_MAC "mac" +#define VDEV_NETVSC_PROBE_MS 1000 + +#define NETVSC_CLASS_ID "{f8615163-df3e-46c5-913f-f2d2f965ed0e}" #define DRV_LOG(level, ...) \ rte_log(RTE_LOG_ ## level, \ @@ -25,12 +50,495 @@ /** Driver-specific log messages type. */ static int vdev_netvsc_logtype; +/** Context structure for a vdev_netvsc instance. */ +struct vdev_netvsc_ctx { + LIST_ENTRY(vdev_netvsc_ctx) entry; /**< Next entry in list. */ + unsigned int id; /**< Unique ID. */ + char name[64]; /**< Unique name. */ + char devname[64]; /**< Fail-safe instance name. */ + char devargs[256]; /**< Fail-safe device arguments. */ + char if_name[IF_NAMESIZE]; /**< NetVSC netdevice name. */ + unsigned int if_index; /**< NetVSC netdevice index. */ + struct ether_addr if_addr; /**< NetVSC MAC address. */ + int pipe[2]; /**< Fail-safe communication pipe. */ + char yield[256]; /**< PCI sub-device arguments. */ +}; + +/** Context list is common to all driver instances. */ +static LIST_HEAD(, vdev_netvsc_ctx) vdev_netvsc_ctx_list = + LIST_HEAD_INITIALIZER(vdev_netvsc_ctx_list); + +/** Number of entries in context list. */ +static unsigned int vdev_netvsc_ctx_count; + /** Number of driver instances relying on context list. */ static unsigned int vdev_netvsc_ctx_inst; /** + * Destroy a vdev_netvsc context instance. + * + * @param ctx + * Context to destroy. + */ +static void +vdev_netvsc_ctx_destroy(struct vdev_netvsc_ctx *ctx) +{ + if (ctx->pipe[0] != -1) + close(ctx->pipe[0]); + if (ctx->pipe[1] != -1) + close(ctx->pipe[1]); + free(ctx); +} + +/** + * Iterate over system network interfaces. + * + * This function runs a given callback function for each netdevice found on + * the system. + * + * @param func + * Callback function pointer. List traversal is aborted when this function + * returns a nonzero value. + * @param ... + * Variable parameter list passed as @p va_list to @p func. + * + * @return + * 0 when the entire list is traversed successfully, a negative error code + * in case or failure, or the nonzero value returned by @p func when list + * traversal is aborted. + */ +static int +vdev_netvsc_foreach_iface(int (*func)(const struct if_nameindex *iface, + const struct ether_addr *eth_addr, + va_list ap), ...) +{ + struct if_nameindex *iface = if_nameindex(); + int s = socket(PF_INET, SOCK_DGRAM, IPPROTO_IP); + unsigned int i; + int ret = 0; + + if (!iface) { + ret = -ENOBUFS; + DRV_LOG(ERR, "cannot retrieve system network interfaces"); + goto error; + } + if (s == -1) { + ret = -errno; + DRV_LOG(ERR, "cannot open socket: %s", rte_strerror(errno)); + goto error; + } + for (i = 0; iface[i].if_name; ++i) { + struct ifreq req; + struct ether_addr eth_addr; + va_list ap; + + strncpy(req.ifr_name, iface[i].if_name, sizeof(req.ifr_name)); + if (ioctl(s, SIOCGIFHWADDR, &req) == -1) { + DRV_LOG(WARNING, "cannot retrieve information about" + " interface \"%s\": %s", + req.ifr_name, rte_strerror(errno)); + continue; + } + if (req.ifr_hwaddr.sa_family != ARPHRD_ETHER) { + DRV_LOG(DEBUG, "interface %s is non-ethernet device", + req.ifr_name); + continue; + } + memcpy(eth_addr.addr_bytes, req.ifr_hwaddr.sa_data, + RTE_DIM(eth_addr.addr_bytes)); + va_start(ap, func); + ret = func(&iface[i], ð_addr, ap); + va_end(ap); + if (ret) + break; + } +error: + if (s != -1) + close(s); + if (iface) + if_freenameindex(iface); + return ret; +} + +/** + * Determine if a network interface is NetVSC. + * + * @param[in] iface + * Pointer to netdevice description structure (name and index). + * + * @return + * A nonzero value when interface is detected as NetVSC. In case of error, + * rte_errno is updated and 0 returned. + */ +static int +vdev_netvsc_iface_is_netvsc(const struct if_nameindex *iface) +{ + static const char temp[] = "/sys/class/net/%s/device/class_id"; + char path[sizeof(temp) + IF_NAMESIZE]; + FILE *f; + int ret; + int len = 0; + + ret = snprintf(path, sizeof(path), temp, iface->if_name); + if (ret == -1 || (size_t)ret >= sizeof(path)) { + rte_errno = ENOBUFS; + return 0; + } + f = fopen(path, "r"); + if (!f) { + rte_errno = errno; + return 0; + } + ret = fscanf(f, NETVSC_CLASS_ID "%n", &len); + if (ret == EOF) + rte_errno = errno; + ret = len == (int)strlen(NETVSC_CLASS_ID); + fclose(f); + return ret; +} + +/** + * Retrieve network interface data from sysfs symbolic link. + * + * @param[out] buf + * Output data buffer. + * @param size + * Output buffer size. + * @param[in] if_name + * Netdevice name. + * @param[in] relpath + * Symbolic link path relative to netdevice sysfs entry. + * + * @return + * 0 on success, a negative error code otherwise. + */ +static int +vdev_netvsc_sysfs_readlink(char *buf, size_t size, const char *if_name, + const char *relpath) +{ + int ret; + + ret = snprintf(buf, size, "/sys/class/net/%s/%s", if_name, relpath); + if (ret == -1 || (size_t)ret >= size) + return -ENOBUFS; + ret = readlink(buf, buf, size); + if (ret == -1) + return -errno; + if ((size_t)ret >= size - 1) + return -ENOBUFS; + buf[ret] = '\0'; + return 0; +} + +/** + * Probe a network interface to associate with vdev_netvsc context. + * + * This function determines if the network device matches the properties of + * the NetVSC interface associated with the vdev_netvsc context and + * communicates its bus address to the fail-safe PMD instance if so. + * + * It is normally used with vdev_netvsc_foreach_iface(). + * + * @param[in] iface + * Pointer to netdevice description structure (name and index). + * @param[in] eth_addr + * MAC address associated with @p iface. + * @param ap + * Variable arguments list comprising: + * + * - struct vdev_netvsc_ctx *ctx: + * Context to associate network interface with. + * + * @return + * A nonzero value when interface matches, 0 otherwise or in case of + * error. + */ +static int +vdev_netvsc_device_probe(const struct if_nameindex *iface, + const struct ether_addr *eth_addr, + va_list ap) +{ + struct vdev_netvsc_ctx *ctx = va_arg(ap, struct vdev_netvsc_ctx *); + char buf[RTE_MAX(sizeof(ctx->yield), 256u)]; + const char *addr; + size_t len; + int ret; + + /* Skip non-matching or unwanted NetVSC interfaces. */ + if (ctx->if_index == iface->if_index) { + if (!strcmp(ctx->if_name, iface->if_name)) + return 0; + DRV_LOG(DEBUG, + "NetVSC interface \"%s\" (index %u) renamed \"%s\"", + ctx->if_name, ctx->if_index, iface->if_name); + strncpy(ctx->if_name, iface->if_name, sizeof(ctx->if_name)); + return 0; + } + if (vdev_netvsc_iface_is_netvsc(iface)) + return 0; + if (!is_same_ether_addr(eth_addr, &ctx->if_addr)) + return 0; + /* Look for associated PCI device. */ + ret = vdev_netvsc_sysfs_readlink(buf, sizeof(buf), iface->if_name, + "device/subsystem"); + if (ret) + return 0; + addr = strrchr(buf, '/'); + addr = addr ? addr + 1 : buf; + if (strcmp(addr, "pci")) + return 0; + ret = vdev_netvsc_sysfs_readlink(buf, sizeof(buf), iface->if_name, + "device"); + if (ret) + return 0; + addr = strrchr(buf, '/'); + addr = addr ? addr + 1 : buf; + len = strlen(addr); + if (!len) + return 0; + /* Send PCI device argument to fail-safe PMD instance. */ + if (strcmp(addr, ctx->yield)) + DRV_LOG(DEBUG, "associating PCI device \"%s\" with NetVSC" + " interface \"%s\" (index %u)", addr, ctx->if_name, + ctx->if_index); + memmove(buf, addr, len + 1); + addr = buf; + buf[len] = '\n'; + ret = write(ctx->pipe[1], addr, len + 1); + buf[len] = '\0'; + if (ret == -1) { + if (errno == EINTR || errno == EAGAIN) + return 1; + DRV_LOG(WARNING, "cannot associate PCI device name \"%s\" with" + " interface \"%s\": %s", addr, ctx->if_name, + rte_strerror(errno)); + return 1; + } + if ((size_t)ret != len + 1) { + /* + * Attempt to override previous partial write, no need to + * recover if that fails. + */ + ret = write(ctx->pipe[1], "\n", 1); + (void)ret; + return 1; + } + fsync(ctx->pipe[1]); + memcpy(ctx->yield, addr, len + 1); + return 1; +} + +/** + * Alarm callback that regularly probes system network interfaces. + * + * This callback runs at a frequency determined by VDEV_NETVSC_PROBE_MS as + * long as an vdev_netvsc context instance exists. + * + * @param arg + * Ignored. + */ +static void +vdev_netvsc_alarm(__rte_unused void *arg) +{ + struct vdev_netvsc_ctx *ctx; + int ret; + + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) { + ret = vdev_netvsc_foreach_iface(vdev_netvsc_device_probe, ctx); + if (ret) + break; + } + if (!vdev_netvsc_ctx_count) + return; + ret = rte_eal_alarm_set(VDEV_NETVSC_PROBE_MS * 1000, + vdev_netvsc_alarm, NULL); + if (ret < 0) { + DRV_LOG(ERR, "unable to reschedule alarm callback: %s", + rte_strerror(-ret)); + } +} + +/** + * Probe a NetVSC interface to generate a vdev_netvsc context from. + * + * This function instantiates vdev_netvsc contexts either for all NetVSC + * devices found on the system or only a subset provided as device + * arguments. + * + * It is normally used with vdev_netvsc_foreach_iface(). + * + * @param[in] iface + * Pointer to netdevice description structure (name and index). + * @param[in] eth_addr + * MAC address associated with @p iface. + * @param ap + * Variable arguments list comprising: + * + * - const char *name: + * Name associated with current driver instance. + * + * - struct rte_kvargs *kvargs: + * Device arguments provided to current driver instance. + * + * - unsigned int specified: + * Number of specific netdevices provided as device arguments. + * + * - unsigned int *matched: + * The number of specified netdevices matched by this function. + * + * @return + * A nonzero value when interface matches, 0 otherwise or in case of + * error. + */ +static int +vdev_netvsc_netvsc_probe(const struct if_nameindex *iface, + const struct ether_addr *eth_addr, + va_list ap) +{ + const char *name = va_arg(ap, const char *); + struct rte_kvargs *kvargs = va_arg(ap, struct rte_kvargs *); + unsigned int specified = va_arg(ap, unsigned int); + unsigned int *matched = va_arg(ap, unsigned int *); + unsigned int i; + struct vdev_netvsc_ctx *ctx; + int ret; + + /* Probe all interfaces when none are specified. */ + if (specified) { + for (i = 0; i != kvargs->count; ++i) { + const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + + if (!strcmp(pair->key, VDEV_NETVSC_ARG_IFACE)) { + if (!strcmp(pair->value, iface->if_name)) + break; + } else if (!strcmp(pair->key, VDEV_NETVSC_ARG_MAC)) { + struct ether_addr tmp; + + if (sscanf(pair->value, + "%" SCNx8 ":%" SCNx8 ":%" SCNx8 ":" + "%" SCNx8 ":%" SCNx8 ":%" SCNx8, + &tmp.addr_bytes[0], + &tmp.addr_bytes[1], + &tmp.addr_bytes[2], + &tmp.addr_bytes[3], + &tmp.addr_bytes[4], + &tmp.addr_bytes[5]) != 6) { + DRV_LOG(ERR, + "invalid MAC address format" + " \"%s\"", + pair->value); + return -EINVAL; + } + if (is_same_ether_addr(eth_addr, &tmp)) + break; + } + } + if (i == kvargs->count) + return 0; + ++(*matched); + } + /* Weed out interfaces already handled. */ + LIST_FOREACH(ctx, &vdev_netvsc_ctx_list, entry) + if (ctx->if_index == iface->if_index) + break; + if (ctx) { + if (!specified) + return 0; + DRV_LOG(WARNING, + "interface \"%s\" (index %u) is already handled," + " skipping", + iface->if_name, iface->if_index); + return 0; + } + if (!vdev_netvsc_iface_is_netvsc(iface)) { + if (!specified) + return 0; + DRV_LOG(WARNING, + "interface \"%s\" (index %u) is not NetVSC," + " skipping", + iface->if_name, iface->if_index); + return 0; + } + /* Create interface context. */ + ctx = calloc(1, sizeof(*ctx)); + if (!ctx) { + ret = -errno; + DRV_LOG(ERR, "cannot allocate context for interface \"%s\": %s", + iface->if_name, rte_strerror(errno)); + goto error; + } + ctx->id = vdev_netvsc_ctx_count; + strncpy(ctx->if_name, iface->if_name, sizeof(ctx->if_name)); + ctx->if_index = iface->if_index; + ctx->if_addr = *eth_addr; + ctx->pipe[0] = -1; + ctx->pipe[1] = -1; + ctx->yield[0] = '\0'; + if (pipe(ctx->pipe) == -1) { + ret = -errno; + DRV_LOG(ERR, + "cannot allocate control pipe for interface \"%s\": %s", + ctx->if_name, rte_strerror(errno)); + goto error; + } + for (i = 0; i != RTE_DIM(ctx->pipe); ++i) { + int flf = fcntl(ctx->pipe[i], F_GETFL); + + if (flf != -1 && + fcntl(ctx->pipe[i], F_SETFL, flf | O_NONBLOCK) != -1) + continue; + ret = -errno; + DRV_LOG(ERR, "cannot toggle non-blocking flag on control file" + " descriptor #%u (%d): %s", i, ctx->pipe[i], + rte_strerror(errno)); + goto error; + } + /* Generate virtual device name and arguments. */ + i = 0; + ret = snprintf(ctx->name, sizeof(ctx->name), "%s_id%u", + name, ctx->id); + if (ret == -1 || (size_t)ret >= sizeof(ctx->name)) + ++i; + ret = snprintf(ctx->devname, sizeof(ctx->devname), "net_failsafe_%s", + ctx->name); + if (ret == -1 || (size_t)ret >= sizeof(ctx->devname)) + ++i; + ret = snprintf(ctx->devargs, sizeof(ctx->devargs), + "fd(%d),dev(net_tap_%s,remote=%s)", + ctx->pipe[0], ctx->name, ctx->if_name); + if (ret == -1 || (size_t)ret >= sizeof(ctx->devargs)) + ++i; + if (i) { + ret = -ENOBUFS; + DRV_LOG(ERR, "generated virtual device name or argument list" + " too long for interface \"%s\"", ctx->if_name); + goto error; + } + /* Request virtual device generation. */ + DRV_LOG(DEBUG, "generating virtual device \"%s\" with arguments \"%s\"", + ctx->devname, ctx->devargs); + vdev_netvsc_foreach_iface(vdev_netvsc_device_probe, ctx); + ret = rte_eal_hotplug_add("vdev", ctx->devname, ctx->devargs); + if (ret) + goto error; + LIST_INSERT_HEAD(&vdev_netvsc_ctx_list, ctx, entry); + ++vdev_netvsc_ctx_count; + DRV_LOG(DEBUG, "added NetVSC interface \"%s\" to context list", + ctx->if_name); + return 0; +error: + if (ctx) + vdev_netvsc_ctx_destroy(ctx); + return ret; +} + +/** * Probe NetVSC interfaces. * + * This function probes system netdevices according to the specified device + * arguments and starts a periodic alarm callback to notify the resulting + * fail-safe PMD instances of their sub-devices whereabouts. + * * @param dev * Virtual device context for driver instance. * @@ -49,12 +557,40 @@ const char *args = rte_vdev_device_args(dev); struct rte_kvargs *kvargs = rte_kvargs_parse(args ? args : "", vdev_netvsc_arg); + unsigned int specified = 0; + unsigned int matched = 0; + unsigned int i; + int ret; DRV_LOG(DEBUG, "invoked as \"%s\", using arguments \"%s\"", name, args); if (!kvargs) { DRV_LOG(ERR, "cannot parse arguments list"); goto error; } + for (i = 0; i != kvargs->count; ++i) { + const struct rte_kvargs_pair *pair = &kvargs->pairs[i]; + + if (!strcmp(pair->key, VDEV_NETVSC_ARG_IFACE) || + !strcmp(pair->key, VDEV_NETVSC_ARG_MAC)) + ++specified; + } + rte_eal_alarm_cancel(vdev_netvsc_alarm, NULL); + /* Gather interfaces. */ + ret = vdev_netvsc_foreach_iface(vdev_netvsc_netvsc_probe, name, kvargs, + specified, &matched); + if (ret < 0) + goto error; + if (matched < specified) + DRV_LOG(WARNING, + "some of the specified parameters did not match" + " recognized network interfaces"); + ret = rte_eal_alarm_set(VDEV_NETVSC_PROBE_MS * 1000, + vdev_netvsc_alarm, NULL); + if (ret < 0) { + DRV_LOG(ERR, "unable to schedule alarm callback: %s", + rte_strerror(-ret)); + goto error; + } error: if (kvargs) rte_kvargs_free(kvargs); @@ -65,6 +601,9 @@ /** * Remove driver instance. * + * The alarm callback and underlying vdev_netvsc context instances are only + * destroyed after the last PMD instance is removed. + * * @param dev * Virtual device context for driver instance. * @@ -74,7 +613,16 @@ static int vdev_netvsc_vdev_remove(__rte_unused struct rte_vdev_device *dev) { - --vdev_netvsc_ctx_inst; + if (--vdev_netvsc_ctx_inst) + return 0; + rte_eal_alarm_cancel(vdev_netvsc_alarm, NULL); + while (!LIST_EMPTY(&vdev_netvsc_ctx_list)) { + struct vdev_netvsc_ctx *ctx = LIST_FIRST(&vdev_netvsc_ctx_list); + + LIST_REMOVE(ctx, entry); + --vdev_netvsc_ctx_count; + vdev_netvsc_ctx_destroy(ctx); + } return 0; } -- 1.8.3.1