From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20062.outbound.protection.outlook.com [40.107.2.62]) by dpdk.org (Postfix) with ESMTP id 9FB8C56A1 for ; Sun, 13 Aug 2017 14:25:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=6Jf9sqjTGnyYsfUIjEP/SJK4PlSbqKvgQCCBwL4MmrI=; b=qG+htBfZfe5ePWhunny7i+QzmlkOSY9Ku/nVcV6Xx2FqdqvtoT9DqEqEAYqABID02oOYlbzgZG8XhPlpaV/8NMYk+z2Ft/mAAXzd4xnlOx6j4Kdh0iUMGCBzfh3p53VF2bFdh3IaqejcJye2Ogf+RgieTj5119neNcbH3szSHVQ= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; Received: from mellanox.com (37.142.13.130) by DB6PR0502MB3047.eurprd05.prod.outlook.com (2603:10a6:4:9f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1341.17; Sun, 13 Aug 2017 12:25:57 +0000 From: Matan Azrad To: Adrien Mazarguil , Nelio Laranjeiro Cc: dev@dpdk.org Date: Sun, 13 Aug 2017 15:25:11 +0300 Message-Id: <1502627112-53405-1-git-send-email-matan@mellanox.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [37.142.13.130] X-ClientProxiedBy: VI1PR08CA0114.eurprd08.prod.outlook.com (2603:10a6:800:d4::16) To DB6PR0502MB3047.eurprd05.prod.outlook.com (2603:10a6:4:9f::7) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 595908a1-faf8-4c86-d3f0-08d4e246737f X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(300000502095)(300135100095)(22001)(2017030254152)(48565401081)(300000503095)(300135400095)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:DB6PR0502MB3047; X-Microsoft-Exchange-Diagnostics: 1; DB6PR0502MB3047; 3:maz3kkjKB7FxwFd785bCTcmuiIeg4lqLaLlNlQYo7onPWwp+N3OLefVebeSFnbpavxsr6TMzdIZD7pC+lE5p04K9eto5++l2NYNIrdYJoA5kx88auGnm50l3dorq66/kdswRQXuuqXxUGEmGXoTxkg3fRJiqTLITVJsZq3jUUc1YcEQmVIALafG6gcrgCJappn6CPuR3pHx6H/OIyo3+JfbHf3jfXlrDfBPTcDIWOQWAWij5n8sL+COWxRu9dwCx; 25:ZzmJ66cfcL8VcL3jJ/Xvphjv732CBKMMPBXQvoV+VIY3bypHEyQyE9sS0Ni+UaCRMfVLv09PtZKXl9rfNnJSJHTT/wG721b0e/3HCHDG2OEremqafuEap8LV88iFkqV/i09Vkk0FSZiH1g/jcB36aM6xNcRCCLQgRB3SsBogUiu8hWR4CidfbixqdpG/AVXEJ58+fofOMcbyKWzulDBs0qnB1F5pK5L2wNaDqKPvO7Ynp0Com10HavBhsPF2m6SZBTiv2/MYZhx+3rwp38BuyRIOYf6pz4oir0Jp7fEhUy0oX+iMASWq8Z0v0XBCGOrU9cwsUdpExiuQ/JmYLBVJ6w==; 31:ck4Dn/J1dRRwKrXTfBRRFVvxsGCwgv4WcpOuWrOe/5WZwFJYimMWCaiV+fMHhz2JKpAFDv1dfKOGSzd6RHZgUNa2fdM1YhzWbbTd3kD8+xQDrVrB/N3xoEVot5xsvE1E/wF3lco6Cb9iFM6weJyWSjY4W0Z4Cr9IjB1WsAn9JzT4/iKKEdd1tWtHIMLfKRRyvZ/EfIMJ8/rrYGXiXqvPlNYpmfVdJx0CDIV13nR5zqA= X-MS-TrafficTypeDiagnostic: DB6PR0502MB3047: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; DB6PR0502MB3047; 20:7TcVkuHURSeiwx+49EqTepDzbXUv7Md2H2R04kGmzfFu7ZEWRCK3P4GTErBMaaslRSP9rQzl3DZSJmB7s+EsrS66++rf9kxABMAihxJ4V36Bjbbt7AvMmnw8CR//odCdHgUoWiOhg2VtIyr/DBb9ZFmEIovrKSUT8+US6fD+vYOQ8NFBH8PbQOhRm8QuTqOH8rA56OQAYv1Pu2YRIo9PPuw23+j4vVntbnTGjcuxesPgUvjDDATLeS8SZQUx54jrEb/bSRRe9/T5PpQiX7yEC27eSWFP2MI9SX7GQZeYp4S8YuBTWEvHqIZ3Bcl74wlMoQfPgPRBrzhQLily/aMshIz0f0XCZisiD+SIwoM9sPK8NMDbJfMtbT3jZF4HgLlBxpEG/F4XcXcT7F+OMbhR9fCIT1zMVClliaFUgmNOYso8EvxE66l4kXN/uIo4TfwdBqe/9aLGPrkmF1jbb1eUjeeSOElh++xUegjQcLg9KtqdPKI5lltgwI2c0Nx7snk0; 4:4ft7GoNAZXPSOkmI894OuAG14dzzAzTo+6Ot/9nC2mcTm13PawZeJWbToyA/AZ6dIU1dOYtjgVABa1AlIUb2ttyLr/aZHQ3/78YLSZBRhVSdTb4AfIvt06cQyhUOV7N8RnHdNzksC8Byjigo0tM8EmQNNc4YbrkVRH7KTy3T9uSsoS+3J78fnh2uQf9nqY0j3ONYFh8dUbbSkPdLScmN3fPXCvDP2xRX70KdCTlw7BDEmv/wkYlSZV9JgMAQ9KUs X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(3002001)(6055026)(6041248)(20161123564025)(20161123560025)(20161123558100)(20161123562025)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DB6PR0502MB3047; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DB6PR0502MB3047; X-Forefront-PRVS: 03982FDC1D X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(7370300001)(4630300001)(6009001)(39860400002)(199003)(189002)(6116002)(3846002)(68736007)(5660300001)(50986999)(47776003)(36756003)(7350300001)(101416001)(6666003)(53936002)(21086003)(33646002)(55016002)(25786009)(478600001)(105586002)(106356001)(33026002)(97736004)(4326008)(42186005)(189998001)(69596002)(81166006)(81156014)(50226002)(86362001)(8676002)(48376002)(305945005)(5003940100001)(50466002)(2906002)(66066001)(7736002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0502MB3047; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DB6PR0502MB3047; 23:EX23b9rrh4fWcKoK1p6vqr2Sxxhl6pdSvIgYxpB?= =?us-ascii?Q?bd8ajt2y7lNvUJDRKNFPm6vyzAOq+3EvHuuZ03+fERLT1p5sm/q/IfGLOlPj?= =?us-ascii?Q?QgDO4imOeNyNvUTx2YGb1ueaohGYfdyWR3azmIBM0aPSSfBK+qeed+f5xRqu?= =?us-ascii?Q?D8hLx6+Tc8ixjW6ASYBB/hkoe7XJAMfa5RRB6jeb3g9dtLtbdRckkGst41bh?= =?us-ascii?Q?LysM2/JkR16nmXzdmUDPvy9HzsRjqI4JjPDCrJd/7vMbKPJmbCAY0E09qxIc?= =?us-ascii?Q?0hpRot+e2X1oKztz95Pzei+7UJm8jkOsAVs4iyT0aPUPR2uCtitj/mhcb2+z?= =?us-ascii?Q?2ZI6ynrErngerW/Wrn6CJ/3TXZaSuami94MiNOLA3N+kkp1n5vFbsPCkozRW?= =?us-ascii?Q?EpjkqOn/dpdBs3QZxfscIYc2O/i8+3NzZDPXbPY/J0IWzA/zBQ9j8E9+aPjj?= =?us-ascii?Q?RzKPlxSIZ68Q5/FjxWhLAV2Dm0CreSXIaPvO0XfdSwkXltATxxYJkqXiSbG8?= =?us-ascii?Q?xJlzo4muWXqpnLTTVVamRj+4NIqdf+2LcMcYgS5VAGH7/Wuy9rw7I5Orklld?= =?us-ascii?Q?7K7L5vH0bIJpWSyZ02Zbya2xHevSgieVxlteu6U01uNCFjMweOFaKmglyqI1?= =?us-ascii?Q?CuU5zBsgpFY3FVfixmYcS1sO3cu3+oDQOMfnRmEGZKAF9AI/74faY87jKtCc?= =?us-ascii?Q?/mxqD8k+G/2rHkzb3afBIdA6xsrdVMGLfIOPavXp+f/a3fSYegv8CFfFHc9P?= =?us-ascii?Q?9qZWA4akZMzlgVpJxMm1Hhq90XSLW3Lo5t6A29huYkriyqpBnVZrMJ4hGNJw?= =?us-ascii?Q?NnJhXAsggxdWuj1NzTqRw2yaMyh7bvyALOPKbVDcgrw3qRphamLt1rhcvosj?= =?us-ascii?Q?9cGEIO0wC/U2/fQLvV5S4UHMhqXX3VsariT1eep3u2nCwfcjP8oxf0VXGjAs?= =?us-ascii?Q?Cve7/1fvGZmoFaSNBy7/s8/JsL7WipLv+oqUlZjuO7421zCWvJm8ixikUoPX?= =?us-ascii?Q?jWs7O2rMKT58+nYlRx+92LylO7WhGrRZZKDpoT67Lsgi2zKrNphSzijDQQNm?= =?us-ascii?Q?JbG1ySEA=3D?= X-Microsoft-Exchange-Diagnostics: 1; DB6PR0502MB3047; 6:YRigJX00q0tq/hJLi4WcgrrbvGcb5YVh33/G/mLjv0FRHJ9pCG7W3luz6Ei44AgkeJrdAsdaaps9JBmBv8YNaCiB0GtkZDVDR/u8Z6ER22vRomtDkhOzm5bZcF3fI0VaJs80okg8X/U6fcVSGrA4LnSyN/WQO7/xu5Y/YtyL3aKIOv47xpZjitwxcNlNq8sAJs2M1kuMp4HhEJbejaEiCJ3khAIDqUA+hf7MMZOzYehu33+F+c28Pq2oI5qIuACq4HtpJ9x27SNFfTLCXGPvxHmFScMxFAGH0wZkUsWwsWWMT2dMCYKJmuQlzI+D0HXo7T4xTF+9/E2DWSOwNdkOIw==; 5:XVDMKjqmWXXI1g02YemIgrGKt8Ot9zwXMYV8jP4bUfa88cBOeq8qiCtZAbzRfwNUMmhWk1vA976kZCV7K5rowZYJu4L3WboB7N6+xGuJR6qqaBRmDff6NxqCsiE6MLfgffo1GMjP4EkrovCZgKWc7w==; 24:g1RKGIyM52aztsDyAJEhB3LysZGJYr0P4+k3CHAgFN8tfwCeOfnEV7CUH4A0MEw5USswf6jre4NWw+mm2ftW1Nw+frFZ5GpR0FomHmnEDEQ=; 7:Jwr22j6gaIovcjqDCP5iOpzOnYVrNORDSTwAcIkzXp0tlc5NLMmeSVvi9wXaTJcQpLl2CzbEDWVmzEaG3jjOqTonxe+3Z6U/xN+YiWXnlCpKZjZlLzViF9C24xFcsg2qUJ/E7qAJfmNSeW7FHdVgAJ9/FPRoOdQq9mpn5c5awj0Tz0kwpsNMvkeU8uSusyCAQfy10u9C4PhtYX59UxtbdLZQ8pnCYHmszoUCOb/DXSc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2017 12:25:57.1615 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0502MB3047 Subject: [dpdk-dev] [PATCH 1/2] net/mlx5: support device removal event X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Aug 2017 12:25:59 -0000 Extend the LSC event handling to support the device removal as well. The Verbs library may send several related events, which are different from LSC event. The mlx5 event handling has been made capable of receiving and signaling several event types at once. This support includes next: 1. Removal event detection according to the user configuration. 2. Calling to all registered mlx5 removal callbacks. 3. Capabilities extension to include removal interrupt handling. Signed-off-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 2 +- drivers/net/mlx5/mlx5_ethdev.c | 100 +++++++++++++++++++++++++++-------------- 2 files changed, 68 insertions(+), 34 deletions(-) Hi This patch based on top of last Nelio mlx5 cleanup patches. diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index bd66a7c..1a3d7f1 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -865,7 +865,7 @@ static struct rte_pci_driver mlx5_driver = { }, .id_table = mlx5_pci_id_map, .probe = mlx5_pci_probe, - .drv_flags = RTE_PCI_DRV_INTR_LSC, + .drv_flags = RTE_PCI_DRV_INTR_LSC | RTE_PCI_DRV_INTR_RMV, }; /** diff --git a/drivers/net/mlx5/mlx5_ethdev.c b/drivers/net/mlx5/mlx5_ethdev.c index 57f6237..404d8f4 100644 --- a/drivers/net/mlx5/mlx5_ethdev.c +++ b/drivers/net/mlx5/mlx5_ethdev.c @@ -1112,47 +1112,75 @@ mlx5_ibv_device_to_pci_addr(const struct ibv_device *device, } /** - * Link status handler. + * Update the link status. + * Set alarm if the device link status is inconsistent. * * @param priv * Pointer to private structure. - * @param dev - * Pointer to the rte_eth_dev structure. * * @return - * Nonzero if the callback process can be called immediately. + * Zero if alarm is not set and the link status is consistent. */ static int -priv_dev_link_status_handler(struct priv *priv, struct rte_eth_dev *dev) +priv_link_status_alarm_update(struct priv *priv) +{ + struct rte_eth_link *link = &priv->dev->data->dev_link; + + mlx5_link_update(priv->dev, 0); + if (((link->link_speed == 0) && link->link_status) || + ((link->link_speed != 0) && !link->link_status)) { + if (!priv->pending_alarm) { + /* Inconsistent status, check again later. */ + priv->pending_alarm = 1; + rte_eal_alarm_set(MLX5_ALARM_TIMEOUT_US, + mlx5_dev_link_status_handler, + priv->dev); + } + return 1; + } else if (unlikely(priv->pending_alarm)) { + /* In case of link interrupt while link alarm was setting. */ + priv->pending_alarm = 0; + rte_eal_alarm_cancel(mlx5_dev_link_status_handler, priv->dev); + } + return 0; +} + +/** + * Device status handler. + * + * @param priv + * Pointer to private structure. + * @param events + * Pointer to event flags holder. + * + * @return + * Events bitmap of callback process which can be called immediately. + */ +static uint32_t +priv_dev_status_handler(struct priv *priv) { struct ibv_async_event event; - struct rte_eth_link *link = &dev->data->dev_link; - int ret = 0; + uint32_t ret = 0; /* Read all message and acknowledge them. */ for (;;) { if (ibv_get_async_event(priv->ctx, &event)) break; - - if (event.event_type != IBV_EVENT_PORT_ACTIVE && - event.event_type != IBV_EVENT_PORT_ERR) + if ((event.event_type == IBV_EVENT_PORT_ACTIVE || + event.event_type == IBV_EVENT_PORT_ERR) && + (priv->dev->data->dev_conf.intr_conf.lsc == 1)) + ret |= (1 << RTE_ETH_EVENT_INTR_LSC); + else if (event.event_type == IBV_EVENT_DEVICE_FATAL && + priv->dev->data->dev_conf.intr_conf.rmv == 1) + ret |= (1 << RTE_ETH_EVENT_INTR_RMV); + else DEBUG("event type %d on port %d not handled", - event.event_type, event.element.port_num); + event.event_type, event.element.port_num); ibv_ack_async_event(&event); } - mlx5_link_update(dev, 0); - if (((link->link_speed == 0) && link->link_status) || - ((link->link_speed != 0) && !link->link_status)) { - if (!priv->pending_alarm) { - /* Inconsistent status, check again later. */ - priv->pending_alarm = 1; - rte_eal_alarm_set(MLX5_ALARM_TIMEOUT_US, - mlx5_dev_link_status_handler, - dev); - } - } else { - ret = 1; - } + if (ret & (1 << RTE_ETH_EVENT_INTR_LSC)) + if (priv_link_status_alarm_update(priv)) + ret &= ~(1 << RTE_ETH_EVENT_INTR_LSC); return ret; } @@ -1172,11 +1200,11 @@ mlx5_dev_link_status_handler(void *arg) priv_lock(priv); assert(priv->pending_alarm == 1); priv->pending_alarm = 0; - ret = priv_dev_link_status_handler(priv, dev); + ret = priv_link_status_alarm_update(priv); priv_unlock(priv); - if (ret) + if (!ret) _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, - NULL); + NULL); } /** @@ -1192,14 +1220,17 @@ mlx5_dev_interrupt_handler(void *cb_arg) { struct rte_eth_dev *dev = cb_arg; struct priv *priv = dev->data->dev_private; - int ret; + uint32_t events; priv_lock(priv); - ret = priv_dev_link_status_handler(priv, dev); + events = priv_dev_status_handler(priv); priv_unlock(priv); - if (ret) + if (events & (1 << RTE_ETH_EVENT_INTR_LSC)) _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, - NULL); + NULL); + if (events & (1 << RTE_ETH_EVENT_INTR_RMV)) + _rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RMV, NULL, + NULL); } /** @@ -1213,7 +1244,8 @@ mlx5_dev_interrupt_handler(void *cb_arg) void priv_dev_interrupt_handler_uninstall(struct priv *priv, struct rte_eth_dev *dev) { - if (!dev->data->dev_conf.intr_conf.lsc) + if (!dev->data->dev_conf.intr_conf.lsc && + !dev->data->dev_conf.intr_conf.rmv) return; rte_intr_callback_unregister(&priv->intr_handle, mlx5_dev_interrupt_handler, @@ -1238,7 +1270,8 @@ priv_dev_interrupt_handler_install(struct priv *priv, struct rte_eth_dev *dev) { int rc, flags; - if (!dev->data->dev_conf.intr_conf.lsc) + if (!dev->data->dev_conf.intr_conf.lsc && + !dev->data->dev_conf.intr_conf.rmv) return; assert(priv->ctx->async_fd > 0); flags = fcntl(priv->ctx->async_fd, F_GETFL); @@ -1246,6 +1279,7 @@ priv_dev_interrupt_handler_install(struct priv *priv, struct rte_eth_dev *dev) if (rc < 0) { INFO("failed to change file descriptor async event queue"); dev->data->dev_conf.intr_conf.lsc = 0; + dev->data->dev_conf.intr_conf.rmv = 0; } else { priv->intr_handle.fd = priv->ctx->async_fd; priv->intr_handle.type = RTE_INTR_HANDLE_EXT; -- 2.7.4