From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <matan@mellanox.com>
Received: from EUR03-DB5-obe.outbound.protection.outlook.com
 (mail-eopbgr40070.outbound.protection.outlook.com [40.107.4.70])
 by dpdk.org (Postfix) with ESMTP id 71D4D1B607;
 Thu,  2 Nov 2017 16:42:44 +0100 (CET)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com;
 s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version;
 bh=X8qcFZLiBr9hdWrc5suVkS78hvJKlTu9FvfVg7+Y7Tk=;
 b=vzOA+u0JKv5a9BT4zPbNQoYj1xDDewGnmu2I8RLsSgYTu/3UkcQH6eXj2d2T8Gw4P7nvYFd78cTH0dwp+qqZH3bghAy2ZGTEx/EuEkE+o7TWZ4aZGqK4gJlvYC9GZYl5X2qSxZeKilzH4BSyR7GegQaBCcRGPtHBb3rRRGuV8Wg=
Authentication-Results: spf=none (sender IP is )
 smtp.mailfrom=matan@mellanox.com; 
Received: from mellanox.com (37.142.13.130) by
 VI1PR0502MB3664.eurprd05.prod.outlook.com (2603:10a6:803:f::23) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.178.6; Thu, 2 Nov
 2017 15:42:42 +0000
From: Matan Azrad <matan@mellanox.com>
To: Adrien Mazarguil <adrien.mazarguil@6wind.com>,
 Gaetan Rivet <gaetan.rivet@6wind.com>
Cc: dev@dpdk.org,
	stable@dpdk.org
Date: Thu,  2 Nov 2017 15:42:02 +0000
Message-Id: <1509637324-13525-2-git-send-email-matan@mellanox.com>
X-Mailer: git-send-email 1.8.3.1
In-Reply-To: <1509637324-13525-1-git-send-email-matan@mellanox.com>
References: <1509637324-13525-1-git-send-email-matan@mellanox.com>
MIME-Version: 1.0
Content-Type: text/plain
X-Originating-IP: [37.142.13.130]
X-ClientProxiedBy: DB6PR0202CA0021.eurprd02.prod.outlook.com
 (2603:10a6:4:29::31) To VI1PR0502MB3664.eurprd05.prod.outlook.com
 (2603:10a6:803:f::23)
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 73b4c78a-8150-439d-6ee6-08d522085b6c
X-MS-Office365-Filtering-HT: Tenant
X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0;
 RULEID:(22001)(48565401081)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603199);
 SRVR:VI1PR0502MB3664; 
X-Microsoft-Exchange-Diagnostics: 1; VI1PR0502MB3664;
 3:ZBP2zpNpm2cByEuUQt/+zEEkdmyMWjTRtXMgJncZp6USBXZ0tcZOVcPLRg4lDmeZ/T7zXlD/X2umkADgP/tXiE6dZDnCDXK/TPryKoHC3M8NxOBjq/PcMzHfe0LgVtI6CwpR2V1VkfCTVJg9LVdPy0CPrYAzzj3lW+ogTDYwelbFPafQ5DEWmFgNBWwv4XXVgb3WQ8dVtp0oCCQPzivj8CR+DiZQYBOgq8Ep4vKPjTJEWsLBKcame7qqEu3ToTgG;
 25:KQgzDu/oFZEEnkLvyJFfVxtQbubzooSL/xrY7SOSM3V0nk0GFyV57m8vDZPw+mSOIYr6zB42znG+Qqb4w3ibncm624f36a+87D7zoBBQtMKTor+WrzIbZ2bTyYO0/6eJdxcr7eU8zu1uA87GXNsmQsbLgxgq61uhNF01xU8COe4Ev5OpHs7p3PNdVxuwNfIcEunB0uDi8Vh720hnLaTFazsd1G1XzwWeNEYFDYv2raSTmg+HEvHHnfiGcZPja4XlTKFPHD4iW/i3RIs7zjFBVKkj2bfDOQAk9uvQFUBAy/TpPbZ4VLnYG3XR+v/a1Gt+MMriqFYP93+8JMWTSYAlPQ==;
 31:kEqn1ZQ2DgNPGV9A44GQyOs5Qmx91FuEw5gLAJk74E6May4yJaemQB80t+aExZBDfFrumt7lOs23ODen4pEuhqGrR0Zazus2IxyaoxQRurS7YCwEVprJS3LMJdG3yPRwjifZZqXSwWuBCeFKN0vm/JWPsXXWtf3d9Wd8QTW+ISLsU7n3C15sYgastM+trZi0RQeVaPtXbx0SkWdV7kC4PbvHMBpMt/NoHRP5+V+3zJQ=
X-MS-TrafficTypeDiagnostic: VI1PR0502MB3664:
X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr
X-Microsoft-Exchange-Diagnostics: 1; VI1PR0502MB3664;
 20:q4Gax4VGO54UkV3p2NBppP3qQrz3gWgOibFHiOakKAcNV2PDQc8auJiXv9tzxvjtANsz//s+IX0fhAtY2u4T4POU5kNwcDg4VEUcKZWdwVywzvRI6CZ0Zbw3mJ78mp2MP8+3Ps2TYTF1Mldvq1NaKCw3WbQ0UVtB+GX18BBWktzggImIAes2OKG1vg0FYD9br8VhJMMFqQbeCS9LehDo9fOjqAnLOiNLVOQ2cNvfan2KlpuHifww0xFOGVa82FXdWHnoGZAcg1cRFEAPjix4aFeet26tg+re/ZLAqtNkbjkftcx3ClLyUoiqd97D2IeksGSbNNGFi9d4Vc9pObtk2W/q0+AcFNLWvtnPcuNF1+mv4QBaiJLSdRHKPce5j/Mddh82xtedF2mZI+OVIFushRCK6/44ZkwjC8EBzS3gA15sqVF+fwKC6hTR5oNANNLWCPiNCwwuHova/tyiYvocrFvA66JiOyEis+itDJRftjNJ4EbBNSkfBrIgY61p9O/Q;
 4:9So1tbeHHYP3xxC0Qxic+TD1h7akLIAOW6iXEeMuoRQh+EhL+xO4nd3WtmabSIWvOphhyqHcYfQ7oVPnyhNiA1vCYN+k96yaru4TIZEBp4F5+7UT4MoIgaFxSJ5O62LxBJ7YILwW4pyWJxb92e6zLxWAOZeJZ/lWriNVjNxPYJFr2o0oOrCsN2IXwU26c8imMNRPV2XHLq5Y3clHUy2epnZqn4n9lOZl9vDbQtjM3sAsrGS2CEs0i34ZYqimJJIGv0U5wv74kqRrLxwR4g8JsbBb5awcI9m2AirvHlRRIOXXawkTlyvHtm98u9SAqhGF
X-Exchange-Antispam-Report-Test: UriScan:(278428928389397);
X-Microsoft-Antispam-PRVS: <VI1PR0502MB3664FDB83F6B2E3B71A33E00D25C0@VI1PR0502MB3664.eurprd05.prod.outlook.com>
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0;
 RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3002001)(100000703101)(100105400095)(3231020)(6055026)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123555025)(20161123562025)(20161123558100)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);
 SRVR:VI1PR0502MB3664; BCL:0; PCL:0;
 RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);
 SRVR:VI1PR0502MB3664; 
X-Forefront-PRVS: 047999FF16
X-Forefront-Antispam-Report: SFV:NSPM;
 SFS:(10009020)(6009001)(376002)(346002)(39860400002)(199003)(189002)(8936002)(48376002)(50466002)(2906002)(81166006)(81156014)(8676002)(3846002)(6116002)(106356001)(69596002)(105586002)(33646002)(2950100002)(6666003)(4720700003)(76176999)(50986999)(5660300001)(101416001)(5003940100001)(189998001)(7736002)(36756003)(53936002)(97736004)(25786009)(316002)(33026002)(16586007)(86362001)(47776003)(4326008)(66066001)(16526018)(50226002)(478600001)(68736007)(110136005)(55016002)(21086003)(305945005);
 DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0502MB3664; H:mellanox.com; FPR:; SPF:None;
 PTR:InfoNoRecords; A:1; MX:1; LANG:en; 
Received-SPF: None (protection.outlook.com: mellanox.com does not designate
 permitted sender hosts)
X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; VI1PR0502MB3664;
 23:kN3ee64Uj+wNT6m7kjICb665++IZz4/7vx7XW2J?=
 =?us-ascii?Q?6P4SrYwvLiiJ3whVBysMm5OyLZJAET61QUH0iMuamnnKsr1hb7QKDNjToO3g?=
 =?us-ascii?Q?XHXspzdB/UNcKEMKjwP2RQYjOJq0R9QxGFWbLv186S+bMcwYsBonoFycDbvl?=
 =?us-ascii?Q?vFUwk75pO9mG2g1EICfv9OfUnQnLaGPDKpkNllrzQ3fiE9I/eU0vcg8MAL4v?=
 =?us-ascii?Q?SpAgdhEdIPLYZkRL5k0rQg5OZR+woc8q0T84NKDh34Zi+q2KmnHiKHdyf/Dm?=
 =?us-ascii?Q?VMe45S7Kv7J0OidOoCtUt1i7Cz9m7MvsUXUT55k8xA8zy8VEsX5bHSUf2mPG?=
 =?us-ascii?Q?/UqEG6Si84MsEUHRIBcKrSWNIlPxGMTUaAWzQCRlRlTw8GplZFD2SDISwV5N?=
 =?us-ascii?Q?rPTQ/ZovrdUOrit/kjD+GLqvhlMDWq4oQP1C4J/3cp9aUJgkpzQjdstqF+hO?=
 =?us-ascii?Q?NY+K6xPeiBdP7u1Fab0T3M1X20uZg7sGaeQIzFL+/gM2GJo9EE+PyAcd7SVX?=
 =?us-ascii?Q?CYOkiJy9DM6+CyIVLQz2EAn+UdwTAS+I7iAYucUdGoOr1IDWiHLBVvB1HPL4?=
 =?us-ascii?Q?eQAaIa9gN6bdntezD3ywK0hvqH8IudP8TXfflHwsCU0Wf7HEw0WxmDj4qQC4?=
 =?us-ascii?Q?BkRA0aUngQVLSvF/SzVh4ps64ric5+9c5Cl/kGzFFDlNYQTPa0WNnjaBdO2t?=
 =?us-ascii?Q?jnl92mSn0VTJHKlbs5+BlQFoXpenV5YGLrthb+da69Zbqf9XMVB7GS3a2Ix9?=
 =?us-ascii?Q?RGM9Tzpe+XiOXhFKyhyZNu/FoSokROIROwP6nqxBIBRddaUC07qmx15k58Kv?=
 =?us-ascii?Q?NvA7psFyz36DI43duyAfq20CrwPw7DRUceTznMfRdl+p9rWuYndsOMCzMLrg?=
 =?us-ascii?Q?UJhggGzJJJyewWqWThp1ihBCeNlIwDZ9LUFIhOVVnvstLMndWCWKv4lxTAF7?=
 =?us-ascii?Q?Pat2+1c/2/UD7GmF1rds3ZRScG4TE6Y6H6zaiG0FK6TapBRrDurEgphxM3+Q?=
 =?us-ascii?Q?Ywf1fzHFj2/6AnCe0EpjWxlFvHNkleR3uP4mHlBrHHfzprNFiBCuFWLg1jc5?=
 =?us-ascii?Q?nmQxfPTdlMBTxl3/rxjhcltyaM3+oSRg1soaWrK10XzwkMMjN189h9p1m3Hq?=
 =?us-ascii?Q?nagQBmusINIgdxMQIcTOBTIqCrygYsI8e?=
X-Microsoft-Exchange-Diagnostics: 1; VI1PR0502MB3664;
 6:NtCh8702V9eQ831er7+LVndKa5+/DgwJvxcmal4XpNcPuIAjSsJFAZmuAUk/LBQTNd/b5nQoSP18s4cXN50u24BWOM9XfAkBV04JXqYgUeRbo1qRTy/VtRJt7BsvKbByUNo2IHgo6XJaxi2eV6u56F5p0H8yvLnMDc4JAlwXzQqid92z+d6yASu7qjzAieLDvFr/KyPzafPjqZ4RWn/g8ABR0lAKVIxvGjd4TmvLExri9H5c7ufaY05gvQ8eFagol+OmOH1R8g8VpkfajOZri96xCMEoyzYWoIbe31aLmjiVoWagR2iPNi2vetdmgeqyJVNlemuRryxub8RsQ4PNS1LGXFjRCwbiGb7C3rbrWgw=;
 5:veL1CqAEnmfrTHVKza/vCYqHytxluzf2oRz0Z4WBoXQc/+7M7bfgpp7gC8SARlkY/qskRxWGsFZP+1Ym+XOOo/B6qL3l8YWy50wRDHbJ1Qm3yQGa+W7PDEQvftiIeS2E7FK1pD0Mak+xuDK64LJ4sSW3g7gWx8byG4lr7n3Qs0U=;
 24:Lv8D9/vElVlOmEYj5tXPEPNkyXI3BoY32DxyxaFwPfnEqNz/1AaUH7+o+LdpsLqv8WYeEeCL3cwA0UzuiBMQSXEGGhxZN5wswj2qW66u8GU=;
 7:sB9NFFKfqAJj0pVq5GdEmVMHx5SswGKDJBgoKFfVssI8u5yE5FRwnUTz1S9dmStI3g0+pEPHsvOtq8ah/bjVqrEVajoCX+Mrkb5Qj7u4+NasScuGqy01KT+EXMWNII5R9M6ww3qj/VVr3H9pQVqIhprGZYhWrmmqs5iRFcYWevsHlF3QjAWbjgjBjXdCbzJRUUSv2wlfdgPW4Z1oVGZ0hpbEWn4ezInLS7/7IVptJMloX69/9CM1pbCuZCeksD5v
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-OriginatorOrg: Mellanox.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Nov 2017 15:42:42.2333 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 73b4c78a-8150-439d-6ee6-08d522085b6c
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0502MB3664
Subject: [dpdk-stable] [PATCH 1/3] net/failsafe: fix removal handling lack
X-BeenThere: stable@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches for DPDK stable branches <stable.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/stable>,
 <mailto:stable-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/stable/>
List-Post: <mailto:stable@dpdk.org>
List-Help: <mailto:stable-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/stable>,
 <mailto:stable-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Thu, 02 Nov 2017 15:42:45 -0000

There is time between the physical removal of the device until
sub-device PMDs get a RMV interrupt. At this time DPDK PMDs and
applications still don't know about the removal and may call sub-device
control operation which should return an error.

In previous code this error is reported to the application contrary to
fail-safe principle that the app should not be aware of device removal.

Define a removal error that each sub-device PMD should return in case
of an error caused by removal event; The special error is -ENODEV.

Add an error check in each relevant control command error flow and
prevent an error report to application when its value is -ENODEV.

Fixes: a46f8d5 ("net/failsafe: add fail-safe PMD")
Fixes: b737a1e ("net/failsafe: support flow API")
Cc: stable@dpdk.org

Signed-off-by: Matan Azrad <matan@mellanox.com>
---
 doc/guides/nics/fail_safe.rst                   |  7 +++++++
 doc/guides/prog_guide/env_abstraction_layer.rst |  3 +++
 drivers/net/failsafe/failsafe_flow.c            | 16 +++++++++------
 drivers/net/failsafe/failsafe_ops.c             | 27 ++++++++++++++++---------
 drivers/net/failsafe/failsafe_private.h         |  8 ++++++++
 5 files changed, 45 insertions(+), 16 deletions(-)

diff --git a/doc/guides/nics/fail_safe.rst b/doc/guides/nics/fail_safe.rst
index c4e3d2e..5023fc4 100644
--- a/doc/guides/nics/fail_safe.rst
+++ b/doc/guides/nics/fail_safe.rst
@@ -193,6 +193,13 @@ any time. The fail-safe PMD will register a callback for such event and react
 accordingly. It will try to safely stop, close and uninit the sub-device having
 emitted this event, allowing it to free its eventual resources.
 
+When fail-safe PMD gets -ENODEV error from control command sent to removable
+sub-devices, it assumes that the error reason is device removal. In this case
+fail-safe returns success value to application. The PMD controlling the
+sub-device is still responsible to emit a removal event (RMV) in addition to
+returning -ENODEV from control operations after the device has been physically
+removed. Only the reception of this event unregisters it on the fail-safe side.
+
 Fail-safe glossary
 ------------------
 
diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst
index 4775eb3..bd2fd87 100644
--- a/doc/guides/prog_guide/env_abstraction_layer.rst
+++ b/doc/guides/prog_guide/env_abstraction_layer.rst
@@ -213,6 +213,9 @@ device having emitted a Device Removal Event. In such case, calling
 callback. Care must be taken not to close the device from the interrupt handler
 context. It is necessary to reschedule such closing operation.
 
+Unsuccessful control operations (for those that return errors) may return
+-ENODEV after the device is physically unplugged.
+
 Blacklisting
 ~~~~~~~~~~~~
 
diff --git a/drivers/net/failsafe/failsafe_flow.c b/drivers/net/failsafe/failsafe_flow.c
index 153ceee..ce9b769 100644
--- a/drivers/net/failsafe/failsafe_flow.c
+++ b/drivers/net/failsafe/failsafe_flow.c
@@ -87,7 +87,7 @@
 		DEBUG("Calling rte_flow_validate on sub_device %d", i);
 		ret = rte_flow_validate(PORT_ID(sdev),
 				attr, patterns, actions, error);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_flow_validate failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
@@ -111,7 +111,8 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		flow->flows[i] = rte_flow_create(PORT_ID(sdev),
 				attr, patterns, actions, error);
-		if (flow->flows[i] == NULL) {
+		if (flow->flows[i] == NULL &&
+			!SUBDEV_REMOVED(sdev, -rte_errno)) {
 			ERROR("Failed to create flow on sub_device %d",
 				i);
 			goto err;
@@ -150,7 +151,7 @@
 			continue;
 		local_ret = rte_flow_destroy(PORT_ID(sdev),
 				flow->flows[i], error);
-		if (local_ret) {
+		if (local_ret && !SUBDEV_REMOVED(sdev, local_ret)) {
 			ERROR("Failed to destroy flow on sub_device %d: %d",
 					i, local_ret);
 			if (ret == 0)
@@ -175,7 +176,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling rte_flow_flush on sub_device %d", i);
 		ret = rte_flow_flush(PORT_ID(sdev), error);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_flow_flush failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
@@ -199,8 +200,11 @@
 
 	sdev = TX_SUBDEV(dev);
 	if (sdev != NULL) {
-		return rte_flow_query(PORT_ID(sdev),
+		int ret = rte_flow_query(PORT_ID(sdev),
 				flow->flows[SUB_ID(sdev)], type, arg, error);
+
+		if (!SUBDEV_REMOVED(sdev, ret))
+			return ret;
 	}
 	WARN("No active sub_device to query about its flow");
 	return -1;
@@ -223,7 +227,7 @@
 			WARN("flow isolation mode of sub_device %d in incoherent state.",
 				i);
 		ret = rte_flow_isolate(PORT_ID(sdev), set, error);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_flow_isolate failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
diff --git a/drivers/net/failsafe/failsafe_ops.c b/drivers/net/failsafe/failsafe_ops.c
index f460551..cc7ab7f 100644
--- a/drivers/net/failsafe/failsafe_ops.c
+++ b/drivers/net/failsafe/failsafe_ops.c
@@ -314,7 +314,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling rte_eth_dev_set_link_up on sub_device %d", i);
 		ret = rte_eth_dev_set_link_up(PORT_ID(sdev));
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_eth_dev_set_link_up failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
@@ -333,7 +333,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling rte_eth_dev_set_link_down on sub_device %d", i);
 		ret = rte_eth_dev_set_link_down(PORT_ID(sdev));
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_eth_dev_set_link_down failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
@@ -418,7 +418,7 @@
 				rx_queue_id,
 				nb_rx_desc, socket_id,
 				rx_conf, mb_pool);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("RX queue setup failed for sub_device %d", i);
 			goto free_rxq;
 		}
@@ -484,7 +484,7 @@
 				tx_queue_id,
 				nb_tx_desc, socket_id,
 				tx_conf);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("TX queue setup failed for sub_device %d", i);
 			goto free_txq;
 		}
@@ -563,7 +563,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling link_update on sub_device %d", i);
 		ret = (SUBOPS(sdev, link_update))(ETH(sdev), wait_to_complete);
-		if (ret && ret != -1) {
+		if (ret && ret != -1  && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Link update failed for sub_device %d with error %d",
 			      i, ret);
 			return ret;
@@ -587,6 +587,7 @@
 fs_stats_get(struct rte_eth_dev *dev,
 	     struct rte_eth_stats *stats)
 {
+	struct rte_eth_stats backup;
 	struct sub_device *sdev;
 	uint8_t i;
 	int ret;
@@ -596,14 +597,20 @@
 		struct rte_eth_stats *snapshot = &sdev->stats_snapshot.stats;
 		uint64_t *timestamp = &sdev->stats_snapshot.timestamp;
 
+		rte_memcpy(&backup, snapshot, sizeof(backup));
 		ret = rte_eth_stats_get(PORT_ID(sdev), snapshot);
 		if (ret) {
+			if (SUBDEV_REMOVED(sdev, ret)) {
+				rte_memcpy(snapshot, &backup, sizeof(backup));
+				goto inc;
+			}
 			ERROR("Operation rte_eth_stats_get failed for sub_device %d with error %d",
 				  i, ret);
 			*timestamp = 0;
 			return ret;
 		}
 		*timestamp = rte_rdtsc();
+inc:
 		failsafe_stats_increment(stats, snapshot);
 	}
 	return 0;
@@ -716,7 +723,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling rte_eth_dev_set_mtu on sub_device %d", i);
 		ret = rte_eth_dev_set_mtu(PORT_ID(sdev), mtu);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_eth_dev_set_mtu failed for sub_device %d with error %d",
 			      i, ret);
 			return ret;
@@ -735,7 +742,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling rte_eth_dev_vlan_filter on sub_device %d", i);
 		ret = rte_eth_dev_vlan_filter(PORT_ID(sdev), vlan_id, on);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_eth_dev_vlan_filter failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
@@ -769,7 +776,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling rte_eth_dev_flow_ctrl_set on sub_device %d", i);
 		ret = rte_eth_dev_flow_ctrl_set(PORT_ID(sdev), fc_conf);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_eth_dev_flow_ctrl_set failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
@@ -806,7 +813,7 @@
 	RTE_ASSERT(index < FAILSAFE_MAX_ETHADDR);
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		ret = rte_eth_dev_mac_addr_add(PORT_ID(sdev), mac_addr, vmdq);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_eth_dev_mac_addr_add failed for sub_device %"
 			      PRIu8 " with error %d", i, ret);
 			return ret;
@@ -848,7 +855,7 @@
 	FOREACH_SUBDEV_STATE(sdev, i, dev, DEV_ACTIVE) {
 		DEBUG("Calling rte_eth_dev_filter_ctrl on sub_device %d", i);
 		ret = rte_eth_dev_filter_ctrl(PORT_ID(sdev), type, op, arg);
-		if (ret) {
+		if (ret && !SUBDEV_REMOVED(sdev, ret)) {
 			ERROR("Operation rte_eth_dev_filter_ctrl failed for sub_device %d"
 			      " with error %d", i, ret);
 			return ret;
diff --git a/drivers/net/failsafe/failsafe_private.h b/drivers/net/failsafe/failsafe_private.h
index d81cc3c..ee81b70 100644
--- a/drivers/net/failsafe/failsafe_private.h
+++ b/drivers/net/failsafe/failsafe_private.h
@@ -262,6 +262,14 @@ int failsafe_eth_lsc_event_callback(uint16_t port_id,
 	(ETH(s)->dev_ops->ops)
 
 /**
+ * s: (struct sub_device *)
+ * e: (int) error
+ */
+#define SUBDEV_REMOVED(s, e) \
+	(s->remove || \
+	 (((e) == -ENODEV) && (ETH(s)->data->dev_flags & RTE_ETH_DEV_INTR_RMV)))
+
+/**
  * Atomic guard
  */
 
-- 
1.8.3.1