From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E7D3DA04C7; Tue, 15 Sep 2020 09:00:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 750E0160; Tue, 15 Sep 2020 09:00:15 +0200 (CEST) Received: from nat-hk.nvidia.com (nat-hk.nvidia.com [203.18.50.4]) by dpdk.org (Postfix) with ESMTP id 476A6160 for ; Tue, 15 Sep 2020 09:00:12 +0200 (CEST) Received: from hkpgpgate102.nvidia.com (Not Verified[10.18.92.77]) by nat-hk.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Tue, 15 Sep 2020 15:00:11 +0800 Received: from HKMAIL101.nvidia.com ([10.18.16.10]) by hkpgpgate102.nvidia.com (PGP Universal service); Tue, 15 Sep 2020 00:00:11 -0700 X-PGP-Universal: processed; by hkpgpgate102.nvidia.com on Tue, 15 Sep 2020 00:00:11 -0700 Received: from HKMAIL102.nvidia.com (10.18.16.11) by HKMAIL101.nvidia.com (10.18.16.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Tue, 15 Sep 2020 07:00:07 +0000 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.172) by HKMAIL102.nvidia.com (10.18.16.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Tue, 15 Sep 2020 07:00:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WIleajO+soQggbXB4hM4X6PVmkWXVcX4Q+slPC6E00LfcYG7sq5plM9tqEBmsfGhiaHgG24m72NoODWZ/NgFmx22TTANSyTmxZX0XN3ySEiiQ3/RqjrZ5/fI2Na80apmAJ2QhC0lAGITwGsHJ+79FwDnJdfGW0uFFybrNKCTthZYrE+taeHlHdY/c22Cnnk6W+mnNnH+KwJ14M0I51cwmDDYqpLUJWN/91rDw04WI6M7inEHQns5q0c+0lFw8UarRtin1qWV9UudwjTimLtQJfVGYwl7jqe2HdQ2FDTFx+WaRFE6VtXiwBp1/KL8AF2aJOP5Q6ldg36mQgGGH5xttw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UmPq9VEWrRmosCI456J9q3ga/gYrzNoKerjtdF5CQiE=; b=BhgTzNRIihueRt2JG++/kZ1EfTvIhegmERg1XtKU1xMmz4V0A6b7rfGPd0UqPlicf+IXavSOD7z2PAwTYtV5bCWI/kmqnfTgIeoMJnjGk5AYmAY2iyKOezc2ZmMBxskD0A/AgbftIvBxfxuw6VlVhJbN7nj42KoH6++/5YjTZ0A93i3zZ6FsZCuKVI4WVHixSffzddEZ9xX27t0Ho3nNZglWwHlSWEcj5fADooiQZ+kwt7tczFBccxlXwOIeLoBHzjT8z9C6AtSc9FQc6YQU7sz2V0WGymYEZHmhoW0W72CcK0DrycLPve2duTPwaJv/xKbTu413/ir3pkhp9mV2fA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MW2PR12MB2492.namprd12.prod.outlook.com (2603:10b6:907:8::19) by MWHPR12MB1149.namprd12.prod.outlook.com (2603:10b6:300:c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3370.16; Tue, 15 Sep 2020 07:00:03 +0000 Received: from MW2PR12MB2492.namprd12.prod.outlook.com ([fe80::f164:e7bb:8264:f84f]) by MW2PR12MB2492.namprd12.prod.outlook.com ([fe80::f164:e7bb:8264:f84f%5]) with mapi id 15.20.3370.019; Tue, 15 Sep 2020 07:00:03 +0000 From: Matan Azrad To: NBU-Contact-longli , Stephen Hemminger CC: "matan@mellanox.com" , "grive@u246.net" , "dev@dpdk.org" , Raslan Darawsheh Thread-Topic: [dpdk-dev] [PATCH] net/vdev_netvsc: handle removal of associated pci device Thread-Index: AQHWhHw23ILUeXN8QkudN0MmJsMwS6lcrkbQgAyA/gCAACCzsA== Date: Tue, 15 Sep 2020 07:00:03 +0000 Message-ID: References: <20200819175333.19601-1-stephen@networkplumber.org> <20200906113309.4a320400@hermes.lan> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=dd1eb5fe-0f02-48d0-bd8a-9577addb9a09; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2020-09-15T03:57:27Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; authentication-results: microsoft.com; dkim=none (message not signed) header.d=none;microsoft.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [77.126.81.41] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 15308e8a-328f-4b27-5bea-08d85944f897 x-ms-traffictypediagnostic: MWHPR12MB1149: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: dMta69Ewf6IsU7Ys6+haQGlpJHWMcLNMDj9Va59tFfHTGuo1gw6YNozG+aWmXqZvm/eCghrACdSHnOropu1KfexR5X7O3Tjl1ORnGaec0nGTEhq+glTRqW3nn4z9Oz2p2U2Xs0Tey9njieWYH5TS1Uy/BneHjYxh5pkyK5xM3d3DF6P/3lwOk/MZhVxPYX7WFtAIs7GirTiwAxU2CI279VePaILeLqC/xDrSKKEWSUs0I66ycLahssM9mMbbDiF2Cmp2A+XtGNsnNlbwoK9by8hNFhNwiZxff1svPgD+y/g70mk7HZJTCaYl2xfZSJ51ja2zQqxs/B8BPmGivW5ywzKIoXld4eVTCfAGdqnrY8sMRMNit612f4DS+KL6QBrA x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MW2PR12MB2492.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(136003)(376002)(346002)(39860400002)(316002)(8676002)(2906002)(66446008)(186003)(478600001)(55016002)(66476007)(64756008)(66556008)(110136005)(9686003)(83380400001)(76116006)(26005)(8936002)(66946007)(86362001)(33656002)(52536014)(107886003)(71200400001)(5660300002)(7696005)(4326008)(6506007)(54906003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: APYRD21H6kLB9Ml2YyrYpETH2dE4hZJ29yngxRooWw5grTBNInlMNwV4k2Jpvm3Poxvpok/YVLz1Mwj/O1vQDPw8tJH//OtDw4NtoBwauACgImGdpjQJp8EQk3hN/csXHQi+ADJntoBW4pBliO4BUdQh4jquNKZGzPqXkhdFqdifkYzSV7sVgPFtH8nOicLSRLEaEqOxDWNPtov+NkfNnHTLzYXrpkjROJnANHQtYsWreM76Sl+0GW5zgmVZOKxIaBikVBJ4j8BAHEK//Ha5BZTd44WWPSl2QIg+YNq5GTEGPiZuPIfjBu8BqC+s2UjpneGRnrSV/r7Ly9eRUuANnQizFrJHe2Cofc7r4K4N36ZDHcE7FTAne/2bOKufbcT4+YKrKaccS1afwlATd4ADBnVjO3hqvttbgK4n2k5Ow8WW24SYbkLLA71B7TNXKmHQh0nRleHXUnCytCiDHrTFgROr7/w/OKad9HmIUVdoLNWCSbst+bX6/1eTYMIRQB/ecuZKlkeJuIS9D9XTsIk9KEIXCVVo44X1bmP9QyG9VxX8Urs9AbcshIATACa3CspAQiKQcS3B19BujxzY6JO5NY14DEVcIIkG7XCU44nUafHW1+JzLjqyB6May2n8HIyVfWsWglZyydob1y+YiRKf7A== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MW2PR12MB2492.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 15308e8a-328f-4b27-5bea-08d85944f897 X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Sep 2020 07:00:03.4403 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ChtUv8fEB9ULze4TIG+eVIoH+6fX0Xy+n5In9o9mb/5xhesMcoHfvbIx7UoGgKgyl4jlSXWaMLM5NTKyRgmd3w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1149 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1600153211; bh=UmPq9VEWrRmosCI456J9q3ga/gYrzNoKerjtdF5CQiE=; h=X-PGP-Universal:ARC-Seal:ARC-Message-Signature: ARC-Authentication-Results:From:To:CC:Subject:Thread-Topic: Thread-Index:Date:Message-ID:References:In-Reply-To: Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:msip_labels:authentication-results: x-originating-ip:x-ms-publictraffictype: x-ms-office365-filtering-correlation-id:x-ms-traffictypediagnostic: x-ld-processed:x-ms-exchange-transport-forked: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:Content-Type: Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=IKFGuGEY7GIcGT7lCNjXasXQqShA1RaiSJQFrbkaMVumgx2jHbJZadyK3XFaqePIf bSEOBclW9fZKlp5gCZCBEf3NGZTGkSgwlUyFs1i/Y7K93PRB2TawG2ZE52F5sxGNVW QxtFkySCYZ8espquxv8Z0vkrhM2uw55VqtKb0vNS55nL8ZYO4oEf1DAmsbcdkz+nI7 0Ccn7LWWWPwHk5GHPiIyWCYP/o4QhhfjuqO9JfsNa2EHepJB8YNkf38AR9+hYhSYJF kncK9OwsDh6CwYqWi68wDkfYg91zG3G6DM7ElUu6kLDGtusWmDEZ3l8/ovT2dPem1Q y/xO93bcEckOw== Subject: Re: [dpdk-dev] [PATCH] net/vdev_netvsc: handle removal of associated pci device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Li From: Long Li > >Subject: Re: [dpdk-dev] [PATCH] net/vdev_netvsc: handle removal of > >associated pci device > > > >Hi Stephen > > > >From: Stephen Hemminger: > >> On Sun, 6 Sep 2020 12:38:18 +0000 > >> Matan Azrad wrote: > >> > >> > Hi Stephen > >> > > >> > From: Stephen Hemminger: > >> > > The vdev_netvsc was not detecting when the associated PCI device > >> > > (SRIOV) was removed. Because of that it would keep feeding the > >> > > same > >> > > (removed) device to failsafe PMD which would then unsuccessfully > >> > > try and probe for it. > >> > > > >> > > Change to use a mark/sweep method to detect that PCI device was > >> > > removed, and also only tell failsafe about new PCI devices. > >> > > Vdev_netvsc does not have to keep stuffing the pipe with the same > >> > > already existing PCI device. > >> > > >> > As I know, the vdev_netvsc driver doesn't call to failsafe if the > >> > PCI device is > >> not detected by the readlink command(considered as removed)... > >> > Am I missing something? > >> > >> The original code is broken because ctx_yield is not cleared, it > >> keeps sending the same value. > > > >Looking on the code again, It looks like ctx->yield has no effect on > >the next pipe write, It is just used for log. > > > >After the PCI interface matching to the netvsc interface, the pipe > >write is triggered only if the readlink commands success to see the > >plugged-in PCI > >device: > >readlink /sys/class/net/[iface]/device/subsystem shows "pci" > >readlink /sys/class/net/[iface]/device shows the pci device ID. > > > >So, the assumption is when the above readlink failed on the interface > >the device is removed(plugged-out) and the fd write will not happen. > > > >The code will continue to retry probe again and again until success > >only for plugged-in pci device matched the netvsc device. >=20 > Hi Matan, >=20 > The original code keeps writing to pipe even it's the same PCI device. Yes, the vdev_netvsc writes any plugged-in device to the associated netvsc = device fd. > The > new code writes to pipe for a new device, only once. See the following co= de: >=20 > + /* Skip if this is same device already sent to failsafe */ > + if (strcmp(addr, ctx->yield) =3D=3D 0) > + return 0; >=20 I understand you want to optimize the pipe writing to be written only after= plugged-in hot event. The current solution suffers from race: the PCI device may be plugged-out a= nd plugged-in in short time shorter than the driver alarm delay, then the P= CI device plugged-in detection will lost. My suggestion: Add validation to the plugged-in device probing state and that it is owned = by failsafe(using ownership API) - don't write the pipe if so. Matan > This patch also saves lots of CPU since it no longer writes to pipe all t= he time. > You are correct about the code will continue to probe on a new PCI device= . > But someone has to do it to handle hot-add. >=20 > Thanks, > Long >=20 >=20 > > > >> It looks like device removal and add was never tested. > > > >This is basic test we have to test plug-in plug-out and it passed every > >day in the last years. > > > >Maybe something new and special in your setup? > > > >> If you test removal you will see that vdev_netvsc: > >> 1. Sends same PCI device repeatedly to failsafe (every alarm call) > >> This is harmless, but useless. > >> 2. When device is removed, keeps doing #1