From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A4E3E4591C; Fri, 6 Sep 2024 11:20:29 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 54D8142E12; Fri, 6 Sep 2024 11:20:29 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2043.outbound.protection.outlook.com [40.107.223.43]) by mails.dpdk.org (Postfix) with ESMTP id 5E5C64029E for ; Fri, 6 Sep 2024 11:20:27 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=F1Dy58HVifXb9l4uvT1f6R1e61WZVYaOdVoOBlcEeHVFf6vP3VCliXsC+dCPUDr+E/Y9Pq+xmi/89p5go+9CG3R4iVnRjOHuBRJFi1cXbgkXARtoP90EjuAfplijqXtrNqyFC8+XBIvdqSgouLEJfMDG2IwEFje+JTXzdZaTaIHb2VO+lWzWY8g8xjSOnGbiCGVYbizfgIImglTsg4kMNu5xbr5p8q8xwvrYQdCb6f6O1V/AWk+L/Z5D9jsGAHvvllJnTmHLRdenBV44B8MIWGYO9lIDHbQUxAmUSOksvNyVk0K04WN4q3oZNe0VzuQwhdFlUGOMTXKW9urWwX4yuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q508NGhTkBF5aMmuyM4PAlvLQ4Ucdec0RYDrRNqckXE=; b=N8hnwPzDJKmm59Y0DAn8hQ+zeMotGiCptwbVTsUHGulq05UeHFM9Go3MM7dAakMJcj9aotDW3DTh1mjuWctdMWaVHkf4lAnH+xOCP3+NC+QybHjHTx4MHhzB/NoCdeY8JndI1CA4xK+DX8R4thKvCF0CphRN1zBfaRXXPXSbxj0078+M1SSSxyrbmX/XFeFdmeseBkwO358X2AyFyWFKKXF/cJ97AajViMZnDoqEPZCSLKBLd4GirW4ySHgCd5NPhgeqiYVI/jRZARwEnNJkNICFWz8ffSAXDwmo7ypexRwhWvR8gE+6kq5N8vCmGV2ocSpxULwb8EYwWFUR40Txuw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q508NGhTkBF5aMmuyM4PAlvLQ4Ucdec0RYDrRNqckXE=; b=E4paWSStWcYvZT6cxq5s4LFZR1vJhCwnkDDjuFm2bFDZyYVl0c4i0KlZQ6JdcN9yGI+GQPf1GLr+SaBU/t8ru80VBDjUlYMV4iGgXtsPqAHXQ2JXuLHQs9B28olJW/lrxxWw5x6T/M2fMx0a8g8cxwCJBsrXTeyLEpxk8c7tiddT1k511diF7dfVX2WfsNCx+t79wmxT6xNaa0FU7GKbbi/ruYzZ4dbWVH81052QELbL4VfIHe0M/sxvKf6sVXzEMbR0rk+RxLAZDwLkBiuHdXMa9uldXgeN/UzVS1FQO+wjDaTClOiiuEVtMtr5GiKJTN4R1magbAq/ZX+KFXC24A== Received: from IA0PR12MB8983.namprd12.prod.outlook.com (2603:10b6:208:490::17) by MW4PR12MB6974.namprd12.prod.outlook.com (2603:10b6:303:207::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7939.17; Fri, 6 Sep 2024 09:20:24 +0000 Received: from IA0PR12MB8983.namprd12.prod.outlook.com ([fe80::68e7:8370:b169:fa0e]) by IA0PR12MB8983.namprd12.prod.outlook.com ([fe80::68e7:8370:b169:fa0e%6]) with mapi id 15.20.7939.017; Fri, 6 Sep 2024 09:20:22 +0000 From: Chenbo Xia To: Maxime Coquelin CC: "dev@dpdk.org" , "david.marchand@redhat.com" Subject: Re: [PATCH 2/2] vhost: add reconnection support to VDUSE Thread-Topic: [PATCH 2/2] vhost: add reconnection support to VDUSE Thread-Index: AQHa/5+Tb2hkJpD2pkWfDp9/zoprPbJKWaMA Date: Fri, 6 Sep 2024 09:20:22 +0000 Message-ID: <3543971A-6C8E-4F28-A7D4-F8C222F3C193@nvidia.com> References: <20240905142602.1264345-1-maxime.coquelin@redhat.com> <20240905142602.1264345-3-maxime.coquelin@redhat.com> In-Reply-To: <20240905142602.1264345-3-maxime.coquelin@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: IA0PR12MB8983:EE_|MW4PR12MB6974:EE_ x-ms-office365-filtering-correlation-id: 09b5c1d9-9a7f-4eed-0827-08dcce552246 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|1800799024|376014|366016|38070700018; x-microsoft-antispam-message-info: =?us-ascii?Q?cvDLqQJ9mQPn8WMw5Mb+73qUM2ts9o+aGGp/3344vS+6tgo85PjB4xLscVr6?= =?us-ascii?Q?L1jH7tRmr0Srz7UenYd8G+PuZydzRMH5XG2WDpifc6zP0w5/7ex9lH3HOAHt?= =?us-ascii?Q?jOANCL3ocvawamBtijrPvhanzU+mAIUMt3BIc0/MiY6JHdaAmCycdNmYF86X?= =?us-ascii?Q?4y6pwWPjtCT9/bDFE6v0VLe9+yoEPtV0enNUYY53ZICVv7uhvkxqQ5LGC0nO?= =?us-ascii?Q?svVyEdT8T+S4bitlSl7lLKGShqLobfAteAkjI6CQ4ARp7muImnMwW8dIjiIu?= =?us-ascii?Q?SiKJZCaER31lFdNlmUUslF2CFFY/c3Bmu4OS6urnSv6nqvPYbJu0ZkU6shlw?= =?us-ascii?Q?eCnzKG3rpN95Hv2FndRu+yl/0kXcDPR4qCTyFh/mr6a1JnCilRiboQIs6RjR?= =?us-ascii?Q?2RsQeMccAGuZQSr5ApYelhWeUjqLHSZYNQcL9dRwMLmSxadigaTNjPewIiA9?= =?us-ascii?Q?dUJamyUoa17RfiaBUnIaUp2f+uO2Cs2wuHSUh7lxkNCI3f/XPczhfa4tiWsd?= =?us-ascii?Q?UZPF4LSO2A9SwI2J60WVsmJ+Bbi5335qpVlJbJXsEyT7hlR6nPNh0Ed+6wJX?= =?us-ascii?Q?O9ybcTcbfcoiHbTNpe8fadW3ld/QugtV+q4fajuIf/tpAmo8uCUTKf0klJMd?= =?us-ascii?Q?bmzhl7I3CzGy8Stu7azOJHJ6oCSSEQ/O8Mr4H+K7YpCo0sRfDHFFIv3YFLxO?= =?us-ascii?Q?JQN4r3lRBli6i+09rllRNLZ6tHmZNkgvp/Hp7pxTp2rNq1+vztNpFjK6M6eB?= =?us-ascii?Q?eCQKbkz6G5WoQz0kgVGSbEbk9DkNShnb/fzoRpcl+SampHoue+FVG+K41rIk?= =?us-ascii?Q?J8u0FaFGK82sdzghN/uMLthah+Gb0RspkXjsHGVgk1Ms4W3+qz5jjr2vPbWa?= =?us-ascii?Q?XBQ/9pDTbRyts5qlwiErzHnGX5LSclwLV7pyGVhT5sjgYwr+3CNaa7kyR0EA?= =?us-ascii?Q?KqbzJg+SBCmiNfL/zEYLr4z6qMP1z1SnYGE2ciEauh/v0BYUTEzStXVeTjAV?= =?us-ascii?Q?ayC9CiOuvf6c509+vQkDIq71lXK0mIApV5SRTgPPQ8rxMG9FeeRRrSdmsRmG?= =?us-ascii?Q?O+sHlkFBK2SgUCCBbzzMX4boonlJmxU4cFh2S1I3BOF5+f7ZakIEV9QsPskH?= =?us-ascii?Q?3n+jMZNkIKEGZHObp+MLqP4KeBeY2JcOpu1QMzT0/FtX/JYevjUTJ9o4Dzck?= =?us-ascii?Q?im618EbMWp7O3LGy/ixrfBASkHVVde4Jnv3zaWmfkfZ10YXxIdq/O0josv+1?= =?us-ascii?Q?yBDn81UUnBshgwRG1tw7Bw7RHkBh6cd0+5hHnRAZX+paViurBX8ISA8ADD25?= =?us-ascii?Q?O6VhVxMn9aPVX8L/i8au6+GDkL7/FVcWZvKUXVnb9uOF0/KCzEt8Y12wYzu9?= =?us-ascii?Q?xW0lwYGxhge9vUDBjDSBRgAbHzV9OWPTA0rqSGfExua+hFx2Ng=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:IA0PR12MB8983.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016)(38070700018); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?LR+XRBUxjUbu0dWkQ7uognYYojvloSJy0xmbWX4M4uwuYLIHxOerfkEuMsi5?= =?us-ascii?Q?0UH6RFbFRF7XMRpma6YuRaBlmhFSGH5TJ/0kcTICw4cOX6Qq1hSwBk4Cm/20?= =?us-ascii?Q?CjUkqxAW9KrwK6SeYDdXv8tHAS3eRCc5yKXsZ02Fhty7vqJ7EM/IQCzGb/Bs?= =?us-ascii?Q?XC/8j5+0S9NA4gvBc8kPs76FEyOIVAm5dggh7N9n1LsUfcpWSBSH4zW0YDgU?= =?us-ascii?Q?YSCBH0onqcHAwe4jYpvQh5Qlv5PTwfDVZ7VVNAhqkjkUlDuZESSLMiCtEV7o?= =?us-ascii?Q?0Qf2bzbm1PE+VsaYc1GYsPmji4qLbUfSVkk4rz0S6uFeEppihYg+PzjLqMFF?= =?us-ascii?Q?f+Vb8UoBWyg3qyJ/ouKt/n4eRqUp0eQ9RWr6Hs2OsA+N8P1xwjnUxzfByRdh?= =?us-ascii?Q?CWx50a/xMg3suaa7/gLDiSRTJTsTTVbXAYRadX8cmGWk6+NAZMFdnYSjaHMG?= =?us-ascii?Q?7iSkLFuKoQKKqirc98cFwgmVnj6BHBHccjjx/2aErGt2wHhFpokutX5zrDwb?= =?us-ascii?Q?FhVH6JGXvR+wIjxkZJP1yEBQz6bRMGvkgCHgyhwRK8N3RbVckqr5l/d9N6kp?= =?us-ascii?Q?UZoQLoQbz7rW2XmMFzuw/g7NjdB+lnlCx8SFROYDe0ULXV39wW8nFLaGopaV?= =?us-ascii?Q?Vc0wX6NN/ByQLKLoSDijbBXwezP5D14QY4E+f/rcaFPF0MkfxSddZjyh1VEY?= =?us-ascii?Q?OSTRpuSxcOx7fja95sRjpCbpVpLxuDhX9fQsHViS6OcOGjZsXIU0cXBB/nO+?= =?us-ascii?Q?OfWb5hWvC+OaoDWq03IOcm/FGkDWkrTwKTy/ckpN8JFn82wCdQDSxiHyA4tt?= =?us-ascii?Q?LJLTOJyUV2sPOyTYAcTTWSa+k0FcPGXOdgq7SBlUV7oXGWhp+ZpzyIjFdof9?= =?us-ascii?Q?Vp1qFvLD3stDCw9Hlo8ftcuafkzX+bQ5sShPUQ2SrMxXCGVNpBwNaxN9MVNC?= =?us-ascii?Q?35Kbndg+i4Y4bHv4EkNlvY23ILlM64tSx3OmknLm8k5BQxc2P7a43H2BIt0n?= =?us-ascii?Q?TEdMiV+r+BoZf2PGyt6u7KZrvAc/3fhOjiyqXsKygY5lF9SgB9cykemXvdQa?= =?us-ascii?Q?FrJcaaju70TigjITRGaXf5QNHmwjlCq3ORSY9yx+XmAFJRrf2obvYsPkJdlD?= =?us-ascii?Q?axvXLW00z8GaltmQUn6AjvynZ9S2flm3KqdGZ6YcxLftZUIeXXdYG9nckuq0?= =?us-ascii?Q?2SfSnJw4RC2shn+EXJ6VEqNQQjdq+c0lFtwTSbgOG8adeNuBTPa0LPDKiiX5?= =?us-ascii?Q?NDJhfiI2ATvskx/8ujIt+GOf0Wc3VxUsHKHH6ooUi67ik9sSWJhw/lDHCSNc?= =?us-ascii?Q?DZMd9PiqTklrn5hq2C9L72lZkS0xVBIEOnLixs09bliTbCHxtA/lnyDUkf9G?= =?us-ascii?Q?i9p9TR/dCjyulY7XZlpUz9TACVed1yci4fp4UNqHL+E23sgecFPCQujI/kkK?= =?us-ascii?Q?JACZCmCaKzTVoV1VMx1ZnZf/U1Li+3BZQI7YfS+Sm4fbBCjW8ZNRrmZ4Sqm0?= =?us-ascii?Q?ESbKzMVrENhDW/XS8/4dzjXzBpnLszRbqV7KNOPBm2bOBGpHbu46uR34R3RQ?= =?us-ascii?Q?J4F6aldm3MwgzA+B4Uruv6+XE+Z9WEPQ4N2ojmzY?= Content-Type: text/plain; charset="us-ascii" Content-ID: <653EA2332AD28C47B1129265C455B270@namprd12.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: IA0PR12MB8983.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 09b5c1d9-9a7f-4eed-0827-08dcce552246 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2024 09:20:22.2026 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xEtwcwEagKYllpZPd0GhD2t/22Lgq9Cgvxfzo4tAYTXZYEWBMlwscIoczPTjOrWkrREzTB3LM6cz3xzvimlisA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6974 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Message-ID: <20240906092022.huyWoLEZ6iBOwKo3R_Z8BK-wR98RkulzEY2Mz8kSPXw@z> Hi Maxime, > On Sep 5, 2024, at 22:26, Maxime Coquelin wr= ote: >=20 > External email: Use caution opening links or attachments >=20 >=20 > This patch enables VDUSE reconnection support making use of > the newly introduced reconnection mechanism in Vhost > library. >=20 > At DPDK VDUSE device creation time, there are two > possibilities: > 1. The Kernel VDUSE device does not exist: > a. A reconnection file named after the VUDSE device name > is created in VDUSE tmpfs. > b. The file is truncated to 'struct vhost_reconnect_data' > size, and mmapped. > c. Negotiated features, Virtio status... are saved for > sanity checks at reconnect time. > 2. The Kernel VDUSE device already exists: > a. Exit with failure if no reconnect file exists for > this device. > b. Open and mmap the reconnect file. > c. Perform sanity check to ensure features are compatible. > d. Restore virtqueues' available indexes at startup time. >=20 > Then at runtime, the virtqueues' available index are logged by > the Vhost reconnection mechanism. >=20 > At DPDK VDUSE device destruction time, there are two > possibilities: > 1. The Kernel VDUSE device destruction succeed, which > means it is no more attached to the vDPA bus. The > reconnection file is unmapped and then removed. > 2. The Kernel VDUSE device destruction failed, meaning it > is no more attached to the vDPA bus. The reconnection > file is unmapped but not removed to make possible later > reconnection. >=20 > Signed-off-by: Maxime Coquelin > --- > lib/vhost/vduse.c | 280 +++++++++++++++++++++++++++++++++++++++------- > 1 file changed, 241 insertions(+), 39 deletions(-) >=20 > diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c > index c66602905c..bd0e492d62 100644 > --- a/lib/vhost/vduse.c > +++ b/lib/vhost/vduse.c > @@ -136,7 +136,7 @@ vduse_control_queue_event(int fd, void *arg, int *rem= ove __rte_unused) > } >=20 > static void > -vduse_vring_setup(struct virtio_net *dev, unsigned int index) > +vduse_vring_setup(struct virtio_net *dev, unsigned int index, bool recon= nect) > { > struct vhost_virtqueue *vq =3D dev->virtqueue[index]; > struct vhost_vring_addr *ra =3D &vq->ring_addrs; > @@ -152,6 +152,19 @@ vduse_vring_setup(struct virtio_net *dev, unsigned i= nt index) > return; > } >=20 > + if (reconnect) { > + vq->last_avail_idx =3D vq->reconnect_log->last_avail_idx; > + vq->last_used_idx =3D vq->reconnect_log->last_avail_idx; > + } else { > + vq->last_avail_idx =3D vq_info.split.avail_index; > + vq->last_used_idx =3D vq_info.split.avail_index; > + } > + vq->size =3D vq_info.num; > + vq->ready =3D true; > + vq->enabled =3D vq_info.ready; > + ra->desc_user_addr =3D vq_info.desc_addr; > + ra->avail_user_addr =3D vq_info.driver_addr; > + ra->used_user_addr =3D vq_info.device_addr; > VHOST_CONFIG_LOG(dev->ifname, INFO, "VQ %u info:", index); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tnum: %u", vq_info.num); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tdesc_addr: %llx", > @@ -162,15 +175,6 @@ vduse_vring_setup(struct virtio_net *dev, unsigned i= nt index) > (unsigned long long)vq_info.device_addr); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tavail_idx: %u", vq_info.spl= it.avail_index); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tready: %u", vq_info.ready); > - > - vq->last_avail_idx =3D vq_info.split.avail_index; > - vq->size =3D vq_info.num; > - vq->ready =3D true; > - vq->enabled =3D vq_info.ready; > - ra->desc_user_addr =3D vq_info.desc_addr; > - ra->avail_user_addr =3D vq_info.driver_addr; > - ra->used_user_addr =3D vq_info.device_addr; > - > vq->kickfd =3D eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); > if (vq->kickfd < 0) { > VHOST_CONFIG_LOG(dev->ifname, ERR, "Failed to init kickfd = for VQ %u: %s", > @@ -267,7 +271,7 @@ vduse_vring_cleanup(struct virtio_net *dev, unsigned = int index) > } >=20 > static void > -vduse_device_start(struct virtio_net *dev) > +vduse_device_start(struct virtio_net *dev, bool reconnect) > { > unsigned int i, ret; >=20 > @@ -287,6 +291,15 @@ vduse_device_start(struct virtio_net *dev) > return; > } >=20 > + if (reconnect && dev->features !=3D dev->reconnect_log->features)= { > + VHOST_CONFIG_LOG(dev->ifname, ERR, > + "Mismatch between reconnect file features= 0x%" PRIx64 " & device features 0x%" PRIx64, Checkpatch reports long line > + dev->reconnect_log->features, dev->featur= es); > + return; > + } > + > + dev->reconnect_log->features =3D dev->features; > + > VHOST_CONFIG_LOG(dev->ifname, INFO, "Negotiated Virtio features: 0= x%" PRIx64, > dev->features); >=20 > @@ -300,7 +313,7 @@ vduse_device_start(struct virtio_net *dev) > } >=20 > for (i =3D 0; i < dev->nr_vring; i++) > - vduse_vring_setup(dev, i); > + vduse_vring_setup(dev, i, reconnect); >=20 > dev->flags |=3D VIRTIO_DEV_READY; >=20 > @@ -373,6 +386,7 @@ vduse_events_handler(int fd, void *arg, int *remove _= _rte_unused) > req.s.status); > old_status =3D dev->status; > dev->status =3D req.s.status; > + dev->reconnect_log->status =3D dev->status; > resp.result =3D VDUSE_REQ_RESULT_OK; > break; > case VDUSE_UPDATE_IOTLB: > @@ -398,7 +412,7 @@ vduse_events_handler(int fd, void *arg, int *remove _= _rte_unused) >=20 > if ((old_status ^ dev->status) & VIRTIO_DEVICE_STATUS_DRIVER_OK) { > if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) > - vduse_device_start(dev); > + vduse_device_start(dev, false); > else > vduse_device_stop(dev); > } > @@ -407,10 +421,64 @@ vduse_events_handler(int fd, void *arg, int *remove= __rte_unused) > vduse_req_id_to_str(req.type), req.type); > } >=20 > +static char vduse_reconnect_dir[PATH_MAX]; > +static bool vduse_reconnect_path_set; > + > +static int > +vduse_reconnect_path_init(void) > +{ > + const char *directory; > + int ret; > + > + /* from RuntimeDirectory=3D see systemd.exec */ > + directory =3D getenv("RUNTIME_DIRECTORY"); > + if (directory =3D=3D NULL) { > + /* > + * Used standard convention defined in > + * XDG Base Directory Specification and > + * Filesystem Hierarchy Standard. > + */ > + if (getuid() =3D=3D 0) > + directory =3D "/var/run"; > + else > + directory =3D getenv("XDG_RUNTIME_DIR") ? : "/tmp= "; > + } > + > + ret =3D snprintf(vduse_reconnect_dir, sizeof(vduse_reconnect_dir)= , "%s/dpdk/vduse", > + directory); > + if (ret < 0 || ret =3D=3D sizeof(vduse_reconnect_dir)) { > + VHOST_CONFIG_LOG("vduse", ERR, "Error creating VDUSE reco= nnect path name"); > + return -1; > + } > + > + ret =3D mkdir(vduse_reconnect_dir, 0700); > + if (ret < 0 && errno !=3D EEXIST) { > + VHOST_CONFIG_LOG("vduse", ERR, "Error creating '%s': %s", > + vduse_reconnect_dir, strerror(errno)); > + return -1; > + } > + > + VHOST_CONFIG_LOG("vduse", INFO, "Created VDUSE reconnect director= y in %s", > + vduse_reconnect_dir); > + > + return 0; > +} > + > +static void > +vduse_reconnect_handler(int fd, void *arg, int *remove) > +{ > + struct virtio_net *dev =3D arg; > + > + vduse_device_start(dev, true); > + > + close(fd); > + *remove =3D 1; > +} > + > int > vduse_device_create(const char *path, bool compliant_ol_flags) > { > - int control_fd, dev_fd, vid, ret; > + int control_fd, dev_fd, vid, ret, reco_fd; > uint32_t i, max_queue_pairs, total_queues; > struct virtio_net *dev; > struct virtio_net_config vnet_config =3D {{ 0 }}; > @@ -418,6 +486,9 @@ vduse_device_create(const char *path, bool compliant_= ol_flags) > uint64_t features; > struct vduse_dev_config *dev_config =3D NULL; > const char *name =3D path + strlen("/dev/vduse/"); > + char reconnect_file[PATH_MAX]; > + struct vhost_reconnect_data *reconnect_log =3D NULL; > + bool reconnect =3D false; >=20 > if (vduse.fdset =3D=3D NULL) { > vduse.fdset =3D fdset_init("vduse-evt"); > @@ -427,6 +498,20 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > } > } >=20 > + if (vduse_reconnect_path_set =3D=3D false) { > + if (vduse_reconnect_path_init() < 0) { > + VHOST_CONFIG_LOG(path, ERR, "failed to initialize= reconnect path"); > + return -1; > + } > + vduse_reconnect_path_set =3D true; > + } > + > + ret =3D snprintf(reconnect_file, sizeof(reconnect_file), "%s/%s",= vduse_reconnect_dir, name); > + if (ret < 0 || ret =3D=3D sizeof(reconnect_file)) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to create vduse recon= nect path name"); > + return -1; > + } > + > control_fd =3D open(VDUSE_CTRL_PATH, O_RDWR); > if (control_fd < 0) { > VHOST_CONFIG_LOG(name, ERR, "Failed to open %s: %s", > @@ -441,14 +526,6 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > goto out_ctrl_close; > } >=20 > - dev_config =3D malloc(offsetof(struct vduse_dev_config, config) + > - sizeof(vnet_config)); > - if (!dev_config) { > - VHOST_CONFIG_LOG(name, ERR, "Failed to allocate VDUSE con= fig"); > - ret =3D -1; > - goto out_ctrl_close; > - } > - > ret =3D rte_vhost_driver_get_features(path, &features); > if (ret < 0) { > VHOST_CONFIG_LOG(name, ERR, "Failed to get backend feature= s"); > @@ -469,23 +546,97 @@ vduse_device_create(const char *path, bool complian= t_ol_flags) > else > total_queues +=3D 1; /* Includes ctrl queue */ >=20 > - vnet_config.max_virtqueue_pairs =3D max_queue_pairs; > - memset(dev_config, 0, sizeof(struct vduse_dev_config)); > + if (access(path, F_OK) =3D=3D 0) { > + VHOST_CONFIG_LOG(name, INFO, "Device already exists, reco= nnecting..."); > + reconnect =3D true; > + > + reco_fd =3D open(reconnect_file, O_RDWR, 0600); > + if (reco_fd < 0) { > + if (errno =3D=3D ENOENT) { > + VHOST_CONFIG_LOG(name, ERR, "Missing reco= nnect file (%s)", > + reconnect_file); > + } else { > + VHOST_CONFIG_LOG(name, ERR, "Failed to op= en reconnect file %s (%s)", > + reconnect_file, strerror(= errno)); > + } Seems no {} is needed for if-else? > + ret =3D -1; > + goto out_ctrl_close; > + } > + > + reconnect_log =3D mmap(NULL, sizeof(*reconnect_log), PROT= _READ | PROT_WRITE, > + MAP_SHARED, reco_fd, 0); > + close(reco_fd); > + if (reconnect_log =3D=3D MAP_FAILED) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to mmap recon= nect file %s (%s)", > + reconnect_file, strerror(errno)); > + ret =3D -1; > + goto out_ctrl_close; > + } > + } else { > + reco_fd =3D open(reconnect_file, O_CREAT | O_EXCL | O_RDW= R, 0600); > + if (reco_fd < 0) { > + if (errno =3D=3D EEXIST) { > + VHOST_CONFIG_LOG(name, ERR, "Reconnect fi= le %s exists but not the device", > + reconnect_file); > + } else { > + VHOST_CONFIG_LOG(name, ERR, "Failed to op= en reconnect file %s (%s)", > + reconnect_file, strerror(= errno)); > + } > + ret =3D -1; > + goto out_ctrl_close; > + } >=20 > - strncpy(dev_config->name, name, VDUSE_NAME_MAX - 1); > - dev_config->device_id =3D VIRTIO_ID_NET; > - dev_config->vendor_id =3D 0; > - dev_config->features =3D features; > - dev_config->vq_num =3D total_queues; > - dev_config->vq_align =3D sysconf(_SC_PAGE_SIZE); > - dev_config->config_size =3D sizeof(struct virtio_net_config); > - memcpy(dev_config->config, &vnet_config, sizeof(vnet_config)); > + ret =3D ftruncate(reco_fd, sizeof(*reconnect_log)); > + if (ret < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to truncate r= econnect file %s (%s)", > + reconnect_file, strerror(errno)); > + close(reco_fd); > + goto out_ctrl_close; > + } >=20 > - ret =3D ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); > - if (ret < 0) { > - VHOST_CONFIG_LOG(name, ERR, "Failed to create VDUSE devic= e: %s", > - strerror(errno)); > - goto out_free; > + reconnect_log =3D mmap(NULL, sizeof(*reconnect_log), PROT= _READ | PROT_WRITE, > + MAP_SHARED, reco_fd, 0); > + close(reco_fd); > + if (reconnect_log =3D=3D MAP_FAILED) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to mmap recon= nect file %s (%s)", > + reconnect_file, strerror(errno)); > + ret =3D -1; > + goto out_ctrl_close; > + } > + > + reconnect_log->version =3D 0; > + > + dev_config =3D malloc(offsetof(struct vduse_dev_config, c= onfig) + > + sizeof(vnet_config)); > + if (!dev_config) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to allocate V= DUSE config"); > + ret =3D -1; > + goto out_ctrl_close; > + } > + > + vnet_config.max_virtqueue_pairs =3D max_queue_pairs; > + memset(dev_config, 0, sizeof(struct vduse_dev_config)); > + > + rte_strscpy(dev_config->name, name, VDUSE_NAME_MAX - 1); > + dev_config->device_id =3D VIRTIO_ID_NET; > + dev_config->vendor_id =3D 0; > + dev_config->features =3D features; > + dev_config->vq_num =3D total_queues; > + dev_config->vq_align =3D sysconf(_SC_PAGE_SIZE); > + dev_config->config_size =3D sizeof(struct virtio_net_conf= ig); > + memcpy(dev_config->config, &vnet_config, sizeof(vnet_conf= ig)); > + > + ret =3D ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); > + if (ret < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to create VDU= SE device: %s", > + strerror(errno)); > + goto out_free; > + } > + > + memcpy(&reconnect_log->config, &vnet_config, sizeof(vnet_= config)); > + reconnect_log->nr_vrings =3D total_queues; > + free(dev_config); > + dev_config =3D NULL; > } >=20 > dev_fd =3D open(path, O_RDWR); > @@ -519,10 +670,15 @@ vduse_device_create(const char *path, bool complian= t_ol_flags) > strncpy(dev->ifname, path, IF_NAME_SZ - 1); > dev->vduse_ctrl_fd =3D control_fd; > dev->vduse_dev_fd =3D dev_fd; > + dev->reconnect_log =3D reconnect_log; > + if (reconnect) > + dev->status =3D dev->reconnect_log->status; > + > vhost_setup_virtio_net(dev->vid, true, compliant_ol_flags, true, t= rue); >=20 > for (i =3D 0; i < total_queues; i++) { > struct vduse_vq_config vq_cfg =3D { 0 }; > + struct vhost_virtqueue *vq; >=20 > ret =3D alloc_vring_queue(dev, i); > if (ret) { > @@ -530,6 +686,12 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > goto out_dev_destroy; > } >=20 > + vq =3D dev->virtqueue[i]; > + vq->reconnect_log =3D &reconnect_log->vring[i]; > + > + if (reconnect) > + continue; > + > vq_cfg.index =3D i; > vq_cfg.max_size =3D 1024; >=20 > @@ -549,7 +711,28 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > goto out_dev_destroy; > } >=20 > - free(dev_config); > + if (reconnect && dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) { > + reco_fd =3D eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); > + if (reco_fd < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to create rec= o_fd: %s", > + strerror(errno)); > + ret =3D -1; > + goto out_dev_destroy; > + } > + > + ret =3D fdset_add(vduse.fdset, reco_fd, vduse_reconnect_h= andler, NULL, dev); > + if (ret) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to add fd %d = to vduse fdset", > + dev->vduse_dev_fd); Should print reco_fd > + goto out_dev_destroy; > + } > + > + ret =3D eventfd_write(reco_fd, (eventfd_t)1); > + if (ret < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to write to r= econnect eventfd"); > + goto out_dev_destroy; > + } > + } Maybe I missed something, why we need to implement like this instead of dir= ectly call vduse_device_start? Thanks, Chenbo >=20 > return 0; >=20 > @@ -587,6 +770,9 @@ vduse_device_destroy(const char *path) > if (vid =3D=3D RTE_MAX_VHOST_DEVICE) > return -1; >=20 > + if (dev->reconnect_log) > + munmap(dev->reconnect_log, sizeof(*dev->reconnect_log)); > + > vduse_device_stop(dev); >=20 > fdset_del(vduse.fdset, dev->vduse_dev_fd); > @@ -597,10 +783,26 @@ vduse_device_destroy(const char *path) > } >=20 > if (dev->vduse_ctrl_fd >=3D 0) { > + char reconnect_file[PATH_MAX]; > + > ret =3D ioctl(dev->vduse_ctrl_fd, VDUSE_DESTROY_DEV, name)= ; > - if (ret) > + if (ret) { > VHOST_CONFIG_LOG(name, ERR, "Failed to destroy VDU= SE device: %s", > strerror(errno)); > + } else { > + /* > + * VDUSE device was no more attached to the vDPA = bus, > + * so we can remove the reconnect file. > + */ > + ret =3D snprintf(reconnect_file, sizeof(reconnect= _file), "%s/%s", > + vduse_reconnect_dir, name); > + if (ret < 0 || ret =3D=3D sizeof(reconnect_file)) > + VHOST_CONFIG_LOG(name, ERR, > + "Failed to create vduse r= econnect path name"); > + else > + unlink(reconnect_file); > + } > + > close(dev->vduse_ctrl_fd); > dev->vduse_ctrl_fd =3D -1; > } > -- > 2.46.0 >=20