From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C8DBE4591A; Fri, 6 Sep 2024 09:15:10 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B5F0F42EC6; Fri, 6 Sep 2024 09:15:10 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2064.outbound.protection.outlook.com [40.107.92.64]) by mails.dpdk.org (Postfix) with ESMTP id 9684D42EC4 for ; Fri, 6 Sep 2024 09:15:09 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UwCQ0VKeTlMh7bc/ylDJJgZiRDbkoVSUzvma6gMQe2uM+/2WtnMNqN+IJbmbId3UqXfDlA7HLLyzLnKJ95OzruG1y9t0kmt80d54D+XmOp2xJWRvU52ztJ1v8KgaA9HHdshvy/KwCrgAYp2W8Xid2h4W9e96vCmh8IiGBIzUuv4hz8Aw46sVxm6wq5lYX63F7NwQr741noJFuCzZqungpbEqJ+pb2qGSUEkol0GwM5uXYOpiyH7k9Z552Er8vs5eIiEWmc5gm8/2mHyJYxrFSH5e2sHFqcGwDxjEDJN2rp6lL73QnJVac6Bvgs99KKClr4XVmlMHDss6OydhdInNyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q508NGhTkBF5aMmuyM4PAlvLQ4Ucdec0RYDrRNqckXE=; b=DLi2Z2Sff4JktiDClsMa8IuZK7LOu2BUEO6TppA0mk4rqZJCqa7q56vguLoBPP8L9QOk688CvI2HtajlclBws1Ezoquu38D/vS2SU6I2DcseRGUqLj85SCOTBzzOdzAAUnnfKUyCG8/Ls/afXxnbcA/GV/A1wUUYcC0Yvs05v47NRADSmLT/ugimfI48TBBKByY/iu5DH/UdE6v2UWemaR9r4zr+MJXJ8hzkUCEZJHpzllqM3B7+U0Oz97qopwPdLkB/0KV4Quxqh3IUinpO/KMTBeuz5xmaeck+HN3vrPK5jyJLBl/tB2PrlwoRrOSwISQ2G/xzgtZC4kWT3E6nkA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q508NGhTkBF5aMmuyM4PAlvLQ4Ucdec0RYDrRNqckXE=; b=bRVNZ/0CwkW6SR/fG2PpiHfG87Ic9NR3uyQxaFrPUCWnlA+ydqj/eL4XMowLhE0EbjowJ7CgPXXHQ5A15va8D/Qt3q4058llZaxfECbeEgwJx6a6jvOZ4v8KJJm2WqAumijj4HDvzZdONVuHSoSkqH5dzoHYSgXACWy6P/uCJniX5yGEEOxcY4x6Gq82wJ7zFB5OuJQ33CfxYi2PiqrG2oztmlgxD8OriygcDCjr/Iz6YSX/pdbQtWrxFHA2j5LcYsnltxyaDw9XAiM4EsTsHFQYqjRqUQCTSNXLjGUaE8m6r6IWdj+6Le88bpQRR5apxIhx5LDu69NPXz5N7QJpiQ== Received: from CYYPR12MB8992.namprd12.prod.outlook.com (2603:10b6:930:bc::21) by PH0PR12MB7471.namprd12.prod.outlook.com (2603:10b6:510:1e9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.28; Fri, 6 Sep 2024 07:15:06 +0000 Received: from CYYPR12MB8992.namprd12.prod.outlook.com ([fe80::a722:c2ca:42c7:aaed]) by CYYPR12MB8992.namprd12.prod.outlook.com ([fe80::a722:c2ca:42c7:aaed%7]) with mapi id 15.20.7918.024; Fri, 6 Sep 2024 07:15:06 +0000 From: Chenbo Xia To: Maxime Coquelin CC: "dev@dpdk.org" , "david.marchand@redhat.com" Subject: Re: [PATCH 2/2] vhost: add reconnection support to VDUSE Thread-Topic: [PATCH 2/2] vhost: add reconnection support to VDUSE Thread-Index: AQHa/5+Tb2hkJpD2pkWfDp9/zoprPbJKWaMA Date: Fri, 6 Sep 2024 07:15:06 +0000 Message-ID: <3543971A-6C8E-4F28-A7D4-F8C222F3C193@nvidia.com> References: <20240905142602.1264345-1-maxime.coquelin@redhat.com> <20240905142602.1264345-3-maxime.coquelin@redhat.com> In-Reply-To: <20240905142602.1264345-3-maxime.coquelin@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CYYPR12MB8992:EE_|PH0PR12MB7471:EE_ x-ms-office365-filtering-correlation-id: 62cc0ecb-3afc-4e75-f6ff-08dcce43a295 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|376014|366016|1800799024|38070700018; x-microsoft-antispam-message-info: =?us-ascii?Q?jOPEweBQUeLeZ3YKN/N99zprZj5fOW5xHJVyUcizMShQkn2dj8aOJWVGpavl?= =?us-ascii?Q?1H8gmn4QRCCvN3wjyu8aJj7vwJ8pJLZctCVk1kvFWsN7RrX6C0DUFcx8dMWh?= =?us-ascii?Q?MnN97fWhXAX4Hbg2MFrAfq2hK6a3Mo1IN3xJN3S5LfHrFU1QL2hfHZIpylJm?= =?us-ascii?Q?kCt5sTfuu5yIhaUxJqKvtqO9iYEUd0H9TBuPi04YqyWxTC0sblnM884OODJw?= =?us-ascii?Q?Q5KEXaKKEtviYj5IBjQS2NZQp28eg7pW4ve3bZhzejIhz6ct63gfVl8hyTAl?= =?us-ascii?Q?CoVgOsqvFdqG3e7NV4gNpPVcz94TmrfzxVI46NxJzb6bsr1ILSdxF6WxJSlF?= =?us-ascii?Q?26In/kv7zud2ZGtihQpqxr2wUovy5cAI2FZQnTEhuGr4tAySBNHSRzD29FwC?= =?us-ascii?Q?qotwsiL3T6ngIen7/1K9p9U6s/k1xRRcAivhx4XMkwR0F8q6P4L7RmDTHkrk?= =?us-ascii?Q?HIEknu7YeGaspIe9qwwgyuhDAPsThnbJljsc5O97iyCBGygcULSuddqgJO7w?= =?us-ascii?Q?nKTQSADL6xj7BkkHr22XRe6emkySvcHJUQ7dpmyZAkuVA30Jbzze7caY4MxH?= =?us-ascii?Q?pbCJ9OhMr1sO7yeW+XY8VfmMlEAlhAC9+mmz1FqfEYFl0sSm2xye7npieJGM?= =?us-ascii?Q?pTiggZGrdM+e/JG8cMssrYLXSz20Q3Go6+B1stOniXqqHG5tGPh5wVGF7DRY?= =?us-ascii?Q?RMPbTEEbENAt+n+afz0CB5xv6RurkfJSu1qdPMDiHHgD4CuFiyf4Bnkudo6y?= =?us-ascii?Q?J0x3ZMCzOfGGneaNjkheXJcmbSn3w4Z9PxrU2m3DVpJ7TnYWyysOHjrCokQU?= =?us-ascii?Q?0+iy9ePQ5jQeiKfT8bE4/8nW8VKKlcw2FZRNr7fWO7c1Kq2FB1T6xn6vSX08?= =?us-ascii?Q?F4oXfIuL1+Ll0g2b78qqhyP0bNGyjS/nf5Tvz+9XeUFi9YPWyN8kbBGoIGc/?= =?us-ascii?Q?4uuqT9610Vkp1OSzVoz803jpFgeHDtjjtQ9c0oeETAnK7rkS01F8LCfqhUX4?= =?us-ascii?Q?gfm8CinEJL+B30Dz2oy5GoMCAX6iyrPH0NjNgCPay7H0z9aajqy/S8ZoIY0L?= =?us-ascii?Q?5KuOekKjkPterJWXSDDQxIA2MSXKmYP2gU4sGsrUcHGZRzmwkNIBEiGBBeYE?= =?us-ascii?Q?cSNxh+y7eXXOggZA1TP6o3MRIVpc7HhHF/2+5Wvs/sLLASsAWMRW5uNIFrX4?= =?us-ascii?Q?R5lFvAmcadqmrGdiDTQwOVVyDw1WL3WGY2WRvCE9HGGgCg51nk8J5S8OgSJ+?= =?us-ascii?Q?erINo/y3kCQNwqE6l3v3Wj4gHfpojcU6dt/eJuvBy8ZYVMYlSxBYtvy2jOEr?= =?us-ascii?Q?o2Iu2wxLyyxjbGQaw6UefYBudfMOrqBwljuz6kIC/bluQVpk8m5vp5k4VnF7?= =?us-ascii?Q?WlmP610MQETmLoA1VvyOXwsVXYg0kuz91zQjLv1JYGynqfxlcw=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CYYPR12MB8992.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024)(38070700018); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?im17JPFFvKXies+GaMG0p3ApcU8wSqJNA2N1ybliZAOUdow4FFHYWMdxmCWL?= =?us-ascii?Q?O5mQKn2zZ1qLa6GwyaHPqOFFP1NwFkGSOXGOjhL7wxJXZFfqDAt5JePK+T7e?= =?us-ascii?Q?0x6Zio4gxPn3vRPJ/lVNYXl5CVgiwoh7uQJAWnq74LszKICu+752OAliFgFF?= =?us-ascii?Q?lVWiz1Ts7/NAILshTSsYHXiruuU+YMR+tAt61MuDRVt3UBmY6IFoMxLTkara?= =?us-ascii?Q?2XG9n6d/c1ONdqovzcwkDzzXLm7CeWsHyBmWEMwuOXIWBl/1JkTUJqti6iJ9?= =?us-ascii?Q?QzjtH6JxGWfGPPehXZy+IVm7B9Q/wiF3DRN9cIWQeKSIQJQQx1hLmlE+4A4b?= =?us-ascii?Q?2eXExWOO3Nnk4d2HZNHwgM9TqMpcHq7xO9HRSsj2mHBHSw47aQA0YcesoCjn?= =?us-ascii?Q?HezZIMR/7inXxu2z4qZyu9lt2cZffBRYkSupyjhg//KEitIW/udy9BxwutSW?= =?us-ascii?Q?TTEzYlbinmkYYmAItZ7/2ED23tuCrMV2hapu1m217/wNnwyFfuDO5i3HU4QP?= =?us-ascii?Q?Gilm90Mk+4dIGnVgfYkPd5ZCMUVQGgfSV4m4eHfksd+oHZkW7wQz+GDIfGSD?= =?us-ascii?Q?tg5tduccCJ2QqffjCA0+/+G7W+d8soorl26r1zdQMFX88HT+eojT/jd+3fZn?= =?us-ascii?Q?pYSIZwv6uc1fnSzoorBBfe6d8HyXWZBmIHkaBjldZUMwM48Yf2vxH1Em09Bb?= =?us-ascii?Q?+Yl/tGkt2D75Q3tKKN57yfhAcQrpkqvBpRXf6nxzUHm7A6PCSN5tOjqZCfEM?= =?us-ascii?Q?CXvJuBkcnIgX1Cy0gR1qnOc6+xgiJKyJNWV0ifVk+KQPYj83TYRZZSPl9cS+?= =?us-ascii?Q?nOfvHT6HwG4KK/DUz9EK4IJV484xJvSO720Qs/IIrFJxngi7da1AZIANem5r?= =?us-ascii?Q?lH5XG49D3SDK4QyuB9y8ldUFl5/KbIlBQFRpsT2OhzN7DHSkZX80kz586Hrj?= =?us-ascii?Q?JsVbcMsrMhy88YsB21ZHCLGl9WNdwqMuVPjrYMYYEf0si677SInr9qh4QcW/?= =?us-ascii?Q?PAfvRTK5HnELphhapbXqn7Hlne8ENvQzXyFvRAJr7ZQw5JEw7DiLEAA3HXOK?= =?us-ascii?Q?kw+AC993HeC0nzdreZ9wNziUnNiJHFE2FzYUgN14hzfOmYpcNhyzXDMipsKI?= =?us-ascii?Q?GRCVhFwLfyTaFCoYEHVwvze4u8R0feHrPN6khZ0BziEnwMDLPqyiCAhSjYwp?= =?us-ascii?Q?nvc3oeH/MUcnBSaZFu1tBOQDfp0SW0XJEp4RnfXs46fNLk15VXCQDCcyGu1N?= =?us-ascii?Q?5g9Z4JX2/vjSYxRrWShNsiN8qKLKu9g8/1re9Sr2kKAo6gY4udzn0GsWQ6qc?= =?us-ascii?Q?NGVSEmBj7lYNeraOH1Ff2vibRuiPyQcfj/jmJc/GRdWPFHg7mCkcaDAp4m2x?= =?us-ascii?Q?RbvxhlGlz5DPJ14zbOVaoSmwgiUO2MfaBRhtijRWDKou6F8u8dZbAbkPOcgK?= =?us-ascii?Q?I4+yCJUqh7wOA/0hKCzQmomIJydnMVoEa+cZBltkznUXGRnERqOAVVD3ziRa?= =?us-ascii?Q?v1gOqji+ouj7yayjmQpY2b+C8qUlf56b/k61coGEmGWvgJ7541yhebOlrdMp?= =?us-ascii?Q?5rliWYK3VUgztJS4OCijEjJHX7Za0jY03wBq9AWhwxcUDcDhc2DgzB81zX8O?= =?us-ascii?Q?Og=3D=3D?= Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CYYPR12MB8992.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 62cc0ecb-3afc-4e75-f6ff-08dcce43a295 X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Sep 2024 07:15:06.5533 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bxdSsuYWXbzi3vvTT7Xj+de7dh/eBJZ0gTUB40YDF6oxWSxAB5Ir73qoBr38G4ksUvn0nvvpNxqvyt8V4u91eg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7471 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Message-ID: <20240906071506.8d2Z9tzv5a1g9gKjZjs_i4VvFRq5KCUHgF0Gd_zjpMI@z> Hi Maxime, > On Sep 5, 2024, at 22:26, Maxime Coquelin wr= ote: >=20 > External email: Use caution opening links or attachments >=20 >=20 > This patch enables VDUSE reconnection support making use of > the newly introduced reconnection mechanism in Vhost > library. >=20 > At DPDK VDUSE device creation time, there are two > possibilities: > 1. The Kernel VDUSE device does not exist: > a. A reconnection file named after the VUDSE device name > is created in VDUSE tmpfs. > b. The file is truncated to 'struct vhost_reconnect_data' > size, and mmapped. > c. Negotiated features, Virtio status... are saved for > sanity checks at reconnect time. > 2. The Kernel VDUSE device already exists: > a. Exit with failure if no reconnect file exists for > this device. > b. Open and mmap the reconnect file. > c. Perform sanity check to ensure features are compatible. > d. Restore virtqueues' available indexes at startup time. >=20 > Then at runtime, the virtqueues' available index are logged by > the Vhost reconnection mechanism. >=20 > At DPDK VDUSE device destruction time, there are two > possibilities: > 1. The Kernel VDUSE device destruction succeed, which > means it is no more attached to the vDPA bus. The > reconnection file is unmapped and then removed. > 2. The Kernel VDUSE device destruction failed, meaning it > is no more attached to the vDPA bus. The reconnection > file is unmapped but not removed to make possible later > reconnection. >=20 > Signed-off-by: Maxime Coquelin > --- > lib/vhost/vduse.c | 280 +++++++++++++++++++++++++++++++++++++++------- > 1 file changed, 241 insertions(+), 39 deletions(-) >=20 > diff --git a/lib/vhost/vduse.c b/lib/vhost/vduse.c > index c66602905c..bd0e492d62 100644 > --- a/lib/vhost/vduse.c > +++ b/lib/vhost/vduse.c > @@ -136,7 +136,7 @@ vduse_control_queue_event(int fd, void *arg, int *rem= ove __rte_unused) > } >=20 > static void > -vduse_vring_setup(struct virtio_net *dev, unsigned int index) > +vduse_vring_setup(struct virtio_net *dev, unsigned int index, bool recon= nect) > { > struct vhost_virtqueue *vq =3D dev->virtqueue[index]; > struct vhost_vring_addr *ra =3D &vq->ring_addrs; > @@ -152,6 +152,19 @@ vduse_vring_setup(struct virtio_net *dev, unsigned i= nt index) > return; > } >=20 > + if (reconnect) { > + vq->last_avail_idx =3D vq->reconnect_log->last_avail_idx; > + vq->last_used_idx =3D vq->reconnect_log->last_avail_idx; > + } else { > + vq->last_avail_idx =3D vq_info.split.avail_index; > + vq->last_used_idx =3D vq_info.split.avail_index; > + } > + vq->size =3D vq_info.num; > + vq->ready =3D true; > + vq->enabled =3D vq_info.ready; > + ra->desc_user_addr =3D vq_info.desc_addr; > + ra->avail_user_addr =3D vq_info.driver_addr; > + ra->used_user_addr =3D vq_info.device_addr; > VHOST_CONFIG_LOG(dev->ifname, INFO, "VQ %u info:", index); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tnum: %u", vq_info.num); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tdesc_addr: %llx", > @@ -162,15 +175,6 @@ vduse_vring_setup(struct virtio_net *dev, unsigned i= nt index) > (unsigned long long)vq_info.device_addr); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tavail_idx: %u", vq_info.spl= it.avail_index); > VHOST_CONFIG_LOG(dev->ifname, INFO, "\tready: %u", vq_info.ready); > - > - vq->last_avail_idx =3D vq_info.split.avail_index; > - vq->size =3D vq_info.num; > - vq->ready =3D true; > - vq->enabled =3D vq_info.ready; > - ra->desc_user_addr =3D vq_info.desc_addr; > - ra->avail_user_addr =3D vq_info.driver_addr; > - ra->used_user_addr =3D vq_info.device_addr; > - > vq->kickfd =3D eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); > if (vq->kickfd < 0) { > VHOST_CONFIG_LOG(dev->ifname, ERR, "Failed to init kickfd = for VQ %u: %s", > @@ -267,7 +271,7 @@ vduse_vring_cleanup(struct virtio_net *dev, unsigned = int index) > } >=20 > static void > -vduse_device_start(struct virtio_net *dev) > +vduse_device_start(struct virtio_net *dev, bool reconnect) > { > unsigned int i, ret; >=20 > @@ -287,6 +291,15 @@ vduse_device_start(struct virtio_net *dev) > return; > } >=20 > + if (reconnect && dev->features !=3D dev->reconnect_log->features)= { > + VHOST_CONFIG_LOG(dev->ifname, ERR, > + "Mismatch between reconnect file features= 0x%" PRIx64 " & device features 0x%" PRIx64, Checkpatch reports long line > + dev->reconnect_log->features, dev->featur= es); > + return; > + } > + > + dev->reconnect_log->features =3D dev->features; > + > VHOST_CONFIG_LOG(dev->ifname, INFO, "Negotiated Virtio features: 0= x%" PRIx64, > dev->features); >=20 > @@ -300,7 +313,7 @@ vduse_device_start(struct virtio_net *dev) > } >=20 > for (i =3D 0; i < dev->nr_vring; i++) > - vduse_vring_setup(dev, i); > + vduse_vring_setup(dev, i, reconnect); >=20 > dev->flags |=3D VIRTIO_DEV_READY; >=20 > @@ -373,6 +386,7 @@ vduse_events_handler(int fd, void *arg, int *remove _= _rte_unused) > req.s.status); > old_status =3D dev->status; > dev->status =3D req.s.status; > + dev->reconnect_log->status =3D dev->status; > resp.result =3D VDUSE_REQ_RESULT_OK; > break; > case VDUSE_UPDATE_IOTLB: > @@ -398,7 +412,7 @@ vduse_events_handler(int fd, void *arg, int *remove _= _rte_unused) >=20 > if ((old_status ^ dev->status) & VIRTIO_DEVICE_STATUS_DRIVER_OK) { > if (dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) > - vduse_device_start(dev); > + vduse_device_start(dev, false); > else > vduse_device_stop(dev); > } > @@ -407,10 +421,64 @@ vduse_events_handler(int fd, void *arg, int *remove= __rte_unused) > vduse_req_id_to_str(req.type), req.type); > } >=20 > +static char vduse_reconnect_dir[PATH_MAX]; > +static bool vduse_reconnect_path_set; > + > +static int > +vduse_reconnect_path_init(void) > +{ > + const char *directory; > + int ret; > + > + /* from RuntimeDirectory=3D see systemd.exec */ > + directory =3D getenv("RUNTIME_DIRECTORY"); > + if (directory =3D=3D NULL) { > + /* > + * Used standard convention defined in > + * XDG Base Directory Specification and > + * Filesystem Hierarchy Standard. > + */ > + if (getuid() =3D=3D 0) > + directory =3D "/var/run"; > + else > + directory =3D getenv("XDG_RUNTIME_DIR") ? : "/tmp= "; > + } > + > + ret =3D snprintf(vduse_reconnect_dir, sizeof(vduse_reconnect_dir)= , "%s/dpdk/vduse", > + directory); > + if (ret < 0 || ret =3D=3D sizeof(vduse_reconnect_dir)) { > + VHOST_CONFIG_LOG("vduse", ERR, "Error creating VDUSE reco= nnect path name"); > + return -1; > + } > + > + ret =3D mkdir(vduse_reconnect_dir, 0700); > + if (ret < 0 && errno !=3D EEXIST) { > + VHOST_CONFIG_LOG("vduse", ERR, "Error creating '%s': %s", > + vduse_reconnect_dir, strerror(errno)); > + return -1; > + } > + > + VHOST_CONFIG_LOG("vduse", INFO, "Created VDUSE reconnect director= y in %s", > + vduse_reconnect_dir); > + > + return 0; > +} > + > +static void > +vduse_reconnect_handler(int fd, void *arg, int *remove) > +{ > + struct virtio_net *dev =3D arg; > + > + vduse_device_start(dev, true); > + > + close(fd); > + *remove =3D 1; > +} > + > int > vduse_device_create(const char *path, bool compliant_ol_flags) > { > - int control_fd, dev_fd, vid, ret; > + int control_fd, dev_fd, vid, ret, reco_fd; > uint32_t i, max_queue_pairs, total_queues; > struct virtio_net *dev; > struct virtio_net_config vnet_config =3D {{ 0 }}; > @@ -418,6 +486,9 @@ vduse_device_create(const char *path, bool compliant_= ol_flags) > uint64_t features; > struct vduse_dev_config *dev_config =3D NULL; > const char *name =3D path + strlen("/dev/vduse/"); > + char reconnect_file[PATH_MAX]; > + struct vhost_reconnect_data *reconnect_log =3D NULL; > + bool reconnect =3D false; >=20 > if (vduse.fdset =3D=3D NULL) { > vduse.fdset =3D fdset_init("vduse-evt"); > @@ -427,6 +498,20 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > } > } >=20 > + if (vduse_reconnect_path_set =3D=3D false) { > + if (vduse_reconnect_path_init() < 0) { > + VHOST_CONFIG_LOG(path, ERR, "failed to initialize= reconnect path"); > + return -1; > + } > + vduse_reconnect_path_set =3D true; > + } > + > + ret =3D snprintf(reconnect_file, sizeof(reconnect_file), "%s/%s",= vduse_reconnect_dir, name); > + if (ret < 0 || ret =3D=3D sizeof(reconnect_file)) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to create vduse recon= nect path name"); > + return -1; > + } > + > control_fd =3D open(VDUSE_CTRL_PATH, O_RDWR); > if (control_fd < 0) { > VHOST_CONFIG_LOG(name, ERR, "Failed to open %s: %s", > @@ -441,14 +526,6 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > goto out_ctrl_close; > } >=20 > - dev_config =3D malloc(offsetof(struct vduse_dev_config, config) + > - sizeof(vnet_config)); > - if (!dev_config) { > - VHOST_CONFIG_LOG(name, ERR, "Failed to allocate VDUSE con= fig"); > - ret =3D -1; > - goto out_ctrl_close; > - } > - > ret =3D rte_vhost_driver_get_features(path, &features); > if (ret < 0) { > VHOST_CONFIG_LOG(name, ERR, "Failed to get backend feature= s"); > @@ -469,23 +546,97 @@ vduse_device_create(const char *path, bool complian= t_ol_flags) > else > total_queues +=3D 1; /* Includes ctrl queue */ >=20 > - vnet_config.max_virtqueue_pairs =3D max_queue_pairs; > - memset(dev_config, 0, sizeof(struct vduse_dev_config)); > + if (access(path, F_OK) =3D=3D 0) { > + VHOST_CONFIG_LOG(name, INFO, "Device already exists, reco= nnecting..."); > + reconnect =3D true; > + > + reco_fd =3D open(reconnect_file, O_RDWR, 0600); > + if (reco_fd < 0) { > + if (errno =3D=3D ENOENT) { > + VHOST_CONFIG_LOG(name, ERR, "Missing reco= nnect file (%s)", > + reconnect_file); > + } else { > + VHOST_CONFIG_LOG(name, ERR, "Failed to op= en reconnect file %s (%s)", > + reconnect_file, strerror(= errno)); > + } Seems no {} is needed for if-else? > + ret =3D -1; > + goto out_ctrl_close; > + } > + > + reconnect_log =3D mmap(NULL, sizeof(*reconnect_log), PROT= _READ | PROT_WRITE, > + MAP_SHARED, reco_fd, 0); > + close(reco_fd); > + if (reconnect_log =3D=3D MAP_FAILED) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to mmap recon= nect file %s (%s)", > + reconnect_file, strerror(errno)); > + ret =3D -1; > + goto out_ctrl_close; > + } > + } else { > + reco_fd =3D open(reconnect_file, O_CREAT | O_EXCL | O_RDW= R, 0600); > + if (reco_fd < 0) { > + if (errno =3D=3D EEXIST) { > + VHOST_CONFIG_LOG(name, ERR, "Reconnect fi= le %s exists but not the device", > + reconnect_file); > + } else { > + VHOST_CONFIG_LOG(name, ERR, "Failed to op= en reconnect file %s (%s)", > + reconnect_file, strerror(= errno)); > + } > + ret =3D -1; > + goto out_ctrl_close; > + } >=20 > - strncpy(dev_config->name, name, VDUSE_NAME_MAX - 1); > - dev_config->device_id =3D VIRTIO_ID_NET; > - dev_config->vendor_id =3D 0; > - dev_config->features =3D features; > - dev_config->vq_num =3D total_queues; > - dev_config->vq_align =3D sysconf(_SC_PAGE_SIZE); > - dev_config->config_size =3D sizeof(struct virtio_net_config); > - memcpy(dev_config->config, &vnet_config, sizeof(vnet_config)); > + ret =3D ftruncate(reco_fd, sizeof(*reconnect_log)); > + if (ret < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to truncate r= econnect file %s (%s)", > + reconnect_file, strerror(errno)); > + close(reco_fd); > + goto out_ctrl_close; > + } >=20 > - ret =3D ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); > - if (ret < 0) { > - VHOST_CONFIG_LOG(name, ERR, "Failed to create VDUSE devic= e: %s", > - strerror(errno)); > - goto out_free; > + reconnect_log =3D mmap(NULL, sizeof(*reconnect_log), PROT= _READ | PROT_WRITE, > + MAP_SHARED, reco_fd, 0); > + close(reco_fd); > + if (reconnect_log =3D=3D MAP_FAILED) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to mmap recon= nect file %s (%s)", > + reconnect_file, strerror(errno)); > + ret =3D -1; > + goto out_ctrl_close; > + } > + > + reconnect_log->version =3D 0; > + > + dev_config =3D malloc(offsetof(struct vduse_dev_config, c= onfig) + > + sizeof(vnet_config)); > + if (!dev_config) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to allocate V= DUSE config"); > + ret =3D -1; > + goto out_ctrl_close; > + } > + > + vnet_config.max_virtqueue_pairs =3D max_queue_pairs; > + memset(dev_config, 0, sizeof(struct vduse_dev_config)); > + > + rte_strscpy(dev_config->name, name, VDUSE_NAME_MAX - 1); > + dev_config->device_id =3D VIRTIO_ID_NET; > + dev_config->vendor_id =3D 0; > + dev_config->features =3D features; > + dev_config->vq_num =3D total_queues; > + dev_config->vq_align =3D sysconf(_SC_PAGE_SIZE); > + dev_config->config_size =3D sizeof(struct virtio_net_conf= ig); > + memcpy(dev_config->config, &vnet_config, sizeof(vnet_conf= ig)); > + > + ret =3D ioctl(control_fd, VDUSE_CREATE_DEV, dev_config); > + if (ret < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to create VDU= SE device: %s", > + strerror(errno)); > + goto out_free; > + } > + > + memcpy(&reconnect_log->config, &vnet_config, sizeof(vnet_= config)); > + reconnect_log->nr_vrings =3D total_queues; > + free(dev_config); > + dev_config =3D NULL; > } >=20 > dev_fd =3D open(path, O_RDWR); > @@ -519,10 +670,15 @@ vduse_device_create(const char *path, bool complian= t_ol_flags) > strncpy(dev->ifname, path, IF_NAME_SZ - 1); > dev->vduse_ctrl_fd =3D control_fd; > dev->vduse_dev_fd =3D dev_fd; > + dev->reconnect_log =3D reconnect_log; > + if (reconnect) > + dev->status =3D dev->reconnect_log->status; > + > vhost_setup_virtio_net(dev->vid, true, compliant_ol_flags, true, t= rue); >=20 > for (i =3D 0; i < total_queues; i++) { > struct vduse_vq_config vq_cfg =3D { 0 }; > + struct vhost_virtqueue *vq; >=20 > ret =3D alloc_vring_queue(dev, i); > if (ret) { > @@ -530,6 +686,12 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > goto out_dev_destroy; > } >=20 > + vq =3D dev->virtqueue[i]; > + vq->reconnect_log =3D &reconnect_log->vring[i]; > + > + if (reconnect) > + continue; > + > vq_cfg.index =3D i; > vq_cfg.max_size =3D 1024; >=20 > @@ -549,7 +711,28 @@ vduse_device_create(const char *path, bool compliant= _ol_flags) > goto out_dev_destroy; > } >=20 > - free(dev_config); > + if (reconnect && dev->status & VIRTIO_DEVICE_STATUS_DRIVER_OK) { > + reco_fd =3D eventfd(0, EFD_NONBLOCK | EFD_CLOEXEC); > + if (reco_fd < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to create rec= o_fd: %s", > + strerror(errno)); > + ret =3D -1; > + goto out_dev_destroy; > + } > + > + ret =3D fdset_add(vduse.fdset, reco_fd, vduse_reconnect_h= andler, NULL, dev); > + if (ret) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to add fd %d = to vduse fdset", > + dev->vduse_dev_fd); Should print reco_fd > + goto out_dev_destroy; > + } > + > + ret =3D eventfd_write(reco_fd, (eventfd_t)1); > + if (ret < 0) { > + VHOST_CONFIG_LOG(name, ERR, "Failed to write to r= econnect eventfd"); > + goto out_dev_destroy; > + } > + } Maybe I missed something, why we need to implement like this instead of dir= ectly call vduse_device_start? Thanks, Chenbo >=20 > return 0; >=20 > @@ -587,6 +770,9 @@ vduse_device_destroy(const char *path) > if (vid =3D=3D RTE_MAX_VHOST_DEVICE) > return -1; >=20 > + if (dev->reconnect_log) > + munmap(dev->reconnect_log, sizeof(*dev->reconnect_log)); > + > vduse_device_stop(dev); >=20 > fdset_del(vduse.fdset, dev->vduse_dev_fd); > @@ -597,10 +783,26 @@ vduse_device_destroy(const char *path) > } >=20 > if (dev->vduse_ctrl_fd >=3D 0) { > + char reconnect_file[PATH_MAX]; > + > ret =3D ioctl(dev->vduse_ctrl_fd, VDUSE_DESTROY_DEV, name)= ; > - if (ret) > + if (ret) { > VHOST_CONFIG_LOG(name, ERR, "Failed to destroy VDU= SE device: %s", > strerror(errno)); > + } else { > + /* > + * VDUSE device was no more attached to the vDPA = bus, > + * so we can remove the reconnect file. > + */ > + ret =3D snprintf(reconnect_file, sizeof(reconnect= _file), "%s/%s", > + vduse_reconnect_dir, name); > + if (ret < 0 || ret =3D=3D sizeof(reconnect_file)) > + VHOST_CONFIG_LOG(name, ERR, > + "Failed to create vduse r= econnect path name"); > + else > + unlink(reconnect_file); > + } > + > close(dev->vduse_ctrl_fd); > dev->vduse_ctrl_fd =3D -1; > } > -- > 2.46.0 >=20