From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9260CA00C3; Fri, 13 May 2022 12:10:13 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8364E4280B; Fri, 13 May 2022 12:10:13 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id DAA2E4114F for ; Fri, 13 May 2022 12:10:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1652436612; x=1683972612; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=RN/Qw63QWycoTWYM+JTcqiVJESztz22RiJYnqcIamCc=; b=OxYR5s8IQCDHq+qihC1YL32RY6+bAYsma/KfU8p0WQhkm7HOZeQFskoD 5+ShbDYvTj1OUH4VGA2Avls9uHShxKeTpR0A1BjtTrMDSzh956hgYWm3y 8LGTjqZJVJE8sP5Ur0g6EmDiPRG4+RnWnfXwuM9/SEHjPSd4YDsRfNxW8 Ea4z2LyBWgd2YzoUF4QphDD3YiAfuYzG+GPU/l8zJkCRhxDwMwcMXN7/A jwDGfCaKFvzOydLCzxorKj7/lH4K+jIdGs3zsfhTNfUC55b3rkYUrfnm/ edr5gWN1aVQC6byk/1l36y/tSKslkouOawb1j9+Ra4du2NCuc5MpU8paz Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10345"; a="257815638" X-IronPort-AV: E=Sophos;i="5.91,221,1647327600"; d="scan'208";a="257815638" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 May 2022 03:10:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.91,221,1647327600"; d="scan'208";a="603790334" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by orsmga001.jf.intel.com with ESMTP; 13 May 2022 03:10:10 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Fri, 13 May 2022 03:10:10 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Fri, 13 May 2022 03:10:09 -0700 Received: from fmsmsx612.amr.corp.intel.com ([10.18.126.92]) by fmsmsx612.amr.corp.intel.com ([10.18.126.92]) with mapi id 15.01.2308.027; Fri, 13 May 2022 03:10:09 -0700 From: "Pei, Andy" To: "Xia, Chenbo" , "dev@dpdk.org" CC: "maxime.coquelin@redhat.com" , "Cao, Gang" , "Liu, Changpeng" Subject: RE: [PATCH v7 14/18] vdpa/ifc: add interrupt and handle for virtio blk Thread-Topic: [PATCH v7 14/18] vdpa/ifc: add interrupt and handle for virtio blk Thread-Index: AQHYWheJNfItJEBMQUy4XKS8tVemhK0cLmEwgABzm0A= Date: Fri, 13 May 2022 10:10:09 +0000 Message-ID: References: <1643093258-47258-2-git-send-email-andy.pei@intel.com> <1651048206-282372-1-git-send-email-andy.pei@intel.com> <1651048206-282372-15-git-send-email-andy.pei@intel.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.6.401.20 dlp-reaction: no-action x-originating-ip: [10.239.127.36] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org HI Chenbo, Thanks for your reply. My reply is inline. > -----Original Message----- > From: Xia, Chenbo > Sent: Friday, May 13, 2022 10:53 AM > To: Pei, Andy ; dev@dpdk.org > Cc: maxime.coquelin@redhat.com; Cao, Gang ; Liu, > Changpeng > Subject: RE: [PATCH v7 14/18] vdpa/ifc: add interrupt and handle for virt= io > blk >=20 > > -----Original Message----- > > From: Pei, Andy > > Sent: Wednesday, April 27, 2022 4:30 PM > > To: dev@dpdk.org > > Cc: Xia, Chenbo ; maxime.coquelin@redhat.com; > > Cao, Gang ; Liu, Changpeng > > > > Subject: [PATCH v7 14/18] vdpa/ifc: add interrupt and handle for > > virtio blk >=20 > Better be: vdpa/ifc: add interrupt handling for config space >=20 Sure. I will fix it in next version. > > > > Create a thread to poll and relay config space change interrupt. > > Use VHOST_USER_SLAVE_CONFIG_CHANGE_MSG to info qemu. >=20 > Inform QEMU. You don't need to save words in commit log. The commit log > should be as detailed as possible to make readers understand quickly what > the commit is doing :) >=20 Sure. I will fix it in next version. > > > > Signed-off-by: Andy Pei > > --- > > drivers/vdpa/ifc/ifcvf_vdpa.c | 112 > > ++++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 112 insertions(+) > > > > diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c > > b/drivers/vdpa/ifc/ifcvf_vdpa.c index 5a8cf1c..0e94e1f 100644 > > --- a/drivers/vdpa/ifc/ifcvf_vdpa.c > > +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c > > @@ -53,7 +53,9 @@ struct ifcvf_internal { > > int vfio_group_fd; > > int vfio_dev_fd; > > pthread_t tid; /* thread for notify relay */ > > + pthread_t intr_tid; /* thread for intr relay */ >=20 > Thread for virtio-blk config space change interrupt relay >=20 Sure. > > int epfd; > > + int csc_fd; >=20 > csc_epfd >=20 OK. > > int vid; > > struct rte_vdpa_device *vdev; > > uint16_t max_queues; > > @@ -558,6 +560,107 @@ struct rte_vdpa_dev_info { > > return 0; > > } > > > > +static void > > +virtio_interrupt_handler(struct ifcvf_internal *internal) { > > + int vid =3D internal->vid; > > + int ret; > > + > > + ret =3D rte_vhost_slave_config_change(vid, 1); > > + if (ret) > > + DRV_LOG(ERR, "failed to notify the guest about configuration > > space change."); > > +} > > + > > +static void * > > +intr_relay(void *arg) > > +{ > > + struct ifcvf_internal *internal =3D (struct ifcvf_internal *)arg; > > + struct epoll_event csc_event; > > + struct epoll_event ev; > > + uint64_t buf; > > + int nbytes; > > + int csc_fd, csc_val =3D 0; > > + > > + csc_fd =3D epoll_create(1); > > + if (csc_fd < 0) { > > + DRV_LOG(ERR, "failed to create epoll for config space > > change."); > > + return NULL; > > + } > > + > > + ev.events =3D EPOLLIN | EPOLLPRI | EPOLLRDHUP | EPOLLHUP; > > + ev.data.fd =3D rte_intr_fd_get(internal->pdev->intr_handle); > > + if (epoll_ctl(csc_fd, EPOLL_CTL_ADD, > > + rte_intr_fd_get(internal->pdev->intr_handle), &ev) < 0) { > > + DRV_LOG(ERR, "epoll add error: %s", strerror(errno)); > > + return NULL; >=20 > Close the epfd and set to -1 if err. >=20 I check other epoll usage in DPDK, it seems most usage do not close the epf= d and set to -1. I am not sure whether it is needed. > > + } > > + > > + internal->csc_fd =3D csc_fd; > > + > > + for (;;) { > > + csc_val =3D epoll_wait(csc_fd, &csc_event, 1, -1); > > + if (csc_val < 0) { > > + if (errno =3D=3D EINTR) > > + continue; > > + DRV_LOG(ERR, "epoll_wait return fail\n"); >=20 > Save '\n', it's not needed for DRV_LOG. Please check other DRV_LOGs >=20 OK > > + return NULL; > > + } else if (csc_val =3D=3D 0) { > > + continue; > > + } else { > > + /* csc_val > 0 */ > > + nbytes =3D read(csc_event.data.fd, &buf, 8); > > + if (nbytes < 0) { > > + if (errno =3D=3D EINTR || errno =3D=3D EWOULDBLOCK) >=20 > EAGAIN should also be this case? >=20 Yes, it will be add in next version. > > + continue; > > + DRV_LOG(ERR, "Error reading from file > > descriptor %d: %s\n", > > + csc_event.data.fd, > > + strerror(errno)); > > + return NULL; > > + } else if (nbytes =3D=3D 0) { > > + DRV_LOG(ERR, "Read nothing from file > > descriptor %d\n", > > + csc_event.data.fd); > > + continue; > > + } else { > > + virtio_interrupt_handler(internal); > > + } > > + } > > + } > > + > > + return NULL; > > +} > > + > > +static int > > +setup_intr_relay(struct ifcvf_internal *internal) { > > + int ret; > > + > > + ret =3D pthread_create(&internal->intr_tid, NULL, intr_relay, > > + (void *)internal); >=20 > EAL API: rte_ctrl_thread_create, will be preferred. >=20 Sure, I will use " rte_ctrl_thread_create " in next version. > > + if (ret) { > > + DRV_LOG(ERR, "failed to create notify relay pthread."); > > + return -1; > > + } > > + return 0; > > +} > > + > > +static int > > +unset_intr_relay(struct ifcvf_internal *internal) { > > + void *status; > > + > > + if (internal->intr_tid) { > > + pthread_cancel(internal->intr_tid); > > + pthread_join(internal->intr_tid, &status); > > + } > > + internal->intr_tid =3D 0; > > + > > + if (internal->csc_fd >=3D 0) > > + close(internal->csc_fd); > > + internal->csc_fd =3D -1; > > + > > + return 0; > > +} > > + > > static int > > update_datapath(struct ifcvf_internal *internal) { @@ -584,10 > > +687,16 @@ struct rte_vdpa_dev_info { > > if (ret) > > goto err; > > > > + ret =3D setup_intr_relay(internal); > > + if (ret) > > + goto err; > > + >=20 > But this is not needed for net, right? As I said, we should include valid= ation > for net also. >=20 > Thanks, > Chenbo >=20 For net device, especially the harden virtio device, fabric plug in or out= will cause config change. I think net device may also need this interrupt, but I am not sure. > > rte_atomic32_set(&internal->running, 1); > > } else if (rte_atomic32_read(&internal->running) && > > (!rte_atomic32_read(&internal->started) || > > !rte_atomic32_read(&internal->dev_attached))) { > > + ret =3D unset_intr_relay(internal); > > + > > ret =3D unset_notify_relay(internal); > > if (ret) > > goto err; > > @@ -880,6 +989,9 @@ struct rte_vdpa_dev_info { > > /* stop the direct IO data path */ > > unset_notify_relay(internal); > > vdpa_ifcvf_stop(internal); > > + > > + unset_intr_relay(internal); > > + > > vdpa_disable_vfio_intr(internal); > > > > ret =3D rte_vhost_host_notifier_ctrl(vid, RTE_VHOST_QUEUE_ALL, > false); > > -- > > 1.8.3.1 >=20