From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DD07EA0351; Thu, 6 Aug 2020 12:28:23 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C897E1C0C3; Thu, 6 Aug 2020 12:28:23 +0200 (CEST) Received: from CNSHJSMIN03.NOKIA-SBELL.COM (cnshjsmin03.app.nokia-sbell.com [116.246.26.71]) by dpdk.org (Postfix) with ESMTP id 9BCBE2BB9 for ; Fri, 31 Jul 2020 12:15:29 +0200 (CEST) X-AuditID: ac189297-db1ff7000001b940-57-5f23ef3ea869 Received: from CNSHPPEXCH1603.nsn-intra.net (Unknown_Domain [135.251.51.103]) (using TLS with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by CNSHJSMIN03.NOKIA-SBELL.COM (Symantec Messaging Gateway) with SMTP id 26.81.47424.E3FE32F5; Fri, 31 Jul 2020 18:15:26 +0800 (HKT) Received: from CNSHPPEXCH1601.nsn-intra.net (135.251.51.101) by CNSHPPEXCH1603.nsn-intra.net (135.251.51.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1847.3; Fri, 31 Jul 2020 18:15:25 +0800 Received: from CNSHPPEXCH1601.nsn-intra.net ([135.251.51.101]) by CNSHPPEXCH1601.nsn-intra.net ([135.251.51.101]) with mapi id 15.01.1847.007; Fri, 31 Jul 2020 18:15:25 +0800 From: "Yan, Xiaoping (NSB - CN/Hangzhou)" To: "tiwei.bie@intel.com" , "dev@dpdk.org" Thread-Topic: interrupt mode issue for virtio device Thread-Index: AdZnHufx4Oal14w3Qie12raAWKZYsQ== Date: Fri, 31 Jul 2020 10:15:25 +0000 Message-ID: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [135.251.51.115] MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrGLMWRmVeSWpSXmKPExsXS/ts4XdfuvXK8wdKFmhbvPm1nstja8J/J gcnj14KlrB6L97xkCmCK4rJJSc3JLEst0rdL4Mo4d+Era8GmqYwVm3eqNTBuambsYuTkkBAw kZh++QCQzcUhJHCISeLxjsfMEM5fRomTV06wQjibGCWmfPzPAtLCJuAh8eLEVmYQW0TAT2LL nMesILawgL7Eukmf2CHiJhJ3er6zdTFyANl6Ej3380DCLAKqErde94ON4RWwk3j96S+YzSgg KzHt0X0mEJtZQFzi1pP5TBDXCUgs2XOeGcIWlXj5+B8ryEgJASWJvg1Q5akSx6dcZ4QYKShx cuYTlgmMQrOQTJqFpGwWkjKIuI7Egt2f2CBsbYllC18zw9hnDjxmQhZfwMi+ilHa2S/YwyvY 19PPwFjPz9/b01E32MnVx0fP2d93EyMwatZITJq+g/H5rA96hxiZOBgPMUpwMCuJ8LZzKcQL 8aYkVlalFuXHF5XmpBYfYpTmYFES5523SD5eSCA9sSQ1OzW1ILUIJsvEwSnVwDSN+X//f7lF Zc2z9ijVKZckOIpfElU/k7GIfalmRfgP1uwItYhC2YUpnH/s+/dosj6aJ3qsryX744XAHS8u vNtXWrlS9+bmer6/8XMSgoKeSv3fm3NIVEPeiz2z7tULf6VG7b81J0U6fzT+nDv97OW1768K lWq7TpbJXnTM9AL/10tWQf/cGzS9jvZuYdOdti1+bZHP38e8M6aYH2QqMjiqqXBu1uHv//T7 9nyf4PUmK0tgesmxQoeTRvHFh31mpp4wn2f471eR8iXryYlzLXk26TuwbPvuE1j8/1dkq46y xIVYn4Rt81xPzTpSNZXjUfUllmszPl+/Vedf92Rl1fRVs1Nb1sqeTdZKPpdSf1WJpTgj0VCL uag4EQDXJk9MCQMAAA== X-Mailman-Approved-At: Thu, 06 Aug 2020 12:28:22 +0200 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] interrupt mode issue for virtio device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, I have a dpdk application run in interrupt mode(like examples\l3fwd-power) = for a virtio device Network devices using DPDK-compatible driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:00:04.0 'Virtio network device 1000' drv=3Dvfio-pci unused=3Digb_uio It works (application is waken up when packet come in) in some host infrast= ructure, but doesn't work ((application is NOT waken up when packet come in= )) in some other. In the host where it doesn't work, I found the interrupt increases in cat /= proc/interrupts for this device. It seems the fd in efds (value 34) mismatch the fds added to epfd (value 33= ) where my application is wating. The other 2 ports (they are sriov VF device) works normally. (gdb) print *(rte_eth_devices[0]->intr_handle) $1 =3D {{vfio_dev_fd =3D 13, uio_cfg_fd =3D 13}, fd =3D 14, type =3D RTE_IN= TR_HANDLE_VFIO_MSIX, max_intr =3D 2, nb_efd =3D 1, efd_counter_size =3D 0 '\000', efds =3D {34, 0 }, elist= =3D {{status =3D 1, fd =3D 21, epfd =3D 33, epdata =3D { event =3D 2147483651, data =3D 0x0, cb_fun =3D 0x6c1b10 , cb_arg =3D 0x7a592f8}}, {status =3D 0, fd =3D 0, epfd =3D 0, epdata =3D {event =3D 0, data =3D 0x0, cb_fun =3D 0x0, cb= _arg =3D 0x0}} }, intr_vec =3D 0x2ab525a0f80} (gdb) print *(rte_eth_devices[1]->intr_handle) $2 =3D {{vfio_dev_fd =3D 16, uio_cfg_fd =3D 16}, fd =3D 17, type =3D RTE_IN= TR_HANDLE_VFIO_MSIX, max_intr =3D 2, nb_efd =3D 1, efd_counter_size =3D 0 '\000', efds =3D {22, 0 }, elist= =3D {{status =3D 1, fd =3D 22, epfd =3D 33, epdata =3D { event =3D 2147483651, data =3D 0x100, cb_fun =3D 0x6c1b10 , cb_arg =3D 0x7a5c178}}, {status =3D 0, fd =3D 0, epfd =3D 0, epdata =3D {event =3D 0, data =3D 0x0, cb_fun =3D 0x0, cb= _arg =3D 0x0}} }, intr_vec =3D 0x2ab525a09c0} (gdb) print *(rte_eth_devices[2]->intr_handle) $3 =3D {{vfio_dev_fd =3D 19, uio_cfg_fd =3D 19}, fd =3D 20, type =3D RTE_IN= TR_HANDLE_VFIO_MSIX, max_intr =3D 2, nb_efd =3D 1, efd_counter_size =3D 0 '\000', efds =3D {23, 0 }, elist= =3D {{status =3D 1, fd =3D 23, epfd =3D 33, epdata =3D { event =3D 2147483651, data =3D 0x200, cb_fun =3D 0x6c1b10 , cb_arg =3D 0x7a59ab8}}, {status =3D 0, fd =3D 0, epfd =3D 0, epdata =3D {event =3D 0, data =3D 0x0, cb_fun =3D 0x0, cb= _arg =3D 0x0}} }, intr_vec =3D 0x2ab525a0400} so although eventfd-count for 34 is increasing, my application is not waken= up # grep -r eventfd-count /proc/1910/fdinfo/ ... /proc/1910/fdinfo/28:eventfd-count: 0 /proc/1910/fdinfo/30:eventfd-count: 114b5da /proc/1910/fdinfo/31:eventfd-count: 0 /proc/1910/fdinfo/34:eventfd-count: 2054 The epoll fd 33 seems doesn't include eventfd 34 # cat /proc/1910/fdinfo/33 pos: 0 flags: 02 mnt_id: 11 tfd: 21 events: 8000001b data: 7a59390 pos:0 ino:2514 sdev:b tfd: 22 events: 8000001b data: 7a5c210 pos:0 ino:2514 sdev:b tfd: 23 events: 8000001b data: 7a59b50 pos:0 ino:2514 sdev:b tfd: 31 events: 8000001b data: 7fffffffe230 pos:0 ino:2514 sdev:b My application register event like this(like examples\l3fwd-power): rte_eth_dev_rx_intr_ctl_q(portid, queueid,RTE_EPOLL_PER_THREAD, RTE_INTR_EV= ENT_ADD, (void *)((uintptr_t)data)); I doubt the efds has changed after the register, but can't find how this co= uld happen... Any idea please? DPDK version in use: 18.05 I saw 1 commit in 19.05 might be relevant.(in my case, there is secondary d= pdk process, my application is the primary process) Could you please confirm? commit 1c8489da561be16ac73c6dab01db816af249713f Author: Tiwei Bie Date: Mon Mar 25 12:12:15 2019 +0800 net/virtio-user: fix multi-process support This patch fixes the multi-process support for virtio-user. Currently virtio-user just provides some limited secondary process supports. Only some basic operations can be done in secondary process on virtio-user port, e.g. getting port stats. Actions which will trigger the communication with vhost backend can't be done in secondary process for now, as the fds are not synced between processes. The processing of server mode devargs is also moved into virtio_user_dev_init(). Fixes: cdb068f031c6 ("bus/vdev: scan by multi-process channel") Fixes: ee27edbe0c10 ("drivers/net: share vdev data to secondary process= ") Cc: stable@dpdk.org Signed-off-by: Tiwei Bie Reviewed-by: Maxime Coquelin Thank you. Best regards Yan Xiaoping