From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DDAFE42532 for ; Thu, 7 Sep 2023 09:21:22 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B0FB2402BD; Thu, 7 Sep 2023 09:21:22 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 1CEF0402BC for ; Thu, 7 Sep 2023 09:21:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694071279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AKLx2NGN//v/VI7wh0ndLeXyTNT7rdZzY36hq5myZto=; b=gupu8IIT8aErOkFZzriwvNdQrynCFidiUnhiEi+889Q4LjmditlW847QhWl92LTXOFwz8f kSN32PcehEAY3of58o1h1fSMa/D74bzWtaIBUW26PV7XFGeHpAyLrPi2JtMGGhzQHnl8o1 6QEOl5qPtZzkTPqdWu4ZcK0hZ9WWBSk= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-632-yRN-s-46PXyG6JtbyK2Euw-1; Thu, 07 Sep 2023 03:21:16 -0400 X-MC-Unique: yRN-s-46PXyG6JtbyK2Euw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 48D221C08969; Thu, 7 Sep 2023 07:21:16 +0000 (UTC) Received: from [10.39.208.8] (unknown [10.39.208.8]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 388041121314; Thu, 7 Sep 2023 07:21:15 +0000 (UTC) Message-ID: <3112e62d-b589-84d3-68a3-dc951f399dd9@redhat.com> Date: Thu, 7 Sep 2023 09:21:13 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: Implementing a simple TAP PMD to dpdk-vhost structure To: =?UTF-8?B?Tmljb2xzb24gS2VuICjjg4vjgrPjg6vjgr3jg7Mg44Kx44OzKQ==?= , David Marchand Cc: "users@dpdk.org" , "Xia, Chenbo" References: From: Maxime Coquelin In-Reply-To: X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On 9/7/23 08:22, Nicolson Ken (ニコルソン ケン) wrote: > Hi David, > > Hmm, maybe the issue is at my end - to configure QEMU, I followed the tutorial at https://www.redhat.com/en/blog/hands-vhost-user-warm-welcome-dpdk > >> Are you seeing the vhost port getting initialised in testpmd output? > > Yes, I get a lot of VHOST_CONFIG messages, ending with: > VHOST_CONFIG: (/tmp/sock0) virtio is now ready for processing. > Rx csum will be done in SW, may impact performance. > > *** > UPDATE: I've been reading more of the manual while doing the troubleshooting below, and I think I've found a major issue while checking the Guest OS. > According to https://doc.dpdk.org/guides-21.11/linux_gsg/linux_drivers.html > > dmesg | tail > ... > [ 1297.875090] vfio-pci: probe of 0000:31:00.0 failed with error -22 > > I get the above in the Guest when trying to do devbind, and I also get this on both Host and Guest: > > cat /boot/config-$(uname -r) | grep NOIOMMU > CONFIG_VFIO_NOIOMMU=y > > Should that actually be "N"? Does "is not set" equal to no IOMMU? I should follow the grubby settings given on that RedHat page, I think. If that is off, then I would guess that that could very well be a source of all my issues. CONFIG_VFIO_NOIOMMU=y is valid but it just build noiommu support, what you need is to enable it at probe time: # modprobe vfio enable_unsafe_noiommu_mode=y # cat /sys/module/vfio/parameters/enable_unsafe_noiommu_mode Y But I understood you wanted to inject packets in the guest kernel. If this is true it should not be necessary, the Virtio device has to be bound to Kernel Virtio-net driver in the guest, not to VFIO/Virtio PMD. Maxime > > Back to the previous contents: > *** > > Now, I try your example - in another terminal on the Host, I'll use "ping -v -I dtap0 -6 fe80::5054:ff:fe01:7d00", which is the address of the interface created in the Guest OS by QEMU (hmm, my seems to have been ignored), after I do "sudo ip link set enp9s0 up" there. > > First, ping output (if I don't use "-I dtap0", nothing happens): > $ ping -v -I dtap0 -6 fe80::5054:ff:fe01:7d00 > ping: Warning: source address might be selected on device other than: dtap0 > PING fe80::5054:ff:fe01:7d00(fe80::5054:ff:fe01:7d00) from :: dtap0: 56 data bytes > From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=1 Destination unreachable: Address unreachable > From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=2 Destination unreachable: Address unreachable > From fe80::20d4:2dff:fe67:3768%dtap0 icmp_seq=3 Destination unreachable: Address unreachable > ^C > --- fe80::5054:ff:fe01:7d00 ping statistics --- > 4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3055ms > > testpmd> set verbose 3 > Change verbose level from 0 to 3 > testpmd> start > io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP allocation mode: native > Logical Core 1 (socket 0) forwards packets on 2 streams: > RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01 > RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 > > Then for each for the 6 attempted pings I see: > > port 0/queue 0: received 1 packets > src=22:D4:2D:67:37:68 - dst=33:33:FF:01:7D:00 - pool=mb_pool_0 - type=0x86dd - length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6 - sw ptype: L2_ETHER L3_IPV6 - l2_len=14 - l3_len=40 - Receive queue=0x0 > ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN > port 1/queue 0: sent 1 packets > src=22:D4:2D:67:37:68 - dst=33:33:FF:01:7D:00 - pool=mb_pool_0 - type=0x86dd - length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6 - sw ptype: L2_ETHER L3_IPV6 - l2_len=14 - l3_len=40 - Send queue=0x0 > ol_flags: RTE_MBUF_F_TX_L4_NO_CKSUM > > Thanks for the help, > Ken > > -----Original Message----- > From: David Marchand > Sent: Wednesday, September 6, 2023 5:41 PM > To: Nicolson Ken (ニコルソン ケン) > Cc: users@dpdk.org; Maxime Coquelin ; Xia, Chenbo > Subject: Re: Implementing a simple TAP PMD to dpdk-vhost structure > > On Wed, Sep 6, 2023 at 9:53 AM Nicolson Ken (ニコルソン ケン) > wrote: >>> Alternatively, did you consider using testpmd with the vhost pmd instead ? >> >> I've tried that before, but as far as I can see from net/vhost/rte_eth_host.c it uses rte_vost_enqueue/dequeue_burst() to basically act as a loopback for the Guest OS. I use: >> >> $ sudo dpdk-testpmd -l 0-3 -n 4 --vdev 'net_tap0' --vdev >> 'net_vhost1,iface=/tmp/sock0,client=1' -- -i >> >> But if I feed data in using "tcpreplay -I dtap0 ...", "show port stats all" shows everything going into the TAP but nothing is forward to vhost. > > Well, pinging from a dtap0 netdev in the host to a virtio-net netdev in a guest works me. > > testpmd> set verbose 3 > Change verbose level from 0 to 3 > testpmd> start > io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP allocation mode: native Logical Core 1 (socket 1) forwards packets on 2 streams: > RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=02:00:00:00:00:01 > RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=02:00:00:00:00:00 > > port 0/queue 0: received 16 packets > src=26:9B:E2:29:7E:C6 - dst=33:33:00:00:00:16 - pool=mb_pool_0 - type=0x86dd - length=90 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6_EXT > - sw ptype: L2_ETHER L3_IPV6_EXT - l2_len=14 - l3_len=48 - Receive > queue=0x0 > ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN > src=26:9B:E2:29:7E:C6 - dst=33:33:00:00:00:16 - pool=mb_pool_0 - type=0x86dd - length=90 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6_EXT > - sw ptype: L2_ETHER L3_IPV6_EXT - l2_len=14 - l3_len=48 - Receive > queue=0x0 > ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN > src=26:9B:E2:29:7E:C6 - dst=33:33:FF:29:7E:C6 - pool=mb_pool_0 - type=0x86dd - length=86 - nb_segs=1 - hw ptype: L2_ETHER L3_IPV6 - sw > ptype: L2_ETHER L3_IPV6 - l2_len=14 - l3_len=40 - Receive queue=0x0 > ol_flags: RTE_MBUF_F_RX_L4_CKSUM_UNKNOWN RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN RTE_MBUF_F_RX_OUTER_L4_CKSUM_UNKNOWN > > > Are you seeing the vhost port getting initialised in testpmd output? > How are you sure that nothing is forwarded? > > > -- > David Marchand >