From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from sonic307-38.consmr.mail.ne1.yahoo.com (sonic307-38.consmr.mail.ne1.yahoo.com [66.163.190.61]) by dpdk.org (Postfix) with ESMTP id 70BBD1B3C2 for ; Thu, 12 Oct 2017 17:34:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1507822466; bh=9ByGeSVxpZRMYUjIrJNNuJuFBirf+ziXzVrE08Ab93M=; h=Date:From:To:In-Reply-To:References:Subject:From:Subject; b=kwtGPII1dQiVL72reWl8AqQFeSZc2B9brkjXPyWwxplI2cqoUrExCj3Mi3uCenInKDAubFxcJ9yYNFSxv2NvlPlGmUsgQS2wgHKdJOXVrBsX/aMwFQ11yeKgGhp9BngP+/cZqFAZfYW1hEVSJBiN8y5ESOOyZivEtmoC83PKQM+40gyM4LWBZTbCTDu8HOplf4UAPWcWsnUlAR0XmT1HfV/jRpHGM0MHL6gwqDkZEWRaF3EOTCdNZ3mmFws9UA4QZzabOsDtyYfguHt/4awIt+P8Q8EsoSIZDWi3G/kTaJgJjX8Ayn3emRaXHu/psTuFv8GqI+lSk52T2ReqiRxiyg== X-YMail-OSG: dIBfwBEVM1mpxDBTMUJModN_4W.2IYykvvEeLPW1R9APCHB51ynWbiicH2e1Cz1 GtHuqfQRkBuudW8Xo.L7ufEki9PaDOGK8SQYZlE1Zl9ZEdI_eHonHxh86WHqVOWBSiEarpOKeI5P YUReIZSgvQTKpFjnft2FjJKy7OesA0Rq6Dg3iXFzejawcXxpOSNiV.HYbildP5pUObQEoLmlktEX SA13DVAcgUmdXzL7vUT9FPSzGSMYojqmlDYn78eS5en2XJcIHACSPmuWyqP3Q.17ki8XepCmUqKO 0qldGYDDDr_3DFY6wgsAT8Ofl97KLz_C_nRLXT61JBOcGGMt.3ltb5oUhJ6f8YvBhYqbya88zusN CyQnr4DSzTur_KsoBZ2bp4IvHi7UIhX1xfjezGeKnjYvygyJ5r.IYLAOa0.b9VgyWcxNa65MykJo dXvHejFmMNw2JDYwk.HPkp6WjDSHTGEMgF7NF.iaOhw2D66TI7OPHAJ9wni7T Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.ne1.yahoo.com with HTTP; Thu, 12 Oct 2017 15:34:26 +0000 Date: Thu, 12 Oct 2017 15:30:25 +0000 (UTC) From: zhi ma To: "dev@dpdk.org" Message-ID: <1780964727.557952.1507822225600@mail.yahoo.com> In-Reply-To: <824172623.112272.1507760060188@mail.yahoo.com> References: <824172623.112272.1507760060188.ref@mail.yahoo.com> <824172623.112272.1507760060188@mail.yahoo.com> MIME-Version: 1.0 X-Mailer: WebService/1.1.10726 YMailNorrin Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Safari/604.1.38 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Need help to run test_pmd via sr-iov setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Oct 2017 15:34:27 -0000 =20 Need help to run test_pmd via sr-iov setup my steps:1. turn on intel vt in bios2. set "intel_iommu=3Don" in kernel boo= t parameter3. update host grub and reboot4. create VF:=C2=A0 =C2=A0 echo 1 = > /sys/class/net/eth17/device sriov_numvfs=C2=A0 =C2=A0 echo 1 > /sys/class= /net/eth18/device sriov_numvfs=C2=A0 =C2=A0 cat /sys/class/net/eth17/device= sriov_numvfs=C2=A0 =C2=A0 1=C2=A0 =C2=A0 cat /sys/class/net/eth18/device s= riov_numvfs=C2=A0 =C2=A0 1=C2=A05. set Mac=C2=A0 =C2=A0 ip link set eth17 v= f 0 mac aa:bb:cc:dd:00:00=C2=A0 =C2=A0 ip link set eth18 vf 0 mac aa:bb:cc:= dd:01:00=C2=A06. rmmod ixgbevf=C2=A07. start ubuntu 14.04 vm with KVM virt-= manager, add two PCI host device (Virtual function devices)=C2=A08. run dpd= k app --- test_pmd in vm (two ports connected back to back)=C2=A0 =C2=A0 st= art tx_first=C2=A0 =C2=A0 stop=C2=A0 =C2=A0 no packets=C2=A0=C2=A09. if I r= un test_pmd inside host machine, it works fine. (lots of sent and received = packets, so those two ports connected fine) =C2=A0INTEL X520-SR2=C2=A0both host and guest vm running as ubuntu 14.04 =C2=A0any thing wrong with my setup? Please help part of my virus xml=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
=C2=A0 =C2=A0 =20 >From maxime.coquelin@redhat.com Thu Oct 12 17:39:07 2017 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id ECD981B3CE for ; Thu, 12 Oct 2017 17:39:05 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EFCC57E38F; Thu, 12 Oct 2017 15:39:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com EFCC57E38F Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=maxime.coquelin@redhat.com Received: from localhost.localdomain (ovpn-112-53.ams2.redhat.com [10.36.112.53]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0B21117D5B; Thu, 12 Oct 2017 15:39:02 +0000 (UTC) From: Maxime Coquelin To: yliu@fridaylinux.org, thomas@monjalon.net, dev@dpdk.org Cc: Maxime Coquelin Date: Thu, 12 Oct 2017 17:38:48 +0200 Message-Id: <20171012153850.21837-1-maxime.coquelin@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 12 Oct 2017 15:39:05 +0000 (UTC) Subject: [dpdk-dev] [PATCH 0/2] vhost: IOTLB fixes for -rc1 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Oct 2017 15:39:07 -0000 These two patches fixes issues faced when running the VM on a different socket than DPDK. In this case, the numa_realloc() function is called to reallocate the virtqueue and the virtio-net device structs on the VM's socket. The problem is that doing so corrupts the IOTLB cache list, as the list head is being reallocated, but the first entry in the list is not updated to point to the new list head. It results in all new IOTLB entries that need to be inserted before the first entry in the list to be leaked, as the new head is still pointing to the first entry at the time the realloc happened. Patch 2 addresses this issue by re-initializing the IOTLB cache completely. Doing this also create again the IOTLB mempool on the new socket. This first issue helped to highlight a deadlock that patch 1 fixes. As inserting an entry before the first entry in the list resulted in a leak, it ended up flooding Qemu with IOTLB misses for the same address. The deadlock happen because an optimization was done to lock the iotlb cache lock once per packet burst instead of once per translation. It means that when an IOTLB miss is sent, it is done with the lock held. The problem is that sending an IOTLB miss can block if the socket buffer is full, and this buffer is emptied by the same Qemu thread which is waiting for an IOTLB update to be completed. But it never completes because DPDK waits for the iotlb lock to insert the update into the iotlb cache, hence the deadlock. The fix consists in just unlocking the iotlb lock while sending the IOTLB miss, which is safe as it does not access the iotlb list the lock protects. Maxime Coquelin (2): vhost: fix deadlock on IOTLB miss vhost: fix IOTLB on NUMA realloc lib/librte_vhost/iotlb.c | 1 - lib/librte_vhost/vhost.c | 12 ++++++++++++ lib/librte_vhost/vhost_user.c | 3 +++ 3 files changed, 15 insertions(+), 1 deletion(-) -- 2.13.6