From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id CBB531B343 for ; Thu, 2 Nov 2017 10:40:42 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 14C6A7EA8B; Thu, 2 Nov 2017 09:40:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 14C6A7EA8B Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx04.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=maxime.coquelin@redhat.com Received: from [10.36.112.50] (ovpn-112-50.ams2.redhat.com [10.36.112.50]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D844160CA0; Thu, 2 Nov 2017 09:40:29 +0000 (UTC) To: "Kavanagh, Mark B" , "yliu@fridaylinux.org" Cc: "dev@dpdk.org" , "Horton, Remy" , "Bie, Tiwei" , "mst@redhat.com" , "jfreiman@redhat.com" , "vkaplans@redhat.com" , "jasowang@redhat.com" , "Mcnamara, John" , "Loftus, Ciara" , "Stokes, Ian" References: <20171005083627.27828-1-maxime.coquelin@redhat.com> <20171005083627.27828-2-maxime.coquelin@redhat.com> From: Maxime Coquelin Message-ID: Date: Thu, 2 Nov 2017 10:40:26 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 02 Nov 2017 09:40:42 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH v3 01/19] Revert "vhost: workaround MQ fails to startup" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Nov 2017 09:40:43 -0000 Hi Mark, On 11/01/2017 06:11 PM, Kavanagh, Mark B wrote: > Hi Maxime, > > First off, apologies for the lateness of this reply - I realize that this patch has already been upstreamed. No worries, great to see DPDK integration being tested before v17.11 is released. Is the v17.11 upgrade patch available somewhere? > Unfortunately, during OvS-DPDK regression testing for DPDK v17.11-rc2 just today, a regression involving vHost multiq was detected, and pinpointed to this patch. > > Version info for the components involved during the aforementioned testing is as follows: > DPDK: v17.11-rc2 > OvS: af2e40c ("sparse: eliminate "duplicate initialization") + DPDK v17.11 upgrade patch > QEMU: v2.7.0 > > The regression may be reproduced as follows: > - Set up OvS-DPDK as normal, and add one vHostUser client port: > $OVS_DIR/utilities/ovs-vsctl add-port br0 vhost-user0 -- set Interface vhost-user0 type=dpdkvhostuserclient > > - Start QEMU, specifying 2 ports for the guest interface" > $QEMU_DIR/x86_64-softmmu/qemu-system-x86_64 \ > -cpu host -enable-kvm -m 4096M \ > -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on \ > -numa node,memdev=mem -mem-prealloc \ > -drive file=$VM_IMAGE \ > -chardev socket,id=char0,path=/tmp/sock0,server \ > -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce,queues=2 \ > -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off,mq=on,vectors=6 \ > -nographic" > > - The guest subsequently starts as normal, but then hangs completely, rendering it completely unusable. > The last lines of ovs-vswitchd.log read as follows: > |00051|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE > |00052|dpdk|INFO|VHOST_CONFIG: set queue enable: 1 to qp idx: 1 > |00053|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_FEATURES > > Moving from QEMU v2.7.0 to v2.10.0 resolves the issue. However, herein lies the issue: QEMU v2.10.0 was only released in August of this year; anecdotally, we know that many OvS-DPDK customers use older versions of QEMU (typically, v2.7.0), and are likely un[able|willing] to move. With this patch, a hard dependency on QEMU v2.10 is created for users who want to use the vHU multiq feature in DPDK v17.11 (and subsequently, the upcoming OvS v2.9.0), which IMO will likely be unacceptable for many. Do you mean that upstream Qemu v2.7.0 is used in production? I would expect the customers to use a distro Qemu which should contain relevant fixes, or follow upstream's stable branches. FYI, Qemu v2.9.1 contains a backport of the fix. > One potential solution to this problem is to introduce a compile-time option that would allow the user to [dis|en]able the VHOST_USER_PROTOCOL_F_REPLY_ACK feature - is that something that would be acceptable to you Maxime? Yes, that's one option, but: 1. VHOST_USER_PROTOCOL_F_REPLY_ACK enabled should be the default 2. VHOST_USER_PROTOCOL_F_REPLY_ACK disabled will be less extensively tested. Yuanhan, what do you think? Regards, Maxime > Thanks in advance, > Mark > > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Maxime Coquelin >> Sent: Thursday, October 5, 2017 9:36 AM >> To: dev@dpdk.org; Horton, Remy ; Bie, Tiwei >> ; yliu@fridaylinux.org >> Cc: mst@redhat.com; jfreiman@redhat.com; vkaplans@redhat.com; >> jasowang@redhat.com; Maxime Coquelin >> Subject: [dpdk-dev] [PATCH v3 01/19] Revert "vhost: workaround MQ fails to >> startup" >> >> This reverts commit 04d81227960b5c1cf2f11f492100979ead20c526. >> >> As agreed when this workaround was introduced, it can be reverted >> as Qemu v2.10 that fixes the issue is now out. >> >> The reply-ack feature is required for vhost-user IOMMU support. >> >> Signed-off-by: Maxime Coquelin >> --- >> lib/librte_vhost/vhost_user.h | 6 +----- >> 1 file changed, 1 insertion(+), 5 deletions(-) >> >> diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h >> index 35ebd7190..2ba22dbb0 100644 >> --- a/lib/librte_vhost/vhost_user.h >> +++ b/lib/librte_vhost/vhost_user.h >> @@ -49,14 +49,10 @@ >> #define VHOST_USER_PROTOCOL_F_REPLY_ACK 3 >> #define VHOST_USER_PROTOCOL_F_NET_MTU 4 >> >> -/* >> - * disable REPLY_ACK feature to workaround the buggy QEMU implementation. >> - * Proved buggy QEMU includes v2.7 - v2.9. >> - */ >> #define VHOST_USER_PROTOCOL_FEATURES ((1ULL << VHOST_USER_PROTOCOL_F_MQ) | \ >> (1ULL << VHOST_USER_PROTOCOL_F_LOG_SHMFD) |\ >> (1ULL << VHOST_USER_PROTOCOL_F_RARP) | \ >> - (0ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \ >> + (1ULL << VHOST_USER_PROTOCOL_F_REPLY_ACK) | \ >> (1ULL << VHOST_USER_PROTOCOL_F_NET_MTU)) >> >> typedef enum VhostUserRequest { >> -- >> 2.13.6 >