From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 10D58A04B5; Wed, 30 Sep 2020 17:37:09 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7C3381DB66; Wed, 30 Sep 2020 17:37:08 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by dpdk.org (Postfix) with ESMTP id 5E7531DB65 for ; Wed, 30 Sep 2020 17:37:06 +0200 (CEST) Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601480224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=hZnVql/Sg0jGaPklW6rzzKjPqWJkaCVHv+oXXXDCGvU=; b=Mt2U3jyEiy4JQmzk+XhQ1x+9APb5HJ6qVkLghd7cW2YuBVetEUifC4/OID2EE00d6BSJO+ XR+Id3jZ/Bb4qKI/AvIdlfoxf6LaJru2NmnSpquk9QObAiIY0lSU6FcwxAkVPBEY28WPg3 uAoBsxnMXdtDyhQYkzbH7u6gxk+r5DI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-514-mrn7yqj2OWi763ABMfMthg-1; Wed, 30 Sep 2020 11:37:02 -0400 X-MC-Unique: mrn7yqj2OWi763ABMfMthg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 70CAC1005E63; Wed, 30 Sep 2020 15:37:01 +0000 (UTC) Received: from [10.36.110.36] (unknown [10.36.110.36]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 333B210027A5; Wed, 30 Sep 2020 15:36:58 +0000 (UTC) To: "Xia, Chenbo" , "Zhang, Roy Fan" , "Liu, Changpeng" , "dev@dpdk.org" Cc: "matan@mellanox.com" , "Zawadzki, Tomasz" , "Yigit, Ferruh" References: <1598944057-32690-1-git-send-email-changpeng.liu@intel.com> <8e0e8343-4e85-71f3-8c85-07eb1cff6079@redhat.com> <06542643-cc20-5e3e-26ae-7100005dc97d@redhat.com> <24b547af-2cda-36db-cb66-e5523d169556@redhat.com> <0f8b9daf-3ed7-0ede-8c0c-f2a1bcef9ca4@redhat.com> From: Maxime Coquelin Autocrypt: addr=maxime.coquelin@redhat.com; keydata= mQINBFOEQQIBEADjNLYZZqghYuWv1nlLisptPJp+TSxE/KuP7x47e1Gr5/oMDJ1OKNG8rlNg kLgBQUki3voWhUbMb69ybqdMUHOl21DGCj0BTU3lXwapYXOAnsh8q6RRM+deUpasyT+Jvf3a gU35dgZcomRh5HPmKMU4KfeA38cVUebsFec1HuJAWzOb/UdtQkYyZR4rbzw8SbsOemtMtwOx YdXodneQD7KuRU9IhJKiEfipwqk2pufm2VSGl570l5ANyWMA/XADNhcEXhpkZ1Iwj3TWO7XR uH4xfvPl8nBsLo/EbEI7fbuUULcAnHfowQslPUm6/yaGv6cT5160SPXT1t8U9QDO6aTSo59N jH519JS8oeKZB1n1eLDslCfBpIpWkW8ZElGkOGWAN0vmpLfdyiqBNNyS3eGAfMkJ6b1A24un /TKc6j2QxM0QK4yZGfAxDxtvDv9LFXec8ENJYsbiR6WHRHq7wXl/n8guyh5AuBNQ3LIK44x0 KjGXP1FJkUhUuruGyZsMrDLBRHYi+hhDAgRjqHgoXi5XGETA1PAiNBNnQwMf5aubt+mE2Q5r qLNTgwSo2dpTU3+mJ3y3KlsIfoaxYI7XNsPRXGnZi4hbxmeb2NSXgdCXhX3nELUNYm4ArKBP LugOIT/zRwk0H0+RVwL2zHdMO1Tht1UOFGfOZpvuBF60jhMzbQARAQABtCxNYXhpbWUgQ29x dWVsaW4gPG1heGltZS5jb3F1ZWxpbkByZWRoYXQuY29tPokCOAQTAQIAIgUCV3u/5QIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQyjiNKEaHD4ma2g/+P+Hg9WkONPaY1J4AR7Uf kBneosS4NO3CRy0x4WYmUSLYMLx1I3VH6SVjqZ6uBoYy6Fs6TbF6SHNc7QbB6Qjo3neqnQR1 71Ua1MFvIob8vUEl3jAR/+oaE1UJKrxjWztpppQTukIk4oJOmXbL0nj3d8dA2QgHdTyttZ1H xzZJWWz6vqxCrUqHU7RSH9iWg9R2iuTzii4/vk1oi4Qz7y/q8ONOq6ffOy/t5xSZOMtZCspu Mll2Szzpc/trFO0pLH4LZZfz/nXh2uuUbk8qRIJBIjZH3ZQfACffgfNefLe2PxMqJZ8mFJXc RQO0ONZvwoOoHL6CcnFZp2i0P5ddduzwPdGsPq1bnIXnZqJSl3dUfh3xG5ArkliZ/++zGF1O wvpGvpIuOgLqjyCNNRoR7cP7y8F24gWE/HqJBXs1qzdj/5Hr68NVPV1Tu/l2D1KMOcL5sOrz 2jLXauqDWn1Okk9hkXAP7+0Cmi6QwAPuBT3i6t2e8UdtMtCE4sLesWS/XohnSFFscZR6Vaf3 gKdWiJ/fW64L6b9gjkWtHd4jAJBAIAx1JM6xcA1xMbAFsD8gA2oDBWogHGYcScY/4riDNKXi lw92d6IEHnSf6y7KJCKq8F+Jrj2BwRJiFKTJ6ChbOpyyR6nGTckzsLgday2KxBIyuh4w+hMq TGDSp2rmWGJjASq5Ag0EVPSbkwEQAMkaNc084Qvql+XW+wcUIY+Dn9A2D1gMr2BVwdSfVDN7 0ZYxo9PvSkzh6eQmnZNQtl8WSHl3VG3IEDQzsMQ2ftZn2sxjcCadexrQQv3Lu60Tgj7YVYRM H+fLYt9W5YuWduJ+FPLbjIKynBf6JCRMWr75QAOhhhaI0tsie3eDsKQBA0w7WCuPiZiheJaL 4MDe9hcH4rM3ybnRW7K2dLszWNhHVoYSFlZGYh+MGpuODeQKDS035+4H2rEWgg+iaOwqD7bg CQXwTZ1kSrm8NxIRVD3MBtzp9SZdUHLfmBl/tLVwDSZvHZhhvJHC6Lj6VL4jPXF5K2+Nn/Su CQmEBisOmwnXZhhu8ulAZ7S2tcl94DCo60ReheDoPBU8PR2TLg8rS5f9w6mLYarvQWL7cDtT d2eX3Z6TggfNINr/RTFrrAd7NHl5h3OnlXj7PQ1f0kfufduOeCQddJN4gsQfxo/qvWVB7PaE 1WTIggPmWS+Xxijk7xG6x9McTdmGhYaPZBpAxewK8ypl5+yubVsE9yOOhKMVo9DoVCjh5To5 aph7CQWfQsV7cd9PfSJjI2lXI0dhEXhQ7lRCFpf3V3mD6CyrhpcJpV6XVGjxJvGUale7+IOp sQIbPKUHpB2F+ZUPWds9yyVxGwDxD8WLqKKy0WLIjkkSsOb9UBNzgRyzrEC9lgQ/ABEBAAGJ Ah8EGAECAAkFAlT0m5MCGwwACgkQyjiNKEaHD4nU8hAAtt0xFJAy0sOWqSmyxTc7FUcX+pbD KVyPlpl6urKKMk1XtVMUPuae/+UwvIt0urk1mXi6DnrAN50TmQqvdjcPTQ6uoZ8zjgGeASZg jj0/bJGhgUr9U7oG7Hh2F8vzpOqZrdd65MRkxmc7bWj1k81tOU2woR/Gy8xLzi0k0KUa8ueB iYOcZcIGTcs9CssVwQjYaXRoeT65LJnTxYZif2pfNxfINFzCGw42s3EtZFteczClKcVSJ1+L +QUY/J24x0/ocQX/M1PwtZbB4c/2Pg/t5FS+s6UB1Ce08xsJDcwyOPIH6O3tccZuriHgvqKP yKz/Ble76+NFlTK1mpUlfM7PVhD5XzrDUEHWRTeTJSvJ8TIPL4uyfzhjHhlkCU0mw7Pscyxn DE8G0UYMEaNgaZap8dcGMYH/96EfE5s/nTX0M6MXV0yots7U2BDb4soLCxLOJz4tAFDtNFtA wLBhXRSvWhdBJZiig/9CG3dXmKfi2H+wdUCSvEFHRpgo7GK8/Kh3vGhgKmnnxhl8ACBaGy9n fxjSxjSO6rj4/MeenmlJw1yebzkX8ZmaSi8BHe+n6jTGEFNrbiOdWpJgc5yHIZZnwXaW54QT UhhSjDL1rV2B4F28w30jYmlRmm2RdN7iCZfbyP3dvFQTzQ4ySquuPkIGcOOHrvZzxbRjzMx1 Mwqu3GQ= Message-ID: <42d0c319-314f-80ea-f3d7-21eaf567c2fa@redhat.com> Date: Wed, 30 Sep 2020 17:36:56 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH] vhost: return ready when at least 1 vring is configured X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, On 9/30/20 4:48 AM, Xia, Chenbo wrote: > Hi Fan & Maxime, > > I am thinking that should we move set_features outside of new_device callback > for crypto device? I see that net and blk devices both set features between register > and start, and personally I think this makes sense that device features are set > before device start and ready. How do you think 😊? Indeed, we cannot consider the device to be ready (and so call .new_device callback) if features haven't been negotiated. I agree, rte_vhost_driver_set_features() has to be called before .new_device(), and that's the reason why it takes socket's path and not vid as input. Thanks, Maxime > Thanks, > Chenbo > >> -----Original Message----- >> From: Zhang, Roy Fan >> Sent: Wednesday, September 30, 2020 2:15 AM >> To: Maxime Coquelin ; Liu, Changpeng >> ; dev@dpdk.org >> Cc: matan@mellanox.com; Xia, Chenbo ; Zawadzki, >> Tomasz ; Yigit, Ferruh >> Subject: RE: [dpdk-dev] [PATCH] vhost: return ready when at least 1 vring >> is configured >> >> Hi Maxime, >> >> Thanks for telling me that but vhost_crypto is still breaking. >> >> Virtio-crypto (both LKCF virtio-crypt driver and DPDK virtio-crypto PMD) >> won't create device queue until the crypto session is required to be >> created. Thus vq_is_ready() will never return true during initialization - >> hence new_device() in vhost_crypto sample app will never be triggered. >> Also since rte_vhost_driver_set_features() is called inside >> rte_vhost_crypto_create(), which is triggered by new_device() handler, we >> cannot update the dev->flags to bypass the "if (dev->flags & >> VIRTIO_DEV_BUILTIN_VIRTIO_NET)" check before new_device() is called. >> what the new logic virtio_is_ready() requirement will never meet for >> vhost_crypto. >> >> A way to fix it is with the following change - >> >> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c >> index b00e1f91d..e5263a360 100644 >> --- a/lib/librte_vhost/vhost_user.c >> +++ b/lib/librte_vhost/vhost_user.c >> @@ -1937,6 +1937,14 @@ vhost_user_set_vring_kick(struct virtio_net **pdev, >> struct VhostUserMsg *msg, >> } >> } >> >> + /* virtio-crypto vq is not ready until session is created. Check >> + * here if we need to initialize device again >> + */ >> + if (!(dev->flags & VIRTIO_DEV_RUNNING)) { >> + if (dev->notify_ops->new_device(dev->vid) == 0) >> + dev->flags |= VIRTIO_DEV_RUNNING; >> + } >> + >> return RTE_VHOST_MSG_RESULT_OK; >> } >> >> but I cannot add virtio_is_ready() inside the " if (!(dev->flags & >> VIRTIO_DEV_RUNNING))" check, since we need new_device() is called to set >> the feature flags - if to set the feature flags in the example instead, >> the logic will not right, since virtio-net does not require the user to >> set the flags themselves either. >> >> Regards, >> Fan >> >> >>> -----Original Message----- >>> From: Maxime Coquelin >>> Sent: Tuesday, September 29, 2020 3:06 PM >>> To: Zhang, Roy Fan ; Liu, Changpeng >>> ; dev@dpdk.org >>> Cc: matan@mellanox.com; Xia, Chenbo ; Zawadzki, >>> Tomasz >>> Subject: Re: [dpdk-dev] [PATCH] vhost: return ready when at least 1 >> vring is >>> configured >>> >>> Hi Fan, >>> >>> The patch is already merged in main branch: >>> >>> commit 09424c3f74311555c33d3d4cdc2ca3654ce13b1c >>> Author: Maxime Coquelin >>> Date: Wed Sep 23 11:49:02 2020 +0200 >>> >>> vhost: fix external backends readiness >>> >>> Commit d0fcc38f5fa4 ("vhost: improve device readiness notifications") >>> makes the assumption that every Virtio devices are considered >>> ready for preocessing as soon as first queue pair is configured >>> and enabled. >>> >>> While this is true for Virtio-net, it isn't for Virtio-scsi >>> and Virtio-blk. >>> >>> This patch fixes this by only making this assumption for >>> the builtin Virtio-net backend, and restores back to previous >>> behaviour for other backends. >>> >>> Fixes: d0fcc38f5fa4 ("vhost: improve device readiness notifications") >>> >>> Reported-by: Changpeng Liu >>> Signed-off-by: Maxime Coquelin >>> Signed-off-by: Changpeng Liu >>> Reviewed-by: Chenbo Xia >>> >>> >>> Regards, >>> Maxime >>> >>> On 9/29/20 3:54 PM, Zhang, Roy Fan wrote: >>>> Hi Maxime, >>>> >>>> Vhost-crypto has exactly the same issue. Changpeng's patch fixed it. >>>> Could you give me a shout when your patch is out, so I can have a test? >>>> >>>> Regards, >>>> Fan >>>> >>>>> -----Original Message----- >>>>> From: dev On Behalf Of Maxime Coquelin >>>>> Sent: Wednesday, September 23, 2020 9:05 AM >>>>> To: Liu, Changpeng ; dev@dpdk.org >>>>> Cc: matan@mellanox.com; Xia, Chenbo ; >>> Zawadzki, >>>>> Tomasz >>>>> Subject: Re: [dpdk-dev] [PATCH] vhost: return ready when at least 1 >> vring >>> is >>>>> configured >>>>> >>>>> Hi Changpeng, >>>>> >>>>> On 9/22/20 9:22 AM, Liu, Changpeng wrote: >>>>>> Hi Maxime, >>>>>> >>>>>> The code you wrote still need to check nr_vring is 0 or not, see the >> extra >>> 2 >>>>> lines added below, then it can work with my tests for now, could you >>> submit >>>>> a patch to DPDK to apply the patch? Thanks. >>>>> >>>>> Thanks! You are right. >>>>> >>>>> I'll send the patch now including your fix. >>>>> >>>>>> BTW, dpdk vhost library still has an issue, it's not related with >> commit >>>>> d0fcc38f, the Guest driver may only kick 1 vring even it sends >>> NUM_QUEUES >>>>> with a bigger value, >>>>>> this is quite common in seabios, e.g: virtio_blk will only use 1 >> vring in >>>>> seabios, this means the backend will never get started in BIOS. >>>>>> >>>>> >>>>> If I understand correctly, this is not a regression but has always >> been >>>>> here? >>>>> >>>>> We should work on fixing it anyway, but I'm not sure to have the time >>>>> for v20.11.0. It would be great if you could provide steps to >> reproduce >>>>> it. Maybe file a bug in DPDK tracker? >>>>> >>>>> Thanks, >>>>> Maxime >>>>> >>>>>>> -----Original Message----- >>>>>>> From: Maxime Coquelin >>>>>>> Sent: Monday, September 21, 2020 6:20 PM >>>>>>> To: Liu, Changpeng ; dev@dpdk.org >>>>>>> Cc: matan@mellanox.com; Xia, Chenbo ; >>>>> Zawadzki, >>>>>>> Tomasz >>>>>>> Subject: Re: [PATCH] vhost: return ready when at least 1 vring is >>>>> configured >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 9/21/20 7:03 AM, Liu, Changpeng wrote: >>>>>>>> Hi Maxime, >>>>>>>> >>>>>>>>> -----Original Message----- >>>>>>>>> From: Maxime Coquelin >>>>>>>>> Sent: Friday, September 18, 2020 5:54 PM >>>>>>>>> To: Liu, Changpeng ; dev@dpdk.org >>>>>>>>> Cc: matan@mellanox.com; Xia, Chenbo ; >>>>> Zawadzki, >>>>>>>>> Tomasz >>>>>>>>> Subject: Re: [PATCH] vhost: return ready when at least 1 vring is >>>>> configured >>>>>>>>> >>>>>>>>> Hi Changpeng, >>>>>>>>> >>>>>>>>> On 9/1/20 9:07 AM, Changpeng Liu wrote: >>>>>>>>>> Commit d0fcc38f "vhost: improve device readiness notifications" >>>>>>>>>> needs at least 2 vrings before changing the device state to >>>>>>>>>> ready, this is fine for NET device but not correct for BLK >>>>>>>>>> device. >>>>>>>>>> >>>>>>>>>> The number of vring required should be based on the device >>>>>>>>>> type, e.g. virtio_scsi device needs at least 3 vrings, and >>>>>>>>>> virtio_net needs at least 2 vrings, virtio_blk needs at least >>>>>>>>>> 1 vring. So instead of doing it in vhost library it's better >>>>>>>>>> that the application who uses this library do this check. >>>>>>>>>> >>>>>>>>>> Signed-off-by: Changpeng Liu >>>>>>>>>> --- >>>>>>>>>> lib/librte_vhost/vhost_user.c | 2 +- >>>>>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>>>>>>> >>>>>>>>>> diff --git a/lib/librte_vhost/vhost_user.c >>>>> b/lib/librte_vhost/vhost_user.c >>>>>>>>>> index c3c924f..4d1883c 100644 >>>>>>>>>> --- a/lib/librte_vhost/vhost_user.c >>>>>>>>>> +++ b/lib/librte_vhost/vhost_user.c >>>>>>>>>> @@ -1343,7 +1343,7 @@ >>>>>>>>>> vq->enabled; >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> -#define VIRTIO_DEV_NUM_VQS_TO_BE_READY 2u >>>>>>>>>> +#define VIRTIO_DEV_NUM_VQS_TO_BE_READY 1u >>>>>>>>> >>>>>>>>> I think it would be better to rely on >>> VIRTIO_DEV_BUILTIN_VIRTIO_NET >>>>> to >>>>>>>>> know whether it should wait for 1 or 2 queues to determine if >> ready. >>>>>>>> virtio_scsi needs at least 3 vrings, so both 1 and 2 can't work >> for >>>>> virtio_scsi >>>>>>> device. >>>>>>>> Can we expose an API to let the caller to set the minimum number >> of >>>>> vrings >>>>>>> required by >>>>>>>> virtio device? >>>>>>> >>>>>>> OK, thanks for pointing this out, I missed it. >>>>>>> >>>>>>> I'm not in favor of introducing an new API for this. >>>>>>> I propose to restrict change introduced in commit d0fcc38f to the >>>>>>> builtin net backend. Can you have a try with below patch? >>>>>>> >>>>>>> Thanks in advance, >>>>>>> Maxime >>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> static int >>>>>>>>>> virtio_is_ready(struct virtio_net *dev) >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> diff --git a/lib/librte_vhost/vhost_user.c >>> b/lib/librte_vhost/vhost_user.c >>>>>>> index 501218e192..f571ef93fc 100644 >>>>>>> --- a/lib/librte_vhost/vhost_user.c >>>>>>> +++ b/lib/librte_vhost/vhost_user.c >>>>>>> @@ -1343,21 +1343,25 @@ vq_is_ready(struct virtio_net *dev, struct >>>>>>> vhost_virtqueue *vq) >>>>>>> vq->enabled; >>>>>>> } >>>>>>> >>>>>>> -#define VIRTIO_DEV_NUM_VQS_TO_BE_READY 2u >>>>>>> +#define VIRTIO_BUILTIN_NUM_VQS_TO_BE_READY 2u >>>>>>> >>>>>>> static int >>>>>>> virtio_is_ready(struct virtio_net *dev) >>>>>>> { >>>>>>> struct vhost_virtqueue *vq; >>>>>>> - uint32_t i; >>>>>>> + uint32_t i, nr_vring = dev->nr_vring; >>>>>>> >>>>>>> if (dev->flags & VIRTIO_DEV_READY) >>>>>>> return 1; >>>>>>> >>>>>>> - if (dev->nr_vring < VIRTIO_DEV_NUM_VQS_TO_BE_READY) >>>>>>> - return 0; >>>>>>> + if (dev->flags & VIRTIO_DEV_BUILTIN_VIRTIO_NET) { >>>>>>> + nr_vring = VIRTIO_BUILTIN_NUM_VQS_TO_BE_READY; >>>>>>> + >>>>>>> + if (dev->nr_vring < nr_vring) >>>>>>> + return 0; >>>>>>> + } >>>>>> >>>>>> + if(!nr_vring) >>>>>> + return 0; >>>>>>> >>>>>>> - for (i = 0; i < VIRTIO_DEV_NUM_VQS_TO_BE_READY; i++) { >>>>>>> + for (i = 0; i < nr_vring; i++) { >>>>>>> vq = dev->virtqueue[i]; >>>>>>> >>>>>>> if (!vq_is_ready(dev, vq)) >>>>>> >>>> >