From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 159C62BF1 for ; Wed, 28 Mar 2018 12:56:23 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 32F54403B1E7; Wed, 28 Mar 2018 10:56:22 +0000 (UTC) Received: from [10.36.112.44] (ovpn-112-44.ams2.redhat.com [10.36.112.44]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 257D22026DFD; Wed, 28 Mar 2018 10:56:19 +0000 (UTC) To: "Liu, Changpeng" , "Kulasek, TomaszX" , "yliu@fridaylinux.org" Cc: "Verkamp, Daniel" , "Harris, James R" , "Wodkowski, PawelX" , "dev@dpdk.org" , "Tan, Jianfeng" References: <20180327151737.6640-1-tomaszx.kulasek@intel.com> <20180327153500.10464-1-tomaszx.kulasek@intel.com> <6c556086-bccc-1e55-d490-c21bcc8a6c4b@redhat.com> From: Maxime Coquelin Message-ID: Date: Wed, 28 Mar 2018 12:56:18 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 28 Mar 2018 10:56:22 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.7]); Wed, 28 Mar 2018 10:56:22 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio configuration space messages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Mar 2018 10:56:23 -0000 On 03/28/2018 12:23 PM, Liu, Changpeng wrote: > > >> -----Original Message----- >> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >> Sent: Wednesday, March 28, 2018 6:11 PM >> To: Liu, Changpeng ; Kulasek, TomaszX >> ; yliu@fridaylinux.org >> Cc: Verkamp, Daniel ; Harris, James R >> ; Wodkowski, PawelX >> ; dev@dpdk.org; Tan, Jianfeng >> >> Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio configuration space >> messages >> >> >> >> On 03/28/2018 12:03 PM, Liu, Changpeng wrote: >>> >>> >>>> -----Original Message----- >>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >>>> Sent: Wednesday, March 28, 2018 5:58 PM >>>> To: Liu, Changpeng ; Kulasek, TomaszX >>>> ; yliu@fridaylinux.org >>>> Cc: Verkamp, Daniel ; Harris, James R >>>> ; Wodkowski, PawelX >>>> ; dev@dpdk.org; Tan, Jianfeng >>>> >>>> Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio configuration space >>>> messages >>>> >>>> >>>> >>>> On 03/28/2018 11:50 AM, Liu, Changpeng wrote: >>>>> >>>>> >>>>>> -----Original Message----- >>>>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >>>>>> Sent: Wednesday, March 28, 2018 5:12 PM >>>>>> To: Kulasek, TomaszX ; yliu@fridaylinux.org >>>>>> Cc: Verkamp, Daniel ; Harris, James R >>>>>> ; Wodkowski, PawelX >>>>>> ; dev@dpdk.org; Liu, Changpeng >>>>>> ; Tan, Jianfeng >>>>>> Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio configuration space >>>>>> messages >>>>>> >>>>>> >>>>>> >>>>>> On 03/27/2018 05:35 PM, Tomasz Kulasek wrote: >>>>>>> This patch adds new vhost user messages GET_CONFIG and SET_CONFIG >>>> used >>>>>>> for get/set virtio device's configuration space. >>>>>>> >>>>>>> Signed-off-by: Changpeng Liu >>>>>>> Signed-off-by: Tomasz Kulasek >>>>>>> --- >>>>>>> Changes in v2: >>>>>>> - code cleanup >>>>>>> >>>>>>> lib/librte_vhost/rte_vhost.h | 4 ++++ >>>>>>> lib/librte_vhost/vhost_user.c | 22 ++++++++++++++++++++++ >>>>>>> lib/librte_vhost/vhost_user.h | 16 ++++++++++++++++ >>>>>>> 3 files changed, 42 insertions(+) >>>>>>> >>>>>>> diff --git a/lib/librte_vhost/rte_vhost.h b/lib/librte_vhost/rte_vhost.h >>>>>>> index d332069..fe30518 100644 >>>>>>> --- a/lib/librte_vhost/rte_vhost.h >>>>>>> +++ b/lib/librte_vhost/rte_vhost.h >>>>>>> @@ -84,6 +84,10 @@ struct vhost_device_ops { >>>>>>> int (*new_connection)(int vid); >>>>>>> void (*destroy_connection)(int vid); >>>>>>> >>>>>>> + int (*get_config)(int vid, uint8_t *config, uint32_t config_len); >>>>>>> + int (*set_config)(int vid, uint8_t *config, uint32_t offset, >>>>>>> + uint32_t len, uint32_t flags); >>>>>>> + >>>>>>> void *reserved[2]; /**< Reserved for future extension */ >>>>>> >>>>>> You are breaking the ABI, as you grow the size of the ops struct. >>>>>> >>>>>> Also, I'm wondering if we shouldn't have a different ops for external >>>>>> backends. Here these ops are more intended to the application, we could >>>>>> have a specific ops struct for external backends IMHO. >>>>>> >>>>>>> }; >>>>>>> >>>>>>> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c >>>>>>> index 90ed211..0ed6a5a 100644 >>>>>>> --- a/lib/librte_vhost/vhost_user.c >>>>>>> +++ b/lib/librte_vhost/vhost_user.c >>>>>>> @@ -50,6 +50,8 @@ static const char >>>> *vhost_message_str[VHOST_USER_MAX] >>>>>> = { >>>>>>> [VHOST_USER_NET_SET_MTU] = "VHOST_USER_NET_SET_MTU", >>>>>>> [VHOST_USER_SET_SLAVE_REQ_FD] = >>>>>> "VHOST_USER_SET_SLAVE_REQ_FD", >>>>>>> [VHOST_USER_IOTLB_MSG] = "VHOST_USER_IOTLB_MSG", >>>>>>> + [VHOST_USER_GET_CONFIG] = "VHOST_USER_GET_CONFIG", >>>>>>> + [VHOST_USER_SET_CONFIG] = "VHOST_USER_SET_CONFIG", >>>>>>> }; >>>>>>> >>>>>>> static uint64_t >>>>>>> @@ -1355,6 +1357,7 @@ vhost_user_msg_handler(int vid, int fd) >>>>>>> * would cause a dead lock. >>>>>>> */ >>>>>>> switch (msg.request.master) { >>>>>>> + case VHOST_USER_SET_CONFIG: >>>>>> >>>>>> It seems VHOST_USER_GET_CONFIG is missing here. >>>>>> >>>>>>> case VHOST_USER_SET_FEATURES: >>>>>>> case VHOST_USER_SET_PROTOCOL_FEATURES: >>>>>>> case VHOST_USER_SET_OWNER: >>>>>>> @@ -1380,6 +1383,25 @@ vhost_user_msg_handler(int vid, int fd) >>>>>>> } >>>>>>> >>>>>>> switch (msg.request.master) { >>>>>>> + case VHOST_USER_GET_CONFIG: >>>>>>> + if (dev->notify_ops->get_config(dev->vid, >>>>>> Please check ->get_config is set before calling it. >>>>>> >>>>>>> + msg.payload.config.region, >>>>>>> + msg.payload.config.size) != 0) { >>>>>>> + msg.size = sizeof(uint64_t); >>>>>>> + } >>>>>>> + send_vhost_reply(fd, &msg); >>>>>>> + break; >>>>>>> + case VHOST_USER_SET_CONFIG: >>>>>>> + if ((dev->notify_ops->set_config(dev->vid, >>>>>> Ditto. >>>>>> >>>>>>> + msg.payload.config.region, >>>>>>> + msg.payload.config.offset, >>>>>>> + msg.payload.config.size, >>>>>>> + msg.payload.config.flags)) != 0) { >>>>>>> + ret = 1; >>>>>>> + } else { >>>>>>> + ret = 0; >>>>>>> + } >>>>>> >>>>>> ret = dev->notify_ops->set_config instead? >>>>>>> + break; >>>>>>> case VHOST_USER_GET_FEATURES: >>>>>>> msg.payload.u64 = vhost_user_get_features(dev); >>>>>>> msg.size = sizeof(msg.payload.u64); >>>>>>> diff --git a/lib/librte_vhost/vhost_user.h b/lib/librte_vhost/vhost_user.h >>>>>>> index d4bd604..25cc026 100644 >>>>>>> --- a/lib/librte_vhost/vhost_user.h >>>>>>> +++ b/lib/librte_vhost/vhost_user.h >>>>>>> @@ -14,6 +14,11 @@ >>>>>>> >>>>>>> #define VHOST_MEMORY_MAX_NREGIONS 8 >>>>>>> >>>>>>> +/* >>>>>>> + * Maximum size of virtio device config space >>>>>>> + */ >>>>>>> +#define VHOST_USER_MAX_CONFIG_SIZE 256 >>>>>>> + >>>>>>> #define VHOST_USER_PROTOCOL_F_MQ 0 >>>>>>> #define VHOST_USER_PROTOCOL_F_LOG_SHMFD 1 >>>>>>> #define VHOST_USER_PROTOCOL_F_RARP 2 >>>>>> >>>>>> Shouldn't there be a protocol feature associated to these new messages? >>>>>> Else how QEMU knows the backend supports it or not? >>>>>> >>>>>> I looked at QEMU code and indeed no protocol feature associated, that's >>>>>> strange... >>>>> Nice to have, for now not all the QEMU host driver need to get this >>>> configuration space from slave backend >>>>> when getting start. This message can be used for migration of vhost-user >>>> devices. >>>> >>>> So if QEMU sends this message but the DPDK version does not support it >>>> yet, vhost_user_msg_handler() will return an error ("vhost read >>>> incorrect message") and the socket will be closed. >>>> >>>> How do we overcome this? I think we really need a spec update ASAP, >>>> before QEMU v2.12 is out (-rc1 already). >>>> >>>> Do you have time to take care of this? >>> For now there are no other users except us care about this message, :), it's no >> hurry. >>> I can take this after QEMU 2.12 release adding a new protocol feature bit. >> >> Are you sure? >> If I understand the code correctly, as the guest writes in config regs >> of a virtio-blk device, .set_config callback will be called. > Exactly. >> >> If you have a vhost-user backend, it will receive the SET_CONFIG >> request, no? > For now this only enabled for QEMU vhost-user-blk driver, QEMU virtio-blk driver didn't have such issue. Right. But it will be really painful to manage for example for cross-version live migration. Or when you'll want to use QEMU v2.13+ with a DPDK v18.05 backend, the protocol feature won't be negotiated. Really, this is important to get it right at the beginning. Thanks, Maxime >> >> Cheers, >> Maxime >> >>>> >>>> Thanks, >>>> Maxime