From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id CF2A47F58 for ; Thu, 19 Apr 2018 16:39:53 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C2AF4075A6A; Thu, 19 Apr 2018 14:39:53 +0000 (UTC) Received: from [10.36.112.62] (ovpn-112-62.ams2.redhat.com [10.36.112.62]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 45ED221568B2; Thu, 19 Apr 2018 14:39:51 +0000 (UTC) From: Maxime Coquelin To: "Liu, Changpeng" , "Kulasek, TomaszX" , "yliu@fridaylinux.org" Cc: "Verkamp, Daniel" , "Harris, James R" , "Wodkowski, PawelX" , "dev@dpdk.org" , "Tan, Jianfeng" References: <20180327151737.6640-1-tomaszx.kulasek@intel.com> <20180327153500.10464-1-tomaszx.kulasek@intel.com> <6c556086-bccc-1e55-d490-c21bcc8a6c4b@redhat.com> Message-ID: <980e5103-22ca-e526-9ac5-ee5b775b50ba@redhat.com> Date: Thu, 19 Apr 2018 16:39:49 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 19 Apr 2018 14:39:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Thu, 19 Apr 2018 14:39:53 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio configuration space messages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Apr 2018 14:39:54 -0000 Hi Changpeng, Tomasz, Any chance that you resubmit the series now that the Qemu changes adding a protocol feature flag has been accepted? Cheers, Maxime On 03/28/2018 12:56 PM, Maxime Coquelin wrote: > > > On 03/28/2018 12:23 PM, Liu, Changpeng wrote: >> >> >>> -----Original Message----- >>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >>> Sent: Wednesday, March 28, 2018 6:11 PM >>> To: Liu, Changpeng ; Kulasek, TomaszX >>> ; yliu@fridaylinux.org >>> Cc: Verkamp, Daniel ; Harris, James R >>> ; Wodkowski, PawelX >>> ; dev@dpdk.org; Tan, Jianfeng >>> >>> Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio configuration space >>> messages >>> >>> >>> >>> On 03/28/2018 12:03 PM, Liu, Changpeng wrote: >>>> >>>> >>>>> -----Original Message----- >>>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >>>>> Sent: Wednesday, March 28, 2018 5:58 PM >>>>> To: Liu, Changpeng ; Kulasek, TomaszX >>>>> ; yliu@fridaylinux.org >>>>> Cc: Verkamp, Daniel ; Harris, James R >>>>> ; Wodkowski, PawelX >>>>> ; dev@dpdk.org; Tan, Jianfeng >>>>> >>>>> Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio configuration >>>>> space >>>>> messages >>>>> >>>>> >>>>> >>>>> On 03/28/2018 11:50 AM, Liu, Changpeng wrote: >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Maxime Coquelin [mailto:maxime.coquelin@redhat.com] >>>>>>> Sent: Wednesday, March 28, 2018 5:12 PM >>>>>>> To: Kulasek, TomaszX ; >>>>>>> yliu@fridaylinux.org >>>>>>> Cc: Verkamp, Daniel ; Harris, James R >>>>>>> ; Wodkowski, PawelX >>>>>>> ; dev@dpdk.org; Liu, Changpeng >>>>>>> ; Tan, Jianfeng >>>>>>> Subject: Re: [dpdk-dev] [PATCH v2] vhost: add virtio >>>>>>> configuration space >>>>>>> messages >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 03/27/2018 05:35 PM, Tomasz Kulasek wrote: >>>>>>>> This patch adds new vhost user messages GET_CONFIG and SET_CONFIG >>>>> used >>>>>>>> for get/set virtio device's configuration space. >>>>>>>> >>>>>>>> Signed-off-by: Changpeng Liu >>>>>>>> Signed-off-by: Tomasz Kulasek >>>>>>>> --- >>>>>>>> Changes in v2: >>>>>>>>      - code cleanup >>>>>>>> >>>>>>>>      lib/librte_vhost/rte_vhost.h  |  4 ++++ >>>>>>>>      lib/librte_vhost/vhost_user.c | 22 ++++++++++++++++++++++ >>>>>>>>      lib/librte_vhost/vhost_user.h | 16 ++++++++++++++++ >>>>>>>>      3 files changed, 42 insertions(+) >>>>>>>> >>>>>>>> diff --git a/lib/librte_vhost/rte_vhost.h >>>>>>>> b/lib/librte_vhost/rte_vhost.h >>>>>>>> index d332069..fe30518 100644 >>>>>>>> --- a/lib/librte_vhost/rte_vhost.h >>>>>>>> +++ b/lib/librte_vhost/rte_vhost.h >>>>>>>> @@ -84,6 +84,10 @@ struct vhost_device_ops { >>>>>>>>          int (*new_connection)(int vid); >>>>>>>>          void (*destroy_connection)(int vid); >>>>>>>> >>>>>>>> +    int (*get_config)(int vid, uint8_t *config, uint32_t >>>>>>>> config_len); >>>>>>>> +    int (*set_config)(int vid, uint8_t *config, uint32_t offset, >>>>>>>> +            uint32_t len, uint32_t flags); >>>>>>>> + >>>>>>>>          void *reserved[2]; /**< Reserved for future extension */ >>>>>>> >>>>>>> You are breaking the ABI, as you grow the size of the ops struct. >>>>>>> >>>>>>> Also, I'm wondering if we shouldn't have a different ops for >>>>>>> external >>>>>>> backends. Here these ops are more intended to the application, we >>>>>>> could >>>>>>> have a specific ops struct for external backends IMHO. >>>>>>> >>>>>>>>      }; >>>>>>>> >>>>>>>> diff --git a/lib/librte_vhost/vhost_user.c >>>>>>>> b/lib/librte_vhost/vhost_user.c >>>>>>>> index 90ed211..0ed6a5a 100644 >>>>>>>> --- a/lib/librte_vhost/vhost_user.c >>>>>>>> +++ b/lib/librte_vhost/vhost_user.c >>>>>>>> @@ -50,6 +50,8 @@ static const char >>>>> *vhost_message_str[VHOST_USER_MAX] >>>>>>> = { >>>>>>>>          [VHOST_USER_NET_SET_MTU]  = "VHOST_USER_NET_SET_MTU", >>>>>>>>          [VHOST_USER_SET_SLAVE_REQ_FD]  = >>>>>>> "VHOST_USER_SET_SLAVE_REQ_FD", >>>>>>>>          [VHOST_USER_IOTLB_MSG]  = "VHOST_USER_IOTLB_MSG", >>>>>>>> +    [VHOST_USER_GET_CONFIG] = "VHOST_USER_GET_CONFIG", >>>>>>>> +    [VHOST_USER_SET_CONFIG] = "VHOST_USER_SET_CONFIG", >>>>>>>>      }; >>>>>>>> >>>>>>>>      static uint64_t >>>>>>>> @@ -1355,6 +1357,7 @@ vhost_user_msg_handler(int vid, int fd) >>>>>>>>           * would cause a dead lock. >>>>>>>>           */ >>>>>>>>          switch (msg.request.master) { >>>>>>>> +    case VHOST_USER_SET_CONFIG: >>>>>>> >>>>>>> It seems VHOST_USER_GET_CONFIG is missing here. >>>>>>> >>>>>>>>          case VHOST_USER_SET_FEATURES: >>>>>>>>          case VHOST_USER_SET_PROTOCOL_FEATURES: >>>>>>>>          case VHOST_USER_SET_OWNER: >>>>>>>> @@ -1380,6 +1383,25 @@ vhost_user_msg_handler(int vid, int fd) >>>>>>>>          } >>>>>>>> >>>>>>>>          switch (msg.request.master) { >>>>>>>> +    case VHOST_USER_GET_CONFIG: >>>>>>>> +        if (dev->notify_ops->get_config(dev->vid, >>>>>>> Please check ->get_config is set before calling it. >>>>>>> >>>>>>>> +                msg.payload.config.region, >>>>>>>> +                msg.payload.config.size) != 0) { >>>>>>>> +            msg.size = sizeof(uint64_t); >>>>>>>> +        } >>>>>>>> +        send_vhost_reply(fd, &msg); >>>>>>>> +        break; >>>>>>>> +    case VHOST_USER_SET_CONFIG: >>>>>>>> +        if ((dev->notify_ops->set_config(dev->vid, >>>>>>> Ditto. >>>>>>> >>>>>>>> +                msg.payload.config.region, >>>>>>>> +                msg.payload.config.offset, >>>>>>>> +                msg.payload.config.size, >>>>>>>> +                msg.payload.config.flags)) != 0) { >>>>>>>> +            ret = 1; >>>>>>>> +        } else { >>>>>>>> +            ret = 0; >>>>>>>> +        } >>>>>>> >>>>>>> ret = dev->notify_ops->set_config instead? >>>>>>>> +        break; >>>>>>>>          case VHOST_USER_GET_FEATURES: >>>>>>>>              msg.payload.u64 = vhost_user_get_features(dev); >>>>>>>>              msg.size = sizeof(msg.payload.u64); >>>>>>>> diff --git a/lib/librte_vhost/vhost_user.h >>>>>>>> b/lib/librte_vhost/vhost_user.h >>>>>>>> index d4bd604..25cc026 100644 >>>>>>>> --- a/lib/librte_vhost/vhost_user.h >>>>>>>> +++ b/lib/librte_vhost/vhost_user.h >>>>>>>> @@ -14,6 +14,11 @@ >>>>>>>> >>>>>>>>      #define VHOST_MEMORY_MAX_NREGIONS 8 >>>>>>>> >>>>>>>> +/* >>>>>>>> + * Maximum size of virtio device config space >>>>>>>> + */ >>>>>>>> +#define VHOST_USER_MAX_CONFIG_SIZE 256 >>>>>>>> + >>>>>>>>      #define VHOST_USER_PROTOCOL_F_MQ    0 >>>>>>>>      #define VHOST_USER_PROTOCOL_F_LOG_SHMFD    1 >>>>>>>>      #define VHOST_USER_PROTOCOL_F_RARP    2 >>>>>>> >>>>>>> Shouldn't there be a protocol feature associated to these new >>>>>>> messages? >>>>>>> Else how QEMU knows the backend supports it or not? >>>>>>> >>>>>>> I looked at QEMU code and indeed no protocol feature associated, >>>>>>> that's >>>>>>> strange... >>>>>> Nice to have, for now not all the QEMU host driver need to get this >>>>> configuration space from slave backend >>>>>> when getting start. This message can be used for migration of >>>>>> vhost-user >>>>> devices. >>>>> >>>>> So if QEMU sends this message but the DPDK version does not support it >>>>> yet, vhost_user_msg_handler() will return an error ("vhost read >>>>> incorrect message") and the socket will be closed. >>>>> >>>>> How do we overcome this? I think we really need a spec update ASAP, >>>>> before QEMU v2.12 is out (-rc1 already). >>>>> >>>>> Do you have time to take care of this? >>>> For now there are no other users except us care about this message, >>>> :), it's no >>> hurry. >>>> I can take this after QEMU 2.12 release adding a new protocol >>>> feature bit. >>> >>> Are you sure? >>> If I understand the code correctly, as the guest writes in config regs >>> of a virtio-blk device, .set_config callback will be called. >> Exactly. >>> >>> If you have a vhost-user backend, it will receive the SET_CONFIG >>> request, no? >> For now this only enabled for QEMU vhost-user-blk driver, QEMU >> virtio-blk driver didn't have such issue. > > Right. > But it will be really painful to manage for example for cross-version > live migration. Or when you'll want to use QEMU v2.13+ with a DPDK > v18.05 backend, the protocol feature won't be negotiated. > > Really, this is important to get it right at the beginning. > > Thanks, > Maxime >>> >>> Cheers, >>> Maxime >>> >>>>> >>>>> Thanks, >>>>> Maxime