From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by dpdk.org (Postfix) with ESMTP id 0B54A1BDF8 for ; Wed, 4 Jul 2018 22:20:32 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5C46C11D6F6; Wed, 4 Jul 2018 20:20:31 +0000 (UTC) Received: from [10.36.112.34] (ovpn-112-34.ams2.redhat.com [10.36.112.34]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 68F792026D76; Wed, 4 Jul 2018 20:20:29 +0000 (UTC) To: Tiwei Bie Cc: zhihong.wang@intel.com, jfreimann@redhat.com, dev@dpdk.org, mst@redhat.com, jasowang@redhat.com, wexu@redhat.com References: <20180702081629.29258-1-maxime.coquelin@redhat.com> <20180702081629.29258-15-maxime.coquelin@redhat.com> <20180704062524.GF28826@debian> From: Maxime Coquelin Message-ID: Date: Wed, 4 Jul 2018 22:20:27 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20180704062524.GF28826@debian> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 04 Jul 2018 20:20:31 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.1]); Wed, 04 Jul 2018 20:20:31 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'maxime.coquelin@redhat.com' RCPT:'' Subject: Re: [dpdk-dev] [PATCH v6 14/15] vhost: add notification for packed ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2018 20:20:32 -0000 On 07/04/2018 08:25 AM, Tiwei Bie wrote: > On Mon, Jul 02, 2018 at 10:16:28AM +0200, Maxime Coquelin wrote: >> Signed-off-by: Maxime Coquelin >> --- >> lib/librte_vhost/vhost.c | 73 ++++++++++++++++++++++++++++++++++++---- >> lib/librte_vhost/vhost.h | 71 ++++++++++++++++++++++++++++++++++---- >> lib/librte_vhost/vhost_user.c | 24 +++++++++++++ >> lib/librte_vhost/virtio-packed.h | 11 ++++++ >> lib/librte_vhost/virtio_net.c | 12 +++---- >> 5 files changed, 172 insertions(+), 19 deletions(-) >> >> diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c >> index 8538302c9..78f20c402 100644 >> --- a/lib/librte_vhost/vhost.c >> +++ b/lib/librte_vhost/vhost.c >> @@ -171,6 +171,24 @@ vring_translate_packed(struct virtio_net *dev, struct vhost_virtqueue *vq) >> if (!vq->desc_packed || size != req_size) >> return -1; >> >> + req_size = sizeof(struct vring_packed_desc_event); >> + size = req_size; >> + vq->driver_event = (struct vring_packed_desc_event *)(uintptr_t) >> + vhost_iova_to_vva(dev, >> + vq, vq->ring_addrs.avail_user_addr, >> + &size, VHOST_ACCESS_RW); > > It should be a space instead of a tab after "vq," > > Why not put "vq, vq->ring_addrs.avail_user_addr," > and "vhost_iova_to_vva(dev," on the same line? > >> + if (!vq->driver_event || size != req_size) >> + return -1; >> + >> + req_size = sizeof(struct vring_packed_desc_event); >> + size = req_size; >> + vq->device_event = (struct vring_packed_desc_event *)(uintptr_t) >> + vhost_iova_to_vva(dev, >> + vq, vq->ring_addrs.used_user_addr, > > It's better to put "vhost_iova_to_vva(dev," and > "vq, vq->ring_addrs.used_user_addr," on the same line. > > Currently, it looks like something as this in my editor: > > ¦ vq->device_event = (struct vring_packed_desc_event *)(uintptr_t) > ¦ ¦ vhost_iova_to_vva(dev, > ¦ ¦ ¦ ¦ ¦ vq, vq->ring_addrs.used_user_addr, > ¦ ¦ ¦ ¦ ¦ &size, VHOST_ACCESS_RW); > It looked a bit cleaner in my editor :) This is fixed now. >> + &size, VHOST_ACCESS_RW); >> + if (!vq->device_event || size != req_size) >> + return -1; >> + >> return 0; >> } >> > [...] >> @@ -640,4 +647,54 @@ vhost_vring_call(struct virtio_net *dev, struct vhost_virtqueue *vq) >> } >> } >> >> +static __rte_always_inline void >> +vhost_vring_call_packed(struct virtio_net *dev, struct vhost_virtqueue *vq) >> +{ >> + uint16_t old, new, off, off_wrap, wrap; >> + bool kick = false; >> + >> + > > There is no need to have two blank lines. > >> + /* Flush used desc update. */ > > Just need one space between "/*" and "Flush". > >> + rte_smp_mb(); >> + >> + if (!(dev->features & (1ULL << VIRTIO_RING_F_EVENT_IDX))) { >> + if (vq->driver_event->desc_event_flags != >> + RING_EVENT_FLAGS_DISABLE) >> + kick = true; >> + goto kick; >> + } >> + >> + old = vq->signalled_used; >> + new = vq->last_used_idx; >> + vq->signalled_used = new; >> + >> + if (vq->driver_event->desc_event_flags != RING_EVENT_FLAGS_DESC) { >> + if (vq->driver_event->desc_event_flags != >> + RING_EVENT_FLAGS_DISABLE) >> + kick = true; >> + goto kick; >> + } >> + >> + rte_smp_rmb(); >> + >> + off_wrap = vq->driver_event->desc_event_off_wrap; >> + off = off_wrap & ~(1 << 15); > > Maybe it's better to use: RING_EVENT_OFF_MASK I planned to remove its definition to be consistent with the virtio_ring.h kernel header. > >> + wrap = vq->used_wrap_counter; >> + >> + if (new < old) { >> + new += vq->size; >> + wrap ^= 1; >> + } >> + >> + if (wrap != off_wrap >> 15) >> + off += vq->size; >> + >> + if (vhost_need_event(off, new, old)) >> + kick = true; >> + >> +kick: >> + if (kick) >> + eventfd_write(vq->callfd, (eventfd_t)1); >> +} >> + >> #endif /* _VHOST_NET_CDEV_H_ */ >> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c >> index b2b57de57..bda515bdb 100644 >> --- a/lib/librte_vhost/vhost_user.c >> +++ b/lib/librte_vhost/vhost_user.c >> @@ -523,6 +523,30 @@ translate_ring_addresses(struct virtio_net *dev, int vq_index) >> vq = dev->virtqueue[vq_index]; >> addr = &vq->ring_addrs; >> >> + len = sizeof(struct vring_packed_desc_event); >> + vq->driver_event = (struct vring_packed_desc_event *) >> + (uintptr_t)ring_addr_to_vva(dev, >> + vq, addr->avail_user_addr, &len); >> + if (vq->driver_event == 0 || > > It's better to compare with NULL. Yes. >> + len != sizeof(struct vring_packed_desc_event)) { >> + RTE_LOG(DEBUG, VHOST_CONFIG, >> + "(%d) failed to find driver area address.\n", >> + dev->vid); >> + return dev; >> + } >> + >> + len = sizeof(struct vring_packed_desc_event); >> + vq->device_event = (struct vring_packed_desc_event *) >> + (uintptr_t)ring_addr_to_vva(dev, >> + vq, addr->used_user_addr, &len); >> + if (vq->device_event == 0 || > > It's better to compare with NULL. > >> + len != sizeof(struct vring_packed_desc_event)) { >> + RTE_LOG(DEBUG, VHOST_CONFIG, >> + "(%d) failed to find device area address.\n", >> + dev->vid); >> + return dev; >> + } >> + >> return dev; >> } >> >> diff --git a/lib/librte_vhost/virtio-packed.h b/lib/librte_vhost/virtio-packed.h >> index d386cb6df..ce3b28313 100644 >> --- a/lib/librte_vhost/virtio-packed.h >> +++ b/lib/librte_vhost/virtio-packed.h > > Normally, we name c/h files as something like: virtio_packed.h > > Besides, it'd be better to move the definitions into > vhost.h and define the types and flags for packed > ring only when e.g. VIRTIO_F_RING_PACKED not defined. > Just like how VIRTIO_F_IOMMU_PLATFORM works: > > /* Declare IOMMU related bits for older kernels */ > #ifndef VIRTIO_F_IOMMU_PLATFORM > > #define VIRTIO_F_IOMMU_PLATFORM 33 > > struct vhost_iotlb_msg { > __u64 iova; > __u64 size; > __u64 uaddr; > #define VHOST_ACCESS_RO 0x1 > #define VHOST_ACCESS_WO 0x2 > #define VHOST_ACCESS_RW 0x3 > __u8 perm; > #define VHOST_IOTLB_MISS 1 > #define VHOST_IOTLB_UPDATE 2 > #define VHOST_IOTLB_INVALIDATE 3 > #define VHOST_IOTLB_ACCESS_FAIL 4 > __u8 type; > }; > > #define VHOST_IOTLB_MSG 0x1 > > struct vhost_msg { > int type; > union { > struct vhost_iotlb_msg iotlb; > __u8 padding[64]; > }; > }; > #endif I agree. I replied to Jason series so that the defines name are consistent. >> @@ -19,6 +19,17 @@ struct vring_desc_packed { >> uint16_t flags; >> }; >> >> +#define RING_EVENT_FLAGS_ENABLE 0x0 >> +#define RING_EVENT_FLAGS_DISABLE 0x1 >> +#define RING_EVENT_FLAGS_DESC 0x2 >> +#define RING_EVENT_FLAGS_MASK 0xFFFC > > RING_EVENT_FLAGS_MASK should be 0x3? I actually removed it, but yes I agree. >> +#define RING_EVENT_WRAP_MASK 0x8000 >> +#define RING_EVENT_OFF_MASK 0x7FFF > > It's better to define above macros as VRING_EVENT_* Yes, I change it to VRING_EVENT_F_* as you suggested to Jason. >> + >> +struct vring_packed_desc_event { >> + uint16_t desc_event_off_wrap; >> + uint16_t desc_event_flags; >> +}; >> > [...] >