From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 788E01B3BB for ; Wed, 28 Nov 2018 18:25:57 +0100 (CET) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A57973154853; Wed, 28 Nov 2018 17:25:56 +0000 (UTC) Received: from [10.36.112.54] (ovpn-112-54.ams2.redhat.com [10.36.112.54]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BC9845EDE3; Wed, 28 Nov 2018 17:25:51 +0000 (UTC) To: Jens Freimann , dev@dpdk.org Cc: tiwei.bie@intel.com, Gavin.Hu@arm.com References: <20181102090749.17316-1-jfreimann@redhat.com> <20181102090749.17316-8-jfreimann@redhat.com> From: Maxime Coquelin Message-ID: Date: Wed, 28 Nov 2018 18:25:49 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181102090749.17316-8-jfreimann@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Wed, 28 Nov 2018 17:25:56 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH v10 7/9] net/virtio: add virtio send command packed queue support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Nov 2018 17:25:57 -0000 On 11/2/18 10:07 AM, Jens Freimann wrote: > Use packed virtqueue format when reading and writing descriptors > to/from the ring. > > Signed-off-by: Jens Freimann > --- > drivers/net/virtio/virtio_ethdev.c | 90 ++++++++++++++++++++++++++++++ > 1 file changed, 90 insertions(+) > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > index ed3d5bd2c..4451dd49b 100644 > --- a/drivers/net/virtio/virtio_ethdev.c > +++ b/drivers/net/virtio/virtio_ethdev.c > @@ -141,6 +141,90 @@ static const struct rte_virtio_xstats_name_off rte_virtio_txq_stat_strings[] = { > > struct virtio_hw_internal virtio_hw_internal[RTE_MAX_ETHPORTS]; > > +static struct virtio_pmd_ctrl * > +virtio_pq_send_command(struct virtnet_ctl *cvq, struct virtio_pmd_ctrl *ctrl, > + int *dlen, int pkt_num) > +{ > + struct virtqueue *vq = cvq->vq; > + int head; > + struct vring_desc_packed *desc = vq->ring_packed.desc_packed; > + struct virtio_pmd_ctrl *result; > + int wrap_counter; > + int sum = 0; > + int k; > + > + /* > + * Format is enforced in qemu code: > + * One TX packet for header; > + * At least one TX packet per argument; > + * One RX packet for ACK. > + */ > + head = vq->vq_avail_idx; > + wrap_counter = vq->avail_wrap_counter; > + desc[head].flags = VRING_DESC_F_NEXT; > + desc[head].addr = cvq->virtio_net_hdr_mem; > + desc[head].len = sizeof(struct virtio_net_ctrl_hdr); > + vq->vq_free_cnt--; > + if (++vq->vq_avail_idx >= vq->vq_nentries) { > + vq->vq_avail_idx -= vq->vq_nentries; > + vq->avail_wrap_counter ^= 1; > + } > + > + for (k = 0; k < pkt_num; k++) { > + desc[vq->vq_avail_idx].addr = cvq->virtio_net_hdr_mem > + + sizeof(struct virtio_net_ctrl_hdr) > + + sizeof(ctrl->status) + sizeof(uint8_t) * sum; > + desc[vq->vq_avail_idx].len = dlen[k]; > + desc[vq->vq_avail_idx].flags = VRING_DESC_F_NEXT; > + sum += dlen[k]; > + vq->vq_free_cnt--; > + _set_desc_avail(&desc[vq->vq_avail_idx], > + vq->avail_wrap_counter); > + rte_smp_wmb(); > + vq->vq_free_cnt--; > + if (++vq->vq_avail_idx >= vq->vq_nentries) { > + vq->vq_avail_idx -= vq->vq_nentries; > + vq->avail_wrap_counter ^= 1; > + } > + } > + > + > + desc[vq->vq_avail_idx].addr = cvq->virtio_net_hdr_mem > + + sizeof(struct virtio_net_ctrl_hdr); > + desc[vq->vq_avail_idx].len = sizeof(ctrl->status); > + desc[vq->vq_avail_idx].flags = VRING_DESC_F_WRITE; > + _set_desc_avail(&desc[vq->vq_avail_idx], > + vq->avail_wrap_counter); > + _set_desc_avail(&desc[head], wrap_counter); > + rte_smp_wmb(); > + > + vq->vq_free_cnt--; > + if (++vq->vq_avail_idx >= vq->vq_nentries) { > + vq->vq_avail_idx -= vq->vq_nentries; > + vq->avail_wrap_counter ^= 1; > + } > + > + virtqueue_notify(vq); > + > + /* wait for used descriptors in virtqueue */ > + do { > + rte_rmb(); > + usleep(100); > + } while (!desc_is_used(&desc[head], vq)); Maybe better to poll for !desc_is_used(&desc[vq->vq_used_cons_idx], vq)) to make it clear we're safe using used_wrap_counter? > + > + /* now get used descriptors */ > + while(desc_is_used(&desc[vq->vq_used_cons_idx], vq)) { > + vq->vq_free_cnt++; > + if (++vq->vq_used_cons_idx >= vq->vq_nentries) { > + vq->vq_used_cons_idx -= vq->vq_nentries; > + vq->used_wrap_counter ^= 1; > + } > + } > + > + result = cvq->virtio_net_hdr_mz->addr; > + return result; > +} > + > static int > virtio_send_command(struct virtnet_ctl *cvq, struct virtio_pmd_ctrl *ctrl, > int *dlen, int pkt_num) > @@ -174,6 +258,11 @@ virtio_send_command(struct virtnet_ctl *cvq, struct virtio_pmd_ctrl *ctrl, > memcpy(cvq->virtio_net_hdr_mz->addr, ctrl, > sizeof(struct virtio_pmd_ctrl)); > > + if (vtpci_packed_queue(vq->hw)) { > + result = virtio_pq_send_command(cvq, ctrl, dlen, pkt_num); > + goto out_unlock; > + } > + > /* > * Format is enforced in qemu code: > * One TX packet for header; > @@ -245,6 +334,7 @@ virtio_send_command(struct virtnet_ctl *cvq, struct virtio_pmd_ctrl *ctrl, > > result = cvq->virtio_net_hdr_mz->addr; > > +out_unlock: > rte_spinlock_unlock(&cvq->lock); > return result->status; > } >