From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 8291CA2F6B
for <public@inbox.dpdk.org>; Wed, 9 Oct 2019 07:59:52 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 1B5761C115;
Wed, 9 Oct 2019 07:59:34 +0200 (CEST)
Received: from mga12.intel.com (mga12.intel.com [192.55.52.136])
by dpdk.org (Postfix) with ESMTP id 354F11C07F
for <dev@dpdk.org>; Wed, 9 Oct 2019 07:59:29 +0200 (CEST)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga002.fm.intel.com ([10.253.24.26])
by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
08 Oct 2019 22:59:30 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.67,273,1566889200"; d="scan'208";a="223473380"
Received: from npg-dpdk-virtual-marvin-dev.sh.intel.com ([10.67.119.142])
by fmsmga002.fm.intel.com with ESMTP; 08 Oct 2019 22:59:27 -0700
From: Marvin Liu <yong.liu@intel.com>
To: maxime.coquelin@redhat.com, tiwei.bie@intel.com, zhihong.wang@intel.com,
stephen@networkplumber.org, gavin.hu@arm.com
Cc: dev@dpdk.org,
Marvin Liu <yong.liu@intel.com>
Date: Wed, 9 Oct 2019 21:38:38 +0800
Message-Id: <20191009133849.69002-4-yong.liu@intel.com>
X-Mailer: git-send-email 2.17.1
In-Reply-To: <20191009133849.69002-1-yong.liu@intel.com>
References: <20190925171329.63734-1-yong.liu@intel.com>
<20191009133849.69002-1-yong.liu@intel.com>
Subject: [dpdk-dev] [PATCH v4 03/14] vhost: add batch enqueue function for
packed ring
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Batch enqueue function will first check whether descriptors are cache
aligned. It will also check prerequisites in the beginning. Batch
enqueue function not support chained mbufs, single packet enqueue
function will handle it.
Signed-off-by: Marvin Liu <yong.liu@intel.com>
diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h
index 4cba8c5ef..e241436c7 100644
--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -39,6 +39,10 @@
#define VHOST_LOG_CACHE_NR 32
+#define PACKED_BATCH_SIZE (RTE_CACHE_LINE_SIZE / \
+ sizeof(struct vring_packed_desc))
+#define PACKED_BATCH_MASK (PACKED_BATCH_SIZE - 1)
+
#ifdef SUPPORT_GCC_UNROLL_PRAGMA
#define UNROLL_PRAGMA_PARAM "GCC unroll 4"
#endif
diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c
index 520c4c6a8..5e08f7d9b 100644
--- a/lib/librte_vhost/virtio_net.c
+++ b/lib/librte_vhost/virtio_net.c
@@ -883,6 +883,86 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq,
return pkt_idx;
}
+static __rte_unused int
+virtio_dev_rx_batch_packed(struct virtio_net *dev, struct vhost_virtqueue *vq,
+ struct rte_mbuf **pkts)
+{
+ bool wrap_counter = vq->avail_wrap_counter;
+ struct vring_packed_desc *descs = vq->desc_packed;
+ uint16_t avail_idx = vq->last_avail_idx;
+ uint64_t desc_addrs[PACKED_BATCH_SIZE];
+ struct virtio_net_hdr_mrg_rxbuf *hdrs[PACKED_BATCH_SIZE];
+ uint32_t buf_offset = dev->vhost_hlen;
+ uint64_t lens[PACKED_BATCH_SIZE];
+ uint16_t i;
+
+ if (unlikely(avail_idx & PACKED_BATCH_MASK))
+ return -1;
+
+ if (unlikely((avail_idx + PACKED_BATCH_SIZE) > vq->size))
+ return -1;
+
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++) {
+ if (unlikely(pkts[i]->next != NULL))
+ return -1;
+ if (unlikely(!desc_is_avail(&descs[avail_idx + i],
+ wrap_counter)))
+ return -1;
+ }
+
+ rte_smp_rmb();
+
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++)
+ lens[i] = descs[avail_idx + i].len;
+
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++) {
+ if (unlikely(pkts[i]->pkt_len > (lens[i] - buf_offset)))
+ return -1;
+ }
+
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++)
+ desc_addrs[i] = vhost_iova_to_vva(dev, vq,
+ descs[avail_idx + i].addr,
+ &lens[i],
+ VHOST_ACCESS_RW);
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++) {
+ if (unlikely(lens[i] != descs[avail_idx + i].len))
+ return -1;
+ }
+
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++) {
+ rte_prefetch0((void *)(uintptr_t)desc_addrs[i]);
+ hdrs[i] = (struct virtio_net_hdr_mrg_rxbuf *)
+ (uintptr_t)desc_addrs[i];
+ lens[i] = pkts[i]->pkt_len + dev->vhost_hlen;
+ }
+
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++)
+ virtio_enqueue_offload(pkts[i], &hdrs[i]->hdr);
+
+ vq->last_avail_idx += PACKED_BATCH_SIZE;
+ if (vq->last_avail_idx >= vq->size) {
+ vq->last_avail_idx -= vq->size;
+ vq->avail_wrap_counter ^= 1;
+ }
+
+ UNROLL_PRAGMA(UNROLL_PRAGMA_PARAM)
+ for (i = 0; i < PACKED_BATCH_SIZE; i++) {
+ rte_memcpy((void *)(uintptr_t)(desc_addrs[i] + buf_offset),
+ rte_pktmbuf_mtod_offset(pkts[i], void *, 0),
+ pkts[i]->pkt_len);
+ }
+
+ return 0;
+}
+
static __rte_unused int16_t
virtio_dev_rx_single_packed(struct virtio_net *dev, struct vhost_virtqueue *vq,
struct rte_mbuf *pkt)
--
2.17.1