From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 047F7A00E6 for ; Mon, 8 Jul 2019 13:38:07 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1E063325F; Mon, 8 Jul 2019 13:38:06 +0200 (CEST) Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 7AE72322C for ; Mon, 8 Jul 2019 13:38:04 +0200 (CEST) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20190708113803euoutp010225197c2053d86c708478ffac278fb7~va5T7hpK70548405484euoutp01S for ; Mon, 8 Jul 2019 11:38:03 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20190708113803euoutp010225197c2053d86c708478ffac278fb7~va5T7hpK70548405484euoutp01S DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1562585883; bh=QbmQwsyn+OJezcjvzedooKup9lHrk91k1yVqSPxAFk4=; h=Subject:To:From:Date:In-Reply-To:References:From; b=BRFUupMgtcOrCRcZ8qdWOwU4E83KtnKripCAotYg+8dKPZdJ5NYwJVx1FABCF6AVT FjIzQ78MtntFn7U75z7btYGBuI6jy2zkKXrU9Yf/KEt8Ac7Xy2DIoEUPVS8aw+5cCk UVxBvOSTyp5jH5gZc1QdIf4u8yyym3cs1iag1eIo= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20190708113802eucas1p2d687ecbf85c9d59534455c68fd0f8719~va5TUo4LN3168031680eucas1p2n; Mon, 8 Jul 2019 11:38:02 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 5E.5C.04377.A1B232D5; Mon, 8 Jul 2019 12:38:02 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20190708113801eucas1p25d89717d8b298790326077852c9933c8~va5SlRf2U2463624636eucas1p2a; Mon, 8 Jul 2019 11:38:01 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20190708113801eusmtrp24982e346a2b1539cb57a6a85e9b85aa0~va5SXPvWP2046120461eusmtrp2F; Mon, 8 Jul 2019 11:38:01 +0000 (GMT) X-AuditID: cbfec7f4-12dff70000001119-b7-5d232b1ae81b Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 08.5F.04140.91B232D5; Mon, 8 Jul 2019 12:38:01 +0100 (BST) Received: from [106.109.129.180] (unknown [106.109.129.180]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20190708113801eusmtip2e7a1b2f34732848fb2fd3bc51d21510d~va5R-RIbq0251202512eusmtip2M; Mon, 8 Jul 2019 11:38:01 +0000 (GMT) To: Marvin Liu , tiwei.bie@intel.com, maxime.coquelin@redhat.com, dev@dpdk.org From: Ilya Maximets Message-ID: <92467053-2cdc-300d-410b-c326924d676d@samsung.com> Date: Mon, 8 Jul 2019 14:37:56 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <20190708171320.38802-3-yong.liu@intel.com> Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprJKsWRmVeSWpSXmKPExsWy7djP87pS2sqxBpd/iVq8+7SdyeJK+092 i2Ode1gstjb8Z7LYdXgBiwOrx68FS1k9Fu95yeTxft9VNo++LasYA1iiuGxSUnMyy1KL9O0S uDJ2N99iKvigWbHwAW8D4xalLkYODgkBE4n5q0y7GLk4hARWMEoca7/PCOF8YZSYs/o0K4Tz mVFi2/1vQA4nWMeuDSeZIRLLGSX2/XzPBuF8ZJRYO38TE0iVsEC4xJy+b2wgtohAisSpr78Z QWw2AR2JU6uPMILs5hWwk2jqAxvKIqAi8fP5FHYQW1QgQuLUkXksIDavgKDEyZlPWEDKOQUs JG6vA5vCLCAu0fRlJSuELS+x/e0csHskBPrZJbr3tLJAHOoisfjyWjYIW1ji1fEt7BC2jMT/ nfOZIOx6ifstLxkhmjsYJaYf+geVsJfY8vocO8hiZgFNifW79CHCjhLbvvUzQoKOT+LGW0GI G/gkJm2bzgwR5pXoaBOCqFaR+H1wOTOELSVx891nqAs8JNZvvM44gVFxFpInZyH5bBaSz2Yh 3LCAkWUVo3hqaXFuemqxUV5quV5xYm5xaV66XnJ+7iZGYII5/e/4lx2Mu/4kHWIU4GBU4uHl kFOKFWJNLCuuzD3EKMHBrCTCmxgkHyvEm5JYWZValB9fVJqTWnyIUZqDRUmct5rhQbSQQHpi SWp2ampBahFMlomDU6qBkb28/sQOSQXHzf/cpaU/Pn2S3Pt7rkKOXCqrv/xa5l2FsseEpq9o 6+F4rGXWtib+xOHIRv3pZ/bte/D7AsejyYY75R7rlGzX5TS59PREw9XUBdJ1ckv6RETPdgif EnrOMtds6v34+ZknXJLKOF+x5ZjmnrcNfZgxdcrN3idN14PytfxORuptVGIpzkg01GIuKk4E AHDhVOAsAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprPIsWRmVeSWpSXmKPExsVy+t/xe7qS2sqxBscfqVi8+7SdyeJK+092 i2Ode1gstjb8Z7LYdXgBiwOrx68FS1k9Fu95yeTxft9VNo++LasYA1ii9GyK8ktLUhUy8otL bJWiDS2M9AwtLfSMTCz1DI3NY62MTJX07WxSUnMyy1KL9O0S9DJ2N99iKvigWbHwAW8D4xal LkZODgkBE4ldG04yg9hCAksZJWas4oCIS0n8+HWBFcIWlvhzrYuti5ELqOY9o8TJvVPYQRLC AuESc/q+sYHYIgIpEnumT2WHGGQuMfncK7A4m4COxKnVRxi7GDk4eAXsJJr6wGayCKhI/HwO MUZUIEJi0rWdLCA2r4CgxMmZT1hAyjkFLCRur2MECTMLqEv8mXeJGcIWl2j6spIVwpaX2P52 DvMERsFZSLpnIWmZhaRlFpKWBYwsqxhFUkuLc9Nzi430ihNzi0vz0vWS83M3MQIjZ9uxn1t2 MHa9Cz7EKMDBqMTDyyGnFCvEmlhWXJl7iFGCg1lJhDcxSD5WiDclsbIqtSg/vqg0J7X4EKMp 0G8TmaVEk/OBUZ1XEm9oamhuYWlobmxubGahJM7bIXAwRkggPbEkNTs1tSC1CKaPiYNTqoFR d8/cd24nGSTu1KzLuRW6jNWMZaVcZEHb/YrjTgoJKrVdi0tmtKa/fy3wkmn5z4eKh1j/hJg8 Mj0tGT3n1OvLu0TLpyjZC3L9e3VfwobljtZUJoktCktbP3/TEN1/K3PJqSNBC92uLQteeeSf 1mMP/674F9Mz1ZkmTbt39OChhHBvPv7mbC0OJZbijERDLeai4kQAp3OZwbICAAA= X-CMS-MailID: 20190708113801eucas1p25d89717d8b298790326077852c9933c8 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20190708113801eucas1p25d89717d8b298790326077852c9933c8 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20190708113801eucas1p25d89717d8b298790326077852c9933c8 References: <20190708171320.38802-3-yong.liu@intel.com> Subject: Re: [dpdk-dev] [RFC PATCH 02/13] add vhost packed ring fast enqueue function X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 08.07.2019 20:13, Marvin Liu wrote: > In fast enqueue function, will first check whether descriptors are > cache aligned. Fast enqueue function will check prerequisites in the > beginning. Fast enqueue function do not support chained mbufs, normal > function will handle that. > > Signed-off-by: Marvin Liu > > diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h > index 884befa85..f24026acd 100644 > --- a/lib/librte_vhost/vhost.h > +++ b/lib/librte_vhost/vhost.h > @@ -39,6 +39,8 @@ > > #define VHOST_LOG_CACHE_NR 32 > > +/* Used in fast packed ring functions */ > +#define PACKED_DESC_PER_CACHELINE (RTE_CACHE_LINE_SIZE / sizeof(struct vring_packed_desc)) > /** > * Structure contains buffer address, length and descriptor index > * from vring to do scatter RX. > diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c > index 003aec1d4..b877510da 100644 > --- a/lib/librte_vhost/virtio_net.c > +++ b/lib/librte_vhost/virtio_net.c > @@ -897,6 +897,115 @@ virtio_dev_rx_split(struct virtio_net *dev, struct vhost_virtqueue *vq, > return pkt_idx; > } > > +static __rte_always_inline uint16_t > +virtio_dev_rx_fast_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, > + struct rte_mbuf **pkts) > +{ > + bool wrap_counter = vq->avail_wrap_counter; > + struct vring_packed_desc *descs = vq->desc_packed; > + uint16_t avail_idx = vq->last_avail_idx; > + uint64_t desc_addr, desc_addr1, desc_addr2, desc_addr3, len, len1, > + len2, len3; > + struct virtio_net_hdr_mrg_rxbuf *hdr, *hdr1, *hdr2, *hdr3; > + uint32_t buf_offset = dev->vhost_hlen; > + > + if (unlikely(avail_idx & 0x3)) > + return -1; > + > + if (unlikely(avail_idx < (vq->size - PACKED_DESC_PER_CACHELINE))) Doe it makes sense to check this? If this condition is not 'true', all the code below will access incorrect memory. > + rte_prefetch0((void *)(uintptr_t)&descs[avail_idx + > + PACKED_DESC_PER_CACHELINE]); > + else > + rte_prefetch0((void *)(uintptr_t)&descs[0]); > + > + if (unlikely((pkts[0]->next != NULL) | > + (pkts[1]->next != NULL) | > + (pkts[2]->next != NULL) | > + (pkts[3]->next != NULL))) > + return -1; > + > + if (unlikely(!desc_is_avail(&descs[avail_idx], wrap_counter)) | > + unlikely(!desc_is_avail(&descs[avail_idx + 1], wrap_counter)) | > + unlikely(!desc_is_avail(&descs[avail_idx + 2], wrap_counter)) | > + unlikely(!desc_is_avail(&descs[avail_idx + 3], wrap_counter))) > + return 1; > + > + rte_smp_rmb(); > + > + len = descs[avail_idx].len; > + len1 = descs[avail_idx + 1].len; > + len2 = descs[avail_idx + 2].len; > + len3 = descs[avail_idx + 3].len; > + > + if (unlikely((pkts[0]->pkt_len > (len - buf_offset)) | > + (pkts[1]->pkt_len > (len1 - buf_offset)) | > + (pkts[2]->pkt_len > (len2 - buf_offset)) | > + (pkts[3]->pkt_len > (len3 - buf_offset)))) > + return -1; > + > + desc_addr = vhost_iova_to_vva(dev, vq, > + descs[avail_idx].addr, > + &len, > + VHOST_ACCESS_RW); > + > + desc_addr1 = vhost_iova_to_vva(dev, vq, > + descs[avail_idx + 1].addr, > + &len1, > + VHOST_ACCESS_RW); > + > + desc_addr2 = vhost_iova_to_vva(dev, vq, > + descs[avail_idx + 2].addr, > + &len2, > + VHOST_ACCESS_RW); > + > + desc_addr3 = vhost_iova_to_vva(dev, vq, > + descs[avail_idx + 3].addr, > + &len3, > + VHOST_ACCESS_RW); > + > + if (unlikely((len != descs[avail_idx].len) | > + (len1 != descs[avail_idx + 1].len) | > + (len2 != descs[avail_idx + 2].len) | > + (len3 != descs[avail_idx + 3].len))) > + return -1; > + > + hdr = (struct virtio_net_hdr_mrg_rxbuf *)(uintptr_t)desc_addr; > + hdr1 = (struct virtio_net_hdr_mrg_rxbuf *)(uintptr_t)desc_addr1; > + hdr2 = (struct virtio_net_hdr_mrg_rxbuf *)(uintptr_t)desc_addr2; > + hdr3 = (struct virtio_net_hdr_mrg_rxbuf *)(uintptr_t)desc_addr3; > + > + virtio_enqueue_offload(pkts[0], &hdr->hdr); > + virtio_enqueue_offload(pkts[1], &hdr1->hdr); > + virtio_enqueue_offload(pkts[2], &hdr2->hdr); > + virtio_enqueue_offload(pkts[3], &hdr3->hdr); > + > + len = pkts[0]->pkt_len + dev->vhost_hlen; > + len1 = pkts[1]->pkt_len + dev->vhost_hlen; > + len2 = pkts[2]->pkt_len + dev->vhost_hlen; > + len3 = pkts[3]->pkt_len + dev->vhost_hlen; > + > + vq->last_avail_idx += PACKED_DESC_PER_CACHELINE; The whole function assumes that PACKED_DESC_PER_CACHELINE equals to 4, but if it's not, it will not work correctly. > + if (vq->last_avail_idx >= vq->size) { > + vq->last_avail_idx -= vq->size; > + vq->avail_wrap_counter ^= 1; > + } > + > + rte_memcpy((void *)(uintptr_t)(desc_addr + buf_offset), > + rte_pktmbuf_mtod_offset(pkts[0], void *, 0), > + pkts[0]->pkt_len); > + rte_memcpy((void *)(uintptr_t)(desc_addr1 + buf_offset), > + rte_pktmbuf_mtod_offset(pkts[1], void *, 0), > + pkts[1]->pkt_len); > + rte_memcpy((void *)(uintptr_t)(desc_addr2 + buf_offset), > + rte_pktmbuf_mtod_offset(pkts[2], void *, 0), > + pkts[2]->pkt_len); > + rte_memcpy((void *)(uintptr_t)(desc_addr3 + buf_offset), > + rte_pktmbuf_mtod_offset(pkts[3], void *, 0), > + pkts[3]->pkt_len); > + > + return 0; > +} > + > static __rte_always_inline uint16_t > virtio_dev_rx_normal_packed(struct virtio_net *dev, struct vhost_virtqueue *vq, > struct rte_mbuf *pkt) >