From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout3.w1.samsung.com (mailout3.w1.samsung.com [210.118.77.13]) by dpdk.org (Postfix) with ESMTP id 6366EC48C for ; Fri, 19 Feb 2016 07:32:58 +0100 (CET) Received: from eucpsbgm2.samsung.com (unknown [203.254.199.245]) by mailout3.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0O2S00BF67IXYE70@mailout3.w1.samsung.com> for dev@dpdk.org; Fri, 19 Feb 2016 06:32:57 +0000 (GMT) X-AuditID: cbfec7f5-f79b16d000005389-54-56c6b7193107 Received: from eusync4.samsung.com ( [203.254.199.214]) by eucpsbgm2.samsung.com (EUCPMTA) with SMTP id 7B.48.21385.917B6C65; Fri, 19 Feb 2016 06:32:57 +0000 (GMT) Received: from imaximets.rnd.samsung.ru ([106.109.129.180]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0O2S00G447IL8D70@eusync4.samsung.com>; Fri, 19 Feb 2016 06:32:57 +0000 (GMT) From: Ilya Maximets To: dev@dpdk.org, Huawei Xie , Yuanhan Liu Date: Fri, 19 Feb 2016 09:32:42 +0300 Message-id: <1455863563-15751-4-git-send-email-i.maximets@samsung.com> X-Mailer: git-send-email 2.5.0 In-reply-to: <1455863563-15751-1-git-send-email-i.maximets@samsung.com> References: <1455863563-15751-1-git-send-email-i.maximets@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprCLMWRmVeSWpSXmKPExsVy+t/xa7qS24+FGay9I2Dx7tN2Jov2mWeZ LK60/2S3mDjJxGL60wiLybOlLK5PuMDqwO6x4UQ/q8evBUtZPRbvecnkcefaHjaPeScDPfq2 rGIMYIvisklJzcksSy3St0vgyvi+7g1rwXO+iqkLlzM1MP7k7mLk5JAQMJH4PvMwE4QtJnHh 3nq2LkYuDiGBpYwSc670QTmtTBKt324xglSxCehInFp9BMwWEUiQOLL/NytIEbPAMkaJiycm A43i4BAWiJaYtIIdxGQRUJWY/YQLpJxXwE3ie8sCVpCwhICcxIIL6SBhTgF3iS+bWsAmCgGV 9J3dxTiBkXcBI8MqRtHU0uSC4qT0XCO94sTc4tK8dL3k/NxNjJCw+rqDcekxq0OMAhyMSjy8 FXrHwoRYE8uKK3MPMUpwMCuJ8Or6A4V4UxIrq1KL8uOLSnNSiw8xSnOwKInzztz1PkRIID2x JDU7NbUgtQgmy8TBKdXAuHTOl0im1ZoRhYZWYr5iZi8NXi7Vb56Qub6mxuVJiPKmxfvOeV5b +9E1/mlGi+fEjSu8Ndy03ecwd8icu/5uY/UtX8vFmsLXN02QcNbXfzHfRESeofbL4fKjdz/0 NyY/SGN87LUtVvXX2m+nZ+a8Cgz5e1pb2ahob9OqDVuuqDAGee39+T3xlRJLcUaioRZzUXEi AHJp2tonAgAA Cc: Ilya Maximets , Dyasly Sergey Subject: [dpdk-dev] [PATCH RFC 3/4] vhost: avoid reordering of used->idx and last_used_idx updating. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2016 06:32:58 -0000 Calling rte_vhost_enqueue_burst() simultaneously from different threads for the same queue_id requires additional SMP memory barrier to avoid reordering of used->idx and last_used_idx updates. In case of virtio_dev_rx() memory barrier rte_mb() simply moved one instruction higher. Signed-off-by: Ilya Maximets --- lib/librte_vhost/vhost_rxtx.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/lib/librte_vhost/vhost_rxtx.c b/lib/librte_vhost/vhost_rxtx.c index 9095fb1..a03f687 100644 --- a/lib/librte_vhost/vhost_rxtx.c +++ b/lib/librte_vhost/vhost_rxtx.c @@ -281,10 +281,13 @@ virtio_dev_rx(struct virtio_net *dev, uint16_t queue_id, rte_pause(); *(volatile uint16_t *)&vq->used->idx += count; - vq->last_used_idx = res_end_idx; - /* flush used->idx update before we read avail->flags. */ + /* + * Flush used->idx update to make it visible to virtio and all other + * threads before allowing to modify it. + */ rte_mb(); + vq->last_used_idx = res_end_idx; /* Kick the guest if necessary. */ if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) @@ -586,19 +589,24 @@ virtio_dev_merge_rx(struct virtio_net *dev, uint16_t queue_id, rte_pause(); *(volatile uint16_t *)&vq->used->idx += entry_success; + /* + * Flush used->idx update to make it visible to all + * other threads before allowing to modify it. + */ + rte_smp_wmb(); + vq->last_used_idx = res_cur_idx; } merge_rx_exit: if (likely(pkt_idx)) { - /* flush used->idx update before we read avail->flags. */ + /* Flush used->idx update to make it visible to virtio. */ rte_mb(); /* Kick the guest if necessary. */ if (!(vq->avail->flags & VRING_AVAIL_F_NO_INTERRUPT)) eventfd_write(vq->callfd, (eventfd_t)1); } - return pkt_idx; } -- 2.5.0