From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 835592C37 for ; Wed, 26 Dec 2018 17:37:20 +0100 (CET) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20181226163719euoutp02099af4ec30f67b78a88005f28783a069~z71OJ6faP2550525505euoutp02H for ; Wed, 26 Dec 2018 16:37:19 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20181226163719euoutp02099af4ec30f67b78a88005f28783a069~z71OJ6faP2550525505euoutp02H DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1545842239; bh=LfBH3TDHl9LkDZIZc7qEwyshQNuy4Yr1TUkVvEWQxyo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PCBOkt3MkFIZ+TG8WWtXNe2jXdE25cVK50/r3XWx8MRNk7430V7gHHNguGysWmHNx gkVjOaRiXAXAIrpTPqAAsWwPwY3HrOLvyI/XquQTc7oILSAxyOxPOA3B208CcyKioi zks3mvNAOJS+kHXXjxBu/ihggFa8W6Y8PzyC8kGs= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20181226163718eucas1p1425521a5fde1a3100cfcf27eb9945196~z71NXdzZ_2478724787eucas1p1J; Wed, 26 Dec 2018 16:37:18 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 13.8F.04294.E3EA32C5; Wed, 26 Dec 2018 16:37:18 +0000 (GMT) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20181226163717eucas1p15276eb45e35abe2c9cf3e7c1e0050823~z71Mk0aoH1329613296eucas1p1b; Wed, 26 Dec 2018 16:37:17 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20181226163717eusmtrp181110db1fa5dba523010558663bc7e15~z71MWNGEw1259012590eusmtrp1p; Wed, 26 Dec 2018 16:37:17 +0000 (GMT) X-AuditID: cbfec7f4-835ff700000010c6-9d-5c23ae3e3391 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id F4.10.04128.C3EA32C5; Wed, 26 Dec 2018 16:37:16 +0000 (GMT) Received: from imaximets.rnd.samsung.ru (unknown [106.109.129.180]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20181226163716eusmtip2aeb9ceb9ee024250805b26e87b0b3789~z71LamZUU0525805258eusmtip23; Wed, 26 Dec 2018 16:37:16 +0000 (GMT) From: Ilya Maximets To: dev@dpdk.org, Maxime Coquelin , "Michael S . Tsirkin" , Xiao Wang Cc: Tiwei Bie , Zhihong Wang , jfreimann@redhat.com, Jason Wang , xiaolong.ye@intel.com, alejandro.lucero@netronome.com, Ilya Maximets Date: Wed, 26 Dec 2018 19:37:12 +0300 Message-Id: <20181226163712.31596-1-i.maximets@samsung.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181214153812.3878-1-i.maximets@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprAKsWRmVeSWpSXmKPExsWy7djP87p265RjDB5NYrE492kZk8W7T9uZ LK60/2S3WHbpM5PFuTVLWSyOde5hsfj/6xWrxdaG/0wW+58fZrf488bUYvPFSUwO3B6/Fixl 9Vi85yWTx/Tuh8we7/ddZfPo27KKMYA1issmJTUnsyy1SN8ugStj0Wfegus2FaeXn2dsYPxt 2MXIySEhYCIx9f0Dxi5GLg4hgRWMEh+u7GGGcL4wSlzZchMq85lR4sHel8wwLRdOnWeDSCxn lHhzaBIrhPODUeLeqXcsIFVsAjoSp1YfAWsXEehmlHj19DNYC7PAc0aJF1/usYNUCQv4SNy9 f4EVxGYRUJXYs2AaWDevgLXE4nfHWSD2yUus3nAAbDcnUHzb3B9Qd7SzS7ydqA1hu0g8vHab DcIWlnh1fAs7hC0jcXpyD9Sceon7LS/BLpIQ6GCUmH7oHxNEwl5iy+tzQA0cQNdpSqzfpQ8R dpT4/XM5C0hYQoBP4sZbQZAwM5A5adt0Zogwr0RHmxBEtYrE74PLoS6Tkrj57jPUBR4Se74c hQZQH6PE5f17mCYwys9CWLaAkXEVo3hqaXFuemqxUV5quV5xYm5xaV66XnJ+7iZGYDo5/e/4 lx2Mu/4kHWIU4GBU4uHVWKwcI8SaWFZcmXuIUYKDWUmEl7UMKMSbklhZlVqUH19UmpNafIhR moNFSZy3muFBtJBAemJJanZqakFqEUyWiYNTqoHR/U9EZt38ghAlj3CfU9ob5Mpi9Hcwcq/+ mdZl/a+YecPb5xsy9030fMH9xtvsRd0ll463xi8Lix88LembkXqSa3O5wkvl9LWO8tFRU57v M3SZaP+xahJL4sMJagfbZhybu2fdYd59yVwa4XbnoqOuzgz67mlZPb/s4X+z8LXn7ar1pvl1 KS1VYinOSDTUYi4qTgQARqXcNCMDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrILMWRmVeSWpSXmKPExsVy+t/xe7q265RjDPbqW5z7tIzJ4t2n7UwW V9p/slssu/SZyeLcmqUsFsc697BY/P/1itVia8N/Jov9zw+zW/x5Y2qx+eIkJgduj18LlrJ6 LN7zksljevdDZo/3+66yefRtWcUYwBqlZ1OUX1qSqpCRX1xiqxRtaGGkZ2hpoWdkYqlnaGwe a2VkqqRvZ5OSmpNZllqkb5egl7HoM2/BdZuK08vPMzYw/jbsYuTkkBAwkbhw6jxbFyMXh5DA UkaJ/b+bmSASUhI/fl1ghbCFJf5c64Iq+sYocfN+CzNIgk1AR+LU6iOMIAkRgX5Gid3XbzKD OMwCbxkl+ibdYAepEhbwkbh7H2IUi4CqxJ4F01hAbF4Ba4nF746zQKyQl1i94QDYVE6g+La5 P8BsIQEriUuLn7BOYORbwMiwilEktbQ4Nz232EivODG3uDQvXS85P3cTIzC4tx37uWUHY9e7 4EOMAhyMSjy8GouVY4RYE8uKK3MPMUpwMCuJ8LKWAYV4UxIrq1KL8uOLSnNSiw8xmgIdNZFZ SjQ5Hxh5eSXxhqaG5haWhubG5sZmFkrivOcNKqOEBNITS1KzU1MLUotg+pg4OKUaGJfYnt0T cIN/+jrlDP9YiTKF51N3Zz0L803eExy2pPrNrHIBz0tzH5qdv7BIeYeQ31nHCatOHQ1kzQtQ L/wpMWfBwxvXv3x+IxGkpuGgmKDqXr3hiXWgU7T9fI0bTl/zdKQ71//1uq85TTunvXXa8/C7 O99zvT++9TxQmdKbCU4Py0vvvDHvUmIpzkg01GIuKk4EAOsvMb2EAgAA X-CMS-MailID: 20181226163717eucas1p15276eb45e35abe2c9cf3e7c1e0050823 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20181226163717eucas1p15276eb45e35abe2c9cf3e7c1e0050823 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20181226163717eucas1p15276eb45e35abe2c9cf3e7c1e0050823 References: <20181214153812.3878-1-i.maximets@samsung.com> Subject: [dpdk-dev] [PATCH v2] net/virtio: add platform memory ordering feature support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Dec 2018 16:37:20 -0000 VIRTIO_F_ORDER_PLATFORM is required to use proper memory barriers in case of HW vhost implementations like vDPA. DMA barriers (rte_cio_*) are sufficent for that purpose. Previously known as VIRTIO_F_IO_BARRIER. Signed-off-by: Ilya Maximets --- Version 2: * rebased on current master (packed rings). RFC --> Version 1: * Dropped vendor-specific hack to determine if we need real barriers. * Added VIRTIO_F_ORDER_PLATFORM feature definition and checking. Note: Patch to change the name of the feature from VIRTIO_F_IO_BARRIER to VIRTIO_F_ORDER_PLATFORM is not merged yet: https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg04114.html drivers/net/virtio/virtio_ethdev.c | 2 ++ drivers/net/virtio/virtio_ethdev.h | 3 ++- drivers/net/virtio/virtio_pci.h | 7 ++++++ drivers/net/virtio/virtio_rxtx.c | 16 ++++++------ drivers/net/virtio/virtqueue.h | 39 ++++++++++++++++++++++++------ 5 files changed, 51 insertions(+), 16 deletions(-) diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index 446c338fc..6d461180c 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -1613,6 +1613,8 @@ virtio_init_device(struct rte_eth_dev *eth_dev, uint64_t req_features) if (virtio_negotiate_features(hw, req_features) < 0) return -1; + hw->weak_barriers = !vtpci_with_feature(hw, VIRTIO_F_ORDER_PLATFORM); + if (!hw->virtio_user_dev) { pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev); rte_eth_copy_pci_info(eth_dev, pci_dev); diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h index f8d8a56ab..b8aab7da4 100644 --- a/drivers/net/virtio/virtio_ethdev.h +++ b/drivers/net/virtio/virtio_ethdev.h @@ -35,7 +35,8 @@ 1ULL << VIRTIO_F_VERSION_1 | \ 1ULL << VIRTIO_F_IN_ORDER | \ 1ULL << VIRTIO_F_RING_PACKED | \ - 1ULL << VIRTIO_F_IOMMU_PLATFORM) + 1ULL << VIRTIO_F_IOMMU_PLATFORM | \ + 1ULL << VIRTIO_F_ORDER_PLATFORM) #define VIRTIO_PMD_SUPPORTED_GUEST_FEATURES \ (VIRTIO_PMD_DEFAULT_GUEST_FEATURES | \ diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h index b22b62dad..38a0261da 100644 --- a/drivers/net/virtio/virtio_pci.h +++ b/drivers/net/virtio/virtio_pci.h @@ -129,6 +129,12 @@ struct virtnet_ctl; */ #define VIRTIO_F_IN_ORDER 35 +/* + * This feature indicates that memory accesses by the driver and the device + * are ordered in a way described by the platform. + */ +#define VIRTIO_F_ORDER_PLATFORM 36 + /* The Guest publishes the used index for which it expects an interrupt * at the end of the avail ring. Host should ignore the avail->flags field. */ /* The Host publishes the avail index for which it expects a kick @@ -241,6 +247,7 @@ struct virtio_hw { uint8_t use_simple_rx; uint8_t use_inorder_rx; uint8_t use_inorder_tx; + uint8_t weak_barriers; bool has_tx_offload; bool has_rx_offload; uint16_t port_id; diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c index 2309b71d6..ebb86ef70 100644 --- a/drivers/net/virtio/virtio_rxtx.c +++ b/drivers/net/virtio/virtio_rxtx.c @@ -1152,7 +1152,7 @@ virtio_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); num = likely(nb_used <= nb_pkts) ? nb_used : nb_pkts; if (unlikely(num > VIRTIO_MBUF_BURST_SZ)) @@ -1361,7 +1361,7 @@ virtio_recv_pkts_inorder(void *rx_queue, nb_used = RTE_MIN(nb_used, nb_pkts); nb_used = RTE_MIN(nb_used, VIRTIO_MBUF_BURST_SZ); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); PMD_RX_LOG(DEBUG, "used:%d", nb_used); @@ -1549,7 +1549,7 @@ virtio_recv_mergeable_pkts(void *rx_queue, nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); PMD_RX_LOG(DEBUG, "used:%d", nb_used); @@ -1940,7 +1940,7 @@ virtio_xmit_pkts_packed(void *tx_queue, struct rte_mbuf **tx_pkts, /* Positive value indicates it need free vring descriptors */ if (unlikely(need > 0)) { - virtio_rmb(); + virtio_rmb(hw->weak_barriers); need = RTE_MIN(need, (int)nb_pkts); virtio_xmit_cleanup_packed(vq, need); need = slots - vq->vq_free_cnt; @@ -1988,7 +1988,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) PMD_TX_LOG(DEBUG, "%d packets to xmit", nb_pkts); nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh)) virtio_xmit_cleanup(vq, nb_used); @@ -2030,7 +2030,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) /* Positive value indicates it need free vring descriptors */ if (unlikely(need > 0)) { nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); need = RTE_MIN(need, (int)nb_used); virtio_xmit_cleanup(vq, need); @@ -2086,7 +2086,7 @@ virtio_xmit_pkts_inorder(void *tx_queue, PMD_TX_LOG(DEBUG, "%d packets to xmit", nb_pkts); nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh)) virtio_xmit_cleanup_inorder(vq, nb_used); @@ -2134,7 +2134,7 @@ virtio_xmit_pkts_inorder(void *tx_queue, need = slots - vq->vq_free_cnt; if (unlikely(need > 0)) { nb_used = VIRTQUEUE_NUSED(vq); - virtio_rmb(); + virtio_rmb(hw->weak_barriers); need = RTE_MIN(need, (int)nb_used); virtio_xmit_cleanup_inorder(vq, need); diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h index d8ae5cdec..a66a37f61 100644 --- a/drivers/net/virtio/virtqueue.h +++ b/drivers/net/virtio/virtqueue.h @@ -19,15 +19,40 @@ struct rte_mbuf; /* - * Per virtio_config.h in Linux. + * Per virtio_ring.h in Linux. * For virtio_pci on SMP, we don't need to order with respect to MMIO * accesses through relaxed memory I/O windows, so smp_mb() et al are * sufficient. * + * For using virtio to talk to real devices (eg. vDPA) we do need real + * barriers. */ -#define virtio_mb() rte_smp_mb() -#define virtio_rmb() rte_smp_rmb() -#define virtio_wmb() rte_smp_wmb() +static inline void +virtio_mb(uint8_t weak_barriers) +{ + if (weak_barriers) + rte_smp_mb(); + else + rte_mb(); +} + +static inline void +virtio_rmb(uint8_t weak_barriers) +{ + if (weak_barriers) + rte_smp_rmb(); + else + rte_cio_rmb(); +} + +static inline void +virtio_wmb(uint8_t weak_barriers) +{ + if (weak_barriers) + rte_smp_wmb(); + else + rte_cio_wmb(); +} #ifdef RTE_PMD_PACKET_PREFETCH #define rte_packet_prefetch(p) rte_prefetch1(p) @@ -325,7 +350,7 @@ virtqueue_enable_intr_packed(struct virtqueue *vq) if (vq->event_flags_shadow == RING_EVENT_FLAGS_DISABLE) { - virtio_wmb(); + virtio_wmb(vq->hw->weak_barriers); vq->event_flags_shadow = RING_EVENT_FLAGS_ENABLE; *event_flags = vq->event_flags_shadow; } @@ -391,7 +416,7 @@ void vq_ring_free_inorder(struct virtqueue *vq, uint16_t desc_idx, static inline void vq_update_avail_idx(struct virtqueue *vq) { - virtio_wmb(); + virtio_wmb(vq->hw->weak_barriers); vq->vq_ring.avail->idx = vq->vq_avail_idx; } @@ -423,7 +448,7 @@ virtqueue_kick_prepare_packed(struct virtqueue *vq) { uint16_t flags; - virtio_mb(); + virtio_mb(vq->hw->weak_barriers); flags = vq->ring_packed.device_event->desc_event_flags; return flags != RING_EVENT_FLAGS_DISABLE; -- 2.17.1