From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7A984A051C for ; Tue, 11 Feb 2020 12:29:17 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 712B02952; Tue, 11 Feb 2020 12:29:17 +0100 (CET) Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by dpdk.org (Postfix) with ESMTP id D3C341BFA3 for ; Tue, 11 Feb 2020 12:29:16 +0100 (CET) Received: by mail-wr1-f66.google.com with SMTP id k11so11862426wrd.9 for ; Tue, 11 Feb 2020 03:29:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=76nUVIz97we3YngReLGXTtmMzfSM4JCv+13VJHBU/1A=; b=FeIssa826rSnPuSbH8pebluUC++GG8cSUgHw/JThQVdSxHKGe+anYccruydUnkIzQg WpF0/SZ0CyCej5LrO6UhWXyykeec4dsRZESQPvSypdAsT5oT8Wu/xM6wfWj5RHFQyVqm bghJa6CU2gJsyvj0q1qHv/z7CKHaCOfY9sGD8WBchWzdUoug5SiDpKBUIhGgZ142wCZ2 +yiKyyEaa/txXHtyec0I97kXMGnhDm8i/xcZt+eQAl5L6WTvbw6vV8rC+Sok2Q9cfmBd 22iNlHQd1iM1NYWhMfCH68fyipisPidt59ByI2+o/BiK/n45yCnh+oGprIILUJxoedpO uOxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=76nUVIz97we3YngReLGXTtmMzfSM4JCv+13VJHBU/1A=; b=Uh1/LY8j4MGOsAFLwjWSEvC8T+/Q3920dLVxTRKMTVCo5q/Rosuc92/etX4DPIj+tM pQMXRZCYExRIbpF7EYmRfdcSs5B2CcLdekz5NOBtrCulZic+B8QAR60Git3EQxnlKW9Y Odu8Y9FA8h0yHd76kpQC8oxpL/6ae7oLpqh5i2HUmnYmpXVt0Ki+B2eqU5dti0zrWnUN QP8dRWV9iBYvLTBZT76rpHFNX+iQjXgukQZe238NofFjipXqQytY5mz7L3lZ/hgqirkm qfxrP3SDqfnaIjwdEQbVqZQHFVwZ/I6h4dG1uQ4BeB89dLJzPqtmatpOYAoDgvt9CB0e Op9A== X-Gm-Message-State: APjAAAX2AuZ4/IbunhnpDcfXa+8JklZPeZMtFSzeJskfG5gXnupBdCQO KDav5Q9eyKuupnJjJQUGkP4= X-Google-Smtp-Source: APXvYqx+wEapyQcE6Q761T5bSIVoqx9HTG+N4v5i2bn9C68f05awwY4mjfW2lTXHBHZ0UX+Wco1Zyw== X-Received: by 2002:a5d:6692:: with SMTP id l18mr7861248wru.382.1581420556534; Tue, 11 Feb 2020 03:29:16 -0800 (PST) Received: from localhost ([88.98.246.218]) by smtp.gmail.com with ESMTPSA id a5sm3188390wmb.37.2020.02.11.03.29.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Feb 2020 03:29:16 -0800 (PST) From: luca.boccassi@gmail.com To: Xuan Ding Cc: Maxime Coquelin , dpdk stable Date: Tue, 11 Feb 2020 11:20:40 +0000 Message-Id: <20200211112216.3929-94-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200211112216.3929-1-luca.boccassi@gmail.com> References: <20200211112216.3929-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-stable] patch 'net/virtio-user: fix packed ring server mode' has been queued to stable release 19.11.1 X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Sender: "stable" Hi, FYI, your patch has been queued to stable release 19.11.1 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 02/13/20. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Thanks. Luca Boccassi --- >From 1d8dce55d16abced072aac16a293cf5d3b49fdfb Mon Sep 17 00:00:00 2001 From: Xuan Ding Date: Wed, 15 Jan 2020 06:13:58 +0000 Subject: [PATCH] net/virtio-user: fix packed ring server mode [ upstream commit 6ebbf4109f3529a1af880ce195869cee84647178 ] This patch fixes the situation where data path does not work properly when vhost reconnects to virtio in server mode with packed ring. Currently, virtio and vhost share memory of vring. For split ring, vhost can read the status of descriptors directly from the available ring and the used ring during reconnection. Therefore, the data path can continue. But for packed ring, when reconnecting to virtio, vhost cannot get the status of descriptors via the descriptor ring. By resetting Tx and Rx queues, the data path can restart from the beginning. Fixes: 4c3f5822eb21 ("net/virtio: add packed virtqueue defines") Signed-off-by: Xuan Ding Reviewed-by: Maxime Coquelin --- drivers/net/virtio/virtio_ethdev.c | 4 +- drivers/net/virtio/virtio_user_ethdev.c | 43 +++++++++++++++ drivers/net/virtio/virtqueue.c | 71 +++++++++++++++++++++++++ drivers/net/virtio/virtqueue.h | 4 ++ 4 files changed, 120 insertions(+), 2 deletions(-) diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index 044eb10a70..f9d0ea70db 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -1913,6 +1913,8 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev) goto err_vtpci_init; } + rte_spinlock_init(&hw->state_lock); + /* reset device and negotiate default features */ ret = virtio_init_device(eth_dev, VIRTIO_PMD_DEFAULT_GUEST_FEATURES); if (ret < 0) @@ -2155,8 +2157,6 @@ virtio_dev_configure(struct rte_eth_dev *dev) return -EBUSY; } - rte_spinlock_init(&hw->state_lock); - hw->use_simple_rx = 1; if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER)) { diff --git a/drivers/net/virtio/virtio_user_ethdev.c b/drivers/net/virtio/virtio_user_ethdev.c index 3fc1725736..f3b35d1bdd 100644 --- a/drivers/net/virtio/virtio_user_ethdev.c +++ b/drivers/net/virtio/virtio_user_ethdev.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "virtio_ethdev.h" #include "virtio_logs.h" @@ -25,12 +26,48 @@ #define virtio_user_get_dev(hw) \ ((struct virtio_user_dev *)(hw)->virtio_user_dev) +static void +virtio_user_reset_queues_packed(struct rte_eth_dev *dev) +{ + struct virtio_hw *hw = dev->data->dev_private; + struct virtnet_rx *rxvq; + struct virtnet_tx *txvq; + uint16_t i; + + /* Add lock to avoid queue contention. */ + rte_spinlock_lock(&hw->state_lock); + hw->started = 0; + + /* + * Waitting for datapath to complete before resetting queues. + * 1 ms should be enough for the ongoing Tx/Rx function to finish. + */ + rte_delay_ms(1); + + /* Vring reset for each Tx queue and Rx queue. */ + for (i = 0; i < dev->data->nb_rx_queues; i++) { + rxvq = dev->data->rx_queues[i]; + virtqueue_rxvq_reset_packed(rxvq->vq); + virtio_dev_rx_queue_setup_finish(dev, i); + } + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + txvq = dev->data->tx_queues[i]; + virtqueue_txvq_reset_packed(txvq->vq); + } + + hw->started = 1; + rte_spinlock_unlock(&hw->state_lock); +} + + static int virtio_user_server_reconnect(struct virtio_user_dev *dev) { int ret; int connectfd; struct rte_eth_dev *eth_dev = &rte_eth_devices[dev->port_id]; + struct virtio_hw *hw = eth_dev->data->dev_private; connectfd = accept(dev->listenfd, NULL, NULL); if (connectfd < 0) @@ -51,6 +88,12 @@ virtio_user_server_reconnect(struct virtio_user_dev *dev) dev->features &= dev->device_features; + /* For packed ring, resetting queues is required in reconnection. */ + if (vtpci_packed_queue(hw)) + PMD_INIT_LOG(NOTICE, "Packets on the fly will be dropped" + " when packed ring reconnecting."); + virtio_user_reset_queues_packed(eth_dev); + ret = virtio_user_start_device(dev); if (ret < 0) return -1; diff --git a/drivers/net/virtio/virtqueue.c b/drivers/net/virtio/virtqueue.c index 5ff1e3587e..0b4e3bf3e2 100644 --- a/drivers/net/virtio/virtqueue.c +++ b/drivers/net/virtio/virtqueue.c @@ -141,3 +141,74 @@ virtqueue_rxvq_flush(struct virtqueue *vq) else virtqueue_rxvq_flush_split(vq); } + +int +virtqueue_rxvq_reset_packed(struct virtqueue *vq) +{ + int size = vq->vq_nentries; + struct vq_desc_extra *dxp; + struct virtnet_rx *rxvq; + uint16_t desc_idx; + + vq->vq_used_cons_idx = 0; + vq->vq_desc_head_idx = 0; + vq->vq_avail_idx = 0; + vq->vq_desc_tail_idx = (uint16_t)(vq->vq_nentries - 1); + vq->vq_free_cnt = vq->vq_nentries; + + vq->vq_packed.used_wrap_counter = 1; + vq->vq_packed.cached_flags = VRING_PACKED_DESC_F_AVAIL; + vq->vq_packed.event_flags_shadow = 0; + vq->vq_packed.cached_flags |= VRING_DESC_F_WRITE; + + rxvq = &vq->rxq; + memset(rxvq->mz->addr, 0, rxvq->mz->len); + + for (desc_idx = 0; desc_idx < vq->vq_nentries; desc_idx++) { + dxp = &vq->vq_descx[desc_idx]; + if (dxp->cookie != NULL) { + rte_pktmbuf_free(dxp->cookie); + dxp->cookie = NULL; + } + } + + vring_desc_init_packed(vq, size); + + return 0; +} + +int +virtqueue_txvq_reset_packed(struct virtqueue *vq) +{ + int size = vq->vq_nentries; + struct vq_desc_extra *dxp; + struct virtnet_tx *txvq; + uint16_t desc_idx; + + vq->vq_used_cons_idx = 0; + vq->vq_desc_head_idx = 0; + vq->vq_avail_idx = 0; + vq->vq_desc_tail_idx = (uint16_t)(vq->vq_nentries - 1); + vq->vq_free_cnt = vq->vq_nentries; + + vq->vq_packed.used_wrap_counter = 1; + vq->vq_packed.cached_flags = VRING_PACKED_DESC_F_AVAIL; + vq->vq_packed.event_flags_shadow = 0; + + txvq = &vq->txq; + memset(txvq->mz->addr, 0, txvq->mz->len); + memset(txvq->virtio_net_hdr_mz->addr, 0, + txvq->virtio_net_hdr_mz->len); + + for (desc_idx = 0; desc_idx < vq->vq_nentries; desc_idx++) { + dxp = &vq->vq_descx[desc_idx]; + if (dxp->cookie != NULL) { + rte_pktmbuf_free(dxp->cookie); + dxp->cookie = NULL; + } + } + + vring_desc_init_packed(vq, size); + + return 0; +} diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h index 8d7f197b13..58ad7309ae 100644 --- a/drivers/net/virtio/virtqueue.h +++ b/drivers/net/virtio/virtqueue.h @@ -443,6 +443,10 @@ struct rte_mbuf *virtqueue_detach_unused(struct virtqueue *vq); /* Flush the elements in the used ring. */ void virtqueue_rxvq_flush(struct virtqueue *vq); +int virtqueue_rxvq_reset_packed(struct virtqueue *vq); + +int virtqueue_txvq_reset_packed(struct virtqueue *vq); + static inline int virtqueue_full(const struct virtqueue *vq) { -- 2.20.1 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2020-02-11 11:17:42.152638556 +0000 +++ 0094-net-virtio-user-fix-packed-ring-server-mode.patch 2020-02-11 11:17:38.536003890 +0000 @@ -1,8 +1,10 @@ -From 6ebbf4109f3529a1af880ce195869cee84647178 Mon Sep 17 00:00:00 2001 +From 1d8dce55d16abced072aac16a293cf5d3b49fdfb Mon Sep 17 00:00:00 2001 From: Xuan Ding Date: Wed, 15 Jan 2020 06:13:58 +0000 Subject: [PATCH] net/virtio-user: fix packed ring server mode +[ upstream commit 6ebbf4109f3529a1af880ce195869cee84647178 ] + This patch fixes the situation where data path does not work properly when vhost reconnects to virtio in server mode with packed ring. @@ -16,7 +18,6 @@ and Rx queues, the data path can restart from the beginning. Fixes: 4c3f5822eb21 ("net/virtio: add packed virtqueue defines") -Cc: stable@dpdk.org Signed-off-by: Xuan Ding Reviewed-by: Maxime Coquelin