From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 80101A04F1; Mon, 9 Dec 2019 09:52:12 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B2D7923D; Mon, 9 Dec 2019 09:52:11 +0100 (CET) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 4F8621F5; Mon, 9 Dec 2019 09:52:08 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Dec 2019 00:52:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,294,1571727600"; d="scan'208";a="412684104" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga005.fm.intel.com with ESMTP; 09 Dec 2019 00:52:07 -0800 Received: from fmsmsx123.amr.corp.intel.com (10.18.125.38) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 9 Dec 2019 00:51:59 -0800 Received: from shsmsx107.ccr.corp.intel.com (10.239.4.96) by fmsmsx123.amr.corp.intel.com (10.18.125.38) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 9 Dec 2019 00:51:59 -0800 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.29]) by SHSMSX107.ccr.corp.intel.com ([169.254.9.164]) with mapi id 14.03.0439.000; Mon, 9 Dec 2019 16:51:57 +0800 From: "Liu, Yong" To: "Ding, Xuan" , "maintainer@dpdk.org" CC: "dev@dpdk.org" , "maxime.coquelin@redhat.com" , "Bie, Tiwei" , "Wang, Zhihong" , "Ding, Xuan" , "stable@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v1] net/virtio-user: fix packed ring server mode Thread-Index: AQHVrmXOL+cMfPY2jUmq3Zy0FkC2M6exdzUA Date: Mon, 9 Dec 2019 08:51:57 +0000 Message-ID: <86228AFD5BCD8E4EBFD2B90117B5E81E6341F8D8@SHSMSX103.ccr.corp.intel.com> References: <20191209164939.54806-1-xuan.ding@intel.com> In-Reply-To: <20191209164939.54806-1-xuan.ding@intel.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODFjMTlhOTItMTU2My00MzdjLWE4ODEtOTU3NmEwN2M0NjY4IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoidDRialA3STVFaW83dEREXC9xbXNiaE9CaXVYdE1ndmhXZ0NRREVSbDVaVGxBUkN2TENidkxqZEtsRkNVTGhlbW4ifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v1] net/virtio-user: fix packed ring server mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: dev On Behalf Of Xuan Ding > Sent: Tuesday, December 10, 2019 12:50 AM > To: maintainer@dpdk.org > Cc: dev@dpdk.org; maxime.coquelin@redhat.com; Bie, Tiwei > ; Wang, Zhihong ; Ding, Xuan > ; stable@dpdk.org > Subject: [dpdk-dev] [PATCH v1] net/virtio-user: fix packed ring server > mode >=20 > This patch fixes the situation where datapath does not work properly when > vhost reconnects to virtio in server mode with packed ring. >=20 > Currently, virtio and vhost share memory of vring. For split ring, vhost > can read the status of discriptors directly from the available ring and > the used ring during reconnection. Therefore, the datapath can continue. >=20 > But for packed ring, when reconnecting to virtio, vhost cannot get the > status of discriptors only through the descriptor ring. By resetting Tx > and Rx queues, the datapath can restart from the beginning. >=20 > Fixes: 4c3f5822eb214 ("net/virtio: add packed virtqueue defines") > Cc: stable@dpdk.org >=20 > Signed-off-by: Xuan Ding > --- > drivers/net/virtio/virtio_ethdev.c | 112 +++++++++++++++++++++++- > drivers/net/virtio/virtio_ethdev.h | 3 + > drivers/net/virtio/virtio_user_ethdev.c | 8 ++ > 3 files changed, 121 insertions(+), 2 deletions(-) >=20 > diff --git a/drivers/net/virtio/virtio_ethdev.c > b/drivers/net/virtio/virtio_ethdev.c > index 044eb10a7..c0cb0f23c 100644 > --- a/drivers/net/virtio/virtio_ethdev.c > +++ b/drivers/net/virtio/virtio_ethdev.c > @@ -433,6 +433,94 @@ virtio_init_vring(struct virtqueue *vq) > virtqueue_disable_intr(vq); > } >=20 > +static int > +virtio_user_reset_rx_queues(struct rte_eth_dev *dev, uint16_t queue_idx) > +{ Hi Xuan, This function named as virtio_user_reset, but look like it has no relations= hip with virtio_user. Maybe rename this function to virtqueue_reset and move it to virtqueue.c wi= ll be more suitable.=20 Please also add suffix _packed as this function only workable for packed ri= ng. Thanks, Marvin > + uint16_t vtpci_queue_idx =3D 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX; > + struct virtio_hw *hw =3D dev->data->dev_private; > + struct virtqueue *vq =3D hw->vqs[vtpci_queue_idx]; > + struct virtnet_rx *rxvq; > + struct vq_desc_extra *dxp; > + unsigned int vq_size; > + uint16_t desc_idx, i; > + > + vq_size =3D VTPCI_OPS(hw)->get_queue_num(hw, vtpci_queue_idx); > + Virtqueue size has been set to vq_nentries in virtqueue structure. Do we ne= ed to re-catch it? > + vq->vq_packed.used_wrap_counter =3D 1; > + vq->vq_packed.cached_flags =3D VRING_PACKED_DESC_F_AVAIL; > + vq->vq_packed.event_flags_shadow =3D 0; > + vq->vq_packed.cached_flags |=3D VRING_DESC_F_WRITE; > + > + rxvq =3D &vq->rxq; > + memset(rxvq->mz->addr, 0, rxvq->mz->len); > + > + for (desc_idx =3D 0; desc_idx < vq_size; desc_idx++) { > + dxp =3D &vq->vq_descx[desc_idx]; > + if (dxp->cookie !=3D NULL) { > + rte_pktmbuf_free(dxp->cookie); > + dxp->cookie =3D NULL; > + } > + } > + > + virtio_init_vring(vq); > + > + for (i =3D 0; i < hw->max_queue_pairs; i++) > + if (rxvq->mpool !=3D NULL) > + virtio_dev_rx_queue_setup_finish(dev, i); > + Please add parentheses for multiple lines loop content.=20 > + return 0; > +} > + > +static int > +virtio_user_reset_tx_queues(struct rte_eth_dev *dev, uint16_t queue_idx) > +{ > + uint8_t vtpci_queue_idx =3D 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX; > + struct virtio_hw *hw =3D dev->data->dev_private; > + struct virtqueue *vq =3D hw->vqs[vtpci_queue_idx]; > + struct virtnet_tx *txvq; > + struct vq_desc_extra *dxp; > + unsigned int vq_size; > + uint16_t desc_idx; > + > + vq_size =3D VTPCI_OPS(hw)->get_queue_num(hw, vtpci_queue_idx); > + > + vq->vq_packed.used_wrap_counter =3D 1; > + vq->vq_packed.cached_flags =3D VRING_PACKED_DESC_F_AVAIL; > + vq->vq_packed.event_flags_shadow =3D 0; > + > + txvq =3D &vq->txq; > + memset(txvq->mz->addr, 0, txvq->mz->len); > + memset(txvq->virtio_net_hdr_mz->addr, 0, > + txvq->virtio_net_hdr_mz->len); > + > + for (desc_idx =3D 0; desc_idx < vq_size; desc_idx++) { > + dxp =3D &vq->vq_descx[desc_idx]; > + if (dxp->cookie !=3D NULL) { > + rte_pktmbuf_free(dxp->cookie); > + dxp->cookie =3D NULL; > + } > + } > + > + virtio_init_vring(vq); > + > + return 0; > +} > + > +static int > +virtio_user_reset_queues(struct rte_eth_dev *eth_dev) > +{ > + uint16_t i; > + > + /* Vring reset for each Tx queue and Rx queue. */ > + for (i =3D 0; i < eth_dev->data->nb_rx_queues; i++) > + virtio_user_reset_rx_queues(eth_dev, i); > + > + for (i =3D 0; i < eth_dev->data->nb_rx_queues; i++) > + virtio_user_reset_tx_queues(eth_dev, i); > + > + return 0; > +} > + > static int > virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx) > { > @@ -1913,6 +2001,8 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev) > goto err_vtpci_init; > } >=20 > + rte_spinlock_init(&hw->state_lock); > + > /* reset device and negotiate default features */ > ret =3D virtio_init_device(eth_dev, > VIRTIO_PMD_DEFAULT_GUEST_FEATURES); > if (ret < 0) > @@ -2155,8 +2245,6 @@ virtio_dev_configure(struct rte_eth_dev *dev) > return -EBUSY; > } >=20 > - rte_spinlock_init(&hw->state_lock); > - > hw->use_simple_rx =3D 1; >=20 > if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER)) { > @@ -2421,6 +2509,26 @@ virtio_dev_vlan_offload_set(struct rte_eth_dev > *dev, int mask) > return 0; > } >=20 > +int > +virtio_user_reset_device(struct rte_eth_dev *eth_dev, struct virtio_hw > *hw) > +{ > + /* Add lock to avoid queue contention. */ > + rte_spinlock_lock(&hw->state_lock); > + hw->started =3D 0; > + /* > + * Waitting for datapath to complete before resetting queues. > + * 1 ms should be enough for the ongoing Tx/Rx function to finish. > + */ > + rte_delay_ms(1); > + > + virtio_user_reset_queues(eth_dev); > + > + hw->started =3D 1; > + rte_spinlock_unlock(&hw->state_lock); > + > + return 0; > +} > + > static int > virtio_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info > *dev_info) > { > diff --git a/drivers/net/virtio/virtio_ethdev.h > b/drivers/net/virtio/virtio_ethdev.h > index a10111758..72e9e3ff8 100644 > --- a/drivers/net/virtio/virtio_ethdev.h > +++ b/drivers/net/virtio/virtio_ethdev.h > @@ -49,6 +49,9 @@ >=20 > extern const struct eth_dev_ops virtio_user_secondary_eth_dev_ops; >=20 > +int virtio_user_reset_device(struct rte_eth_dev *eth_dev, > + struct virtio_hw *hw); > + > /* > * CQ function prototype > */ > diff --git a/drivers/net/virtio/virtio_user_ethdev.c > b/drivers/net/virtio/virtio_user_ethdev.c > index 3fc172573..49068a578 100644 > --- a/drivers/net/virtio/virtio_user_ethdev.c > +++ b/drivers/net/virtio/virtio_user_ethdev.c > @@ -31,6 +31,7 @@ virtio_user_server_reconnect(struct virtio_user_dev > *dev) > int ret; > int connectfd; > struct rte_eth_dev *eth_dev =3D &rte_eth_devices[dev->port_id]; > + struct virtio_hw *hw =3D eth_dev->data->dev_private; >=20 > connectfd =3D accept(dev->listenfd, NULL, NULL); > if (connectfd < 0) > @@ -51,6 +52,13 @@ virtio_user_server_reconnect(struct virtio_user_dev > *dev) >=20 > dev->features &=3D dev->device_features; >=20 > + /* > + * * For packed ring, resetting queues > + * is required in reconnection. > + */ > + if (vtpci_packed_queue(hw)) > + virtio_user_reset_device(eth_dev, hw); > + > ret =3D virtio_user_start_device(dev); > if (ret < 0) > return -1; > -- > 2.17.1