From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 400FEA04F5; Thu, 12 Dec 2019 12:08:50 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0923D1BF31; Thu, 12 Dec 2019 12:08:50 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id 9C8DB2BC8; Thu, 12 Dec 2019 12:08:47 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Dec 2019 03:08:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,305,1571727600"; d="scan'208";a="388270203" Received: from fmsmsx107.amr.corp.intel.com ([10.18.124.205]) by orsmga005.jf.intel.com with ESMTP; 12 Dec 2019 03:08:45 -0800 Received: from fmsmsx114.amr.corp.intel.com (10.18.116.8) by fmsmsx107.amr.corp.intel.com (10.18.124.205) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 12 Dec 2019 03:08:45 -0800 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by FMSMSX114.amr.corp.intel.com (10.18.116.8) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 12 Dec 2019 03:08:44 -0800 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.109]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.214]) with mapi id 14.03.0439.000; Thu, 12 Dec 2019 19:08:43 +0800 From: "Ding, Xuan" To: "Liu, Yong" , "maintainer@dpdk.org" CC: "dev@dpdk.org" , "maxime.coquelin@redhat.com" , "Bie, Tiwei" , "Wang, Zhihong" , "stable@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v1] net/virtio-user: fix packed ring server mode Thread-Index: AQHVrmXCicpGOK1Yv06RNaU3I4Ct76ew+OuAgAVgN/A= Date: Thu, 12 Dec 2019 11:08:42 +0000 Message-ID: <3DA54CD954B3144AB0AD8E35A260C0060526255D@shsmsx102.ccr.corp.intel.com> References: <20191209164939.54806-1-xuan.ding@intel.com> <86228AFD5BCD8E4EBFD2B90117B5E81E6341F8D8@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E6341F8D8@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v1] net/virtio-user: fix packed ring server mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Liu, Yong > Sent: Monday, December 9, 2019 4:52 PM > To: Ding, Xuan ; maintainer@dpdk.org > Cc: dev@dpdk.org; maxime.coquelin@redhat.com; Bie, Tiwei > ; Wang, Zhihong ; Ding, Xuan > ; stable@dpdk.org > Subject: RE: [dpdk-dev] [PATCH v1] net/virtio-user: fix packed ring serve= r mode >=20 >=20 >=20 > > -----Original Message----- > > From: dev On Behalf Of Xuan Ding > > Sent: Tuesday, December 10, 2019 12:50 AM > > To: maintainer@dpdk.org > > Cc: dev@dpdk.org; maxime.coquelin@redhat.com; Bie, Tiwei > > ; Wang, Zhihong ; Ding, > > Xuan ; stable@dpdk.org > > Subject: [dpdk-dev] [PATCH v1] net/virtio-user: fix packed ring server > > mode > > > > This patch fixes the situation where datapath does not work properly > > when vhost reconnects to virtio in server mode with packed ring. > > > > Currently, virtio and vhost share memory of vring. For split ring, > > vhost can read the status of discriptors directly from the available > > ring and the used ring during reconnection. Therefore, the datapath can > continue. > > > > But for packed ring, when reconnecting to virtio, vhost cannot get the > > status of discriptors only through the descriptor ring. By resetting > > Tx and Rx queues, the datapath can restart from the beginning. > > > > Fixes: 4c3f5822eb214 ("net/virtio: add packed virtqueue defines") > > Cc: stable@dpdk.org > > > > Signed-off-by: Xuan Ding > > --- > > drivers/net/virtio/virtio_ethdev.c | 112 +++++++++++++++++++++++- > > drivers/net/virtio/virtio_ethdev.h | 3 + > > drivers/net/virtio/virtio_user_ethdev.c | 8 ++ > > 3 files changed, 121 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/net/virtio/virtio_ethdev.c > > b/drivers/net/virtio/virtio_ethdev.c > > index 044eb10a7..c0cb0f23c 100644 > > --- a/drivers/net/virtio/virtio_ethdev.c > > +++ b/drivers/net/virtio/virtio_ethdev.c > > @@ -433,6 +433,94 @@ virtio_init_vring(struct virtqueue *vq) > > virtqueue_disable_intr(vq); > > } > > > > +static int > > +virtio_user_reset_rx_queues(struct rte_eth_dev *dev, uint16_t > > +queue_idx) { >=20 > Hi Xuan, > This function named as virtio_user_reset, but look like it has no relatio= nship with > virtio_user. > Maybe rename this function to virtqueue_reset and move it to virtqueue.c = will > be more suitable. > Please also add suffix _packed as this function only workable for packed = ring. >=20 > Thanks, > Marvin >=20 Hi Marvin, this function only works for virtqueue, so move it to virtqueue.= c is better. Two functions' name will be changed to virtqueue_rxvq_reset_packed and virt= queue_txvq_reset_packed, respectively. Thank you for your suggestion. Xuan > > + uint16_t vtpci_queue_idx =3D 2 * queue_idx + VTNET_SQ_RQ_QUEUE_IDX; > > + struct virtio_hw *hw =3D dev->data->dev_private; > > + struct virtqueue *vq =3D hw->vqs[vtpci_queue_idx]; > > + struct virtnet_rx *rxvq; > > + struct vq_desc_extra *dxp; > > + unsigned int vq_size; > > + uint16_t desc_idx, i; > > + > > + vq_size =3D VTPCI_OPS(hw)->get_queue_num(hw, vtpci_queue_idx); > > + > Virtqueue size has been set to vq_nentries in virtqueue structure. Do we = need to > re-catch it? >=20 Virtqueue size doesn't need to be re-catched, so vq->vq_nentries will be us= ed instead. > > + vq->vq_packed.used_wrap_counter =3D 1; > > + vq->vq_packed.cached_flags =3D VRING_PACKED_DESC_F_AVAIL; > > + vq->vq_packed.event_flags_shadow =3D 0; > > + vq->vq_packed.cached_flags |=3D VRING_DESC_F_WRITE; > > + > > + rxvq =3D &vq->rxq; > > + memset(rxvq->mz->addr, 0, rxvq->mz->len); > > + > > + for (desc_idx =3D 0; desc_idx < vq_size; desc_idx++) { > > + dxp =3D &vq->vq_descx[desc_idx]; > > + if (dxp->cookie !=3D NULL) { > > + rte_pktmbuf_free(dxp->cookie); > > + dxp->cookie =3D NULL; > > + } > > + } > > + > > + virtio_init_vring(vq); > > + > > + for (i =3D 0; i < hw->max_queue_pairs; i++) > > + if (rxvq->mpool !=3D NULL) > > + virtio_dev_rx_queue_setup_finish(dev, i); > > + >=20 > Please add parentheses for multiple lines loop content. Thank you for your notice. >=20 > > + return 0; > > +} > > + > > +static int > > +virtio_user_reset_tx_queues(struct rte_eth_dev *dev, uint16_t > > +queue_idx) { > > + uint8_t vtpci_queue_idx =3D 2 * queue_idx + VTNET_SQ_TQ_QUEUE_IDX; > > + struct virtio_hw *hw =3D dev->data->dev_private; > > + struct virtqueue *vq =3D hw->vqs[vtpci_queue_idx]; > > + struct virtnet_tx *txvq; > > + struct vq_desc_extra *dxp; > > + unsigned int vq_size; > > + uint16_t desc_idx; > > + > > + vq_size =3D VTPCI_OPS(hw)->get_queue_num(hw, vtpci_queue_idx); > > + > > + vq->vq_packed.used_wrap_counter =3D 1; > > + vq->vq_packed.cached_flags =3D VRING_PACKED_DESC_F_AVAIL; > > + vq->vq_packed.event_flags_shadow =3D 0; > > + > > + txvq =3D &vq->txq; > > + memset(txvq->mz->addr, 0, txvq->mz->len); > > + memset(txvq->virtio_net_hdr_mz->addr, 0, > > + txvq->virtio_net_hdr_mz->len); > > + > > + for (desc_idx =3D 0; desc_idx < vq_size; desc_idx++) { > > + dxp =3D &vq->vq_descx[desc_idx]; > > + if (dxp->cookie !=3D NULL) { > > + rte_pktmbuf_free(dxp->cookie); > > + dxp->cookie =3D NULL; > > + } > > + } > > + > > + virtio_init_vring(vq); > > + > > + return 0; > > +} > > + > > +static int > > +virtio_user_reset_queues(struct rte_eth_dev *eth_dev) { > > + uint16_t i; > > + > > + /* Vring reset for each Tx queue and Rx queue. */ > > + for (i =3D 0; i < eth_dev->data->nb_rx_queues; i++) > > + virtio_user_reset_rx_queues(eth_dev, i); > > + > > + for (i =3D 0; i < eth_dev->data->nb_rx_queues; i++) > > + virtio_user_reset_tx_queues(eth_dev, i); > > + > > + return 0; > > +} > > + > > static int > > virtio_init_queue(struct rte_eth_dev *dev, uint16_t vtpci_queue_idx) > > { @@ -1913,6 +2001,8 @@ eth_virtio_dev_init(struct rte_eth_dev > > *eth_dev) > > goto err_vtpci_init; > > } > > > > + rte_spinlock_init(&hw->state_lock); > > + > > /* reset device and negotiate default features */ > > ret =3D virtio_init_device(eth_dev, > > VIRTIO_PMD_DEFAULT_GUEST_FEATURES); > > if (ret < 0) > > @@ -2155,8 +2245,6 @@ virtio_dev_configure(struct rte_eth_dev *dev) > > return -EBUSY; > > } > > > > - rte_spinlock_init(&hw->state_lock); > > - > > hw->use_simple_rx =3D 1; > > > > if (vtpci_with_feature(hw, VIRTIO_F_IN_ORDER)) { @@ -2421,6 > +2509,26 > > @@ virtio_dev_vlan_offload_set(struct rte_eth_dev *dev, int mask) > > return 0; > > } > > > > +int > > +virtio_user_reset_device(struct rte_eth_dev *eth_dev, struct > > +virtio_hw > > *hw) > > +{ > > + /* Add lock to avoid queue contention. */ > > + rte_spinlock_lock(&hw->state_lock); > > + hw->started =3D 0; > > + /* > > + * Waitting for datapath to complete before resetting queues. > > + * 1 ms should be enough for the ongoing Tx/Rx function to finish. > > + */ > > + rte_delay_ms(1); > > + > > + virtio_user_reset_queues(eth_dev); > > + > > + hw->started =3D 1; > > + rte_spinlock_unlock(&hw->state_lock); > > + > > + return 0; > > +} > > + > > static int > > virtio_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info > > *dev_info) > > { > > diff --git a/drivers/net/virtio/virtio_ethdev.h > > b/drivers/net/virtio/virtio_ethdev.h > > index a10111758..72e9e3ff8 100644 > > --- a/drivers/net/virtio/virtio_ethdev.h > > +++ b/drivers/net/virtio/virtio_ethdev.h > > @@ -49,6 +49,9 @@ > > > > extern const struct eth_dev_ops virtio_user_secondary_eth_dev_ops; > > > > +int virtio_user_reset_device(struct rte_eth_dev *eth_dev, > > + struct virtio_hw *hw); > > + > > /* > > * CQ function prototype > > */ > > diff --git a/drivers/net/virtio/virtio_user_ethdev.c > > b/drivers/net/virtio/virtio_user_ethdev.c > > index 3fc172573..49068a578 100644 > > --- a/drivers/net/virtio/virtio_user_ethdev.c > > +++ b/drivers/net/virtio/virtio_user_ethdev.c > > @@ -31,6 +31,7 @@ virtio_user_server_reconnect(struct virtio_user_dev > > *dev) > > int ret; > > int connectfd; > > struct rte_eth_dev *eth_dev =3D &rte_eth_devices[dev->port_id]; > > + struct virtio_hw *hw =3D eth_dev->data->dev_private; > > > > connectfd =3D accept(dev->listenfd, NULL, NULL); > > if (connectfd < 0) > > @@ -51,6 +52,13 @@ virtio_user_server_reconnect(struct virtio_user_dev > > *dev) > > > > dev->features &=3D dev->device_features; > > > > + /* > > + * * For packed ring, resetting queues > > + * is required in reconnection. > > + */ > > + if (vtpci_packed_queue(hw)) > > + virtio_user_reset_device(eth_dev, hw); > > + > > ret =3D virtio_user_start_device(dev); > > if (ret < 0) > > return -1; > > -- > > 2.17.1