From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 2E7442BFA for ; Wed, 2 Nov 2016 02:38:55 +0100 (CET) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga104.jf.intel.com with ESMTP; 01 Nov 2016 18:38:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,582,1473145200"; d="scan'208";a="1053814825" Received: from fmsmsx107.amr.corp.intel.com ([10.18.124.205]) by orsmga001.jf.intel.com with ESMTP; 01 Nov 2016 18:38:54 -0700 Received: from fmsmsx121.amr.corp.intel.com (10.18.125.36) by fmsmsx107.amr.corp.intel.com (10.18.124.205) with Microsoft SMTP Server (TLS) id 14.3.248.2; Tue, 1 Nov 2016 18:38:54 -0700 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by fmsmsx121.amr.corp.intel.com (10.18.125.36) with Microsoft SMTP Server (TLS) id 14.3.248.2; Tue, 1 Nov 2016 18:38:54 -0700 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.206]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.96]) with mapi id 14.03.0248.002; Wed, 2 Nov 2016 09:38:48 +0800 From: "Yao, Lei A" To: Olivier Matz , "dev@dpdk.org" , "yuanhan.liu@linux.intel.com" Thread-Topic: [dpdk-dev] [PATCH v3 02/12] net/virtio: setup and start cq in configure callback Thread-Index: AQHSJVyQQTPnOO6a0UW7Wx8OTEcoXaDFCBpQ Date: Wed, 2 Nov 2016 01:38:47 +0000 Message-ID: <2DBBFF226F7CF64BAFCA79B681719D9537F234A7@shsmsx102.ccr.corp.intel.com> References: <1469088510-7552-1-git-send-email-olivier.matz@6wind.com> <1476368171-18176-1-git-send-email-olivier.matz@6wind.com> <1476368171-18176-3-git-send-email-olivier.matz@6wind.com> In-Reply-To: <1476368171-18176-3-git-send-email-olivier.matz@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dprovan@bivio.net" Subject: Re: [dpdk-dev] [PATCH v3 02/12] net/virtio: setup and start cq in configure callback X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Nov 2016 01:38:57 -0000 Hi, Olivier During the validation work with v16.11-rc2, I find that this patch will cau= se VM crash if enable virtio bonding in VM. Could you have a check at your = side? The following is steps at my side. Thanks a lot 1. bind PF port to igb_uio. modprobe uio insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko ./tools/dpdk-devbind.py --bind=3Digb_uio 84:00.1 2. start vhost switch. ./examples/vhost/build/vhost-switch -c 0x1c0000 -n 4 --socket-mem 4096,4096= - -p 0x1 --mergeable 0 --vm2vm 0 --socket-file ./vhost-net 3. bootup one vm with four virtio net device qemu-system-x86_64 \ -name vm0 -enable-kvm -chardev socket,path=3D/tmp/vm0_qga0.sock,server,nowa= it,id=3Dvm0_qga0 \ -device virtio-serial -device virtserialport,chardev=3Dvm0_qga0,name=3Dorg.= qemu.guest_agent.0 \ -daemonize -monitor unix:/tmp/vm0_monitor.sock,server,nowait \ -net nic,vlan=3D0,macaddr=3D00:00:00:c7:56:64,addr=3D1f \ net user,vlan=3D0,hostfwd=3Dtcp:10.239.129.127:6107:22 \ -chardev socket,id=3Dchar0,path=3D./vhost-net \ -netdev type=3Dvhost-user,id=3Dnetdev0,chardev=3Dchar0,vhostforce \ -device virtio-net-pci,netdev=3Dnetdev0,mac=3D52:54:00:00:00:01 \ -chardev socket,id=3Dchar1,path=3D./vhost-net \ -netdev type=3Dvhost-user,id=3Dnetdev1,chardev=3Dchar1,vhostforce \ -device virtio-net-pci,netdev=3Dnetdev1,mac=3D52:54:00:00:00:02 \ -chardev socket,id=3Dchar2,path=3D./vhost-net \ -netdev type=3Dvhost-user,id=3Dnetdev2,chardev=3Dchar2,vhostforce \ -device virtio-net-pci,netdev=3Dnetdev2,mac=3D52:54:00:00:00:03 \ -chardev socket,id=3Dchar3,path=3D./vhost-net \ -netdev type=3Dvhost-user,id=3Dnetdev3,chardev=3Dchar3,vhostforce \ -device virtio-net-pci,netdev=3Dnetdev3,mac=3D52:54:00:00:00:04 \ -cpu host -smp 8 -m 4096 \ -object memory-backend-file,id=3Dmem,size=3D4096M,mem-path=3D/mnt/huge,shar= e=3Don \ -numa node,memdev=3Dmem -mem-prealloc -drive file=3D/home/osimg/ubuntu16.im= g -vnc :10 4. on vm: bind virtio net device to igb_uio modprobe uio insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko tools/dpdk-devbind.py --bind=3Digb_uio 00:04.0 00:05.0 00:06.0 00:07.0 5. startup test_pmd app ./x86_64-native-linuxapp-gcc/app/testpmd -c 0x1f -n 4 - -i --txqflags=3D0xf= 00 --disable-hw-vlan-filter 6. create one bonding device (port 4) create bonded device 0 0 (the first 0: mode, the second: the socket number) show bonding config 4 7. bind port 0, 1, 2 to port 4 add bonding slave 0 4 add bonding slave 1 4 add bonding slave 2 4 port start 4 Result: just after port start 4(port 4 is bonded port), the vm shutdown imm= ediately. BRs Lei -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Olivier Matz Sent: Thursday, October 13, 2016 10:16 PM To: dev@dpdk.org; yuanhan.liu@linux.intel.com Cc: Ananyev, Konstantin ; Chandran, Sugesh ; Richardson, Bruce ; = Tan, Jianfeng ; Zhang, Helin ; adrien.mazarguil@6wind.com; stephen@networkplumber.org; dprovan@bivio.ne= t; Wang, Xiao W ; maxime.coquelin@redhat.com Subject: [dpdk-dev] [PATCH v3 02/12] net/virtio: setup and start cq in conf= igure callback Move the configuration of control queue in the configure callback. This is needed by next commit, which introduces the reinitialization of the= device in the configure callback to change the feature flags. Therefore, the control queue will have to be restarted at the same place. As virtio_dev_cq_queue_setup() is called from a place where config->max_virtqueue_pairs is not available, we need to store this in the private structure. It replaces max_rx_queues and max_tx_queues which ha= ve the same value. The log showing the value of max_rx_queues and max_tx_qu= eues is also removed since config->max_virtqueue_pairs is already displayed= above. Signed-off-by: Olivier Matz Reviewed-by: Maxime Coquelin --- drivers/net/virtio/virtio_ethdev.c | 43 +++++++++++++++++++---------------= ---- drivers/net/virtio/virtio_ethdev.h | 4 ++-- drivers/net/virtio/virtio_pci.h | 3 +-- 3 files changed, 24 insertions(+), 26 deletions(-) diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio= _ethdev.c index 77ca569..f3921ac 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -552,6 +552,9 @@ virtio_dev_close(struct rte_eth_dev *dev) if (hw->started =3D=3D 1) virtio_dev_stop(dev); =20 + if (hw->cvq) + virtio_dev_queue_release(hw->cvq->vq); + /* reset the NIC */ if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) vtpci_irq_config(hw, VIRTIO_MSI_NO_VECTOR); @@ -1191,16 +1194,7 @@ virti= o_init_device(struct rte_eth_dev *eth_dev) config->max_virtqueue_pairs =3D 1; } =20 - hw->max_rx_queues =3D - (VIRTIO_MAX_RX_QUEUES < config->max_virtqueue_pairs) ? - VIRTIO_MAX_RX_QUEUES : config->max_virtqueue_pairs; - hw->max_tx_queues =3D - (VIRTIO_MAX_TX_QUEUES < config->max_virtqueue_pairs) ? - VIRTIO_MAX_TX_QUEUES : config->max_virtqueue_pairs; - - virtio_dev_cq_queue_setup(eth_dev, - config->max_virtqueue_pairs * 2, - SOCKET_ID_ANY); + hw->max_queue_pairs =3D config->max_virtqueue_pairs; =20 PMD_INIT_LOG(DEBUG, "config->max_virtqueue_pairs=3D%d", config->max_virtqueue_pairs); @@ -1211,19 +1205,15 @@ virtio_init_device(struct rte_eth_dev *eth_dev) config->mac[2], config->mac[3], config->mac[4], config->mac[5]); } else { - hw->max_rx_queues =3D 1; - hw->max_tx_queues =3D 1; + PMD_INIT_LOG(DEBUG, "config->max_virtqueue_pairs=3D1"); + hw->max_queue_pairs =3D 1; } =20 - PMD_INIT_LOG(DEBUG, "hw->max_rx_queues=3D%d hw->max_tx_queues=3D%d", - hw->max_rx_queues, hw->max_tx_queues); if (pci_dev) PMD_INIT_LOG(DEBUG, "port %d vendorID=3D0x%x deviceID=3D0x%x", eth_dev->data->port_id, pci_dev->id.vendor_id, pci_dev->id.device_id); =20 - virtio_dev_cq_start(eth_dev); - return 0; } =20 @@ -1285,7 +1275,6 @@ static int eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev) { struct rte_pci_device *pci_dev; - struct virtio_hw *hw =3D eth_dev->data->dev_private; =20 PMD_INIT_FUNC_TRACE(); =20 @@ -1301,9 +1290,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev) eth_dev->tx_pkt_burst =3D NULL; eth_dev->rx_pkt_burst =3D NULL; =20 - if (hw->cvq) - virtio_dev_queue_release(hw->cvq->vq); - rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs =3D NULL; =20 @@ -1352,6 +1338,7 @@ virtio_dev_configure(struct rte_eth_dev *dev) { const struct rte_eth_rxmode *rxmode =3D &dev->data->dev_conf.rxmode; struct virtio_hw *hw =3D dev->data->dev_private; + int ret; =20 PMD_INIT_LOG(DEBUG, "configure"); =20 @@ -1360,6 +1347,16 @@ virtio_dev_configure(struct rte_eth_dev *dev) return -EINVAL; } =20 + /* Setup and start control queue */ + if (vtpci_with_feature(hw, VIRTIO_NET_F_CTRL_VQ)) { + ret =3D virtio_dev_cq_queue_setup(dev, + hw->max_queue_pairs * 2, + SOCKET_ID_ANY); + if (ret < 0) + return ret; + virtio_dev_cq_start(dev); + } + hw->vlan_strip =3D rxmode->hw_vlan_strip; =20 if (rxmode->hw_vlan_filter @@ -1553,8 +1550,10 @@ virtio_dev_info_get(struct rte_eth_dev *dev, struct = rte_eth_dev_info *dev_info) dev_info->driver_name =3D dev->driver->pci_drv.driver.name; else dev_info->driver_name =3D "virtio_user PMD"; - dev_info->max_rx_queues =3D (uint16_t)hw->max_rx_queues; - dev_info->max_tx_queues =3D (uint16_t)hw->max_tx_queues; + dev_info->max_rx_queues =3D + RTE_MIN(hw->max_queue_pairs, VIRTIO_MAX_RX_QUEUES); + dev_info->max_tx_queues =3D + RTE_MIN(hw->max_queue_pairs, VIRTIO_MAX_TX_QUEUES); dev_info->min_rx_bufsize =3D VIRTIO_MIN_RX_BUFSIZE; dev_info->max_rx_pktlen =3D VIRTIO_MAX_RX_PKTLEN; dev_info->max_mac_addrs =3D VIRTIO_MAX_MAC_ADDRS; diff --git a/drivers/ne= t/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h index 04d626b..dc18341 100644 --- a/drivers/net/virtio/virtio_ethdev.h +++ b/drivers/net/virtio/virtio_ethdev.h @@ -47,8 +47,8 @@ #define PAGE_SIZE 4096 #endif =20 -#define VIRTIO_MAX_RX_QUEUES 128 -#define VIRTIO_MAX_TX_QUEUES 128 +#define VIRTIO_MAX_RX_QUEUES 128U +#define VIRTIO_MAX_TX_QUEUES 128U #define VIRTIO_MAX_MAC_ADDRS 64 #define VIRTIO_MIN_RX_BUFSIZE 64 #define VIRTIO_MAX_RX_PKTLEN 9728 diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pc= i.h index b8295a7..6930cd6 100644 --- a/drivers/net/virtio/virtio_pci.h +++ b/drivers/net/virtio/virtio_pci.h @@ -246,8 +246,7 @@ struct virtio_hw { struct virtnet_ctl *cvq; struct rte_pci_ioport io; uint64_t guest_features; - uint32_t max_tx_queues; - uint32_t max_rx_queues; + uint32_t max_queue_pairs; uint16_t vtnet_hdr_size; uint8_t vlan_strip; uint8_t use_msix; -- 2.8.1