From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f46.google.com (mail-pa0-f46.google.com [209.85.220.46]) by dpdk.org (Postfix) with ESMTP id DF73D9A88 for ; Wed, 10 Feb 2016 04:40:37 +0100 (CET) Received: by mail-pa0-f46.google.com with SMTP id yy13so4664689pab.3 for ; Tue, 09 Feb 2016 19:40:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igel-co-jp.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=OeWL+6B1CVt/p8zUwCtdrdzQmwA1MSJYCD4GIqbS0Ew=; b=XW8wTaqFYpaP6fjwCHggQu+uG2ecTyNAZRsxRNkFFX0D9hzNv3lTHjjhBXJTf3kp98 nOKrm2Hhz+zcaSitC+CXGySeStF31vAWuCPdglQtaZlLP3xj+O0tFV9mR83ItwEhvCwR vrKfpwzUPouZDMjKlxgeErOB4E/0yHfKZyaXWHFp2rTWeCXsn5rYvdZjgkgsimNHTF6L CMEmc/qE6iRyFniG4CFvUXRR3kM19jxVu0QELTByNDEymn2wZLC/D/BGnW9AsXngo+7z +h93qlRA8EASvq2LBG2SzRJbW6Kj0yMlZXrylEEaPr8KFujfvQAdjLguznFv7TsB/X9p VOig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=OeWL+6B1CVt/p8zUwCtdrdzQmwA1MSJYCD4GIqbS0Ew=; b=TOIZRd2kIHYqstSooonVLFfpr1KBBFdM+5Esz6DfmCM2yMVede2EfYcH5dxZBLnBzn QXmL6F7HElRB+FBkKsxRYUWoTSMzeBTl8kRtk6J5jmabq5PvvCXObCqbPF+wYXrtGD3q LA8jV9oa4PunyA7UMDUFZk2kYm3hU/Sm/TeL1pOKqKtXxv4q76yw/GQiUzweSTLprFkf 2ZtZwjLnHOrU/Gxp/WOYzIesvNkhUc7s0dlEMlwd6c9CjxpLyi0jLCWINhAawBq6gk00 gs05+8EUIrZirenDu6uDK86zIDnTMaGGdCYhBAsbUEB0FOnwoJw+iEoUa/CS1XiJfMBP XcYg== X-Gm-Message-State: AG10YOQOF0lo2icn/nPXNDK4Nb00qoYb0pjZeRasFtdewK3bj/sm0h84MInKKyl71+OKaw== X-Received: by 10.67.7.42 with SMTP id cz10mr40711302pad.158.1455075637307; Tue, 09 Feb 2016 19:40:37 -0800 (PST) Received: from localhost.localdomain (napt.igel.co.jp. [219.106.231.132]) by smtp.gmail.com with ESMTPSA id m75sm1001576pfj.38.2016.02.09.19.40.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 09 Feb 2016 19:40:36 -0800 (PST) From: Tetsuya Mukawa To: dev@dpdk.org Date: Wed, 10 Feb 2016 12:40:12 +0900 Message-Id: <1455075613-3605-5-git-send-email-mukawa@igel.co.jp> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1455075613-3605-1-git-send-email-mukawa@igel.co.jp> References: <1455075613-3605-1-git-send-email-mukawa@igel.co.jp> In-Reply-To: <1453374478-30996-6-git-send-email-mukawa@igel.co.jp> References: <1453374478-30996-6-git-send-email-mukawa@igel.co.jp> Subject: [dpdk-dev] [PATCH v2 4/5] virtio: Add support for qtest virtio-net PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Feb 2016 03:40:38 -0000 The patch adds a new virtio-net PMD configuration that allows the PMD to work on host as if the PMD is in VM. Here is new configuration for virtio-net PMD. - CONFIG_RTE_VIRTIO_VDEV_QTEST To use this mode, EAL needs map all hugepages as one file. Also the file should be mapped between (1 << 31) and (1 << 44). To allocate like above, add "--single-file" and "--qtest-virtio" option to application command line. To prepare virtio-net device on host, the users need to invoke QEMU process in special qtest mode. This mode is mainly used for testing QEMU devices from outer process. In this mode, no guest runs. Here is QEMU command line. $ qemu-system-x86_64 \ -machine pc-i440fx-1.4,accel=qtest \ -display none -qtest-log /dev/null \ -qtest unix:/tmp/socket,server \ -netdev type=tap,script=/etc/qemu-ifup,id=net0,queues=1 \ -device virtio-net-pci,netdev=net0,mq=on,disable-modern=false,addr=3 \ -chardev socket,id=chr1,path=/tmp/ivshmem,server \ -device ivshmem,size=1G,chardev=chr1,vectors=1,addr=4 * Should use qemu-2.5.1, or above. * QEMU process is needed per port. * virtio-1.0 device are only supported. * The vhost backends like vhost-net and vhost-user can be specified. * In most cases, just using above command is enough, but you can also specify other QEMU virtio-net options. * Only checked "pc-i440fx-1.4" machine, but may work with other machines. It depends on a machine has piix3 south bridge. If the machine doesn't have, virtio-net PMD cannot receive status changed interrupts. * Should not add "--enable-kvm" to QEMU command line. After invoking QEMU, the PMD can connect to QEMU process using unix domain sockets. Over these sockets, virtio-net, ivshmem and piix3 device in QEMU are probed by the PMD. Here is example of command line. $ testpmd -c f -n 1 -m 1024 --no-pci --single-file --qtest-virtio \ --vdev="eth_qtest_virtio0,qtest=/tmp/socket,ivshmem=/tmp/ivshmem"\ -- --disable-hw-vlan --txqflags=0xf00 -i Please specify same unix domain sockets and memory size in both QEMU and DPDK command lines like above. The share memory size should be power of 2, because ivshmem only accepts such memory size. Signed-off-by: Tetsuya Mukawa --- config/common_linuxapp | 1 + drivers/net/virtio/Makefile | 4 + drivers/net/virtio/qtest.c | 1342 ++++++++++++++++++++++++++++++++++++ drivers/net/virtio/qtest.h | 65 ++ drivers/net/virtio/virtio_ethdev.c | 383 +++++++++- drivers/net/virtio/virtio_pci.c | 364 +++++++++- drivers/net/virtio/virtio_pci.h | 5 +- 7 files changed, 2122 insertions(+), 42 deletions(-) create mode 100644 drivers/net/virtio/qtest.c create mode 100644 drivers/net/virtio/qtest.h diff --git a/config/common_linuxapp b/config/common_linuxapp index f76e162..7cbf50d 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -539,3 +539,4 @@ CONFIG_RTE_TEST_PMD_RECORD_BURST_STATS=n # Enable virtio support for container # CONFIG_RTE_VIRTIO_VDEV=y +CONFIG_RTE_VIRTIO_VDEV_QTEST=y diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile index ef920f9..6c11378 100644 --- a/drivers/net/virtio/Makefile +++ b/drivers/net/virtio/Makefile @@ -56,6 +56,10 @@ ifeq ($(CONFIG_RTE_VIRTIO_VDEV),y) SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += vhost_embedded.c endif +ifeq ($(CONFIG_RTE_VIRTIO_VDEV_QTEST),y) + SRCS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += qtest.c +endif + # this lib depends upon: DEPDIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += lib/librte_eal lib/librte_ether DEPDIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += lib/librte_mempool lib/librte_mbuf diff --git a/drivers/net/virtio/qtest.c b/drivers/net/virtio/qtest.c new file mode 100644 index 0000000..418214f --- /dev/null +++ b/drivers/net/virtio/qtest.c @@ -0,0 +1,1342 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 IGEL Co., Ltd. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IGEL Co., Ltd. nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "virtio_pci.h" +#include "virtio_logs.h" +#include "virtio_ethdev.h" +#include "qtest.h" + +#define NB_BAR 6 + +/* PIIX3 configuration registers */ +#define PIIX3_REG_ADDR_PIRQA 0x60 +#define PIIX3_REG_ADDR_PIRQB 0x61 +#define PIIX3_REG_ADDR_PIRQC 0x62 +#define PIIX3_REG_ADDR_PIRQD 0x63 + +/* Device information */ +#define VIRTIO_NET_DEVICE_ID 0x1000 +#define VIRTIO_NET_VENDOR_ID 0x1af4 +#define VIRTIO_NET_IRQ_NUM 10 +#define IVSHMEM_DEVICE_ID 0x1110 +#define IVSHMEM_VENDOR_ID 0x1af4 +#define IVSHMEM_PROTOCOL_VERSION 0 +#define PIIX3_DEVICE_ID 0x7000 +#define PIIX3_VENDOR_ID 0x8086 + +/* ------------------------------------------------------------ + * IO port mapping of qtest guest + * ------------------------------------------------------------ + * 0x0000 - 0xbfff : not used + * 0xc000 - 0xc03f : virtio-net(BAR0) + * 0xc040 - 0xffff : not used + * + * ------------------------------------------------------------ + * Memory mapping of qtest quest + * ------------------------------------------------------------ + * 0x00000000_00000000 - 0x00000000_3fffffff : not used + * 0x00000000_40000000 - 0x00000000_40000fff : virtio-net(BAR1) + * 0x00000000_40001000 - 0x00000000_40ffffff : not used + * 0x00000000_41000000 - 0x00000000_417fffff : virtio-net(BAR4) + * 0x00000000_41800000 - 0x00000000_41ffffff : not used + * 0x00000000_42000000 - 0x00000000_420000ff : ivshmem(BAR0) + * 0x00000000_42000100 - 0x00000000_42ffffff : not used + * 0x00000000_80000000 - 0xffffffff_ffffffff : ivshmem(BAR2) + * + * We can only specify start address of a region. The region size + * will be defined by the device implementation in QEMU. + * The size will be pow of 2 accroding to the PCI specification. + * Also, the region start address should be aligned by region size. + * + * BAR2 of ivshmem will be mmapped address of DPDK application memory. + * So this address will be dynamically changed, but not to overlap + * others, it shoulb be mmaped between above addresses. Such allocation + * is done by EAL. Check rte_eal_get_free_region() also. + */ +#define VIRTIO_NET_IO_START 0xc000 +#define VIRTIO_NET_MEMORY1_START 0x40000000 +#define VIRTIO_NET_MEMORY2_START 0x41000000 +#define IVSHMEM_MEMORY_START 0x42000000 + +#define PCI_CONFIG_ADDR(_bus, _device, _function, _offset) ( \ + (1 << 31) | ((_bus) & 0xff) << 16 | ((_device) & 0x1f) << 11 | \ + ((_function) & 0x7) << 8 | ((_offset) & 0xfc)) + +static char interrupt_message[32]; + +enum qtest_pci_bar_type { + QTEST_PCI_BAR_DISABLE = 0, + QTEST_PCI_BAR_IO, + QTEST_PCI_BAR_MEMORY_UNDER_1MB, + QTEST_PCI_BAR_MEMORY_32, + QTEST_PCI_BAR_MEMORY_64 +}; + +struct qtest_pci_bar { + enum qtest_pci_bar_type type; + uint8_t addr; + uint64_t region_start; + uint64_t region_size; +}; + +struct qtest_session; +TAILQ_HEAD(qtest_pci_device_list, qtest_pci_device); +struct qtest_pci_device { + TAILQ_ENTRY(qtest_pci_device) next; + const char *name; + uint16_t device_id; + uint16_t vendor_id; + uint8_t bus_addr; + uint8_t device_addr; + struct qtest_pci_bar bar[NB_BAR]; + int (*init)(struct qtest_session *s, struct qtest_pci_device *dev); +}; + +union qtest_pipefds { + struct { + int pipefd[2]; + }; + struct { + int readfd; + int writefd; + }; +}; + +struct qtest_session { + int qtest_socket; + pthread_mutex_t qtest_session_lock; + + struct qtest_pci_device_list head; + int ivshmem_socket; + + pthread_t event_th; + char *evq; + char *evq_dequeue_ptr; + size_t evq_total_len; + + union qtest_pipefds msgfds; + + pthread_t intr_th; + int eventfd; + rte_atomic16_t enable_intr; + rte_intr_callback_fn cb; + void *cb_arg; + struct rte_eth_dev_data *eth_data; +}; + +static int +qtest_raw_send(int fd, char *buf, size_t count) +{ + size_t len = count; + size_t total_len = 0; + int ret = 0; + + while (len > 0) { + ret = write(fd, buf, len); + if (ret == -1) { + if (errno == EINTR) + continue; + return ret; + } + if (ret == (int)len) + break; + total_len += ret; + buf += ret; + len -= ret; + } + return total_len + ret; +} + +static int +qtest_raw_recv(int fd, char *buf, size_t count) +{ + size_t len = count; + size_t total_len = 0; + int ret = 0; + + while (len > 0) { + ret = read(fd, buf, len); + if (ret <= 0) { + if (errno == EINTR) { + continue; + } + return ret; + } + if (ret == (int)len) + break; + if (*(buf + ret - 1) == '\n') + break; + total_len += ret; + buf += ret; + len -= ret; + } + return total_len + ret; +} + +/* + * To know QTest protocol specification, see below QEMU source code. + * - qemu/qtest.c + * If qtest socket is closed, qtest_raw_in and qtest_raw_read will return 0. + */ +static uint32_t +qtest_raw_in(struct qtest_session *s, uint16_t addr, char type) +{ + char buf[64]; + int ret; + + if ((type != 'l') && (type != 'w') && (type != 'b')) + rte_panic("Invalid value\n"); + + snprintf(buf, sizeof(buf), "in%c 0x%x\n", type, addr); + /* write to qtest socket */ + ret = qtest_raw_send(s->qtest_socket, buf, strlen(buf)); + /* read reply from event handler */ + ret = qtest_raw_recv(s->msgfds.readfd, buf, sizeof(buf)); + if (ret < 0) + return 0; + + buf[ret] = '\0'; + return strtoul(buf + strlen("OK "), NULL, 16); +} + +static void +qtest_raw_out(struct qtest_session *s, uint16_t addr, uint32_t val, char type) +{ + char buf[64]; + int ret __rte_unused; + + if ((type != 'l') && (type != 'w') && (type != 'b')) + rte_panic("Invalid value\n"); + + snprintf(buf, sizeof(buf), "out%c 0x%x 0x%x\n", type, addr, val); + /* write to qtest socket */ + ret = qtest_raw_send(s->qtest_socket, buf, strlen(buf)); + /* read reply from event handler */ + ret = qtest_raw_recv(s->msgfds.readfd, buf, sizeof(buf)); +} + +static uint32_t +qtest_raw_read(struct qtest_session *s, uint64_t addr, char type) +{ + char buf[64]; + int ret; + + if ((type != 'l') && (type != 'w') && (type != 'b')) + rte_panic("Invalid value\n"); + + snprintf(buf, sizeof(buf), "read%c 0x%lx\n", type, addr); + /* write to qtest socket */ + ret = qtest_raw_send(s->qtest_socket, buf, strlen(buf)); + /* read reply from event handler */ + ret = qtest_raw_recv(s->msgfds.readfd, buf, sizeof(buf)); + if (ret < 0) + return 0; + + buf[ret] = '\0'; + return strtoul(buf + strlen("OK "), NULL, 16); +} + +static void +qtest_raw_write(struct qtest_session *s, uint64_t addr, uint32_t val, char type) +{ + char buf[64]; + int ret __rte_unused; + + if ((type != 'l') && (type != 'w') && (type != 'b')) + rte_panic("Invalid value\n"); + + snprintf(buf, sizeof(buf), "write%c 0x%lx 0x%x\n", type, addr, val); + /* write to qtest socket */ + ret = qtest_raw_send(s->qtest_socket, buf, strlen(buf)); + /* read reply from event handler */ + ret = qtest_raw_recv(s->msgfds.readfd, buf, sizeof(buf)); +} + +/* + * qtest_pci_inX/outX are used for accessing PCI configuration space. + * The functions are implemented based on PCI configuration space + * specification. + * Accroding to the spec, access size of read()/write() should be 4 bytes. + */ +static int +qtest_pci_inb(struct qtest_session *s, uint8_t bus, uint8_t device, + uint8_t function, uint8_t offset) +{ + uint32_t tmp; + + tmp = PCI_CONFIG_ADDR(bus, device, function, offset); + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_out(s, 0xcf8, tmp, 'l'); + tmp = qtest_raw_in(s, 0xcfc, 'l'); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot unlock mutex\n"); + + return (tmp >> ((offset & 0x3) * 8)) & 0xff; +} + +static void +qtest_pci_outb(struct qtest_session *s, uint8_t bus, uint8_t device, + uint8_t function, uint8_t offset, uint8_t value) +{ + uint32_t addr, tmp, pos; + + addr = PCI_CONFIG_ADDR(bus, device, function, offset); + pos = (offset % 4) * 8; + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_out(s, 0xcf8, addr, 'l'); + tmp = qtest_raw_in(s, 0xcfc, 'l'); + tmp = (tmp & ~(0xff << pos)) | (value << pos); + + qtest_raw_out(s, 0xcf8, addr, 'l'); + qtest_raw_out(s, 0xcfc, tmp, 'l'); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot unlock mutex\n"); +} + +static uint32_t +qtest_pci_inl(struct qtest_session *s, uint8_t bus, uint8_t device, + uint8_t function, uint8_t offset) +{ + uint32_t tmp; + + tmp = PCI_CONFIG_ADDR(bus, device, function, offset); + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_out(s, 0xcf8, tmp, 'l'); + tmp = qtest_raw_in(s, 0xcfc, 'l'); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot unlock mutex\n"); + + return tmp; +} + +static void +qtest_pci_outl(struct qtest_session *s, uint8_t bus, uint8_t device, + uint8_t function, uint8_t offset, uint32_t value) +{ + uint32_t tmp; + + tmp = PCI_CONFIG_ADDR(bus, device, function, offset); + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_out(s, 0xcf8, tmp, 'l'); + qtest_raw_out(s, 0xcfc, value, 'l'); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot unlock mutex\n"); +} + +static uint64_t +qtest_pci_inq(struct qtest_session *s, uint8_t bus, uint8_t device, + uint8_t function, uint8_t offset) +{ + uint32_t tmp; + uint64_t val; + + tmp = PCI_CONFIG_ADDR(bus, device, function, offset); + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_out(s, 0xcf8, tmp, 'l'); + val = (uint64_t)qtest_raw_in(s, 0xcfc, 'l'); + + tmp = PCI_CONFIG_ADDR(bus, device, function, offset + 4); + + qtest_raw_out(s, 0xcf8, tmp, 'l'); + val |= (uint64_t)qtest_raw_in(s, 0xcfc, 'l') << 32; + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot unlock mutex\n"); + + return val; +} + +static void +qtest_pci_outq(struct qtest_session *s, uint8_t bus, uint8_t device, + uint8_t function, uint8_t offset, uint64_t value) +{ + uint32_t tmp; + + tmp = PCI_CONFIG_ADDR(bus, device, function, offset); + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_out(s, 0xcf8, tmp, 'l'); + qtest_raw_out(s, 0xcfc, (uint32_t)(value & 0xffffffff), 'l'); + + tmp = PCI_CONFIG_ADDR(bus, device, function, offset + 4); + + qtest_raw_out(s, 0xcf8, tmp, 'l'); + qtest_raw_out(s, 0xcfc, (uint32_t)(value >> 32), 'l'); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot unlock mutex\n"); +} + +/* + * qtest_in/out are used for accessing ioport of qemu guest. + * qtest_read/write are used for accessing memory of qemu guest. + */ +uint32_t +qtest_in(struct virtio_hw *hw, uint16_t addr, char type) +{ + struct qtest_session *s = (struct qtest_session *)hw->qsession; + uint32_t val; + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + val = qtest_raw_in(s, addr, type); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + return val; +} + +void +qtest_out(struct virtio_hw *hw, uint16_t addr, uint64_t val, char type) +{ + struct qtest_session *s = (struct qtest_session *)hw->qsession; + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_out(s, addr, val, type); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); +} + +uint32_t +qtest_read(struct virtio_hw *hw, uint64_t addr, char type) +{ + struct qtest_session *s = (struct qtest_session *)hw->qsession; + uint32_t val; + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + val = qtest_raw_read(s, addr, type); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + return val; +} + +void +qtest_write(struct virtio_hw *hw, uint64_t addr, uint64_t val, char type) +{ + struct qtest_session *s = (struct qtest_session *)hw->qsession; + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + qtest_raw_write(s, addr, val, type); + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); +} + +static struct qtest_pci_device * +qtest_find_device(struct qtest_session *s, const char *name) +{ + struct qtest_pci_device *dev; + + TAILQ_FOREACH(dev, &s->head, next) { + if (strcmp(dev->name, name) == 0) + return dev; + } + return NULL; +} + +/* + * The function is used for reading pci configuration space of specifed device. + */ +int +qtest_read_pci_cfg(struct virtio_hw *hw, const char *name, + void *buf, size_t len, off_t offset) +{ + struct qtest_session *s = (struct qtest_session *)hw->qsession; + struct qtest_pci_device *dev; + uint32_t i; + uint8_t *p = buf; + + dev = qtest_find_device(s, name); + if (dev == NULL) { + PMD_DRV_LOG(ERR, "Cannot find specified device: %s\n", name); + return -1; + } + + for (i = 0; i < len; i++) { + *(p + i) = qtest_pci_inb(s, + dev->bus_addr, dev->device_addr, 0, offset + i); + } + + return 0; +} + +static struct qtest_pci_bar * +qtest_get_bar(struct virtio_hw *hw, const char *name, uint8_t bar) +{ + struct qtest_session *s = (struct qtest_session *)hw->qsession; + struct qtest_pci_device *dev; + + if (bar >= NB_BAR) { + PMD_DRV_LOG(ERR, "Invalid bar is specified: %u\n", bar); + return NULL; + } + + dev = qtest_find_device(s, name); + if (dev == NULL) { + PMD_DRV_LOG(ERR, "Cannot find specified device: %s\n", name); + return NULL; + } + + if (dev->bar[bar].type == QTEST_PCI_BAR_DISABLE) { + PMD_DRV_LOG(ERR, "Cannot find valid BAR(%s): %u\n", name, bar); + return NULL; + } + + return &dev->bar[bar]; +} + +int +qtest_get_bar_addr(struct virtio_hw *hw, const char *name, + uint8_t bar, uint64_t **addr) +{ + struct qtest_pci_bar *bar_ptr; + + bar_ptr = qtest_get_bar(hw, name, bar); + if (bar_ptr == NULL) + return -1; + + *addr = (uint64_t *)bar_ptr->region_start; + return 0; +} + +int +qtest_get_bar_size(struct virtio_hw *hw, const char *name, + uint8_t bar, uint64_t *size) +{ + struct qtest_pci_bar *bar_ptr; + + bar_ptr = qtest_get_bar(hw, name, bar); + if (bar_ptr == NULL) + return -1; + + *size = bar_ptr->region_size; + return 0; +} + +int +qtest_intr_enable(void *data) +{ + struct virtio_hw *hw = ((struct rte_eth_dev_data *)data)->dev_private; + struct qtest_session *s; + + s = (struct qtest_session *)hw->qsession; + rte_atomic16_set(&s->enable_intr, 1); + + return 0; +} + +int +qtest_intr_disable(void *data) +{ + struct virtio_hw *hw = ((struct rte_eth_dev_data *)data)->dev_private; + struct qtest_session *s; + + s = (struct qtest_session *)hw->qsession; + rte_atomic16_set(&s->enable_intr, 0); + + return 0; +} + +void +qtest_intr_callback_register(void *data, + rte_intr_callback_fn cb, void *cb_arg) +{ + struct virtio_hw *hw = ((struct rte_eth_dev_data *)data)->dev_private; + struct qtest_session *s; + + s = (struct qtest_session *)hw->qsession; + s->cb = cb; + s->cb_arg = cb_arg; + rte_atomic16_set(&s->enable_intr, 1); +} + +void +qtest_intr_callback_unregister(void *data, + rte_intr_callback_fn cb __rte_unused, + void *cb_arg __rte_unused) +{ + struct virtio_hw *hw = ((struct rte_eth_dev_data *)data)->dev_private; + struct qtest_session *s; + + s = (struct qtest_session *)hw->qsession; + rte_atomic16_set(&s->enable_intr, 0); + s->cb = NULL; + s->cb_arg = NULL; +} + +static void * +qtest_intr_handler(void *data) { + struct qtest_session *s = (struct qtest_session *)data; + eventfd_t value; + int ret; + + for (;;) { + ret = eventfd_read(s->eventfd, &value); + if (ret < 0) + return NULL; + s->cb(NULL, s->cb_arg); + } + return NULL; +} + +static int +qtest_intr_initialize(void *data) +{ + struct virtio_hw *hw = ((struct rte_eth_dev_data *)data)->dev_private; + struct qtest_session *s; + char buf[64]; + int ret; + + s = (struct qtest_session *)hw->qsession; + + /* This message will come when interrupt occurs */ + snprintf(interrupt_message, sizeof(interrupt_message), + "IRQ raise %d", VIRTIO_NET_IRQ_NUM); + + snprintf(buf, sizeof(buf), "irq_intercept_in ioapic\n"); + + if (pthread_mutex_lock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + /* To enable interrupt, send "irq_intercept_in" message to QEMU */ + ret = qtest_raw_send(s->qtest_socket, buf, strlen(buf)); + if (ret < 0) { + pthread_mutex_unlock(&s->qtest_session_lock); + return -1; + } + + /* just ignore QEMU response */ + ret = qtest_raw_recv(s->msgfds.readfd, buf, sizeof(buf)); + if (ret < 0) { + pthread_mutex_unlock(&s->qtest_session_lock); + return -1; + } + + if (pthread_mutex_unlock(&s->qtest_session_lock) < 0) + rte_panic("Cannot lock mutex\n"); + + return 0; +} + +static void +qtest_event_send(struct qtest_session *s, char *buf) +{ + int ret; + + if (strncmp(buf, interrupt_message, strlen(interrupt_message)) == 0) { + if (rte_atomic16_read(&s->enable_intr) == 0) + return; + + /* relay interrupt to eventfd */ + ret = eventfd_write(s->eventfd, 1); + if (ret < 0) + rte_panic("cannot relay interrupt\n"); + } else { + /* relay normal message to pipe */ + ret = qtest_raw_send(s->msgfds.writefd, buf, strlen(buf)); + if (ret < 0) + rte_panic("cannot relay normal message\n"); + } +} + +static void +qtest_close_one_socket(int *fd) +{ + if (*fd > 0) { + close(*fd); + *fd = -1; + } +} + +static void +qtest_close_sockets(struct qtest_session *s) +{ + qtest_close_one_socket(&s->qtest_socket); + qtest_close_one_socket(&s->msgfds.readfd); + qtest_close_one_socket(&s->msgfds.writefd); + qtest_close_one_socket(&s->eventfd); + qtest_close_one_socket(&s->ivshmem_socket); +} + +static void +qtest_event_enqueue(struct qtest_session *s, char *buf) +{ + size_t len = strlen(buf); + char *dest; + + if (s->evq == NULL) { + /* allocate one more byte for '\0' */ + s->evq = malloc(len + 1); + if (s->evq == NULL) + rte_panic("Cannot allocate memory\n"); + + s->evq_dequeue_ptr = s->evq; + s->evq_total_len = len + 1; + dest = s->evq; + } else { + size_t offset = s->evq_dequeue_ptr - s->evq; + + s->evq = realloc(s->evq, s->evq_total_len + len); + if (s->evq == NULL) + rte_panic("Cannot re-allocate memory\n"); + + s->evq_dequeue_ptr = s->evq + offset; + dest = s->evq + s->evq_total_len - 1; + s->evq_total_len += len; + } + + strncpy(dest, buf, len); + dest[len] = '\0'; +} + +static char * +qtest_event_dequeue(struct qtest_session *s) +{ + char *head, *next_head; + + head = s->evq_dequeue_ptr; + + /* make sure message is terminated by '\n' */ + next_head = strchr(s->evq_dequeue_ptr, '\n'); + if (next_head == NULL) + return NULL; + + /* set next dequeue pointer */ + s->evq_dequeue_ptr = next_head + 1; + + return head; +} + +static void +qtest_event_flush(struct qtest_session *s) +{ + if (s->evq) { + free(s->evq); + s->evq = NULL; + s->evq_dequeue_ptr = NULL; + s->evq_total_len = 0; + } +} + +/* + * This thread relays QTest response using pipe and eventfd. + * The function is needed because we need to separate IRQ message from others. + */ +static void * +qtest_event_handler(void *data) { + struct qtest_session *s = (struct qtest_session *)data; + char buf[64]; + char *p; + int ret; + + for (;;) { + memset(buf, 0, sizeof(buf)); + ret = qtest_raw_recv(s->qtest_socket, buf, sizeof(buf)); + if (ret <= 0) { + PMD_DRV_LOG(EMERG, + "Port %u: qtest connection was closed.\n" + "Please detach the port, then start QEMU " + "and attach the port again.\n", + s->eth_data->port_id); + qtest_close_sockets(s); + qtest_event_flush(s); + return NULL; + } + + qtest_event_enqueue(s, buf); + + /* in the case of incomplete message, receive again */ + p = &buf[sizeof(buf) - 1]; + if ((*p != '\0') && (*p != '\n')) + continue; + + /* may receive multiple messages at the same time */ + while ((p = qtest_event_dequeue(s)) != NULL) + qtest_event_send(s, p); + + qtest_event_flush(s); + } + return NULL; +} + +static int +qtest_init_piix3_device(struct qtest_session *s, struct qtest_pci_device *dev) +{ + uint8_t bus, device, virtio_net_slot = 0; + struct qtest_pci_device *tmpdev; + uint8_t pcislot2regaddr[] = { 0xff, + 0xff, + 0xff, + PIIX3_REG_ADDR_PIRQC, + PIIX3_REG_ADDR_PIRQD, + PIIX3_REG_ADDR_PIRQA, + PIIX3_REG_ADDR_PIRQB}; + + bus = dev->bus_addr; + device = dev->device_addr; + + PMD_DRV_LOG(INFO, + "Find %s on virtual PCI bus: %04x:%02x:00.0\n", + dev->name, bus, device); + + /* Get slot id that is connected to virtio-net */ + TAILQ_FOREACH(tmpdev, &s->head, next) { + if (strcmp(tmpdev->name, "virtio-net") == 0) { + virtio_net_slot = tmpdev->device_addr; + break; + } + } + + if (virtio_net_slot == 0) + return -1; + + /* + * Set interrupt routing for virtio-net device. + * Here is i440fx/piix3 connection settings + * --------------------------------------- + * PCI Slot3 -> PIRQC + * PCI Slot4 -> PIRQD + * PCI Slot5 -> PIRQA + * PCI Slot6 -> PIRQB + */ + if (pcislot2regaddr[virtio_net_slot] != 0xff) { + qtest_pci_outb(s, bus, device, 0, + pcislot2regaddr[virtio_net_slot], + VIRTIO_NET_IRQ_NUM); + } + + return 0; +} + +/* + * Common initialization of PCI device. + * To know detail, see pci specification. + */ +static int +qtest_init_pci_device(struct qtest_session *s, struct qtest_pci_device *dev) +{ + uint8_t i, bus, device; + uint32_t val; + uint64_t val64; + + bus = dev->bus_addr; + device = dev->device_addr; + + PMD_DRV_LOG(INFO, + "Find %s on virtual PCI bus: %04x:%02x:00.0\n", + dev->name, bus, device); + + /* Check header type */ + val = qtest_pci_inb(s, bus, device, 0, PCI_HEADER_TYPE); + if (val != PCI_HEADER_TYPE_NORMAL) { + PMD_DRV_LOG(ERR, "Unexpected header type %d\n", val); + return -1; + } + + /* Check BAR type */ + for (i = 0; i < NB_BAR; i++) { + val = qtest_pci_inl(s, bus, device, 0, dev->bar[i].addr); + + switch (dev->bar[i].type) { + case QTEST_PCI_BAR_IO: + if ((val & 0x1) != PCI_BASE_ADDRESS_SPACE_IO) + dev->bar[i].type = QTEST_PCI_BAR_DISABLE; + break; + case QTEST_PCI_BAR_MEMORY_UNDER_1MB: + if ((val & 0x1) != PCI_BASE_ADDRESS_SPACE_MEMORY) + dev->bar[i].type = QTEST_PCI_BAR_DISABLE; + if ((val & 0x6) != PCI_BASE_ADDRESS_MEM_TYPE_1M) + dev->bar[i].type = QTEST_PCI_BAR_DISABLE; + break; + case QTEST_PCI_BAR_MEMORY_32: + if ((val & 0x1) != PCI_BASE_ADDRESS_SPACE_MEMORY) + dev->bar[i].type = QTEST_PCI_BAR_DISABLE; + if ((val & 0x6) != PCI_BASE_ADDRESS_MEM_TYPE_32) + dev->bar[i].type = QTEST_PCI_BAR_DISABLE; + break; + case QTEST_PCI_BAR_MEMORY_64: + + if ((val & 0x1) != PCI_BASE_ADDRESS_SPACE_MEMORY) + dev->bar[i].type = QTEST_PCI_BAR_DISABLE; + if ((val & 0x6) != PCI_BASE_ADDRESS_MEM_TYPE_64) + dev->bar[i].type = QTEST_PCI_BAR_DISABLE; + break; + case QTEST_PCI_BAR_DISABLE: + break; + } + } + + /* Enable device */ + val = qtest_pci_inl(s, bus, device, 0, PCI_COMMAND); + val |= PCI_COMMAND_IO | PCI_COMMAND_MEMORY | PCI_COMMAND_MASTER; + qtest_pci_outl(s, bus, device, 0, PCI_COMMAND, val); + + /* Calculate BAR size */ + for (i = 0; i < NB_BAR; i++) { + switch (dev->bar[i].type) { + case QTEST_PCI_BAR_IO: + case QTEST_PCI_BAR_MEMORY_UNDER_1MB: + case QTEST_PCI_BAR_MEMORY_32: + qtest_pci_outl(s, bus, device, 0, + dev->bar[i].addr, 0xffffffff); + val = qtest_pci_inl(s, bus, device, + 0, dev->bar[i].addr); + dev->bar[i].region_size = ~(val & 0xfffffff0) + 1; + break; + case QTEST_PCI_BAR_MEMORY_64: + qtest_pci_outq(s, bus, device, 0, + dev->bar[i].addr, 0xffffffffffffffff); + val64 = qtest_pci_inq(s, bus, device, + 0, dev->bar[i].addr); + dev->bar[i].region_size = + ~(val64 & 0xfffffffffffffff0) + 1; + break; + case QTEST_PCI_BAR_DISABLE: + break; + } + } + + /* Set BAR region */ + for (i = 0; i < NB_BAR; i++) { + switch (dev->bar[i].type) { + case QTEST_PCI_BAR_IO: + case QTEST_PCI_BAR_MEMORY_UNDER_1MB: + case QTEST_PCI_BAR_MEMORY_32: + qtest_pci_outl(s, bus, device, 0, dev->bar[i].addr, + dev->bar[i].region_start); + PMD_DRV_LOG(INFO, "Set BAR of %s device: 0x%lx - 0x%lx\n", + dev->name, dev->bar[i].region_start, + dev->bar[i].region_start + dev->bar[i].region_size); + break; + case QTEST_PCI_BAR_MEMORY_64: + qtest_pci_outq(s, bus, device, 0, dev->bar[i].addr, + dev->bar[i].region_start); + PMD_DRV_LOG(INFO, "Set BAR of %s device: 0x%lx - 0x%lx\n", + dev->name, dev->bar[i].region_start, + dev->bar[i].region_start + dev->bar[i].region_size); + break; + case QTEST_PCI_BAR_DISABLE: + break; + } + } + + return 0; +} + +static int +qtest_find_pci_device(struct qtest_session *s, const char *name, + struct rte_pci_addr *addr) +{ + struct qtest_pci_device *dev; + uint32_t val; + + PMD_DRV_LOG(INFO, "PCI address of %s is %04x:%02x:%02x.%02x\n", name, + addr->domain, addr->bus, addr->devid, addr->function); + + dev = qtest_find_device(s, name); + if (dev == NULL) + goto error; + + val = qtest_pci_inl(s, addr->bus, addr->devid, addr->function, 0); + if (val == ((uint32_t)dev->device_id << 16 | dev->vendor_id)) { + dev->bus_addr = addr->bus; + dev->device_addr = addr->devid; + return 0; + } + +error: + PMD_DRV_LOG(ERR, "%s isn' found on %04x:%02x:%02x.%02x\n", name, + addr->domain, addr->bus, addr->devid, addr->function); + return -1; +} + +static int +qtest_init_pci_devices(struct qtest_session *s, + struct rte_pci_addr *virtio_addr, + struct rte_pci_addr *ivshmem_addr, + struct rte_pci_addr *piix3_addr) +{ + struct qtest_pci_device *dev; + int ret; + + + /* Try to find devices */ + ret = qtest_find_pci_device(s, "virtio-net", virtio_addr); + if (ret < 0) + return -1; + + ret = qtest_find_pci_device(s, "ivshmem", ivshmem_addr); + if (ret < 0) + return -1; + + ret = qtest_find_pci_device(s, "piix3", piix3_addr); + if (ret < 0) + return -1; + + /* Initialize devices */ + TAILQ_FOREACH(dev, &s->head, next) { + ret = dev->init(s, dev); + if (ret != 0) + return ret; + } + + return 0; +} + +struct rte_pci_id +qtest_get_pci_id_of_virtio_net(void) +{ + struct rte_pci_id id = {VIRTIO_NET_DEVICE_ID, + VIRTIO_NET_VENDOR_ID, PCI_ANY_ID, PCI_ANY_ID}; + + return id; +} + +static int +qtest_register_target_devices(struct qtest_session *s) +{ + struct qtest_pci_device *virtio_net, *ivshmem, *piix3; + const struct rte_memseg *ms; + + ms = rte_eal_get_physmem_layout(); + /* if EAL memory size isn't pow of 2, ivshmem will refuse it */ + if ((ms[0].len & (ms[0].len - 1)) != 0) { + PMD_DRV_LOG(ERR, "memory size must be power of 2\n"); + return -1; + } + + virtio_net = malloc(sizeof(*virtio_net)); + if (virtio_net == NULL) + return -1; + + ivshmem = malloc(sizeof(*ivshmem)); + if (ivshmem == NULL) + return -1; + + piix3 = malloc(sizeof(*piix3)); + if (piix3 == NULL) + return -1; + + memset(virtio_net, 0, sizeof(*virtio_net)); + memset(ivshmem, 0, sizeof(*ivshmem)); + + TAILQ_INIT(&s->head); + + virtio_net->name = "virtio-net"; + virtio_net->device_id = VIRTIO_NET_DEVICE_ID; + virtio_net->vendor_id = VIRTIO_NET_VENDOR_ID; + virtio_net->init = qtest_init_pci_device; + virtio_net->bar[0].addr = PCI_BASE_ADDRESS_0; + virtio_net->bar[0].type = QTEST_PCI_BAR_IO; + virtio_net->bar[0].region_start = VIRTIO_NET_IO_START; + virtio_net->bar[1].addr = PCI_BASE_ADDRESS_1; + virtio_net->bar[1].type = QTEST_PCI_BAR_MEMORY_32; + virtio_net->bar[1].region_start = VIRTIO_NET_MEMORY1_START; + virtio_net->bar[4].addr = PCI_BASE_ADDRESS_4; + virtio_net->bar[4].type = QTEST_PCI_BAR_MEMORY_64; + virtio_net->bar[4].region_start = VIRTIO_NET_MEMORY2_START; + TAILQ_INSERT_TAIL(&s->head, virtio_net, next); + + ivshmem->name = "ivshmem"; + ivshmem->device_id = IVSHMEM_DEVICE_ID; + ivshmem->vendor_id = IVSHMEM_VENDOR_ID; + ivshmem->init = qtest_init_pci_device; + ivshmem->bar[0].addr = PCI_BASE_ADDRESS_0; + ivshmem->bar[0].type = QTEST_PCI_BAR_MEMORY_32; + ivshmem->bar[0].region_start = IVSHMEM_MEMORY_START; + ivshmem->bar[2].addr = PCI_BASE_ADDRESS_2; + ivshmem->bar[2].type = QTEST_PCI_BAR_MEMORY_64; + /* In host mode, only one memory segment is vaild */ + ivshmem->bar[2].region_start = (uint64_t)ms[0].addr; + TAILQ_INSERT_TAIL(&s->head, ivshmem, next); + + /* piix3 is needed to route irqs from virtio-net to ioapic */ + piix3->name = "piix3"; + piix3->device_id = PIIX3_DEVICE_ID; + piix3->vendor_id = PIIX3_VENDOR_ID; + piix3->init = qtest_init_piix3_device; + TAILQ_INSERT_TAIL(&s->head, piix3, next); + + return 0; +} + +static int +qtest_send_message_to_ivshmem(int sock_fd, uint64_t client_id, int shm_fd) +{ + struct iovec iov; + struct msghdr msgh; + size_t fdsize = sizeof(int); + char control[CMSG_SPACE(fdsize)]; + struct cmsghdr *cmsg; + int ret; + + memset(&msgh, 0, sizeof(msgh)); + iov.iov_base = &client_id; + iov.iov_len = sizeof(client_id); + + msgh.msg_iov = &iov; + msgh.msg_iovlen = 1; + + if (shm_fd >= 0) { + msgh.msg_control = &control; + msgh.msg_controllen = sizeof(control); + cmsg = CMSG_FIRSTHDR(&msgh); + cmsg->cmsg_len = CMSG_LEN(fdsize); + cmsg->cmsg_level = SOL_SOCKET; + cmsg->cmsg_type = SCM_RIGHTS; + memcpy(CMSG_DATA(cmsg), &shm_fd, fdsize); + } + + do { + ret = sendmsg(sock_fd, &msgh, 0); + } while (ret < 0 && errno == EINTR); + + if (ret < 0) { + PMD_DRV_LOG(ERR, "sendmsg error\n"); + return ret; + } + + return ret; +} + +static int +qtest_setup_shared_memory(struct qtest_session *s) +{ + int shm_fd, num, ret; + struct back_file *huges; + + num = rte_eal_get_backfile_info(&huges); + if (num != 1) { + PMD_DRV_LOG(ERR, + "Not supported memory configuration\n"); + return -1; + } + + shm_fd = open(huges[0].filepath, O_RDWR); + if (shm_fd < 0) { + PMD_DRV_LOG(ERR, + "Cannot open file: %s\n", huges[0].filepath); + return -1; + } + + /* send our protocol version first */ + ret = qtest_send_message_to_ivshmem(s->ivshmem_socket, + IVSHMEM_PROTOCOL_VERSION, -1); + if (ret < 0) { + PMD_DRV_LOG(ERR, + "Failed to send protocol version to ivshmem\n"); + return -1; + } + + /* send client id */ + ret = qtest_send_message_to_ivshmem(s->ivshmem_socket, 0, -1); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to send VMID to ivshmem\n"); + return -1; + } + + /* send message to ivshmem */ + ret = qtest_send_message_to_ivshmem(s->ivshmem_socket, -1, shm_fd); + if (ret < 0) { + PMD_DRV_LOG(ERR, "Failed to file descriptor to ivshmem\n"); + return -1; + } + + close(shm_fd); + + return 0; +} + +int +qtest_vdev_init(struct rte_eth_dev_data *data, + int qtest_socket, int ivshmem_socket, + struct rte_pci_addr *virtio_addr, + struct rte_pci_addr *ivshmem_addr, + struct rte_pci_addr *piix3_addr) +{ + struct virtio_hw *hw = ((struct rte_eth_dev_data *)data)->dev_private; + struct qtest_session *s; + int ret; + + s = rte_zmalloc(NULL, sizeof(*s), RTE_CACHE_LINE_SIZE); + + ret = pipe(s->msgfds.pipefd); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to initialize message pipe\n"); + return -1; + } + + s->eventfd = eventfd(0, 0); + if (s->eventfd < 0) { + PMD_DRV_LOG(ERR, "Failed to open eventfd\n"); + return -1; + } + + ret = qtest_register_target_devices(s); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to initialize qtest session\n"); + return -1; + } + + ret = pthread_mutex_init(&s->qtest_session_lock, NULL); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to initialize mutex\n"); + return -1; + } + + rte_atomic16_set(&s->enable_intr, 0); + s->qtest_socket = qtest_socket; + s->ivshmem_socket = ivshmem_socket; + s->eth_data = data; + hw->qsession = (void *)s; + + ret = pthread_create(&s->event_th, NULL, qtest_event_handler, s); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to create event handler\n"); + return -1; + } + + ret = pthread_create(&s->intr_th, NULL, qtest_intr_handler, s); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to create interrupt handler\n"); + return -1; + } + + ret = qtest_intr_initialize(data); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to initialize interrupt\n"); + return -1; + } + + ret = qtest_setup_shared_memory(s); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to setup shared memory\n"); + return -1; + } + + ret = qtest_init_pci_devices(s, virtio_addr, ivshmem_addr, piix3_addr); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Failed to initialize devices\n"); + return -1; + } + + return 0; +} + +static void +qtest_remove_target_devices(struct qtest_session *s) +{ + struct qtest_pci_device *dev, *next; + + for (dev = TAILQ_FIRST(&s->head); dev != NULL; dev = next) { + next = TAILQ_NEXT(dev, next); + TAILQ_REMOVE(&s->head, dev, next); + free(dev); + } +} + +void +qtest_vdev_uninit(struct rte_eth_dev_data *data) +{ + struct virtio_hw *hw = ((struct rte_eth_dev_data *)data)->dev_private; + struct qtest_session *s; + + s = (struct qtest_session *)hw->qsession; + + qtest_close_sockets(s); + qtest_event_flush(s); + + pthread_cancel(s->event_th); + pthread_join(s->event_th, NULL); + + pthread_cancel(s->intr_th); + pthread_join(s->intr_th, NULL); + + pthread_mutex_destroy(&s->qtest_session_lock); + + qtest_remove_target_devices(s); + + rte_free(s); + hw->qsession = NULL; +} diff --git a/drivers/net/virtio/qtest.h b/drivers/net/virtio/qtest.h new file mode 100644 index 0000000..965e985 --- /dev/null +++ b/drivers/net/virtio/qtest.h @@ -0,0 +1,65 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 IGEL Co., Ltd. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of IGEL Co., Ltd. nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _VIRTIO_QTEST_H_ +#define _VIRTIO_QTEST_H_ + +#include "virtio_pci.h" + +#define QTEST_DRV_NAME "eth_qtest_virtio" + +int qtest_vdev_init(struct rte_eth_dev_data *data, int qtest_socket, + int ivshmem_socket, struct rte_pci_addr *virtio_addr, + struct rte_pci_addr *ivshmem_addr, + struct rte_pci_addr *piix3_addr); +void qtest_vdev_uninit(struct rte_eth_dev_data *data); +void qtest_intr_callback_register(void *data, + rte_intr_callback_fn cb, void *cb_arg); +void qtest_intr_callback_unregister(void *data, + rte_intr_callback_fn cb, void *cb_arg); +int qtest_intr_enable(void *data); +int qtest_intr_disable(void *data); +struct rte_pci_id qtest_get_pci_id_of_virtio_net(void); + +uint32_t qtest_in(struct virtio_hw *, uint16_t, char type); +void qtest_out(struct virtio_hw *, uint16_t, uint64_t, char type); +uint32_t qtest_read(struct virtio_hw *, uint64_t, char type); +void qtest_write(struct virtio_hw *, uint64_t, uint64_t, char type); +int qtest_read_pci_cfg(struct virtio_hw *hw, const char *name, + void *buf, size_t len, off_t offset); +int qtest_get_bar_addr(struct virtio_hw *hw, const char *name, + uint8_t bar, uint64_t **addr); +int qtest_get_bar_size(struct virtio_hw *hw, const char *name, + uint8_t bar, uint64_t *size); + +#endif /* _VIRTIO_QTEST_H_ */ diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index c3e877a..4d3df85 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -36,6 +36,10 @@ #include #include #include +#ifdef RTE_VIRTIO_VDEV_QTEST +#include +#include +#endif #include #include @@ -60,6 +64,9 @@ #include "virtqueue.h" #include "virtio_rxtx.h" +#ifdef RTE_VIRTIO_VDEV_QTEST +#include "qtest.h" +#endif static int eth_virtio_dev_init(struct rte_eth_dev *eth_dev); static int eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev); @@ -387,7 +394,7 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev, return -ENOMEM; } } -#ifdef RTE_VIRTIO_VDEV +#if defined(RTE_VIRTIO_VDEV) || defined(RTE_VIRTIO_VDEV_QTEST) else vq->vq_ring_mem = (phys_addr_t)mz->addr; /* Use vaddr!!! */ #endif @@ -431,7 +438,7 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev, if (virtio_dev_check(dev, RTE_ETH_DEV_PCI, NULL, 0)) vq->virtio_net_hdr_mem = mz->phys_addr; -#ifdef RTE_VIRTIO_VDEV +#if defined(RTE_VIRTIO_VDEV) || defined(RTE_VIRTIO_VDEV_QTEST) else vq->virtio_net_hdr_mem = (phys_addr_t)mz->addr; #endif @@ -441,7 +448,7 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev, if (virtio_dev_check(dev, RTE_ETH_DEV_PCI, NULL, 0)) vq->offset = offsetof(struct rte_mbuf, buf_physaddr); -#ifdef RTE_VIRTIO_VDEV +#if defined(RTE_VIRTIO_VDEV) || defined(RTE_VIRTIO_VDEV_QTEST) else vq->offset = offsetof(struct rte_mbuf, buf_addr); #endif @@ -999,6 +1006,23 @@ virtio_interrupt_handler(__rte_unused struct rte_intr_handle *handle, isr = vtpci_isr(hw); PMD_DRV_LOG(INFO, "interrupt status = %#x", isr); +#ifdef RTE_VIRTIO_VDEV_QTEST + if (virtio_dev_check(dev, RTE_ETH_DEV_VIRTUAL, QTEST_DRV_NAME, 0)) { + if (qtest_intr_enable(dev->data) < 0) + PMD_DRV_LOG(ERR, "interrupt enable failed"); + /* + * If last qtest message is interrupt, 'isr' will be 0 + * becasue socket has been closed already. + * But still we want to notice this event to EAL. + * So just ignore isr value. + */ + if (virtio_dev_link_update(dev, 0) == 0) + _rte_eth_dev_callback_process(dev, + RTE_ETH_EVENT_INTR_LSC); + return; + } +#endif + if (virtio_dev_check(dev, RTE_ETH_DEV_PCI, NULL, 0)) if (rte_intr_enable(&dev->pci_dev->intr_handle) < 0) PMD_DRV_LOG(ERR, "interrupt enable failed"); @@ -1055,9 +1079,16 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev) pci_dev = eth_dev->pci_dev; if (virtio_dev_check(eth_dev, RTE_ETH_DEV_PCI, NULL, 0)) { - if (vtpci_init(pci_dev, hw) < 0) + if (vtpci_init(eth_dev, hw) < 0) return -1; } +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, + RTE_ETH_DEV_VIRTUAL, QTEST_DRV_NAME, 0)) { + if (vtpci_init(eth_dev, hw) < 0) + return -1; + } +#endif /* Reset the device although not necessary at startup */ vtpci_reset(hw); @@ -1077,6 +1108,13 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev) rte_eth_copy_pci_info(eth_dev, pci_dev); } +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, + RTE_ETH_DEV_VIRTUAL, QTEST_DRV_NAME, 0)) { + if (!vtpci_with_feature(hw, VIRTIO_NET_F_STATUS)) + pci_dev->driver->drv_flags &= ~RTE_PCI_DRV_INTR_LSC; + } +#endif rx_func_get(eth_dev); @@ -1165,6 +1203,26 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev) virtio_interrupt_handler, eth_dev); } +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, RTE_ETH_DEV_VIRTUAL, + QTEST_DRV_NAME, 0)) { + struct rte_pci_id id; + + id = qtest_get_pci_id_of_virtio_net(); + RTE_SET_USED(id); + + PMD_INIT_LOG(DEBUG, "port %d vendorID=0x%x deviceID=0x%x", + eth_dev->data->port_id, + id.vendor_id, id.device_id); + + /* Setup interrupt callback */ + if (virtio_dev_check(eth_dev, RTE_ETH_DEV_VIRTUAL, + NULL, RTE_ETH_DEV_INTR_LSC)) + qtest_intr_callback_register(eth_dev->data, + virtio_interrupt_handler, eth_dev); + } +#endif + virtio_dev_cq_start(eth_dev); return 0; @@ -1202,7 +1260,15 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev) virtio_interrupt_handler, eth_dev); - rte_eal_pci_unmap_device(pci_dev); +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, RTE_ETH_DEV_VIRTUAL, + QTEST_DRV_NAME, RTE_ETH_DEV_INTR_LSC)) + qtest_intr_callback_unregister(eth_dev->data, + virtio_interrupt_handler, eth_dev); +#endif + + if (virtio_dev_check(eth_dev, RTE_ETH_DEV_PCI, NULL, 0)) + rte_eal_pci_unmap_device(pci_dev); PMD_INIT_LOG(DEBUG, "dev_uninit completed"); @@ -1284,16 +1350,34 @@ virtio_dev_start(struct rte_eth_dev *dev) /* check if lsc interrupt feature is enabled */ if (dev->data->dev_conf.intr_conf.lsc) { - if (!virtio_dev_check(dev, RTE_ETH_DEV_PCI, - NULL, RTE_PCI_DRV_INTR_LSC)) { + int pdev_has_lsc = 0, vdev_has_lsc = 0; + + pdev_has_lsc = virtio_dev_check(dev, RTE_ETH_DEV_PCI, + NULL, RTE_PCI_DRV_INTR_LSC); +#ifdef RTE_VIRTIO_VDEV_QTEST + vdev_has_lsc = virtio_dev_check(dev, RTE_ETH_DEV_VIRTUAL, + QTEST_DRV_NAME, RTE_ETH_DEV_INTR_LSC); +#endif + + if ((!pdev_has_lsc) && (!vdev_has_lsc)) { PMD_DRV_LOG(ERR, "link status not supported by host"); return -ENOTSUP; } - if (rte_intr_enable(&dev->pci_dev->intr_handle) < 0) { - PMD_DRV_LOG(ERR, "interrupt enable failed"); - return -EIO; + if (pdev_has_lsc) { + if (rte_intr_enable(&dev->pci_dev->intr_handle) < 0) { + PMD_DRV_LOG(ERR, "interrupt enable failed"); + return -EIO; + } } +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (vdev_has_lsc) { + if (qtest_intr_enable(dev->data) < 0) { + PMD_DRV_LOG(ERR, "interrupt enable failed"); + return -EIO; + } + } +#endif } /* Initialize Link state */ @@ -1390,8 +1474,15 @@ virtio_dev_stop(struct rte_eth_dev *dev) PMD_INIT_LOG(DEBUG, "stop"); - if (dev->data->dev_conf.intr_conf.lsc) - rte_intr_disable(&dev->pci_dev->intr_handle); + if (dev->data->dev_conf.intr_conf.lsc) { + if (virtio_dev_check(dev, RTE_ETH_DEV_PCI, NULL, 0)) + rte_intr_disable(&dev->pci_dev->intr_handle); +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(dev, RTE_ETH_DEV_VIRTUAL, + QTEST_DRV_NAME, 0)) + qtest_intr_disable(dev->data); +#endif + } memset(&link, 0, sizeof(link)); virtio_dev_atomic_write_link_status(dev, &link); @@ -1628,3 +1719,271 @@ static struct rte_driver rte_cvio_driver = { PMD_REGISTER_DRIVER(rte_cvio_driver); #endif + +#ifdef RTE_VIRTIO_VDEV_QTEST + +#define ETH_VIRTIO_NET_ARG_QTEST_PATH "qtest" +#define ETH_VIRTIO_NET_ARG_IVSHMEM_PATH "ivshmem" +#define ETH_VIRTIO_NET_ARG_VIRTIO_NET_ADDR "virtio-net-addr" +#define ETH_VIRTIO_NET_ARG_IVSHMEM_ADDR "ivshmem-addr" +#define ETH_VIRTIO_NET_ARG_PIIX3_ADDR "piix3-addr" + +static const char *valid_qtest_args[] = { + ETH_VIRTIO_NET_ARG_QTEST_PATH, + ETH_VIRTIO_NET_ARG_IVSHMEM_PATH, + ETH_VIRTIO_NET_ARG_VIRTIO_NET_ADDR, + ETH_VIRTIO_NET_ARG_IVSHMEM_ADDR, + ETH_VIRTIO_NET_ARG_PIIX3_ADDR, + NULL +}; + +static int +get_socket_path_arg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + int ret, fd, loop = 100; + int *pfd = extra_args; + struct sockaddr_un sa = {0}; + + if ((value == NULL) || (extra_args == NULL)) + return -EINVAL; + + fd = socket(AF_UNIX, SOCK_STREAM, 0); + if (fd < 0) + return -1; + + sa.sun_family = AF_UNIX; + strncpy(sa.sun_path, value, sizeof(sa.sun_path)); + + while (loop--) { + /* + * may need to wait for qtest and ivshmem + * sockets are prepared by QEMU. + */ + ret = connect(fd, (struct sockaddr *)&sa, + sizeof(struct sockaddr_un)); + if (ret == 0) + break; + else + usleep(100000); + } + + if (ret != 0) { + close(fd); + return -1; + } + + *pfd = fd; + + return 0; +} + +static int +get_pci_addr_arg(const char *key __rte_unused, + const char *value, void *extra_args) +{ + struct rte_pci_addr *addr = extra_args; + + if ((value == NULL) || (extra_args == NULL)) + return -EINVAL; + + if (eal_parse_pci_DomBDF(value, addr) != 0) + return -1; + + if (addr->domain != 0) + return -1; + + return 0; +} + +static struct rte_eth_dev * +virtio_net_eth_dev_alloc(const char *name) +{ + struct rte_eth_dev *eth_dev; + struct rte_eth_dev_data *data; + struct virtio_hw *hw; + + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); + if (eth_dev == NULL) + rte_panic("cannot alloc rte_eth_dev\n"); + + data = eth_dev->data; + + hw = rte_zmalloc(NULL, sizeof(*hw), 0); + if (!hw) + rte_panic("malloc virtio_hw failed\n"); + + data->dev_private = hw; + eth_dev->driver = &rte_virtio_pmd; + return eth_dev; +} + +static int +virtio_net_eth_pmd_parse_socket_path(struct rte_kvargs *kvlist, + const char *option, int *socket) +{ + int ret; + + if (rte_kvargs_count(kvlist, option) == 1) { + ret = rte_kvargs_process(kvlist, option, + &get_socket_path_arg, socket); + if (ret != 0) { + PMD_INIT_LOG(ERR, + "Failed to connect to %s socket", option); + return -1; + } + } else { + PMD_INIT_LOG(ERR, "No argument specified for %s", option); + return -1; + } + + return 0; +} + +static int +virtio_net_eth_pmd_parse_pci_addr(struct rte_kvargs *kvlist, + const char *option, struct rte_pci_addr *addr, + struct rte_pci_addr *default_addr) +{ + int ret; + + if (rte_kvargs_count(kvlist, option) == 1) { + ret = rte_kvargs_process(kvlist, option, + &get_pci_addr_arg, addr); + if (ret != 0) { + PMD_INIT_LOG(ERR, + "Specified invalid address in '%s'", option); + return -1; + } + } else + *addr = *default_addr; + + return 0; +} + +/* + * Initialization when "CONFIG_RTE_VIRTIO_VDEV_QTEST" is enabled. + */ +static int +rte_qtest_virtio_pmd_init(const char *name, const char *params) +{ + struct rte_kvargs *kvlist = NULL; + struct rte_eth_dev *eth_dev = NULL; + int ret, qtest_sock = 0, ivshmem_sock = 0; + struct rte_pci_addr virtio_addr, ivshmem_addr, piix3_addr, default_addr; + + if (params == NULL || params[0] == '\0') + goto error; + + kvlist = rte_kvargs_parse(params, valid_qtest_args); + if (!kvlist) { + PMD_INIT_LOG(ERR, "error when parsing param"); + return -EFAULT; + } + + ret = virtio_net_eth_pmd_parse_socket_path(kvlist, + ETH_VIRTIO_NET_ARG_IVSHMEM_PATH, &ivshmem_sock); + if (ret < 0) + goto error; + + ret = virtio_net_eth_pmd_parse_socket_path(kvlist, + ETH_VIRTIO_NET_ARG_QTEST_PATH, &qtest_sock); + if (ret < 0) + goto error; + + default_addr.domain = 0; + default_addr.bus = 0; + default_addr.function = 0; + + default_addr.devid = 3; + ret = virtio_net_eth_pmd_parse_pci_addr(kvlist, + ETH_VIRTIO_NET_ARG_VIRTIO_NET_ADDR, + &virtio_addr, &default_addr); + if (ret < 0) + goto error; + + default_addr.devid = 4; + ret = virtio_net_eth_pmd_parse_pci_addr(kvlist, + ETH_VIRTIO_NET_ARG_IVSHMEM_ADDR, + &ivshmem_addr, &default_addr); + if (ret < 0) + goto error; + + default_addr.devid = 1; + ret = virtio_net_eth_pmd_parse_pci_addr(kvlist, + ETH_VIRTIO_NET_ARG_PIIX3_ADDR, + &piix3_addr, &default_addr); + if (ret < 0) + goto error; + + eth_dev = virtio_net_eth_dev_alloc(name); + if (eth_dev == NULL) + goto error; + + ret = qtest_vdev_init(eth_dev->data, qtest_sock, ivshmem_sock, + &virtio_addr, &ivshmem_addr, &piix3_addr); + if (ret < 0) + goto error; + + /* originally, this will be called in rte_eal_pci_probe() */ + ret = eth_virtio_dev_init(eth_dev); + if (ret < 0) + goto error; + + eth_dev->driver = NULL; + eth_dev->data->dev_flags |= RTE_ETH_DEV_DETACHABLE; + eth_dev->data->kdrv = RTE_KDRV_NONE; + eth_dev->data->drv_name = QTEST_DRV_NAME; + + rte_kvargs_free(kvlist); + return 0; + +error: + if (qtest_sock) + close (qtest_sock); + if (ivshmem_sock) + close (ivshmem_sock); + rte_kvargs_free(kvlist); + return -EFAULT; +} + +/* + * Finalization when "CONFIG_RTE_VIRTIO_VDEV_QTEST" is enabled. + */ +static int +rte_qtest_virtio_pmd_uninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + int ret; + + if (name == NULL) + return -EINVAL; + + /* find the ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -ENODEV; + + ret = eth_virtio_dev_uninit(eth_dev); + if (ret != 0) + return -EFAULT; + + qtest_vdev_uninit(eth_dev->data); + rte_free(eth_dev->data->dev_private); + + ret = rte_eth_dev_release_port(eth_dev); + if (ret != 0) + return -EFAULT; + + return 0; +} + +static struct rte_driver rte_qtest_virtio_driver = { + .name = QTEST_DRV_NAME, + .type = PMD_VDEV, + .init = rte_qtest_virtio_pmd_init, + .uninit = rte_qtest_virtio_pmd_uninit, +}; + +PMD_REGISTER_DRIVER(rte_qtest_virtio_driver); +#endif /* RTE_VIRTIO_VDEV_QTEST */ diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c index e04c0db..f4a2711 100644 --- a/drivers/net/virtio/virtio_pci.c +++ b/drivers/net/virtio/virtio_pci.c @@ -37,10 +37,15 @@ #include #endif +#include "virtio_ethdev.h" #include "virtio_pci.h" #include "virtio_logs.h" #include "virtqueue.h" +#ifdef RTE_VIRTIO_VDEV_QTEST +#include "qtest.h" +#endif + /* * Following macros are derived from linux/pci_regs.h, however, * we can't simply include that header here, as there is no such @@ -449,6 +454,220 @@ static const struct virtio_pci_ops modern_ops = { }; +#ifdef RTE_VIRTIO_VDEV_QTEST +static inline uint8_t +qtest_read8(struct virtio_hw *hw, uint8_t *addr) +{ + return qtest_read(hw, (uint64_t)addr, 'b'); +} + +static inline void +qtest_write8(struct virtio_hw *hw, uint8_t val, uint8_t *addr) +{ + return qtest_write(hw, (uint64_t)addr, val, 'b'); +} + +static inline uint16_t +qtest_read16(struct virtio_hw *hw, uint16_t *addr) +{ + return qtest_read(hw, (uint64_t)addr, 'w'); +} + +static inline void +qtest_write16(struct virtio_hw *hw, uint16_t val, uint16_t *addr) +{ + return qtest_write(hw, (uint64_t)addr, val, 'w'); +} + +static inline uint32_t +qtest_read32(struct virtio_hw *hw, uint32_t *addr) +{ + return qtest_read(hw, (uint64_t)addr, 'l'); +} + +static inline void +qtest_write32(struct virtio_hw *hw, uint32_t val, uint32_t *addr) +{ + return qtest_write(hw, (uint64_t)addr, val, 'l'); +} + +static inline void +qtest_write64_twopart(struct virtio_hw *hw, + uint64_t val, uint32_t *lo, uint32_t *hi) +{ + qtest_write32(hw, val & ((1ULL << 32) - 1), lo); + qtest_write32(hw, val >> 32, hi); +} + +static void +qtest_modern_read_dev_config(struct virtio_hw *hw, size_t offset, + void *dst, int length) +{ + int i; + uint8_t *p; + uint8_t old_gen, new_gen; + + do { + old_gen = qtest_read8(hw, &hw->common_cfg->config_generation); + + p = dst; + for (i = 0; i < length; i++) + *p++ = qtest_read8(hw, (uint8_t *)hw->dev_cfg + offset + i); + + new_gen = qtest_read8(hw, &hw->common_cfg->config_generation); + } while (old_gen != new_gen); +} + +static void +qtest_modern_write_dev_config(struct virtio_hw *hw, size_t offset, + const void *src, int length) +{ + int i; + const uint8_t *p = src; + + for (i = 0; i < length; i++) + qtest_write8(hw, *p++, (uint8_t *)hw->dev_cfg + offset + i); +} + +static uint64_t +qtest_modern_get_features(struct virtio_hw *hw) +{ + uint32_t features_lo, features_hi; + + qtest_write32(hw, 0, &hw->common_cfg->device_feature_select); + features_lo = qtest_read32(hw, &hw->common_cfg->device_feature); + + qtest_write32(hw, 1, &hw->common_cfg->device_feature_select); + features_hi = qtest_read32(hw, &hw->common_cfg->device_feature); + + return ((uint64_t)features_hi << 32) | features_lo; +} + +static void +qtest_modern_set_features(struct virtio_hw *hw, uint64_t features) +{ + qtest_write32(hw, 0, &hw->common_cfg->guest_feature_select); + qtest_write32(hw, features & ((1ULL << 32) - 1), + &hw->common_cfg->guest_feature); + + qtest_write32(hw, 1, &hw->common_cfg->guest_feature_select); + qtest_write32(hw, features >> 32, + &hw->common_cfg->guest_feature); +} + +static uint8_t +qtest_modern_get_status(struct virtio_hw *hw) +{ + return qtest_read8(hw, &hw->common_cfg->device_status); +} + +static void +qtest_modern_set_status(struct virtio_hw *hw, uint8_t status) +{ + qtest_write8(hw, status, &hw->common_cfg->device_status); +} + +static void +qtest_modern_reset(struct virtio_hw *hw) +{ + modern_set_status(hw, VIRTIO_CONFIG_STATUS_RESET); + modern_get_status(hw); +} + +static uint8_t +qtest_modern_get_isr(struct virtio_hw *hw) +{ + return qtest_read8(hw, hw->isr); +} + +static uint16_t +qtest_modern_set_config_irq(struct virtio_hw *hw, uint16_t vec) +{ + qtest_write16(hw, vec, &hw->common_cfg->msix_config); + return qtest_read16(hw, &hw->common_cfg->msix_config); +} + +static uint16_t +qtest_modern_get_queue_num(struct virtio_hw *hw, uint16_t queue_id) +{ + qtest_write16(hw, queue_id, &hw->common_cfg->queue_select); + return qtest_read16(hw, &hw->common_cfg->queue_size); +} + +static void +qtest_modern_setup_queue(struct virtio_hw *hw, struct virtqueue *vq) +{ + uint64_t desc_addr, avail_addr, used_addr; + uint16_t notify_off; + + desc_addr = (uint64_t)vq->mz->addr; + avail_addr = desc_addr + vq->vq_nentries * sizeof(struct vring_desc); + used_addr = RTE_ALIGN_CEIL(avail_addr + offsetof(struct vring_avail, + ring[vq->vq_nentries]), + VIRTIO_PCI_VRING_ALIGN); + + qtest_write16(hw, vq->vq_queue_index, &hw->common_cfg->queue_select); + + qtest_write64_twopart(hw, desc_addr, &hw->common_cfg->queue_desc_lo, + &hw->common_cfg->queue_desc_hi); + qtest_write64_twopart(hw, avail_addr, &hw->common_cfg->queue_avail_lo, + &hw->common_cfg->queue_avail_hi); + qtest_write64_twopart(hw, used_addr, &hw->common_cfg->queue_used_lo, + &hw->common_cfg->queue_used_hi); + + notify_off = qtest_read16(hw, &hw->common_cfg->queue_notify_off); + vq->notify_addr = (void *)((uint8_t *)hw->notify_base + + notify_off * hw->notify_off_multiplier); + + qtest_write16(hw, 1, &hw->common_cfg->queue_enable); + + PMD_INIT_LOG(DEBUG, "queue %u addresses:", vq->vq_queue_index); + PMD_INIT_LOG(DEBUG, "\t desc_addr: %" PRIx64, desc_addr); + PMD_INIT_LOG(DEBUG, "\t aval_addr: %" PRIx64, avail_addr); + PMD_INIT_LOG(DEBUG, "\t used_addr: %" PRIx64, used_addr); + PMD_INIT_LOG(DEBUG, "\t notify addr: %p (notify offset: %u)", + vq->notify_addr, notify_off); +} + +static void +qtest_modern_del_queue(struct virtio_hw *hw, struct virtqueue *vq) +{ + qtest_write16(hw, vq->vq_queue_index, &hw->common_cfg->queue_select); + + qtest_write64_twopart(hw, 0, &hw->common_cfg->queue_desc_lo, + &hw->common_cfg->queue_desc_hi); + qtest_write64_twopart(hw, 0, &hw->common_cfg->queue_avail_lo, + &hw->common_cfg->queue_avail_hi); + qtest_write64_twopart(hw, 0, &hw->common_cfg->queue_used_lo, + &hw->common_cfg->queue_used_hi); + + qtest_write16(hw, 0, &hw->common_cfg->queue_enable); +} + +static void +qtest_modern_notify_queue(struct virtio_hw *hw __rte_unused, struct virtqueue *vq) +{ + qtest_write16(hw, 1, vq->notify_addr); +} + +static const struct virtio_pci_ops qtest_modern_ops = { + .read_dev_cfg = qtest_modern_read_dev_config, + .write_dev_cfg = qtest_modern_write_dev_config, + .reset = qtest_modern_reset, + .get_status = qtest_modern_get_status, + .set_status = qtest_modern_set_status, + .get_features = qtest_modern_get_features, + .set_features = qtest_modern_set_features, + .get_isr = qtest_modern_get_isr, + .set_config_irq = qtest_modern_set_config_irq, + .get_queue_num = qtest_modern_get_queue_num, + .setup_queue = qtest_modern_setup_queue, + .del_queue = qtest_modern_del_queue, + .notify_queue = qtest_modern_notify_queue, +}; +#endif /* RTE_VIRTIO_VDEV_QTEST */ + + void vtpci_read_dev_config(struct virtio_hw *hw, size_t offset, void *dst, int length) @@ -522,12 +741,16 @@ vtpci_irq_config(struct virtio_hw *hw, uint16_t vec) } static void * -get_cfg_addr(struct rte_pci_device *dev, struct virtio_pci_cap *cap) +get_cfg_addr(struct rte_eth_dev *eth_dev, + struct virtio_hw *hw, + struct virtio_pci_cap *cap) { + struct rte_pci_device *pci_dev = eth_dev->pci_dev; uint8_t bar = cap->bar; uint32_t length = cap->length; uint32_t offset = cap->offset; - uint8_t *base; + uint8_t *base = NULL; + uint64_t size = 0; if (bar > 5) { PMD_INIT_LOG(ERR, "invalid bar: %u", bar); @@ -540,14 +763,27 @@ get_cfg_addr(struct rte_pci_device *dev, struct virtio_pci_cap *cap) return NULL; } - if (offset + length > dev->mem_resource[bar].len) { + if (virtio_dev_check(eth_dev, RTE_ETH_DEV_PCI, NULL, 0)) { + size = pci_dev->mem_resource[bar].len; + base = pci_dev->mem_resource[bar].addr; + } +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, RTE_ETH_DEV_VIRTUAL, + QTEST_DRV_NAME, 0)) { + qtest_get_bar_size(hw, "virtio-net", bar, &size); + qtest_get_bar_addr(hw, "virtio-net", bar, (uint64_t **)&base); + } +#else + RTE_SET_USED(hw); +#endif + + if (offset + length > size) { PMD_INIT_LOG(ERR, "invalid cap: overflows bar space: %u > %" PRIu64, - offset + length, dev->mem_resource[bar].len); + offset + length, size); return NULL; } - base = dev->mem_resource[bar].addr; if (base == NULL) { PMD_INIT_LOG(ERR, "bar %u base addr is NULL", bar); return NULL; @@ -557,25 +793,48 @@ get_cfg_addr(struct rte_pci_device *dev, struct virtio_pci_cap *cap) } static int -virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw) +virtio_read_pci_config(struct rte_eth_dev *eth_dev, + struct virtio_hw *hw, + void *buf, size_t len, off_t offset) { + struct rte_pci_device *pci_dev = eth_dev->pci_dev; + int ret = -1; + + if (virtio_dev_check(eth_dev, RTE_ETH_DEV_PCI, NULL, 0)) + ret = rte_eal_pci_read_config(pci_dev, buf, len, offset); +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, RTE_ETH_DEV_VIRTUAL, + QTEST_DRV_NAME, 0)) + ret = qtest_read_pci_cfg(hw, "virtio-net", buf, len, offset); +#else + RTE_SET_USED(hw); +#endif + + return ret; +} + +static int +virtio_read_caps(struct rte_eth_dev *eth_dev, struct virtio_hw *hw) +{ + struct rte_pci_device *pci_dev = eth_dev->pci_dev; uint8_t pos; struct virtio_pci_cap cap; int ret; - if (rte_eal_pci_map_device(dev)) { + if ((eth_dev->dev_type == RTE_ETH_DEV_PCI) && + (rte_eal_pci_map_device(pci_dev) < 0)) { PMD_INIT_LOG(DEBUG, "failed to map pci device!"); return -1; } - ret = rte_eal_pci_read_config(dev, &pos, 1, PCI_CAPABILITY_LIST); + ret = virtio_read_pci_config(eth_dev, hw, &pos, 1, PCI_CAPABILITY_LIST); if (ret < 0) { PMD_INIT_LOG(DEBUG, "failed to read pci capability list"); return -1; } while (pos) { - ret = rte_eal_pci_read_config(dev, &cap, sizeof(cap), pos); + ret = virtio_read_pci_config(eth_dev, hw, &cap, sizeof(cap), pos); if (ret < 0) { PMD_INIT_LOG(ERR, "failed to read pci cap at pos: %x", pos); @@ -595,18 +854,19 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw) switch (cap.cfg_type) { case VIRTIO_PCI_CAP_COMMON_CFG: - hw->common_cfg = get_cfg_addr(dev, &cap); + hw->common_cfg = get_cfg_addr(eth_dev, hw, &cap); break; case VIRTIO_PCI_CAP_NOTIFY_CFG: - rte_eal_pci_read_config(dev, &hw->notify_off_multiplier, + virtio_read_pci_config(eth_dev, hw, + &hw->notify_off_multiplier, 4, pos + sizeof(cap)); - hw->notify_base = get_cfg_addr(dev, &cap); + hw->notify_base = get_cfg_addr(eth_dev, hw, &cap); break; case VIRTIO_PCI_CAP_DEVICE_CFG: - hw->dev_cfg = get_cfg_addr(dev, &cap); + hw->dev_cfg = get_cfg_addr(eth_dev, hw, &cap); break; case VIRTIO_PCI_CAP_ISR_CFG: - hw->isr = get_cfg_addr(dev, &cap); + hw->isr = get_cfg_addr(eth_dev, hw, &cap); break; } @@ -631,31 +891,77 @@ next: return 0; } +static int +vtpci_modern_init(struct rte_eth_dev *eth_dev, struct virtio_hw *hw) +{ + struct rte_pci_device *pci_dev = eth_dev->pci_dev; + + PMD_INIT_LOG(INFO, "modern virtio pci detected."); + + if (virtio_dev_check(eth_dev, RTE_ETH_DEV_PCI, NULL, 0)) { + hw->vtpci_ops = &modern_ops; + pci_dev->driver->drv_flags |= RTE_PCI_DRV_INTR_LSC; + } +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, RTE_ETH_DEV_VIRTUAL, NULL, 0)) { + hw->vtpci_ops = &qtest_modern_ops; + eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC; + } +#endif + + hw->modern = 1; + + return 0; +} + +static int +vtpci_legacy_init(struct rte_eth_dev *eth_dev, struct virtio_hw *hw) +{ + struct rte_pci_device *pci_dev = eth_dev->pci_dev; + struct virtio_pci_cap cap; + + PMD_INIT_LOG(INFO, "trying with legacy virtio pci."); + if (virtio_dev_check(eth_dev, RTE_ETH_DEV_PCI, NULL, 0)) { + if (legacy_virtio_resource_init(pci_dev, hw) < 0) + return -1; + + hw->vtpci_ops = &legacy_ops; + hw->use_msix = legacy_virtio_has_msix(&pci_dev->addr); + } +#ifdef RTE_VIRTIO_VDEV_QTEST + else if (virtio_dev_check(eth_dev, RTE_ETH_DEV_VIRTUAL, + QTEST_DRV_NAME, 0)) { + PMD_INIT_LOG(ERR, "Legacy virtio device isn't supported."); + return -1; + } +#endif + + cap.bar = cap.length = cap.offset = 0; + hw->modern = 0; + + return 0; +} + int -vtpci_init(struct rte_pci_device *dev, struct virtio_hw *hw) +vtpci_init(struct rte_eth_dev *eth_dev, struct virtio_hw *hw) { - hw->dev = dev; + struct rte_pci_device *pci_dev = eth_dev->pci_dev; + int ret; + + hw->dev = pci_dev; /* * Try if we can succeed reading virtio pci caps, which exists * only on modern pci device. If failed, we fallback to legacy * virtio handling. */ - if (virtio_read_caps(dev, hw) == 0) { - PMD_INIT_LOG(INFO, "modern virtio pci detected."); - hw->vtpci_ops = &modern_ops; - hw->modern = 1; - dev->driver->drv_flags |= RTE_PCI_DRV_INTR_LSC; - return 0; - } + if (virtio_read_caps(eth_dev, hw) == 0) + ret = vtpci_modern_init(eth_dev, hw); + else + ret = vtpci_legacy_init(eth_dev, hw); - PMD_INIT_LOG(INFO, "trying with legacy virtio pci."); - if (legacy_virtio_resource_init(dev, hw) < 0) + if (ret < 0) return -1; - hw->vtpci_ops = &legacy_ops; - hw->use_msix = legacy_virtio_has_msix(&dev->addr); - hw->modern = 0; - return 0; } diff --git a/drivers/net/virtio/virtio_pci.h b/drivers/net/virtio/virtio_pci.h index ae6777d..41268a7 100644 --- a/drivers/net/virtio/virtio_pci.h +++ b/drivers/net/virtio/virtio_pci.h @@ -242,6 +242,9 @@ struct virtio_net_config; struct virtio_hw { struct virtqueue *cvq; +#ifdef RTE_VIRTIO_VDEV_QTEST + void *qsession; +#endif struct rte_pci_ioport io; uint64_t guest_features; uint32_t max_tx_queues; @@ -306,7 +309,7 @@ vtpci_with_feature(struct virtio_hw *hw, uint64_t bit) /* * Function declaration from virtio_pci.c */ -int vtpci_init(struct rte_pci_device *, struct virtio_hw *); +int vtpci_init(struct rte_eth_dev *, struct virtio_hw *); void vtpci_reset(struct virtio_hw *); void vtpci_reinit_complete(struct virtio_hw *); -- 2.1.4