From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by dpdk.org (Postfix) with ESMTP id ED8177CD8 for ; Fri, 2 Feb 2018 12:33:47 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 02 Feb 2018 03:33:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,448,1511856000"; d="scan'208";a="171132737" Received: from unknown (HELO dpdk98.sh.intel.com) ([10.67.111.140]) by orsmga004.jf.intel.com with ESMTP; 02 Feb 2018 03:33:45 -0800 From: Zhihong Wang To: dev@dpdk.org Cc: jianfeng.tan@intel.com, tiwei.bie@intel.com, maxime.coquelin@redhat.com, yliu@fridaylinux.org, cunming.liang@intel.com, xiao.w.wang@intel.com, dan.daly@intel.com, Zhihong Wang Date: Fri, 2 Feb 2018 18:28:53 -0500 Message-Id: <1517614137-62926-4-git-send-email-zhihong.wang@intel.com> X-Mailer: git-send-email 2.7.5 In-Reply-To: <1517614137-62926-1-git-send-email-zhihong.wang@intel.com> References: <1517614137-62926-1-git-send-email-zhihong.wang@intel.com> Subject: [dpdk-dev] [PATCH 3/7] vhost: support selective datapath X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Feb 2018 11:33:48 -0000 This patch introduces support for selective datapath in DPDK vhost-user lib to enable various types of virtio-compatible devices to do data transfer with virtio driver directly to enable acceleration. The default datapath is the existing software implementation, more options will be available when new engines are registered. An engine is a group of virtio-compatible devices under a single address. The engine driver includes: 1. A set of engine ops is defined in rte_vdpa_eng_ops to perform engine init, uninit, and attributes reporting. 2. A set of device ops is defined in rte_vdpa_dev_ops for virtio devices in the engine to do device specific operations: a. dev_conf: Called to configure the actual device when the virtio device becomes ready. b. dev_close: Called to close the actual device when the virtio device is stopped. c. vring_state_set: Called to change the state of the vring in the actual device when vring state changes. d. feature_set: Called to set the negotiated features to device. e. migration_done: Called to allow the device to response to RARP sending. Signed-off-by: Zhihong Wang --- lib/librte_vhost/Makefile | 4 +- lib/librte_vhost/rte_vdpa.h | 113 +++++++++++++++++++++++++++++++++++++++ lib/librte_vhost/vdpa.c | 125 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 240 insertions(+), 2 deletions(-) create mode 100644 lib/librte_vhost/rte_vdpa.h create mode 100644 lib/librte_vhost/vdpa.c diff --git a/lib/librte_vhost/Makefile b/lib/librte_vhost/Makefile index 5d6c6ab..37044ac 100644 --- a/lib/librte_vhost/Makefile +++ b/lib/librte_vhost/Makefile @@ -22,9 +22,9 @@ LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev -lrte_net # all source are stored in SRCS-y SRCS-$(CONFIG_RTE_LIBRTE_VHOST) := fd_man.c iotlb.c socket.c vhost.c \ - vhost_user.c virtio_net.c + vhost_user.c virtio_net.c vdpa.c # install includes -SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h +SYMLINK-$(CONFIG_RTE_LIBRTE_VHOST)-include += rte_vhost.h rte_vdpa.h include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_vhost/rte_vdpa.h b/lib/librte_vhost/rte_vdpa.h new file mode 100644 index 0000000..729849b --- /dev/null +++ b/lib/librte_vhost/rte_vdpa.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +#ifndef _RTE_VDPA_H_ +#define _RTE_VDPA_H_ + +/** + * @file + * + * Device specific vhost lib + */ + +#include +#include +#include "rte_vhost.h" + +#define MAX_VDPA_ENGINE_NUM 128 +#define MAX_VDPA_NAME_LEN 128 + + +struct rte_vdpa_eng_addr { + union { + uint8_t __dummy[64]; + + struct { + struct rte_pci_addr pci_addr; + }; + }; +}; + +struct rte_vdpa_eng_info { + char name[MAX_VDPA_NAME_LEN]; + struct rte_vdpa_eng_addr *addr; +}; + +struct rte_vdpa_eng_attr { + uint64_t features; + uint64_t protocol_features; + uint32_t queue_num; + uint32_t dev_num; +}; + +/* register/remove engine */ +typedef int (*vdpa_eng_init_t)(int eid, struct rte_vdpa_eng_addr *addr); +typedef int (*vdpa_eng_uninit_t)(int eid); + +/* query info of this engine */ +typedef int (*vdpa_info_query_t)(int eid, + struct rte_vdpa_eng_attr *attr); + +/* driver configure/close the port based on connection */ +typedef int (*vdpa_dev_conf_t)(int vid); +typedef int (*vdpa_dev_close_t)(int vid); + +/* enable/disable this vring */ +typedef int (*vdpa_vring_state_set_t)(int vid, int vring, int state); + +/* set features when changed */ +typedef int (*vdpa_feature_set_t)(int vid); + +/* destination operations when migration done, e.g. send rarp */ +typedef int (*vdpa_migration_done_t)(int vid); + +/* device ops */ +struct rte_vdpa_dev_ops { + vdpa_dev_conf_t dev_conf; + vdpa_dev_close_t dev_close; + vdpa_vring_state_set_t vring_state_set; + vdpa_feature_set_t feature_set; + vdpa_migration_done_t migration_done; +}; + +/* engine ops */ +struct rte_vdpa_eng_ops { + vdpa_eng_init_t eng_init; + vdpa_eng_uninit_t eng_uninit; + vdpa_info_query_t info_query; +}; + +struct rte_vdpa_eng_driver { + const char *name; + struct rte_vdpa_eng_ops eng_ops; + struct rte_vdpa_dev_ops dev_ops; +} __rte_cache_aligned; + +struct rte_vdpa_engine { + struct rte_vdpa_eng_info eng_info; + struct rte_vdpa_eng_driver *eng_drv; +} __rte_cache_aligned; + +extern struct rte_vdpa_engine *vdpa_engines[]; +extern uint32_t vdpa_engine_num; + +/* engine management */ +int rte_vdpa_register_engine(const char *name, struct rte_vdpa_eng_addr *addr); +int rte_vdpa_unregister_engine(int eid); + +int rte_vdpa_find_engine_id(struct rte_vdpa_eng_addr *addr); + +int rte_vdpa_info_query(int eid, struct rte_vdpa_eng_attr *attr); + +/* driver register api */ +void rte_vdpa_register_driver(struct rte_vdpa_eng_driver *drv); + +#define RTE_VDPA_REGISTER_DRIVER(nm, drv) \ +RTE_INIT(vdpainitfn_ ##nm); \ +static void vdpainitfn_ ##nm(void) \ +{ \ + rte_vdpa_register_driver(&drv); \ +} \ + +#endif /* _RTE_VDPA_H_ */ diff --git a/lib/librte_vhost/vdpa.c b/lib/librte_vhost/vdpa.c new file mode 100644 index 0000000..1fc2d0b --- /dev/null +++ b/lib/librte_vhost/vdpa.c @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2018 Intel Corporation + */ + +/** + * @file + * + * Device specific vhost lib + */ + +#include +#include "rte_vdpa.h" +#include "vhost.h" + +static struct rte_vdpa_eng_driver *vdpa_drivers[MAX_VDPA_ENGINE_NUM]; +static uint32_t vdpa_driver_num; + +struct rte_vdpa_engine *vdpa_engines[MAX_VDPA_ENGINE_NUM]; +uint32_t vdpa_engine_num; + +static int is_same_eng(struct rte_vdpa_eng_addr *a, struct rte_vdpa_eng_addr *b) +{ + if (a->pci_addr.domain != b->pci_addr.domain + || a->pci_addr.bus != b->pci_addr.bus + || a->pci_addr.devid != b->pci_addr.devid + || a->pci_addr.function != b->pci_addr.function) + return -1; + + return 0; +} + +void rte_vdpa_register_driver(struct rte_vdpa_eng_driver *driver) +{ + if (vdpa_driver_num >= MAX_VDPA_ENGINE_NUM) + return; + + vdpa_drivers[vdpa_driver_num] = driver; + vdpa_driver_num++; +} + +int rte_vdpa_register_engine(const char *name, struct rte_vdpa_eng_addr *addr) +{ + static int engine_idx; + + struct rte_vdpa_engine *eng; + struct rte_vdpa_eng_driver *cur; + char engine_name[MAX_VDPA_NAME_LEN]; + int i; + + for (i = 0; i < MAX_VDPA_ENGINE_NUM; ++i) { + eng = vdpa_engines[i]; + if (eng && is_same_eng(eng->eng_info.addr, addr) == 0) + return i; + } + + sprintf(engine_name, "vdpa-%s-%d", name, engine_idx++); + eng = rte_zmalloc(engine_name, sizeof(struct rte_vdpa_engine), + RTE_CACHE_LINE_SIZE); + if (!eng) + return -1; + + for (i = 0; i < MAX_VDPA_ENGINE_NUM; ++i) { + cur = vdpa_drivers[i]; + if (cur && 0 == strncmp(name, cur->name, + MAX_VDPA_NAME_LEN)) { + eng->eng_drv = cur; + strcpy(eng->eng_info.name, name); + eng->eng_info.addr = addr; + for (i = 0; i < MAX_VDPA_ENGINE_NUM; ++i) { + if (vdpa_engines[i]) + continue; + vdpa_engines[i] = eng; + if (eng->eng_drv->eng_ops.eng_init) + eng->eng_drv->eng_ops.eng_init(i, addr); + vdpa_engine_num++; + return i; + } + } + } + + return -1; +} + +int rte_vdpa_unregister_engine(int eid) +{ + if (eid < 0 || eid >= MAX_VDPA_ENGINE_NUM || vdpa_engines[eid] + == NULL) + return -1; + + if (vdpa_engines[eid]->eng_drv->eng_ops.eng_uninit) + vdpa_engines[eid]->eng_drv->eng_ops.eng_uninit(eid); + + rte_free(vdpa_engines[eid]); + vdpa_engines[eid] = NULL; + vdpa_engine_num--; + + return eid; +} + +int rte_vdpa_find_engine_id(struct rte_vdpa_eng_addr *addr) +{ + struct rte_vdpa_engine *eng; + int i; + + for (i = 0; i < MAX_VDPA_ENGINE_NUM; ++i) { + eng = vdpa_engines[i]; + if (eng && is_same_eng(eng->eng_info.addr, addr) == 0) + return i; + } + + return -1; +} + +int rte_vdpa_info_query(int eid, struct rte_vdpa_eng_attr *attr) +{ + if (eid < 0 || eid >= MAX_VDPA_ENGINE_NUM || vdpa_engines[eid] + == NULL) + return -1; + + if (vdpa_engines[eid]->eng_drv->eng_ops.info_query == NULL) + return -1; + + return vdpa_engines[eid]->eng_drv->eng_ops.info_query(eid, attr); + +} -- 2.7.5