From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C0AB1A0A02; Thu, 14 Jan 2021 07:19:35 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5F32F140D5F; Thu, 14 Jan 2021 07:19:33 +0100 (CET) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mails.dpdk.org (Postfix) with ESMTP id C139C140D5C for ; Thu, 14 Jan 2021 07:19:31 +0100 (CET) IronPort-SDR: 5PHYBBWoRSRujyb2ucBq8AkIYToDvzWuEs2HurabaaauyYzkw6MsK2N5yt6lwFjfr6jDIXWKZ7 ZPgy6T+WwDMw== X-IronPort-AV: E=McAfee;i="6000,8403,9863"; a="175735621" X-IronPort-AV: E=Sophos;i="5.79,346,1602572400"; d="scan'208";a="175735621" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jan 2021 22:19:31 -0800 IronPort-SDR: Z+SQGIfyQt9laUVvxkh/4UqtvRDqTVaf8S9bwNrZah79djzL8Sd3eQY+Fh0N2jW+6auvfbwZNK UdUmWDLMGQ4w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,346,1602572400"; d="scan'208";a="349068948" Received: from npg-dpdk-virtio-xiachenbo-nw.sh.intel.com ([10.67.119.123]) by orsmga003.jf.intel.com with ESMTP; 13 Jan 2021 22:19:28 -0800 From: Chenbo Xia To: dev@dpdk.org, thomas@monjalon.net, david.marchand@redhat.com Cc: stephen@networkplumber.org, cunming.liang@intel.com, xiuchun.lu@intel.com, miao.li@intel.com, jingjing.wu@intel.com, beilei.xing@intel.com Date: Thu, 14 Jan 2021 14:14:03 +0800 Message-Id: <20210114061411.39166-2-chenbo.xia@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210114061411.39166-1-chenbo.xia@intel.com> References: <20201218073851.93609-1-chenbo.xia@intel.com> <20210114061411.39166-1-chenbo.xia@intel.com> Subject: [dpdk-dev] [PATCH v2 1/9] lib: introduce vfio-user library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch introduces vfio-user library, which follows vfio-user protocol v1.0. As vfio-user has server and client implementation, this patch introduces basic structures and internal functions that will be used by both server and client. Signed-off-by: Chenbo Xia Signed-off-by: Xiuchun Lu --- MAINTAINERS | 4 + lib/librte_vfio_user/meson.build | 9 ++ lib/librte_vfio_user/version.map | 3 + lib/librte_vfio_user/vfio_user_base.c | 211 ++++++++++++++++++++++++++ lib/librte_vfio_user/vfio_user_base.h | 65 ++++++++ lib/meson.build | 1 + 6 files changed, 293 insertions(+) create mode 100644 lib/librte_vfio_user/meson.build create mode 100644 lib/librte_vfio_user/version.map create mode 100644 lib/librte_vfio_user/vfio_user_base.c create mode 100644 lib/librte_vfio_user/vfio_user_base.h diff --git a/MAINTAINERS b/MAINTAINERS index 6787b15dcc..91b8b2ccc1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1541,6 +1541,10 @@ M: Nithin Dabilpuram M: Pavan Nikhilesh F: lib/librte_node/ +Vfio-user - EXPERIMENTAL +M: Chenbo Xia +M: Xiuchun Lu +F: lib/librte_vfio_user/ Test Applications ----------------- diff --git a/lib/librte_vfio_user/meson.build b/lib/librte_vfio_user/meson.build new file mode 100644 index 0000000000..0f6407b80f --- /dev/null +++ b/lib/librte_vfio_user/meson.build @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +if not is_linux + build = false + reason = 'only supported on Linux' +endif + +sources = files('vfio_user_base.c') diff --git a/lib/librte_vfio_user/version.map b/lib/librte_vfio_user/version.map new file mode 100644 index 0000000000..33c1b976f1 --- /dev/null +++ b/lib/librte_vfio_user/version.map @@ -0,0 +1,3 @@ +EXPERIMENTAL { + local: *; +}; diff --git a/lib/librte_vfio_user/vfio_user_base.c b/lib/librte_vfio_user/vfio_user_base.c new file mode 100644 index 0000000000..b9fdff5b02 --- /dev/null +++ b/lib/librte_vfio_user/vfio_user_base.c @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#include +#include +#include + +#include "vfio_user_base.h" + +int vfio_user_log_level; + +const char *vfio_user_msg_str[VFIO_USER_MAX] = { + [VFIO_USER_NONE] = "VFIO_USER_NONE", + [VFIO_USER_VERSION] = "VFIO_USER_VERSION", +}; + +void +vfio_user_close_msg_fds(struct vfio_user_msg *msg) +{ + int i; + + for (i = 0; i < msg->fd_num; i++) + close(msg->fds[i]); +} + +int +vfio_user_check_msg_fdnum(struct vfio_user_msg *msg, int expected_fds) +{ + if (msg->fd_num == expected_fds) + return 0; + + VFIO_USER_LOG(ERR, "Expect %d FDs for request %s, received %d\n", + expected_fds, vfio_user_msg_str[msg->cmd], msg->fd_num); + + vfio_user_close_msg_fds(msg); + + return -1; +} + +static int +vfio_user_recv_fd_msg(int sockfd, char *buf, int buflen, int *fds, + int max_fds, int *fd_num) +{ + struct iovec iov; + struct msghdr msgh; + char control[CMSG_SPACE(max_fds * sizeof(int))]; + struct cmsghdr *cmsg; + int fd_sz, got_fds = 0; + int ret, i; + + *fd_num = 0; + + memset(&msgh, 0, sizeof(msgh)); + iov.iov_base = buf; + iov.iov_len = buflen; + + msgh.msg_iov = &iov; + msgh.msg_iovlen = 1; + msgh.msg_control = control; + msgh.msg_controllen = sizeof(control); + + ret = recvmsg(sockfd, &msgh, 0); + if (ret <= 0) { + if (ret) + VFIO_USER_LOG(DEBUG, "recvmsg failed\n"); + return ret; + } + + if (msgh.msg_flags & (MSG_TRUNC | MSG_CTRUNC)) { + VFIO_USER_LOG(ERR, "Message is truncated\n"); + return -1; + } + + for (cmsg = CMSG_FIRSTHDR(&msgh); cmsg != NULL; + cmsg = CMSG_NXTHDR(&msgh, cmsg)) { + if ((cmsg->cmsg_level == SOL_SOCKET) && + (cmsg->cmsg_type == SCM_RIGHTS)) { + fd_sz = cmsg->cmsg_len - CMSG_LEN(0); + got_fds = fd_sz / sizeof(int); + if (got_fds >= max_fds) { + /* Invalid message, close fds */ + int *close_fd = (int *)CMSG_DATA(cmsg); + for (i = 0; i < got_fds; i++) { + close_fd += i; + close(*close_fd); + } + VFIO_USER_LOG(ERR, "fd num exceeds max " + "in vfio-user msg\n"); + return -1; + } + *fd_num = got_fds; + memcpy(fds, CMSG_DATA(cmsg), got_fds * sizeof(int)); + break; + } + } + + /* Make unused file descriptors invalid */ + while (got_fds < max_fds) + fds[got_fds++] = -1; + + return ret; +} + +int +vfio_user_recv_msg(int sockfd, struct vfio_user_msg *msg) +{ + int ret; + + ret = vfio_user_recv_fd_msg(sockfd, (char *)msg, VFIO_USER_MSG_HDR_SIZE, + msg->fds, VFIO_USER_MAX_FD, &msg->fd_num); + if (ret <= 0) { + return ret; + } else if (ret != VFIO_USER_MSG_HDR_SIZE) { + VFIO_USER_LOG(ERR, "Read unexpected header size\n"); + ret = -1; + goto err; + } + + if (msg->size > VFIO_USER_MSG_HDR_SIZE) { + if (msg->size > (sizeof(msg->payload) + + VFIO_USER_MSG_HDR_SIZE)) { + VFIO_USER_LOG(ERR, "Read invalid msg size: %d\n", + msg->size); + ret = -1; + goto err; + } + + ret = read(sockfd, &msg->payload, + msg->size - VFIO_USER_MSG_HDR_SIZE); + if (ret <= 0) + goto err; + if (ret != (int)(msg->size - VFIO_USER_MSG_HDR_SIZE)) { + VFIO_USER_LOG(ERR, "Read payload failed\n"); + ret = -1; + goto err; + } + } + + return ret; +err: + vfio_user_close_msg_fds(msg); + return ret; +} + +static int +vfio_user_send_fd_msg(int sockfd, char *buf, int buflen, int *fds, int fd_num) +{ + + struct iovec iov; + struct msghdr msgh; + size_t fdsize = fd_num * sizeof(int); + char control[CMSG_SPACE(fdsize)]; + struct cmsghdr *cmsg; + int ret; + + memset(&msgh, 0, sizeof(msgh)); + iov.iov_base = buf; + iov.iov_len = buflen; + + msgh.msg_iov = &iov; + msgh.msg_iovlen = 1; + + if (fds && fd_num > 0) { + msgh.msg_control = control; + msgh.msg_controllen = sizeof(control); + cmsg = CMSG_FIRSTHDR(&msgh); + cmsg->cmsg_len = CMSG_LEN(fdsize); + cmsg->cmsg_level = SOL_SOCKET; + cmsg->cmsg_type = SCM_RIGHTS; + memcpy(CMSG_DATA(cmsg), fds, fdsize); + } else { + msgh.msg_control = NULL; + msgh.msg_controllen = 0; + } + + do { + ret = sendmsg(sockfd, &msgh, MSG_NOSIGNAL); + } while (ret < 0 && errno == EINTR); + + if (ret < 0) { + VFIO_USER_LOG(ERR, "sendmsg error\n"); + return ret; + } + + return ret; +} + +int +vfio_user_send_msg(int sockfd, struct vfio_user_msg *msg) +{ + if (!msg) + return 0; + + return vfio_user_send_fd_msg(sockfd, (char *)msg, + msg->size, msg->fds, msg->fd_num); +} + +int +vfio_user_reply_msg(int sockfd, struct vfio_user_msg *msg) +{ + if (!msg) + return 0; + + msg->flags |= VFIO_USER_NEED_NO_RP; + msg->flags |= VFIO_USER_TYPE_REPLY; + + return vfio_user_send_msg(sockfd, msg); +} + +RTE_LOG_REGISTER(vfio_user_log_level, lib.vfio, INFO); diff --git a/lib/librte_vfio_user/vfio_user_base.h b/lib/librte_vfio_user/vfio_user_base.h new file mode 100644 index 0000000000..34106cc606 --- /dev/null +++ b/lib/librte_vfio_user/vfio_user_base.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020 Intel Corporation + */ + +#ifndef _VFIO_USER_BASE_H +#define _VFIO_USER_BASE_H + +#include + +#define VFIO_USER_MAX_FD 1024 +#define VFIO_USER_MAX_VERSION_DATA 512 + +extern int vfio_user_log_level; +extern const char *vfio_user_msg_str[]; + +#define VFIO_USER_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, vfio_user_log_level, \ + "VFIO_USER: " fmt, ## args) + +struct vfio_user_socket { + char *sock_addr; + int sock_fd; + int dev_id; +}; + +typedef enum VFIO_USER_CMD_TYPE { + VFIO_USER_NONE = 0, + VFIO_USER_VERSION = 1, + VFIO_USER_MAX = 2, +} VFIO_USER_CMD_TYPE; + +struct vfio_user_version { + uint16_t major; + uint16_t minor; + /* Version data (JSON), for now not supported */ + uint8_t ver_data[VFIO_USER_MAX_VERSION_DATA]; +}; + +struct vfio_user_msg { + uint16_t msg_id; + uint16_t cmd; + uint32_t size; +#define VFIO_USER_TYPE_CMD (0x0) /* Message type is COMMAND */ +#define VFIO_USER_TYPE_REPLY (0x1 << 0) /* Message type is REPLY */ +#define VFIO_USER_NEED_NO_RP (0x1 << 4) /* Message needs no reply */ +#define VFIO_USER_ERROR (0x1 << 5) /* Reply message has error */ + uint32_t flags; + uint32_t err; /* Valid in reply, optional */ + union { + struct vfio_user_version ver; + } payload; + int fds[VFIO_USER_MAX_FD]; + int fd_num; +}; + +#define VFIO_USER_MSG_HDR_SIZE offsetof(struct vfio_user_msg, payload.ver) + +void vfio_user_close_msg_fds(struct vfio_user_msg *msg); +int vfio_user_check_msg_fdnum(struct vfio_user_msg *msg, int expected_fds); +void vfio_user_close_msg_fds(struct vfio_user_msg *msg); +int vfio_user_recv_msg(int sockfd, struct vfio_user_msg *msg); +int vfio_user_send_msg(int sockfd, struct vfio_user_msg *msg); +int vfio_user_reply_msg(int sockfd, struct vfio_user_msg *msg); + +#endif diff --git a/lib/meson.build b/lib/meson.build index ed00f89146..b7fbfcc95b 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -28,6 +28,7 @@ libraries = [ 'rib', 'reorder', 'sched', 'security', 'stack', 'vhost', # ipsec lib depends on net, crypto and security 'ipsec', + 'vfio_user', #fib lib depends on rib 'fib', # add pkt framework libs which use other libs from above -- 2.17.1