From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2AAC7A0A03; Mon, 18 Jan 2021 21:37:14 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 836EF140ECC; Mon, 18 Jan 2021 21:36:17 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id A16B1140ECB for ; Mon, 18 Jan 2021 21:36:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611002176; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yml1ZHk0/sPBhwk6DKJg761DGVwXIXBMVkCNSNRcrCo=; b=Yh65UemqkOO4Xf6NDaQ2YLJDvysDld2bv0RjnmgIlV0jpVi7KJFEoCzcXrpHqo8FvazxuE +HJDjyPCvdWhyMT+ssn8Dm6kiEj8EO7Kp0ryFFuDPmblbfH5rMHFifJ5OBTCprN4m1GrfP pwp17PV87jYv2Cg0nto5MAmL2lOvn7s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-585-Nqq_JIWLNcOv3wxv13_QyA-1; Mon, 18 Jan 2021 15:36:14 -0500 X-MC-Unique: Nqq_JIWLNcOv3wxv13_QyA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E60D518C8C03; Mon, 18 Jan 2021 20:36:12 +0000 (UTC) Received: from [10.36.110.29] (unknown [10.36.110.29]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 79A5610013C0; Mon, 18 Jan 2021 20:36:08 +0000 (UTC) To: "Xia, Chenbo" , "dev@dpdk.org" , "olivier.matz@6wind.com" , "amorenoz@redhat.com" , "david.marchand@redhat.com" References: <20201220211405.313012-1-maxime.coquelin@redhat.com> <20201220211405.313012-40-maxime.coquelin@redhat.com> From: Maxime Coquelin Message-ID: Date: Mon, 18 Jan 2021 21:36:07 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 1/11/21 9:02 AM, Xia, Chenbo wrote: > Hi Maxime, > >> -----Original Message----- >> From: Maxime Coquelin >> Sent: Monday, December 21, 2020 5:14 AM >> To: dev@dpdk.org; Xia, Chenbo ; olivier.matz@6wind.com; >> amorenoz@redhat.com; david.marchand@redhat.com >> Cc: Maxime Coquelin >> Subject: [PATCH 39/40] net/virtio: move Vhost-kernel data to its backend >> >> As done earlier for Vhost-user, this patch moves the >> Vhost-Kernel specific data to its backend file. >> >> Signed-off-by: Maxime Coquelin >> --- >> drivers/net/virtio/virtio_user/vhost_kernel.c | 78 +++++++++++++++---- >> .../net/virtio/virtio_user/virtio_user_dev.c | 43 ++-------- >> .../net/virtio/virtio_user/virtio_user_dev.h | 7 +- >> 3 files changed, 72 insertions(+), 56 deletions(-) > > > >> vhost_kernel_setup(struct virtio_user_dev *dev) >> { >> int vhostfd; >> - uint32_t i; >> + uint32_t q, i; >> + struct vhost_kernel_data *data; >> + >> + data = malloc(sizeof(*data)); >> + if (!data) { >> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate Vhost-kernel data", >> dev->path); >> + return -1; >> + } >> + >> + data->vhostfds = malloc(dev->max_queue_pairs * sizeof(int)); >> + data->tapfds = malloc(dev->max_queue_pairs * sizeof(int)); >> + if (!data->vhostfds || !data->tapfds) { >> + PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path); >> + return -1; >> + } >> + >> + for (q = 0; q < dev->max_queue_pairs; ++q) { >> + data->vhostfds[q] = -1; >> + data->tapfds[q] = -1; >> + } > > Forget to do 'dev->backend_data = data'? 😊 > > Besides, > > When I do test 'virtio_user for exceptional path', iperf just cannot normally > send/recv pkts. Testpmd have some pkts (<10), guess tcp connection cannot be > established. I have not tried to locate the root cause. Just thinking it's > better to tell you first. Thanks for reporting. I did a git bisect and could find the issue: static int vhost_kernel_set_vring_kick(struct virtio_user_dev *dev, struct vhost_vring_file *file) { return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_CALL, file); } static int vhost_kernel_set_vring_call(struct virtio_user_dev *dev, struct vhost_vring_file *file) { return vhost_kernel_set_vring_file(dev, VHOST_SET_VRING_KICK, file); } The kick and call request were inverted! Once fixed, I can txonly without getting stuck once queue is full. I will run more Vhost-kernel tests, but I think that was the issue you found. Maxime > Thanks! > Chenbo > >> >> get_vhost_kernel_max_regions(); >> >> for (i = 0; i < dev->max_queue_pairs; ++i) { >> vhostfd = open(dev->path, O_RDWR); >> if (vhostfd < 0) { >> - PMD_DRV_LOG(ERR, "fail to open %s, %s", >> - dev->path, strerror(errno)); >> + PMD_DRV_LOG(ERR, "fail to open %s, %s", dev->path, >> strerror(errno)); >> return -1; >> } >> >> - dev->vhostfds[i] = vhostfd; >> + data->vhostfds[i] = vhostfd; >> } >> >> return 0; >> @@ -360,6 +392,19 @@ vhost_kernel_setup(struct virtio_user_dev *dev) >> static int >> vhost_kernel_destroy(struct virtio_user_dev *dev) >> { >> + struct vhost_kernel_data *data = dev->backend_data; >> + uint32_t i; >> + >> + for (i = 0; i < dev->max_queue_pairs; ++i) { >> + close(data->vhostfds[i]); >> + if (data->tapfds[i] >= 0) >> + close(data->tapfds[i]); >> + } >> + >> + free(data->vhostfds); >> + free(data->tapfds); >> + free(data); >> + >> return 0; >> } >> >> @@ -395,14 +440,15 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev >> *dev, >> int vhostfd; >> int tapfd; >> int req_mq = (dev->max_queue_pairs > 1); >> + struct vhost_kernel_data *data = dev->backend_data; >> >> - vhostfd = dev->vhostfds[pair_idx]; >> + vhostfd = data->vhostfds[pair_idx]; >> >> if (dev->qp_enabled[pair_idx] == enable) >> return 0; >> >> if (!enable) { >> - tapfd = dev->tapfds[pair_idx]; >> + tapfd = data->tapfds[pair_idx]; >> if (vhost_kernel_set_backend(vhostfd, -1) < 0) { >> PMD_DRV_LOG(ERR, "fail to set backend for vhost kernel"); >> return -1; >> @@ -415,8 +461,8 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev, >> return 0; >> } >> >> - if (dev->tapfds[pair_idx] >= 0) { >> - tapfd = dev->tapfds[pair_idx]; >> + if (data->tapfds[pair_idx] >= 0) { >> + tapfd = data->tapfds[pair_idx]; >> if (vhost_kernel_tap_set_offload(tapfd, dev->features) == -1) >> return -1; >> if (req_mq && vhost_kernel_tap_set_queue(tapfd, true) < 0) { >> @@ -439,7 +485,7 @@ vhost_kernel_enable_queue_pair(struct virtio_user_dev *dev, >> return -1; >> } >> >> - dev->tapfds[pair_idx] = tapfd; >> + data->tapfds[pair_idx] = tapfd; >> >> set_backend: >> if (vhost_kernel_set_backend(vhostfd, tapfd) < 0) { >> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.c >> b/drivers/net/virtio/virtio_user/virtio_user_dev.c >> index 93a9ce2cd2..154aecc209 100644 >> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.c >> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.c >> @@ -397,11 +397,6 @@ virtio_user_mem_event_cb(enum rte_mem_event type >> __rte_unused, >> static int >> virtio_user_dev_setup(struct virtio_user_dev *dev) >> { >> - uint32_t q; >> - >> - dev->vhostfds = NULL; >> - dev->tapfds = NULL; >> - >> if (dev->is_server) { >> if (dev->backend_type != VIRTIO_USER_BACKEND_VHOST_USER) { >> PMD_DRV_LOG(ERR, "Server mode only supports vhost-user!"); >> @@ -409,34 +404,21 @@ virtio_user_dev_setup(struct virtio_user_dev *dev) >> } >> } >> >> - if (dev->backend_type == VIRTIO_USER_BACKEND_VHOST_USER) { >> + switch (dev->backend_type) { >> + case VIRTIO_USER_BACKEND_VHOST_USER: >> dev->ops = &virtio_ops_user; >> - } else if (dev->backend_type == >> - VIRTIO_USER_BACKEND_VHOST_KERNEL) { >> + break; >> + case VIRTIO_USER_BACKEND_VHOST_KERNEL: >> dev->ops = &virtio_ops_kernel; >> - >> - dev->vhostfds = malloc(dev->max_queue_pairs * >> - sizeof(int)); >> - dev->tapfds = malloc(dev->max_queue_pairs * >> - sizeof(int)); >> - if (!dev->vhostfds || !dev->tapfds) { >> - PMD_INIT_LOG(ERR, "(%s) Failed to allocate FDs", dev->path); >> - return -1; >> - } >> - >> - for (q = 0; q < dev->max_queue_pairs; ++q) { >> - dev->vhostfds[q] = -1; >> - dev->tapfds[q] = -1; >> - } >> - } else if (dev->backend_type == >> - VIRTIO_USER_BACKEND_VHOST_VDPA) { >> + break; >> + case VIRTIO_USER_BACKEND_VHOST_VDPA: >> dev->ops = &virtio_ops_vdpa; >> - } else { >> + break; >> + default: >> PMD_DRV_LOG(ERR, "(%s) Unknown backend type", dev->path); >> return -1; >> } >> >> - >> if (dev->ops->setup(dev) < 0) { >> PMD_INIT_LOG(ERR, "(%s) Failed to setup backend\n", dev->path); >> return -1; >> @@ -592,15 +574,6 @@ virtio_user_dev_uninit(struct virtio_user_dev *dev) >> close(dev->callfds[i]); >> close(dev->kickfds[i]); >> } >> - if (dev->vhostfds) { >> - for (i = 0; i < dev->max_queue_pairs; ++i) { >> - close(dev->vhostfds[i]); >> - if (dev->tapfds[i] >= 0) >> - close(dev->tapfds[i]); >> - } >> - free(dev->vhostfds); >> - free(dev->tapfds); >> - } >> >> free(dev->ifname); >> >> diff --git a/drivers/net/virtio/virtio_user/virtio_user_dev.h >> b/drivers/net/virtio/virtio_user/virtio_user_dev.h >> index a429dcc57c..5a2c9d38dd 100644 >> --- a/drivers/net/virtio/virtio_user/virtio_user_dev.h >> +++ b/drivers/net/virtio/virtio_user/virtio_user_dev.h >> @@ -31,11 +31,6 @@ struct virtio_user_dev { >> /* for vhost_vdpa backend */ >> int vhostfd; >> >> - /* for vhost_kernel backend */ >> - char *ifname; >> - int *vhostfds; >> - int *tapfds; >> - >> /* for both vhost_user and vhost_kernel */ >> int callfds[VIRTIO_MAX_VIRTQUEUES]; >> int kickfds[VIRTIO_MAX_VIRTQUEUES]; >> @@ -55,6 +50,8 @@ struct virtio_user_dev { >> uint16_t port_id; >> uint8_t mac_addr[RTE_ETHER_ADDR_LEN]; >> char path[PATH_MAX]; >> + char *ifname; >> + >> union { >> struct vring vrings[VIRTIO_MAX_VIRTQUEUES]; >> struct vring_packed packed_vrings[VIRTIO_MAX_VIRTQUEUES]; >> -- >> 2.29.2 >