From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3FE23A0C46; Fri, 18 Jun 2021 10:01:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2F30E410DE; Fri, 18 Jun 2021 10:01:43 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 07B2140150 for ; Fri, 18 Jun 2021 10:01:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624003301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B4cjpUNC9/qTa5SqtihJC2BffPyI53NombEDJyQTF44=; b=S98sYlsAj8Wfce4DgoTibbkwn5hf+05lPyFiePBdMnsTdnuydtkzNnuU5qVhBWgMiq7fq8 CiSD9t3FBeW1pNc8lOVUI6ikmNXply0SK8mz9ZVj3Ehta+MklgWiQaIXxqYutCKOugBiME LJhXxM/cC7xaUkEX9RwUAr6nV8Tm8Ms= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-324-tVqwegyIOoGsJji1rl1wgw-1; Fri, 18 Jun 2021 04:01:40 -0400 X-MC-Unique: tVqwegyIOoGsJji1rl1wgw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 878578030D6; Fri, 18 Jun 2021 08:01:39 +0000 (UTC) Received: from [10.36.110.21] (unknown [10.36.110.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A4A355C22A; Fri, 18 Jun 2021 08:01:33 +0000 (UTC) To: "Xia, Chenbo" , "dev@dpdk.org" , "david.marchand@redhat.com" References: <20210617153739.178011-1-maxime.coquelin@redhat.com> <20210617153739.178011-6-maxime.coquelin@redhat.com> From: Maxime Coquelin Message-ID: Date: Fri, 18 Jun 2021 10:01:32 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v4 5/7] vhost: improve NUMA reallocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 6/18/21 6:34 AM, Xia, Chenbo wrote: > Hi Maxime, > >> -----Original Message----- >> From: Maxime Coquelin >> Sent: Thursday, June 17, 2021 11:38 PM >> To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo >> Cc: Maxime Coquelin >> Subject: [PATCH v4 5/7] vhost: improve NUMA reallocation >> >> This patch improves the numa_realloc() function by making use >> of rte_realloc_socket(), which takes care of the memory copy >> and freeing of the old data. >> >> Suggested-by: David Marchand >> Signed-off-by: Maxime Coquelin >> --- >> lib/vhost/vhost_user.c | 195 ++++++++++++++++++----------------------- >> 1 file changed, 86 insertions(+), 109 deletions(-) >> >> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c >> index 6e7b327ef8..0590ef6d14 100644 >> --- a/lib/vhost/vhost_user.c >> +++ b/lib/vhost/vhost_user.c >> @@ -480,144 +480,121 @@ vhost_user_set_vring_num(struct virtio_net **pdev, >> static struct virtio_net* >> numa_realloc(struct virtio_net *dev, int index) >> { >> - int oldnode, newnode; >> + int node, dev_node; >> struct virtio_net *old_dev; >> - struct vhost_virtqueue *old_vq, *vq; >> - struct vring_used_elem *new_shadow_used_split; >> - struct vring_used_elem_packed *new_shadow_used_packed; >> - struct batch_copy_elem *new_batch_copy_elems; >> + struct vhost_virtqueue *vq; >> + struct batch_copy_elem *bce; >> + struct guest_page *gp; >> + struct rte_vhost_memory *mem; >> + size_t mem_size; >> int ret; >> >> old_dev = dev; >> - vq = old_vq = dev->virtqueue[index]; >> - >> - ret = get_mempolicy(&newnode, NULL, 0, old_vq->desc, >> - MPOL_F_NODE | MPOL_F_ADDR); >> + vq = dev->virtqueue[index]; >> >> - /* check if we need to reallocate vq */ >> - ret |= get_mempolicy(&oldnode, NULL, 0, old_vq, >> - MPOL_F_NODE | MPOL_F_ADDR); >> + ret = get_mempolicy(&node, NULL, 0, vq->desc, MPOL_F_NODE | MPOL_F_ADDR); >> if (ret) { >> - VHOST_LOG_CONFIG(ERR, >> - "Unable to get vq numa information.\n"); >> + VHOST_LOG_CONFIG(ERR, "Unable to get virtqueue %d numa >> information.\n", index); >> return dev; >> } >> - if (oldnode != newnode) { >> - if (vq->ready) { >> - vq->ready = false; >> - vhost_user_notify_queue_state(dev, index, 0); >> - } >> >> - VHOST_LOG_CONFIG(INFO, >> - "reallocate vq from %d to %d node\n", oldnode, newnode); >> - vq = rte_malloc_socket(NULL, sizeof(*vq), 0, newnode); >> - if (!vq) >> - return dev; >> + if (vq->ready) { >> + vq->ready = false; >> + vhost_user_notify_queue_state(dev, index, 0); >> + } >> >> - memcpy(vq, old_vq, sizeof(*vq)); >> + vq = rte_realloc_socket(vq, sizeof(*vq), 0, node); >> + if (!vq) { >> + VHOST_LOG_CONFIG(ERR, "Failed to realloc virtqueue %d on >> node %d\n", >> + index, node); >> + return dev; >> + } >> >> - if (vq_is_packed(dev)) { >> - new_shadow_used_packed = rte_malloc_socket(NULL, >> - vq->size * >> - sizeof(struct vring_used_elem_packed), >> - RTE_CACHE_LINE_SIZE, >> - newnode); >> - if (new_shadow_used_packed) { >> - rte_free(vq->shadow_used_packed); >> - vq->shadow_used_packed = new_shadow_used_packed; >> - } >> - } else { >> - new_shadow_used_split = rte_malloc_socket(NULL, >> - vq->size * >> - sizeof(struct vring_used_elem), >> - RTE_CACHE_LINE_SIZE, >> - newnode); >> - if (new_shadow_used_split) { >> - rte_free(vq->shadow_used_split); >> - vq->shadow_used_split = new_shadow_used_split; >> - } >> - } >> + if (vq != dev->virtqueue[index]) { >> + VHOST_LOG_CONFIG(INFO, "reallocated virtqueue on node %d\n", node); >> + dev->virtqueue[index] = vq; >> + vhost_user_iotlb_init(dev, index); >> + } >> + >> + if (vq_is_packed(dev)) { >> + struct vring_used_elem_packed *sup; >> >> - new_batch_copy_elems = rte_malloc_socket(NULL, >> - vq->size * sizeof(struct batch_copy_elem), >> - RTE_CACHE_LINE_SIZE, >> - newnode); >> - if (new_batch_copy_elems) { >> - rte_free(vq->batch_copy_elems); >> - vq->batch_copy_elems = new_batch_copy_elems; >> + sup = rte_realloc_socket(vq->shadow_used_packed, vq->size * >> sizeof(*sup), >> + RTE_CACHE_LINE_SIZE, node); >> + if (!sup) { >> + VHOST_LOG_CONFIG(ERR, "Failed to realloc shadow packed on >> node %d\n", node); >> + return dev; >> } >> + vq->shadow_used_packed = sup; >> > > Above blank line could be deleted? OK, will do. Thanks, Maxime > Thanks, > Chenbo >