From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8C274A0C46; Fri, 18 Jun 2021 09:40:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2C046410E7; Fri, 18 Jun 2021 09:40:59 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id A7F55410D7 for ; Fri, 18 Jun 2021 09:40:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1624002057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z4Z7CRu+LRixLIuXeyX4X7hNHyuvdSdCMu4He/2X/bM=; b=Li8R1Vfh9kT5u/8ACxHGqP2jJB1IL9PLUu5x/o5Rsa3WIt/MtT4z4D7KeIec+qaCMccYIp zE3fm5ikZ67dPDDDk0Nj/jFJd3JUDgbU07kMeB7t6zxsbdiHFqzxDSuHdfoQcRgznuMy1r QA0GjnZnapzeE/IsM68XlVwkRWBXrMs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-514-KQZlYyNoMwy0ryTj7lZ1DQ-1; Fri, 18 Jun 2021 03:40:54 -0400 X-MC-Unique: KQZlYyNoMwy0ryTj7lZ1DQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0440F1019982; Fri, 18 Jun 2021 07:40:54 +0000 (UTC) Received: from [10.36.110.21] (unknown [10.36.110.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2767217D53; Fri, 18 Jun 2021 07:40:48 +0000 (UTC) To: "Xia, Chenbo" , "dev@dpdk.org" , "david.marchand@redhat.com" Cc: "stable@dpdk.org" References: <20210617153739.178011-1-maxime.coquelin@redhat.com> <20210617153739.178011-2-maxime.coquelin@redhat.com> From: Maxime Coquelin Message-ID: <0f06a4d5-7db9-0485-e0b3-6695b71345bb@redhat.com> Date: Fri, 18 Jun 2021 09:40:47 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v4 1/7] vhost: fix missing memory table NUMA realloc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 6/18/21 6:34 AM, Xia, Chenbo wrote: > Hi Maxime, > >> -----Original Message----- >> From: Maxime Coquelin >> Sent: Thursday, June 17, 2021 11:38 PM >> To: dev@dpdk.org; david.marchand@redhat.com; Xia, Chenbo >> Cc: Maxime Coquelin ; stable@dpdk.org >> Subject: [PATCH v4 1/7] vhost: fix missing memory table NUMA realloc >> >> When the guest allocates virtqueues on a different NUMA node >> than the one the Vhost metadata are allocated, both the Vhost >> device struct and the virtqueues struct are reallocated. >> >> However, reallocating the Vhost memory table was missing, which >> likely causes iat least one cross-NUMA accesses for every burst >> of packets. > > 'at least' ? yes. >> >> This patch reallocates this table on the same NUMA node as the >> other metadata. >> >> Fixes: 552e8fd3d2b4 ("vhost: simplify memory regions handling") >> Cc: stable@dpdk.org >> >> Reported-by: David Marchand >> Signed-off-by: Maxime Coquelin >> --- >> lib/vhost/vhost_user.c | 15 +++++++++++++++ >> 1 file changed, 15 insertions(+) >> >> diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c >> index 8f0eba6412..031e3bfa2f 100644 >> --- a/lib/vhost/vhost_user.c >> +++ b/lib/vhost/vhost_user.c >> @@ -557,6 +557,9 @@ numa_realloc(struct virtio_net *dev, int index) > > As we are realloc more things now, the comment above 'numa_realloc(XXX)' > should also be changed like: > > Reallocate related data structure to make them on the same numa node as > the memory of vring descriptor. Agree, I'll put this: " Reallocate virtio_dev, vhost_virtqueue and related data structures to make them on the same numa node as the memory of vring descriptor. " Thanks, Maxime > Thanks, > Chenbo > >> goto out; >> } >> if (oldnode != newnode) { >> + struct rte_vhost_memory *old_mem; >> + ssize_t mem_size; >> + >> VHOST_LOG_CONFIG(INFO, >> "reallocate dev from %d to %d node\n", >> oldnode, newnode); >> @@ -568,6 +571,18 @@ numa_realloc(struct virtio_net *dev, int index) >> >> memcpy(dev, old_dev, sizeof(*dev)); >> rte_free(old_dev); >> + >> + mem_size = sizeof(struct rte_vhost_memory) + >> + sizeof(struct rte_vhost_mem_region) * dev->mem->nregions; >> + old_mem = dev->mem; >> + dev->mem = rte_malloc_socket(NULL, mem_size, 0, newnode); >> + if (!dev->mem) { >> + dev->mem = old_mem; >> + goto out; >> + } >> + >> + memcpy(dev->mem, old_mem, mem_size); >> + rte_free(old_mem); >> } >> >> out: >> -- >> 2.31.1 >