From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 67B73A00C2; Wed, 17 Mar 2021 11:20:57 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2363C140F95; Wed, 17 Mar 2021 11:20:57 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by mails.dpdk.org (Postfix) with ESMTP id B11CA140F8B for ; Wed, 17 Mar 2021 11:20:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615976455; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CNHn0XCaFwM8Ah7uLsTp+9/PUrCmaU26a1Pm3v5i+/0=; b=GNopCZR5wJ5wDt3HZkQPnPHC4Le04k9rzRNfZx5ebm1rMHmkVWDLCzBG6RPQxIMIze0PI7 QRnwOxWlRtdoxSHyw4lH4weRPmeoY/7cDMnajIjDeJGGXVJIDGvGC4SddwSSutwHiEjERQ LtF0uVGaCOKHRn7JSI9Q90mAREji4Nk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-145-K0C3jynGNKC6U8jc4ifC-w-1; Wed, 17 Mar 2021 06:20:53 -0400 X-MC-Unique: K0C3jynGNKC6U8jc4ifC-w-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 58A6F107ACCD; Wed, 17 Mar 2021 10:20:52 +0000 (UTC) Received: from [10.36.110.14] (unknown [10.36.110.14]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 42589690FE; Wed, 17 Mar 2021 10:20:47 +0000 (UTC) To: David Marchand Cc: dev , "Xia, Chenbo" , Adrian Moreno Zapata , Olivier Matz , bnemeth@redhat.com References: <20210316124153.503928-1-maxime.coquelin@redhat.com> <20210316124153.503928-3-maxime.coquelin@redhat.com> From: Maxime Coquelin Message-ID: <9746d67b-45c7-9cdc-f458-ff769870fa40@redhat.com> Date: Wed, 17 Mar 2021 11:20:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2 2/3] vhost: move dirty logging cache out of the virtqueue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 3/16/21 2:13 PM, David Marchand wrote: > On Tue, Mar 16, 2021 at 1:42 PM Maxime Coquelin > wrote: >> >> This patch moves the per-virtqueue's dirty logging cache >> out of the virtqueue struct, by allocating it dynamically >> only when live-migration is enabled. >> >> It saves 8 cachelines in vhost_virtqueue struct. >> >> Signed-off-by: Maxime Coquelin >> --- >> lib/librte_vhost/vhost.c | 14 ++++++++++++++ >> lib/librte_vhost/vhost.h | 2 +- >> lib/librte_vhost/vhost_user.c | 25 +++++++++++++++++++++++++ >> 3 files changed, 40 insertions(+), 1 deletion(-) >> >> diff --git a/lib/librte_vhost/vhost.c b/lib/librte_vhost/vhost.c >> index 5a7c0c6cff..c3490ce897 100644 >> --- a/lib/librte_vhost/vhost.c >> +++ b/lib/librte_vhost/vhost.c >> @@ -145,6 +145,10 @@ __vhost_log_cache_sync(struct virtio_net *dev, struct vhost_virtqueue *vq) >> if (unlikely(!dev->log_base)) >> return; >> >> + /* No cache, nothing to sync */ >> + if (unlikely(!vq->log_cache)) >> + return; >> + >> rte_atomic_thread_fence(__ATOMIC_RELEASE); >> >> log_base = (unsigned long *)(uintptr_t)dev->log_base; >> @@ -177,6 +181,14 @@ vhost_log_cache_page(struct virtio_net *dev, struct vhost_virtqueue *vq, >> uint32_t offset = page / (sizeof(unsigned long) << 3); >> int i; >> >> + if (unlikely(!vq->log_cache)) { >> + /* No logging cache allocated, write dirty log map directly */ >> + rte_smp_wmb(); > > We try not to reintroduce full barriers (checkpatch caught this). Agree, the rebase introduced it again silently. > >> + vhost_log_page((uint8_t *)(uintptr_t)dev->log_base, page); >> + >> + return; >> + } >> + >> for (i = 0; i < vq->log_cache_nb_elem; i++) { >> struct log_cache_entry *elem = vq->log_cache + i; >> >> @@ -354,6 +366,8 @@ free_vq(struct virtio_net *dev, struct vhost_virtqueue *vq) >> } >> rte_free(vq->batch_copy_elems); >> rte_mempool_free(vq->iotlb_pool); >> + if (vq->log_cache) >> + rte_free(vq->log_cache); > > No if() needed. Yes, will fix in next revision. > >> rte_free(vq); >> } >> >> diff --git a/lib/librte_vhost/vhost.h b/lib/librte_vhost/vhost.h >> index 717f410548..3a71dfeed9 100644 >> --- a/lib/librte_vhost/vhost.h >> +++ b/lib/librte_vhost/vhost.h >> @@ -183,7 +183,7 @@ struct vhost_virtqueue { >> bool used_wrap_counter; >> bool avail_wrap_counter; >> >> - struct log_cache_entry log_cache[VHOST_LOG_CACHE_NR]; >> + struct log_cache_entry *log_cache; >> uint16_t log_cache_nb_elem; >> >> rte_rwlock_t iotlb_lock; >> diff --git a/lib/librte_vhost/vhost_user.c b/lib/librte_vhost/vhost_user.c >> index a60bb945ad..0f452d6ff3 100644 >> --- a/lib/librte_vhost/vhost_user.c >> +++ b/lib/librte_vhost/vhost_user.c >> @@ -2022,6 +2022,11 @@ vhost_user_get_vring_base(struct virtio_net **pdev, >> rte_free(vq->batch_copy_elems); >> vq->batch_copy_elems = NULL; >> >> + if (vq->log_cache) { >> + rte_free(vq->log_cache); >> + vq->log_cache = NULL; >> + } > > Idem. > > Thanks, Maxime