From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 85B17430F1; Thu, 24 Aug 2023 16:18:26 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 20C4440EE1; Thu, 24 Aug 2023 16:18:26 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 8EA894067B for ; Thu, 24 Aug 2023 16:18:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692886704; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8DYSuA6QigJmwKNof4vvtu6arJdt7WbLJ+Q70+Mrt48=; b=OkXJFTf2uj5vghmLF1YGPjgV6MewKAAfHKC5NiOJy/z3ZTABC7woAqzgAH4S2bMvSjkyRP 6nVMrWiTZ/Gdy5xRdvQHJFfGHWhZ5bSSLeHFjpwhv9Xv5I5PkSCqPr+B0vFqRxikvAIGls fXWxad2vauBXZQjnb9QN8P7cCsV0TyQ= Received: from mail-lf1-f69.google.com (mail-lf1-f69.google.com [209.85.167.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-462-m-k7tzCfOHSySo2N_wlXyQ-1; Thu, 24 Aug 2023 10:18:23 -0400 X-MC-Unique: m-k7tzCfOHSySo2N_wlXyQ-1 Received: by mail-lf1-f69.google.com with SMTP id 2adb3069b0e04-50084bf5da1so4680314e87.0 for ; Thu, 24 Aug 2023 07:18:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692886698; x=1693491498; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8DYSuA6QigJmwKNof4vvtu6arJdt7WbLJ+Q70+Mrt48=; b=HWIeAyqO1ThpQBi73iQFtu2KCMv1pNC1sXqX+IITBUN9+xOAo3PPAIdcbCvveIRIJW pmOB8T36zxXuXpVOLSrQdiLsQWA/8BA83RIeOJYN+RBrzULZVDFwe8MVs6cuz0NoFBhb wnNOVWKgYAWoepTovjBFTXvUhkhbBafAJXHCGgXKKpXiOI9wPJD0uuaQZNcWtB0APPAZ mN+k1/H5vAIPYRb6pGN+GCUYBRXoFOXWmc7BPp5/b/wwXvMTLi42xB6QFBzZFURKG25a 0J3d9/tIPaoSxVzLlmSdH7nce2NGuMhpb0Xb7eH217JcIqQWQNtyqzq6WEOOWsTI4Pta EYeg== X-Gm-Message-State: AOJu0YyiyMxQNRLx05ZXOyAH8iHLntovL7PLMrDqZoq6Hs/zW4ICbIfM 6jTLqgr1MTiSTja7BPjzVL1PQzw49w9ud3xli/p+KLbzsTyuSdIdGp4Iy1BBvhAk+abBwEGnWSz tqvPA8ZjMbRfvegvn8DI= X-Received: by 2002:a05:6512:3e18:b0:4f8:770f:1b08 with SMTP id i24-20020a0565123e1800b004f8770f1b08mr14123641lfv.13.1692886698661; Thu, 24 Aug 2023 07:18:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGy0Zm77wlDyUai3JAVqENq1sBS1VkWe1oq/p1V8RomMP8iCsX2ZI4pF8Dwa+pQpIF8/fFTNBQQE/tgo1ovK10= X-Received: by 2002:a05:6512:3e18:b0:4f8:770f:1b08 with SMTP id i24-20020a0565123e1800b004f8770f1b08mr14123617lfv.13.1692886698341; Thu, 24 Aug 2023 07:18:18 -0700 (PDT) MIME-Version: 1.0 References: <1680558751-17931-1-git-send-email-roretzla@linux.microsoft.com> <1691781658-32520-1-git-send-email-roretzla@linux.microsoft.com> <1691781658-32520-7-git-send-email-roretzla@linux.microsoft.com> <98CBD80474FA8B44BF855DF32C47DC35D87B32@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D87B32@smartserver.smartshare.dk> From: David Marchand Date: Thu, 24 Aug 2023 16:18:06 +0200 Message-ID: Subject: Re: [PATCH v11 06/16] eal: use prefetch intrinsics To: =?UTF-8?Q?Morten_Br=C3=B8rup?= , Tyler Retzlaff , Maxime Coquelin Cc: dev@dpdk.org, Bruce Richardson , Konstantin Ananyev , Ciara Power , thomas@monjalon.net X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Aug 24, 2023 at 2:47=E2=80=AFPM Morten Br=C3=B8rup wrote: > > From: David Marchand [mailto:david.marchand@redhat.com] > > However, I am a bit puzzled why the prefetch change makes the compiler > > consider this loop differently. > > We have the same constructs everywhere in this library and x86_64 > > builds are fine... > > That is indeed the relevant question here! > > Perhaps the compiler somehow ignores the "const" part of the parameter gi= ven to the "asm" (in rte_prefetch0()) for 64 bit arch, but not for 32 bit a= rch? It is possible to reproduce the issue with current DPDK tree (with existing prefetch implementation in asm) and removing all rte_prefetch0 calls from the async rx path: diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c index d7624d18c8..6f941cf27d 100644 --- a/lib/vhost/virtio_net.c +++ b/lib/vhost/virtio_net.c @@ -748,8 +748,6 @@ map_one_desc(struct virtio_net *dev, struct vhost_virtqueue *vq, if (unlikely(!desc_addr)) return -1; - rte_prefetch0((void *)(uintptr_t)desc_addr); - buf_vec[vec_id].buf_iova =3D desc_iova; buf_vec[vec_id].buf_addr =3D desc_addr; buf_vec[vec_id].buf_len =3D desc_chunck_len; @@ -1808,8 +1806,6 @@ virtio_dev_rx_async_submit_split(struct virtio_net *dev, struct vhost_virtqueue */ avail_head =3D __atomic_load_n(&vq->avail->idx, __ATOMIC_ACQUIRE); - rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size - 1)]= ); - async_iter_reset(async); for (pkt_idx =3D 0; pkt_idx < count; pkt_idx++) { @@ -1997,7 +1993,6 @@ virtio_dev_rx_async_packed_batch_enqueue(struct virtio_net *dev, uint16_t i; vhost_for_each_try_unroll(i, 0, PACKED_BATCH_SIZE) { - rte_prefetch0((void *)(uintptr_t)desc_addrs[i]); desc =3D vhost_iova_to_vva(dev, vq, desc_addrs[i], &lens[i], VHOST_ACCESS_RW); hdrs[i] =3D (struct virtio_net_hdr_mrg_rxbuf *)(uintptr_t)d= esc; lens[i] =3D pkts[i]->pkt_len + @@ -2106,8 +2101,6 @@ virtio_dev_rx_async_submit_packed(struct virtio_net *dev, struct vhost_virtqueue uint16_t i; do { - rte_prefetch0(&vq->desc_packed[vq->last_avail_idx]); - if (count - pkt_idx >=3D PACKED_BATCH_SIZE) { if (!virtio_dev_rx_async_packed_batch(dev, vq, &pkts[pkt_idx], dma_id, vchan_id)) { If any prefetch is left, the warning disappears. So the asm usage probably impacts/disables some compiler feature, for this code path. --=20 David Marchand