From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 62C0F430FE; Fri, 25 Aug 2023 10:44:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 50F3E40695; Fri, 25 Aug 2023 10:44:19 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id E9556400D5 for ; Fri, 25 Aug 2023 10:44:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1692953057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5trKUOtE0WDcwsEMWOLe2Qln+Lam1rnTZ14KlwFYPTo=; b=Kbq63GufbfA4dyevWmPxw6nwEfsZWv0rBFK2sw/+ydv05rxfUUtSvDeB72Haantt86V4Ip YAlAEU0VmXDH8U6mR/bp4GpnUl+eyXIyOewvr9mywNJA/w+MzimTnrfOuPvHJ85AZ07Xdg N/EWM5s2RUqlQnstoY2nCCWi2XqEwao= Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-471-AJvq7dYQMASe5maUlaOwfA-1; Fri, 25 Aug 2023 04:44:15 -0400 X-MC-Unique: AJvq7dYQMASe5maUlaOwfA-1 Received: by mail-lf1-f71.google.com with SMTP id 2adb3069b0e04-5007ac1c653so753313e87.2 for ; Fri, 25 Aug 2023 01:44:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692953054; x=1693557854; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5trKUOtE0WDcwsEMWOLe2Qln+Lam1rnTZ14KlwFYPTo=; b=AsFynTR3kaup4O6PsfmDfg4rOSm7nBl7x52OBCtoJhXYAQPMa32byZKPUVMdwp/ym0 +RAjoUPqJughwMEU/bW38gFw5ia1qiZm0IHHtP94EKK7HNz30UF5LbmT26Sc40iNYPsj F7V9WjbJr5PFuRvqmqafWb+N1eS97OoIcKigdjfy6IjekakN6xWa4fEDcdaT6AMWAvPO aOA8kM5G02+oBYeXlqZOgdDjJnjD4uJzGTd67/6dH+BDg1g250Y4fICFBWuSg3qrl2BC H46F/i8O/k9GbU9Qa9bR04pdE9TDCey8wLhFYOq7ryiVEGU49zz18lVfI2Ukasyd5nf1 AFFQ== X-Gm-Message-State: AOJu0YwL3l+5ouUnY2TM2ocxZSea6rJKKxggeOxGDav0SFudSovx9aQe ih9l43po+Yu7Q4b4Jp22sN0OdDI+NtAvjGU0Th23TkACWoVklvyyv7YSVrAKDt9y/Ru7oX8bRCC 6Z8VPnDzuAvfsiRiu7GU= X-Received: by 2002:a05:6512:308b:b0:4fd:b223:92c with SMTP id z11-20020a056512308b00b004fdb223092cmr13298927lfd.60.1692953054199; Fri, 25 Aug 2023 01:44:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IECDfu+gRaN++lEf5jWoABEgfCfmR9CoX5PRcTnSiNl4h63Cpu/ZVgQKgTQR/OQEfTQcOqdiENBCO7PWiyMoAc= X-Received: by 2002:a05:6512:308b:b0:4fd:b223:92c with SMTP id z11-20020a056512308b00b004fdb223092cmr13298919lfd.60.1692953053878; Fri, 25 Aug 2023 01:44:13 -0700 (PDT) MIME-Version: 1.0 References: <1680558751-17931-1-git-send-email-roretzla@linux.microsoft.com> <1691781658-32520-1-git-send-email-roretzla@linux.microsoft.com> <1691781658-32520-7-git-send-email-roretzla@linux.microsoft.com> <98CBD80474FA8B44BF855DF32C47DC35D87B32@smartserver.smartshare.dk> <20230824155358.GA17491@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> In-Reply-To: <20230824155358.GA17491@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> From: David Marchand Date: Fri, 25 Aug 2023 10:44:01 +0200 Message-ID: Subject: Re: [PATCH v11 06/16] eal: use prefetch intrinsics To: Tyler Retzlaff , Maxime Coquelin Cc: =?UTF-8?Q?Morten_Br=C3=B8rup?= , dev@dpdk.org, Bruce Richardson , Konstantin Ananyev , Ciara Power , thomas@monjalon.net X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Aug 24, 2023 at 5:54=E2=80=AFPM Tyler Retzlaff wrote: > > On Thu, Aug 24, 2023 at 04:18:06PM +0200, David Marchand wrote: > > On Thu, Aug 24, 2023 at 2:47=E2=80=AFPM Morten Br=C3=B8rup wrote: > > > > From: David Marchand [mailto:david.marchand@redhat.com] > > > > However, I am a bit puzzled why the prefetch change makes the compi= ler > > > > consider this loop differently. > > > > We have the same constructs everywhere in this library and x86_64 > > > > builds are fine... > > > > > > That is indeed the relevant question here! > > > > > > Perhaps the compiler somehow ignores the "const" part of the paramete= r given to the "asm" (in rte_prefetch0()) for 64 bit arch, but not for 32 b= it arch? > > > > It is possible to reproduce the issue with current DPDK tree (with > > existing prefetch implementation in asm) and removing all > > rte_prefetch0 calls from the async rx path: > > > > diff --git a/lib/vhost/virtio_net.c b/lib/vhost/virtio_net.c > > index d7624d18c8..6f941cf27d 100644 > > --- a/lib/vhost/virtio_net.c > > +++ b/lib/vhost/virtio_net.c > > @@ -748,8 +748,6 @@ map_one_desc(struct virtio_net *dev, struct > > vhost_virtqueue *vq, > > if (unlikely(!desc_addr)) > > return -1; > > > > - rte_prefetch0((void *)(uintptr_t)desc_addr); > > - > > buf_vec[vec_id].buf_iova =3D desc_iova; > > buf_vec[vec_id].buf_addr =3D desc_addr; > > buf_vec[vec_id].buf_len =3D desc_chunck_len; > > @@ -1808,8 +1806,6 @@ virtio_dev_rx_async_submit_split(struct > > virtio_net *dev, struct vhost_virtqueue > > */ > > avail_head =3D __atomic_load_n(&vq->avail->idx, __ATOMIC_ACQUIR= E); > > > > - rte_prefetch0(&vq->avail->ring[vq->last_avail_idx & (vq->size -= 1)]); > > - > > async_iter_reset(async); > > > > for (pkt_idx =3D 0; pkt_idx < count; pkt_idx++) { > > @@ -1997,7 +1993,6 @@ virtio_dev_rx_async_packed_batch_enqueue(struct > > virtio_net *dev, > > uint16_t i; > > > > vhost_for_each_try_unroll(i, 0, PACKED_BATCH_SIZE) { > > - rte_prefetch0((void *)(uintptr_t)desc_addrs[i]); > > desc =3D vhost_iova_to_vva(dev, vq, desc_addrs[i], > > &lens[i], VHOST_ACCESS_RW); > > hdrs[i] =3D (struct virtio_net_hdr_mrg_rxbuf *)(uintptr= _t)desc; > > lens[i] =3D pkts[i]->pkt_len + > > @@ -2106,8 +2101,6 @@ virtio_dev_rx_async_submit_packed(struct > > virtio_net *dev, struct vhost_virtqueue > > uint16_t i; > > > > do { > > - rte_prefetch0(&vq->desc_packed[vq->last_avail_idx]); > > - > > if (count - pkt_idx >=3D PACKED_BATCH_SIZE) { > > if (!virtio_dev_rx_async_packed_batch(dev, vq, > > &pkts[pkt_idx], > > dma_id, vchan_id)) { > > > > > > If any prefetch is left, the warning disappears. > > So the asm usage probably impacts/disables some compiler feature, for > > this code path. > > So as an aside one of the reasons often cited as to why intrinsics are > preferred over inline asm is that inline asm can't be considered when > the compiler performs optimization. It could be that when moving to use > the intrinsics the scope of what can be optimized around the prefetch0 > calls can be used for optimization leading to the observations here. Agreed. While we decide the next steps on the vhost library, and for making progress on this series, I decided to keep the previous asm implementation and put intrinsics under a #ifdef MSVC. --=20 David Marchand