From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6BF0343C20; Wed, 28 Feb 2024 15:19:00 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E6F1E40E4A; Wed, 28 Feb 2024 15:18:59 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id F087B40DCD for ; Wed, 28 Feb 2024 15:18:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1709129937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rhv9PAuk+OnrqNnkKZjAzxHDLa3Iv5nZfUzEMAnvrQY=; b=JAm492mQ2+aol9eWCaDyo17KlFWUE8Ns2PsjcVmWiGG2vyPSlzY8M/JARTqqqZcSzYfDur /WmWhBubw0iyANJ56evDgofq7qD1InIYc0LL19GMClq12DsqbjAJxPafA/sZ1MDLHepNlg aIaVZlz4U3BbC5Pi36k9lT4Fpe58rUU= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-98-GjAGeVHDNi2Vk3YsRqAvsw-1; Wed, 28 Feb 2024 09:18:54 -0500 X-MC-Unique: GjAGeVHDNi2Vk3YsRqAvsw-1 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-2d2a5e2e7c3so13094871fa.2 for ; Wed, 28 Feb 2024 06:18:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709129932; x=1709734732; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Rhv9PAuk+OnrqNnkKZjAzxHDLa3Iv5nZfUzEMAnvrQY=; b=VSFfYizYgBzk7aKeb5Gm6J+zsCsm+yG6h+uHkYmICGf3MjoRusM0CPc9ZKqB8/RpGf C7gDKaRHkjlk6XYpET5K3LiMO7piFhKMZWD34AxRXFX5sOQl21UkeKux21MiQg5VSGji 0uYXA7aho+ujjJgWum9OpdLHuUtJymBeaRTZfIfNVT0aQhXL5LiFwCkPeM7g9hbSXTl1 otEgdscAcP3dQBw1TKydqLc42W/9IyXOtHCldXGbMMLmnYWIYB6vDr4f6xg2jUjOTTHQ cRw4lROAjF+34BN/IKqUzgsPD4xGI2iGNmoj+OAZgqpNt2Ox7DldonKuZ5oP61NqV3/v h3vg== X-Gm-Message-State: AOJu0YxHJLB78pBccoXdBQdKk3n6jqk+8CYOu7lMvFqCfa5gNqpIUFk8 kAgU9X+VRjiOMJbcVuOt03Ry5Yur6vFHxg8happTX/8KmZzZDXNsaJpnmuB5Pu4KlW2H3W/KrSL mM5aWCzEeJ9vcRVhD5jX4y28TPu4p892gs94Z3XJkOToCH62oc1d+ugsO4DkFRYjnbAEY52ZUmE YfIxhgNbMgB/W7hd0= X-Received: by 2002:a05:651c:2019:b0:2d2:29c2:e7ea with SMTP id s25-20020a05651c201900b002d229c2e7eamr7454392ljo.24.1709129932669; Wed, 28 Feb 2024 06:18:52 -0800 (PST) X-Google-Smtp-Source: AGHT+IECL00JzplNO3Uhr7nAc5/jc2FblWIvjqENn1bgNIXh46iqV9pECdOhyHJ5gN8Hz7bJHxBv+iACxtX5K8J+ocA= X-Received: by 2002:a05:651c:2019:b0:2d2:29c2:e7ea with SMTP id s25-20020a05651c201900b002d229c2e7eamr7454375ljo.24.1709129932322; Wed, 28 Feb 2024 06:18:52 -0800 (PST) MIME-Version: 1.0 References: <1706657173-26166-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-1-git-send-email-roretzla@linux.microsoft.com> <1709012499-12813-21-git-send-email-roretzla@linux.microsoft.com> In-Reply-To: <1709012499-12813-21-git-send-email-roretzla@linux.microsoft.com> From: David Marchand Date: Wed, 28 Feb 2024 15:18:40 +0100 Message-ID: Subject: Re: [PATCH v6 20/23] mbuf: remove and stop using rte marker fields To: Tyler Retzlaff Cc: dev@dpdk.org, Ajit Khaparde , Andrew Boyer , Andrew Rybchenko , Bruce Richardson , Chenbo Xia , Chengwen Feng , Dariusz Sosnowski , David Christensen , Hyong Youb Kim , Jerin Jacob , Jie Hai , Jingjing Wu , John Daley , Kevin Laatz , Kiran Kumar K , Konstantin Ananyev , Maciej Czekaj , Matan Azrad , Maxime Coquelin , Nithin Dabilpuram , Ori Kam , Ruifeng Wang , Satha Rao , Somnath Kotur , Suanming Mou , Sunil Kumar Kori , Viacheslav Ovsiienko , Yisen Zhuang , Yuying Zhang , mb@smartsharesystems.com, Thomas Monjalon X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Feb 27, 2024 at 6:44=E2=80=AFAM Tyler Retzlaff wrote: > > RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Remove > RTE_MARKER fields from rte_mbuf struct. > > Maintain alignment of fields after removed cacheline1 marker by placing > C11 alignas(RTE_CACHE_LINE_MIN_SIZE). > > Update implementation of rte_mbuf_prefetch_part1() and > rte_mbuf_prefetch_part2() inline functions calculate pointer for > prefetch of cachline0 and cachline1 without using removed markers. > > Update static_assert of rte_mbuf struct fields to reference data_off and > packet_type fields that occupy the original offsets of the marker > fields. > > Signed-off-by: Tyler Retzlaff > --- > doc/guides/rel_notes/release_24_03.rst | 9 ++++++++ > lib/mbuf/rte_mbuf.h | 4 ++-- > lib/mbuf/rte_mbuf_core.h | 39 +++++++++++++---------------= ------ > 3 files changed, 26 insertions(+), 26 deletions(-) > > diff --git a/doc/guides/rel_notes/release_24_03.rst b/doc/guides/rel_note= s/release_24_03.rst > index 879bb49..67750f2 100644 > --- a/doc/guides/rel_notes/release_24_03.rst > +++ b/doc/guides/rel_notes/release_24_03.rst > @@ -156,6 +156,15 @@ Removed Items > The application reserved statically defined logtypes ``RTE_LOGTYPE_USE= R1..RTE_LOGTYPE_USER8`` > are still defined. > > +* mbuf: ``RTE_MARKER`` fields ``cacheline0`` ``cacheline1`` > + ``rx_descriptor_fields1`` and ``RTE_MARKER64`` field ``rearm_data`` > + have been removed from ``struct rte_mbuf``. > + Prefetch of ``cacheline0`` and ``cacheline1`` may be achieved through > + ``rte_mbuf_prefetch_part1()`` and ``rte_mbuf_prefetch_part2()`` inline > + functions respectively. > + Access to ``rearm_data`` and ``rx_descriptor_fields1`` should be > + through new inline functions ``rte_mbuf_rearm_data()`` and > + ``rte_mbuf_rx_descriptor_fields1()`` respectively. > > API Changes > ----------- > diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h > index aa7495b..61cda20 100644 > --- a/lib/mbuf/rte_mbuf.h > +++ b/lib/mbuf/rte_mbuf.h > @@ -108,7 +108,7 @@ > static inline void > rte_mbuf_prefetch_part1(struct rte_mbuf *m) > { > - rte_prefetch0(&m->cacheline0); > + rte_prefetch0(m); > } > > /** > @@ -126,7 +126,7 @@ > rte_mbuf_prefetch_part2(struct rte_mbuf *m) > { > #if RTE_CACHE_LINE_SIZE =3D=3D 64 > - rte_prefetch0(&m->cacheline1); > + rte_prefetch0(RTE_PTR_ADD(m, RTE_CACHE_LINE_MIN_SIZE)); > #else > RTE_SET_USED(m); > #endif > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h > index 36551c2..4e06f15 100644 > --- a/lib/mbuf/rte_mbuf_core.h > +++ b/lib/mbuf/rte_mbuf_core.h > @@ -18,6 +18,7 @@ > > #include > #include > +#include > #include > > #include > @@ -467,8 +468,6 @@ enum { > * The generic rte_mbuf, containing a packet mbuf. > */ > struct rte_mbuf { > - RTE_MARKER cacheline0; > - > void *buf_addr; /**< Virtual address of segment buffer.= */ > #if RTE_IOVA_IN_MBUF > /** > @@ -495,7 +494,6 @@ struct rte_mbuf { > * To obtain a pointer to rearm_data use the rte_mbuf_rearm_data(= ) > * accessor instead of directly referencing through the data_off = field. > */ > - RTE_MARKER64 rearm_data; > uint16_t data_off; One subtile change of removing the marker is that fields may not be aligned as before. #if RTE_IOVA_IN_MBUF /** * Physical address of segment buffer. * This field is undefined if the build is configured to use only * virtual address as IOVA (i.e. RTE_IOVA_IN_MBUF is 0). * Force alignment to 8-bytes, so as to ensure we have the exact * same mbuf cacheline0 layout for 32-bit and 64-bit. This makes * working on vector drivers easier. */ rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); #else /** * Next segment of scattered packet. * This field is valid when physical address field is undefined. * Otherwise next pointer in the second cache line will be used. */ struct rte_mbuf *next; #endif When building ! RTE_IOVA_IN_MBUF on a 32 bits arch, the next pointer is not force aligned to 64bits. Which has a cascade effect on data_off alignement. In file included from ../lib/mbuf/rte_mbuf_core.h:19, from ../lib/mbuf/rte_mbuf.h:42, from ../lib/mbuf/rte_mbuf_dyn.c:18: ../lib/mbuf/rte_mbuf_core.h:676:1: error: static assertion failed: "data_of= f" 676 | static_assert(!(offsetof(struct rte_mbuf, data_off) !=3D | ^~~~~~~~~~~~~ I hope reviewers pay attention to the alignment changes when removing those markers. This is not trivial to catch in the CI. --=20 David Marchand