From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7F78BA0C47; Tue, 15 Jun 2021 14:16:30 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 021BA4067A; Tue, 15 Jun 2021 14:16:30 +0200 (CEST) Received: from smartserver.smartsharesystems.com (smartserver.smartsharesystems.com [77.243.40.215]) by mails.dpdk.org (Postfix) with ESMTP id 45F1A40140 for ; Tue, 15 Jun 2021 14:16:29 +0200 (CEST) X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Date: Tue, 15 Jun 2021 14:16:27 +0200 Message-ID: <98CBD80474FA8B44BF855DF32C47DC35C6185B@smartserver.smartshare.dk> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: mbuf next field belongs in the first cacheline Thread-Index: Addh4EQkoS7VIMOORbmYSgnV6rTOtA== From: =?iso-8859-1?Q?Morten_Br=F8rup?= To: "Olivier Matz" , "Matan Azrad" , "Shahaf Shuler" , "Viacheslav Ovsiienko" Cc: Subject: [dpdk-dev] mbuf next field belongs in the first cacheline X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" MBUF and MLX5 maintainers, I'm picking up an old discussion, which you might consider pursuing. = Feel free to ignore, if you consider this discussion irrelevant or = already closed and done with. The Techboard has previously discussed the organization of the mbuf = fields. Ref: = http://mails.dpdk.org/archives/dev/2020-November/191859.html It was concluded that there was no measured performance difference if = the "pool" or "next" field was in the first cacheline, so it was decided = to put the "pool" field in the first cacheline. And further optimizing = the mbuf field organization could be reconsidered later. I have been looking at it. In theory it should not be required to touch = the "pool" field at RX. But the "next" field must be written for = segmented packets. I think you could achieve an RX performance gain in the MLX5 driver if = the mbuf structure was changed so the "next" and "pool" fields were = swapped (i.e. putting "next" in the first cacheline), and = /drivers/net/mlx5/mlx5_rx.c line 821 was modified to replace "rep =3D = rte_mbuf_raw_alloc(seg->pool)" with something conceptually like "rep =3D = rte_mbuf_raw_alloc(rxq->pool)". Then you don't have to touch the mbuf's = "pool" field (residing in the second cacheline with this change) during = RX. This way, you would only touch the mbuf's first cacheline during RX. My suggested optimization might be purely theoretical: Many applications = touch the mbuf's second cacheline shortly after RX anyway. If you don't pursue this mbuf reorganization, the comment to the mbuf's = cacheline1 field is incorrect and should be updated: - /* second cache line - fields only used in slow path or on TX */ + /* second cache line - fields mainly used in slow path or on TX */ -Morten