mbuf fast free optimization idea

mbuf fast free optimization idea

[Morten Brørup]

When RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE is used, the usage pattern of some of the mbuf fields differs:

- The "pool" pointer becomes rarely used. (Only one packet per burst.)
- The "next" pointer becomes unused.
- The "tx_offload" field is still used. (Set by application, read by driver.)

This means that the 2nd cache line in the mbuf now only contains one hot field, "tx_offload".

So, if we swap the "tx_offload" field (in the 2nd cache line) with the "pool" field (in the 1st cache line), the 2nd cache line becomes rarely used.

This should provide a performance improvement for applications (using RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) when under cache pressure.

PS: Back in the days, the mbuf description said that the 1st cache line was for RX purposes, and the 2nd cache line was for TX purposes. This is not true anymore, so let's not limit out thinking by that (obsolete) design.


Med venlig hilsen / Kind regards,
-Morten Brørup

Re: mbuf fast free optimization idea

[Asaf Penso]

[-- Attachment #1: Type: text/plain, Size: 1390 bytes --]

+ @Slava Ovsiienko<mailto:viacheslavo@nvidia.com>

Regards,
Asaf Penso
________________________________
From: Morten Brørup <mb@smartsharesystems.com>
Sent: Saturday, August 26, 2023 1:04:18 PM
To: olivier.matz@6wind.com <olivier.matz@6wind.com>; NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>; Shahaf Shuler <shahafs@nvidia.com>
Cc: dev@dpdk.org <dev@dpdk.org>
Subject: mbuf fast free optimization idea

When RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE is used, the usage pattern of some of the mbuf fields differs:

- The "pool" pointer becomes rarely used. (Only one packet per burst.)
- The "next" pointer becomes unused.
- The "tx_offload" field is still used. (Set by application, read by driver.)

This means that the 2nd cache line in the mbuf now only contains one hot field, "tx_offload".

So, if we swap the "tx_offload" field (in the 2nd cache line) with the "pool" field (in the 1st cache line), the 2nd cache line becomes rarely used.

This should provide a performance improvement for applications (using RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE) when under cache pressure.

PS: Back in the days, the mbuf description said that the 1st cache line was for RX purposes, and the 2nd cache line was for TX purposes. This is not true anymore, so let's not limit out thinking by that (obsolete) design.


Med venlig hilsen / Kind regards,
-Morten Brørup


[-- Attachment #2: Type: text/html, Size: 2184 bytes --]