DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>,
	Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
	"Yigit, Ferruh" <ferruh.yigit@intel.com>,
	"david.marchand@redhat.com" <david.marchand@redhat.com>,
	"Richardson, Bruce" <bruce.richardson@intel.com>,
	"olivier.matz@6wind.com" <olivier.matz@6wind.com>,
	"jerinj@marvell.com" <jerinj@marvell.com>,
	"viacheslavo@nvidia.com" <viacheslavo@nvidia.com>,
	"ajit.khaparde@broadcom.com" <ajit.khaparde@broadcom.com>,
	"honnappa.nagarahalli@arm.com" <honnappa.nagarahalli@arm.com>,
	"maxime.coquelin@redhat.com" <maxime.coquelin@redhat.com>,
	"stephen@networkplumber.org" <stephen@networkplumber.org>,
	"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>
Subject: Re: [dpdk-dev] [PATCH 15/15] mbuf: move pool pointer in hotter first half
Date: Thu, 29 Oct 2020 14:15:41 +0000
Message-ID: <BYAPR11MB3301BA165C37FE07B8537E119A140@BYAPR11MB3301.namprd11.prod.outlook.com> (raw)
In-Reply-To: <2815114.5jsDjZKEVh@thomas>



> 
> 29/10/2020 11:50, Andrew Rybchenko:
> > On 10/29/20 12:27 PM, Thomas Monjalon wrote:
> > > The mempool pointer in the mbuf struct is moved
> > > from the second to the first half.
> > > It should increase performance on most systems having 64-byte cache line,
> > > i.e. mbuf is split in two cache lines.
> > > On such system, the first half (also called first cache line) is hotter
> > > than the second one where the pool pointer was.
> > >
> > > Moving this field gives more space to dynfield1.
> > >
> > > This is how the mbuf layout looks like (pahole-style):
> > >
> > > word  type                              name                byte  size
> > >  0    void *                            buf_addr;         /*   0 +  8 */
> > >  1    rte_iova_t                        buf_iova          /*   8 +  8 */
> > >       /* --- RTE_MARKER64               rearm_data;                   */
> > >  2    uint16_t                          data_off;         /*  16 +  2 */
> > >       uint16_t                          refcnt;           /*  18 +  2 */
> > >       uint16_t                          nb_segs;          /*  20 +  2 */
> > >       uint16_t                          port;             /*  22 +  2 */
> > >  3    uint64_t                          ol_flags;         /*  24 +  8 */
> > >       /* --- RTE_MARKER                 rx_descriptor_fields1;        */
> > >  4    uint32_t             union        packet_type;      /*  32 +  4 */
> > >       uint32_t                          pkt_len;          /*  36 +  4 */
> > >  5    uint16_t                          data_len;         /*  40 +  2 */
> > >       uint16_t                          vlan_tci;         /*  42 +  2 */
> > >  5.5  uint64_t             union        hash;             /*  44 +  8 */
> > >  6.5  uint16_t                          vlan_tci_outer;   /*  52 +  2 */
> > >       uint16_t                          buf_len;          /*  54 +  2 */
> > >  7    struct rte_mempool *              pool;             /*  56 +  8 */
> > >       /* --- RTE_MARKER                 cacheline1;                   */
> > >  8    struct rte_mbuf *                 next;             /*  64 +  8 */
> > >  9    uint64_t             union        tx_offload;       /*  72 +  8 */
> > > 10    uint16_t                          priv_size;        /*  80 +  2 */
> > >       uint16_t                          timesync;         /*  82 +  2 */
> > >       uint32_t                          seqn;             /*  84 +  4 */
> > > 11    struct rte_mbuf_ext_shared_info * shinfo;           /*  88 +  8 */
> > > 12    uint64_t                          dynfield1[4];     /*  96 + 32 */
> > > 16    /* --- END                                             128      */
> > >
> > > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> >
> > I'd like to understand why pool is chosen instead of, for
> > example, next pointer.
> >
> > Pool is used on housekeeping when driver refills Rx ring or
> > free completed Tx mbufs. Free thresholds try to avoid it on
> > every Rx/Tx burst (if possible).
> >
> > Next is used for multi-segment Tx and scattered (and buffer
> > split) Rx. IMHO the key question here is we consider these
> > use cases as common and priority to optimize. If yes, I'd
> > vote to have next on the first cacheline.

Between these two I also would probably lean towards *next*
(after all _free_ also has to access/update next).
As another alternative to consider: tx_offload.
It is also used quite widely. 

> >
> > I'm not sure. Just trying to hear a bit more about it.
> .
> That's a good question.
> Clearly pool and next are good options.
> The best would be to have some benchmarks.
> If one use case shows no benefit, the decision is easier.
> 
> If you prefer, we can leave this last patch for -rc3.
> 


  reply	other threads:[~2020-10-29 14:15 UTC|newest]

Thread overview: 170+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-29  9:27 [dpdk-dev] [PATCH 00/15] remove mbuf timestamp Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 01/15] eventdev: remove software Rx timestamp Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 02/15] mbuf: add Rx timestamp dynamic flag Thomas Monjalon
2020-10-29  9:58   ` Andrew Rybchenko
2020-10-29 18:19     ` Ajit Khaparde
2020-10-29  9:27 ` [dpdk-dev] [PATCH 03/15] ethdev: register mbuf field and flags for timestamp Thomas Monjalon
2020-10-29 10:08   ` Andrew Rybchenko
2020-10-29 10:12     ` Thomas Monjalon
2020-10-29 10:33       ` Andrew Rybchenko
2020-10-29 10:46         ` Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 04/15] latency: switch timestamp to dynamic mbuf field Thomas Monjalon
2020-10-29 10:13   ` Andrew Rybchenko
2020-10-29 10:40     ` Thomas Monjalon
2020-10-29 14:20   ` Pattan, Reshma
2020-10-29 16:15     ` Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 05/15] net/ark: " Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 06/15] net/dpaa2: " Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 07/15] net/mlx5: fix dynamic mbuf offset lookup check Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 08/15] net/mlx5: switch timestamp to dynamic mbuf field Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 09/15] net/nfb: " Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 10/15] net/octeontx2: " Thomas Monjalon
2020-10-29 11:02   ` Andrew Rybchenko
2020-10-29 11:34     ` Thomas Monjalon
2020-10-29 11:37       ` Andrew Rybchenko
2020-10-29 11:52     ` Slava Ovsiienko
2020-10-30 12:41       ` Jerin Jacob
2020-11-01 16:12         ` Thomas Monjalon
2020-11-01 20:00         ` Andrew Rybchenko
2020-10-29  9:27 ` [dpdk-dev] [PATCH 11/15] net/pcap: " Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 12/15] app/testpmd: " Thomas Monjalon
2020-10-29 10:20   ` Andrew Rybchenko
2020-10-29 10:43     ` Thomas Monjalon
2020-10-29 10:52       ` Andrew Rybchenko
2020-10-29  9:27 ` [dpdk-dev] [PATCH 13/15] examples/rxtx_callbacks: switch timestamp to dynamic field Thomas Monjalon
2020-10-29 10:21   ` Andrew Rybchenko
2020-10-29 10:44     ` Thomas Monjalon
2020-10-29  9:27 ` [dpdk-dev] [PATCH 14/15] mbuf: remove deprecated timestamp field Thomas Monjalon
2020-10-29 10:23   ` Andrew Rybchenko
2020-10-29 18:18     ` Ajit Khaparde
2020-10-29 14:48   ` Kinsella, Ray
2020-10-29  9:27 ` [dpdk-dev] [PATCH 15/15] mbuf: move pool pointer in hotter first half Thomas Monjalon
2020-10-29 10:50   ` Andrew Rybchenko
2020-10-29 10:56     ` Thomas Monjalon
2020-10-29 14:15       ` Ananyev, Konstantin [this message]
2020-10-29 18:45         ` Ajit Khaparde
2020-10-31 18:20           ` [dpdk-dev] [PATCH 15/15] mbuf: move pool pointer in hotterfirst half Morten Brørup
2020-10-31 20:40             ` Thomas Monjalon
2020-11-01  9:12               ` Morten Brørup
2020-11-01 16:21                 ` Thomas Monjalon
2020-11-01 16:38                 ` Thomas Monjalon
2020-11-01 20:59                   ` Morten Brørup
2020-11-02 15:58                     ` Thomas Monjalon
2020-11-03 12:10                       ` Morten Brørup
2020-11-03 12:25                         ` Bruce Richardson
2020-11-03 13:46                           ` Morten Brørup
2020-11-03 13:50                             ` Bruce Richardson
2020-11-03 14:03                               ` Morten Brørup
2020-11-03 14:02                         ` Slava Ovsiienko
2020-11-03 15:03                           ` Morten Brørup
2020-11-04 15:00                             ` Olivier Matz
2020-11-05  0:25                               ` Ananyev, Konstantin
2020-11-05  9:04                                 ` Morten Brørup
2020-11-05  9:35                                 ` Morten Brørup
2020-11-05 10:29                                   ` Bruce Richardson
2020-10-29 14:42   ` [dpdk-dev] [PATCH 15/15] mbuf: move pool pointer in hotter first half Kinsella, Ray
2020-11-01 18:06 ` [dpdk-dev] [PATCH v2 00/14] remove mbuf timestamp Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 01/14] eventdev: remove software Rx timestamp Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 02/14] mbuf: add Rx timestamp dynamic flag Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 03/14] ethdev: register mbuf field and flags for timestamp Thomas Monjalon
2020-11-02 15:39     ` Olivier Matz
2020-11-02 16:52       ` Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 04/14] latency: switch timestamp to dynamic mbuf field Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 05/14] net/ark: " Thomas Monjalon
2020-11-02 15:32     ` Olivier Matz
2020-11-02 16:10       ` Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 06/14] net/dpaa2: " Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 07/14] net/mlx5: fix dynamic mbuf offset lookup check Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 08/14] net/mlx5: switch timestamp to dynamic mbuf field Thomas Monjalon
2020-11-02  5:08     ` Ruifeng Wang
2020-11-02 23:20     ` David Christensen
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 09/14] net/nfb: " Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 10/14] net/octeontx2: " Thomas Monjalon
2020-11-01 18:28     ` Jerin Jacob
2020-11-02  9:38       ` Thomas Monjalon
2020-11-02 11:01         ` Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 11/14] net/pcap: " Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 12/14] app/testpmd: " Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 13/14] examples/rxtx_callbacks: switch timestamp to dynamic field Thomas Monjalon
2020-11-01 18:06   ` [dpdk-dev] [PATCH v2 14/14] mbuf: remove deprecated timestamp field Thomas Monjalon
2020-11-02 15:41     ` Olivier Matz
2020-11-02 15:47       ` David Marchand
2020-11-02 15:49         ` Thomas Monjalon
2020-11-01 18:08   ` [dpdk-dev] [PATCH v2 00/14] remove mbuf timestamp Thomas Monjalon
2020-11-03  0:13 ` [dpdk-dev] [PATCH v3 00/16] " Thomas Monjalon
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 01/16] eventdev: remove software Rx timestamp Thomas Monjalon
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 02/16] mbuf: add Rx timestamp flag and helpers Thomas Monjalon
2020-11-03  9:33     ` Olivier Matz
2020-11-03  9:59       ` Thomas Monjalon
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 03/16] latency: switch Rx timestamp to dynamic mbuf field Thomas Monjalon
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 04/16] net/ark: " Thomas Monjalon
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 05/16] net/dpaa2: " Thomas Monjalon
2020-11-03  9:18     ` Hemant Agrawal
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 06/16] net/mlx5: fix dynamic mbuf offset lookup check Thomas Monjalon
2020-11-03  8:12     ` Slava Ovsiienko
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 07/16] net/mlx5: switch Rx timestamp to dynamic mbuf field Thomas Monjalon
2020-11-03  8:12     ` Slava Ovsiienko
2020-11-03  0:13   ` [dpdk-dev] [PATCH v3 08/16] net/nfb: " Thomas Monjalon
2020-11-03 10:20     ` Olivier Matz
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 09/16] net/octeontx2: " Thomas Monjalon
2020-11-03 10:52     ` Harman Kalra
2020-11-03 11:22       ` Thomas Monjalon
2020-11-03 12:21         ` Thomas Monjalon
2020-11-03 14:23           ` [dpdk-dev] [EXT] " Harman Kalra
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 10/16] net/pcap: " Thomas Monjalon
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 11/16] app/testpmd: " Thomas Monjalon
2020-11-03 10:23     ` Olivier Matz
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 12/16] examples/rxtx_callbacks: switch timestamp to dynamic field Thomas Monjalon
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 13/16] ethdev: add doxygen comment for Rx timestamp API Thomas Monjalon
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 14/16] mbuf: remove deprecated timestamp field Thomas Monjalon
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 15/16] mbuf: add Tx timestamp registration helper Thomas Monjalon
2020-11-03  0:14   ` [dpdk-dev] [PATCH v3 16/16] ethdev: include mbuf registration in Tx timestamp API Thomas Monjalon
2020-11-03  7:54     ` Slava Ovsiienko
2020-11-03  9:00   ` [dpdk-dev] [PATCH v3 00/16] remove mbuf timestamp David Marchand
2020-11-03 12:21 ` [dpdk-dev] [PATCH v4 " Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 01/16] eventdev: remove software Rx timestamp Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 02/16] mbuf: add Rx timestamp flag and helpers Thomas Monjalon
2020-11-03 12:34     ` Andrew Rybchenko
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 03/16] latency: switch Rx timestamp to dynamic mbuf field Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 04/16] net/ark: " Thomas Monjalon
2020-11-03 12:37     ` Andrew Rybchenko
2020-11-03 13:08       ` Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 05/16] net/dpaa2: " Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 06/16] net/mlx5: fix dynamic mbuf offset lookup check Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 07/16] net/mlx5: switch Rx timestamp to dynamic mbuf field Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 08/16] net/nfb: " Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 09/16] net/octeontx2: " Thomas Monjalon
2020-11-03 12:21   ` [dpdk-dev] [PATCH v4 10/16] net/pcap: " Thomas Monjalon
2020-11-03 12:22   ` [dpdk-dev] [PATCH v4 11/16] app/testpmd: " Thomas Monjalon
2020-11-03 12:22   ` [dpdk-dev] [PATCH v4 12/16] examples/rxtx_callbacks: switch timestamp to dynamic field Thomas Monjalon
2020-11-03 12:22   ` [dpdk-dev] [PATCH v4 13/16] ethdev: add doxygen comment for Rx timestamp API Thomas Monjalon
2020-11-03 12:40     ` Andrew Rybchenko
2020-11-03 12:22   ` [dpdk-dev] [PATCH v4 14/16] mbuf: remove deprecated timestamp field Thomas Monjalon
2020-11-03 12:22   ` [dpdk-dev] [PATCH v4 15/16] mbuf: add Tx timestamp registration helper Thomas Monjalon
2020-11-03 12:42     ` Andrew Rybchenko
2020-11-03 12:22   ` [dpdk-dev] [PATCH v4 16/16] ethdev: include mbuf registration in Tx timestamp API Thomas Monjalon
2020-11-03 12:45     ` Andrew Rybchenko
2020-11-03 14:09 ` [dpdk-dev] [PATCH v5 00/16] remove mbuf timestamp Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 01/16] eventdev: remove software Rx timestamp Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 02/16] mbuf: add Rx timestamp flag and helpers Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 03/16] latency: switch Rx timestamp to dynamic mbuf field Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 04/16] net/ark: " Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 05/16] net/dpaa2: " Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 06/16] net/mlx5: fix dynamic mbuf offset lookup check Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 07/16] net/mlx5: switch Rx timestamp to dynamic mbuf field Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 08/16] net/nfb: " Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 09/16] net/octeontx2: " Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 10/16] net/pcap: " Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 11/16] app/testpmd: " Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 12/16] examples/rxtx_callbacks: switch timestamp to dynamic field Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 13/16] ethdev: add doxygen comment for Rx timestamp API Thomas Monjalon
2020-11-03 19:07     ` Ajit Khaparde
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 14/16] mbuf: remove deprecated timestamp field Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 15/16] mbuf: add Tx timestamp registration helper Thomas Monjalon
2020-11-03 14:09   ` [dpdk-dev] [PATCH v5 16/16] ethdev: include mbuf registration in Tx timestamp API Thomas Monjalon
2020-11-03 14:17   ` [dpdk-dev] [PATCH v5 00/16] remove mbuf timestamp Olivier Matz
2020-11-03 14:44     ` Thomas Monjalon
2020-11-03 16:08   ` Stephen Hemminger
2020-11-03 16:20     ` Thomas Monjalon
2020-11-03 17:42       ` Stephen Hemminger
2020-11-03 17:55         ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BYAPR11MB3301BA165C37FE07B8537E119A140@BYAPR11MB3301.namprd11.prod.outlook.com \
    --to=konstantin.ananyev@intel.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=hemant.agrawal@nxp.com \
    --cc=honnappa.nagarahalli@arm.com \
    --cc=jerinj@marvell.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=olivier.matz@6wind.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git