DPDK patches and discussions
 help / color / mirror / Atom feed
From: Jerin Jacob <jerinjacobk@gmail.com>
To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
Cc: dpdk-dev <dev@dpdk.org>, Thomas Monjalon <thomas@monjalon.net>,
	 "Yigit, Ferruh" <ferruh.yigit@intel.com>,
	Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
	 "Yang, Qiming" <qiming.yang@intel.com>,
	"Zhang, Qi Z" <qi.z.zhang@intel.com>,
	 "Xing, Beilei" <beilei.xing@intel.com>,
	"techboard@dpdk.org" <techboard@dpdk.org>
Subject: Re: [dpdk-dev] [RFC 0/7] hide eth dev related structures
Date: Mon, 27 Sep 2021 21:44:08 +0530	[thread overview]
Message-ID: <CALBAE1OBa+pAB=4Fpq8WNaYRdha3Hg3P9LPygwtybmqgX9XVeQ@mail.gmail.com> (raw)
In-Reply-To: <DM6PR11MB4491BD7D7ED73F07AA2B21B39AA29@DM6PR11MB4491.namprd11.prod.outlook.com>

On Wed, Sep 22, 2021 at 8:38 PM Ananyev, Konstantin
<konstantin.ananyev@intel.com> wrote:
>
>
> > > Hi Jerin,
> > >
> > > > > NOTE: This is just an RFC to start further discussion and collect the feedback.
> > > > > Due to significant amount of work, changes required are applied only to two
> > > > > PMDs so far: net/i40e and net/ice.
> > > > > So to build it you'll need to add:
> > > > > -Denable_drivers='common/*,mempool/*,net/ice,net/i40e'
> > > > > to your config options.
> > > >
> > > > >
> > > > > That approach was selected to avoid(/minimize) possible performance losses.
> > > > >
> > > > > So far I done only limited amount functional and performance testing.
> > > > > Didn't spot any functional problems, and performance numbers
> > > > > remains the same before and after the patch on my box (testpmd, macswap fwd).
> > > >
> > > >
> > > > Based on testing on octeonxt2. We see some regression in testpmd and
> > > > bit on l3fwd too.
> > > >
> > > > Without patch: 73.5mpps/core in testpmd iofwd
> > > > With out patch: 72 5mpps/core in testpmd iofwd
> > > >
> > > > Based on my understanding it is due to additional indirection.
> > >
> > > From your patch below, it looks like not actually additional indirection,
> > > but extra memory dereference - func and dev pointers are now stored
> > > at different places.
> >
> > Yup. I meant the same. We are on the same page.
> >
> > > Plus the fact that now we dereference rte_eth_devices[]
> > > data inside PMD function. Which probably prevents compiler and CPU to load
> > >  rte_eth_devices[port_id].data and rte_eth_devices[port_id]. pre_tx_burst_cbs[queue_id]
> > > in advance before calling actual RX/TX function.
> >
> > Yes.
> >
> > > About your approach: I don’t mind to add extra opaque 'void *data' pointer,
> > > but would prefer not to expose callback invocations code into inline function.
> > > Main reason for that - I think it still need to be reworked to allow adding/removing
> > > callbacks without stopping the device. Something similar to what was done for cryptodev
> > > callbacks. To be able to do that in future without another ABI breakage callbacks related part
> > > needs to be kept internal.
> > > Though what we probably can do: add two dynamic arrays of opaque pointers to  rte_eth_burst_api.
> > > One for rx/tx queue data pointers, second for rx/tx callback pointers.
> > > To be more specific, something like:
> > >
> > > typedef uint16_t (*rte_eth_rx_burst_t)( void *rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts, void *cbs);
> > > typedef uint16_t (*rte_eth_tx_burst_t)(void *txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts, void *cbs);
> > > ....
> > >
> > > struct rte_eth_burst_api {
> > >         rte_eth_rx_burst_t rx_pkt_burst;
> > >         /**< PMD receive function. */
> > >         rte_eth_tx_burst_t tx_pkt_burst;
> > >         /**< PMD transmit function. */
> > >         rte_eth_tx_prep_t tx_pkt_prepare;
> > >         /**< PMD transmit prepare function. */
> > >         rte_eth_rx_queue_count_t rx_queue_count;
> > >         /**< Get the number of used RX descriptors. */
> > >         rte_eth_rx_descriptor_status_t rx_descriptor_status;
> > >         /**< Check the status of a Rx descriptor. */
> > >         rte_eth_tx_descriptor_status_t tx_descriptor_status;
> > >         /**< Check the status of a Tx descriptor. */
> > >         struct {
> > >                  void **queue_data;   /* point to rte_eth_devices[port_id].data-> rx_queues */
> > >                  void **cbs;                  /*  points to rte_eth_devices[port_id].post_rx_burst_cbs */
> > >        } rx_data, tx_data;
> > > } __rte_cache_aligned;
> > >
> > > static inline uint16_t
> > > rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id,
> > >                  struct rte_mbuf **rx_pkts, const uint16_t nb_pkts)
> > > {
> > >        struct rte_eth_burst_api *p;
> > >
> > >         if (port_id >= RTE_MAX_ETHPORTS || queue_id >= RTE_MAX_QUEUES_PER_PORT)
> > >                 return 0;
> > >
> > >       p =  &rte_eth_burst_api[port_id];
> > >       return p->rx_pkt_burst(p->rx_data.queue_data[queue_id], rx_pkts, nb_pkts, p->rx_data.cbs[queue_id]);
> >
> >
> >
> > That works.
> >
> >
> > > }
> > >
> > > Same for TX.
> > >
> > > If that looks ok to everyone, I'll try to prepare next version based on that.
> >
> >
> > Looks good to me.
> >
> > > In theory that should avoid extra dereference problem and even reduce indirection.
> > > As a drawback data->rxq/txq should always be allocated for RTE_MAX_QUEUES_PER_PORT entries,
> > > but I presume that’s not a big deal.
> > >
> > > As a side question - is there any reason why rte_ethdev_trace_rx_burst() is invoked at very last point,
> > > while rte_ethdev_trace_tx_burst()  after CBs but before actual tx_pkt_burst()?
> > > It would make things simpler if tracng would always be done either on entrance or exit of rx/tx_burst.
> >
> > exit is fine.
> >
> > >
> > > >
> > > > My suggestion to fix the problem by:
> > > > Removing the additional `data` redirection and pull callback function
> > > > pointers back
> > > > and keep rest as opaque as done in the existing patch like [1]
> > > >
> > > > I don't believe this has any real implication on future ABI stability
> > > > as we will not be adding
> > > > any new item in rte_eth_fp in any way as new features can be added in slowpath
> > > > rte_eth_dev as mentioned in the patch.
> >
> > Ack
> >
> > I will happy to test again after the rework and report any performance
> > issues if any.
>
> Thanks Jerin, v2 is out:
> https://patches.dpdk.org/project/dpdk/list/?series=19084
> Please have a look, when you'll get a chance.

Tested the series. Looks good, No performance issue.

>

  reply	other threads:[~2021-09-27 16:14 UTC|newest]

Thread overview: 112+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-20 16:28 Konstantin Ananyev
2021-08-20 16:28 ` [dpdk-dev] [RFC 1/7] eth: move ethdev 'burst' API into separate structure Konstantin Ananyev
2021-08-20 16:28 ` [dpdk-dev] [RFC 2/7] eth: make drivers to use new API for Rx Konstantin Ananyev
2021-09-06 18:41   ` Ferruh Yigit
2021-09-14 14:28     ` Ananyev, Konstantin
2021-08-20 16:28 ` [dpdk-dev] [RFC 3/7] eth: make drivers to use new API for Tx Konstantin Ananyev
2021-08-20 16:28 ` [dpdk-dev] [RFC 4/7] eth: make drivers to use new API for Tx prepare Konstantin Ananyev
2021-08-20 16:28 ` [dpdk-dev] [RFC 5/7] eth: make drivers to use new API to obtain descriptor status Konstantin Ananyev
2021-08-20 16:28 ` [dpdk-dev] [RFC 6/7] eth: make drivers to use new API for Rx queue count Konstantin Ananyev
2021-08-20 16:28 ` [dpdk-dev] [RFC 7/7] eth: hide eth dev related structures Konstantin Ananyev
2021-08-26 12:37 ` [dpdk-dev] [RFC 0/7] " Jerin Jacob
2021-09-06 18:09   ` Ferruh Yigit
2021-09-14 13:33   ` Ananyev, Konstantin
2021-09-15  9:45     ` Jerin Jacob
2021-09-22 15:08       ` Ananyev, Konstantin
2021-09-27 16:14         ` Jerin Jacob [this message]
2021-09-28  9:37           ` Ananyev, Konstantin
2021-09-22 14:09 ` [dpdk-dev] [RFC v2 0/5] " Konstantin Ananyev
2021-09-22 14:09   ` [dpdk-dev] [RFC v2 1/5] ethdev: allocate max space for internal queue array Konstantin Ananyev
2021-09-22 14:09   ` [dpdk-dev] [RFC v2 2/5] ethdev: change input parameters for rx_queue_count Konstantin Ananyev
2021-09-23  5:51     ` Wang, Haiyue
2021-09-22 14:09   ` [dpdk-dev] [RFC v2 3/5] ethdev: copy ethdev 'burst' API into separate structure Konstantin Ananyev
2021-09-23  5:58     ` Wang, Haiyue
2021-09-27 18:01       ` Jerin Jacob
2021-09-28  9:42         ` Ananyev, Konstantin
2021-09-22 14:09   ` [dpdk-dev] [RFC v2 4/5] ethdev: make burst functions to use new flat array Konstantin Ananyev
2021-09-22 14:09   ` [dpdk-dev] [RFC v2 5/5] ethdev: hide eth dev related structures Konstantin Ananyev
2021-10-01 14:02   ` [dpdk-dev] [PATCH v3 0/7] " Konstantin Ananyev
2021-10-01 14:02     ` [dpdk-dev] [PATCH v3 1/7] ethdev: allocate max space for internal queue array Konstantin Ananyev
2021-10-01 16:48       ` Ferruh Yigit
2021-10-01 14:02     ` [dpdk-dev] [PATCH v3 2/7] ethdev: change input parameters for rx_queue_count Konstantin Ananyev
2021-10-01 14:02     ` [dpdk-dev] [PATCH v3 3/7] ethdev: copy ethdev 'fast' API into separate structure Konstantin Ananyev
2021-10-01 14:02     ` [dpdk-dev] [PATCH v3 4/7] ethdev: make burst functions to use new flat array Konstantin Ananyev
2021-10-01 16:46       ` Ferruh Yigit
2021-10-01 17:40         ` Ananyev, Konstantin
2021-10-04  8:46           ` Ferruh Yigit
2021-10-04  9:20             ` Ananyev, Konstantin
2021-10-04 10:13               ` Ferruh Yigit
2021-10-04 11:17                 ` Ananyev, Konstantin
2021-10-01 14:02     ` [dpdk-dev] [PATCH v3 5/7] ethdev: add API to retrieve multiple ethernet addresses Konstantin Ananyev
2021-10-01 14:02     ` [dpdk-dev] [PATCH v3 6/7] ethdev: remove legacy Rx descriptor done API Konstantin Ananyev
2021-10-01 14:02     ` [dpdk-dev] [PATCH v3 7/7] ethdev: hide eth dev related structures Konstantin Ananyev
2021-10-01 16:53       ` Ferruh Yigit
2021-10-01 17:04         ` Ferruh Yigit
2021-10-01 17:02     ` [dpdk-dev] [PATCH v3 0/7] " Ferruh Yigit
2021-10-04 13:55     ` [dpdk-dev] [PATCH v4 " Konstantin Ananyev
2021-10-04 13:55       ` [dpdk-dev] [PATCH v4 1/7] ethdev: allocate max space for internal queue array Konstantin Ananyev
2021-10-05 12:09         ` Thomas Monjalon
2021-10-05 16:45           ` Ananyev, Konstantin
2021-10-05 16:49             ` Thomas Monjalon
2021-10-05 12:21         ` Thomas Monjalon
2021-10-04 13:55       ` [dpdk-dev] [PATCH v4 2/7] ethdev: change input parameters for rx_queue_count Konstantin Ananyev
2021-10-04 13:55       ` [dpdk-dev] [PATCH v4 3/7] ethdev: copy ethdev 'fast' API into separate structure Konstantin Ananyev
2021-10-05 13:09         ` Thomas Monjalon
2021-10-05 16:41           ` Ananyev, Konstantin
2021-10-05 16:48             ` Thomas Monjalon
2021-10-05 17:04               ` Ananyev, Konstantin
2021-10-04 13:56       ` [dpdk-dev] [PATCH v4 4/7] ethdev: make burst functions to use new flat array Konstantin Ananyev
2021-10-05  9:54         ` David Marchand
2021-10-05 10:13           ` Ananyev, Konstantin
2021-10-04 13:56       ` [dpdk-dev] [PATCH v4 5/7] ethdev: add API to retrieve multiple ethernet addresses Konstantin Ananyev
2021-10-05 13:13         ` Thomas Monjalon
2021-10-05 16:35           ` Ananyev, Konstantin
2021-10-05 16:45             ` Thomas Monjalon
2021-10-05 17:12               ` Ananyev, Konstantin
2021-10-05 17:41                 ` Thomas Monjalon
2021-10-04 13:56       ` [dpdk-dev] [PATCH v4 6/7] ethdev: remove legacy Rx descriptor done API Konstantin Ananyev
2021-10-05 13:14         ` Thomas Monjalon
2021-10-05 16:21           ` Ananyev, Konstantin
2021-10-04 13:56       ` [dpdk-dev] [PATCH v4 7/7] ethdev: hide eth dev related structures Konstantin Ananyev
2021-10-05 10:04         ` David Marchand
2021-10-05 10:43           ` Ferruh Yigit
2021-10-05 11:37             ` David Marchand
2021-10-05 15:57               ` Ananyev, Konstantin
2021-10-05 13:24         ` Thomas Monjalon
2021-10-05 16:19           ` Ananyev, Konstantin
2021-10-05 16:25             ` Thomas Monjalon
2021-10-06 16:42       ` [dpdk-dev] [PATCH v4 0/7] " Ali Alnubani
2021-10-06 17:26         ` Ali Alnubani
2021-10-07 11:27       ` [dpdk-dev] [PATCH v5 " Konstantin Ananyev
2021-10-07 11:27         ` [dpdk-dev] [PATCH v5 1/7] ethdev: remove legacy Rx descriptor done API Konstantin Ananyev
2021-10-07 11:27         ` [dpdk-dev] [PATCH v5 2/7] ethdev: allocate max space for internal queue array Konstantin Ananyev
2021-10-11  9:20           ` Andrew Rybchenko
2021-10-11 16:25             ` Ananyev, Konstantin
2021-10-11 17:15               ` Andrew Rybchenko
2021-10-11 23:06                 ` Ananyev, Konstantin
2021-10-12  5:47                   ` Andrew Rybchenko
2021-10-07 11:27         ` [dpdk-dev] [PATCH v5 3/7] ethdev: change input parameters for rx_queue_count Konstantin Ananyev
2021-10-11  8:06           ` Andrew Rybchenko
2021-10-12 17:59           ` Hyong Youb Kim (hyonkim)
2021-10-07 11:27         ` [dpdk-dev] [PATCH v5 4/7] ethdev: copy fast-path API into separate structure Konstantin Ananyev
2021-10-09 12:05           ` fengchengwen
2021-10-11  1:18             ` fengchengwen
2021-10-11  8:39               ` Andrew Rybchenko
2021-10-11 15:24               ` Ananyev, Konstantin
2021-10-11  8:35             ` Andrew Rybchenko
2021-10-11 15:15             ` Ananyev, Konstantin
2021-10-11  8:25           ` Andrew Rybchenko
2021-10-11 16:52             ` Ananyev, Konstantin
2021-10-11 17:22               ` Andrew Rybchenko
2021-10-07 11:27         ` [dpdk-dev] [PATCH v5 5/7] ethdev: make fast-path functions to use new flat array Konstantin Ananyev
2021-10-11  9:02           ` Andrew Rybchenko
2021-10-11 15:47             ` Ananyev, Konstantin
2021-10-11 17:03               ` Andrew Rybchenko
2021-10-07 11:27         ` [dpdk-dev] [PATCH v5 6/7] ethdev: add API to retrieve multiple ethernet addresses Konstantin Ananyev
2021-10-11  9:09           ` Andrew Rybchenko
2021-10-07 11:27         ` [dpdk-dev] [PATCH v5 7/7] ethdev: hide eth dev related structures Konstantin Ananyev
2021-10-11  9:20           ` Andrew Rybchenko
2021-10-11 15:54             ` Ananyev, Konstantin
2021-10-11 17:04               ` Andrew Rybchenko
2021-10-08 18:13         ` [dpdk-dev] [PATCH v5 0/7] " Slava Ovsiienko
2021-10-11  9:22         ` Andrew Rybchenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CALBAE1OBa+pAB=4Fpq8WNaYRdha3Hg3P9LPygwtybmqgX9XVeQ@mail.gmail.com' \
    --to=jerinjacobk@gmail.com \
    --cc=andrew.rybchenko@oktetlabs.ru \
    --cc=beilei.xing@intel.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=qi.z.zhang@intel.com \
    --cc=qiming.yang@intel.com \
    --cc=techboard@dpdk.org \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).