From: Jerin Jacob <jerinjacobk@gmail.com>
To: Ferruh Yigit <ferruh.yigit@amd.com>
Cc: jerinj@marvell.com, dev@dpdk.org,
Thomas Monjalon <thomas@monjalon.net>,
Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>,
ferruh.yigit@xilinx.com, ajit.khaparde@broadcom.com,
aboyer@pensando.io, beilei.xing@intel.com,
bruce.richardson@intel.com, chas3@att.com, chenbo.xia@intel.com,
ciara.loftus@intel.com, dsinghrawat@marvell.com,
ed.czeck@atomicrules.com, evgenys@amazon.com, grive@u256.net,
g.singh@nxp.com, zhouguoyang@huawei.com, haiyue.wang@intel.com,
hkalra@marvell.com, heinrich.kuhn@corigine.com,
hemant.agrawal@nxp.com, hyonkim@cisco.com, igorch@amazon.com,
irusskikh@marvell.com, jgrajcia@cisco.com,
jasvinder.singh@intel.com, jianwang@trustnetic.com,
jiawenwu@trustnetic.com, jingjing.wu@intel.com,
johndale@cisco.com, john.miller@atomicrules.com,
linville@tuxdriver.com, keith.wiles@intel.com,
kirankumark@marvell.com, oulijun@huawei.com, lironh@marvell.com,
longli@microsoft.com, mw@semihalf.com, spinler@cesnet.cz,
matan@nvidia.com, matt.peters@windriver.com,
maxime.coquelin@redhat.com, mk@semihalf.com, humin29@huawei.com,
pnalla@marvell.com, ndabilpuram@marvell.com,
qiming.yang@intel.com, qi.z.zhang@intel.com, radhac@marvell.com,
rahul.lakkireddy@chelsio.com, rmody@marvell.com,
rosen.xu@intel.com, sachin.saxena@oss.nxp.com,
skoteshwar@marvell.com, shshaikh@marvell.com,
shaibran@amazon.com, shepard.siegel@atomicrules.com,
asomalap@amd.com, somnath.kotur@broadcom.com,
sthemmin@microsoft.com, steven.webster@windriver.com,
skori@marvell.com, mtetsuyah@gmail.com, vburru@marvell.com,
viacheslavo@nvidia.com, xiao.w.wang@intel.com,
cloud.wangxiaoyun@huawei.com, yisen.zhuang@huawei.com,
yongwang@vmware.com, xuanziyang2@huawei.com,
cristian.dumitrescu@intel.com
Subject: Re: [dpdk-dev] [v1] ethdev: support Tx queue used count
Date: Thu, 18 Jan 2024 14:36:21 +0530 [thread overview]
Message-ID: <CALBAE1NxtWzz0DZr5Nm04aSiO9xWiZWwNB_wevFM+_Rnakynkw@mail.gmail.com> (raw)
In-Reply-To: <5251f00a-a039-460d-805b-0325cfced32b@amd.com>
On Fri, Jan 12, 2024 at 5:04 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>
> On 1/11/2024 3:17 PM, jerinj@marvell.com wrote:
> > From: Jerin Jacob <jerinj@marvell.com>
> >
> > Introduce a new API to retrieve the number of used descriptors
> > in a Tx queue. Applications can leverage this API in the fast path to
> > inspect the Tx queue occupancy and take appropriate actions based on the
> > available free descriptors.
> >
> > A notable use case could be implementing Random Early Discard (RED)
> > in software based on Tx queue occupancy.
> >
> > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > ---
> >
>
> As we are adding a new API and dev_ops, is a driver implementation and
> testpmd/example implementation planned for this release?
Yes.
>
>
> > rfc..v1:
> > - Updated API similar to rte_eth_rx_queue_count() where it returns
> > "used" count instead of "free" count
> >
> > diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst
> > index f7d9980849..0d5a8733fc 100644
> > --- a/doc/guides/nics/features.rst
> > +++ b/doc/guides/nics/features.rst
> > @@ -962,6 +962,16 @@ management (see :doc:`../prog_guide/power_man` for more details).
> >
> > * **[implements] eth_dev_ops**: ``get_monitor_addr``
> >
> > +.. _nic_features_tx_queue_used_count:
> > +
> > +Tx queue count
> > +--------------
> > +
> > +Supports to get the number of used descriptors of a Tx queue.
> > +
> > +* **[implements] eth_dev_ops**: ``tx_queue_count``.
> > +* **[related] API**: ``rte_eth_tx_queue_count()``.
> > +
> >
>
> Can you please keep the order same with 'default.ini' file,
> I recognized there is already some mismatch in order but we can fix them
> later.
Ack
>
> > .. _nic_features_other:
> >
> > Other dev ops not represented by a Feature
> > diff --git a/doc/guides/nics/features/default.ini b/doc/guides/nics/features/default.ini
> > index 806cb033ff..3ef6d45c0e 100644
> > --- a/doc/guides/nics/features/default.ini
> > +++ b/doc/guides/nics/features/default.ini
> > @@ -59,6 +59,7 @@ Packet type parsing =
> > Timesync =
> > Rx descriptor status =
> > Tx descriptor status =
> > +Tx queue count =
> >
>
> Existing Rx queue count is not documented, if we are documenting this,
> can you please add "Rx queue count" in a separate patch?
I will do.
>
> > Basic stats =
> > Extended stats =
> > Stats per queue =
> > diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h
> > index b482cd12bb..f05f68a67c 100644
> > --- a/lib/ethdev/ethdev_driver.h
> > +++ b/lib/ethdev/ethdev_driver.h
> > @@ -58,6 +58,8 @@ struct rte_eth_dev {
> > eth_rx_queue_count_t rx_queue_count;
> > /** Check the status of a Rx descriptor */
> > eth_rx_descriptor_status_t rx_descriptor_status;
> > + /** Get the number of used Tx descriptors */
> > + eth_tx_queue_count_t tx_queue_count;
> > /** Check the status of a Tx descriptor */
> > eth_tx_descriptor_status_t tx_descriptor_status;
> > /** Pointer to PMD transmit mbufs reuse function */
> > diff --git a/lib/ethdev/ethdev_private.c b/lib/ethdev/ethdev_private.c
> > index a656df293c..626524558a 100644
> > --- a/lib/ethdev/ethdev_private.c
> > +++ b/lib/ethdev/ethdev_private.c
> > @@ -273,6 +273,7 @@ eth_dev_fp_ops_setup(struct rte_eth_fp_ops *fpo,
> > fpo->tx_pkt_prepare = dev->tx_pkt_prepare;
> > fpo->rx_queue_count = dev->rx_queue_count;
> > fpo->rx_descriptor_status = dev->rx_descriptor_status;
> > + fpo->tx_queue_count = dev->tx_queue_count;
> > fpo->tx_descriptor_status = dev->tx_descriptor_status;
> > fpo->recycle_tx_mbufs_reuse = dev->recycle_tx_mbufs_reuse;
> > fpo->recycle_rx_descriptors_refill = dev->recycle_rx_descriptors_refill;
> > diff --git a/lib/ethdev/ethdev_trace_points.c b/lib/ethdev/ethdev_trace_points.c
> > index 91f71d868b..e618414392 100644
> > --- a/lib/ethdev/ethdev_trace_points.c
> > +++ b/lib/ethdev/ethdev_trace_points.c
> > @@ -481,6 +481,9 @@ RTE_TRACE_POINT_REGISTER(rte_eth_trace_count_aggr_ports,
> > RTE_TRACE_POINT_REGISTER(rte_eth_trace_map_aggr_tx_affinity,
> > lib.ethdev.map_aggr_tx_affinity)
> >
> > +RTE_TRACE_POINT_REGISTER(rte_eth_trace_tx_queue_count,
> > + lib.ethdev.tx_queue_count)
> > +
>
> Can you please group this with 'tx_burst' & 'call_tx_callbacks' above?
Ack
>
> > RTE_TRACE_POINT_REGISTER(rte_flow_trace_copy,
> > lib.ethdev.flow.copy)
> >
> > diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h
> > index 21e3a21903..af59da9652 100644
> > --- a/lib/ethdev/rte_ethdev.h
> > +++ b/lib/ethdev/rte_ethdev.h
> > @@ -6803,6 +6803,80 @@ rte_eth_recycle_mbufs(uint16_t rx_port_id, uint16_t rx_queue_id,
> > __rte_experimental
> > int rte_eth_buffer_split_get_supported_hdr_ptypes(uint16_t port_id, uint32_t *ptypes, int num);
> >
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
> > + *
> > + * Get the number of used descriptors of a Tx queue
> > + *
> > + * This function retrieves the number of used descriptors of a transmit queue.
> > + * Applications can use this API in the fast path to inspect Tx queue occupancy and take
> > + * appropriate actions based on the available free descriptors.
> > + * An example action could be implementing the Random Early Discard (RED).
> >
>
> Above is good, but do you think does it help to mention that intended
> usacase is QoS, to address the risk Bruce mentioned?
I will add following notes in next version.
* @note There is no requirement to call this function before
rte_eth_tx_burst() invocation.
* @note Utilize this function exclusively when the caller needs to
determine the used queue count
* across all descriptors of a Tx queue. If the use case only involves
checking the status of a
* specific descriptor slot, opt for rte_eth_tx_descriptor_status() instead.
>
>
> > + *
> > + * Since it's a fast-path function, no check is performed on port_id and
> > + * tx_queue_id. The caller must therefore ensure that the port is enabled
> > + * and the queue is configured and running.
> > + *
> > + * @param port_id
> > + * The port identifier of the device.
> > + * @param tx_queue_id
> > + * The index of the transmit queue.
> > + * The value must be in the range [0, nb_tx_queue - 1] previously supplied
> > + * to rte_eth_dev_configure().
> > + * @return
> > + * The number of used descriptors in the specific queue, or:
> > + * - (-ENODEV) if *port_id* is invalid. Enabled only when RTE_ETHDEV_DEBUG_TX is enabled
> > + * - (-EINVAL) if *queue_id* is invalid. Enabled only when RTE_ETHDEV_DEBUG_TX is enabled
> > + * - (-ENOTSUP) if the device does not support this function.
> > + *
> > + * @note This function is designed for fast-path use.
> > + */
> > +__rte_experimental
> > +static inline int
> > +rte_eth_tx_queue_count(uint16_t port_id, uint16_t tx_queue_id)
> > +{
> > + struct rte_eth_fp_ops *fops;
> > + void *qd;
> > + int rc;
> > +
> > +#ifdef RTE_ETHDEV_DEBUG_TX
> > + if (port_id >= RTE_MAX_ETHPORTS || !rte_eth_dev_is_valid_port(port_id)) {
> > + RTE_ETHDEV_LOG_LINE(ERR, "Invalid port_id=%u", port_id);
> > + rc = -ENODEV;
> > + rte_eth_trace_tx_queue_count(port_id, tx_queue_id, rc);
> > + return rc;
> > + }
> > +
> > + rc = -EINVAL;
> > + if (tx_queue_id >= RTE_MAX_QUEUES_PER_PORT) {
> > + RTE_ETHDEV_LOG_LINE(ERR, "Invalid Tx queue_id=%u for port_id=%u",
> > + tx_queue_id, port_id);
> > + rte_eth_trace_tx_queue_count(port_id, tx_queue_id, rc);
> > + return rc;
> > + }
> > +#endif
> > +
> > + /* Fetch pointer to Tx queue data */
> > + fops = &rte_eth_fp_ops[port_id];
> > + qd = fops->txq.data[tx_queue_id];
> > +
> > +#ifdef RTE_ETHDEV_DEBUG_TX
> > + if (qd == NULL) {
> > + RTE_ETHDEV_LOG_LINE(ERR, "Invalid Tx queue_id=%u for port_id=%u",
> > + tx_queue_id, port_id);
> > + rte_eth_trace_tx_queue_count(port_id, tx_queue_id, rc);
> > + return rc;
> > + }
> > +#endif
> > + if (fops->tx_queue_count == NULL)
> > + return -ENOTSUP;
> > +
> > + rc = fops->tx_queue_count(qd);
> >
>
> Can you please put an empty line here to have a separate block for trace
> code?
Ack
>
> > + rte_eth_trace_tx_queue_count(port_id, tx_queue_id, rc);
> > +
> > + return rc;
> > +}
> > #ifdef __cplusplus
> > }
> > #endif
> > diff --git a/lib/ethdev/rte_ethdev_core.h b/lib/ethdev/rte_ethdev_core.h
> > index 4bfaf79c6c..d3f09f390d 100644
> > --- a/lib/ethdev/rte_ethdev_core.h
> > +++ b/lib/ethdev/rte_ethdev_core.h
> > @@ -60,6 +60,9 @@ typedef uint16_t (*eth_recycle_tx_mbufs_reuse_t)(void *txq,
> > /** @internal Refill Rx descriptors with the recycling mbufs */
> > typedef void (*eth_recycle_rx_descriptors_refill_t)(void *rxq, uint16_t nb);
> >
> > +/** @internal Get number of used descriptors on a transmit queue. */
> > +typedef int (*eth_tx_queue_count_t)(void *txq);
> > +
>
> Can you please move it above 'tx_descriptor_status', to keep same order
> kept in many other locations:
> rx_queue_count
> rx_descriptor_status
> tx_queue_count
> tx_descriptor_status
Ack
>
>
> > /**
> > * @internal
> > * Structure used to hold opaque pointers to internal ethdev Rx/Tx
> > @@ -116,7 +119,9 @@ struct rte_eth_fp_ops {
> > eth_tx_descriptor_status_t tx_descriptor_status;
> > /** Copy used mbufs from Tx mbuf ring into Rx. */
> > eth_recycle_tx_mbufs_reuse_t recycle_tx_mbufs_reuse;
> > - uintptr_t reserved2[2];
> > + /** Get the number of used Tx descriptors. */
> > + eth_tx_queue_count_t tx_queue_count;
> >
>
> Similarly, can you please move it above 'tx_descriptor_status'?
Ack
>
>
> > + uintptr_t reserved2[1];
> > /**@}*/
> >
> > } __rte_cache_aligned;
> > diff --git a/lib/ethdev/rte_ethdev_trace_fp.h b/lib/ethdev/rte_ethdev_trace_fp.h
> > index 186271c9ff..c98c488433 100644
> > --- a/lib/ethdev/rte_ethdev_trace_fp.h
> > +++ b/lib/ethdev/rte_ethdev_trace_fp.h
> > @@ -73,6 +73,14 @@ RTE_TRACE_POINT_FP(
> > rte_trace_point_emit_u64(count);
> > )
> >
> > +RTE_TRACE_POINT_FP(
> > + rte_eth_trace_tx_queue_count,
> > + RTE_TRACE_POINT_ARGS(uint16_t port_id, uint16_t tx_queue_id, int rc),
> > + rte_trace_point_emit_u16(port_id);
> > + rte_trace_point_emit_u16(tx_queue_id);
> > + rte_trace_point_emit_int(rc);
> > +)
> > +
>
> Existing 'rx_queue_count' is missing tracepoints, for consistency, can
> you please add that too in a separate patch?
I will do.
>
>
> > #ifdef __cplusplus
> > }
> > #endif
> > diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map
> > index 5c4917c020..e03830902a 100644
> > --- a/lib/ethdev/version.map
> > +++ b/lib/ethdev/version.map
> > @@ -316,6 +316,9 @@ EXPERIMENTAL {
> > rte_eth_recycle_rx_queue_info_get;
> > rte_flow_group_set_miss_actions;
> > rte_flow_calc_table_hash;
> > +
> > + # added in 24.03
> > + rte_eth_tx_queue_count;
> > };
> >
> > INTERNAL {
>
next prev parent reply other threads:[~2024-01-18 9:06 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-19 17:29 [dpdk-dev] [RFC] ethdev: support Tx queue free descriptor query jerinj
2024-01-04 13:16 ` Dumitrescu, Cristian
2024-01-04 13:35 ` Jerin Jacob
2024-01-04 14:21 ` Konstantin Ananyev
2024-01-04 18:29 ` Thomas Monjalon
2024-01-05 9:57 ` Jerin Jacob
2024-01-05 10:03 ` Thomas Monjalon
2024-01-05 11:12 ` Konstantin Ananyev
2024-01-08 20:54 ` Morten Brørup
2024-01-09 14:45 ` Jerin Jacob
2024-01-04 21:17 ` Thomas Monjalon
2024-01-05 9:54 ` Jerin Jacob
2024-01-05 10:02 ` Thomas Monjalon
2024-01-08 10:54 ` Bruce Richardson
2024-01-08 21:15 ` Morten Brørup
2024-01-09 8:47 ` Bruce Richardson
2024-01-12 10:56 ` Ferruh Yigit
2024-01-11 15:17 ` [dpdk-dev] [v1] ethdev: support Tx queue used count jerinj
2024-01-11 16:17 ` Andrew Rybchenko
2024-01-12 6:56 ` Jerin Jacob
2024-01-11 16:20 ` Morten Brørup
2024-01-12 6:59 ` Jerin Jacob
2024-01-11 17:00 ` Stephen Hemminger
2024-01-12 7:01 ` Jerin Jacob
2024-01-12 16:30 ` Stephen Hemminger
2024-01-12 8:02 ` David Marchand
2024-01-12 9:29 ` Jerin Jacob
2024-01-12 11:34 ` Ferruh Yigit
2024-01-12 12:11 ` David Marchand
2024-01-12 14:25 ` Ferruh Yigit
2024-01-12 12:29 ` Morten Brørup
2024-01-12 14:29 ` Ferruh Yigit
2024-01-18 9:06 ` Jerin Jacob [this message]
2024-01-12 12:33 ` Konstantin Ananyev
2024-01-16 6:37 ` Jerin Jacob
2024-01-18 10:17 ` Konstantin Ananyev
2024-01-18 11:21 ` Jerin Jacob
2024-01-18 13:36 ` Morten Brørup
2024-01-19 9:52 ` Konstantin Ananyev
2024-01-19 10:32 ` Morten Brørup
2024-01-12 16:52 ` Stephen Hemminger
2024-01-18 9:47 ` [dpdk-dev] [v2] " jerinj
2024-01-22 13:00 ` Konstantin Ananyev
2024-01-23 11:46 ` Ferruh Yigit
2024-02-07 20:30 ` Ferruh Yigit
2024-01-29 15:03 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CALBAE1NxtWzz0DZr5Nm04aSiO9xWiZWwNB_wevFM+_Rnakynkw@mail.gmail.com \
--to=jerinjacobk@gmail.com \
--cc=aboyer@pensando.io \
--cc=ajit.khaparde@broadcom.com \
--cc=andrew.rybchenko@oktetlabs.ru \
--cc=asomalap@amd.com \
--cc=beilei.xing@intel.com \
--cc=bruce.richardson@intel.com \
--cc=chas3@att.com \
--cc=chenbo.xia@intel.com \
--cc=ciara.loftus@intel.com \
--cc=cloud.wangxiaoyun@huawei.com \
--cc=cristian.dumitrescu@intel.com \
--cc=dev@dpdk.org \
--cc=dsinghrawat@marvell.com \
--cc=ed.czeck@atomicrules.com \
--cc=evgenys@amazon.com \
--cc=ferruh.yigit@amd.com \
--cc=ferruh.yigit@xilinx.com \
--cc=g.singh@nxp.com \
--cc=grive@u256.net \
--cc=haiyue.wang@intel.com \
--cc=heinrich.kuhn@corigine.com \
--cc=hemant.agrawal@nxp.com \
--cc=hkalra@marvell.com \
--cc=humin29@huawei.com \
--cc=hyonkim@cisco.com \
--cc=igorch@amazon.com \
--cc=irusskikh@marvell.com \
--cc=jasvinder.singh@intel.com \
--cc=jerinj@marvell.com \
--cc=jgrajcia@cisco.com \
--cc=jianwang@trustnetic.com \
--cc=jiawenwu@trustnetic.com \
--cc=jingjing.wu@intel.com \
--cc=john.miller@atomicrules.com \
--cc=johndale@cisco.com \
--cc=keith.wiles@intel.com \
--cc=kirankumark@marvell.com \
--cc=linville@tuxdriver.com \
--cc=lironh@marvell.com \
--cc=longli@microsoft.com \
--cc=matan@nvidia.com \
--cc=matt.peters@windriver.com \
--cc=maxime.coquelin@redhat.com \
--cc=mk@semihalf.com \
--cc=mtetsuyah@gmail.com \
--cc=mw@semihalf.com \
--cc=ndabilpuram@marvell.com \
--cc=oulijun@huawei.com \
--cc=pnalla@marvell.com \
--cc=qi.z.zhang@intel.com \
--cc=qiming.yang@intel.com \
--cc=radhac@marvell.com \
--cc=rahul.lakkireddy@chelsio.com \
--cc=rmody@marvell.com \
--cc=rosen.xu@intel.com \
--cc=sachin.saxena@oss.nxp.com \
--cc=shaibran@amazon.com \
--cc=shepard.siegel@atomicrules.com \
--cc=shshaikh@marvell.com \
--cc=skori@marvell.com \
--cc=skoteshwar@marvell.com \
--cc=somnath.kotur@broadcom.com \
--cc=spinler@cesnet.cz \
--cc=steven.webster@windriver.com \
--cc=sthemmin@microsoft.com \
--cc=thomas@monjalon.net \
--cc=vburru@marvell.com \
--cc=viacheslavo@nvidia.com \
--cc=xiao.w.wang@intel.com \
--cc=xuanziyang2@huawei.com \
--cc=yisen.zhuang@huawei.com \
--cc=yongwang@vmware.com \
--cc=zhouguoyang@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).