From: David Marchand <david.marchand@redhat.com>
To: Ferruh Yigit <ferruh.yigit@amd.com>
Cc: dev@dpdk.org, Aman Singh <aman.deep.singh@intel.com>,
Yuying Zhang <yuying.zhang@intel.com>,
Robin Jarry <rjarry@redhat.com>
Subject: Re: [PATCH 6/6] app/testpmd: factorize fwd engine Tx
Date: Mon, 20 Feb 2023 17:33:40 +0100 [thread overview]
Message-ID: <CAJFAV8xmVzZk=XqPJFfaoSLbWetbKXf1oX0VC0uyDpzP11KxaQ@mail.gmail.com> (raw)
In-Reply-To: <e6a8232a-a3f7-32a6-09c8-e9aa834e989b@amd.com>
On Tue, Feb 14, 2023 at 7:16 PM Ferruh Yigit <ferruh.yigit@amd.com> wrote:
>
> On 1/24/2023 10:47 AM, David Marchand wrote:
> > Reduce code duplication by introducing a helper that takes care of
> > transmitting, retrying if enabled and incrementing tx counter.
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
>
> <...>
>
> > diff --git a/app/test-pmd/noisy_vnf.c b/app/test-pmd/noisy_vnf.c
> > index 937d5a1d7d..3875590132 100644
> > --- a/app/test-pmd/noisy_vnf.c
> > +++ b/app/test-pmd/noisy_vnf.c
> > @@ -93,30 +93,6 @@ sim_memory_lookups(struct noisy_config *ncf, uint16_t nb_pkts)
> > }
> > }
> >
> > -static uint16_t
> > -do_retry(uint16_t nb_rx, uint16_t nb_tx, struct rte_mbuf **pkts,
> > - struct fwd_stream *fs)
> > -{
> > - uint32_t retry = 0;
> > -
> > - while (nb_tx < nb_rx && retry++ < burst_tx_retry_num) {
> > - rte_delay_us(burst_tx_delay_time);
> > - nb_tx += rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
> > - &pkts[nb_tx], nb_rx - nb_tx);
> > - }
> > -
> > - return nb_tx;
> > -}
> > -
> > -static uint32_t
> > -drop_pkts(struct rte_mbuf **pkts, uint16_t nb_rx, uint16_t nb_tx)
> > -{
> > - if (nb_tx < nb_rx)
> > - rte_pktmbuf_free_bulk(&pkts[nb_tx], nb_rx - nb_tx);
> > -
> > - return nb_rx - nb_tx;
> > -}
> > -
> > /*
> > * Forwarding of packets in noisy VNF mode. Forward packets but perform
> > * memory operations first as specified on cmdline.
> > @@ -156,38 +132,23 @@ pkt_burst_noisy_vnf(struct fwd_stream *fs)
> >
> > if (!ncf->do_buffering) {
> > sim_memory_lookups(ncf, nb_rx);
> > - nb_tx = rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
> > - pkts_burst, nb_rx);
> > - if (unlikely(nb_tx < nb_rx) && fs->retry_enabled)
> > - nb_tx += do_retry(nb_rx, nb_tx, pkts_burst, fs);
> > - inc_tx_burst_stats(fs, nb_tx);
> > - fs->tx_packets += nb_tx;
> > - fs->fwd_dropped += drop_pkts(pkts_burst, nb_rx, nb_tx);
> > + nb_tx = common_fwd_stream_transmit(fs, pkts_burst, nb_rx);
> >
>
> 'nb_tx' is not used or necessary in this context, so assignment is not
> necassary.
>
> PS:
> In the latest next-net head, there is a 'goto' here instead of return,
> but that is becuase of recording cycles, becuase of optimization in this
> set (patch 1/6) that needs to turn back to 'return' that is what I did
> while applying patch.
This part changed a bit after rebasing and I think nb_tx is still
needed in this context.
Please have a look at v2.
>
> > kreturn true;
> > }
> >
> > fifo_free = rte_ring_free_count(ncf->f);
> > if (fifo_free >= nb_rx) {
> > - nb_enqd = rte_ring_enqueue_burst(ncf->f,
> > - (void **) pkts_burst, nb_rx, NULL);
> > - if (nb_enqd < nb_rx)
> > - fs->fwd_dropped += drop_pkts(pkts_burst,
> > - nb_rx, nb_enqd);
> > - } else {
> > - nb_deqd = rte_ring_dequeue_burst(ncf->f,
> > - (void **) tmp_pkts, nb_rx, NULL);
> > - nb_enqd = rte_ring_enqueue_burst(ncf->f,
> > - (void **) pkts_burst, nb_deqd, NULL);
> > - if (nb_deqd > 0) {
> > - nb_tx = rte_eth_tx_burst(fs->tx_port,
> > - fs->tx_queue, tmp_pkts,
> > - nb_deqd);
> > - if (unlikely(nb_tx < nb_rx) && fs->retry_enabled)
> > - nb_tx += do_retry(nb_rx, nb_tx, tmp_pkts, fs);
> > - inc_tx_burst_stats(fs, nb_tx);
> > - fs->fwd_dropped += drop_pkts(tmp_pkts, nb_deqd, nb_tx);
> > + nb_enqd = rte_ring_enqueue_burst(ncf->f, (void **) pkts_burst, nb_rx, NULL);
> > + if (nb_enqd < nb_rx) {
> > + fs->fwd_dropped += nb_rx - nb_enqd;
> > + rte_pktmbuf_free_bulk(&pkts_burst[nb_enqd], nb_rx - nb_enqd);
>
> Why not keep 'drop_pkts()' for this block, it is easier to read with it.
That would be the only case where drop_pkts() is used.
I am not convinced about readability but if you feel strongly about
it, I don't mind restoring it (in a v3, I'll post v2 unchanged on this
matter).
>
> > }
> > + } else {
> > + nb_deqd = rte_ring_dequeue_burst(ncf->f, (void **) tmp_pkts, nb_rx, NULL);
> > + nb_enqd = rte_ring_enqueue_burst(ncf->f, (void **) pkts_burst, nb_deqd, NULL);
> > + if (nb_deqd > 0)
> > + nb_tx = common_fwd_stream_transmit(fs, tmp_pkts, nb_deqd);
>
> 'nb_tx' assignment looks wrong,
> function returns 'nb_dropped' not 'nb_tx'. 'nb_tx' used below to detect
> if flush needed ('needs_flush'), so 'needs_flush' may be set wrong
> becuase dropped packet is used instead number of Tx packets.
Indeed, this is wrong, I had caught the issue when preparing v2 after
rebasing over Robin series.
I had changed common_fwd_stream_transmit() so it returns the number of
transmitted packets which is more in line with rte_eth_tx_burst().
This is easier to understand too.
>
> > }
> >
> > sim_memory_lookups(ncf, nb_enqd);
> > @@ -204,15 +165,9 @@ pkt_burst_noisy_vnf(struct fwd_stream *fs)
> > needs_flush = delta_ms >= noisy_tx_sw_buf_flush_time &&
> > noisy_tx_sw_buf_flush_time > 0 && !nb_tx;
> > while (needs_flush && !rte_ring_empty(ncf->f)) {
> > - unsigned int sent;
> > nb_deqd = rte_ring_dequeue_burst(ncf->f, (void **)tmp_pkts,
> > MAX_PKT_BURST, NULL);
> > - sent = rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
> > - tmp_pkts, nb_deqd);
> > - if (unlikely(sent < nb_deqd) && fs->retry_enabled)
> > - nb_tx += do_retry(nb_rx, nb_tx, tmp_pkts, fs);
> > - inc_tx_burst_stats(fs, nb_tx);
> > - fs->fwd_dropped += drop_pkts(tmp_pkts, nb_deqd, sent);
> > + nb_tx = common_fwd_stream_transmit(fs, tmp_pkts, nb_deqd);
>
> similaryly 'nb_tx' assignment can be wrong here, and 'nb_tx' used for
> return value to hint if record cycle is required, using number of
> dropped packets (what 'common_fwd_stream_transmit()' returns) gives
> wrong result.
Yes, and there are other issues in this part which I moved to a
separate patch for v2.
>
> > ncf->prev_time = rte_get_timer_cycles();
> > }
> >
> > diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
> > index e6b28b4748..71ff70f55b 100644
> > --- a/app/test-pmd/testpmd.h
> > +++ b/app/test-pmd/testpmd.h
> > @@ -870,6 +870,36 @@ common_fwd_stream_receive(struct fwd_stream *fs, struct rte_mbuf **burst,
> > return nb_rx;
> > }
> >
> > +/* Returns count of dropped packets. */
> > +static inline uint16_t
> > +common_fwd_stream_transmit(struct fwd_stream *fs, struct rte_mbuf **burst,
> > + unsigned int count)
>
> I would use 'nb_pkts' instead of 'count' as variable name since it is
> more common, but of course this is subjective.
Ok for me.
I did the same renaming for the rx helper.
>
> > +{
> > + uint16_t nb_tx;
> > + uint32_t retry;
> > +
> > + nb_tx = rte_eth_tx_burst(fs->tx_port, fs->tx_queue, burst, count);
> > + /*
> > + * Retry if necessary
> > + */
> > + if (unlikely(nb_tx < count) && fs->retry_enabled) {
> > + retry = 0;
> > + while (nb_tx < count && retry++ < burst_tx_retry_num) {
> > + rte_delay_us(burst_tx_delay_time);
> > + nb_tx += rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
> > + &burst[nb_tx], count - nb_tx);
> > + }
> > + }
> > + fs->tx_packets += nb_tx;
> > + inc_tx_burst_stats(fs, nb_tx);
> > + if (unlikely(nb_tx < count)) {
> > + fs->fwd_dropped += (count - nb_tx);
> > + rte_pktmbuf_free_bulk(&burst[nb_tx], count - nb_tx);
> > + }
> > +
> > + return count - nb_tx;
>
> Instead of returning number of dropped packets, what about returning
> number of packets sent ('nb_tx')?
> Intuitively this is what expected from a function named
> 'common_fwd_stream_transmit()', and even if it is more optimised to
> return number of dropped packet, this may have only a little impact on
> the caller code.
Ah ah, well, I thought the same after rebasing v1.
This change is in v2.
>
>
> And even 'fs->tx_packets' updated withing the function, updating it
> externally and explicitly feels me as it clarifies usage more, although
> this part is up to you, I mean usage like:
> fs->tx_packets += common_fwd_stream_transmit(fs, pkts, nb_pkts);
We would have fs->tx_packets managed by fwd engine code, but
fwd_dropped by the common helper?
I prefer fwd engines do not have to deal with the stats at all.
We have a lot of fwd engines, and small issues are often
(re-)introduced with new fwd engines.
A centralised approach is more robust.
Thanks Ferruh.
--
David Marchand
next prev parent reply other threads:[~2023-02-20 16:33 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-24 10:47 [PATCH 0/6] Testpmd code cleanup David Marchand
2023-01-24 10:47 ` [PATCH 1/6] app/testpmd: factorize core cycles record David Marchand
2023-02-14 18:14 ` Ferruh Yigit
2023-01-24 10:47 ` [PATCH 2/6] app/testpmd: don't send unprepared packets David Marchand
2023-02-14 18:14 ` Ferruh Yigit
2023-01-24 10:47 ` [PATCH 3/6] app/testpmd: bulk free mbufs David Marchand
2023-02-14 18:14 ` Ferruh Yigit
2023-01-24 10:47 ` [PATCH 4/6] app/testpmd: factorize fwd engine init David Marchand
2023-02-14 18:14 ` Ferruh Yigit
2023-01-24 10:47 ` [PATCH 5/6] app/testpmd: factorize fwd engine Rx David Marchand
2023-02-14 18:15 ` Ferruh Yigit
2023-01-24 10:47 ` [PATCH 6/6] app/testpmd: factorize fwd engine Tx David Marchand
2023-02-14 11:03 ` Singh, Aman Deep
2023-02-14 18:17 ` Ferruh Yigit
2023-02-16 8:01 ` Singh, Aman Deep
2023-02-16 10:07 ` Ferruh Yigit
2023-02-14 18:16 ` Ferruh Yigit
2023-02-20 16:33 ` David Marchand [this message]
2023-01-25 13:50 ` [PATCH 0/6] Testpmd code cleanup Robin Jarry
2023-02-14 18:22 ` Ferruh Yigit
2023-02-20 15:02 ` David Marchand
2023-02-20 16:40 ` [PATCH v2 0/9] " David Marchand
2023-02-20 16:40 ` [PATCH v2 1/9] app/testpmd: fix Tx preparation in checksum engine David Marchand
2023-02-20 16:40 ` [PATCH v2 2/9] app/testpmd: fix packet count in ieee15888 engine David Marchand
2023-02-20 16:40 ` [PATCH v2 3/9] app/testpmd: rework ieee1588 engine fwd configuration David Marchand
2023-02-24 9:11 ` Singh, Aman Deep
2023-02-20 16:40 ` [PATCH v2 4/9] app/testpmd: fix packet transmission in noisy VNF engine David Marchand
2023-02-20 16:40 ` [PATCH v2 5/9] app/testpmd: bulk free mbufs David Marchand
2023-02-20 16:41 ` [PATCH v2 6/9] app/testpmd: factorize core cycles record David Marchand
2023-02-20 16:41 ` [PATCH v2 7/9] app/testpmd: factorize fwd engine init David Marchand
2023-02-20 16:41 ` [PATCH v2 8/9] app/testpmd: factorize fwd engine Rx David Marchand
2023-02-20 16:41 ` [PATCH v2 9/9] app/testpmd: factorize fwd engine Tx David Marchand
2023-02-20 18:34 ` [PATCH v3 0/9] Testpmd code cleanup David Marchand
2023-02-20 18:34 ` [PATCH v3 1/9] app/testpmd: fix Tx preparation in checksum engine David Marchand
2023-02-20 18:34 ` [PATCH v3 2/9] app/testpmd: fix packet count in ieee15888 engine David Marchand
2023-02-24 8:24 ` Singh, Aman Deep
2023-02-20 18:34 ` [PATCH v3 3/9] app/testpmd: rework ieee1588 engine fwd configuration David Marchand
2023-02-28 18:50 ` Ferruh Yigit
2023-02-20 18:34 ` [PATCH v3 4/9] app/testpmd: fix packet transmission in noisy VNF engine David Marchand
2023-02-28 18:51 ` Ferruh Yigit
2023-02-20 18:34 ` [PATCH v3 5/9] app/testpmd: bulk free mbufs David Marchand
2023-02-20 18:45 ` Stephen Hemminger
2023-02-20 18:34 ` [PATCH v3 6/9] app/testpmd: factorize core cycles record David Marchand
2023-02-20 18:35 ` [PATCH v3 7/9] app/testpmd: factorize fwd engine init David Marchand
2023-02-20 18:35 ` [PATCH v3 8/9] app/testpmd: factorize fwd engine Rx David Marchand
2023-02-20 18:35 ` [PATCH v3 9/9] app/testpmd: factorize fwd engine Tx David Marchand
2023-02-28 18:35 ` Ferruh Yigit
2023-02-28 18:54 ` [PATCH v3 0/9] Testpmd code cleanup Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJFAV8xmVzZk=XqPJFfaoSLbWetbKXf1oX0VC0uyDpzP11KxaQ@mail.gmail.com' \
--to=david.marchand@redhat.com \
--cc=aman.deep.singh@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@amd.com \
--cc=rjarry@redhat.com \
--cc=yuying.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).