From: Jianbo Liu <jianbo.liu@linaro.org>
To: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"Zhang, Helin" <helin.zhang@intel.com>,
"jerin.jacob@caviumnetworks.com"
<jerin.jacob@caviumnetworks.com>
Subject: Re: [dpdk-dev] [PATCH 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function
Date: Sat, 4 Feb 2017 11:37:26 +0800 [thread overview]
Message-ID: <CAP4Qi39SAgzZhYy9-RWHC4v=p-Bf30s32FnA8jSFqviW94YXSA@mail.gmail.com> (raw)
In-Reply-To: <2601191342CEEE43887BDE71AB9772583F1109B4@irsmsx105.ger.corp.intel.com>
On 3 February 2017 at 19:38, Ananyev, Konstantin
<konstantin.ananyev@intel.com> wrote:
>
>
>> -----Original Message-----
>> From: Jianbo Liu [mailto:jianbo.liu@linaro.org]
>> Sent: Friday, February 3, 2017 6:22 AM
>> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
>> Cc: dev@dpdk.org; Zhang, Helin <helin.zhang@intel.com>; jerin.jacob@caviumnetworks.com
>> Subject: Re: [PATCH 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function
>>
>> On 2 February 2017 at 00:19, Ananyev, Konstantin
>> <konstantin.ananyev@intel.com> wrote:
>> > Hi,
>> >
>> >> -----Original Message-----
>> >> From: Jianbo Liu [mailto:jianbo.liu@linaro.org]
>> >> Sent: Monday, December 19, 2016 6:09 AM
>> >> To: dev@dpdk.org; Zhang, Helin <helin.zhang@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>;
>> >> jerin.jacob@caviumnetworks.com
>> >> Cc: Jianbo Liu <jianbo.liu@linaro.org>
>> >> Subject: [PATCH 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function
>> >>
>> >> To get better performance, Rx bulk alloc recv function will scan 8 descriptors
>> >> in one time, but the statuses are not consistent on ARM platform because
>> >> the memory allocated for Rx descriptors is cacheable hugepages.
>> >> This patch is to calculate the number of received packets by scanning DD bit
>> >> sequentially, and stops when meeting the first packet with DD bit unset.
>> >>
>> >> Signed-off-by: Jianbo Liu <jianbo.liu@linaro.org>
>> >> ---
>> >> drivers/net/ixgbe/ixgbe_rxtx.c | 12 ++++++++----
>> >> 1 file changed, 8 insertions(+), 4 deletions(-)
>> >>
>> >> diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
>> >> index b2d9f45..2866bdb 100644
>> >> --- a/drivers/net/ixgbe/ixgbe_rxtx.c
>> >> +++ b/drivers/net/ixgbe/ixgbe_rxtx.c
>> >> @@ -1402,17 +1402,21 @@ ixgbe_rx_scan_hw_ring(struct ixgbe_rx_queue *rxq)
>> >> for (i = 0; i < RTE_PMD_IXGBE_RX_MAX_BURST;
>> >> i += LOOK_AHEAD, rxdp += LOOK_AHEAD, rxep += LOOK_AHEAD) {
>> >> /* Read desc statuses backwards to avoid race condition */
>> >> - for (j = LOOK_AHEAD-1; j >= 0; --j)
>> >> + for (j = LOOK_AHEAD - 1; j >= 0; --j) {
>> >> s[j] = rte_le_to_cpu_32(rxdp[j].wb.upper.status_error);
>> >> -
>> >> - for (j = LOOK_AHEAD - 1; j >= 0; --j)
>> >> pkt_info[j] = rte_le_to_cpu_32(rxdp[j].wb.lower.
>> >> lo_dword.data);
>> >> + }
>> >> +
>> >> + rte_smp_rmb();
>> >
>> > If reads can be reordered, shouldn't we fill pkt_info[] after smp_rmb() here?
>>
>> The barrier is to forbid the reordering from the following readings,
>> which will count the number of actual received packets.
>
> What I meant is that if you'll keep reading from both rxdp[].wb.lower and rxdp[].wb.upper
> before rmb, then nothing would prevent cpu from reorder these reads in any way it likes
> (if we are talking about cpus with read reordering allowed), right?
> So it can end up with the following order:
>
> rxdp[N].wb.lower
> rxdp[N].wb.upper
>
> or even:
>
> rxdp[N-1].wb.lower
> rxdp[N].wb.lower
> rxdp[N-1].wb.upper
> rxdp[N].wb.upper
>
> In such cases pkt_info[] may contain invalid data.
Yes, it's possible. I'll send v2.
Thanks!
>
>> And as wb.uper and wb.lower of one descriptor are in the same
>> cacheline, could it be better to read them at the same time?.
>
> It could be, but I think for the sake of data integrity we have to make sure that
> cpu would never read any other RXD field before wb.upper. status_error, see above.
>
> BTW, the following code might re-read both wb.upper and wb.lower anyway.
> So I don't think you'll save many cycles here anyway.
>
>>
>> > As another nit - with rmb() in and because you are looking the first gap in s[] now,
>> > no need to read TXDs in backward order.
>>
>> Reading backward is just to keep as it is for x86 platform.
>
> With the change you introducing, I don't think it is necessary any more.
>
> Konstantin
>
>>
>> > How it looks to me (as a suggestion):
>> >
>> > for (j = 0; j != LOOK_AHEAD; j++)
>> > s[j] = rte_le_to_cpu_32(rxdp[j].wb.upper.status_error);
>> >
>> > rte_smp_rmb();
>> >
>> > for (j = 0; j < LOOK_AHEAD && (s[j] & IXGBE_RXDADV_STAT_DD) != 0; j++)
>> > ;
>> >
>> > for (j = 0; j < nb_dd; ++j) {
>> > pkt_info[j] = rte_le_to_cpu_32(rxdp[j].wb.lower.lo_dword.data);
>> > ....
>> >
>> > Konstantin
>> >
>> >
>> >>
>> >> /* Compute how many status bits were set */
>> >> nb_dd = 0;
>> >> for (j = 0; j < LOOK_AHEAD; ++j)
>> >> - nb_dd += s[j] & IXGBE_RXDADV_STAT_DD;
>> >> + if (s[j] & IXGBE_RXDADV_STAT_DD)
>> >> + ++nb_dd;
>> >> + else
>> >> + break;
>> >>
>> >> nb_rx += nb_dd;
>> >>
>> >> --
>> >> 2.4.11
>> >
next prev parent reply other threads:[~2017-02-04 3:37 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-19 6:09 Jianbo Liu
2016-12-19 6:09 ` [dpdk-dev] [PATCH 2/2] net/ixgbe: calculate correct number of received packets for ARM NEON-version vPMD Jianbo Liu
2016-12-21 10:08 ` Jerin Jacob
2016-12-21 11:03 ` Bruce Richardson
2016-12-22 1:18 ` Jianbo Liu
2016-12-22 1:05 ` Jianbo Liu
2017-02-01 16:19 ` [dpdk-dev] [PATCH 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function Ananyev, Konstantin
2017-02-03 6:22 ` Jianbo Liu
2017-02-03 11:38 ` Ananyev, Konstantin
2017-02-04 3:37 ` Jianbo Liu [this message]
2017-02-04 9:37 ` [dpdk-dev] [PATCH v2 " Jianbo Liu
2017-02-04 9:37 ` [dpdk-dev] [PATCH v2 2/2] net/ixgbe: calculate correct number of received packets for ARM NEON-version vPMD Jianbo Liu
2017-02-04 13:26 ` [dpdk-dev] [PATCH v2 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function Ananyev, Konstantin
2017-02-08 18:02 ` Ferruh Yigit
2017-02-08 18:53 ` Ananyev, Konstantin
2017-02-08 19:53 ` Ananyev, Konstantin
2017-02-09 3:49 ` Jianbo Liu
2017-02-04 16:37 ` [dpdk-dev] [PATCH v3 " Jianbo Liu
2017-02-04 16:37 ` [dpdk-dev] [PATCH v3 2/2] net/ixgbe: calculate correct number of received packets for ARM NEON-version vPMD Jianbo Liu
2017-02-04 16:39 ` [dpdk-dev] [PATCH v3 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function Jianbo Liu
2017-02-09 4:05 ` [dpdk-dev] [PATCH v4 " Jianbo Liu
2017-02-09 4:05 ` [dpdk-dev] [PATCH v4 2/2] net/ixgbe: calculate correct number of received packets for ARM NEON-version vPMD Jianbo Liu
2017-02-09 12:43 ` Ferruh Yigit
2017-02-09 12:39 ` [dpdk-dev] [PATCH v4 1/2] net/ixgbe: calculate the correct number of received packets in bulk alloc function Ferruh Yigit
2017-02-09 12:42 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAP4Qi39SAgzZhYy9-RWHC4v=p-Bf30s32FnA8jSFqviW94YXSA@mail.gmail.com' \
--to=jianbo.liu@linaro.org \
--cc=dev@dpdk.org \
--cc=helin.zhang@intel.com \
--cc=jerin.jacob@caviumnetworks.com \
--cc=konstantin.ananyev@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).