From: Andrew Rybchenko <arybchenko@solarflare.com>
To: Ilya Matveychikov <matvejchikov@gmail.com>,
Adrien Mazarguil <adrien.mazarguil@6wind.com>
Cc: <dev@dpdk.org>
Subject: Re: [dpdk-dev] A question about (poor) rte_ethdev internal rx/tx callbacks design
Date: Tue, 14 Nov 2017 09:24:27 +0300 [thread overview]
Message-ID: <70e9ecfe-2f65-980c-7e47-eb8734ad3256@solarflare.com> (raw)
In-Reply-To: <2FF46D73-66D4-4E6B-8509-DD0CEEFF12D3@gmail.com>
On 11/13/2017 10:33 PM, Ilya Matveychikov wrote:
>> On Nov 13, 2017, at 9:15 PM, Adrien Mazarguil <adrien.mazarguil@6wind.com> wrote:
>>
>> On Mon, Nov 13, 2017 at 02:56:23PM +0400, Ilya Matveychikov wrote:
>>>> On Nov 13, 2017, at 2:39 PM, Adrien Mazarguil <adrien.mazarguil@6wind.com> wrote:
>>>>
>>>> On Sat, Nov 11, 2017 at 09:18:45PM +0400, Ilya Matveychikov wrote:
>>>>> Folks,
>>>>>
>>>>> Are you serious with it:
>>>>>
>>>>> typedef uint16_t (*eth_rx_burst_t)(void *rxq,
>>>>> struct rte_mbuf **rx_pkts,
>>>>> uint16_t nb_pkts);
>>>>> typedef uint16_t (*eth_tx_burst_t)(void *txq,
>>>>> struct rte_mbuf **tx_pkts,
>>>>> uint16_t nb_pkts);
>>>>>
>>>>> I’m not surprised that every PMD stores port_id in every and each queue as having just the queue as an argument doesn’t allow to get the device. So the question is - why not to use something like:
>>>>>
>>>>> typedef uint16_t (*eth_rx_burst_t)(void *dev, uint16_t queue_id,
>>>>> struct rte_mbuf **rx_pkts,
>>>>> uint16_t nb_pkts);
>>>>> typedef uint16_t (*eth_tx_burst_t)(void *dev, uint16_t queue_id,
>>>>> struct rte_mbuf **tx_pkts,
>>>>> uint16_t nb_pkts);
>>>> I assume it's since the rte_eth_[rt]x_burst() wrappers already pay the price
>>>> for that indirection, doing it twice would be redundant.
>>> No need to do it twice, agree. We can pass dev pointer as well as queue, not just the queue’s
>>> index.
>>>
>>>> Basically the cost of storing a back-pointer to dev or a queue index in each
>>>> Rx/Tx queue structure is minor compared to saving a couple of CPU cycles
>>>> wherever we can.
>>> Not sure about it. More data to store - more cache space to occupy. Note that every queue has
>>> at least 4 bytes more than it actually needs. And RTE_MAX_QUEUES_PER_PORT is defined
>>> by it’s default to 1024. So we may have 4k extra for each port....
>> Note that queues are only allocated if requested by application, there's
>> really not much overhead involved.
> Yeah, mostly you are right here.
>
>> Also to echo Konstantin's reply and clarify mine, PMDs normally do not
>> access this structure from their data plane. This pointer, if needed, is
>> normally stored away from hot regions accessed during TX/RX, usually at the
>> end of TX/RX structures and only for the convenience of management
>> operations. It therefore has no measurable impact on the CPU cache.
>>
> I did a research of how drivers implements rx/tx queues and now I want to share the information
> and some thoughts about it (see the info at the end):
>
> 1) All drivers have tx/rx queues defined as structures
> 2) Current design implies that it’s enough to pass opaque rx/tx queue to the driver and frankly
> speaking it is. But..
> 3) Most of drivers wants to get not only the queue’s pointer but at least queue_id and port_id and
> most of them wants to have the pointer to internal devices’ data also.
>
> The way each driver solves (3) issue is data duplication. In other words, every queue used to have
> such the information (queue_id, port_id and dev_priv pointer) inside.
>
> My question was and still about such the design. Not sure that it’s the best way to do it keeping in
> mind that queue_id may be calculated using pointer difference and port_id may be stored just only
> once per device. But it’ll require to change internal interface, sure.
>
> And as I promised here is the result of the research on rx/tx queues:
<...>
> drivers/net/sfq:
> struct sfc_ef10_rxq { dp { queue_id, port_id } }
> struct sfc_ef10_txq { dp { queue_id, port_id } }
Which are not used on data/fast path (as Adrien state above). So, it
does not affect cache usage efficient etc.
next prev parent reply other threads:[~2017-11-14 6:24 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-11 17:18 Ilya Matveychikov
2017-11-12 2:43 ` Thomas Monjalon
2017-11-13 10:39 ` Adrien Mazarguil
2017-11-13 10:56 ` Ilya Matveychikov
2017-11-13 17:15 ` Adrien Mazarguil
2017-11-13 19:33 ` Ilya Matveychikov
2017-11-14 6:24 ` Andrew Rybchenko [this message]
2017-11-13 10:58 ` Ananyev, Konstantin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=70e9ecfe-2f65-980c-7e47-eb8734ad3256@solarflare.com \
--to=arybchenko@solarflare.com \
--cc=adrien.mazarguil@6wind.com \
--cc=dev@dpdk.org \
--cc=matvejchikov@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).