From: "Rao, Nikhil" <nikhil.rao@intel.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Cc: gage.eads@intel.com, dev@dpdk.org, thomas@monjalon.net,
bruce.richardson@intel.com, harry.van.haaren@intel.com,
hemant.agrawal@nxp.com, nipun.gupta@nxp.com,
narender.vangati@intel.com,
Abhinandan Gujjar <abhinandan.gujjar@intel.com>
Subject: Re: [dpdk-dev] [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues
Date: Tue, 1 Aug 2017 14:10:26 +0530 [thread overview]
Message-ID: <7b9ca757-f428-3675-b997-794ec6e96f2a@intel.com> (raw)
In-Reply-To: <20170729151252.GA25166@jerin>
On 7/29/2017 8:42 PM, Jerin Jacob wrote:
> -----Original Message-----
>> Date: Thu, 27 Jul 2017 16:28:29 +0530
>> From: "Rao, Nikhil" <nikhil.rao@intel.com>
>> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> CC: gage.eads@intel.com, dev@dpdk.org, thomas@monjalon.net,
>> bruce.richardson@intel.com, harry.van.haaren@intel.com,
>> hemant.agrawal@nxp.com, nipun.gupta@nxp.com, narender.vangati@intel.com,
>> Abhinandan Gujjar <abhinandan.gujjar@intel.com>, nikhil.rao@intel.com
>> Subject: Re: [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues
>> User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
>> Thunderbird/52.2.1
>>
>>
>>
>> In the case of a SW thread we would like to use the servicing weight
>> specified in the queue to do WRR across <ports, queues[]>, in keeping with
>
> OK, then lets work together to address in transparent manner where it
> works for HW and SW.
>
>> the adaper per <eventdev, eth port> model, one way to do this is to use the
>> same cfg.service_name in the rte_event_eth_rx_adapter_configure() call.
>>
>> However this creates a few difficulties/inconsistencies:
>
> I agree. If we are thinking about WRR across <ports,queues[]> then above
> proposal implementation creates inconsistencies. On the other side, it create challenges
> with HW implementation to have unified adapter API works for both HW and SW.
>
>>
>> 1)Service has the notion of a socket id. Multiple event dev IDs can be
>> included in the same service, each event dev has a socket ID -> this seems
>> to be an inconsistency that shouldn’t be allowed by design.
>>
>> 2)Say, the Rx event adapter doesn’t drop packets (could be configurable),
>> i.e, if events cannot be enqueued into the event device, these remain in a
>> buffer, when the buffer fills up packets aren’t dequeued from the eth
>> device.
>>
>> In the simplest case the Rx event adapter service has a single <event
>> device, event port> across multiple eth ports, it dequeues from the wrr[]
>> and buffers events, bulk enqueues BATCH_SIZE events into the <event device,
>> event port>.
>>
>> With adapters having different <event device, event port> code can be
>> optimized so that adapters that have a common <event device, event port> can
>> be made to refer to a common enqueue buffer { event dev, event port, buffer
>> } structure but this adds more book keeping in the code.
>>
>> 3)Every adapter can be configured with max_nb_rx ( a max nb of packets that
>> it can process in any invocation) – but the max_nb_rx seems like a service
>> level parameter instead of it being a summation across adapters.
>>
>> 1 & 3 could be solved by restricting the adapters to the same (as in the
>> first rte_event_eth_rx_adapter_configure() call) socket ID, and perhaps
>> using the max value of max_nb_rx or using the same value of max_nb_rx across
>> adapters. #2 is doable but has a bit of code complexity to handle the
>> generic case.
>>
>> Before we go there, I wanted to check if there is an alternative possible
>> that would remove the difficulties above. Essentially allow multiple ports
>> within an adapter but avoid the problem of the inconsistent <eventdev, port>
>> combinations when using multiple ports with a single eventdev.
>>
>> Instead of
>> ==
>> rte_event_eth_rx_adapter_create()
>> rte_event_eth_rx_adapter_get_info();
>> rte_event_eth_rx_adapter_configure();
>> rte_event_eth_rx_adapter_queue_add();
>> ==
>>
>> How about ?
>> ==
>>
>> rte_event_eth_rx_adapter_get_info(uint8_t dev_id, uint8_t eth_port_id,
>> struct rte_event_eth_rx_adap_info *info);
>>
>> struct rte_event_eth_rx_adap_info {
>> uint32_t cap;
>>
>> /* adapter has inbuilt port, no need to create producer port */
>> #define RTE_EVENT_ETHDEV_CAP_INBUILT_PORT (1ULL << 0)
>> /* adapter does not need service function */
>> #define RTE_EVENT_ETHDEV_CAP_NO_SERVICE_FUNC (1ULL << 1)
>>
>> }
>>
>> rte_event_eth_rx_adapter_conf cfg;
>> cfg.event_port = event_port;
>> cfg.service_name = “rx_adapter_service”;
>
> Does application need to specify the service name? IMO, it better a
> component(rx_adapter) defines it name and format and expose in rte_event_eth_rx_adapter.h
>
I have had the application specify the name. so that it can call
struct rte_service_spec *rte_service_get_by_name(const char *name);
followed by
int32_t rte_service_enable_on_lcore(struct rte_service_spec *service,
uint32_t lcore);
given that the application knows the socket id of the event device
associated with the adapter.
>>
>> // all ports in eth_port_id[] have cap =
>> //!RTE_EVENT_ETHDEV_CAP_INBUILT_PORT
>> // && ! RTE_EVENT_ETHDEV_CAP_NO_SERVICE_FUNC
>> rte_event_eth_rx_adapter_create(dev_id, eth_port_id[], N, id, &cfg);
>
> The downside might be:
> - application has different flow based on based on the capability.
> Listing down a few capabilities/limitation below.
>
>> ===
>> int rte_event_eth_rx_adapter_queue_add() would need a port id in the N>1
>> port case, that can be ignored if the adapter doesn’t need it (N=1).
>>
>> thanks for reading the long email, thoughts ?
>
> I have bit another thought to solve the above mentioned downside.
>
> - Theme is based on your original rx adapter proposal but with eventpmd
> ops(not adapter ops).i.e Reuse as much of your existing Rx adapter
> implementation as common code and add hooks for HW based adapters. For
> example, before you add <ethdev, queue_id> to "rx_poll" in eth_poll_wrr_calc(),
> Check for eventdev PMD ops is available adding it HW. If yes, Don't add in "rx_poll"
>
> adapter_api
> ------------
> int rte_event_eth_rx_adapter_create(id, rte_event_eth_rx_adapter_conf *conf)
> int rte_event_eth_rx_adapter_queue_add(uint8_t id, uint8_t eth_dev_id, int32_t rx_queue_id, rte_event_eth_rx_adapter_queue_conf *conf);
>
> eventdev PMD op api(not as adapter PMD as discussed earlier)
> -------------------
>
> 1) typedef uint64_t (*eventdev_rx_adap_capa)(struct rte_eventdev *dev, uint8_t ethdev_id)
>
> Return the adapter capability of a given eventdev when it needs to
> connected to a specific ethdev_id
>
Doesn't the capability of a <eventdev, ethdev> also need to be made
available to the application as an adapter API ?
for e.g., so the application can call
rte_event_eth_rx_adapter_queue_add() with rx_queue_id = -1 if
RX_ADAPTER_CAP_ADD_QUEUE is not set.
In the same manner, if I understand RX_ADAPTER_CAP_SET_FLOW_ID, the
application would have provide a flow ID in
rte_event_eth_rx_adapter_queue_add(), if RX_ADAPTER_CAP_SET_FLOW_ID is
not set.
Also if a given <eventdev, ethdev> are connected in HW then is the
servicing weight of a queue applicable ?
> Possible capability values based on my understating for existing SW and Cavium
> HW PMD. NXP folks can add new ones.
>
> - RX_ADAPTER_CAP_INBUILT_PORT - /* adapter has inbuilt port, no need to create producer port by common code */
> - RX_ADAPTER_CAP_SET_FLOW_ID - /* adapter capable of setting RTE_ETH_RX_EVENT_ADAPTER_QUEUE_FLOW_ID_VALID */
> - RX_ADAPTER_CAP_ADD_QUEUE /* adapter capable of adding any specific
> ethdev rx queue to any eventdev queue. Some eventdev PMD has a limitation
> that once a < ethdev_id , queue_id> connected to specific eventdev queue,
> all the all queues_id under the same ethdev_id need to be connected to
> same eventdev queue. aka works only on the rte_event_eth_rx_adapter_queue_conf.rx_queue_id == -1 mode, */
>
>
> 2) typedef int (*eventdev_rx_adap_add)(struct rte_eventdev *dev, uint8_t ethdev_id, int queue_id, rte_event_eth_rx_adapter_queue_conf *conf));
> -- if implemented by eventdev PMD and returns zero then COMMON code does not need to poll */
>
>
> 3) typedef int (*eventdev_rx_adap_del)(struct rte_eventdev *dev, uint8_t ethdev_id, int queue_id)
> -- remove previously added
>
>
> *** Haven't spend a lot of time on API/macro name.Please use better naming conversion.
>
>
> Another notes based on your existing implementation + eventdev ops scheme
>
> 1) rte_event_eth_rx_adapter_creates() registers service function by
> default. It should be delayed to when common adapter code find a device
> with !RX_ADAPTER_CAP_INBUILT_PORT cap on rte_event_eth_rx_adapter_queue_add()
>
> 2) Do we need rx_adapter start/stop functions?
Yes.
>
> 3) If it happens to be case where rte_event_eth_rx_adapter_queue_add()
> use only RX_ADAPTER_CAP_INBUILT_PORT then common code should not
> create any service.
Yes.
>
> 4) If adapter uses one port with service core and other one with HW
> adapter. rte_event_eth_rx_adapter_stats.rx_packets will be not updated
> correctly, We need eventdev PMD ops to get those stats. If we agree
> overall PMD ops + adapter API partitioning then we can refine additionally
> eventpmd for stats etc or xstat based scheme etc.
Wouldnt the rx_packets be the sum of the service core thread packet
count and the count provided by the eventdev pmd ops and be obtained
from a call to
rte_event_eth_rx_adapter_stats_get(uint8_t id,
struct rte_event_eth_rx_adapter_stats *stats);
>
> 5) specifying rte_event_eth_rx_adapter_conf.rx_event_port_id on
> rte_event_eth_rx_adapter_create() would waste one HW eventdev port if its
> happen to be used RX_ADAPTER_CAP_INBUILT_PORT on rte_event_eth_rx_adapter_queue_add().
> unlike SW eventdev port, HW eventdev ports are costly so I think, We
> need to have another eventdev PMD ops to create service/producer ports.
> Or any other scheme that creates rte_event_eth_rx_adapter_conf.rx_event_port_id
> on demand by common code.
>
One solution is:
struct rte_event_eth_rx_adapter_conf {
uint8_t dev_id;
int (*conf_cb)(uint8_t id, uint8_t port_id, uint32_t flags, struct
rte_event_eth_rx_adapter_conf *conf);
unsigned int max_nb_rx;
int event_port_id;
char service_name[];
}
Where dev_id and conf_cb have to be specified in the create call, but
event_port_id and service_name will be filled in when conf_cb() is
invoked if required. flags will specify what parameters of conf need to
be filled in. The conf parameter can be separated into a different
struct rte_event_eth_rx_adapter_ext_conf {
unsigned int max_nb_rx;
int event_port_id;
char service_name[];
}
Nikhil
next prev parent reply other threads:[~2017-08-01 8:40 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-09 20:38 [dpdk-dev] [RFC] " Gage Eads
2017-05-11 16:38 ` Jerin Jacob
2017-05-16 20:51 ` Thomas Monjalon
2017-05-24 4:30 ` Rao, Nikhil
2017-06-19 10:05 ` Jerin Jacob
2017-06-26 13:19 ` Jerin Jacob
2017-06-28 6:47 ` Rao, Nikhil
2017-07-06 21:52 ` [dpdk-dev] [PATCH 1/2] " Nikhil Rao
2017-07-06 14:18 ` Jerin Jacob
2017-07-07 6:21 ` Rao, Nikhil
2017-07-07 15:03 ` Jerin Jacob
2017-07-07 15:57 ` Jerin Jacob
2017-07-10 6:14 ` Rao, Nikhil
2017-07-10 10:41 ` Jerin Jacob
2017-07-13 3:26 ` Rao, Nikhil
2017-07-13 18:45 ` Jerin Jacob
2017-07-27 10:58 ` Rao, Nikhil
2017-07-29 15:12 ` Jerin Jacob
2017-07-31 3:57 ` Nipun Gupta
2017-07-31 15:31 ` Jerin Jacob
2017-08-01 8:40 ` Rao, Nikhil [this message]
2017-08-01 16:42 ` Jerin Jacob
2017-08-02 19:19 ` Eads, Gage
2017-08-03 6:23 ` Jerin Jacob
2017-08-09 2:23 ` Eads, Gage
2017-08-09 16:19 ` Jerin Jacob
2017-08-09 19:24 ` Eads, Gage
2017-08-10 16:53 ` Jerin Jacob
2017-08-14 8:48 ` Rao, Nikhil
2017-08-14 11:11 ` Jerin Jacob
2017-08-16 5:06 ` Rao, Nikhil
2017-08-11 5:25 ` Rao, Nikhil
2017-08-11 9:49 ` Jerin Jacob
2017-09-04 6:37 ` Jerin Jacob
2017-07-06 21:52 ` [dpdk-dev] [PATCH 2/2] eventdev: add event eth rx adapter unit tests Nikhil Rao
2017-07-24 10:10 ` [dpdk-dev] [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues Nipun Gupta
2017-07-24 10:24 ` Jerin Jacob
2017-07-24 11:37 ` Nipun Gupta
2017-07-24 10:32 ` Van Haaren, Harry
2017-07-24 13:06 ` Nipun Gupta
2017-07-24 13:33 ` Van Haaren, Harry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7b9ca757-f428-3675-b997-794ec6e96f2a@intel.com \
--to=nikhil.rao@intel.com \
--cc=abhinandan.gujjar@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=gage.eads@intel.com \
--cc=harry.van.haaren@intel.com \
--cc=hemant.agrawal@nxp.com \
--cc=jerin.jacob@caviumnetworks.com \
--cc=narender.vangati@intel.com \
--cc=nipun.gupta@nxp.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).