[dpdk-dev] Eventdev DSW and eth_rx_adapters

[dpdk-dev] Eventdev DSW and eth_rx_adapters

[Venky Venkatesh]

Hi,
I am relatively new to DPDK and am trying to use the eventdev library.
The sw_evdev runs on a single core (service core). And then there is rte_event_eth_rx_adapter_ which links the sw_evdev to the ethdev. This adapter is also service core based. The DSW runs on all cores – and thus doesn’t use service cores. If we use the existing adapter, in the DSW paradigm:

  *   The benefit of DSW is diminished as the packets would first have to go thru a possible choke point viz. the service core.
  *   I don’t have an understanding of this: the eventdev port corresponding to packet RX would be on a service core due to the adapter being on the service core. Therefore when the adapter calls event_enqueue to enqueue into the scheduler, does it mean that the DSW portion would also run on the service core as well?
  *   Besides, (more importantly) the service core model is also not preferable to us as it sets aside cores outside of the application – which is a costly trade for us.

How then is the user of the eventdev-with-DSW expected to interface with ethdev? Should the application call the ethdev directly and then call the eventdev-with-DSW-underneath directly (much like what the current adapter is doing)? Is there something already existing for this so that we don’t reinvent the wheel?

Thanks
-Venky

Re: [dpdk-dev] Eventdev DSW and eth_rx_adapters

[Mattias Rönnblom]

On 2018-12-18 02:46, Venky Venkatesh wrote:
> Hi,
> I am relatively new to DPDK and am trying to use the eventdev library.
> The sw_evdev runs on a single core (service core). And then there is rte_event_eth_rx_adapter_ which links the sw_evdev to the ethdev. This adapter is also service core based. The DSW runs on all cores – and thus doesn’t use service cores. If we use the existing adapter, in the DSW paradigm:
> 
>    *   The benefit of DSW is diminished as the packets would first have to go thru a possible choke point viz. the service core.

Yes, but the practical implications may not be as great as you think. A 
single service core will be able to handle (e.g. relay) a fair amount of 
events into/out-of an event device.

You are however not forced to use service cores. You may use one or more 
of your workers to feed an event device.

Another option is to do your application work as a service (and thus run 
the whole thing on service cores). Unorthodox design, and not something 
I've tried in practice, but might work.

While the service cores concept is simple and effective, it's also not 
very flexible and can lead to inefficiencies and unnecessary 
bottlenecks. The key issue is that it might be difficult in practice to 
distribute the work of running on the different services and the 
application work across the lcores, especially if the load on the 
different tasks differs across work loads. A dynamic load distribution 
mechanism would be preferable. Such a mechanism might however prove 
difficult to devise, in particular since the tool of DPDK's disposal is 
cooperative multitasking, in one form or the other.

>    *   I don’t have an understanding of this: the eventdev port corresponding to packet RX would be on a service core due to the adapter being on the service core. Therefore when the adapter calls event_enqueue to enqueue into the scheduler, does it mean that the DSW portion would also run on the service core as well?

Yes, the service cores will be an "participating lcore" (as I think I 
called it in DSW), in case they use an eventdev port.

>    *   Besides, (more importantly) the service core model is also not preferable to us as it sets aside cores outside of the application – which is a costly trade for us.
> 
> How then is the user of the eventdev-with-DSW expected to interface with ethdev? Should the application call the ethdev directly and then call the eventdev-with-DSW-underneath directly (much like what the current adapter is doing)? Is there something already existing for this so that we don’t reinvent the wheel?
> 


See above for your last two questions.

An overall comment to your questions is that the DSW user should act in 
the same way - pretty much - as would he use for example the SW device.

Re: [dpdk-dev] Eventdev DSW and eth_rx_adapters

[Venky Venkatesh]

On 12/18/18, 10:53 PM, "Mattias Rönnblom" <mattias.ronnblom@ericsson.com> wrote:

    On 2018-12-18 02:46, Venky Venkatesh wrote:
    > Hi,
    > I am relatively new to DPDK and am trying to use the eventdev library.
    > The sw_evdev runs on a single core (service core). And then there is rte_event_eth_rx_adapter_ which links the sw_evdev to the ethdev. This adapter is also service core based. The DSW runs on all cores – and thus doesn’t use service cores. If we use the existing adapter, in the DSW paradigm:
    > 
    >    *   The benefit of DSW is diminished as the packets would first have to go thru a possible choke point viz. the service core.
    
    Yes, but the practical implications may not be as great as you think. A 
    single service core will be able to handle (e.g. relay) a fair amount of 
    events into/out-of an event device.
    
    You are however not forced to use service cores. You may use one or more 
    of your workers to feed an event device.

[VV]: Just to be sure we are on the same page: The workers in a sort of alternating fashion call the ethdev to dequeue and enqueue_new into the eventdev and of course the dequeue from the eventdev (sort of analogous to the code that you provided with integrated producer and consumer). 
As I said in the earlier email, the service core option (to run an adapter, even if it's performance is not an issue) is a bit too costly for us. 
    
    Another option is to do your application work as a service (and thus run 
    the whole thing on service cores). Unorthodox design, and not something 
    I've tried in practice, but might work.
    
    While the service cores concept is simple and effective, it's also not 
    very flexible and can lead to inefficiencies and unnecessary 
    bottlenecks. The key issue is that it might be difficult in practice to 
    distribute the work of running on the different services and the 
    application work across the lcores, especially if the load on the 
    different tasks differs across work loads. A dynamic load distribution 
    mechanism would be preferable. Such a mechanism might however prove 
    difficult to devise, in particular since the tool of DPDK's disposal is 
    cooperative multitasking, in one form or the other.


    
    >    *   I don’t have an understanding of this: the eventdev port corresponding to packet RX would be on a service core due to the adapter being on the service core. Therefore when the adapter calls event_enqueue to enqueue into the scheduler, does it mean that the DSW portion would also run on the service core as well?
    
    Yes, the service cores will be an "participating lcore" (as I think I 
    called it in DSW), in case they use an eventdev port.
    
    >    *   Besides, (more importantly) the service core model is also not preferable to us as it sets aside cores outside of the application – which is a costly trade for us.
    > 
    > How then is the user of the eventdev-with-DSW expected to interface with ethdev? Should the application call the ethdev directly and then call the eventdev-with-DSW-underneath directly (much like what the current adapter is doing)? Is there something already existing for this so that we don’t reinvent the wheel?
    > 
    
    
    See above for your last two questions.
    
    An overall comment to your questions is that the DSW user should act in 
    the same way - pretty much - as would he use for example the SW device.
 

Re: [dpdk-dev] Eventdev DSW and eth_rx_adapters

[Mattias Rönnblom]

On 2018-12-19 19:37, Venky Venkatesh wrote:
> 
> 
> On 12/18/18, 10:53 PM, "Mattias Rönnblom" <mattias.ronnblom@ericsson.com> wrote:
> 
>      On 2018-12-18 02:46, Venky Venkatesh wrote:
>      > Hi,
>      > I am relatively new to DPDK and am trying to use the eventdev library.
>      > The sw_evdev runs on a single core (service core). And then there is rte_event_eth_rx_adapter_ which links the sw_evdev to the ethdev. This adapter is also service core based. The DSW runs on all cores – and thus doesn’t use service cores. If we use the existing adapter, in the DSW paradigm:
>      >
>      >    *   The benefit of DSW is diminished as the packets would first have to go thru a possible choke point viz. the service core.
>      
>      Yes, but the practical implications may not be as great as you think. A
>      single service core will be able to handle (e.g. relay) a fair amount of
>      events into/out-of an event device.
>      
>      You are however not forced to use service cores. You may use one or more
>      of your workers to feed an event device.
> 
> [VV]: Just to be sure we are on the same page: The workers in a sort of alternating fashion call the ethdev to dequeue and enqueue_new into the eventdev and of course the dequeue from the eventdev (sort of analogous to the code that you provided with integrated producer and consumer).

Yes.