Re: [dpdk-dev] [RFC] ethdev: support hairpin queue

["Wu, Jingjing" <jingjing.wu@intel.com>]

Hi, Ori

Thanks for the explanation. I have more question below.

Thanks
Jingjing

> -----Original Message-----
> From: Ori Kam [mailto:orika@mellanox.com]
> Sent: Thursday, September 5, 2019 1:45 PM
> To: Wu, Jingjing <jingjing.wu@intel.com>; Thomas Monjalon <thomas@monjalon.net>;
> Yigit, Ferruh <ferruh.yigit@intel.com>; arybchenko@solarflare.com; Shahaf Shuler
> <shahafs@mellanox.com>; Slava Ovsiienko <viacheslavo@mellanox.com>; Alex
> Rosenbaum <alexr@mellanox.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [RFC] ethdev: support hairpin queue
> 
> Hi Wu,
> Thanks for your comments PSB,
> 
> Ori
> 
> > -----Original Message-----
> > From: Wu, Jingjing <jingjing.wu@intel.com>
> > Sent: Thursday, September 5, 2019 7:01 AM
> > To: Ori Kam <orika@mellanox.com>; Thomas Monjalon
> > <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>;
> > arybchenko@solarflare.com; Shahaf Shuler <shahafs@mellanox.com>; Slava
> > Ovsiienko <viacheslavo@mellanox.com>; Alex Rosenbaum
> > <alexr@mellanox.com>
> > Cc: dev@dpdk.org
> > Subject: RE: [dpdk-dev] [RFC] ethdev: support hairpin queue
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ori Kam
> > > Sent: Tuesday, August 13, 2019 9:38 PM
> > > To: thomas@monjalon.net; Yigit, Ferruh <ferruh.yigit@intel.com>;
> > > arybchenko@solarflare.com; shahafs@mellanox.com;
> > viacheslavo@mellanox.com;
> > > alexr@mellanox.com
> > > Cc: dev@dpdk.org; orika@mellanox.com
> > > Subject: [dpdk-dev] [RFC] ethdev: support hairpin queue
> > >
> > > This RFC replaces RFC[1].
> > >
> > > The hairpin feature (different name can be forward) acts as "bump on the
> > wire",
> > > meaning that a packet that is received from the wire can be modified using
> > > offloaded action and then sent back to the wire without application
> > intervention
> > > which save CPU cycles.
> > >
> > > The hairpin is the inverse function of loopback in which application
> > > sends a packet then it is received again by the
> > > application without being sent to the wire.
> > >
> > > The hairpin can be used by a number of different NVF, for example load
> > > balancer, gateway and so on.
> > >
> > > As can be seen from the hairpin description, hairpin is basically RX queue
> > > connected to TX queue.
> > >
> > > During the design phase I was thinking of two ways to implement this
> > > feature the first one is adding a new rte flow action. and the second
> > > one is create a special kind of queue.
> > >
> > > The advantages of using the queue approch:
> > > 1. More control for the application. queue depth (the memory size that
> > > should be used).
> > > 2. Enable QoS. QoS is normaly a parametr of queue, so in this approch it
> > > will be easy to integrate with such system.
> >
> >
> > Which kind of QoS?
> 
> For example latency , packet rate those kinds of makes sense in the queue level.
> I know we don't have any current support but I think we will have during the next year.
> 
Where would be the QoS API loading? TM API? Or propose other new?
> >
> > > 3. Native integression with the rte flow API. Just setting the target
> > > queue/rss to hairpin queue, will result that the traffic will be routed
> > > to the hairpin queue.
> > > 4. Enable queue offloading.
> > >
> > Looks like the hairpin queue is just hardware queue, it has no relationship with
> > host memory. It makes the queue concept a little bit confusing. And why do we
> > need to setup queues, maybe some info in eth_conf is enough?
> 
> Like stated above it makes sense to have queue related parameters.
> For example I can think of application that most packets are going threw that hairpin
> queue, but some control packets are
> from the application. So the application can configure the QoS between those two
> queues. In addtion this will enable the application
> to use the queue like normal queue from rte_flow (see comment below) and every other
> aspect.
> 
Yes, it is typical use case. And rte_flow is used to classify to different queue?
If I understand correct, your hairpin queue is using host memory/or on-card memory for buffering, but CPU cannot touch it, all the packet processing is done by NIC.
Queue is created, where the queue ID is used? Tx queue ID may be used as action of rte_flow? I still don't understand where the hairpin Rx queue ID be used. 
In my opinion, if no rx/tx function, it should not be a true queue from host view. 

> >
> > Not sure how your hardware make the hairpin work? Use rte_flow for packet
> > modification offload? Then how does HW distribute packets to those hardware
> > queue, classification? If So, why not just extend rte_flow with the hairpin
> > action?
> >
> 
> You are correct, the application uses rte_flow and just points the traffic to the requested
> hairpin queue/rss.
> We could have added a new rte_flow command. The reasons we didn't:
> 1. Like stated above some of the hairpin makes sense in queue level.
> 2.  In the near future, we will also want to support hairpin between different ports. This
> makes much more
> sense using queues.
> 
> > > Each hairpin Rxq can be connected Txq / number of Txqs which can belong to
> > a
> > > different ports assuming the PMD supports it. The same goes the other
> > > way each hairpin Txq can be connected to one or more Rxqs.
> > > This is the reason that both the Txq setup and Rxq setup are getting the
> > > hairpin configuration structure.
> > >
> > > From PMD prespctive the number of Rxq/Txq is the total of standard
> > > queues + hairpin queues.
> > >
> > > To configure hairpin queue the user should call
> > > rte_eth_rx_hairpin_queue_setup / rte_eth_tx_hairpin_queue_setup insteed
> > > of the normal queue setup functions.
> >
> > If the new API introduced to avoid ABI change, would one API
> > rte_eth_rx_hairpin_setup be enough?
> 
> I'm not sure I understand your comment.
> The rx_hairpin_setup was created for two main reasons:
> 1. Avoid API change.
> 2. I think it is more correct to use different API since the parameters are different.
> 
I mean not use queue setup concept, set hairpin feature through one hairpin configuration API.

> The reason we have both rx and tx setup functions is that we want the user to have
> control binding the two queues.
> It is most important when we will advance to hairpin between ports.

Hairpin between ports? It looks like switch but not hairpin, right?
> 
> >
> > Thanks
> > Jingjing
> 
> Thanks,
> Ori