Re: [dpdk-dev] [PATCH 0/9] net/cxgbe: updates for rte_flow support

[Rahul Lakkireddy <rahul.lakkireddy@chelsio.com>]

On Wednesday, March 03/18/20, 2020 at 16:07:07 +0100, Thomas Monjalon wrote:
> 18/03/2020 14:06, Rahul Lakkireddy:
> > Hi Thomas,
> > 
> > On Wednesday, March 03/18/20, 2020 at 13:09:47 +0100, Thomas Monjalon wrote:
> > > 11/03/2020 10:05, Rahul Lakkireddy:
> > > > From: Karra Satwik <kaara.satwik@chelsio.com>
> > > > 
> > > > This series of patches contain rte_flow support for matching
> > > > Q-in-Q VLAN, IP TOS, PF, and VF fields. Also, adds Destination
> > > > MAC rewrite and Source MAC rewrite actions.
> > > > 
> > > > Apart from the 4-tuple (IP src/dst addresses and TCP/UDP src/dst
> > > > port addresses), there are only 40-bits available to match other
> > > > fields in packet headers. Currently, the combination of packet
> > > > header fields to match are configured via filterMode for LETCAM
> > > > filters and filterMask for HASH filters in firmware config files
> > > > (t5/t6-config.txt). Adapter needs to be reflashed with new firmware
> > > > config file everytime the combinations need to be changed. To avoid
> > > > this, a new firmware API is available to dynamically change the
> > > > combination before completing full adapter initialization. So, 2
> > > > new devargs filtermode and filtermask are added to dynamically
> > > > select the combinations during runtime.
> > > 
> > > Please, could you explain why you are using devargs for flow matching,
> > > instead of using the common and generic rte_flow API?
> > 
> > The devargs are being used to configure the TCAM in hardware on
> > what header fields need to be matched in packets by the TCAM. The
> > actual filter rules are still being inserted using rte_flow API.
> > 
> > Apart from the 4-tuple (src/dst IP, src/dst port addresses), there
> > are only 40-bits available for each filter rule to match other
> > header fields. Hardware supports matching ethertype (16-bit),
> > DST MAC (9-bit MPS index), Inner VLAN (16-bit), Outer VLAN (16-bit),
> > IP Protocol (8-bit), IP TOS (8-bit), Ingress Physical Port (3-bit),
> > and PFVF (17-bit) for now. It's not possible to write a filter rule
> > which wants to match all the above fields, which is far beyond 40-bits
> > available. So, the devargs are being used to control which of the
> > above fields that user wants to configure the TCAM to match. Note
> > that once the TCAM is configured, "all" the rules in the TCAM can
> > only match the selected fields in the combination. They can't match
> > any other header fields.
> 
> In case a rule is not possible, are you rejecting it at the validation stage?
> 

Yes, the rule is rejected if it can't be offloaded. Any rule
that has fields that are unsupported or haven't been configured
to be matched by TCAM are rejected. This validation is being done
in the driver's registered callback of rte_flow_validate().

> > For example, let's say user wants to match ethertype (16-bit),
> > DST MAC (9-bit MPS index), and IP protocol (8-bit) in all filter
> > rules. Then, they would configure the TCAM with the {ethertype,
> > DST MAC, and IP protocol} combination. All rules that the user
> > wants to insert into TCAM can only have the above header fields,
> > alongside the default 4-tuple (src/dst IP, src/dst port addresses).
> > 
> > There are 2 regions in hardware. One for matching wild-card filter
> > rules and the other for matching exact-match rules. The "filtermode"
> > devarg controls the 40-bit combination for wild-card filter rules and
> > the "filtermask" devarg controls the combination for exact-match rules.
> 
> I see an issue with this approach. I will explain below.
> 
> An application is written to use some flow rules.
> The application requirements are expressed by the app developper
> through the API (rte_flow).
> In your case, the user must be aware of what the application expects
> and fill the right devargs, according to what the dev wrote.
> Why bothering the user with this constraint?
> 
> I understand the hardware must be prepared in advance.
> I think this configuration must be done through API.
> One workaround is to manage this HW limitation in a PMD-specific API.
> A good solution would be to express this requirement with rte_flow.
> 

The TCAM must be configured on what fields it is expected to match
before the hardware initialization completes; i.e. during PCIe probe
itself. Devargs is the only API I know that can provide this kind
of configuration information at this early stage. Based on what
fields the TCAM has been configured to match, the resources in
hardware are allocated accordingly.

Note that this configuration is not on a per rule basis. All the
rules must only have the fields that the TCAM had been configured to
match. Any rule that doesn't adhere to this will be rejected.

> One idea: can we use rte_flow_validate() to fill the requirements?
> The PMD requirements are empty at the beginning and they are filled
> with the first calls to rte_flow_validate().
> 
> Maybe we also need to express the capabilities/limitations.
> Example: is there a maximum number of rules?
> maximum number of protocols to match?
> maximum number of bits to match?
> 
> I suppose it is not easy to implement. Comments?
> 

Perhaps the hardware's rte_flow offload capabilities can be exposed
similar to how we're exposing the Tx/Rx offloads today, via the
rte_eth_dev_info structure. Alternatively, we can add a separate
rte_flow_dev_info structure and let PMDs fill their rte_flow related
capabilities. Maybe even add a rte_flow_dev_infos_get() API for this
purpose?

Thanks,
Rahul