Re: [dpdk-dev] [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API

[Tiwei Bie <tiwei.bie@intel.com>]

On Wed, Jan 04, 2017 at 11:14:25PM +0800, Ananyev, Konstantin wrote:
> 
> 
> > -----Original Message-----
> > From: Bie, Tiwei
> > Sent: Wednesday, January 4, 2017 2:39 PM
> > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> > Cc: dev@dpdk.org; adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Zhang, Helin <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang,
> > Xiao W <xiao.w.wang@intel.com>
> > Subject: Re: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> > 
> > On Wed, Jan 04, 2017 at 10:21:08PM +0800, Ananyev, Konstantin wrote:
> > > Hi Twei,
> > >
> > > > -----Original Message-----
> > > > From: Bie, Tiwei
> > > > Sent: Wednesday, January 4, 2017 7:22 AM
> > > > To: dev@dpdk.org
> > > > Cc: adrien.mazarguil@6wind.com; Lu, Wenzhuo <wenzhuo.lu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> > > > olivier.matz@6wind.com; thomas.monjalon@6wind.com; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Zhang, Helin
> > > > <helin.zhang@intel.com>; Dai, Wei <wei.dai@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> > > > Subject: [PATCH v5 3/8] ethdev: reserve capability flags for PMD-specific API
> > > >
> > > > Reserve a Tx capability flag and a Rx capability flag, that can be
> > > > used by PMD to define its own capability flags when implementing the
> > > > PMD-specific API.
> > > >
> > > > Suggested-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
> > > > Signed-off-by: Tiwei Bie <tiwei.bie@intel.com>
> > > > Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > > > ---
> > > >  lib/librte_ether/rte_ethdev.h | 2 ++
> > > >  1 file changed, 2 insertions(+)
> > > >
> > > > diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
> > > > index d465825..8800b39 100644
> > > > --- a/lib/librte_ether/rte_ethdev.h
> > > > +++ b/lib/librte_ether/rte_ethdev.h
> > > > @@ -857,6 +857,7 @@ struct rte_eth_conf {
> > > >  #define DEV_RX_OFFLOAD_TCP_LRO     0x00000010
> > > >  #define DEV_RX_OFFLOAD_QINQ_STRIP  0x00000020
> > > >  #define DEV_RX_OFFLOAD_OUTER_IPV4_CKSUM 0x00000040
> > > > +#define DEV_RX_OFFLOAD_RESERVED_0  0x00000080 /**< Used for PMD-specific API. */
> > > >
> > > >  /**
> > > >   * TX offload capabilities of a device.
> > > > @@ -874,6 +875,7 @@ struct rte_eth_conf {
> > > >  #define DEV_TX_OFFLOAD_GRE_TNL_TSO      0x00000400    /**< Used for tunneling packet. */
> > > >  #define DEV_TX_OFFLOAD_IPIP_TNL_TSO     0x00000800    /**< Used for tunneling packet. */
> > > >  #define DEV_TX_OFFLOAD_GENEVE_TNL_TSO   0x00001000    /**< Used for tunneling packet. */
> > > > +#define DEV_TX_OFFLOAD_RESERVED_0  0x00002000 /**< Used for PMD-specific API. */
> > > >
> > > >  /**
> > > >   * Ethernet device information
> > > > --
> > > > 2.7.4
> > >
> > > I am not sure how that supposed to work and how user should know that DEV_RX_OFFLOAD_RESERVED_0
> > > is actually a MACSEC for ixgbe?
> > 
> > Users are not supposed to use DEV_RX_OFFLOAD_RESERVED_0, instead, they
> > should use the capabilities and the likes defined in rte_pmd_ixgbe.h
> > where the PMD-specifics APIs are declared:
> > 
> > /**
> >  * If these flags are advertised by the PMD, the NIC supports the MACsec
> >  * offload. The incoming MACsec traffics can be offloaded transparently
> >  * after the MACsec offload is configured correctly by the application.
> >  * And the application can set the PKT_TX_IXGBE_MACSEC flag in mbufs to
> >  * enable the MACsec offload for the packets to be transmitted.
> >  */
> > #define DEV_RX_OFFLOAD_IXGBE_MACSEC_STRIP	DEV_RX_OFFLOAD_RESERVED_0
> > #define DEV_TX_OFFLOAD_IXGBE_MACSEC_INSERT	DEV_TX_OFFLOAD_RESERVED_0
> > 
> > /**
> >  * This event will occur when the PN counter in a MACsec connection
> >  * reach the exhaustion threshold.
> >  */
> > #define RTE_ETH_EVENT_IXGBE_MACSEC		RTE_ETH_EVENT_RESERVED_0
> > 
> > /**
> >  * Offload the MACsec. This flag must be set by the application in mbuf
> >  * to enable this offload feature for a packet to be transmitted.
> >  */
> > #define PKT_TX_IXGBE_MACSEC			PKT_TX_RESERVED_0
> > 
> > PMD-specific APIs can only be used on the corresponding driver/device,
> > so different PMD can share the same reserved bit to represent different
> > things when implementing their own PMD-specific APIs.
> 
> Ok, and why do we need it?
> Why we can't just have PKT_TX_MACSEC straightway?
> What are you trying to gain here?
> Is it just for future opportunity to save an extra bit in mbuf.ol_flags?
> I don't think we are short of bits here right now, and we don't consume them exra-fast
> to start to worry about it.
> Again, if it *really* would be for ixgbe only forever, and y don't want to waste a bit in tx_olflags,
> why not to introduce device specific ol_flags in mbuf second cache line?
> Probably uint16_t would be enough for that.
> 
> > 
> > > Another question what to do if you would like to create a bonded device over two devices with different NIC types?
> > > As I understand you can end up in situation when  DEV_RX_OFFLOAD_RESERVED_0  would mean different capabilities.
> > > Why not to have this MACSEC capability and ol_flag value as generic ones, as you have them in previous versions of your patch?
> > 
> > Those flags are only used in PMD-specific APIs. I don't think we could
> > use the PMD-specific APIs provided by a certain PMD on a bonded device.
> 
> I understand that.
> My question was: suppose user would like to create a bonded device over 2 NICs.
> One of them is ixgbe, while other would be some other type.
> In future get_dev_info() for each of them might return DEV_RX_OFFLOAD_RESERVED_0  bit as set.
> But it would mean completely different thing.
> How bonded device would know that to deal properly?
> 
> Another example - user has 2 NICs of different type and would like to send the same packet on both of them simultaneously.
> As PKT_TX_RESERVED might mean different things for these devices, and user would like to use let say
> PKT_TX_IXGBE_MACSEC on one of them, he would need to do a copy of them, instead just increment a refcnt. 
> 
> Similar issues might arise at RX handling: user got a packet with PKT_RX_RESERVED_0 set.
> What does it really mean if there are different NIC types in the system?
> The only way to answer that question, as I can see,  is to keep track from what NIC that packet was received.
> Which I suppose, is not always convenient.
> 

The main purpose is to put the PMD-specific APIs in a separate
namespace instead of mixing the PMD-specific APIs and global APIs
up, and also save the bits in mbuf.ol_flags.

There are other ways to achieve this goal, such as introducing
the PMD specific ol_flags in mbuf second cache line as you said.
I just thought defining some reserved bits seems to be the most
simple way which won't introduce many changes.

What's your suggestions? Should I just revert the changes and
define the generic flags directly?

Thanks & regards,
Tiwei Bie