Re: [dpdk-dev] [RFC] ethdev: reserve the RX offload most-significant bits for PMD scartch

[Stephen Hemminger <stephen@networkplumber.org>]

On Fri, 21 Jun 2019 07:43:13 +0000
"Wang, Haiyue" <haiyue.wang@intel.com> wrote:

> I know my patch is ugly for making customers happy, I will try to find other method not to
> break the beautiful rte_ethdev desgin.
> 
> Really thanks for your reply. This helps me understand the PMD design practice more.
> 
> BR,
> Haiyue
> 
> From: Andrew Rybchenko [mailto:arybchenko@solarflare.com]
> Sent: Friday, June 21, 2019 15:40
> To: Wang, Haiyue <haiyue.wang@intel.com>; dev@dpdk.org
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Thomas Monjalon <thomas@monjalon.net>
> Subject: Re: [dpdk-dev] [RFC] ethdev: reserve the RX offload most-significant bits for PMD scartch
> 
> On 6/21/19 10:37 AM, Wang, Haiyue wrote:
> Then this is not so generic if a workaround is needed. In other words, no one is so perfect. ☺
> 
> Yes, it is a bug. No one is perfect.
> 
> 
> 
> BR,
> Haiyue
> 
> From: Andrew Rybchenko [mailto:arybchenko@solarflare.com]
> Sent: Friday, June 21, 2019 15:34
> To: Wang, Haiyue <haiyue.wang@intel.com><mailto:haiyue.wang@intel.com>; dev@dpdk.org<mailto:dev@dpdk.org>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com><mailto:ferruh.yigit@intel.com>; Thomas Monjalon <thomas@monjalon.net><mailto:thomas@monjalon.net>
> Subject: Re: [dpdk-dev] [RFC] ethdev: reserve the RX offload most-significant bits for PMD scartch
> 
> On 6/21/19 4:12 AM, Wang, Haiyue wrote:
> Not so frightening in real world for an application to be aware of its NICs:
> https://github.com/Juniper/contrail-vrouter/blob/master/dpdk/vr_dpdk_ethdev.c#L387<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Juniper_contrail-2Dvrouter_blob_master_dpdk_vr-5Fdpdk-5Fethdev.c-23L387&d=DwMGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=flTOx6Av679My7o_iScb5sOlLD68bpUyE2RUtfW3SWQ&m=XSIm84nALkE7O1aeqpJkVJJWzepVsGEJsTeiDCxoLK4&s=L1vEJ5GeVHbammKc0iJn0YdoeKf0GqeeNJy-q5xCi6E&e=>
> 
> 
> 
> 
> In this particular case it is just a workaround for bonding and bnxt.
> Driver name is provided and sufficient to make it possible when
> absolutely required.
> 
> 
> 
> 
> 
> Yes, we need to avoid this kind of design.
> 
> BR,
> Haiyue
> 
> From: Andrew Rybchenko [mailto:arybchenko@solarflare.com]
> Sent: Friday, June 21, 2019 02:30
> To: Wang, Haiyue <haiyue.wang@intel.com><mailto:haiyue.wang@intel.com>; dev@dpdk.org<mailto:dev@dpdk.org>
> Cc: Yigit, Ferruh <ferruh.yigit@intel.com><mailto:ferruh.yigit@intel.com>; Thomas Monjalon <thomas@monjalon.net><mailto:thomas@monjalon.net>
> Subject: Re: [dpdk-dev] [RFC] ethdev: reserve the RX offload most-significant bits for PMD scartch
> 
> CC ethdev maintainers
> 
> On 6/20/19 10:25 AM, Haiyue Wang wrote:
> 
> Generally speaking, the DEV_RX_OFFLOAD_xxx for RX offload capabilities
> 
> of a device is one-bit field definition, it has 64 different values at
> 
> most.
> 
> 
> 
> Nowdays the receiving queue of NIC has rich features not just checksum
> 
> offload, like it can extract the network protocol header fields to its
> 
> RX descriptors for quickly handling. But this kind of feature is not so
> 
> common, and it is hardware related. Normally, this can be done through
> 
> rte_devargs driver parameters, but the scope is per device. This is not
> 
> so nice for per queue design.
> 
> 
> 
> The per queue API 'rte_eth_rx_queue_setup' and data structure 'struct
> 
> rte_eth_rxconf' are stable now, and be common for all PMDs. For keeping
> 
> the ethdev API & ABI compatibility, and the application can make good
> 
> use of the NIC's specific features, reserving the most-significant bits
> 
> of RX offload seems an compromise method.
> 
> 
> 
> Then the PMDs redefine these bits as they want, all PMDs share the same
> 
> bit positions and expose their new definitions with the header file.
> 
> 
> 
> The experimental reserved bits number is 6 currently. Tt can be one-bit
> 
> for each features up to the the maximum number 6. It can also be some
> 
> bits encoding: e.g, 6 bits can stand for 63 maximum number of features.
> 
> 
> 
> We call these reserved bits as DEV_RX_OFFLOAD_PMD_SCRATCH. And the left
> 
> unused bits number is : 64 - 19 (currently defined) - 6 (PMD scartch) =
> 
> 39.
> 
> 
> 
> This is not so nice for applications, they need to check PMD's driver
> 
> name for lookuping their DEV_RX_OFFLOAD_PMD_SCRATCH definitions. But it
> 
> is good for the applications to make use of the hardware compatibility.
> 
> 
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com><mailto:haiyue.wang@intel.com>
> 
> I would say that it very bad for applications. It sounds like reserved bits
> in registers which have meaning in fact and sometimes different meaning.
> Of course, it is not that bad when rules are defined, but such kind of
> features tend to spread and clutter up interfaces. IMHO, the feature is
> really frightening.

There are two issues. First, having more OFFLOAD capability feature bits
is good. As long as these feature bits are well defined. If only one vendor
implements that feature that is fine. Another vendor can implement the
same thing, and application can know what it is asking for.

The other issue is the limited number of feature bits. I expect that some
time soon the bits will have to grow into an array and cause API/ABI
break. That can be fixed when the last bit is exhausted.