Re: [dpdk-dev] [RFC PATCH 0/4] ethdev new offloads API

[Jerin Jacob <jerin.jacob@caviumnetworks.com>]

-----Original Message-----
> Date: Mon, 28 Aug 2017 12:57:13 +0200
> From: Thomas Monjalon <thomas@monjalon.net>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Cc: dev@dpdk.org, Shahaf Shuler <shahafs@mellanox.com>
> Subject: Re: [dpdk-dev] [RFC PATCH 0/4] ethdev new offloads API
> 
> 28/08/2017 07:00, Jerin Jacob:
> > From: Shahaf Shuler <shahafs@mellanox.com>
> > > Friday, August 25, 2017 1:32 PM, Jerin Jacob:
> > > > >
> > > > > The new API does not have an equivalent for the below Tx flags:
> > > > >
> > > > > * ETH_TXQ_FLAGS_NOREFCOUNT
> > > > > * ETH_TXQ_FLAGS_NOMULTMEMP
> > > > 
> > > > IMO, it make sense to keep those flags as PMD optimization if an application
> > > > does not need reference count and multi mempool in the application.
> > > > As example, An non trivial application like l3fwd does not need both of them.
> > > 
> > > The l3fwd application is yet another simple example from DPDK tree. Am not sure that a complete vRouter/vSwitch implementation is with the same characteristics.
> > 
> > But not all dpdk applications are complete vRouter/vSwitch implementation.
> > 
> > > Moreover, I think the fact there is an application which is able to use it is not enough.  IMO there needs to be some basic functionality always provided by the PMDs and not controlled by flags.
> > > For example, let's say we have an application which always sends the mbufs with the same ol_flags, or even with the same length.
> > 
> > Does ETH_TXQ_FLAGS_NOREFCOUNT comes in same category like mbuf->ol_flags?
> > 
> > > Will it make sense to add more flags to control it?
> > > Will it makes sense to run RFC2544 benchmark with testpmd io forwarding with those flags? 
> > > 
> > > If the answer is yes, maybe those flags (and others to follow) belong on different location on ethdev. However for sure they are not offloads.
> > 
> > I am not sure about the reason for opting out mempool related flags.
> > In the context of HW assisted external mempool managers, Enabling reference count is an offload
> > from Ethernet device.
> > For example, with external HW assisted mempool, ethdev driver needs to
> > have different way of forming TXQ descriptor in case if reference count
> > is enabled(as, in the case of HW assisted mempool managers, bu default,
> > HW frees the packet on send)
> > 
> > I am fine with moving the flags to some where else if it is make sense to you.But
> > from PMD optimization perspective, I think it is important have these flags.
> 
> Why not.

> We can have a function to enable such optimizations.

OK

> However I am not sure ethdev is the right place as these hints apply
> to any mbuf.

I think, We are talking about the mbuf behavior when working with ethdev TXQ
here. Right? IMO, it make senses in the ethdev layer.

We pulled in ETH_TXQ_FLAGS_NOMULTSEGS flag for the rework even though it
is related to mbuf. If you think, ETH_TXQ_FLAGS_NOREFCOUNT and
ETH_TXQ_FLAGS_NOMULTMEMP should NOT a be PER TXQ configurable flag then it can
be moved to Port level TXQ configuration at

function: rte_eth_dev_configure()
struct rte_eth_conf::struct rte_eth_txmode:

I think it is important for The ARM64 architecture to optimize for the
application which does not need reference counting and single pool
configuration like l3fwd.

Reasons:
- The NPU class ethdev hardwares are tightly coupled with external
  mempool ops with HW offload, and there is provision to utilize these features
- For the general purpose arm64 perspective, At least for the low-end systems,
  the cache hierarchy is quite different from x86. So its costly to deference
  the mbuf area(which stores the mempool handler) after the Tx free. We
  can support both use cases, just that it should configurable based on
  the flags by the application.

> Please Jerin, could you work on moving these settings in a new API?

Sure. Once the generic code is in place. We are committed to fix the
PMDs by 18.02.

Jerin