Re: [dpdk-dev] [PATCH v2] ethdev: document RSS default key and types

[Ori Kam <orika@mellanox.com>]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Ophir Munk
> Sent: Friday, November 9, 2018 10:14 AM
> To: Yongseok Koh <yskoh@mellanox.com>; Adrien Mazarguil
> <adrien.mazarguil@6wind.com>; Andrew Rybchenko
> <arybchenko@solarflare.com>
> Cc: Ferruh Yigit <ferruh.yigit@intel.com>; dev@dpdk.org; Thomas Monjalon
> <thomas@monjalon.net>; Asaf Penso <asafp@mellanox.com>; Shahaf Shuler
> <shahafs@mellanox.com>; Olga Shern <olgas@mellanox.com>
> Subject: Re: [dpdk-dev] [PATCH v2] ethdev: document RSS default key and types
> 
> 
> 
> > -----Original Message-----
> > From: Yongseok Koh
> > Sent: Friday, November 09, 2018 1:07 AM
> > To: Ophir Munk <ophirmu@mellanox.com>; Adrien Mazarguil
> > <adrien.mazarguil@6wind.com>; Andrew Rybchenko
> > <arybchenko@solarflare.com>
> > Cc: Ferruh Yigit <ferruh.yigit@intel.com>; dev@dpdk.org; Thomas Monjalon
> > <thomas@monjalon.net>; Asaf Penso <asafp@mellanox.com>; Shahaf
> > Shuler <shahafs@mellanox.com>; Olga Shern <olgas@mellanox.com>
> > Subject: Re: [dpdk-dev] [PATCH v2] ethdev: document RSS default key and
> > types
> >
> >
> > > On Nov 8, 2018, at 12:50 AM, Ophir Munk <ophirmu@mellanox.com>
> > wrote:
> > >
> > >
> > >
> > >> -----Original Message-----
> > >> From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com]
> > >> Sent: Wednesday, November 07, 2018 6:41 PM
> > >> To: Ophir Munk <ophirmu@mellanox.com>
> > >> Cc: Ferruh Yigit <ferruh.yigit@intel.com>; Andrew Rybchenko
> > >> <arybchenko@solarflare.com>; dev@dpdk.org; Thomas Monjalon
> > >> <thomas@monjalon.net>; Asaf Penso <asafp@mellanox.com>; Shahaf
> > Shuler
> > >> <shahafs@mellanox.com>; Olga Shern <olgas@mellanox.com>
> > >> Subject: Re: [dpdk-dev] [PATCH v2] ethdev: document RSS default key
> > >> and types
> > >>
> > >> On Wed, Nov 07, 2018 at 03:13:07PM +0000, Ophir Munk wrote:
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com]
> > >>>> Sent: Wednesday, November 07, 2018 4:06 PM
> > >>>> To: Ophir Munk <ophirmu@mellanox.com>
> > >>>> Cc: Ferruh Yigit <ferruh.yigit@intel.com>; Andrew Rybchenko
> > >>>> <arybchenko@solarflare.com>; dev@dpdk.org; Thomas Monjalon
> > >>>> <thomas@monjalon.net>; Asaf Penso <asafp@mellanox.com>; Shahaf
> > >>>> Shuler <shahafs@mellanox.com>; Olga Shern <olgas@mellanox.com>
> > >>>> Subject: Re: [dpdk-dev] [PATCH v2] ethdev: document RSS default key
> > >>>> and types
> > >>>>
> > >>>> On Wed, Nov 07, 2018 at 12:39:24PM +0000, Ophir Munk wrote:
> > >>>>>> -----Original Message-----
> > >>>>>> From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com]
> > >>>>>> Sent: Wednesday, November 07, 2018 11:31 AM
> > >>>>>> To: Ophir Munk <ophirmu@mellanox.com>
> > >>>>>> Cc: Ferruh Yigit <ferruh.yigit@intel.com>; Andrew Rybchenko
> > >>>>>> <arybchenko@solarflare.com>; dev@dpdk.org; Thomas Monjalon
> > >>>>>> <thomas@monjalon.net>; Asaf Penso <asafp@mellanox.com>;
> > >> Shahaf
> > >>>>>> Shuler <shahafs@mellanox.com>; Olga Shern
> > >> <olgas@mellanox.com>
> > >>>>>> Subject: Re: [dpdk-dev] [PATCH v2] ethdev: document RSS default
> > >>>>>> key and types
> > >>>>>>
> > >>>>>> On Wed, Nov 07, 2018 at 09:23:42AM +0000, Ophir Munk wrote:
> > >>>>>>> struct rte_flow_action_rss include fields 'key' and 'types'.
> > >>>>>>> Field 'key' is a pointer to bytes array (uint8_t *) which
> > >>>>>>> contains the specific RSS hash key.
> > >>>>>>> If an application is only interested in default RSS operation it
> > >>>>>>> should not care about the specific hash key. The application can
> > >>>>>>> set the hash key to NULL such that any PMD uses its
> > >> default RSS key.
> > >>>>>>>
> > >>>>>>> Field 'types' is a uint64_t bits flag used to specify a specific
> > >>>>>>> RSS hash type such as ETH_RSS_IP (see ETH_RSS_*).
> > >>>>>>> If an application does not care about the specific RSS type it
> > >>>>>>> can set this field to 0 such that any PMD uses its default type.
> > >>>>>>>
> > >>>>>>> Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
> > >>>>>>> ---
> > >>>>>>> lib/librte_ethdev/rte_flow.h | 9 +++++++--
> > >>>>>>> 1 file changed, 7 insertions(+), 2 deletions(-)
> > >>>>>>>
> > >>>>>>> diff --git a/lib/librte_ethdev/rte_flow.h
> > >>>>>>> b/lib/librte_ethdev/rte_flow.h index c0fe879..ca9e135 100644
> > >>>>>>> --- a/lib/librte_ethdev/rte_flow.h
> > >>>>>>> +++ b/lib/librte_ethdev/rte_flow.h
> > >>>>>>> @@ -1782,10 +1782,15 @@ struct rte_flow_action_rss {
> > >>>>>>> 	 * through.
> > >>>>>>> 	 */
> > >>>>>>> 	uint32_t level;
> > >>>>>>> -	uint64_t types; /**< Specific RSS hash types (see
> > >> ETH_RSS_*). */
> > >>>>>>> +	/**
> > >>>>>>> +	 * Specific RSS hash types (see ETH_RSS_*),
> > >>>>>>> +	 * or 0 for PMD specific default.
> > >>>>>>> +	 */
> > >>>>>>> +	uint64_t types;
> > >>>>>>> 	uint32_t key_len; /**< Hash key length in bytes. */
> > >>>>>>> 	uint32_t queue_num; /**< Number of entries in @p queue.
> > >> */
> > >>>>>>> -	const uint8_t *key; /**< Hash key. */
> > >>>>>>> +	/** Hash key, or NULL for PMD specific default key. */
> > >>>>>>> +	const uint8_t *key;
> > >>>>>>
> > >>>>>> I'd suggest to document that on key_len instead. If key_len is
> > >>>>>> nonzero, key cannot be NULL anyway.
> > >>>>>
> > >>>>> The decision if a key/len combination is valid is done in the PMD
> > >>>>> action
> > >>>> validation API.
> > >>>>> For example, in MLX5 key==NULL and key_len==40 is accepted.
> > >>>>> The combination key==NULL and key_len==0 should always succeeds,
> > >>>> however the "must" parameter for RSS default is key==NULL and not
> > >>>> key_len==0.
> > >>>>
> > >>>> I understand this is how the mlx5 PMD implemented it, but my point
> > >>>> is that it makes more sense API-wise to define key_len == 0 as the
> > >>>> trigger for a default RSS hash key than key == NULL.
> > >>>>
> > >>>> My suggestion is to follow the same trend as memcpy(), mmap(),
> > >>>> snprintf() and other well-known functions that take a size when
> > >>>> dealing with NULL/undefined pointers. Only size matters! :)
> > >>>>
> > >>>
> > >>> Please let's stay backward compatible and consistent with previous
> > >>> dpdk releases where key==NULL is used in struct rte_eth_rss_conf
> > >>> (see
> > >> code snippet in [1]).
> > >>
> > >> And I thought I wouldn't hear again from that structure after
> > >> ac8d22de2394
> > >> ("ethdev: flatten RSS configuration in flow API") got rid of it in
> > >> rte_flow
> > >> :)
> > >>
> > >> There is no need to be backward compatible with it particularly if
> > >> doing so improves consistency (assuming you agree it does).
> > >>
> > >> Take "rss_hf" versus "types" fields for instance [2]. Unsetting the
> > >> former disables RSS completely while unsetting the latter provides
> > >> default RSS settings for that flow rule, simply because a RSS action
> > >> that doesn't do RSS makes no sense (one could provide a single queue for
> > that).
> > >>
> > >> Regarding "key_len", have a look at this other message [3] in the
> > >> same thread which clearly states that i40e expects a zero key length
> > >> to trigger default RSS, it doesn't rely on a NULL pointer.
> > >>
> > >> Generally speaking, I'm pushing for zero values in action objects to
> > >> result in a safe default behavior. A nonzero key_len with a NULL key
> > >> may result in a crash, while the reverse is inherently safer since
> > >> one doesn't even need to check the size or pointer validity, e.g.:
> > >>
> > >> memcpy(target_device_rss_stuff, rss_conf->key, rss_conf->key_len);
> > >>
> > >> What's not to like?
> > >>
> > >
> > > Some think key==NULL && key_len!=0 is senseless while other think
> > key!=NULL && key_len==0 is senseless.
> > > Let's agree on both key==NULL && key_len==0.
> >
> > A bug's already been found by our QA team. The following command
> > crashes.
> >   flow create 0 ingress pattern eth / ipv6 / udp / end actions rss queues 0 end
> > level 2 types udp end key_len 40 / end
> >
> > due to null pointer access in the following code:
> >
> > lib/librte_ethdev/rte_flow.c
> > @@ -464,7 +464,7 @@ rte_flow_conv_action_conf(void *buf, const size_t
> > size,
> >                            }),
> >                            size > sizeof(*dst.rss) ? sizeof(*dst.rss) : size);
> >                 off = sizeof(*dst.rss);
> >                 if (src.rss->key_len) {
> >                         off = RTE_ALIGN_CEIL(off, sizeof(*dst.rss->key));
> >                         tmp = sizeof(*src.rss->key) * src.rss->key_len;
> >                         if (size >= off + tmp)
> >                                 dst.rss->key = rte_memcpy
> >                                         ((void *)((uintptr_t)dst.rss + off),
> >                                          src.rss->key, tmp);
> >                         off += tmp;
> >                 }
> >
> >
> > I wanted to submit a patch to add a sanity check,
> >
> > -               if (src.rss->key_len) {
> > +               if (src.rss->key && src.rss->key_len) {
> >
> > but looks like we should conclude this thread first?
> > Or, does the fix make any sense regardless of having key_len=0 or key=null
> > for default key?
> > Having more sanity check is no harm usually...
> >
> >
> > Thanks,
> > Yongseok
> >
> 
> The setfault is the result of commit a4391f8bae ("app/testpmd: set default RSS
> key as null").
> Reverting this commit should fix the segfault but it also means there is no way
> to set default key (key=NULL) with testpmd.
> Need to check if this is only a testpmd limitation and not all applications
> limitation.
> 
> We should decide how an application can set default RSS without knowing
> anything about keys.
> 

I agree with Adrian that the main criteria should be the length.
Maybe the set default RSS in testpmd should get new parameter.

> >
> > >
> > >> [2] "Re: [PATCH v2 07/15] ethdev: flatten RSS configuration in flow API"
> > >>
> > >>
> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma
> > >> i
> > >> ls.dpdk.org%2Farchives%2Fdev%2F2018-
> > >>
> > April%2F096235.html&amp;data=02%7C01%7Cophirmu%40mellanox.com%
> > >>
> > 7C4fdd2e5a838047c152c408d644cfe439%7Ca652971c7d2e4d9ba6a4d14925
> > >>
> > 6f461b%7C0%7C0%7C636772057013784575&amp;sdata=6JsomMO68lJoiNd
> > >> ehRaNdn2xG9BSAgqCjxwpJJJOP8A%3D&amp;reserved=0
> > >>
> > >> [3] "Re: [dpdk-dev] [PATCH v6 07/16] ethdev: flatten RSS configuration in
> > >>    flow API"
> > >>
> > >>
> > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fma
> > >> i
> > >> ls.dpdk.org%2Farchives%2Fdev%2F2018-
> > >>
> > May%2F100128.html&amp;data=02%7C01%7Cophirmu%40mellanox.com%7
> > >>
> > C4fdd2e5a838047c152c408d644cfe439%7Ca652971c7d2e4d9ba6a4d149256
> > >>
> > f461b%7C0%7C0%7C636772057013784575&amp;sdata=TXDTKLho8qyKlH%2
> > >> FqBturGHWhvdfaACuMoqvl0eOjfcU%3D&amp;reserved=0
> > >>
> > >>> We should avoid confusing applications with setting key==NULL with
> > >> legacy RSS and key_len==0 with rte_flows.
> > >>>
> > >>> With regard to trends APIs - I was thinking about free() where a
> > >>> NULL pointer is acceptable :)
> > >>
> > >> Yeah, though free() doesn't take a size. On the other hand the
> > >> behavior of
> > >> realloc() is much more interesting when faced with a zero size :)
> > >> That's probably one of the few exceptions so I guess my point still stands.
> > >>
> > >>>
> > >>> [1]
> > >>> File lib/librte_ethdev/rte_ethdev.h:
> > >>>
> > >>> /**
> > >>> * A structure used to configure the Receive Side Scaling (RSS)
> > >>> feature
> > >>> * of an Ethernet port.
> > >>> * If not NULL, the *rss_key* pointer of the *rss_conf* structure
> > >>> points
> > >>> * to an array holding the RSS key to use for hashing specific header
> > >>> * fields of received packets. The length of this array should be
> > >>> indicated
> > >>> * by *rss_key_len* below. Otherwise, a default random hash key is
> > >>> used by
> > >>> * the device driver.
> > >>> *
> > >>> * The *rss_key_len* field of the *rss_conf* structure indicates the
> > >>> length
> > >>> * in bytes of the array pointed by *rss_key*. To be compatible, this
> > >>> length
> > >>> * will be checked in i40e only. Others assume 40 bytes to be used as
> > >> before.
> > >>> *
> > >>> * ..........
> > >>> */
> > >>> struct rte_eth_rss_conf {
> > >>> 	uint8_t *rss_key;    /**< If not NULL, 40-byte hash key. */
> > >>> 	uint8_t rss_key_len; /**< hash key length in bytes. */
> > >>> 	uint64_t rss_hf;     /**< Hash functions to apply - see below. */
> > >>> };
> > >>
> > >> --
> > >> Adrien Mazarguil
> > >> 6WIND

Best,
Ori