Re: [dpdk-dev] [dpdk-stable] [PATCH] net/iavf: fix multi-process shared data

["Zhang, Qi Z" <qi.z.zhang@intel.com>]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Ferruh Yigit
> Sent: Thursday, September 30, 2021 6:57 PM
> To: Yu, DapengX <dapengx.yu@intel.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing,
> Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; stable@dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-stable] [PATCH] net/iavf: fix multi-process shared
> data
> 
> On 9/30/2021 10:11 AM, Yu, DapengX wrote:
> >
> >
> >> -----Original Message-----
> >> From: Yigit, Ferruh <ferruh.yigit@intel.com>
> >> Sent: Thursday, September 30, 2021 12:28 AM
> >> To: Yu, DapengX <dapengx.yu@intel.com>; Richardson, Bruce
> >> <bruce.richardson@intel.com>; Ananyev, Konstantin
> >> <konstantin.ananyev@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>;
> >> Xing, Beilei <beilei.xing@intel.com>
> >> Cc: dev@dpdk.org; stable@dpdk.org
> >> Subject: Re: [dpdk-stable] [PATCH] net/iavf: fix multi-process shared
> >> data
> >>
> >> On 9/28/2021 4:37 AM, dapengx.yu@intel.com wrote:
> >>> From: Dapeng Yu <dapengx.yu@intel.com>
> >>>
> >>> When the iavf_adapter instance is not initialized completedly in the
> >>> primary process, the secondary process accesses its "rte_eth_dev"
> >>> member, it causes secondary process crash.
> >>>
> >>> This patch replaces adapter->eth_dev with rte_eth_devices[port_id]
> >>> in the data paths where rte_eth_dev instance is accessed.
> >>>
> >>> Fixes: f978c1c9b3b5 ("net/iavf: add RSS hash parsing in AVX path")
> >>> Fixes: 9c9aa0040344 ("net/iavf: add offload path for Rx AVX512 flex
> >>> descriptor")
> >>> Fixes: 63660ea3ee0b ("net/iavf: add RSS hash parsing in SSE path")
> >>> Cc: stable@dpdk.org
> >>>
> >>> Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
> >>> ---
> >>>  drivers/net/iavf/iavf_rxtx_vec_avx2.c   | 5 +++--
> >>>  drivers/net/iavf/iavf_rxtx_vec_avx512.c | 5 +++--
> >>>  drivers/net/iavf/iavf_rxtx_vec_sse.c    | 3 ++-
> >>>  3 files changed, 8 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/drivers/net/iavf/iavf_rxtx_vec_avx2.c
> >>> b/drivers/net/iavf/iavf_rxtx_vec_avx2.c
> >>> index 475070e036..59b086ade5 100644
> >>> --- a/drivers/net/iavf/iavf_rxtx_vec_avx2.c
> >>> +++ b/drivers/net/iavf/iavf_rxtx_vec_avx2.c
> >>> @@ -525,6 +525,7 @@ _iavf_recv_raw_pkts_vec_avx2_flex_rxd(struct
> >>> iavf_rx_queue *rxq,  #define IAVF_DESCS_PER_LOOP_AVX 8
> >>>
> >>>  	const uint32_t *type_table = rxq->vsi->adapter->ptype_tbl;
> >>> +	struct rte_eth_dev *dev = &rte_eth_devices[rxq->port_id];
> >>>
> >>
> >> It is not good idea to access global variable directly from the driver.
> > In "lib/ethdev/rte_ethdev.h", the global variable rte_eth_devices is used.
> > So I think use it in a PMD should be also acceptable since it is just read.
> 
> It is expected for ehtdev APIs to access the array. Application knows only
> port_id, ethdev layer converts this port_id to device struct by accessing the
> global array, and drivers should be able to operate only with its device.
> 
> > rte_eth_rx_burst(uint16_t port_id, uint16_t queue_id,
> > 		 struct rte_mbuf **rx_pkts, const uint16_t nb_pkts) {
> > 	struct rte_eth_dev *dev = &rte_eth_devices[port_id];
> >
> >>
> >> The problem definition is correct, eth_dev is unique per process, so
> >> it can't be saved to a shared struct.
> >>
> >> But here I assume real intention is to be able to access PMD specific
> >> data from queue struct, for this what about storing
> >> 'rte_eth_dev_data' in the 'iavf_rx_queue', this should sove the
> >> problem without accessing the global variable.
> >
> > The intention is to read the offload properties of device
> > configuration, so it not queue specific or PMD specific. It is already in public
> data structure.
> > If it is stored in 'iavf_rx_queue' again, the data will be duplicate.
> >
> 
> I can see the intention. This is more design concern, you can access to that
> data structure doesn't mean you should.
> 
> You will just store the pointer of the 'data', is it duplication?

+1, access rte_eth_devices directly is not a good practice in PMD. 

I think to fix the knowing issue, we can just replace eth_dev with eth_dev_data in iavf_adapter. (this is actually what PF's fix do)

And to avoid long pointer chain like " rxq->vsi->adapter->eth_dev->data->dev_conf.rxmode.offloads" in data path.

We should introduce per queue cache, but this could be in a separate patch.