RE: [PATCH v8 05/12] net/nfp: add flower PF setup logic

[Chaoyong He <chaoyong.he@corigine.com>]

> On 9/9/2022 3:36 AM, Chaoyong He wrote:
> >> On 9/8/2022 9:44 AM, Chaoyong He wrote:
> >>> Adds the vNIC initialization logic for the flower PF vNIC. The
> >>> flower firmware exposes this vNIC for the purposes of fallback
> >>> traffic in the switchdev use-case.
> >>>
> >>> Adds minimal dev_ops for this PF device. Because the device is being
> >>> exposed externally to DPDK it should also be configured using DPDK
> >>> helpers like rte_eth_configure(). For these helpers to work the
> >>> flower logic needs to implements a minimal set of dev_ops.
> >>>
> >>> Signed-off-by: Chaoyong He <chaoyong.he@corigine.com>
> >>> Reviewed-by: Niklas Söderlund <niklas.soderlund@corigine.com>
> 
> <...>
> 
> >>> +static int
> >>> +nfp_flower_init_pf_vnic(struct nfp_net_hw *hw) {
> >>> +	int ret;
> >>> +	uint16_t i;
> >>> +	uint16_t n_txq;
> >>> +	uint16_t n_rxq;
> >>> +	uint16_t port_id;
> >>> +	unsigned int numa_node;
> >>> +	struct rte_mempool *mp;
> >>> +	struct nfp_pf_dev *pf_dev;
> >>> +	struct rte_eth_dev *eth_dev;
> >>> +	struct nfp_app_fw_flower *app_fw_flower;
> >>> +
> >>> +	static const struct rte_eth_conf port_conf = {
> >>> +		.rxmode = {
> >>> +			.mq_mode  = RTE_ETH_MQ_RX_RSS,
> >>> +			.offloads = RTE_ETH_RX_OFFLOAD_CHECKSUM,
> >>> +		},
> >>> +		.txmode = {
> >>> +			.mq_mode = RTE_ETH_MQ_TX_NONE,
> >>> +		},
> >>> +	};
> >>> +
> >>> +	/* Set up some pointers here for ease of use */
> >>> +	pf_dev = hw->pf_dev;
> >>> +	app_fw_flower = NFP_PRIV_TO_APP_FW_FLOWER(pf_dev-
> >>> app_fw_priv);
> >>> +
> >>> +	/*
> >>> +	 * Perform the "common" part of setting up a flower vNIC.
> >>> +	 * Mostly reading configuration from hardware.
> >>> +	 */
> >>> +	ret = nfp_flower_init_vnic_common(hw, "pf_vnic");
> >>> +	if (ret != 0)
> >>> +		goto done;
> >>> +
> >>> +	hw->eth_dev = rte_eth_dev_allocate("nfp_pf_vnic");
> >>> +	if (hw->eth_dev == NULL) {
> >>> +		ret = -ENOMEM;
> >>> +		goto done;
> >>> +	}
> >>> +
> >>> +	/* Grab the pointer to the newly created rte_eth_dev here */
> >>> +	eth_dev = hw->eth_dev;
> >>> +
> >>> +	numa_node = rte_socket_id();
> >>> +
> >>> +	/* Fill in some of the eth_dev fields */
> >>> +	eth_dev->device = &pf_dev->pci_dev->device;
> >>> +	eth_dev->data->dev_private = hw;
> >>> +
> >>> +	/* Create a mbuf pool for the PF */
> >>> +	app_fw_flower->pf_pktmbuf_pool = nfp_flower_pf_mp_create();
> >>> +	if (app_fw_flower->pf_pktmbuf_pool == NULL) {
> >>> +		ret = -ENOMEM;
> >>> +		goto port_release;
> >>> +	}
> >>> +
> >>> +	mp = app_fw_flower->pf_pktmbuf_pool;
> >>> +
> >>> +	/* Add Rx/Tx functions */
> >>> +	eth_dev->dev_ops = &nfp_flower_pf_vnic_ops;
> >>> +
> >>> +	/* PF vNIC gets a random MAC */
> >>> +	eth_dev->data->mac_addrs = rte_zmalloc("mac_addr",
> >> RTE_ETHER_ADDR_LEN, 0);
> >>> +	if (eth_dev->data->mac_addrs == NULL) {
> >>> +		ret = -ENOMEM;
> >>> +		goto mempool_cleanup;
> >>> +	}
> >>> +
> >>> +	rte_eth_random_addr(eth_dev->data->mac_addrs->addr_bytes);
> >>> +	rte_eth_dev_probing_finish(eth_dev);
> >>> +
> >>> +	/* Configure the PF device now */
> >>> +	n_rxq = hw->max_rx_queues;
> >>> +	n_txq = hw->max_tx_queues;
> >>> +	port_id = hw->eth_dev->data->port_id;
> >>> +
> >>> +	ret = rte_eth_dev_configure(port_id, n_rxq, n_txq, &port_conf);
> >>
> >> Still not sure about PMD calling 'rte_eth_dev_configure()', can you
> >> please give more details on what specific configuration is expected with
> that call?
> >
> > The main configuration we need is the number of rx/tx queue.
> > So we should use the internal api `eth_dev_rx/tx_queue_config` to instead?
> >
> 
> nb_rx_q/nb_tx_q are parameters provided by user (via
> rte_eth_dev_configure()), won't is wrong for PMD to set a value on its own?
> 
> Why nb_rx_q/nb_tx_q are required in the probe() stage? Probe stage is not
> to configure the device.

Our nfp card use `control message` to exchange message between PMD and firmware when we use flower firmware.
The control message is in the form of pkt and we use a `ctrl vNIC` ehtdev as the agent to send and receive these pkts.
e.g., if we want to create representor port, the PMD must send the corresponding control message to firmware.

This `ctrl vNIC` is totally user app Invisible,  to make it able to send and receive pkt, we must do some configure steps to this ethdev ourselves firstly.
We can don't use 'rte_eth_dev_configure()', but we still should do the needed configure steps to make sure the device can work.