Re: Regarding Mellanox bifurcated driver on Azure

[Prashant Upadhyaya <praupadhyaya@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3834 bytes --]

On Wed, 30 Apr 2025 at 22:21, Stephen Hemminger <stephen@networkplumber.org>
wrote:

> On Wed, 30 Apr 2025 22:00:29 +0530
> Prashant Upadhyaya <praupadhyaya@gmail.com> wrote:
>
> > On Wed, 30 Apr 2025 at 19:58, Stephen Hemminger <
> stephen@networkplumber.org>
> > wrote:
> >
> > > On Wed, 30 Apr 2025 13:00:53 +0530
> > > Prashant Upadhyaya <praupadhyaya@gmail.com> wrote:
> > >
> > > > > With DPDK on Azure, an application should never use the VF
> directly.
> > > > > It needs to use either netvsc PMD which handles both the vmbus
> (slow
> > > path)
> > > > > and VF (fast path) combined. Or use the older
> vdev_netvsc/failsafe/tap
> > > > > combination.
> > > > > The latter uses a virtual device to make a failsafe PMD which then
> does
> > > > > a combination of TAP (via kernel slow path) and MLX5 VF.  The
> failsafe
> > > PMD
> > > > > is what is exposed for application usage.
> > > > >
> > > > > The limitations are not explicitly mentioned in the documentation
> but:
> > > > >   - don't use VF directly in application
> > > > >   - there is no support for bifurcation where some packets go to
> kernel
> > > > >     and some to DPDK
> > > > >   - there is only very limited support for rte_flow; that is with
> > > failsafe
> > > > > PMD
> > > > >     (not netvsc PMD) and the limitations are that the emulation
> of
> > > rte_flow
> > > > >     in the TAP device only supports a few things.
> > > > >
> > > >
> > > > Thanks Stephen, the above information was very instructive.
> > > > If I do use the Netvsc PMD with the latest DPDK, will my DPDK app
> get the
> > > > non IP packets like ARP, please confirm.
> > > > I quickly tried the Netvsc PMD but don't seem to be getting the ARP
> > > packets
> > > > in still.
> > > > When you mention "The failsafe PMD is what is exposed for application
> > > > usage", what is the meaning of this, are the apps expected to use
> > > failsafe
> > > > PMD, please suggest.
> > > >
> > > > Regards
> > > > -Prashant
> > >
> > > ARP handled differently in virtual network environments. The ARP
> packets
> > > sent
> > > get consumed and replied to by the network infrastructure (in all
> virtual
> > > networks
> > > not just Azure). Non-IP packets always show up on the synthetic VMBus
> > > device.
> > >
> > > Current docs are here:
> > >
> > >
> https://learn.microsoft.com/en-us/azure/virtual-network/setup-dpdk?tabs=redhat
> > >
> > > See vdev_netvsc for picture.
> > > https://doc.dpdk.org/guides/nics/vdev_netvsc.html
> > >
> > >
> > > Thanks again Stephen. I finally was able to run the netvsc pmd and it
> is
> > detecting the ports.
> > However, for every accelerated networking interface of Azure, it detects
> > 'two' ports. This is presumably for controlling both the slow and fast
> path
> > ?
> > This poses an issue for my app as it wanted to see only 'one' interface
> in
> > its control as a lot of business logic is kind of tied to it.
> > So a question -- am I observing correctly that DPDK, in case of netvsc,
> > will enumerate two ports for each accelerated networking interface ?
> >
> > Regards
> > -Prashant
>
> The hidden VF interfaces are "owned" in DPDK.
> If you use the standard API's in ethdev it will skip the owned interfaces.
>
> /**
>  * Macro to iterate over all enabled and ownerless ethdev ports.
>  */
> #define RTE_ETH_FOREACH_DEV(p) \
>         RTE_ETH_FOREACH_DEV_OWNED_BY(p, RTE_ETH_DEV_NO_OWNER)
>
>
It seems that the owner is updated at the port only after the port is
'started' -- wanted to confirm if I should go through the normal motions of
rte_eth_dev_configure, rte_eth_rx_queue_setup, rte_eth_tx_queue_setup,
rte_eth_dev_start
for 'both' the ports and after the link is detected (and the owner is set
by DPDK), do I iterate like you have suggested, and call the tx/rx burst
api's only on the non-owned port numbers for I/O.

[-- Attachment #2: Type: text/html, Size: 5517 bytes --]