From: Slava Ovsiienko <viacheslavo@nvidia.com>
To: Jan Viktorin <viktorin@cesnet.cz>
Cc: "dev@dpdk.org" <dev@dpdk.org>, Asaf Penso <asafp@nvidia.com>,
Shahaf Shuler <shahafs@nvidia.com>,
Matan Azrad <matan@nvidia.com>
Subject: Re: [dpdk-dev] [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev mode
Date: Mon, 15 Mar 2021 19:31:25 +0000 [thread overview]
Message-ID: <DM6PR12MB3753ECA40CD54808E32EBF98DF6C9@DM6PR12MB3753.namprd12.prod.outlook.com> (raw)
In-Reply-To: <20210315171039.16b3965e@tanguero.localdomain>
Hi, Jan
1. Let's suppose we have the one port NIC, PCI BDF 08:00.0, SR-IOV enabled, 0 VFs is configured,
netdev name is ens1f0 (depends on distro and kernel version).
2. Then, we configure 3 VFs with SR-IOV, we get :
- 08:00.0 - PF, ens1f0
- 08:00.1 - VF0, ens1f1
- 08:00.2 - VF1, ens1f2
- 08:00.3 - VF2, ens1f3
3. Then, we should unbind VFs ONLY, PF should be kept untouched.
- 08:00.0 - PF, ens1f0
- 08:00.1 - VF0, -
- 08:00.2 - VF1, -
- 08:00.3 - VF2, -
4. Then, configure switchdev mode
- 08:00.0 - PF, ens1f0
- 08:00.1 - VF0, -
- 08:00.2 - VF1, -
- 08:00.3 - VF2, -
+ we get 3 representor netdevs:
ens1f0_0 (no dedicated PCI device)
ens1f0_1 (no dedicated PCI device)
ens1f0_2 (no dedicated PCI device)
5. Then, bind VFs back (assume we are no going to map them to VMs)
- 08:00.0 - PF, ens1f0
- 08:00.1 - VF0, ens1f1
- 08:00.2 - VF1, ens1f2
- 08:00.3 - VF2, ens1f3
- ens1f0_0 (no dedicated PCI device)
- ens1f0_1 (no dedicated PCI device)
- ens1f0_2 (no dedicated PCI device)
6. As result we should get 7 netdevs - 1 PF (Uplink representor), 3 VFs, 3 representors (no PCI device, pure netdev)
With best regards, Slava
> -----Original Message-----
> From: Jan Viktorin <viktorin@cesnet.cz>
> Sent: Monday, March 15, 2021 18:11
> To: Slava Ovsiienko <viacheslavo@nvidia.com>
> Cc: dev@dpdk.org; Asaf Penso <asafp@nvidia.com>; Shahaf Shuler
> <shahafs@nvidia.com>; Matan Azrad <matan@nvidia.com>
> Subject: Re: [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev
> mode
>
> Hello Salva,
>
> On Mon, 15 Mar 2021 15:53:51 +0000
> Slava Ovsiienko <viacheslavo@nvidia.com> wrote:
>
> > Hi, Jan
> >
> > Yes, bullet [4] explicitly requires to unbind VFs, and detach the netdevs
> from the mlx5_core driver.
> > Otherwise, kernel driver refuses to be configured with switchdev mode in
> [5]. So, [4] can't be skipped.
> > After setting swithdev mode, VFs can be bound back (if it is needed, and
> these ones are not mapped to VMs):
>
> OK, but I believe that it is **not possible** to follow the rule [5].
> The guide explicitly says in [4] "can be rebind **after** the switchdev
> mode".
> Just, if you unbind the device, there is no way how to configure the
> switchdev mode, this is the contradiction I mentioned in the commit. You
> cannot configure switchdev mode because the interface is gone and the path
> /sys/class/net/<net device>/compat/devlink/mode no longer exists.
>
> So, maybe, just the formulation is wrong. So, what is the **exact
> right** way how to do it? I would change the commit accordingly. Just, let's
> make it right. Would it work this way?
>
> # echo -n "<device pci address>" > /sys/bus/pci/drivers/mlx5_core/unbind
> # echo -n "<device pci address>" > /sys/bus/pci/drivers/mlx5_core/bind
> # echo switchdev > /sys/class/net/<net device>/compat/devlink/mode
>
> It is good to mention that after the rebind, the <net device> can change.
>
> Regards,
> Jan
>
> >
> > echo -n "<device pci address>" > > /sys/bus/pci/drivers/mlx5_core/bind
> >
> > With best regards,
> > Slava
> >
> > > -----Original Message-----
> > > From: Jan Viktorin <iviktorin@fit.vutbr.cz>
> > > Sent: Monday, March 15, 2021 17:34
> > > To: dev@dpdk.org
> > > Cc: Jan Viktorin <viktorin@cesnet.cz>; Asaf Penso
> > > <asafp@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Slava
> > > Ovsiienko <viacheslavo@nvidia.com>; Matan Azrad
> <matan@nvidia.com>
> > > Subject: [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev
> > > mode
> > >
> > > From: Jan Viktorin <viktorin@cesnet.cz>
> > >
> > > The step 4 is a contradiction. It advices to unbind the device from
> > > the mlx5_core which removes the associated system network interface
> (e.g.
> > > eth0). In the step 5, the same system network interface (e.g. eth0)
> > > is required to exist.
> > >
> > > Signed-off-by: Jan Viktorin <viktorin@cesnet.cz>
> > > ---
> > > doc/guides/nics/mlx5.rst | 6 +-----
> > > 1 file changed, 1 insertion(+), 5 deletions(-)
> > >
> > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst
> > > index 0a2dc3dee..122d8e0fc 100644
> > > --- a/doc/guides/nics/mlx5.rst
> > > +++ b/doc/guides/nics/mlx5.rst
> > > @@ -1370,11 +1370,7 @@ the DPDK application.
> > >
> > > echo <num of vfs > /sys/class/net/<net
> > > device>/device/sriov_numvfs
> > >
> > > -4. Unbind the device (can be rebind after the switchdev mode)::
> > > -
> > > - echo -n "<device pci address>" >
> > > /sys/bus/pci/drivers/mlx5_core/unbind
> > > -
> > > -5. Enable switchdev mode::
> > > +4. Enable switchdev mode::
> > >
> > > echo switchdev > /sys/class/net/<net
> > > device>/compat/devlink/mode
> > >
> > > --
> > > 2.30.1
> >
next prev parent reply other threads:[~2021-03-15 19:31 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-15 15:33 [dpdk-dev] [PATCH 1/2] net/mlx5: fix typos Jan Viktorin
2021-03-15 15:33 ` [dpdk-dev] [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev mode Jan Viktorin
2021-03-15 15:53 ` Slava Ovsiienko
2021-03-15 16:11 ` Jan Viktorin
2021-03-15 19:31 ` Slava Ovsiienko [this message]
2021-03-15 19:48 ` Jan Viktorin
2021-03-15 19:52 ` Slava Ovsiienko
2021-03-15 20:30 ` Jan Viktorin
2021-07-15 14:51 ` Raslan Darawsheh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM6PR12MB3753ECA40CD54808E32EBF98DF6C9@DM6PR12MB3753.namprd12.prod.outlook.com \
--to=viacheslavo@nvidia.com \
--cc=asafp@nvidia.com \
--cc=dev@dpdk.org \
--cc=matan@nvidia.com \
--cc=shahafs@nvidia.com \
--cc=viktorin@cesnet.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).