DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Xia, Chenbo" <chenbo.xia@intel.com>
To: Parav Pandit <parav@nvidia.com>,
	NBU-Contact-Thomas Monjalon <thomas@monjalon.net>,
	"Yigit, Ferruh" <ferruh.yigit@intel.com>
Cc: "Xueming(Steven) Li" <xuemingl@nvidia.com>,
	Slava Ovsiienko <viacheslavo@nvidia.com>,
	"dev@dpdk.org" <dev@dpdk.org>, Jason Gunthorpe <jgg@nvidia.com>
Subject: Re: [dpdk-dev] [RFC 00/14] mlx5: support SubFunction
Date: Tue, 15 Jun 2021 11:19:01 +0000	[thread overview]
Message-ID: <MN2PR11MB4063B8711879B9DBF42105439C309@MN2PR11MB4063.namprd11.prod.outlook.com> (raw)
In-Reply-To: <PH0PR12MB548193B30CD9F6AC7B814602DC309@PH0PR12MB5481.namprd12.prod.outlook.com>

Hi Parav,

> -----Original Message-----
> From: Parav Pandit <parav@nvidia.com>
> Sent: Tuesday, June 15, 2021 1:43 PM
> To: Xia, Chenbo <chenbo.xia@intel.com>; NBU-Contact-Thomas Monjalon
> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>
> Cc: Xueming(Steven) Li <xuemingl@nvidia.com>; Slava Ovsiienko
> <viacheslavo@nvidia.com>; dev@dpdk.org; Jason Gunthorpe <jgg@nvidia.com>
> Subject: RE: [dpdk-dev] [RFC 00/14] mlx5: support SubFunction
> 
> 
> 
> > From: Xia, Chenbo <chenbo.xia@intel.com>
> > Sent: Tuesday, June 15, 2021 11:03 AM
> >
> > Hi Parav,
> >
> > > -----Original Message-----
> > > From: Parav Pandit <parav@nvidia.com>
> > > Sent: Tuesday, June 15, 2021 12:05 PM
> > > To: Xia, Chenbo <chenbo.xia@intel.com>; NBU-Contact-Thomas Monjalon
> > > <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>
> > > Cc: Xueming(Steven) Li <xuemingl@nvidia.com>; Slava Ovsiienko
> > > <viacheslavo@nvidia.com>; dev@dpdk.org; Jason Gunthorpe
> > > <jgg@nvidia.com>
> > > Subject: RE: [dpdk-dev] [RFC 00/14] mlx5: support SubFunction
> > >
> > > Hi Chenbo,
> > >
> > > > From: Xia, Chenbo <chenbo.xia@intel.com>
> > > > Sent: Tuesday, June 15, 2021 7:41 AM
> > > >
> > > > Hi Thomas,
> > > >
> > > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > > > Sent: Friday, June 11, 2021 3:54 PM
> > > [..]
> > >
> > > >
> > > > Yes. In our term it's called Assignable Device Interface (ADI)
> > > > introduced in Intel Scalable IOV
> > > > (https://01.org/blogs/2019/assignable-interfaces-intel-
> > > > scalable-i/o-virtualization-linux)
> > > >
> > > > And vfio-mdev is chosen to be the software framework for it. I start
> > > > to
> > > realize
> > > > there is difference between SF and ADI: SF considers multi-function
> > > > devices which may include net/regex/vdpa/...
> > > Yes. net, rdma, vdpa, regex ++.
> > > And eventually vfio_device to map to VM too.
> > >
> > > Non mdev framework is chosen so that all the use cases of kernel only,
> > > or user only or mix modes can be supported.
> >
> > OK. Got it.
> >
> > >
> > > > But ADI only focuses on the
> > > > virtualization of the devices and splitting devices to logic parts
> > > > and
> > > providing
> > > > huge number of interfaces to host APP. I think SF also considers
> > > > this but is mainly used for multi-function devices (like DPU in your
> term?
> > > > Correct me if I'm wrong).
> > > >
> > > SF also supports DPU mode too but it is in addition to above use cases.
> > > SF will expose mdev (or a vfio_device) to map to a VM.
> >
> > So your SW actually supports vfio-mdev? I suppose the device-specific mdev
> > Kernel module is out-of-tree?
> >
> mlx5 driver doesn't support vfio_device for SFs.
> Kernel plumbing for PASID assignment to SF is WIP currently kernel community.
> We do not have any out-of-tree kernel module.
> 
> > Just FYI:
> >
> > We are introducing a new mdev bus for DPDK:
> > http://patchwork.dpdk.org/project/dpdk/cover/20210601030644.3318-1-
> > chenbo.xia@intel.com/
> >
> I am yet to read about it. But I am not sure what value does it add.
> A user can open a vfio device using vfio subsystem and operate on it.
> A vfio device can be a create as a result of binding PCI VF/PF to vfio-pci
> driver or a SF by binding SF to vfio_foo driver.

Yes, in general it is the way. For vfio-mdev, it works as binding the vfio-mdev
to parent device and echo uuid to create a virtual device. VFIO APP like DPDK,
as you said, should work similar with VFIO UAPI for vfio-pci devices or mdev-based
devices. But currently DPDK only cares about vfio-pci devices and does not care
things for other cases like mdev-based pci devices. For example, it does not scan
/sys/bus/mdev and it always uses pci bdf as device address, which mdev-based pci
devices do not have. Therefore I sent that patchset.

> There is kernel work in progress to use vfio core as library.

OK. Could you share me some link to it? Much appreciated.

> So we do not anticipate to use add mdev layer and uuid to create a vfio device
> for a SF.

OK. For now, we are following the vfio-mdev standard, using UUID to create vfio
devices.

> 
> For Intel, ADI will never has any netdevs or rdma dev?

I think technically it could have. But for some devices like our dma devices, it's
just using mdev:

https://www.spinics.net/lists/kvm/msg244417.html

Thanks,
Chenbo


  reply	other threads:[~2021-06-15 11:19 UTC|newest]

Thread overview: 108+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-27 13:37 Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 01/14] common/mlx5: add common device driver Xueming Li
2021-06-10  9:51   ` Thomas Monjalon
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 00/14] net/mlx5: support Sub-Function Xueming Li
2021-07-21 14:37     ` [dpdk-dev] [PATCH v4 00/16] " Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 01/16] common/mlx5: rename eth device class name Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 02/16] common/mlx5: add common device driver Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 03/16] common/mlx5: move description of PCI sysfs functions Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 04/16] common/mlx5: support auxiliary bus Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 05/16] common/mlx5: get PCI device address from any bus Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 06/16] net/mlx5: remove PCI dependency Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 07/16] net/mlx5: migrate to bus-agnostic common driver Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 08/16] net/mlx5: support SubFunction Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 09/16] net/mlx5: check max Verbs port number Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 10/16] regex/mlx5: migrate to common driver Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 11/16] vdpa/mlx5: define driver name as macro Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 12/16] vdpa/mlx5: remove PCI specifics Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 13/16] vdpa/mlx5: support SubFunction Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 14/16] compress/mlx5: migrate to common driver Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 15/16] crypto/mlx5: " Xueming Li
2021-07-21 14:37       ` [dpdk-dev] [PATCH v4 16/16] common/mlx5: clean up legacy PCI bus driver Xueming Li
2021-07-21 22:24       ` [dpdk-dev] [PATCH v4 00/16] net/mlx5: support Sub-Function Thomas Monjalon
2021-07-22  3:03         ` Xueming(Steven) Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 01/14] common/mlx5: add common device driver Xueming Li
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 00/14] net/mlx5: support Sub-Function Xueming Li
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 01/14] common/mlx5: add common device driver Xueming Li
2021-07-14  5:58       ` Slava Ovsiienko
2021-07-18 18:28       ` Thomas Monjalon
2021-07-19  4:05         ` Xueming(Steven) Li
2021-07-19  2:53       ` [dpdk-dev] [PATCH v3 00/15] net/mlx5: support Sub-Function Xueming Li
2021-07-19  2:53       ` [dpdk-dev] [PATCH v3 01/15] common/mlx5: rename eth device class name Xueming Li
2021-07-19  2:53       ` [dpdk-dev] [PATCH v3 02/15] common/mlx5: add common device driver Xueming Li
2021-07-19  2:53       ` [dpdk-dev] [PATCH v3 03/15] common/mlx5: move description of PCI sysfs functions Xueming Li
2021-07-19  2:53       ` [dpdk-dev] [PATCH v3 04/15] common/mlx5: support auxiliary bus Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 05/15] common/mlx5: get PCI device address from any bus Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 06/15] net/mlx5: remove PCI dependency Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 07/15] net/mlx5: migrate to bus-agnostic common driver Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 08/15] net/mlx5: support SubFunction Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 09/15] net/mlx5: check max Verbs port number Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 10/15] regex/mlx5: migrate to common driver Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 11/15] vdpa/mlx5: define driver name as macro Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 12/15] vdpa/mlx5: remove PCI specifics Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 13/15] vdpa/mlx5: support SubFunction Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 14/15] compress/mlx5: migrate to common driver Xueming Li
2021-07-19  2:54       ` [dpdk-dev] [PATCH v3 15/15] common/mlx5: clean up legacy PCI bus driver Xueming Li
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 02/14] common/mlx5: move description of PCI sysfs functions Xueming Li
2021-07-14  5:58       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 03/14] common/mlx5: support auxiliary bus Xueming Li
2021-07-14  5:58       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 04/14] common/mlx5: get PCI device address from any bus Xueming Li
2021-07-14  5:59       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 05/14] net/mlx5: remove PCI dependency Xueming Li
2021-07-14  5:59       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 06/14] net/mlx5: migrate to bus-agnostic common driver Xueming Li
2021-07-14  5:59       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 07/14] net/mlx5: support SubFunction Xueming Li
2021-07-14  5:59       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 08/14] net/mlx5: check max Verbs port number Xueming Li
2021-07-14  6:00       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 09/14] regex/mlx5: migrate to common driver Xueming Li
2021-07-14  6:00       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 10/14] vdpa/mlx5: define driver name as macro Xueming Li
2021-07-14  6:00       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 11/14] vdpa/mlx5: remove PCI specifics Xueming Li
2021-07-14  6:08       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 12/14] vdpa/mlx5: support SubFunction Xueming Li
2021-07-14  6:01       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 13/14] compress/mlx5: migrate to common driver Xueming Li
2021-07-14  6:01       ` Slava Ovsiienko
2021-07-13 13:14     ` [dpdk-dev] [PATCH v2 14/14] common/mlx5: clean up legacy PCI bus driver Xueming Li
2021-07-14  6:01       ` Slava Ovsiienko
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 02/14] common/mlx5: move description of PCI sysfs functions Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 03/14] common/mlx5: support auxiliary bus Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 04/14] common/mlx5: get PCI device address from any bus Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 05/14] net/mlx5: remove PCI dependency Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 06/14] net/mlx5: migrate to bus-agnostic common driver Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 07/14] net/mlx5: support SubFunction Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 08/14] net/mlx5: check max Verbs port number Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 09/14] regex/mlx5: migrate to common driver Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 10/14] vdpa/mlx5: define driver name as macro Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 11/14] vdpa/mlx5: remove PCI specifics Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 12/14] vdpa/mlx5: support SubFunction Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 13/14] compress/mlx5: migrate to common driver Xueming Li
2021-06-16  4:09   ` [dpdk-dev] [PATCH v1 14/14] common/mlx5: clean up legacy PCI bus driver Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 02/14] common/mlx5: move description of PCI sysfs functions Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 03/14] net/mlx5: remove PCI dependency Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 04/14] net/mlx5: migrate to bus-agnostic common driver Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 05/14] regex/mlx5: migrate to " Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 06/14] compress/mlx5: " Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 07/14] vdpa/mlx5: fix driver name Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 08/14] vdpa/mlx5: remove PCI specifics Xueming Li
2021-05-27 13:37 ` [dpdk-dev] [RFC 09/14] common/mlx5: clean up legacy PCI bus driver Xueming Li
2021-05-27 14:01 ` [dpdk-dev] [RFC 10/14] bus/auxiliary: introduce auxiliary bus Xueming Li
2021-05-27 14:01   ` [dpdk-dev] [RFC 11/14] common/mlx5: support " Xueming Li
2021-05-27 14:02   ` [dpdk-dev] [RFC 12/14] common/mlx5: get PCI device address from any bus Xueming Li
2021-05-27 14:02   ` [dpdk-dev] [RFC 13/14] vdpa/mlx5: support SubFunction Xueming Li
2021-05-27 14:02   ` [dpdk-dev] [RFC 14/14] net/mlx5: " Xueming Li
2021-06-10 10:33 ` [dpdk-dev] [RFC 00/14] mlx5: " Ferruh Yigit
2021-06-10 13:23   ` Thomas Monjalon
2021-06-11  5:14     ` Xia, Chenbo
2021-06-11  7:54       ` Thomas Monjalon
2021-06-15  2:10         ` Xia, Chenbo
2021-06-15  4:04           ` Parav Pandit
2021-06-15  5:33             ` Xia, Chenbo
2021-06-15  5:43               ` Parav Pandit
2021-06-15 11:19                 ` Xia, Chenbo [this message]
2021-06-15 12:47                   ` Parav Pandit
2021-06-15 15:19                     ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=MN2PR11MB4063B8711879B9DBF42105439C309@MN2PR11MB4063.namprd11.prod.outlook.com \
    --to=chenbo.xia@intel.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=jgg@nvidia.com \
    --cc=parav@nvidia.com \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@nvidia.com \
    --cc=xuemingl@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).