Re: [PATCH v1] bus/pci: revise support PASID control

[Chenbo Xia <chenbox@nvidia.com>]

On Nov 17, 2023, at 01:43, Chen, Mike Ximing <mike.ximing.chen@intel.com> wrote:
> 
> External email: Use caution opening links or attachments
> 
> 
>> -----Original Message-----
>> From: Chenbo Xia <chenbox@nvidia.com>
>> Sent: Tuesday, November 14, 2023 8:54 PM
>> To: Sevincer, Abdullah <abdullah.sevincer@intel.com>
>> Cc: dev@dpdk.org; jerinj@marvell.com; Chen, Mike Ximing
>> <mike.ximing.chen@intel.com>; Richardson, Bruce
>> <bruce.richardson@intel.com>; NBU-Contact-Thomas Monjalon (EXTERNAL)
>> <thomas@monjalon.net>; Marchand, David <david.marchand@redhat.com>;
>> nipun.gupta@amd.com
>> Subject: Re: [PATCH v1] bus/pci: revise support PASID control
>> 
>> On Nov 15, 2023, at 01:39, Sevincer, Abdullah <abdullah.sevincer@intel.com>
>> wrote:
>>> 
>>> External email: Use caution opening links or attachments
>>> 
>>> 
>>>> +I don’t know about the details, so it means for different devices that support
>> PASID, they have different offsets?
>>> 
>>>> +Btw, Is this cap still not exposed to user space in latest kernel?
>>> 
>>> Yes, may be different offsets for different devices.
>> 
>> But why? It’s not standard capability? In my understanding, standard cap should
>> have the same offset definitions for all devices.
> 
> PASID is a part of extended capabilities. Its offset can be different for different devices.
> 
>> 
>>> As of now it is not exposed to user. Bruce's test was on 6.2 generic
>>> kernel (6.2.0-36-generic)
>> 
>> Will kernel plan to support that? I can see the related work was done by Intel but
>> somehow it’s not merged into kernel. Could you give more information on this?
>> 
> Hi Chenbo,
> As you may know there has been a lot of changes in iommu/vfio/SVA/pasid/SIOV
> support in Linux kernel recently. The PASID used to be disabled, but starting with
> kernel 6.2 it is enabled in vfio-pci driver by default.  We did contact the kernel developers
> on this issue. They seem to insist that enabling PASID is needed for whatever new features
> they are developing. This breaks the DLB PF PMD as DLB HW requires the PASID to be
> disable for PF to operate properly (otherwise the HW put DLB in a different mode). We
> will continue to talk to the kernel developers on this issue, but in the meantime would like
> to provide this patch so that DPDK PF PMD can still work with latest kernels.
> 
> In term of exposing the PASID capability to the user space. We are aware of some patches
> Submitted in conjunction to the changes mentioned above, for example,
> https://lkml.iu.edu/hypermail/linux/kernel/2309.3/02380.html
> But we don’t know when and if it will be accepted into the kernel. Hopefully the patch will
> be accepted so we don’t have to use the hard coded offset.
> 
>> If kernel does not want this to be exposed, it means userspace should not access
>> this. No?
>> 
> The action (disabling PASID) only applies the targeted device. In the DLB PF PMD case,
> the DPDK has full control of the device via vfio-pci.  It does not affect kernel and any
> other device's operation.

Thanks for the long explanation.

Hope to see follow-up in DPDK when this capability get exposed to user later :)

/Chenbo

> 
> Thanks
> Mike
>> /Chenbo
>