Re: [dpdk-dev] Question about enable CBDMA with ioat PMD ?

[Liang Ma <liangma@liangbit.com>]

On Wed, Apr 14, 2021 at 11:48:03AM +0100, Bruce Richardson wrote:
> On Wed, Apr 14, 2021 at 11:37:48AM +0100, Liang Ma wrote:
> > On Tue, Apr 13, 2021 at 07:24:41PM +0100, Bruce Richardson wrote:
> > > On Tue, Apr 13, 2021 at 06:36:38PM +0100, Liang Ma wrote:
> > > > Hi Bruce, 
> > > >    I look into current IOAT PMD. I found that looks only support the
> > > >    latest DSA device which is part of SPR(Please correct me if I'm wrong).
> > > >    However, there still is lots existing CPU that support previous CBDMA
> > > >    Intel Quick-DATA technology. SPDK enable those device with uio. 
> > > >    Any hint to enable those CBDMA device with IOAT PMD ? 
> > > >    
> > > 
> > > Hi Liang,
> > > 
> > > These devices are still supported by the driver. In the latest releases the
> > > new device support was added but no older device support was removed.
> > > Please let us know if you do hit any problems with running it on the CBDMA
> > > HW devices, because it should all still be working as before.
> > > 
> > > Regards,
> > > /Bruce
> > Hi Bruce, 
> >    I hit a problem with CBDMA enabling. 
> > 
> >    1. I follow the DPDK  IOAT driver Doc, I bind 0000.04.0 device with
> >       dpdk-devbind.py 
> >    
> >    2. I run dpdk-test without any parameter then dpdk-test complain with 
> > 
> >        EAL: VFIO support initialized
> >        EAL: 0000:00:04.0 VFIO group is not viable! Not all devices in IOMMU
> >        group bound to VFIO or unbound
> >        Requested device 0000:00:04.0 cannot be used
> > 
> >    3. kernel boot parameter include the "intel_iommu=on"
> > 
> >    If I still need some extra iommu settings for CBDMA device ?
> > 
> It's an IOMMU limitation, not just for CBDMA but for other devices like
> NICs too - you need to ensure all subfunctions on a device are all bound to
> the same IOMMU group. So for the CBDMA instance, to use one channel with
> vfio, the other channels on the device also need to be bound to vfio, or
> else not bound to any driver. If bound to the kernel driver, there will be
> IOMMU conflicts as we can't split the channels between kernel and userspace
> address spaces.
> 
> /Bruce
I bind whole device from 00:04.0 to 00:04.7.   ioat PMD  complain
with  "ioat_rawdev_create(): ioat_rawdev_create: Channel appears
locked".  Any hint?
Regards
Liang