From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2F3DBA0548; Sun, 18 Apr 2021 17:00:24 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 06928410E5; Sun, 18 Apr 2021 17:00:24 +0200 (CEST) Received: from sender11-of-o51.zoho.eu (sender11-of-o51.zoho.eu [31.186.226.237]) by mails.dpdk.org (Postfix) with ESMTP id 66E17410D7 for ; Sun, 18 Apr 2021 17:00:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; t=1618758021; cv=none; d=zohomail.eu; s=zohoarc; b=UMIFSirwJ2X2MpEqaE239NRQtIMy2Yzq4msKGKXVtgHr4eynQSmFURUsem7u0Jtjaj8Q53A+jLm0R9oxomhwkuvREnirSufUtbmolr+PXDVfIjThdNes23AHQgopTCAH1M58Lf9LWWkNHdHDaSKKfWwjGl5XRs/haQgkx0U4MKs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.eu; s=zohoarc; t=1618758021; h=Content-Type:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=3fPInJSNbU90XXaDqxEAR/oZuZdyhJ36Buafx/++QWI=; b=SaRVQNXyw1OjvyI1xrtJiOXUubR8mbwS1gnieI9CjG2cd9qLrud/lKMr6v+V7BHK9tPWXviJ9ASLmSmYVc/yV8aP1W7hiEDLkj+Si1WtZKAEU/l0hSxXasPto5Z06tt2QIYR9A9Rvn6Ls13RLO4+1eGwG/6528LKLpOZF2r1wgo= ARC-Authentication-Results: i=1; mx.zohomail.eu; spf=pass smtp.mailfrom=liangma@liangbit.com; dmarc=pass header.from= header.from= Received: from DESKTOP-POQV63C.localdomain (51.37.188.134 [51.37.188.134]) by mx.zoho.eu with SMTPS id 1618758019057700.8554063249254; Sun, 18 Apr 2021 17:00:19 +0200 (CEST) Date: Sun, 18 Apr 2021 16:00:16 +0100 From: Liang Ma To: "Burakov, Anatoly" Cc: Bruce Richardson , dev@dpdk.org Message-ID: <20210418150016.GA41@DESKTOP-POQV63C.localdomain> References: <20210413182441.GC1185@bricha3-MOBL.ger.corp.intel.com> <20210414104803.GA514@bricha3-MOBL.ger.corp.intel.com> <20210414131301.GB514@bricha3-MOBL.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-ZohoMailClient: External Subject: Re: [dpdk-dev] Question about enable CBDMA with ioat PMD ? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Apr 14, 2021 at 04:33:10PM +0100, Burakov, Anatoly wrote: > On 14-Apr-21 2:45 PM, Liang Ma wrote: > > On Wed, Apr 14, 2021 at 02:21:53PM +0100, Bruce Richardson wrote: > > > On Wed, Apr 14, 2021 at 01:52:14PM +0100, Liang Ma wrote: > > > > On Wed, Apr 14, 2021 at 11:48:03AM +0100, Bruce Richardson wrote: > > > > > On Wed, Apr 14, 2021 at 11:37:48AM +0100, Liang Ma wrote: > > > > > > On Tue, Apr 13, 2021 at 07:24:41PM +0100, Bruce Richardson wrote: > > > > > > > On Tue, Apr 13, 2021 at 06:36:38PM +0100, Liang Ma wrote: > > > > > > > > Hi Bruce, > > > > > > > > I look into current IOAT PMD. I found that looks only support the > > > > > > > > latest DSA device which is part of SPR(Please correct me if I'm wrong). > > > > > > > > However, there still is lots existing CPU that support previous CBDMA > > > > > > > > Intel Quick-DATA technology. SPDK enable those device with uio. > > > > > > > > Any hint to enable those CBDMA device with IOAT PMD ? > > > > > > > > > > > > > > Hi Liang, > > > > > > > > > > > > > > These devices are still supported by the driver. In the latest releases the > > > > > > > new device support was added but no older device support was removed. > > > > > > > Please let us know if you do hit any problems with running it on the CBDMA > > > > > > > HW devices, because it should all still be working as before. > > > > > > > > > > > > > > Regards, > > > > > > > /Bruce > > > > > > Hi Bruce, > > > > > > I hit a problem with CBDMA enabling. > > > > > > > > > > > > 1. I follow the DPDK IOAT driver Doc, I bind 0000.04.0 device with > > > > > > dpdk-devbind.py > > > > > > 2. I run dpdk-test without any parameter then dpdk-test complain with > > > > > > > > > > > > EAL: VFIO support initialized > > > > > > EAL: 0000:00:04.0 VFIO group is not viable! Not all devices in IOMMU > > > > > > group bound to VFIO or unbound > > > > > > Requested device 0000:00:04.0 cannot be used > > > > > > > > > > > > 3. kernel boot parameter include the "intel_iommu=on" > > > > > > > > > > > > If I still need some extra iommu settings for CBDMA device ? > > > > > > > > > > > It's an IOMMU limitation, not just for CBDMA but for other devices like > > > > > NICs too - you need to ensure all subfunctions on a device are all bound to > > > > > the same IOMMU group. So for the CBDMA instance, to use one channel with > > > > > vfio, the other channels on the device also need to be bound to vfio, or > > > > > else not bound to any driver. If bound to the kernel driver, there will be > > > > > IOMMU conflicts as we can't split the channels between kernel and userspace > > > > > address spaces. > > > > > > > > > > /Bruce > > > > I bind whole device from 00:04.0 to 00:04.7. ioat PMD complain > > > > with "ioat_rawdev_create(): ioat_rawdev_create: Channel appears > > > > locked". Any hint? > > > > Regards > > > > Liang > > > > > > That should be a warning only, I think and the device should still be > > > available. Is that not the case, or is there a later error? > > > > > > /Bruce > > Good news, that's a warning only message, I checked the code. > > after probe the device, rawdev_autotest all passed ! > > Many thanks for your help ! > > I strongly recommend to add the IOMMU hint of CBDMA(bind whole device) > > into the Doc. I think DSA device don't need to do that right ? > > Best Regards > > Liang > > > > This is documented in our VFIO docs [1]. > > [1] http://doc.dpdk.org/guides/linux_gsg/linux_drivers.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules Thanks Anatoly.