From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4CBDCA0032; Thu, 16 Dec 2021 18:33:29 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C436540143; Thu, 16 Dec 2021 18:33:28 +0100 (CET) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 39F704013F for ; Thu, 16 Dec 2021 18:33:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1639676007; x=1671212007; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=E1gaMeTD7vWzlnK8PO+M0b8vDwUk8cwyXkmiqx71dZ0=; b=lg8xQfU14xyoHf/dnnhD6rLA4Lpoafa5WmpPy8TWyGwsC4HR7lGF7RZF Nntkq2Ydx50pWz0nP67m7aTZz95IWxgrBzoPpIG0UEnMV18ECClPWBhIH i1IFEdfgUtaeEX6QuQtAFXwUdOLM/LEKx4f4sPJtFaFA6Jj15f248XzIj X6CgtcVGAyBS6TgNOV+laSUSBTarkbAKQ0LDYXJ6qcfM+Wt/efv9pmGd3 xnW24PoNDWBjscnr7uq5AoYrZjiJtERKNffdzMs6YEEbmwdHJWpTncHu/ P/W8TKvidRQhKabW1ROgQLjhGMkqEyPaI9h+zaYpCbOl/e4Cmohmf6+Ej g==; X-IronPort-AV: E=McAfee;i="6200,9189,10199"; a="237090370" X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="237090370" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Dec 2021 09:33:21 -0800 X-IronPort-AV: E=Sophos;i="5.88,211,1635231600"; d="scan'208";a="466151567" Received: from bricha3-mobl.ger.corp.intel.com ([10.252.20.185]) by orsmga006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-SHA; 16 Dec 2021 09:33:20 -0800 Date: Thu, 16 Dec 2021 17:33:16 +0000 From: Bruce Richardson To: Lance Richardson Cc: dev , conor.walsh@intel.com, kevin.laatz@intel.com Subject: Re: Using IOAT PMD Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Dec 16, 2021 at 11:34:25AM -0500, Lance Richardson wrote: > On Thu, Dec 16, 2021 at 11:20 AM Bruce Richardson > wrote: > > > > On Thu, Dec 16, 2021 at 11:04:54AM -0500, Lance Richardson wrote: > > > Hi Bruce, > > > > > > I've been looking into using the IOAT PMD, initially with dma_autotest > > > and the dpdk-dma example application. These seem to work fine on > > > SKX with the current main branch, but when I try the same procedure > > > on ICX (binding all 8 devices to vfio-pci in both cases), I get the following > > > output for each device when probed. Is something different needed when > > > using IOAT on ICX vs. SKX? > > > > > > Thanks, > > > Lance > > > > > > EAL: Probe PCI driver: dmadev_ioat (8086:b00) device: 0000:80:01.0 (socket 2) > > > IOAT: ioat_dmadev_probe(): Init 0000:80:01.0 on NUMA node 2 > > > IOAT: ioat_dmadev_create(): ioat_dmadev_create: Channel count == 255 > > > > > > IOAT: ioat_dmadev_create(): ioat_dmadev_create: Channel appears locked > > > > > > IOAT: ioat_dmadev_create(): ioat_dmadev_create: cannot reset device. > > > CHANCMD=0xff, CHANSTS=0xffffffffffffffff, CHANERR=0xffffffff > > > > > > EAL: Releasing PCI mapped resource for 0000:80:01.0 > > > EAL: Calling pci_unmap_resource for 0000:80:01.0 at 0x4102430000 > > > EAL: Requested device 0000:80:01.0 cannot be used > > > > That is strange, the same PMD should work ok on both platforms. This is all > > on latest branch, right? Let me attempt to reproduce and get back to you. > > Hi Bruce, > > That's correct, I'm using the current tip of the main branch, which > seems to be identical to 21.11.0. > > > > /Bruce > > > > PS: Is this a 4-socket system you are running on, since I see "socket 2" > > being described as the socket number for device 80:01.0? > > > It is a two-socket system with sub-NUMA enabled, so it appears as four > NUMA nodes. I'm only binding the devices on the second socket. > Ok, [not that that should affect anything to do with ioat, AFAIK] Tried quickly reproducing the issue on some of our systems and failed to do so. Does this error appear consistently, especially after a reboot? Thanks, /Bruce