From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 278BFA0C41; Tue, 7 Sep 2021 12:12:02 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 180B74114B; Tue, 7 Sep 2021 12:12:02 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id C728C410ED for ; Tue, 7 Sep 2021 12:11:59 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10099"; a="283866378" X-IronPort-AV: E=Sophos;i="5.85,274,1624345200"; d="scan'208";a="283866378" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Sep 2021 03:11:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,274,1624345200"; d="scan'208";a="546736061" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by fmsmga002.fm.intel.com with ESMTP; 07 Sep 2021 03:11:58 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Tue, 7 Sep 2021 03:11:58 -0700 Received: from fmsmsx605.amr.corp.intel.com (10.18.126.85) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Tue, 7 Sep 2021 03:11:58 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Tue, 7 Sep 2021 03:11:58 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.177) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.10; Tue, 7 Sep 2021 03:11:57 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F6NYrBsQiL8vBI37Msxfz8X2kGNCBIiFg6GcnACdNc9NaYsbe2ivACMABy8jpYDVjGhDnDGVa7dFF0XV5aNe8a+8cPNpr9KisznQhILBARkkP1dhBKc9dlC20qMp1amVqpd/yxFtShJWeWRzDlxIAdJmwGYmdRz2TJYH9VSsr1/oaOFjiHiTZPrqBWluH9oKF74U8LjNRybiGyZgBWbkTUB8vv/6XV9xsqSlBvt5HJk3Jm1obNzjuI1Cxk1qDDwYFLkGWdOqmUdKxVehW/DWFbiTOsQTRAJ4lhSrSvvwEGP2Xm7a8MZ+XUcndtkIYz9W9/NKNgNUA2HgYMv6yKUkcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=CsvVI4CZgfiFB9jFD2rjMk7JDP4tJwRuevgTEHAqPi0=; b=W/7+i19CX5TO+bJQT94lqVpSpWucyeGrh9L9PuPw5wl5hzUUb7yvYdAP/NvgBGsi5ZlrTG0R8FIMjFGOjcuXTEEQAlbSUpE8OE7gojEvf8Gx7RP+oqwHdZ6DL96YIsKHbW3k6zc+4u+dwUUBrUL7fqsGoy3VE7mK8P+Ypp6G2GfMymuJ7wvHbMuIiQ0hvgu895ow6XLxHxVRz7qSH2krlEZa0l0ggxPop6PwER+nmsi7u8BUSlA9ZvY8D/DnYdlhg5UwIhnRcqGnNaWcVaIT7CkMDNSj2beTkzcIOTLfSRdnlJNuIXqKAQw8iTSxYdcAZdgA9vcz8T23PAKVNxav1g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CsvVI4CZgfiFB9jFD2rjMk7JDP4tJwRuevgTEHAqPi0=; b=s6Ll1awvBNCUbSajn5Rkgx49N9ea5FctK2emmDGCvbNt2sObBzv+HkbmTMbmu/tMrtNR338UfXOaJPwGAt3wa9K8XHKnvbo4GwFQVDySXzvSG8/1GGZje0dsAfT+U0c1pOxsF0i52C5Ngy2cdNtFzTKOgYlDVn2+D95q5trewqw= Authentication-Results: dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=none action=none header.from=intel.com; Received: from SJ0PR11MB5055.namprd11.prod.outlook.com (2603:10b6:a03:2d9::10) by BYAPR11MB3736.namprd11.prod.outlook.com (2603:10b6:a03:f6::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.22; Tue, 7 Sep 2021 10:11:55 +0000 Received: from SJ0PR11MB5055.namprd11.prod.outlook.com ([fe80::78d4:6f82:aa87:c271]) by SJ0PR11MB5055.namprd11.prod.outlook.com ([fe80::78d4:6f82:aa87:c271%5]) with mapi id 15.20.4478.025; Tue, 7 Sep 2021 10:11:55 +0000 From: Kevin Laatz To: Conor Walsh , , , CC: References: <20210827172550.1522362-1-conor.walsh@intel.com> <20210903111734.2734545-1-conor.walsh@intel.com> <20210903111734.2734545-6-conor.walsh@intel.com> Message-ID: Date: Tue, 7 Sep 2021 11:11:51 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 In-Reply-To: <20210903111734.2734545-6-conor.walsh@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-ClientProxiedBy: DB8PR04CA0028.eurprd04.prod.outlook.com (2603:10a6:10:110::38) To SJ0PR11MB5055.namprd11.prod.outlook.com (2603:10b6:a03:2d9::10) MIME-Version: 1.0 Received: from [192.168.1.15] (109.79.106.71) by DB8PR04CA0028.eurprd04.prod.outlook.com (2603:10a6:10:110::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.14 via Frontend Transport; Tue, 7 Sep 2021 10:11:54 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c574446f-61d0-42cb-25d5-08d971e7ebd0 X-MS-TrafficTypeDiagnostic: BYAPR11MB3736: X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0xX/zwRVcddeo29M/1qflSPRjUcNJQH2hqEEVUf6PfL/eYj9zFNpz6K85S1OVlLioJvFFWGwbczqq+4bBjpBwnzd+cQ5Mor5shtR7yFMLdO0/y3Z4+jNGla8K6UmQ42FD62O39ZRmAFto9ZxRSP0kuw6c0JFOGN03UX5uDrj9QB/go9j26j27GtJRM3XsrpIQs7POJSu3M6XuDxP0Go09WzfnOVTic+gPGP0qvsKs6ULaSNlLdN3oLOmr/KBUGl37ILYMn5STplRXD88cvGPWpMPOgivI1YuQg+YrOr+ip/lAGx0rZFt7wOIbYF1jFri2C6bYBhnBwXW8sm9/K1nNL1wrwD2p9dx8nsJe25wDDU3rIg9WlH52pCbxw9Q8D0Sy6ccP1Vup9T304c0/xU8S/Jc8BO2u62hMacSDx5WAyML76qu24WuzAhwXHG034jTn0RS9XD5PoTSN3Jvl+sXnAKrhOc87Uzuctvdycla1/QjZHmUurW0x5z948HMYdlE4alCIkO8XZE4DkQazWkmSZL0nZ33vME3a/YJZt/LAPf2SHUY1bSfMt/ZXSYCdsPJBqFFH/mb7Kpryg7atqL9LUGZHid1iDzuzT74Qua0cRzZmEEGhdHu9oyaLRhCntcgWr6/iatuMcC1uVWKWBcKnBh/htC9XjWSybGnpFtAJNEiJzoMIEirR9CRyfxhfeGkn0WS+/J1WVhkD9/yFCqLzXY4yzK2tAkqI1LfJSxFmwQ= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB5055.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39860400002)(136003)(376002)(396003)(346002)(366004)(31686004)(478600001)(956004)(186003)(6486002)(83380400001)(26005)(38100700002)(5660300002)(2616005)(8676002)(4326008)(53546011)(66556008)(66476007)(66946007)(6666004)(8936002)(31696002)(44832011)(36756003)(2906002)(16576012)(86362001)(316002)(45980500001)(43740500002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?M2Y2NGlobWxscklSQ2o1ZElxVnh6blZ5NnVlTzF6SHlIWHYzQzlKNE5IMmdy?= =?utf-8?B?RWJRNWJEY2duRUlJUCt0YlZJZWRyeWZXdTg1SGUvS1IyeUpxcWpvMnBUSE1I?= =?utf-8?B?TmFZVmlwWEUxd05NTWdsMW1vQllFRDhQUy9jem9ZcnFvN2xHazJRcG9sU1pN?= =?utf-8?B?UkVzcXFYeGFmYURma0FQUCtieXJKZkZEZXJTK3F3TUFrOUw0ZnpPdkkxVU9M?= =?utf-8?B?WTBEbUVzdThmRm9NU3FzR0hJSUFIYVAyZFhBQkZZRTBvY29RMy9Ma0todGQw?= =?utf-8?B?bS9FL2ZjclVncVNmTmZpUnZLK29BR3phRVFKUFNaRHIwRHdaenNxdUoxNW1C?= =?utf-8?B?WjhyKzlkQ0VGOHdLeDdVVVF6SUczYThueUMvMVJ2YVEyMUJSOWNKaThWekx0?= =?utf-8?B?bnB0cUpuQ2pyUnlwOWhhYVUxK24vOEh6czI5QVl2L1ZXZVhSOG1PYXBOR3Ir?= =?utf-8?B?UzRJNlVpSElQdTBVY0Y0emFwKzhZVXlUZ2gvQzVIZW54bHdiYm9PNXJwKzQ3?= =?utf-8?B?YS9ZQ3ZGL21jWXZNQkVueklld1hGUCtQb1YwOE1mWGM1WkdaTFE5WDZtSzZh?= =?utf-8?B?VXJTU1MvWndZVnBnakFSWGQ2SWVubW1kUnZYVjhwc3BOakZ3Z3Q1VGVxdFM2?= =?utf-8?B?aWFlVFV1NjF3Q1EvZlNSVFhKL1RGL0RTdy91aFFaZUVRWlZXeVVCQ25ReHp0?= =?utf-8?B?cTVpTGdHb3RLZlJDUFRFeDZmd3pDRVBOYW1oaDhvS0tqbUd3anRZRVRlMHgr?= =?utf-8?B?b1gyM3Z5Mk5VWGF2Z2lyUWNKcTJkT0xpMFZLQzA5R0owbW1HaXFoS3FmMXl3?= =?utf-8?B?a1haeGlKZUtTOUl5SlhmL2daOEdTVEorK1VpNkhVV2U4RzlYZDhCU0docm51?= =?utf-8?B?RGJMUUZSZHhiUE1qMHhhOEFpc0xjUWVtWThuTDR3WDRHSk93VmlCL2xhQnJ5?= =?utf-8?B?UWZBd1o1SlIrRnVHaXRzd0J5dDdEUWxKYVpKdGpRYk5nQVg1em9EZ1FqYlJE?= =?utf-8?B?KysvT3RxQzhuc0VVOVV5T2c0Rm40YlBpa1Q0OXpTQ1lVRk9yU2ZTUytNSWF5?= =?utf-8?B?VC85VkpTdlZESjcyS3Iyamx0TGM0cFlycXpicnFROVkyRjA2bFZYS3FNT3Y1?= =?utf-8?B?L0M4YnhpTStaZytMYmNLZE9HTmRkckRRK1NoeHBtck44MlR2ZW11MnUzZ1lT?= =?utf-8?B?SjI3T2VvN28rb3NlVGJCMHFwL3FndnVrR1J4dFJwSUlsNUlVUDFwamNYZmNh?= =?utf-8?B?ZUlWN3V6SEs2WU1kMnlwMHUwTnlibUNxekRGaUZUNVRybTQ5MWxZWlhYeWJi?= =?utf-8?B?Nmo2eVBGQVNZTkt6Yzdtd1JOazRGT1pBUjcvMzFCZ0h6VWlueDJ4Yk9MZ2N1?= =?utf-8?B?d0FQRVNVK1J2K0d3M25FMktQQWZnaXFWUWlUWnlNMnlVWTB6YWJHUW5BMDM0?= =?utf-8?B?TTViODVBZjFCRXJHWGFqMk5KcW01Y0pVelF3NmxaL0hsdGphdmd1dk8yMTc5?= =?utf-8?B?TW9wMW5QdkZvazkzZGtLSmxEV25Rc29JdVExRUd5cWJ3YzgrOGZJNWZuekRh?= =?utf-8?B?bS80V29NWlE0aWQzQXlpNVRIc0J0NDljeDdhTUFSUjhIa2xGUm9RMWl2dy9k?= =?utf-8?B?d3hmZTk4YUkxb1hsSXk3SXBxeDJVZ3N6bXpPSWp3cTdMMkxwS215SE5SOVVK?= =?utf-8?B?WFRHU2xQTU9aRUtYYVJ3aFUveDJCMkpaaDRYMjdKWmV4NVZ2M0Z4ZHNUSGJU?= =?utf-8?Q?pHMHxIUZRFqR0v52NZ5HHBwBCb2cH2YU6lVY4yY?= X-MS-Exchange-CrossTenant-Network-Message-Id: c574446f-61d0-42cb-25d5-08d971e7ebd0 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB5055.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 10:11:55.8382 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IuFifLbUHxkmz/N80H9dgMzibetnwNxkhCxT9IGKvgJMn/dONSrc+PKaI7Nto54pEet9azaPr4vxiVZ3boID4A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3736 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v2 05/10] dma/ioat: add start and stop functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 03/09/2021 12:17, Conor Walsh wrote: > Add start, stop and recover functions for IOAT devices. > > Signed-off-by: Conor Walsh > Signed-off-by: Bruce Richardson > --- > doc/guides/dmadevs/ioat.rst | 3 ++ > drivers/dma/ioat/ioat_dmadev.c | 87 ++++++++++++++++++++++++++++++++++ > 2 files changed, 90 insertions(+) > > diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst > index b6d88fe966..f7742642b5 100644 > --- a/doc/guides/dmadevs/ioat.rst > +++ b/doc/guides/dmadevs/ioat.rst > @@ -86,3 +86,6 @@ The following code shows how the device is configured in ``test_dmadev.c``: > :start-after: Setup of the dmadev device. 8< > :end-before: >8 End of setup of the dmadev device. > :dedent: 1 > + > +Once configured, the device can then be made ready for use by calling the > +``rte_dmadev_start()`` API. > diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c > index 94f9139e0d..9f9feecd49 100644 > --- a/drivers/dma/ioat/ioat_dmadev.c > +++ b/drivers/dma/ioat/ioat_dmadev.c > @@ -73,6 +73,91 @@ ioat_vchan_setup(struct rte_dmadev *dev, uint16_t vchan __rte_unused, > return 0; > } > > +/* Recover IOAT device. */ > +static inline int > +__ioat_recover(struct ioat_dmadev *ioat) > +{ > + uint32_t chanerr, retry = 0; > + uint16_t mask = ioat->qcfg.nb_desc - 1; > + > + /* Clear any channel errors. Reading and writing to chanerr does this. */ > + chanerr = ioat->regs->chanerr; > + ioat->regs->chanerr = chanerr; > + > + /* Reset Channel. */ > + ioat->regs->chancmd = IOAT_CHANCMD_RESET; > + > + /* Write new chain address to trigger state change. */ > + ioat->regs->chainaddr = ioat->desc_ring[(ioat->next_read - 1) & mask].next; > + /* Ensure channel control and status addr are correct. */ > + ioat->regs->chanctrl = IOAT_CHANCTRL_ANY_ERR_ABORT_EN | > + IOAT_CHANCTRL_ERR_COMPLETION_EN; > + ioat->regs->chancmp = ioat->status_addr; > + > + /* Allow HW time to move to the ARMED state. */ > + do { > + rte_pause(); > + retry++; > + } while (ioat->regs->chansts != IOAT_CHANSTS_ARMED && retry < 200); > + > + /* Exit as failure if device is still HALTED. */ > + if (ioat->regs->chansts != IOAT_CHANSTS_ARMED) > + return -1; > + > + /* Store next write as offset as recover will move HW and SW ring out of sync. */ > + ioat->offset = ioat->next_read; > + > + /* Prime status register with previous address. */ > + ioat->status = ioat->desc_ring[(ioat->next_read - 2) & mask].next; > + > + return 0; > +} > + > +/* Start a configured device. */ > +static int > +ioat_dev_start(struct rte_dmadev *dev) > +{ > + struct ioat_dmadev *ioat = dev->dev_private; > + > + if (ioat->qcfg.nb_desc == 0 || ioat->desc_ring == NULL) > + return -EBUSY; > + > + /* Inform hardware of where the descriptor ring is. */ > + ioat->regs->chainaddr = ioat->ring_addr; > + /* Inform hardware of where to write the status/completions. */ > + ioat->regs->chancmp = ioat->status_addr; > + > + /* Prime the status register to be set to the last element. */ > + ioat->status = ioat->ring_addr + ((ioat->qcfg.nb_desc - 1) * DESC_SZ); > + > + printf("IOAT.status: %s [%#lx]\n", > + chansts_readable[ioat->status & IOAT_CHANSTS_STATUS], > + ioat->status); > + > + if ((ioat->regs->chansts & IOAT_CHANSTS_STATUS) == IOAT_CHANSTS_HALTED) { > + IOAT_PMD_WARN("Device HALTED on start, attempting to recover\n"); > + if (__ioat_recover(ioat) != 0) { > + IOAT_PMD_ERR("Device couldn't be recovered"); > + return -1; > + } > + } > + > + return 0; > +} > + > +/* Stop a configured device. */ > +static int > +ioat_dev_stop(struct rte_dmadev *dev) > +{ > + struct ioat_dmadev *ioat = dev->dev_private; > + > + ioat->regs->chancmd = IOAT_CHANCMD_SUSPEND; > + /* Allow the device time to suspend itself. */ > + rte_delay_ms(1); > + > + return 0; It be more beneficial to check if the device has actually suspended before returning. Similar to recover, a timeout could be set by which the device is expected to be suspended. If the device is still not suspended by then, return error. IMO this would be more useful to an application that always returning 0, since the device may still be active. With the above addressed, Reviewed-by: Kevin Laatz