From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D95FEA0C45; Wed, 22 Sep 2021 11:12:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF74041198; Wed, 22 Sep 2021 11:12:34 +0200 (CEST) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id B4C2941196 for ; Wed, 22 Sep 2021 11:12:32 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10114"; a="309106438" X-IronPort-AV: E=Sophos;i="5.85,313,1624345200"; d="scan'208";a="309106438" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2021 02:12:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,313,1624345200"; d="scan'208";a="533622271" Received: from orsmsx605.amr.corp.intel.com ([10.22.229.18]) by fmsmga004.fm.intel.com with ESMTP; 22 Sep 2021 02:12:31 -0700 Received: from orsmsx606.amr.corp.intel.com (10.22.229.19) by ORSMSX605.amr.corp.intel.com (10.22.229.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Wed, 22 Sep 2021 02:12:31 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx606.amr.corp.intel.com (10.22.229.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Wed, 22 Sep 2021 02:12:31 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.57.45) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.12; Wed, 22 Sep 2021 02:12:30 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WVvFgjQ7n/W+/eSSMje2G/2EAi7UTtpYVV5iQTI5rcjSrLvg6nmwVu1y2HrtF2CDCKDmzAKY8k7TX9W7hNR1dsM2NnVvvPg/5JsFdv2YjJMc2UZ5j4dv8rObF4eyhNpQLP98tiQT+tycZnAIjXD7FBleYU6LFAEJbAJLMaxqj06mTwsD0FTqBky+Kc5OoLbEf4pTLg57FRBc7bvatMUgVz2qsSXXylTxVPjKUhI127sd3umZUxQcfbeFQFLl23hPZMdObVO5XdwZ2qwEHTNvvFw70LIgeqAThJEuHaNnW1MFUq7wBE+NO7LHfusPkyXB9LwXbzf8viuJ0UL+EiZSFg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=12KrindMS7OVK+4q7GbhdYsV4lkdUkY+T4N6tZN4xw4=; b=OnAWJhRbnjpOQMUDnTzfmDjIyTDrLeT68r/E4Fu0MuyFvQgHT89QpEpyX2eGsdfe/bUgRWcB+DLk0zk5fggdeLQ4D6yjA+IdCD6Q7m3jHWBlemBgbgtxlamQSoqOD6MMA4VO1G2tgIjbispsSyvxoavJzll0FqpUmOa9W6p7VQ9SAkza666tFMAn1U+BhXS5bdM8u6NC27OWQHTdwuFR4h/dqSmbpZyamRqqxLO9EFA7GOdIoJf2Y4YhSXiUeAJcu8OjpmDUpFsKw8tKZ/T+D3Pyuh98H+5ynbr/Mfx8dfucGkPmlrQZtitosaNR2eCrrympp963jOEjLHItlVPgmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=12KrindMS7OVK+4q7GbhdYsV4lkdUkY+T4N6tZN4xw4=; b=iFQZkL5bQMszPJIzqMINilwslF3Q0WyTqpTBgXYCW2DdKZg0rOm4Su+KBDX3NYxql8O/WDXXtQL/kPZhud5ZpnhoTuYWoyeRu1eQ5DEcJ9SniXv1GOY8fyLJDN1k6I5Q964DcOIQ/3uMnB5YmzOPKp7LHoQB/UQdF9g9bmEoPlE= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; Received: from SJ0PR11MB5055.namprd11.prod.outlook.com (2603:10b6:a03:2d9::10) by BYAPR11MB3589.namprd11.prod.outlook.com (2603:10b6:a03:b3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.13; Wed, 22 Sep 2021 09:12:28 +0000 Received: from SJ0PR11MB5055.namprd11.prod.outlook.com ([fe80::78d4:6f82:aa87:c271]) by SJ0PR11MB5055.namprd11.prod.outlook.com ([fe80::78d4:6f82:aa87:c271%7]) with mapi id 15.20.4523.018; Wed, 22 Sep 2021 09:12:28 +0000 To: fengchengwen , CC: , , References: <20210827172048.558704-1-kevin.laatz@intel.com> <20210917152437.3270330-1-kevin.laatz@intel.com> <20210917152437.3270330-5-kevin.laatz@intel.com> From: Kevin Laatz Message-ID: Date: Wed, 22 Sep 2021 10:12:23 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-ClientProxiedBy: DB7PR05CA0056.eurprd05.prod.outlook.com (2603:10a6:10:2e::33) To SJ0PR11MB5055.namprd11.prod.outlook.com (2603:10b6:a03:2d9::10) MIME-Version: 1.0 Received: from [192.168.1.15] (109.76.83.81) by DB7PR05CA0056.eurprd05.prod.outlook.com (2603:10a6:10:2e::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.15 via Frontend Transport; Wed, 22 Sep 2021 09:12:27 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b5f8a6d3-810f-437b-ff3e-08d97da919d0 X-MS-TrafficTypeDiagnostic: BYAPR11MB3589: X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xz+TRPLypGGqwPzJI9Z7cRDqG3VfehNLDn9bvBMVuA2/X3S41ELCiMriaaXI9bqIws3tkzuceaQSfyRObYB/zP3W5FyCjLvzEPL87Ir7M6a1NFkuU3w0+y2fcjbbMtSshqvjpbc8EST8pkK9MePVpbg/c7wahm05pTBjqxetblFV23Kt7d1tKnmhgqmQSyyyhxST8+i0ce6IDCgvy/+7Sy3dfABlgu5NXsMrhv6QjopmHDWIFmkrubLB3XYroDO1E1F7u5iNBBEDAkG0Qay9KqmqOahY4hLZZkp3MaVjbxfQuj2jzhifWhpRQxLxdMWO8bC3S7LaGK6oPU+BrP2mELn4WdrZGw4V8n2OjBWB9CW7svkllwMOpjzXvHGXGzhdx7kuUnO8gwMCiudPrOOybF0ITEee0bmncruURjpVaTbSyTM1gzNN1wdJzvMlyFf4d61VOGyLYD7hU9KYq/OIsWgVh8oeYCiGieyFcCf/94zHC760z7Qsjl6vP7DlxvfMJ2O4Abr2B6SAu5qplQEWUgnCmAaAzjDRjNJ4UcGXK19UGQv0FwB9msU1/IAA4w/KjsWV6PpgRNYQa4LjIWXIpvDPBdOUWmaR5rL6rM4IbLM9DiI13h7/Ihb9og8O5QLSfJv+/s2xPs6M0j/WZlmPZ3kUDuY3zD+YH/xcW1tnwFf3YHwWcTnfompFb4znHWslNbZqQOQDpr/ELDXeerj3T8RERmRndyXmaWo9qMf+uqY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB5055.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(186003)(5660300002)(16576012)(66476007)(36756003)(53546011)(86362001)(66556008)(2906002)(8936002)(6666004)(956004)(66946007)(6486002)(38100700002)(8676002)(31696002)(316002)(31686004)(107886003)(83380400001)(508600001)(2616005)(4326008)(26005)(44832011)(43740500002)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R3dMdXFLdzNKWkxROHNRY0Y3UmF2YWNZVVpzd0tOendMNUIyM3BhSTlwVHk4?= =?utf-8?B?UnJtOWFPM1lTTUxrMGxIcDFrOStsbHlPU0ZFU01uQmZDTVZSSmVzZlIwOVJQ?= =?utf-8?B?S2NEMWVqU0lTZVNDM3NTWVNlTi9UZDRFMDVac0hyeHdDVW44YTZmazBmY0JF?= =?utf-8?B?ZW0vZWNwaVlnRkJ1T0wzMWRPWW9SeUw2ZXlmUHFId2E2QU5UcDZLWFZCb2Fp?= =?utf-8?B?My9BeXo5eGhiNEZSbTZDU2JuaVAyaDQ4U054S2ZLSmN6Z0FSUnRuT3JmOGZR?= =?utf-8?B?ajB0NFg5MEJqL250YWExd2V5K2s1YjNJQjNlalFmc0h0TFV5U0tMSEE2WDJx?= =?utf-8?B?VnJiZncyUzlPc2FpbC9yMEFpTGRoUmczOGpQdUhkVjRJUUxOZFNvY093WG1J?= =?utf-8?B?QVdDMVRUM2M2elhST0VVMnlvZVhkYUEvc3hHNkVKVlROSUkwc1dYZU1VNllO?= =?utf-8?B?UW8zcS93T3dkTFozWVpSTU8rMy9KbzZhL29tdTkwWDk5TWhPRVh1V1dWSVAy?= =?utf-8?B?dm15anpMZ3d2NmpmQk1ndmpoQU1MalAvcmczRzNsaElJT0dScG1IR3dMMG5E?= =?utf-8?B?RTUyTDB2SjlVVitNOUg5eFNIL0s2WVFrQjJsZFExMm8xY3B4R0R1ZlFaZ3Bz?= =?utf-8?B?Z0drNldkQ2tCSEtZV3l3NGxTSkc1d0ZVNFdrL2kvZXppY0YyUndWSWc4NFBF?= =?utf-8?B?c3JtN0ZQbURNb2ZXRTFGbzk4THFFS3VFSmJadGxwYTFMNTM0Yy8wcjVETUR0?= =?utf-8?B?ZmdtZlhHakhKS2hLS2haZXI5QVRWcUxhVllEWWV5SjBPUEM2aUlEcWlHUlYw?= =?utf-8?B?RGxaOXcrYVdhNzhpajhZRmpVbW9id1EzM1BnYnV0dG1WNlpsTzNJNWxvQ3Zw?= =?utf-8?B?SFBpcUl5cFhhclBzV3J6K0ZuenNoeE1NbXlDUGpubUYyQ0V1R2NVZ0RZblAx?= =?utf-8?B?NW1NUW8zcU9ES3ByY3lLeWZvY3NmNEFPTGw0bzYwOE1NbDRqaUY5Z3FzVTJL?= =?utf-8?B?VUVXdkczd2Yyakt5QnpGT2M2VDljZmJjYzZXQkFuNzFLWXNvTFpuMGl2aHcv?= =?utf-8?B?eUU4bVFYcGUyamV6NHBNanhwa1d1eFkzWXkxQlJnUTNQVGxMSVUvOGU3d1U4?= =?utf-8?B?eXdSbWZXdWViTzU1UFRndVBCQ1oreUF6ZTF5SC9CWFNrWVh4RlR6N0syeUdH?= =?utf-8?B?SnBKWHZYNlMvZkVVeThxbGd3bFlVbUJqWkVkT2xpb0ZodzFWM201SUxsWmJV?= =?utf-8?B?RG5EcHRlY0dGOE9pZ1VUWDlTMTRVbDB0YmdURkgzVlZURUNCMUg2QUJzNXJJ?= =?utf-8?B?ODlyekI1NXZ5MGtTM2N5RG5xS1g2T3NHTE9LcVN3K3VQZ0RlMVV4cjcwcEoz?= =?utf-8?B?eG8zTEVCSG96TGxsMzlxejJrNDgyTkFRL1dpdkd2WDF1dHV6ZFgrcVZRTmJv?= =?utf-8?B?bnZleW93U0VpTHd5RzBmcjF0ZVZOQjNBc1F6SnZoeVN5YkllcStqL2docDg2?= =?utf-8?B?alN1bHlKKzNpcVpYcXhqVHNselBMQXNzdmsvS2V6c05YVFJqeHlzRk5lZllZ?= =?utf-8?B?TUo0UFIyOHFKVW5pTkJyK0MzYmE4QWliRCtnZmo3cGdhUHdYLzhFOGNaVHk1?= =?utf-8?B?Z2xCM3ErZHhycE1SbktjcmdqY09ZZHkxeWRqaDlGN1ZxOWhHUnNuaGV2eFp4?= =?utf-8?B?MHNsOEwzK2VrWlJDdFhaUXg0YW5yUGp6UWlveHNTUkg2K1JnRjNLU0RVMXpK?= =?utf-8?Q?f2nbj5MzRpwCLzg3zE5ZugJwgZGyECTV55pKAJ9?= X-MS-Exchange-CrossTenant-Network-Message-Id: b5f8a6d3-810f-437b-ff3e-08d97da919d0 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB5055.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2021 09:12:28.6843 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3KLsrongbo1J7gcUJelJ2J4qQYxR9+EcHK6Kk7appaU7z1hSfSWKgx7p7ds2iRxyHv5CzOVpq27avFNGT+qE9g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3589 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v5 04/16] dma/idxd: create dmadev instances on bus probe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 22/09/2021 03:04, fengchengwen wrote: > On 2021/9/17 23:24, Kevin Laatz wrote: >> When a suitable device is found during the bus scan/probe, create a dmadev >> instance for each HW queue. Internal structures required for device >> creation are also added. >> >> Signed-off-by: Bruce Richardson >> Signed-off-by: Kevin Laatz >> Reviewed-by: Conor Walsh >> >> --- >> v4: >> - fix 'vdev' naming, changed to 'bus' >> - rebase changes >> --- >> drivers/dma/idxd/idxd_bus.c | 19 ++++++++ >> drivers/dma/idxd/idxd_common.c | 76 ++++++++++++++++++++++++++++++++ >> drivers/dma/idxd/idxd_internal.h | 40 +++++++++++++++++ >> drivers/dma/idxd/meson.build | 1 + >> 4 files changed, 136 insertions(+) >> create mode 100644 drivers/dma/idxd/idxd_common.c >> >> diff --git a/drivers/dma/idxd/idxd_bus.c b/drivers/dma/idxd/idxd_bus.c >> index ef589af30e..b48fa954ed 100644 >> --- a/drivers/dma/idxd/idxd_bus.c >> +++ b/drivers/dma/idxd/idxd_bus.c >> @@ -85,6 +85,18 @@ dsa_get_sysfs_path(void) >> return path ? path : DSA_SYSFS_PATH; >> } >> >> +static int >> +idxd_dev_close(struct rte_dma_dev *dev) >> +{ >> + struct idxd_dmadev *idxd = dev->data->dev_private; >> + munmap(idxd->portal, 0x1000); >> + return 0; >> +} >> + >> +static const struct rte_dma_dev_ops idxd_bus_ops = { >> + .dev_close = idxd_dev_close, >> +}; >> + >> static void * >> idxd_bus_mmap_wq(struct rte_dsa_device *dev) >> { >> @@ -206,6 +218,7 @@ idxd_probe_dsa(struct rte_dsa_device *dev) >> return -1; >> idxd.max_batch_size = ret; >> idxd.qid = dev->addr.wq_id; >> + idxd.u.bus.dsa_id = dev->addr.device_id; >> idxd.sva_support = 1; >> >> idxd.portal = idxd_bus_mmap_wq(dev); >> @@ -214,6 +227,12 @@ idxd_probe_dsa(struct rte_dsa_device *dev) >> return -ENOENT; >> } >> >> + ret = idxd_dmadev_create(dev->wq_name, &dev->device, &idxd, &idxd_bus_ops); >> + if (ret) { >> + IDXD_PMD_ERR("Failed to create rawdev %s", dev->wq_name); >> + return ret; >> + } >> + >> return 0; >> } >> >> diff --git a/drivers/dma/idxd/idxd_common.c b/drivers/dma/idxd/idxd_common.c >> new file mode 100644 >> index 0000000000..8afad637fc >> --- /dev/null >> +++ b/drivers/dma/idxd/idxd_common.c >> @@ -0,0 +1,76 @@ >> +/* SPDX-License-Identifier: BSD-3-Clause >> + * Copyright 2021 Intel Corporation >> + */ >> + >> +#include >> +#include >> +#include >> + >> +#include "idxd_internal.h" >> + >> +#define IDXD_PMD_NAME_STR "dmadev_idxd" >> + >> +int >> +idxd_dmadev_create(const char *name, struct rte_device *dev, >> + const struct idxd_dmadev *base_idxd, >> + const struct rte_dma_dev_ops *ops) >> +{ >> + struct idxd_dmadev *idxd; >> + struct rte_dma_dev *dmadev = NULL; >> + int ret = 0; >> + >> + if (!name) { >> + IDXD_PMD_ERR("Invalid name of the device!"); >> + ret = -EINVAL; >> + goto cleanup; >> + } >> + >> + /* Allocate device structure */ >> + dmadev = rte_dma_pmd_allocate(name, dev->numa_node, >> + sizeof(dmadev->dev_private)); >> + if (dmadev == NULL) { >> + IDXD_PMD_ERR("Unable to allocate raw device"); >> + ret = -ENOMEM; >> + goto cleanup; >> + } >> + dmadev->dev_ops = ops; >> + dmadev->device = dev; >> + >> + idxd = rte_malloc_socket(NULL, sizeof(struct idxd_dmadev), 0, dev->numa_node); >> + if (idxd == NULL) { >> + IDXD_PMD_ERR("Unable to allocate memory for device"); >> + ret = -ENOMEM; >> + goto cleanup; >> + } >> + dmadev->data->dev_private = idxd; >> + dmadev->dev_private = idxd; > The dmadev->dev_private and dmadev->data->dev_private already inited by rte_dma_pmd_allocate, > and the driver only needs to pass in the correct parameters. > > Recommended: > dmadev = rte_dma_pmd_allocate(name, dev->name, sizeof(struct idxd_dmadev)); > > >> + *idxd = *base_idxd; /* copy over the main fields already passed in */ >> + idxd->dmadev = dmadev; >> + >> + /* allocate batch index ring and completion ring. >> + * The +1 is because we can never fully use >> + * the ring, otherwise read == write means both full and empty. >> + */ >> + idxd->batch_comp_ring = rte_zmalloc(NULL, (sizeof(idxd->batch_idx_ring[0]) + >> + sizeof(idxd->batch_comp_ring[0])) * (idxd->max_batches + 1), >> + sizeof(idxd->batch_comp_ring[0])); >> + if (idxd->batch_comp_ring == NULL) { >> + IDXD_PMD_ERR("Unable to reserve memory for batch data\n"); >> + ret = -ENOMEM; >> + goto cleanup; >> + } >> + idxd->batch_idx_ring = (void *)&idxd->batch_comp_ring[idxd->max_batches+1]; >> + idxd->batch_iova = rte_mem_virt2iova(idxd->batch_comp_ring); >> + > Once init one dmadev successful, driver need changes it's state to READY, like: > dmadev->state = RTE_DMA_DEV_READY; > This was useful when call rte_dma_pmd_release: if the state is ready, lib will call > rte_dma_close() to release the dmadev, else it only clean the struct which lib holds. Will make these changes in v6, thanks!