From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8F9341CF1; Tue, 21 Feb 2023 00:21:24 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4325243115; Tue, 21 Feb 2023 00:21:24 +0100 (CET) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2065.outbound.protection.outlook.com [40.107.212.65]) by mails.dpdk.org (Postfix) with ESMTP id 1D0A343114 for ; Tue, 21 Feb 2023 00:21:23 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L/OpxdryzI6yBs94HcIldMw+Xs/lBTPlA6swF9JwGX2TQtA3AuC0WijTrnKA9KwAVJzCsbEEO3+PwLDqnKCKuFmFugw/qe7rBxc4A/KHcGYyhjps43Iv8gLKmiQctB2lkQmhnXmetwTqlMVHbMBBALkT+WnCCiSUsAQm4Ea23mMwK34DpjFOC1SOs49J6L1rrg0JyKadCJ4mOkcbh+VzClwsTeDUnaGUkMjAMsaIcpggihGt5+Rs+CskZG1NzuaLU3k1XR30o3fq56i25DrCswnZOGwb3mg35D4boec0qIvXjGcems6Edw8GfqBJn+K0Gva1nJ/xCy0iTJFlw9Mh3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pQ9Aca9oOXxZaOc0C2AhBhXPo8QzB0ERRvLFeMFx35Q=; b=JLoPPnJrxQLNdskctPTK4KEQVpEbTySDLOIxJiaisHicYh0KdVVPVkmJx/NeuNHrjikgeUEuMU10xXSGHzn3vufFye31mbTdce9xROGqGcWnoaQy7vF0H+rWMuNRZ+nSOfe2cmd9Wby8AhoBJJilNOl95ymEdB+6XSCCwY9YVK/dPtA/Woya/YdmCeObj52lZ3MPoZBcfST0zuJ976grt/DNUfPr8EoNJtiHkjRinO/yzuzQqMhHDSxDnlO9I68oHwxOms4Ijo9QmvRLASN1QqsiOV5OYPx8/ziVm5l3CuYWWXkqDIwT542bjhPAosRNLKRVkdsJeVmKL0ZvwcjvzA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pQ9Aca9oOXxZaOc0C2AhBhXPo8QzB0ERRvLFeMFx35Q=; b=KDB5fq71uZl88zKCzBPz18Gw1hwK8kP5B86yE3QcHbYxCtYr69USdeZc5lc8oKXoNf4UAaSELkclwrjYwXEWYT6e0mYmxxEhWPzO9x+f1MpxW9vz4uNGbq3+860z96MYsP7Ja/C7gsFmP1rmNLGwmfSg9aO8ScXJGcBzLbrP7QU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) by SJ1PR12MB6121.namprd12.prod.outlook.com (2603:10b6:a03:45c::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6111.20; Mon, 20 Feb 2023 23:21:19 +0000 Received: from CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::3614:22ed:ed5:5b48]) by CH2PR12MB4294.namprd12.prod.outlook.com ([fe80::3614:22ed:ed5:5b48%8]) with mapi id 15.20.6111.020; Mon, 20 Feb 2023 23:21:19 +0000 Message-ID: Date: Mon, 20 Feb 2023 23:21:12 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Content-Language: en-US To: Ed Czeck Cc: dev@dpdk.org, stephen@networkplumber.org, Shepard Siegel , John Miller References: <20230217160039.2487085-1-ed.czeck@atomicrules.com> <20230217215923.2561685-1-ed.czeck@atomicrules.com> <20230217215923.2561685-2-ed.czeck@atomicrules.com> <390a5b75-175d-3c03-a690-b062ff5de43b@amd.com> From: Ferruh Yigit Subject: Re: [PATCH v2 2/3] net/ark: support for single function with multiple port In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: LO2P265CA0448.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:e::28) To CH2PR12MB4294.namprd12.prod.outlook.com (2603:10b6:610:a9::11) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PR12MB4294:EE_|SJ1PR12MB6121:EE_ X-MS-Office365-Filtering-Correlation-Id: f7155911-8441-4607-69d4-08db13992bb4 X-LD-Processed: 3dd8961f-e488-4e60-8e11-a82d994e183d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yPFNWxGT+EWOXWJGyQ7ICXxWvXE7KVg5Y2gSjA+Fh4Hjb9idX+o5Ic9Hi3iK4JGptx57doJoVn45FZOCRVMh8FC4xRZO5zmRxP3NAzYR5tb0EpI8q1QhcoHU+QidattevCuKwTABqw46tRv25Uj9QR9ifDUWquKhoZQDN1kbjJKgWKlpl8BBxbnoOUuohB5g7gpQyOt4o2RwIY3F+nh+oP/ebOR+BUKb/Ktafb2LQpq9GJeVDLI2Gk/eDC3SEwEhAiG9ziOD8sZLZavlohfZvvCQBULdYOmQ11WNbJL5hrKQbd/KPRYR9qfCG4uoEz+f0Q2lrPc4VdGicSkd0UOVH4+FVkW6qiv0XOjUKevTyrNvl3ePrBa8N5RyDnsp9YGJZafbuefREKKsOEJYL0DUOca/XAiuWiyYblLnp3P8m16WOAUT0JGqM08SAgkXvznIXt7o0Jsx1hHzvHh/oFj9jYjA1cjeiym/+Zs/jGHZY8kpWKsyyc1dlzSbALXPKoyLTnlE9ikVcaaVpPaxaupQuSEETkNFS4Dce5Iu4sfJu8x3EcqRLWqyEl7qK271mqR9mZUbZkls7Hh0QX4HaQoVWC5sFLNsxn9MXaXZoz0dfuMbZFbHjDGyDaYOw3bP7E2X8F5VmKvLko6g6VGTJX7VfEzlS2pwG0d/7o4zr6ouJt2Py5owEQD35HBpNBvTRWlQp97RMejPB8iaC6lI26r4OrTYJK4y7PalS7wdQKEEoOY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH2PR12MB4294.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230025)(4636009)(136003)(366004)(39860400002)(376002)(396003)(346002)(451199018)(26005)(478600001)(31686004)(6486002)(6512007)(186003)(2616005)(6666004)(83380400001)(8676002)(66556008)(4326008)(66946007)(6916009)(66476007)(8936002)(316002)(41300700001)(53546011)(6506007)(38100700002)(36756003)(54906003)(86362001)(31696002)(2906002)(5660300002)(44832011)(43740500002)(45980500001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?K0NxR05ZUXhyRkRvTFdjazhRcExZNDNCSEdmYkpLOVo2U0FtZjY5aGZza1Bp?= =?utf-8?B?Q1pkS2F3TlI0WVFBd0M1TXo2azBhOFQwNDdVZjlXZjg0SzBlUkliclNSa05F?= =?utf-8?B?T0t4Q29UUWx4WFRlYXdFTmliRTA0VGF3aFlLRmZRanhXL1YrU1BteGNjdnB3?= =?utf-8?B?SjlyM2hWS1FPSUJlMlVwNEsxU2R3aVVNUFBUZjdaeGhjMURSU1M1RW5LUE9w?= =?utf-8?B?aGJlTjRncDBMR1llaXpSWUhYQ0pHb0xSYmFLZ0VQOE9QTmdzQXZnU1BTUFlN?= =?utf-8?B?T0dWalljV2kvSmNTZnJ0dHZsM3dzZTVIZTN0MmpSQzRUN2lEWGt5bjRuWHEw?= =?utf-8?B?WXRSUTNtRjlwUzhiUGJ2SVRhL1pYcW94b3QwRHZYbXFidVdQSjBxeUk0VkJy?= =?utf-8?B?QmVrcFE1N09tTFZucllXRDNjT0NYTUYrbFZ5a014ZHV3OC9WWlEzM29Ec0RY?= =?utf-8?B?ekFybkVVSDFWbzRPQjFxQ3N2bmhoMGlUb2h3TVEwYUdQN0NiellZRk1zMVFU?= =?utf-8?B?MU44YzNOYk1VZUVoc1NrbzJYcFY5S3NRTVhXTjU2Uy9xbDFTQWIyd2pSb0x4?= =?utf-8?B?NmE2eUNYL3lvSjZIZHdrV3o0NzVZclRQVHR0VDhUZjhBRzRzeDhPTW9mcUZR?= =?utf-8?B?NHBaWDlCdjZxT3dETHRJSk85WTdLQi91TnlkVUlxUldpNW9JcUYxZ21Uc25p?= =?utf-8?B?L1d2UWp2RENlU1VxdjZ1clZCb0laYmdDYTNLalQ5eWJydWg0U3VFTy9MQ1U0?= =?utf-8?B?Z0tsK3R5ZDFoUDdBK3FWa3dvVmJVVTNUOW5GdGp6MFNYUTczWmordlk0a2hx?= =?utf-8?B?VTZkRzFQdjhnellHQnFCZEQ0YTZTTWhkQnVnT2hoTUtKei9uTWtaazhNWTNm?= =?utf-8?B?UURmbU9hZ1N4Y2REb3VDM0pDWm5LTlliZmFMek5HZnJhczUzUWU3WUNnVWQr?= =?utf-8?B?c3hWL2dUbFBYSEY3THlSRVdBRGtUWkZySEh6ekNEQWNUU1lmOVFiWjNhM1ZQ?= =?utf-8?B?c2ZQc2VDejVrSnlNdFJZa0lsb1hCVDRSN2NDTEI0QklSKytpMkplaUlsUkxt?= =?utf-8?B?ek5KU0ZLRThaVVZpTEhQbUdLQXQ4bHduT0FGdFdoa1Z2cENKbzZlazNiekNp?= =?utf-8?B?cTE3bWlwK25McnRLY1BqQVpOdXFBTERENmxaRW1BWGRXRW52dGYwYVRMRnhY?= =?utf-8?B?dmE5U1F2UDNLQUF2SmdGV0VzL1RubVJDWmhzbHVEQS9VWnBPcmgyd0d1UEw1?= =?utf-8?B?bWpDcDhRdzRBYU5Xam5ua3pLeEJOUkQyWVVDZnFKekprZytZSytyUFZpaE9P?= =?utf-8?B?QXFMVjRsK1ZTTzNmRlhZYmhlNXVtTTNOVENsaEthMVQ0NitxaE0yK3A2SjZz?= =?utf-8?B?d041MFNRdTJMZm1STFBIbC9ac1YvTkh5ZEZ1cUovb2tEZ2RkalBHUzNxNUVl?= =?utf-8?B?MS9FVmJIVEtZYTdXYVZiSnBZSTFwSlkwTW5mVyswVktpMHRhN1cxOEFZRDRS?= =?utf-8?B?czhwbm82amlaeSt4dUlGM1NvblhIRzJETDBXSkt6U1l0TldjQXl0NWUwWGVG?= =?utf-8?B?ckxnMU9ERm9LckJ0UUsvTWxMN2U2dUtrMXZjdlllY1ZKbEFFeWdFUDQ1TnpM?= =?utf-8?B?NjRLblM4RFNMTkVQcTZoNlZBWFdBMjlIbnhIY0pPbGgrZHhZM0gwMVhPRjIv?= =?utf-8?B?a25KR1h0dnFhbTBDOHIzZlBEcUhTT2Q1ek85bTRqUk83REUvak1selRCMEZt?= =?utf-8?B?M29vSjE3b0l0Y3ZjM3ZTQVo5YmFqV0xCcTZvcWg4ZVBnc2tCTFBoeGc3amFD?= =?utf-8?B?T0FSK2FMRENRSTZpejJ2WVFLZ2dxZU82NmoxZlBtYlJCK0hZQUNBeG9vSlhv?= =?utf-8?B?RUgzUERtaCtCYnBxYytvZXpaWmFZVWpVVFcwdU0xMXViNGJZMU1wMTB4ckdG?= =?utf-8?B?cnZHTWFpNkhaQmRIWFdIcmIrdWt6R3lRUGtVZE5HelF5dmhMZ0pTRVJrdm0w?= =?utf-8?B?OXlPVXpYc0RDL2MrOG8rdzloZTg4STBzUERBenhyQVl4ZHRGelQrY2wzQ3li?= =?utf-8?B?LzJTVGZkenl2WENDOS94OVRtUG4vOStoWFB6b2JzSXZab3FsZXpadGo0TCtp?= =?utf-8?Q?Nqs7HynjMBTePoVSGjKEUGYcJ?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f7155911-8441-4607-69d4-08db13992bb4 X-MS-Exchange-CrossTenant-AuthSource: CH2PR12MB4294.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2023 23:21:18.9724 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JqyfMHW7B3whPLZtI70wZkyn4dq2/DvqtLMM8oQTShOVzhpYasQsQh7ya1ce7DXo X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6121 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2/20/2023 10:09 PM, Ed Czeck wrote: > Hi Ferruh, > We had looked at providing this patch as an LTS backport, but there are > some ties to FPGA firmware which complicate its application.  As this is > a uncommon net/ark feature we support users on an as-needed basis. Got it, you don't want this fix to be backported, that is OK. Still, it is a fix right, instead of a new support, if so can you please reflect this in the commit log? Also below comments seems ignored, can you please response them? Specially I am curious about how come this failed now, was it a testing gap in your end or a FW change caused it, and if it is a FW change this may break previous FW support, right? > Thanks. > Ed. > > On Mon, Feb 20, 2023 at 9:54 AM Ferruh Yigit > wrote: > > On 2/17/2023 9:59 PM, Ed Czeck wrote: > > allows the creation of multiple ports from one ark device via > > the use of ark pmd extension, though the splitting of queues > > Hi Ed, > > As far as I can see "single function with multiple port" support was > already there but this commit is fixing queue index usage for it, if > correct can you please update commit log accordingly? (with fixes line > etc..) > This also helps the fix to be backported to LTS versions. > > > btw, how this feature was working until now, when queue ids "0 to > ark_api_num_queues_per_port()" used for each port? > Was it not tested at all, or is there something changed in FW causing > this issue, if so is there any FW version dependency to this change? > > > > > Add unique dev_private data for each port > > > > Please check comment below related to this one. > > > > > Signed-off-by: Ed Czeck > > > --- > >  drivers/net/ark/ark_ethdev.c    | 14 +++++++++++++- > >  drivers/net/ark/ark_ethdev_rx.c |  6 +++--- > >  drivers/net/ark/ark_ethdev_tx.c |  2 +- > >  drivers/net/ark/ark_global.h    |  4 ++++ > >  4 files changed, 21 insertions(+), 5 deletions(-) > > > > diff --git a/drivers/net/ark/ark_ethdev.c > b/drivers/net/ark/ark_ethdev.c > > index d237e80cf4..96d0c2b0f0 100644 > > --- a/drivers/net/ark/ark_ethdev.c > > +++ b/drivers/net/ark/ark_ethdev.c > > @@ -432,6 +432,7 @@ eth_ark_dev_init(struct rte_eth_dev *dev) > >                       ark->user_ext.dev_get_port_count(dev, > >                                ark->user_data[dev->data->port_id]); > >       ark->num_ports = port_count; > > +     ark->num_queues = ark_api_num_queues_per_port(ark->mpurx.v, > port_count); > >  > >       for (p = 0; p < port_count; p++) { > >               struct rte_eth_dev *eth_dev; > > @@ -457,7 +458,18 @@ eth_ark_dev_init(struct rte_eth_dev *dev) > >               } > >  > >               eth_dev->device = &pci_dev->device; > > -             eth_dev->data->dev_private = ark; > > +             /* Device requires new dev_private data */ > > +             eth_dev->data->dev_private = > > +                     rte_zmalloc_socket(name, > > +                                        sizeof(struct ark_adapter), > > +                                        RTE_CACHE_LINE_SIZE, > > +                                        rte_socket_id()); > > + > > +             memcpy(eth_dev->data->dev_private, ark, > > +                    sizeof(struct ark_adapter))> +           ark > = eth_dev->data->dev_private; > > +             ark->qbase = p * ark->num_queues; > > + > > These each are a new eth_dev, so nothing wrong for each to allocate > device private data, but if the only difference in private data is > 'ark->qbase', it is possible to use 'eth_dev->process_private' for it, > which is per eth_dev. It is up to you. > > > Btw, how there are handled in the secondary process? (previous patch) > Since this code just creates new eth_dev, shouldn't secondary process > needs some code to find and re-use them? > > >               eth_dev->dev_ops = ark->eth_dev->dev_ops; > >               eth_dev->tx_pkt_burst = ark->eth_dev->tx_pkt_burst; > >               eth_dev->rx_pkt_burst = ark->eth_dev->rx_pkt_burst; > > diff --git a/drivers/net/ark/ark_ethdev_rx.c > b/drivers/net/ark/ark_ethdev_rx.c > > index cbc0416bc2..38bc69dff4 100644 > > --- a/drivers/net/ark/ark_ethdev_rx.c > > +++ b/drivers/net/ark/ark_ethdev_rx.c > > @@ -68,7 +68,7 @@ struct ark_rx_queue { > >  static int > >  eth_ark_rx_hw_setup(struct rte_eth_dev *dev, > >                   struct ark_rx_queue *queue, > > -                 uint16_t rx_queue_id __rte_unused, uint16_t > rx_queue_idx) > > +                 uint16_t rx_queue_idx) > >  { > >       rte_iova_t queue_base; > >       rte_iova_t phys_addr_q_base; > > @@ -124,7 +124,7 @@ eth_ark_dev_rx_queue_setup(struct rte_eth_dev > *dev, > >       uint32_t i; > >       int status; > >  > > -     int qidx = queue_idx; > > +     int qidx = ark->qbase + queue_idx; > >  > >       /* We may already be setup, free memory prior to > re-allocation */ > >       if (dev->data->rx_queues[queue_idx] != NULL) { > > @@ -215,7 +215,7 @@ eth_ark_dev_rx_queue_setup(struct rte_eth_dev > *dev, > >       } > >       /* MPU Setup */ > >       if (status == 0) > > -             status = eth_ark_rx_hw_setup(dev, queue, qidx, > queue_idx); > > +             status = eth_ark_rx_hw_setup(dev, queue, queue_idx); > >  > >       if (unlikely(status != 0)) { > >               struct rte_mbuf **mbuf; > > diff --git a/drivers/net/ark/ark_ethdev_tx.c > b/drivers/net/ark/ark_ethdev_tx.c > > index 5940a592a2..4792754f19 100644 > > --- a/drivers/net/ark/ark_ethdev_tx.c > > +++ b/drivers/net/ark/ark_ethdev_tx.c > > @@ -229,7 +229,7 @@ eth_ark_tx_queue_setup(struct rte_eth_dev *dev, > >       struct ark_tx_queue *queue; > >       int status; > >  > > -     int qidx = queue_idx; > > +     int qidx = ark->qbase + queue_idx; > >  > >       if (!rte_is_power_of_2(nb_desc)) { > >               ARK_PMD_LOG(ERR, > > diff --git a/drivers/net/ark/ark_global.h > b/drivers/net/ark/ark_global.h > > index 71d0b53e03..176fbcda17 100644 > > --- a/drivers/net/ark/ark_global.h > > +++ b/drivers/net/ark/ark_global.h > > @@ -112,7 +112,11 @@ struct ark_adapter { > >       ark_pkt_chkr_t pc; > >       ark_pkt_dir_t pd; > >  > > +     /* For single function, multiple ports */ > >       int num_ports; > > +     uint16_t qbase; > > +     uint16_t num_queues; > > > it looks like 'num_queues' only used locally and not needed to be part > of device data, unless there is some more usage planned for future. > > > + > >       bool isvf; > >  > >       /* Packet generator/checker args */ >