From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0BF74A04DD; Tue, 20 Oct 2020 22:08:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 32CAA2B81; Tue, 20 Oct 2020 22:07:01 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id DF87E29D6 for ; Tue, 20 Oct 2020 22:06:57 +0200 (CEST) IronPort-SDR: xsIjj7BhK7V+abXU5XA6E6DwJFw7ASkJDXH+rh5+NCrc3JMe7RmDLA12CuE1Owy9BlDLgPXnJL pAjkfrpT1KnQ== X-IronPort-AV: E=McAfee;i="6000,8403,9780"; a="163782584" X-IronPort-AV: E=Sophos;i="5.77,398,1596524400"; d="scan'208";a="163782584" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2020 13:06:56 -0700 IronPort-SDR: FcXvENa4rVNeX4eyYJ3jRZR7i3X8gwbSAwESm8Jt5Qy5676aOvUhw/tYHBwSBIgV279r+eXgSm HvGXpERCcDaw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,398,1596524400"; d="scan'208";a="348003272" Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86]) by fmsmga004.fm.intel.com with ESMTP; 20 Oct 2020 13:06:55 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 20 Oct 2020 13:06:55 -0700 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Tue, 20 Oct 2020 13:06:54 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Tue, 20 Oct 2020 13:06:54 -0700 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.36.51) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Tue, 20 Oct 2020 13:06:52 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PnUZkZXQeUsWZH85aXBkIdVA0f0Q5I3KSQycPT8GYIRKeJIz6gotMkV32eKkSaOxs4hbyrBhAqBKkCmeBF6Xlj6gJPiRphHzeCaiZYjHCkYRKfrRjc/dsgTBRot6SqajVWjgsHKv06fdc4wed19dKKfDdVE/Fkfq34mpp3afLdAjJfo5lQaen2TWU51KNRw5uQVyB6XhhqJej+9yfqWn1lUnbLq6JP9BZJ25aOgT3JrELGtlsL1MjMx3JudxByREVX1lBczk+6yM/6E+MD8Df+AkJI2IC5nzimIagf171UEYN/XKj0BapcjMoIDcrDYWnWx5WCobsTo/F4Xcoj1fMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fIcBFtASZqA1ULewSb1HxNxQ7SCOEOLA/RhjJZt5ans=; b=LcpJUXDiVju3DWytITmUGJi4Ohc+R2NeJqIpIyBCwTv11+GUdSr+Bl5iD4xyKM7D/4YbCStfOmv+mhcPqxfwv1fEs/vnrILoTSf9LEpCGrh3JaF88GOCPlwDFLk+fhZN25hd1uwEAu+VW650CAQsEGeBRKtxdvFvy/7sGQoYVeYSvrx66y2IHVqIjTnKKMN+B1Tgt3TL4Y6ifssIrDiA9L7fqrRFKm9ALE43tXC4pAlK4AYbpSVdRZwc6X9HZMfC4Fp4pDud+GlqCJK1kdQgADNGWtEpQg8xFl75ps3FT+IYq9JKFPzhbEePGDLKqX/myh6CrZZa3Yjc1ogHeB8qdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fIcBFtASZqA1ULewSb1HxNxQ7SCOEOLA/RhjJZt5ans=; b=bRxi1MUqIqNDepqgm0N+K6AmA0dMLvoEgTd60/t1xp7WJeEqpPRhNDyxU8e/QLvJdsjEJ+6FO1sREDB31gLKg7oDcWyIZzCXf8GNzPz5QYvqh7a6gH1z9w2wUcDbBXP9nTEv4/AbUR+vEREF63R5vFbQH8Q9M4eH/Q1xVAWYh+c= Received: from SN6PR11MB2574.namprd11.prod.outlook.com (2603:10b6:805:59::14) by SA2PR11MB4937.namprd11.prod.outlook.com (2603:10b6:806:118::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.20; Tue, 20 Oct 2020 20:06:50 +0000 Received: from SN6PR11MB2574.namprd11.prod.outlook.com ([fe80::902d:8f85:ea05:a7a]) by SN6PR11MB2574.namprd11.prod.outlook.com ([fe80::902d:8f85:ea05:a7a%4]) with mapi id 15.20.3477.028; Tue, 20 Oct 2020 20:06:50 +0000 From: "Eads, Gage" To: "McDaniel, Timothy" CC: "dev@dpdk.org" , "Carrillo, Erik G" , "Van Haaren, Harry" , "jerinj@marvell.com" Thread-Topic: [PATCH v5 11/22] event/dlb: add port setup Thread-Index: AQHWpLgU8Kn8xdXlmkGjg7GCLRAsE6mg7j6g Date: Tue, 20 Oct 2020 20:06:50 +0000 Message-ID: References: <1596138614-17409-2-git-send-email-timothy.mcdaniel@intel.com> <1602961456-17392-1-git-send-email-timothy.mcdaniel@intel.com> <1602961456-17392-12-git-send-email-timothy.mcdaniel@intel.com> In-Reply-To: <1602961456-17392-12-git-send-email-timothy.mcdaniel@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [68.203.30.51] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 07581357-7b04-4c31-7bd0-08d87533ae59 x-ms-traffictypediagnostic: SA2PR11MB4937: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: rOJB00EmIvMiuTru9G5HqbBkNmG6xHs2CAX66NOEthjNBIuejb2hnl7NbxTR55h2wqTR+tcgPO96Mqqly40ZtnvBlk03xwW4bT8qvWW5UN+9HzXferDEvEKuCqeuGPC00YKF7zXj+4jAAguOzhK81uh1hT2A9a9n15fsgTthJPxDJv6P/S+Mg0gepbdLLCzkOqqX3cqwMlb1dAVJ3hFPhGlJ4Lrz2KuzfrszPolYtNEvWi94sqS0A4ZJT1/preCjChlxFrg6t/z54YGtU91l5vvfeYHiUXDy6lyzpdcMYDnc9o+avOxsjESrn7ecX4JR x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB2574.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(366004)(136003)(39860400002)(346002)(376002)(5660300002)(76116006)(66946007)(316002)(26005)(7696005)(66556008)(66476007)(83380400001)(54906003)(186003)(66446008)(55016002)(64756008)(52536014)(6506007)(86362001)(6862004)(8676002)(33656002)(4326008)(8936002)(71200400001)(9686003)(6636002)(2906002)(478600001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: khH+UF/s0+nN478JHw+ODvHrKHZmkwaJV798FWaE4sc+zFYBzi9aQz2quNR2VZYt/SsUq+QVxTLmdIofyUoPPc37a8Jh2wBPPtylIlZ7oOxxsZ1HjlSVkg6YTGRUMPsXdQWhfEC3b8IpN2vQ6Mt+/hb/xyBBaefFXXLruYgZ04OTtbePvkYze5TOIUPlmZT6J9s95zyR7r9+BOq4W6ziLNXTRtlWpkQ+qJjdlp8N3MwORYiH+LUaVSM21JU8tlC1gu8mLycn2pS3TEmFEAZuWwqhadUmIma2LRijPR2juUds53py0q7dcI3/78oHlHfKw8oNf4IQrMvfg7JjJj0nL/DVVtjG2BkU5rxjE/yLStog0Qfc62A3T1uCo47F7tjWCa7AE6TjEUlEFlU/uuHjPVfCQfeVtf8wXcJketjDz1b4GCo3j14rksqveSadULGE/S8yip7wjkn1ADYnrEBxgC3CmP62VDTWpwKqDJMjzWmc7jelbTUFmkmpo3Kt0ebLWP5ItQF/HRLx/DE9ijsjMzhkvWx05l6zwF9PKso8y9vJSW13lsDtj1lbvjJHvYQdVwTDpI/HGbKOM62Rp2XD2cMEGZqvYS5Q/8In0dN8ZSxLiWuUe/jitdE1reHPr4BpN5aCQfrrQBwJzGnpFCaTIA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB2574.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 07581357-7b04-4c31-7bd0-08d87533ae59 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Oct 2020 20:06:50.0927 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 7o5mQ6Wo45LBoeWJZnhtllBWyy5CmCE/2lzO8ZmMFSPFU9W5q68rj1uiVp0+gXXUEmY3YNQwxj1Je0bosw43UQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR11MB4937 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v5 11/22] event/dlb: add port setup X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" [...] > +static int > +dlb_pf_ldb_port_create(struct dlb_hw_dev *handle, > + struct dlb_create_ldb_port_args *cfg, > + enum dlb_cq_poll_modes poll_mode) > +{ > + struct dlb_dev *dlb_dev =3D (struct dlb_dev *)handle->pf_dev; > + struct dlb_cmd_response response =3D {0}; > + int ret; > + uint8_t *port_base; > + const struct rte_memzone *mz; > + int alloc_sz, qe_sz, cq_alloc_depth; > + rte_iova_t pp_dma_base; > + rte_iova_t pc_dma_base; > + rte_iova_t cq_dma_base; > + int is_dir =3D false; > + > + DLB_INFO(dev->dlb_device, "Entering %s()\n", __func__); > + > + if (poll_mode =3D=3D DLB_CQ_POLL_MODE_STD) > + qe_sz =3D sizeof(struct dlb_dequeue_qe); > + else > + qe_sz =3D RTE_CACHE_LINE_SIZE; > + > + /* The hardware always uses a CQ depth of at least > + * DLB_MIN_HARDWARE_CQ_DEPTH, even though from the user > + * perspective we support a depth as low as 1 for LDB ports. > + */ > + cq_alloc_depth =3D RTE_MAX(cfg->cq_depth, > DLB_MIN_HARDWARE_CQ_DEPTH); > + > + /* Calculate the port memory required, including two cache lines for > + * credit pop counts. Round up to the nearest cache line. > + */ > + alloc_sz =3D 2 * RTE_CACHE_LINE_SIZE + cq_alloc_depth * qe_sz; > + alloc_sz =3D RTE_CACHE_LINE_ROUNDUP(alloc_sz); > + > + port_base =3D dlb_alloc_coherent_aligned(&mz, &pc_dma_base, > + alloc_sz, PAGE_SIZE); > + if (port_base =3D=3D NULL) > + return -ENOMEM; > + > + /* Lock the page in memory */ > + ret =3D rte_mem_lock_page(port_base); > + if (ret < 0) { > + DLB_LOG_ERR("dlb pf pmd could not lock page for device i/o\n"); > + goto create_port_err; > + } > + > + memset(port_base, 0, alloc_sz); > + cq_dma_base =3D (uintptr_t)(pc_dma_base + (2 * > RTE_CACHE_LINE_SIZE)); > + > + ret =3D dlb_hw_create_ldb_port(&dlb_dev->hw, > + handle->domain_id, > + cfg, > + pc_dma_base, > + cq_dma_base, > + &response); > + if (ret) > + goto create_port_err; > + > + pp_dma_base =3D (uintptr_t)dlb_dev->hw.func_kva + PP_BASE(is_dir); > + dlb_port[response.id][DLB_LDB].pp_addr =3D > + (void *)(uintptr_t)(pp_dma_base + (PAGE_SIZE * response.id)); > + > + dlb_port[response.id][DLB_LDB].cq_base =3D > + (void *)(uintptr_t)(port_base + (2 * RTE_CACHE_LINE_SIZE)); > + > + dlb_port[response.id][DLB_LDB].ldb_popcount =3D > + (void *)(uintptr_t)port_base; > + dlb_port[response.id][DLB_LDB].dir_popcount =3D (void *)(uintptr_t) > + (port_base + RTE_CACHE_LINE_SIZE); > + dlb_port[response.id][DLB_LDB].mz =3D mz; > + > + *(struct dlb_cmd_response *)cfg->response =3D response; > + > + DLB_INFO(dev->dlb_device, "Exiting %s() with ret=3D%d\n", __func__, > ret); > + > +create_port_err: Need to free the memzone if the PMD jumps to this label > + > + return ret; > +} > + > +static int > +dlb_pf_dir_port_create(struct dlb_hw_dev *handle, > + struct dlb_create_dir_port_args *cfg, > + enum dlb_cq_poll_modes poll_mode) > +{ > + struct dlb_dev *dlb_dev =3D (struct dlb_dev *)handle->pf_dev; > + struct dlb_cmd_response response =3D {0}; > + int ret; > + uint8_t *port_base; > + const struct rte_memzone *mz; > + int alloc_sz, qe_sz; > + rte_iova_t pp_dma_base; > + rte_iova_t pc_dma_base; > + rte_iova_t cq_dma_base; > + int is_dir =3D true; > + > + DLB_INFO(dev->dlb_device, "Entering %s()\n", __func__); > + > + if (poll_mode =3D=3D DLB_CQ_POLL_MODE_STD) > + qe_sz =3D sizeof(struct dlb_dequeue_qe); > + else > + qe_sz =3D RTE_CACHE_LINE_SIZE; > + > + /* Calculate the port memory required, including two cache lines for > + * credit pop counts. Round up to the nearest cache line. > + */ > + alloc_sz =3D 2 * RTE_CACHE_LINE_SIZE + cfg->cq_depth * qe_sz; > + alloc_sz =3D RTE_CACHE_LINE_ROUNDUP(alloc_sz); > + > + port_base =3D dlb_alloc_coherent_aligned(&mz, &pc_dma_base, > + alloc_sz, PAGE_SIZE); > + if (port_base =3D=3D NULL) > + return -ENOMEM; > + > + /* Lock the page in memory */ > + ret =3D rte_mem_lock_page(port_base); > + if (ret < 0) { > + DLB_LOG_ERR("dlb pf pmd could not lock page for device i/o\n"); > + goto create_port_err; > + } > + > + memset(port_base, 0, alloc_sz); > + cq_dma_base =3D (uintptr_t)(pc_dma_base + (2 * > RTE_CACHE_LINE_SIZE)); > + > + ret =3D dlb_hw_create_dir_port(&dlb_dev->hw, > + handle->domain_id, > + cfg, > + pc_dma_base, > + cq_dma_base, > + &response); > + if (ret) > + goto create_port_err; > + > + pp_dma_base =3D (uintptr_t)dlb_dev->hw.func_kva + PP_BASE(is_dir); > + dlb_port[response.id][DLB_DIR].pp_addr =3D > + (void *)(uintptr_t)(pp_dma_base + (PAGE_SIZE * response.id)); > + > + dlb_port[response.id][DLB_DIR].cq_base =3D > + (void *)(uintptr_t)(port_base + (2 * RTE_CACHE_LINE_SIZE)); > + > + dlb_port[response.id][DLB_DIR].ldb_popcount =3D > + (void *)(uintptr_t)port_base; > + dlb_port[response.id][DLB_DIR].dir_popcount =3D (void *)(uintptr_t) > + (port_base + RTE_CACHE_LINE_SIZE); > + dlb_port[response.id][DLB_DIR].mz =3D mz; > + > + *(struct dlb_cmd_response *)cfg->response =3D response; > + > + DLB_INFO(dev->dlb_device, "Exiting %s() with ret=3D%d\n", __func__, > ret); > + > +create_port_err: > + Ditto Thanks, Gage