From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0047.outbound.protection.outlook.com [104.47.33.47]) by dpdk.org (Postfix) with ESMTP id 4BED658C5 for ; Thu, 29 Sep 2016 16:39:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=5zsvFdCe4Tbs34MwFrDTQWFyfMM7E4XOnzjNYpxw+kM=; b=BfjEHfmud0oAx6malRYRNbo3xTpMmh4NAM63zBG00FErqCgSyKTm3EZLc7WRQbbXIaMTGm2/nHd1PJ4V+FrJfdrEyz9VAna0SKVKVqdTIIaMGlye82dFJcFD0N5Bch0Ctk6qlL4asg0xqCAowSNDaSsilzBsspmpOFHtsAXP2t4= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Maciej.Czekaj@cavium.com; Received: from [10.0.0.58] (31.172.191.173) by DM5PR07MB2892.namprd07.prod.outlook.com (10.168.102.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.639.5; Thu, 29 Sep 2016 14:39:10 +0000 To: "Mcnamara, John" , Kamil Rytarowski , "dev@dpdk.org" References: <1472230448-17490-1-git-send-email-krytarowski@caviumnetworks.com> <1472230448-17490-14-git-send-email-krytarowski@caviumnetworks.com> CC: "zyta.szpak@semihalf.com" , "slawomir.rosek@semihalf.com" , "rad@semihalf.com" , "jerin.jacob@caviumnetworks.com" , Kamil Rytarowski From: Maciej Czekaj Message-ID: <477e359c-c84b-83ef-774a-c7b4eaa208b7@caviumnetworks.com> Date: Thu, 29 Sep 2016 16:38:59 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [31.172.191.173] X-ClientProxiedBy: HE1PR05CA0072.eurprd05.prod.outlook.com (10.164.28.40) To DM5PR07MB2892.namprd07.prod.outlook.com (10.168.102.146) X-MS-Office365-Filtering-Correlation-Id: 00aede11-a5de-4913-a10e-08d3e876612c X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB2892; 2:bnZj5ky6HgRonQBpnJPzOxZ9JKRF8qGl+Ypu+vc2t3bLC3ei+V+x16d3zMU3jsI97WZBlnWv2ZdT1ufv3LaofVxbDVUppJJnwySgJ2Ia2z+lOyiVzev4lkp0XPCo8HCFa8Zl+bs0kgCSFzMccjyA8WAIkx89rjxnRKIQ9rWuBaKfGprFKHfYZEL3gPmqcyh7; 3:lUrP6lNVbTuli+/yfcuHfH1fl3qvvoluqjpQJE1VEE0UpR27Wp5c4LtcDcnUcxSy9CdV1kqfZxf8+g3xVFNiNpzVnjncK8fPxNa2k/BZyjsUFPcKhl58EUQMbnUdztBx X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM5PR07MB2892; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB2892; 25:3NLy865RxrqESudaBPLub6iEDvk6X0FesvjjNEQ6Jrn1lB80KtUmYQlca4iHNObcSM86940vY86QDtmnXImkgHS9KJeq5HBwJDxaD8gxqrNOY//F04l/bZyDbz+yKzAafGs31vyoUBKvjTx8ld1GHx49Ark/vWQ6KoXDC5SU7gucOh5XuWMuKuh+Q0qT+QRy5ZK4NywoGYh4xXaiP9+dfWqQTGI6dP7iz3OD+MvSb39FWh8FNDNZRD15DJtKSd4QwWoAc8LFsINQz27rQ2Kwpi10Nya6Sk49FeSmqSSDvsmBKtr0+97sN5MLFHjnCYnNgX20rtPFObNd009LCP/wgDadfRkLnCuZaW83JVwQJX4EqDBpO3sePJY4NZ3k+OQsYOdAW0rgX7hzWH/+fpf/fG1smluSij5ayyES5UdESDJV693FaJgZJP6Q5CAiljRpeJh3zkOhs1phk/x81nMIEhhka63XC9/OQQWupwZJIg3miwyYF32YwBtZKFte6DwyOmP1Ju1pundC0K3GEJhk4ECh7+LH8cG5ZCx02LQ314gNFwj3L0m9e5Opx4KuvIBkpaTNuFyDoKhCgDRN2Z2rg08mcmDHqxzNv4uVXhxdHc1OTXlOYP22FcVwGb5uVe+15KRtIzyOVwvERTcHRGsWAtnOOZVKnspTdMOlXpUCr/F2FN/sPvF5IRihqiw0QtGOAH2r7mhipzJdHHrupsV1LA4xH5qVOGMAm1LvTXw1Ri/9JgoqKMZz+7e/dfVGIZoF X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB2892; 31:0i1EopxAFuXPVBB5cgEZnzKyWaksCaX8fikhAls8qhZI+d4mQNS7uesDNH3QU3US554VXYuSw0qXxY6mbKkvXszX9SCh99Sdv69qo0+jKh/0rwyzSnkOntcNEaClyV+GTep4b+14k1hAvjJzQYE4Ae/NLSMpbKd2FDaNUHdjNWlGB/szl9lgcaoBfMX+RnKuhC7jYeT9d0Z0Duk5OkT5hK5WwhPy2xwPf/4d+5sfjf0=; 20:ZCjDOEWvL6c1K0oNGDIOywQLPrKq1G2PO4YKBbFuHEaenpUNTaBth7i4EDMaqvjQML6nJ1xCW0Nqq+3tjHIFieynRFUHTUkJ5scsstZ1Kxg9zTjRT8BTNA1zkuN+P4P6TzmQJqQMSEgLwGtsm/zw2oUHrZKuQeazaj8w+gvoQ/u960OD+dE6SvWhTdV/J4niehT67ckFPCyChcc7y/FcG3RVVdZslu75qOnIRNfkVgx1o0oJ7gnaLI+r5e2ja8ROwO0w9xmb8WZHxE5csxi4ciwKSZqVjF/eVButeFBXiWTBK3KXNHjV2C6JcfkmRNaNOd6e+MZDFYf7tpO8jh+DcY8q0sTgI5KSlqGg6U4y0fZ2ShpLV6JC5sYwCpv17pOxELy1zX0a7X/MnR7bhIGqVaOZilCzDqJ6jRfC3YzZ83KO3iUA4ujwKm9MhG7ekf23+c9F8PPQpBc8+wfa6A4LnPC4onBhqcxA5iUq1os2800xt8g1suRHs67n3InnkvK34Hg9NuhT7s3scr8k4HNFq56cOb5NYnSDYFOUhDoTzPvw6TxuwYKyy9DB74FhsU5csAmLLJIY9F4xAE2SaCBLFRukaQOqYfc4vXjDAFlroNg= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:DM5PR07MB2892; BCL:0; PCL:0; RULEID:; SRVR:DM5PR07MB2892; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB2892; 4:jSCHofyOV1nuPqXRE0GdCOM+rlkoT8evMXKp4Nb7aKlded9990eYmoTd+bOjgQZHiWGDxxi9khZPbeMt3WY6zRqRe08oAtin05Gabk6z2O1yok5MlUnOkvTrUIhtO7ExL94nXu3mLJZ8WttTokfr95HC8nP/NcoBnzYvIdUDxQDmUTLJurDSxmqyTXDnZqt7XpSJPWkIbze1y4zZc8VpukSxRULZpQaaBnwO0ppcMKQ7NcSpzMKCFIQifPITvyXu2wrOYagAwkyT4+P+EQOiCrWsZcb4SiQrYIz7x7tbYCNdR+3132OMaUuHhBstAY0LgRjfSWpeXnk7c1XMENk7R8dAktTb7ZpA0By1Hf2D7WK7Abbrjm3itkLk0M3klwVk/wKt2GxwqWwP12zuGZpaYQ== X-Forefront-PRVS: 00808B16F3 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6049001)(6009001)(7916002)(189002)(13464003)(377454003)(50944005)(199003)(8676002)(81156014)(81166006)(2950100002)(189998001)(6666003)(50466002)(64126003)(4001430100002)(68736007)(42882006)(4326007)(230700001)(2906002)(5660300001)(36756003)(47776003)(42186005)(586003)(31686004)(19580405001)(66066001)(19580395003)(65956001)(7846002)(65806001)(6116002)(305945005)(7736002)(83506001)(3846002)(31696002)(92566002)(77096005)(106356001)(5890100001)(105586002)(76176999)(54356999)(50986999)(23746002)(4001350100001)(33646002)(2501003)(101416001)(5001770100001)(97736004)(107886002)(65826007); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR07MB2892; H:[10.0.0.58]; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1; DM5PR07MB2892; 23:2NvtDS+nAnJ2JZhfPqoITIkkC4Hq2E2xUVPXA?= =?Windows-1252?Q?5AtxJIMiSxPcywnfFNK9kd8A27oWA+Sdl6hHfZNUep9pRTVNVJuEizl/?= =?Windows-1252?Q?nSi5zOYHTm5rf7loSZbwjE6Mrg0VEeBdSYsrWPvPO+elhIoAcLcEd6Ks?= =?Windows-1252?Q?sX9II30AFQEg8jcI7V8JSjLPam3BBWmvyTQgFt/OFSS6GxYvI5oBjZ86?= =?Windows-1252?Q?EAm6XaJZKi7sx4t6ceje3HlMAU0Yjr/wtKlAptwveeMegt3LjXeotbvM?= =?Windows-1252?Q?3FRofyQLE3qSg7OlamqDKT2pbId/WeWlLq5VtSMmkE3aOUGSWiG6Efyt?= =?Windows-1252?Q?MPsaxS1WnwwoMfPf+cFBODWMhm2doQGPdHK6004K+sWhOIigCNgPcTNo?= =?Windows-1252?Q?jfj9MRCg4QuNR34gXfaK4//shvw3EChepM4J5Mel+5OBexHH/hE1hNP0?= =?Windows-1252?Q?nZE7ibKtSd6eIy9Ftp1V4Dj3RtDnLPtnjid/2ThcOSANoTCZkBKeh/AR?= =?Windows-1252?Q?W4XGjmsKi7n+Dbr9kphGxsNjyyqtabeARq2RLDVup18U0sTSaz8MaYpz?= =?Windows-1252?Q?YKD0AhTjUFz/QwP3vV8sM+SKo3UqtkpLN2QP+5Ml7UyETHztr4vN1WP6?= =?Windows-1252?Q?HtGg7Crcg4a85E8P6a1ESfCrl3FIAm4VZKV8+m2UchEy9tdC6eOSrkS0?= =?Windows-1252?Q?7MwtLVYOMxxTPH4H1rBp5M/vqf4qeNGDiOeioxveovBL5M4e5DBQrvqd?= =?Windows-1252?Q?8En9nTHZH6D8hB8q6qB/0/MKiszlAgFMH1DEs6kwZOUAIcZEVVt9d4Bm?= =?Windows-1252?Q?ACx1H1zfY5Fi+vUIYrkI5SjsD3hUYu/wlVbPgtWUfjOWcdn7dcWvoTkX?= =?Windows-1252?Q?7c/8b9s8u5m/h2IZmtnOw+a4k5S9+SuyCQSJ8uR+jF5SyvbHv5nJihM/?= =?Windows-1252?Q?W+EXCYM0uqpFzaO6ELiexnvtZtpwDaW2sf4lVWMEpNg6BE0RSp+8xqwb?= =?Windows-1252?Q?FOBqkg4iEyTNsloqt8hcEAqaWnAT7g/xbPn5DOeXqiKKBiXbRIzj21Fl?= =?Windows-1252?Q?DKuYhrJdGOdv2zM+xcdy/Qoi35pvBVK2YSwcYypfPkzK3A1cpIvv0ptJ?= =?Windows-1252?Q?nrVIu9ySPqgrczuXMWytuxpQvcllhE3j4DQjPrfDWwuZ5dBNzow4aBaj?= =?Windows-1252?Q?yKl3TXpUua8NSfLe5Sxns9J2v9t5MRVj5Pq/BHO7mw08zKLR2nDgulJD?= =?Windows-1252?Q?vLfPUZ5hqo22B3ix1U37qPSs9efIW6svIymZJeQ1WpOQYylXuqLOnqNA?= =?Windows-1252?Q?mX6vr8y0k9EukvBue5VGidlJ3eQnAw0NDNJctIM5IHmG3PC1TUIptBB4?= =?Windows-1252?Q?WxrZbRqKaUxlt601QEJPL5Roe9836VSkYn+SMA6FrnsAtif0870rNv/v?= =?Windows-1252?Q?3oo4Tf0W4W2+O8rvcrToSRuDTHEW43lh0hpGQCoCjBQA0h5A1OKFkf5s?= =?Windows-1252?Q?cBpCuqGloyq0b+bYwCooLl8GNuLnMNUkQ74aaf1EKaKX1CMOZQM226g0?= =?Windows-1252?Q?gTIT+PpTFOJvHM=3D?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB2892; 6:k9YmwmLAUfMPNTvZOve1SOPVhOC9/oWtHMg935gd1/EPTEKOXfniHEa1o7gO6ZuI6NO+TxT0FOHxU78JUiwr3eu5tdntMFPdaAK4baWEFUfPl6gFpRiNDJVfr0KE1D+w1yAusksugZ3BlPA7OHo/hS4S9Q4pGxQwm5CYF++295qzSVSIU5l7DRKaGP/SJPhGTE9rhpsJ5X7blGz6ymyjUxszTDqym4pln7MYFJw/CTM1EiRB2/LlHCL3yQvuOCoXoxuOFAlXGvuOADeISv/rsvYROwwEMVSFw+xo2cs7Mgw=; 5:66m00YQQSeW+O/R21FY9p4D7K38eczO6HAu317KeqDcLU6pB8lySNCst1K0oJIZ/HTKZi9MtDrtAjFmaou9H+3IsB/c+vpoKRgEgJTSIag8fjhPOvATDaKgdXOgcfbGhyTSD40ELMsAjeDsn1Wkvzg==; 24:8wQGVHVE3cZmHK+pbvyf8nfoKdLZkmhWqmRMLRHiSM8PkJfFgtUCDaOOuwmwm7mG6jb0AZkFAO03Or+XgZ4zvUyFEXYmBx7lJMXpKL/kpnU=; 7:1HFBOuMmAUM39U4spGlpEGd7/rTM6uRcdEDxBhbwwCwkP5M88Q3a69NkXYCZ8OXJsJQevNSht4bLC3owzEajrlccToYWbBRJ6oV5GyUjng1NPSzS1yKY1lHvSfaYRX24tRLyhCFlgKMSZnAL+9iIRoa4vdp3nnE42GjS4JCTawx/Jfg1zabKI1vvRZ6KFVtUgbav23DGUarD91BbpVWFGHapcS1Nhxjly+rLgzuvmpHzE6k74FMrT3Y06XSUJRNKw1egDdhRylmyryLSxNfky9XXnB2/sHRuopyPNURyVMN0oCEXqJstbm8yG4w/KBh2 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Sep 2016 14:39:10.4465 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR07MB2892 Subject: Re: [dpdk-dev] [PATCH 13/13] net/thunderx: document secondary queue set support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2016 14:39:13 -0000 We will address all the issues in v2. >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Kamil Rytarowski >> Sent: Friday, August 26, 2016 5:54 PM >> To: dev@dpdk.org >> Cc: maciej.czekaj@caviumnetworks.com; zyta.szpak@semihalf.com; >> slawomir.rosek@semihalf.com; rad@semihalf.com; >> jerin.jacob@caviumnetworks.com; Kamil Rytarowski >> >> Subject: [dpdk-dev] [PATCH 13/13] net/thunderx: document secondary queue >> set support >> > > There are some whitespace errors in the docs: > > Applying patch #15435 using 'git am' > Description: [dpdk-dev,13/13] net/thunderx: document secondary queue set support > Applying: net/thunderx: document secondary queue set support > .git/rebase-apply/patch:70: trailing whitespace. > > .git/rebase-apply/patch:74: trailing whitespace. > > .git/rebase-apply/patch:98: trailing whitespace. > > > > Some other minor comments below. > > > >> >> Supported ThunderX SoCs >> ----------------------- >> @@ -322,6 +323,112 @@ This section provides instructions to configure SR- >> IOV with Linux OS. >> #. Refer to section :ref:`Running testpmd ` for >> instruction >> how to launch ``testpmd`` application. >> >> +Multiple Queue Set per DPDK port configuration >> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> + >> +There are two types of VFs: >> + >> +- Primary VF >> +- Secondary VF >> + >> +Each port consist of a primary VF and n secondary VF(s). Each VF provides > > s/consist/consists/ > > >> 8 Tx/Rx queues to a port. >> +In case port is configured to use more than 8 queues, then it requires >> +one (or more) secondary VF. Each secondary VF adds additional 8 queues to >> the queue set. > > There are a few missing definite and indefinite articles missing in the text. > > >> + >> +During PMD driver initialization, the primary VF's are enumerated by >> +checking the specific flag (see sqs message in DPDK boot log - sqs >> indicates secondary queue set). >> +They are at the beginning of VF list (the remain ones are secondary >> VF's). >> + >> +The primary VFs are used as master queue sets. Secondary VFs provid > > s/provid/provide/ > >> +additional queue sets for primary ones. If a port is configured for >> +more then >> +8 queues than it will request for additional queues from secondary VFs. >> + >> +Secondary VFs cannot be shared between primary VFs. >> + >> +Primary VFs are present on the beginning of the 'Network devices using >> +kernel driver' list, secondary VFs are on the remaining on the remaining >> part of the list. >> + >> + .. note:: >> + > > This note and the following one are indented too far. They should be aligned with the margin. > > > >> ... > > >> +Example device binding >> +~~~~~~~~~~~~~~~~~~~~~~ >> + >> +If a system has three interfaces, a total of 18 VF devices will be >> +created on a non-NUMA machine. >> + >> + .. note:: >> + >> + NUMA systems have 12 VFs per port and non-NUMA 6 VFs per port. >> + >> + .. code-block:: console > > This note and code block are indented too far. They should be aligned with the margin. > > >> + >> + # tools/dpdk-devbind.py --status >> + >> + Network devices using DPDK-compatible driver >> + ============================================ >> + >> + >> + Network devices using kernel driver >> + =================================== >> + 0000:01:10.0 'Device a026' if= drv=thunder-BGX unused=vfio- >> pci,uio_pci_generic >> + 0000:01:10.1 'Device a026' if= drv=thunder-BGX unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.0 'Device a01e' if= drv=thunder-nic unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.1 'Device 0011' if=eth0 drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.2 'Device 0011' if=eth1 drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.3 'Device 0011' if=eth2 drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.4 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.5 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.6 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:00.7 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.0 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.1 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.2 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.3 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.4 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.5 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.6 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:01.7 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:02.0 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:02.1 'Device 0011' if= drv=thunder-nicvf unused=vfio- >> pci,uio_pci_generic >> + 0002:01:02.2 'Device 0011' if= drv=thunder-nicvf >> + unused=vfio-pci,uio_pci_generic >> + >> + Other network devices >> + ===================== >> + 0002:00:03.0 'Device a01f' unused=vfio-pci,uio_pci_generic >> + >> + >> +We want to bind two physical interfaces with 24 queues each device, we >> +attach two primary VFs and four secondary queues. In our example we >> choose two 10G interfaces eth1 (0002:01:00.2) and eth2 (0002:01:00.3). >> +We will chose four secondary queue sets from the ending of the list >> (0002:01:01.7-0002:01:02.2). > > s/chose/choose/ > >> + >> + >> +#. Bind two primary VFs to the ``vfio-pci`` driver: >> + >> + .. code-block:: console >> + > > These code blocks are indented correctly. > > > John. >