From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 09A42A04DD;
Thu, 22 Oct 2020 12:04:02 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id C6FA8697A;
Thu, 22 Oct 2020 12:04:01 +0200 (CEST)
Received: from mga09.intel.com (mga09.intel.com [134.134.136.24])
by dpdk.org (Postfix) with ESMTP id 0BFF86947
for <dev@dpdk.org>; Thu, 22 Oct 2020 12:03:59 +0200 (CEST)
IronPort-SDR: LaLHwjYnDLsoEORG7nHhdcJeiCnnN68QC9drajzKDJQ8SjlNDt1tQqRwYmyJEI0foE23BjpA6W
hTOWsrIrwUUA==
X-IronPort-AV: E=McAfee;i="6000,8403,9781"; a="167622341"
X-IronPort-AV: E=Sophos;i="5.77,404,1596524400"; d="scan'208";a="167622341"
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga007.fm.intel.com ([10.253.24.52])
by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
22 Oct 2020 03:03:56 -0700
IronPort-SDR: +Dk8Ih/IiQ3kQj6Xd1udGLVqwVsA76p6+4UvVDvIRWUrSq06m4s1zobm5wNieT2v+vzK//vs3R
5jc83C5Nab1A==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.77,404,1596524400"; d="scan'208";a="302379117"
Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82])
by fmsmga007.fm.intel.com with ESMTP; 22 Oct 2020 03:03:56 -0700
Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by
fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.1713.5; Thu, 22 Oct 2020 03:03:55 -0700
Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by
fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5
via Frontend Transport; Thu, 22 Oct 2020 03:03:55 -0700
Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.103)
by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.1.1713.5; Thu, 22 Oct 2020 03:03:54 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=J+XPEJDG/CIPgNqJ17IQef1M5HOHrqKGVBQkcoT2pzjgCXdCQLHwNjtIqtS3gcCnuOPCRmQqCfG6UMfFrZRf7DLwFISw6U8vPlOvUyzQv790U0hrDMbrJrFyVN68/Dy5MQcExr/RnlKuhwYtgfjj3eR/4W8Voe3C9zFogV4rZ69+5yo41v67QMFuiZ3g70LEKs6GKZR/DRMSemPzeOgZwZzBOPbDsH2BOPKxJAB4E2gvlperCS1fLO48LBhmdSJG9YWodqS8J/8EnTVMYdzVq23HbpNv/XDqpSNleVPIGWLsJ7UZjqrao4yJg4XCT8u8P/j6UvS65KF564gV8n/HTQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=O9dCWLK82CQK70pm+98CN49Rt9jKDldRFrechJ4puME=;
b=QbvWVei9BG83+vGF9647tgKapYnv47aeUtGsdPrJuJp600oj+wYUui0rjV8sBbVW2Fut6FfJciVV3K59rILZroR5Y+IgTomvFqNq4e60OqUifD/NagMcwR2ime2gVa5HOIKt5ivOC9wI3OfGvDDjtYfYUPqZVK0V/sR0h+O7yHDRMQZZKw5ZL2VOPaqNStsC9LezvCcHUbWpZpes0pkL2CtbIi1LKNLA6Rm9R8LAo8SYZdebidA2zJF5P3vl9o0HWs8TZR6Ct8ub6vJsqrK4fXx8RUMnrdyijpEAz++66HN7ZKafT8xpZ7lyzPHzBZvXj4qbH1NHa2hHrTtK2HZqlw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
s=selector2-intel-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=O9dCWLK82CQK70pm+98CN49Rt9jKDldRFrechJ4puME=;
b=EdAUA6qMg0gj2CT4kKHA+kjcLuXv9zJCzBzqXvLd8E2ADKRq4+9FNB18Wn10IK9NEZg+360aJKQ18WujbFAXNgLGO2+KJ7H7FWc/AlGxz6y6DrGKq4hXEOmd/e0Xpe4DAzLCLOxqt7dtqQxN4z83Q3gs2GQQItquovH9HB6Ix6A=
Received: from MN2PR11MB3821.namprd11.prod.outlook.com (2603:10b6:208:f7::24)
by MN2PR11MB3709.namprd11.prod.outlook.com (2603:10b6:208:f3::22)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3499.18; Thu, 22 Oct
2020 10:03:52 +0000
Received: from MN2PR11MB3821.namprd11.prod.outlook.com
([fe80::d0d2:ceff:f645:f67e]) by MN2PR11MB3821.namprd11.prod.outlook.com
([fe80::d0d2:ceff:f645:f67e%7]) with mapi id 15.20.3499.018; Thu, 22 Oct 2020
10:03:52 +0000
From: "Power, Ciara" <ciara.power@intel.com>
To: Thomas Monjalon <thomas@monjalon.net>, "Richardson, Bruce"
<bruce.richardson@intel.com>
CC: "dev@dpdk.org" <dev@dpdk.org>, Igor Russkikh <irusskikh@marvell.com>,
"Yigit, Ferruh" <ferruh.yigit@intel.com>, Harman Kalra <hkalra@marvell.com>,
Ajit Khaparde <ajit.khaparde@broadcom.com>, Hyong Youb Kim
<hyonkim@cisco.com>, Shepard Siegel <shepard.siegel@atomicrules.com>, "Ed
Czeck" <ed.czeck@atomicrules.com>, John Miller <john.miller@atomicrules.com>,
Igor Russkikh <igor.russkikh@aquantia.com>, Pavel Belous
<pavel.belous@aquantia.com>, Somalapuram Amaranath <asomalap@amd.com>, Rasesh
Mody <rmody@marvell.com>, Shahed Shaikh <shshaikh@marvell.com>, Somnath Kotur
<somnath.kotur@broadcom.com>, Rahul Lakkireddy
<rahul.lakkireddy@chelsio.com>, Hemant Agrawal <hemant.agrawal@nxp.com>,
Sachin Saxena <sachin.saxena@oss.nxp.com>, Marcin Wojtas <mw@semihalf.com>,
Michal Krawczyk <mk@semihalf.com>, Guy Tzalik <gtzalik@amazon.com>, "Evgeny
Schemeilin" <evgenys@amazon.com>, Igor Chauskin <igorch@amazon.com>,
Gagandeep Singh <g.singh@nxp.com>, John Daley <johndale@cisco.com>, "Gaetan
Rivet" <grive@u256.net>, "Zhang, Qi Z" <qi.z.zhang@intel.com>, "Wang, Xiao W"
<xiao.w.wang@intel.com>, Ziyang Xuan <xuanziyang2@huawei.com>, Xiaoyun Wang
<cloud.wangxiaoyun@huawei.com>, Guoyang Zhou <zhouguoyang@huawei.com>, "Wei
Hu (Xavier)" <xavier.huwei@huawei.com>, "Min Hu (Connor)"
<humin29@huawei.com>, Yisen Zhuang <yisen.zhuang@huawei.com>, "Xing, Beilei"
<beilei.xing@intel.com>, "Guo, Jia" <jia.guo@intel.com>, "Yang, Qiming"
<qiming.yang@intel.com>, "Wang, Haiyue" <haiyue.wang@intel.com>, "Alfredo
Cardigliano" <cardigliano@ntop.org>, "Xu, Rosen" <rosen.xu@intel.com>,
Shijith Thotton <sthotton@marvell.com>, Srisivasubramanian Srinivasan
<srinivasan@marvell.com>, Jakub Grajciar <jgrajcia@cisco.com>, Matan Azrad
<matan@nvidia.com>, Shahaf Shuler <shahafs@nvidia.com>, Viacheslav Ovsiienko
<viacheslavo@nvidia.com>, Zyta Szpak <zr@semihalf.com>, Liron Himi
<lironh@marvell.com>, Martin Spinler <spinler@cesnet.cz>, Heinrich Kuhn
<heinrich.kuhn@netronome.com>, Jerin Jacob <jerinj@marvell.com>, "Nithin
Dabilpuram" <ndabilpuram@marvell.com>, Kiran Kumar K
<kirankumark@marvell.com>, "Singh, Jasvinder" <jasvinder.singh@intel.com>,
"Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>, Maciej Czekaj
<mczekaj@marvell.com>, Maxime Coquelin <maxime.coquelin@redhat.com>, "Xia,
Chenbo" <chenbo.xia@intel.com>, "Wang, Zhihong" <zhihong.wang@intel.com>
Thread-Topic: [dpdk-dev] [PATCH v7 03/14] doc: remove references to make from
NICs guides
Thread-Index: AQHWp4KxYsO+Gb0V20uEtHKrkCqd9amjTIYAgAAPWBA=
Date: Thu, 22 Oct 2020 10:03:52 +0000
Message-ID: <MN2PR11MB38215F5CE293DB747E438F02E61D0@MN2PR11MB3821.namprd11.prod.outlook.com>
References: <20200807123009.21266-1-ciara.power@intel.com>
<20201021081725.68541-1-ciara.power@intel.com>
<20201021081725.68541-4-ciara.power@intel.com> <3176147.upzOanXTzD@thomas>
In-Reply-To: <3176147.upzOanXTzD@thomas>
Accept-Language: en-IE, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
dlp-version: 11.5.1.3
dlp-product: dlpe-windows
dlp-reaction: no-action
authentication-results: monjalon.net; dkim=none (message not signed)
header.d=none;monjalon.net; dmarc=none action=none header.from=intel.com;
x-originating-ip: [37.228.239.233]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 91523c80-00fd-4a71-abef-08d87671c794
x-ms-traffictypediagnostic: MN2PR11MB3709:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <MN2PR11MB3709DEFA7F0B1A89A804B2DFE61D0@MN2PR11MB3709.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: OPhA2FgzB8S/EZ09l8hXAQBFR00kOvWCXP6TRjptVUQfL5xeVS6rNMEUvGGBnkMu8fpF55aJ3b9V/ifW4IulEo0wtnM6qUmkuFhGDeOBLT/9xpLJ4nnec/KCo3IzlUOLwUdOUwjt7M1cDizHA8IgQA2bN9spTHM8KXkxiE9buFZI42JujyC/pnfxR3/zt346GU4QTJZEzuKTucN0pDfSXz6NYMzpkSJG2xsR/EGLqDxqznRgqPliAhAMdWl5yFNH3otYo74AOKEUZMVam+u8uAszAZWxamnA2SDhzEQDUGTnLsGZZKt3zNHZgdTv+PId9W/UnQ6t4npV/dfVEoRX+CpUDr4lYZJCj68Me7U3CdN5vML5wPpAmAtuZ1peCe5l8KNwy+mpjkoBhFXIPhXsYw==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:MN2PR11MB3821.namprd11.prod.outlook.com; PTR:; CAT:NONE;
SFS:(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(8676002)(55016002)(316002)(110136005)(26005)(186003)(6506007)(7406005)(4326008)(54906003)(478600001)(7696005)(7416002)(8936002)(86362001)(33656002)(2906002)(107886003)(52536014)(76116006)(6636002)(71200400001)(9686003)(66556008)(64756008)(5660300002)(66946007)(66446008)(83380400001)(66476007);
DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: a4yQyU9ppAFoRZLdy9J97XdblHOHYNaV5M25tR5l/QaNtVDmyokjcf9GzHgZCyWQby2vdFZc05V4CNsVGkni+HUveb+sVoj/YVD+bA7cxipF31gLSDjOJpXeOHqLcBBSMyJfnjHzIrHHJNp2+4H3vJfGsK3DsnQPlCOTRLyMyFkvuFxMGRc/c/VDzPgUO4YUNdM/l2o++84vxvi221Gekxxsj9TgUA5eT//V6D5WsH4ulVZgtEi8Zr4SuDzliKuESHRUPHrYQ58pZGAVDFZIDeI7cvOc6wqha1VHfkSERsfOIo99LCr5o8vKO+mi/wirvwWIVQGMgtTnoT2ipr5J83sBXbcgIGUU7RepWntWE139EMjxfC/52WDeCUvJZJP6Ebj1Rv0uxuv1Gl/wUzenGHRag/k6FULDh2n1Qh52l4Uvl0KLNSqfRXXjcahWyob0zdcQ/XvClSF8wvZPWR00nCJ193GGkdUL9KoydOxAuy38pF1wv4nIC5h3WnlnAl8t/1XRQIBXjsJbdKDJoFtwYRdcXJe3ZfDAKG3VkgCbl5Jm/E4SWDrrTXe6aGosA6SHnMzLfFwc0ZzJtn/ShP8scH3VNWFfopSAFSGs7EPmDD39m9Frc2vSc26F2qf+ovwox5Nh0TshOREavS/rpnTbIQ==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MN2PR11MB3821.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 91523c80-00fd-4a71-abef-08d87671c794
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Oct 2020 10:03:52.2028 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: +NKx2eNUOYau5FYiB84cAzlQbDHxN+ymk/lGc3jBd9u7siVjphFoM4eOlu7feJzxFAVUrgNVzoaD8OsIrPJvUQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3709
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [PATCH v7 03/14] doc: remove references to make from
NICs guides
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
Hi Thomas,
Thanks for reviewing, these changes seem fine overall, comments inline.
Thanks,
Ciara
>-----Original Message-----
>From: Thomas Monjalon <thomas@monjalon.net>
>Sent: Thursday 22 October 2020 09:32
<snip>
>Subject: Re: [dpdk-dev] [PATCH v7 03/14] doc: remove references to make
>from NICs guides
>
>Hi,
>
>I would like to apply this series.
>I see some small things that I could fix.
>Please see the comments below to confirm.
>
>21/10/2020 10:17, Ciara Power:
>> -.. _bnx2x_driver-compilation:
>> + .. _bnx2x_driver-compilation:
>
>was it changed by mistake?
>
Yes sorry, that change was unintentional.
>[...]
>> #. Load ``igb_uio`` or ``vfio-pci`` driver:
>>
>> + Before compiling, make sure to enable kmods for the meson build::
>> +
>> + meson configure -Denable_kmods=3Dtrue
>> +
>
>igb_uio is moved so I think this addition is not relevant anymore.
>
Yes agreed, can remove.
>[...]
>> -- ``CONFIG_RTE_LIBRTE_DPAA_DEBUG_DRIVER`` (default ``n``)
>> -
>> - Toggles display of bus configurations and enables a debugging queue
>> - to fetch error (Rx/Tx) packets to driver. By default, packets with
>> errors
>> - (like wrong checksum) are dropped by the hardware.
>> -
>> -- ``CONFIG_RTE_LIBRTE_DPAA_HWDEBUG`` (default ``n``)
>> -
>> - Enables debugging of the Queue and Buffer Manager layer which
>> interacts
>> - with the DPAA hardware.
>
>I feel these explanations should be kept.
>Or do you think it's not worth?
>
Yes sure, keep them.
>[...]
>> -- ``CONFIG_RTE_LIBRTE_DPAA2_DEBUG_DRIVER`` (default ``n``)
>> -
>> - Toggle display of debugging messages/logic
>> -
>> -- ``CONFIG_RTE_LIBRTE_DPAA2_USE_PHYS_IOVA`` (default ``n``)
>> -
>> - Toggle to use physical address vs virtual address for hardware
>accelerators.
>
>Keep these ones?
>
Yes can keep, although one thing to note is I think RTE_LIBRTE_DPAA2_USE_PH=
YS_IOVA is now enabled by default, going by config/meson.build:
dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
>[...]
>> - * **CONFIG_RTE_LIBRTE_ENA_DEBUG_RX** (default n): Enables or
>disables debug
>> - logging of RX logic within the ENA PMD driver.
>> -
>> - * **CONFIG_RTE_LIBRTE_ENA_DEBUG_TX** (default n): Enables or
>disables debug
>> - logging of TX logic within the ENA PMD driver.
>> -
>> - * **CONFIG_RTE_LIBRTE_ENA_COM_DEBUG** (default n): Enables or
>disables debug
>> - logging of low level tx/rx logic in ena_com(base) within the ENA P=
MD
>driver.
>
>Keep? Or debug options not worth?
>
Yes sure, I have no problems keeping them.
>[...]
>> -- ``CONFIG_RTE_IBVERBS_LINK_DLOPEN`` (default **n**)
>> -
>> - Build PMD with additional code to make it loadable without hard
>> - dependencies on **libibverbs** nor **libmlx5**, which may not be
>> installed
>> - on the target system.
>> -
>> - In this mode, their presence is still required for it to run
>> properly,
>> - however their absence won't prevent a DPDK application from
>> starting (with
>> - ``CONFIG_RTE_BUILD_SHARED_LIB`` disabled) and they won't show up
>as
>> - missing with ``ldd(1)``.
>> -
>> - It works by moving these dependencies to a purpose-built rdma-core
>"glue"
>> - plug-in which must either be installed in a directory whose name is
>> based
>> - on ``CONFIG_RTE_EAL_PMD_PATH`` suffixed with ``-glue`` if set, or
>> in a
>> - standard location for the dynamic linker (e.g. ``/lib``) if left to
>> the
>> - default empty string (``""``).
>> -
>> - This option has no performance impact.
>> -
>> -- ``CONFIG_RTE_IBVERBS_LINK_STATIC`` (default **n**)
>> -
>> - Embed static flavor of the dependencies **libibverbs** and
>> **libmlx5**
>> - in the PMD shared library or the executable static binary.
>> -
>> -- ``CONFIG_RTE_LIBRTE_MLX5_DEBUG`` (default **n**)
>> -
>> - Toggle debugging code and stricter compilation flags. Enabling this
>> option
>> - adds additional run-time checks and debugging messages at the cost
>> of
>> - lower performance.
>> -
>> -.. note::
>> -
>> - For BlueField, target should be set to ``arm64-bluefield-linux-gcc``=
. This
>> - will enable ``CONFIG_RTE_LIBRTE_MLX5_PMD`` and set
>``RTE_CACHE_LINE_SIZE`` to
>> - 64. Default armv8a configuration of make build and meson build set i=
t to
>128
>> - then brings performance degradation.
>> -
>> -This option is available in meson:
>> +The ibverbs libraries can be linked with this PMD in a number of
>> +ways, configured by the "ibverbs_link" build option. This can take on
>> +the following values:
>>
>> - ``ibverbs_link`` can be ``static``, ``shared``, or ``dlopen``.
>
>I would reword to include explanations above.
>
I agree, I made that change for the vdpadevs/mlx5 doc (https://patchwork.dp=
dk.org/patch/81678/) , I missed it in these NIC guides.
>[...]
>> -- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX`` (default **n**)
>> -
>> - Toggle display of transmit fast path run-time messages.
>> -
>> -- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX`` (default **n**)
>> -
>> - Toggle display of receive fast path run-time messages.
>> -
>> -- ``CONFIG_RTE_LIBRTE_QEDE_FW`` (default **""**)
>> +- ``RTE_LIBRTE_QEDE_FW`` (default **""**)
>>
>> Gives absolute path of firmware file.
>> ``Eg: "/lib/firmware/qed/qed_init_values-8.40.33.0.bin"``
>> @@ -130,6 +117,16 @@ enabling debugging options may affect system
>performance.
>> CAUTION this option is more for custom firmware, it is not
>> recommended for use under normal condition.
>>
>> +The following options can be enabled with Meson flags.
>> +
>> +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX`` (default **disabled**)
>> +
>> + Toggle display of transmit fast path run-time messages.
>> +
>> +- ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX`` (default **disabled**)
>> +
>> + Toggle display of receive fast path run-time messages.
>
>CONFIG_ should be removed.
>
Agreed, missed that when adding them back in, thanks.