From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 899C4425E4; Wed, 20 Sep 2023 09:35:14 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 77E5740EE3; Wed, 20 Sep 2023 09:35:14 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id A776340EE1 for ; Wed, 20 Sep 2023 09:35:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1695195312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Yi89//i9GD+CqmebZo08FP+5YH6YzaoRHPXOInLMPCw=; b=KhonmLJ2iZA1lguaaZip8TKlkNrlNWdvIMEL3xcrfrNZ13S2sr4IqJ4beofB6zKVh+ovgd 6QVg2axk7eQzeDkNUeL+EB0LyREXynU1PPuPxqAVe0cN+Xa2u8OvYVOrJpb2O3V9fcBPIm IHWN+htRVW0ODiV8a+CbjOElcZ6qqhs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-646-9kZUAEldObys2grwbIi21Q-1; Wed, 20 Sep 2023 03:35:09 -0400 X-MC-Unique: 9kZUAEldObys2grwbIi21Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7E53A101A53B; Wed, 20 Sep 2023 07:35:08 +0000 (UTC) Received: from [10.39.208.35] (unknown [10.39.208.35]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8D36140C6EBF; Wed, 20 Sep 2023 07:35:06 +0000 (UTC) Message-ID: Date: Wed, 20 Sep 2023 09:35:05 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 To: "Roger Melton (rmelton)" , "Dave Johnson (davejo)" , "dev@dpdk.org" , "Sampath Peechu (speechu)" , chenbo.xia@outlook.com Cc: "Malcolm Bumgardner (mbumgard)" , "Chris Brezovec (cbrezove)" , David Marchand References: <63486764-3b44-3299-6830-05435dfd78f3@redhat.com> <827f912f-fc2d-6d41-ba8c-e7f3f9f2e24b@redhat.com> From: Maxime Coquelin Subject: Re: Commit broke 32-bit testpmd app In-Reply-To: X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/mixed; boundary="------------eEnOQcOba2z40BCh0cKmlsRG" Content-Language: en-US X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This is a multi-part message in MIME format. --------------eEnOQcOba2z40BCh0cKmlsRG Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, I tried to reproduce without success(see attached log). I fail to reproduce because buf_iova fits into 32 bits in my case: (gdb) p /x *tx_pkts[0] $4 = { cacheline0 = 0x77b19ec0, buf_addr = 0x77b19f40, buf_iova = 0x49519f40, rearm_data = 0x77b19ed0, However, looking at your report, something like this would work for you: diff --git a/drivers/net/virtio/virtqueue.h b/drivers/net/virtio/virtqueue.h index 9d4aba11a3..38efbc517a 100644 --- a/drivers/net/virtio/virtqueue.h +++ b/drivers/net/virtio/virtqueue.h @@ -124,7 +124,7 @@ virtqueue_store_flags_packed(struct vring_packed_desc *dp, * (virtio-pci and virtio-user). */ #define VIRTIO_MBUF_ADDR(mb, vq) \ - ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + (vq)->mbuf_addr_offset))) + (*(uint64_t *)((uintptr_t)(mb) + (vq)->mbuf_addr_offset)) The problem is that it would likely break Virtio-user en 32bits mode, as this is how it was initially implemented, and got fixed few years ago, as David hinted to me: commit 260aae9ad9621e3e758f1443abb8fcbc25ece07c Author: Jianfeng Tan Date: Wed Apr 19 02:30:33 2017 +0000 net/virtio-user: fix address on 32-bit system virtio-user cannot work on 32-bit system as higher 32-bit of the addr field (64-bit) in the desc is filled with non-zero value which should not happen for a 32-bit system. In case of virtio-user, we use buf_addr of mbuf to fill the virtqueue desc addr. This is a regression bug. For 32-bit system, the first 4 bytes of mbuf is buf_addr, with following 8 bytes for buf_phyaddr. With below wrong definition, both buf_addr and lower 4 bytes buf_phyaddr are obtained to fill the virtqueue desc. #define VIRTIO_MBUF_ADDR(mb, vq) \ (*(uint64_t *)((uintptr_t)(mb) + (vq)->offset)) Fixes: 25f80d108780 ("net/virtio: fix packet corruption") Cc: stable@dpdk.org Signed-off-by: Jianfeng Tan Acked-by: Yuanhan Liu If my understanding is correct, on 32 bits, when mbuf->buf_addr is used (Virtio-user), we need to mask out the higher 4 bytes, while when using Virtio-pci we need the full 64 bits (as the physical addresses used as IOVA on the guest are 64 bits). Regards, Maxime On 9/13/23 15:24, Roger Melton (rmelton) wrote: > +Chris Brezovec > > Hi Maxime, > > Chris from our team is attending the DPDK Summit in Dublin this week. > If you have some time available, we'd appreciate it if he could meet > with you to discuss the 32bit virtio issue we are seeing. > > Regards, > Roger Melton > > On 9/6/23 2:57 PM, Dave Johnson (davejo) wrote: >> >> Hi Maxime, >> >> This email is regarding the following commit: >> >> https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657 >> >> A query had been sent previously on this topic (see below) indicating >> this commit appears to have broken the 32-bit testpmd app and impacted >> one of our products that runs as a 32-bit DPDK application.  We >> consequently backed the commit out of our product but would prefer to >> get a fix for it.  In the earlier exchange, you had asked if we were >> using virtio-pci or virtio-user (we are using virtio-pci) and asked >> for logs which Sampath provided.  It’s been a while, so let me now if >> you need me to send resend those logs or need any other information. >> >> FWIW, I reproduced this using testpmd and noticed that this part of >> the change seems to be the interesting part (in >> drivers/net/virtio/virtqueue.h): >> >> /** >> >> * Return the IOVA (or virtual address in case of virtio-user) of mbuf >> >> * data buffer. >> >> * >> >> * The address is firstly casted to the word size (sizeof(uintptr_t)) >> >> * before casting it to uint64_t. This is to make it work with different >> >> * combination of word size (64 bit and 32 bit) and virtio device >> >> * (virtio-pci and virtio-user). >> >> */ >> >> #define VIRTIO_MBUF_ADDR(mb, vq) \ >> >>       ((uint64_t)(*(uintptr_t *)((uintptr_t)(mb) + >> (vq)->mbuf_addr_offset)) >> >> If I revert just this part of the changeset (by re-using the >> VIRTIO_MBUF_ADDR to return buf_iova which matches what it had used >> previously), then 32-bit testpmd is able to receive traffic again: >> >> #define VIRTIO_MBUF_ADDR(mb, vq) (mb->buf_iova) >> >> Looking at the address produced by each of these, I see the address is >> the same except that the casting results in the upper bits getting >> cleared: >> >> Address from patch (nonworking case) = 0x58e7c900 >> >> Address using buf_iova (working case) = 0x158e7c900 >> >> :: >> >> Address from patch (nonworking case) = 0x58e7bfc0 >> >> Address using buf_iova (working case) = 0x158e7bfc0 >> >> :: >> >> Address from patch (nonworking case) = 0x58e7b680 >> >> Address using buf_iova (working case) = 0x158e7b680 >> >> :: >> >> Regards, Dave >> >> *From: *Sampath Peechu (speechu) >> *Date: *Monday, January 30, 2023 at 3:29 PM >> *To: *Maxime Coquelin , >> chenbo.xia@intel.com , dev@dpdk.org >> *Cc: *Roger Melton (rmelton) , Malcolm Bumgardner >> (mbumgard) >> *Subject: *Re: Commit broke 32-bit testpmd app >> >> Hi Maxime, >> >> Could you please let us know if you got a chance to look at the debugs >> logs I provided? >> >> Thanks, >> >> Sampath >> >> *From: *Sampath Peechu (speechu) >> *Date: *Tuesday, December 6, 2022 at 1:08 PM >> *To: *Maxime Coquelin , >> chenbo.xia@intel.com , dev@dpdk.org >> *Cc: *Roger Melton (rmelton) >> *Subject: *Re: Commit broke 32-bit testpmd app >> >> Hi Maxime, >> >> Did you get a chance to look into this? >> >> Please let me know if you need anything else. >> >> Thanks, >> >> Sampath >> >> *From: *Sampath Peechu (speechu) >> *Date: *Wednesday, November 23, 2022 at 5:15 PM >> *To: *Maxime Coquelin , >> chenbo.xia@intel.com , dev@dpdk.org >> *Cc: *Roger Melton (rmelton) >> *Subject: *Re: Commit broke 32-bit testpmd app >> >> Hi Maxime, >> >> I’m attaching the following for reference. >> >> * Instructions for Centos8 test setup >> * Diffs between the working and non-working versions (working >> version has the problem commit backed out) >> * Working logs (stats show that ping packets from neighbor VM can be >> seen with both 64-bit and 32-bit apps) >> * Non-working logs (stats show that ping packets from neighbor VM >> are seen with 64-bit app but NOT seen with 32-bit app) >> >> ============================ >> >> $ sudo ./usertools/dpdk-devbind.py --status >> >> Network devices using DPDK-compatible driver >> >> ============================================ >> >> 0000:07:00.0 'Virtio network device 1041' drv=igb_uio unused= >> >> 0000:08:00.0 'Virtio network device 1041' drv=igb_uio unused= >> >> Network devices using kernel driver >> >> =================================== >> >> 0000:01:00.0 'Virtio network device 1041' if=enp1s0 drv=virtio-pci >> unused=igb_uio *Active* >> >> … >> >> =========================== >> >> Thanks, >> >> Sampath >> >> *From: *Maxime Coquelin >> *Date: *Tuesday, November 22, 2022 at 4:24 AM >> *To: *Sampath Peechu (speechu) , >> chenbo.xia@intel.com , dev@dpdk.org >> *Cc: *Roger Melton (rmelton) >> *Subject: *Re: Commit broke 32-bit testpmd app >> >> Hi, >> >> In my initial reply (see below), I also asked if you had logs to share. >> And wondered whether it happens with Virtio PCI or Virtio-user? >> >> Regards, >> Maxime >> >> On 11/16/22 00:30, Sampath Peechu (speechu) wrote: >> > ++ dev@dpdk.org > >> > >> > *From: *Maxime Coquelin >> > *Date: *Tuesday, November 15, 2022 at 3:19 AM >> > *To: *Sampath Peechu (speechu) , >> chenbo.xia@intel.com >> > >> > *Cc: *Roger Melton (rmelton) >> > *Subject: *Re: Commit broke 32-bit testpmd app >> > >> > Hi Sampath, >> > >> > >> > Please add dev@dpdk.org, the upstream mailing list, if this is related >> > to the upstream DPDK project.If it is using RHEL DPDK package, please >> > use the appropriate support channels. >> > >> > On 11/14/22 23:55, Sampath Peechu (speechu) wrote: >> >  > Hi Virtio Maintainers team, >> >  > >> >  > This email is regarding the following commit. >> >  > >> >  > >> > >> https://github.com/DPDK/dpdk/commit/ba55c94a7ebc386d2288d6578ed57aad6cb92657 > >> >  > >> >  > The above commit appears to have broken the 32-bit testpmd app (and >> >  > consequently impacted one of our products that runs as a 32-bit DPDK >> >  > app). The 64-bit testpmd app does not appear to be impacted though. >> > >> > We'll need some logs to understand what is going on. >> > Does it happen with virtio-pci or virtio-user? >> > >> > Regards, >> > Maxime >> > >> >  > With the commit in place, we didn’t see any packets going through at >> >  > all. After backing out the commit and rebuilding the 32-bit >> testpmd app >> >  > in our test setup, we were able to pass traffic as expected. >> >  > >> >  > Could you please let us know if this is a known issue? And if >> there is a >> >  > fix available for it? >> >  > >> >  > Thank you, >> >  > >> >  > Sampath Peechu >> >  > >> >  > Cisco Systems >> >  > >> > >> > --------------eEnOQcOba2z40BCh0cKmlsRG Content-Type: text/x-log; charset=UTF-8; name="virtio_32bits.log" Content-Disposition: attachment; filename="virtio_32bits.log" Content-Transfer-Encoding: base64 CgogICAgW3Jvb3RAbG9jYWxob3N0IGRwZGtdIyBmaWxlIC4vYnVpbGQvYXBwL2RwZGstdGVzdHBt ZAogICAgLi9idWlsZC9hcHAvZHBkay10ZXN0cG1kOiBFTEYgMzItYml0IExTQiBleGVjdXRhYmxl LCBJbnRlbCA4MDM4NiwgdmVyc2lvbiAxIChTWVNWKSwgZHluYW1pY2FsbHkgbGlua2VkLCBpbnRl cnByZXRlciAvbGliL2xkLWxpbnV4LnNvLjIsIEJ1aWxkSURbc2hhMV09NTVjZjczZGFmOTUzMGI5 NzI5MGIyNTJlNDhkYTkzOTAxNjNkNDJiNywgZm9yIEdOVS9MaW51eCAzLjIuMCwgd2l0aCBkZWJ1 Z19pbmZvLCBub3Qgc3RyaXBwZWQKICAgIFtyb290QGxvY2FsaG9zdCBkcGRrXSMgdW5hbWUgLWEK ICAgIExpbnV4IGxvY2FsaG9zdC5sb2NhbGRvbWFpbiA2LjIuOS0zMDAuZmMzOC54ODZfNjQgIzEg U01QIFBSRUVNUFRfRFlOQU1JQyBUaHUgTWFyIDMwIDIyOjMyOjU4IFVUQyAyMDIzIHg4Nl82NCBH TlUvTGludXgKICAgIFtyb290QGxvY2FsaG9zdCBkcGRrXSMgLi9idWlsZC9hcHAvZHBkay10ZXN0 cG1kIC0tIC1pCiAgICBFQUw6IERldGVjdGVkIENQVSBsY29yZXM6IDMKICAgIEVBTDogRGV0ZWN0 ZWQgTlVNQSBub2RlczogMQogICAgRUFMOiBEZXRlY3RlZCBzdGF0aWMgbGlua2FnZSBvZiBEUERL CiAgICBFQUw6IE11bHRpLXByb2Nlc3Mgc29ja2V0IC92YXIvcnVuL2RwZGsvcnRlL21wX3NvY2tl dAogICAgRUFMOiBTZWxlY3RlZCBJT1ZBIG1vZGUgJ1BBJwogICAgRUFMOiBWRklPIHN1cHBvcnQg aW5pdGlhbGl6ZWQKICAgIEVBTDogUHJvYmUgUENJIGRyaXZlcjogbmV0X3ZpcnRpbyAoMWFmNDox MDQxKSBkZXZpY2U6IDAwMDA6MDE6MDAuMCAoc29ja2V0IC0xKQogICAgZXRoX3ZpcnRpb19wY2lf aW5pdCgpOiBGYWlsZWQgdG8gaW5pdCBQQ0kgZGV2aWNlCiAgICBFQUw6IFJlcXVlc3RlZCBkZXZp Y2UgMDAwMDowMTowMC4wIGNhbm5vdCBiZSB1c2VkCiAgICBFQUw6IFByb2JlIFBDSSBkcml2ZXI6 IG5ldF92aXJ0aW8gKDFhZjQ6MTA0MSkgZGV2aWNlOiAwMDAwOjA3OjAwLjAgKHNvY2tldCAtMSkK ICAgIEVBTDogVXNpbmcgSU9NTVUgdHlwZSA4IChOby1JT01NVSkKICAgIFRFTEVNRVRSWTogTm8g bGVnYWN5IGNhbGxiYWNrcywgbGVnYWN5IHNvY2tldCBub3QgY3JlYXRlZAogICAgSW50ZXJhY3Rp dmUtbW9kZSBzZWxlY3RlZAogICAgV2FybmluZzogTlVNQSBzaG91bGQgYmUgY29uZmlndXJlZCBt YW51YWxseSBieSB1c2luZyAtLXBvcnQtbnVtYS1jb25maWcgYW5kIC0tcmluZy1udW1hLWNvbmZp ZyBwYXJhbWV0ZXJzIGFsb25nIHdpdGggLS1udW1hLgogICAgdGVzdHBtZDogY3JlYXRlIGEgbmV3 IG1idWYgcG9vbCA8bWJfcG9vbF8wPjogbj0xNjM0NTYsIHNpemU9MjE3Niwgc29ja2V0PTAKICAg IHRlc3RwbWQ6IHByZWZlcnJlZCBtZW1wb29sIG9wcyBzZWxlY3RlZDogcmluZ19tcF9tYwogICAg IAogICAgV2FybmluZyEgcG9ydC10b3BvbG9neT1wYWlyZWQgYW5kIG9kZCBmb3J3YXJkIHBvcnRz IG51bWJlciwgdGhlIGxhc3QgcG9ydCB3aWxsIHBhaXIgd2l0aCBpdHNlbGYuCiAgICAgCiAgICBD b25maWd1cmluZyBQb3J0IDAgKHNvY2tldCAwKQogICAgRUFMOiBFcnJvciBkaXNhYmxpbmcgTVNJ LVggaW50ZXJydXB0cyBmb3IgZmQgMjQKICAgIFBvcnQgMDogNTY6NDg6NEY6NTM6NTQ6MDEKICAg IENoZWNraW5nIGxpbmsgc3RhdHVzZXMuLi4KICAgIERvbmUKICAgIHRlc3RwbWQ+IHNldCBmd2Qg aWNtcGVjaG8KICAgIFNldCBpY21wZWNobyBwYWNrZXQgZm9yd2FyZGluZyBtb2RlCiAgICB0ZXN0 cG1kPiBzdGFydAogICAgaWNtcGVjaG8gcGFja2V0IGZvcndhcmRpbmcgLSBwb3J0cz0xIC0gY29y ZXM9MSAtIHN0cmVhbXM9MSAtIE5VTUEgc3VwcG9ydCBlbmFibGVkLCBNUCBhbGxvY2F0aW9uIG1v ZGU6IG5hdGl2ZQogICAgTG9naWNhbCBDb3JlIDEgKHNvY2tldCAwKSBmb3J3YXJkcyBwYWNrZXRz IG9uIDEgc3RyZWFtczoKICAgICAgUlggUD0wL1E9MCAoc29ja2V0IDApIC0+IFRYIFA9MC9RPTAg KHNvY2tldCAwKSBwZWVyPTAyOjAwOjAwOjAwOjAwOjAwCiAgICAgCiAgICAgIGljbXBlY2hvIHBh Y2tldCBmb3J3YXJkaW5nIHBhY2tldHMvYnVyc3Q9MzIKICAgICAgbmIgZm9yd2FyZGluZyBjb3Jl cz0xIC0gbmIgZm9yd2FyZGluZyBwb3J0cz0xCiAgICAgIHBvcnQgMDogUlggcXVldWUgbnVtYmVy OiAxIFR4IHF1ZXVlIG51bWJlcjogMQogICAgICAgIFJ4IG9mZmxvYWRzPTB4MCBUeCBvZmZsb2Fk cz0weDAKICAgICAgICBSWCBxdWV1ZTogMAogICAgICAgICAgUlggZGVzYz0wIC0gUlggZnJlZSB0 aHJlc2hvbGQ9MAogICAgICAgICAgUlggdGhyZXNob2xkIHJlZ2lzdGVyczogcHRocmVzaD0wIGh0 aHJlc2g9MCAgd3RocmVzaD0wCiAgICAgICAgICBSWCBPZmZsb2Fkcz0weDAKICAgICAgICBUWCBx dWV1ZTogMAogICAgICAgICAgVFggZGVzYz0wIC0gVFggZnJlZSB0aHJlc2hvbGQ9MAogICAgICAg ICAgVFggdGhyZXNob2xkIHJlZ2lzdGVyczogcHRocmVzaD0wIGh0aHJlc2g9MCAgd3RocmVzaD0w CiAgICAgICAgICBUWCBvZmZsb2Fkcz0weDAgLSBUWCBSUyBiaXQgdGhyZXNob2xkPTAKICAgIHRl c3RwbWQ+IHNldCB2ZXJib3NlIDkKICAgIENoYW5nZSB2ZXJib3NlIGxldmVsIGZyb20gMCB0byA5 CiAgICB0ZXN0cG1kPiBwb3J0IDAvcXVldWUgMDogcmVjZWl2ZWQgMSBwYWNrZXRzCiAgICAgIHNy Yz1ERTpEMDowNjpBQzpEQzozQiAtIGRzdD01Njo0ODo0Rjo1Mzo1NDowMSAtIHBvb2w9bWJfcG9v bF8wIC0gdHlwZT0weDA4MDAgLSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlwZTogTDJf RVRIRVIgTDNfSVBWNCAgLSBsMl9sZW49MTQgLSBsM19sZW49MjAgLSBSZWNlaXZlIHF1ZXVlPTB4 MAogICAgICBvbF9mbGFnczogUlRFX01CVUZfRl9SWF9MNF9DS1NVTV9VTktOT1dOIFJURV9NQlVG X0ZfUlhfSVBfQ0tTVU1fVU5LTk9XTiBSVEVfTUJVRl9GX1JYX09VVEVSX0w0X0NLU1VNX1VOS05P V04KICAgICAKICAgIFBvcnQgMCBwa3QtbGVuPTk4IG5iLXNlZ3M9MQogICAgICBFVEg6ICBzcmM9 REU6RDA6MDY6QUM6REM6M0IgZHN0PTU2OjQ4OjRGOjUzOjU0OjAxIHR5cGU9MHgwODAwCiAgICAg IElQVjQ6IHNyYz0xOTIuMTY4LjEwMS4yIGRzdD0xOTIuMTY4LjEwMS4zIHByb3RvPTEgKElDTVAp CiAgICAgIElDTVA6IGVjaG8gcmVxdWVzdCBzZXEgaWQ9MjIzCiAgICBwb3J0IDAvcXVldWUgMDog c2VudCAxIHBhY2tldHMKICAgICAgc3JjPTU2OjQ4OjRGOjUzOjU0OjAxIC0gZHN0PURFOkQwOjA2 OkFDOkRDOjNCIC0gcG9vbD1tYl9wb29sXzAgLSB0eXBlPTB4MDgwMCAtIGxlbmd0aD05OCAtIG5i X3NlZ3M9MSAtIHN3IHB0eXBlOiBMMl9FVEhFUiBMM19JUFY0ICAtIGwyX2xlbj0xNCAtIGwzX2xl bj0yMCAtIFNlbmQgcXVldWU9MHgwCiAgICAgIG9sX2ZsYWdzOiBSVEVfTUJVRl9GX1RYX0w0X05P X0NLU1VNCiAgICBwb3J0IDAvcXVldWUgMDogcmVjZWl2ZWQgMSBwYWNrZXRzCiAgICAgIHNyYz1E RTpEMDowNjpBQzpEQzozQiAtIGRzdD01Njo0ODo0Rjo1Mzo1NDowMSAtIHBvb2w9bWJfcG9vbF8w IC0gdHlwZT0weDA4MDAgLSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlwZTogTDJfRVRI RVIgTDNfSVBWNCAgLSBsMl9sZW49MTQgLSBsM19sZW49MjAgLSBSZWNlaXZlIHF1ZXVlPTB4MAog ICAgICBvbF9mbGFnczogUlRFX01CVUZfRl9SWF9MNF9DS1NVTV9VTktOT1dOIFJURV9NQlVGX0Zf UlhfSVBfQ0tTVU1fVU5LTk9XTiBSVEVfTUJVRl9GX1JYX09VVEVSX0w0X0NLU1VNX1VOS05PV04K ICAgICAKICAgIFBvcnQgMCBwa3QtbGVuPTk4IG5iLXNlZ3M9MQogICAgICBFVEg6ICBzcmM9REU6 RDA6MDY6QUM6REM6M0IgZHN0PTU2OjQ4OjRGOjUzOjU0OjAxIHR5cGU9MHgwODAwCiAgICAgIElQ VjQ6IHNyYz0xOTIuMTY4LjEwMS4yIGRzdD0xOTIuMTY4LjEwMS4zIHByb3RvPTEgKElDTVApCiAg ICAgIElDTVA6IGVjaG8gcmVxdWVzdCBzZXEgaWQ9MjI0CiAgICBwb3J0IDAvcXVldWUgMDogc2Vu dCAxIHBhY2tldHMKICAgICAgc3JjPTU2OjQ4OjRGOjUzOjU0OjAxIC0gZHN0PURFOkQwOjA2OkFD OkRDOjNCIC0gcG9vbD1tYl9wb29sXzAgLSB0eXBlPTB4MDgwMCAtIGxlbmd0aD05OCAtIG5iX3Nl Z3M9MSAtIHN3IHB0eXBlOiBMMl9FVEhFUiBMM19JUFY0ICAtIGwyX2xlbj0xNCAtIGwzX2xlbj0y MCAtIFNlbmQgcXVldWU9MHgwCiAgICAgIG9sX2ZsYWdzOiBSVEVfTUJVRl9GX1RYX0w0X05PX0NL U1VNCiAgICBwb3J0IDAvcXVldWUgMDogcmVjZWl2ZWQgMSBwYWNrZXRzCiAgICAgIHNyYz1ERTpE MDowNjpBQzpEQzozQiAtIGRzdD01Njo0ODo0Rjo1Mzo1NDowMSAtIHBvb2w9bWJfcG9vbF8wIC0g dHlwZT0weDA4MDAgLSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlwZTogTDJfRVRIRVIg TDNfSVBWNCAgLSBsMl9sZW49MTQgLSBsM19sZW49MjAgLSBSZWNlaXZlIHF1ZXVlPTB4MAogICAg ICBvbF9mbGFnczogUlRFX01CVUZfRl9SWF9MNF9DS1NVTV9VTktOT1dOIFJURV9NQlVGX0ZfUlhf SVBfQ0tTVU1fVU5LTk9XTiBSVEVfTUJVRl9GX1JYX09VVEVSX0w0X0NLU1VNX1VOS05PV04KICAg ICAKICAgIFBvcnQgMCBwa3QtbGVuPTk4IG5iLXNlZ3M9MQogICAgICBFVEg6ICBzcmM9REU6RDA6 MDY6QUM6REM6M0IgZHN0PTU2OjQ4OjRGOjUzOjU0OjAxIHR5cGU9MHgwODAwCiAgICAgIElQVjQ6 IHNyYz0xOTIuMTY4LjEwMS4yIGRzdD0xOTIuMTY4LjEwMS4zIHByb3RvPTEgKElDTVApCiAgICAg IElDTVA6IGVjaG8gcmVxdWVzdCBzZXEgaWQ9MjI1CiAgICBwb3J0IDAvcXVldWUgMDogc2VudCAx IHBhY2tldHMKICAgICAgc3JjPTU2OjQ4OjRGOjUzOjU0OjAxIC0gZHN0PURFOkQwOjA2OkFDOkRD OjNCIC0gcG9vbD1tYl9wb29sXzAgLSB0eXBlPTB4MDgwMCAtIGxlbmd0aD05OCAtIG5iX3NlZ3M9 MSAtIHN3IHB0eXBlOiBMMl9FVEhFUiBMM19JUFY0ICAtIGwyX2xlbj0xNCAtIGwzX2xlbj0yMCAt IFNlbmQgcXVldWU9MHgwCiAgICAgIG9sX2ZsYWdzOiBSVEVfTUJVRl9GX1RYX0w0X05PX0NLU1VN CiAgICBwb3J0IDAvcXVldWUgMDogcmVjZWl2ZWQgMSBwYWNrZXRzCiAgICAgIHNyYz1ERTpEMDow NjpBQzpEQzozQiAtIGRzdD01Njo0ODo0Rjo1Mzo1NDowMSAtIHBvb2w9bWJfcG9vbF8wIC0gdHlw ZT0weDA4MDAgLSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlwZTogTDJfRVRIRVIgTDNf SVBWNCAgLSBsMl9sZW49MTQgLSBsM19sZW49MjAgLSBSZWNlaXZlIHF1ZXVlPTB4MAogICAgICBv bF9mbGFnczogUlRFX01CVUZfRl9SWF9MNF9DS1NVTV9VTktOT1dOIFJURV9NQlVGX0ZfUlhfSVBf Q0tTVU1fVU5LTk9XTiBSVEVfTUJVRl9GX1JYX09VVEVSX0w0X0NLU1VNX1VOS05PV04KICAgICAK ICAgIFBvcnQgMCBwa3QtbGVuPTk4IG5iLXNlZ3M9MQogICAgICBFVEg6ICBzcmM9REU6RDA6MDY6 QUM6REM6M0IgZHN0PTU2OjQ4OjRGOjUzOjU0OjAxIHR5cGU9MHgwODAwCiAgICAgIElQVjQ6IHNy Yz0xOTIuMTY4LjEwMS4yIGRzdD0xOTIuMTY4LjEwMS4zIHByb3RvPTEgKElDTVApCiAgICAgIElD TVA6IGVjaG8gcmVxdWVzdCBzZXEgaWQ9MjI2CiAgICBwb3J0IDAvcXVldWUgMDogc2VudCAxIHBh Y2tldHMKICAgICAgc3JjPTU2OjQ4OjRGOjUzOjU0OjAxIC0gZHN0PURFOkQwOjA2OkFDOkRDOjNC IC0gcG9vbD1tYl9wb29sXzAgLSB0eXBlPTB4MDgwMCAtIGxlbmd0aD05OCAtIG5iX3NlZ3M9MSAt IHN3IHB0eXBlOiBMMl9FVEhFUiBMM19JUFY0ICAtIGwyX2xlbj0xNCAtIGwzX2xlbj0yMCAtIFNl bmQgcXVldWU9MHgwCiAgICAgIG9sX2ZsYWdzOiBSVEVfTUJVRl9GX1RYX0w0X05PX0NLU1VNCiAg ICBwb3J0IDAvcXVldWUgMDogcmVjZWl2ZWQgMSBwYWNrZXRzCiAgICAgIHNyYz1ERTpEMDowNjpB QzpEQzozQiAtIGRzdD01Njo0ODo0Rjo1Mzo1NDowMSAtIHBvb2w9bWJfcG9vbF8wIC0gdHlwZT0w eDA4MDAgLSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlwZTogTDJfRVRIRVIgTDNfSVBW NCAgLSBsMl9sZW49MTQgLSBsM19sZW49MjAgLSBSZWNlaXZlIHF1ZXVlPTB4MAogICAgICBvbF9m bGFnczogUlRFX01CVUZfRl9SWF9MNF9DS1NVTV9VTktOT1dOIFJURV9NQlVGX0ZfUlhfSVBfQ0tT VU1fVU5LTk9XTiBSVEVfTUJVRl9GX1JYX09VVEVSX0w0X0NLU1VNX1VOS05PV04KICAgICAKICAg IFBvcnQgMCBwa3QtbGVuPTk4IG5iLXNlZ3M9MQogICAgICBFVEg6ICBzcmM9REU6RDA6MDY6QUM6 REM6M0IgZHN0PTU2OjQ4OjRGOjUzOjU0OjAxIHR5cGU9MHgwODAwCiAgICAgIElQVjQ6IHNyYz0x OTIuMTY4LjEwMS4yIGRzdD0xOTIuMTY4LjEwMS4zIHByb3RvPTEgKElDTVApCiAgICAgIElDTVA6 IGVjaG8gcmVxdWVzdCBzZXEgaWQ9MjI3CiAgICBwb3J0IDAvcXVldWUgMDogc2VudCAxIHBhY2tl dHMKICAgICAgc3JjPTU2OjQ4OjRGOjUzOjU0OjAxIC0gZHN0PURFOkQwOjA2OkFDOkRDOjNCIC0g cG9vbD1tYl9wb29sXzAgLSB0eXBlPTB4MDgwMCAtIGxlbmd0aD05OCAtIG5iX3NlZ3M9MSAtIHN3 IHB0eXBlOiBMMl9FVEhFUiBMM19JUFY0ICAtIGwyX2xlbj0xNCAtIGwzX2xlbj0yMCAtIFNlbmQg cXVldWU9MHgwCiAgICAgIG9sX2ZsYWdzOiBSVEVfTUJVRl9GX1RYX0w0X05PX0NLU1VNCiAgICBw b3J0IDAvcXVldWUgMDogcmVjZWl2ZWQgMSBwYWNrZXRzCiAgICAgIHNyYz1ERTpEMDowNjpBQzpE QzozQiAtIGRzdD01Njo0ODo0Rjo1Mzo1NDowMSAtIHBvb2w9bWJfcG9vbF8wIC0gdHlwZT0weDA4 MDAgLSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlwZTogTDJfRVRIRVIgTDNfSVBWNCAg LSBsMl9sZW49MTQgLSBsM19sZW49MjAgLSBSZWNlaXZlIHF1ZXVlPTB4MAogICAgICBvbF9mbGFn czogUlRFX01CVUZfRl9SWF9MNF9DS1NVTV9VTktOT1dOIFJURV9NQlVGX0ZfUlhfSVBfQ0tTVU1f VU5LTk9XTiBSVEVfTUJVRl9GX1JYX09VVEVSX0w0X0NLU1VNX1VOS05PV04KICAgICAKICAgIFBv cnQgMCBwa3QtbGVuPTk4IG5iLXNlZ3M9MQogICAgICBFVEg6ICBzcmM9REU6RDA6MDY6QUM6REM6 M0IgZHN0PTU2OjQ4OjRGOjUzOjU0OjAxIHR5cGU9MHgwODAwCiAgICAgIElQVjQ6IHNyYz0xOTIu MTY4LjEwMS4yIGRzdD0xOTIuMTY4LjEwMS4zIHByb3RvPTEgKElDTVApCiAgICAgIElDTVA6IGVj aG8gcmVxdWVzdCBzZXEgaWQ9MjI4CiAgICBwb3J0IDAvcXVldWUgMDogc2VudCAxIHBhY2tldHMK ICAgICAgc3JjPTU2OjQ4OjRGOjUzOjU0OjAxIC0gZHN0PURFOkQwOjA2OkFDOkRDOjNCIC0gcG9v bD1tYl9wb29sXzAgLSB0eXBlPTB4MDgwMCAtIGxlbmd0aD05OCAtIG5iX3NlZ3M9MSAtIHN3IHB0 eXBlOiBMMl9FVEhFUiBMM19JUFY0ICAtIGwyX2xlbj0xNCAtIGwzX2xlbj0yMCAtIFNlbmQgcXVl dWU9MHgwCiAgICAgIG9sX2ZsYWdzOiBSVEVfTUJVRl9GX1RYX0w0X05PX0NLU1VNCiAgICBwb3J0 IDAvcXVldWUgMDogcmVjZWl2ZWQgMSBwYWNrZXRzCiAgICAgIHNyYz1ERTpEMDowNjpBQzpEQzoz QiAtIGRzdD01Njo0ODo0Rjo1Mzo1NDowMSAtIHBvb2w9bWJfcG9vbF8wIC0gdHlwZT0weDA4MDAg LSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlwZTogTDJfRVRIRVIgTDNfSVBWNCAgLSBs Ml9sZW49MTQgLSBsM19sZW49MjAgLSBSZWNlaXZlIHF1ZXVlPTB4MAogICAgICBvbF9mbGFnczog UlRFX01CVUZfRl9SWF9MNF9DS1NVTV9VTktOT1dOIFJURV9NQlVGX0ZfUlhfSVBfQ0tTVU1fVU5L Tk9XTiBSVEVfTUJVRl9GX1JYX09VVEVSX0w0X0NLU1VNX1VOS05PV04KICAgICAKICAgIFBvcnQg MCBwa3QtbGVuPTk4IG5iLXNlZ3M9MQogICAgICBFVEg6ICBzcmM9REU6RDA6MDY6QUM6REM6M0Ig ZHN0PTU2OjQ4OjRGOjUzOjU0OjAxIHR5cGU9MHgwODAwCiAgICAgIElQVjQ6IHNyYz0xOTIuMTY4 LjEwMS4yIGRzdD0xOTIuMTY4LjEwMS4zIHByb3RvPTEgKElDTVApCiAgICAgIElDTVA6IGVjaG8g cmVxdWVzdCBzZXEgaWQ9MjI5CiAgICBwb3J0IDAvcXVldWUgMDogc2VudCAxIHBhY2tldHMKICAg ICAgc3JjPTU2OjQ4OjRGOjUzOjU0OjAxIC0gZHN0PURFOkQwOjA2OkFDOkRDOjNCIC0gcG9vbD1t Yl9wb29sXzAgLSB0eXBlPTB4MDgwMCAtIGxlbmd0aD05OCAtIG5iX3NlZ3M9MSAtIHN3IHB0eXBl OiBMMl9FVEhFUiBMM19JUFY0ICAtIGwyX2xlbj0xNCAtIGwzX2xlbj0yMCAtIFNlbmQgcXVldWU9 MHgwCiAgICAgIG9sX2ZsYWdzOiBSVEVfTUJVRl9GX1RYX0w0X05PX0NLU1VNCiAgICBzdG9wcG9y dCAwL3F1ZXVlIDA6IHJlY2VpdmVkIDEgcGFja2V0cwogICAgICBzcmM9REU6RDA6MDY6QUM6REM6 M0IgLSBkc3Q9NTY6NDg6NEY6NTM6NTQ6MDEgLSBwb29sPW1iX3Bvb2xfMCAtIHR5cGU9MHgwODAw IC0gbGVuZ3RoPTk4IC0gbmJfc2Vncz0xIC0gc3cgcHR5cGU6IEwyX0VUSEVSIEwzX0lQVjQgIC0g bDJfbGVuPTE0IC0gbDNfbGVuPTIwIC0gUmVjZWl2ZSBxdWV1ZT0weDAKICAgICAgb2xfZmxhZ3M6 IFJURV9NQlVGX0ZfUlhfTDRfQ0tTVU1fVU5LTk9XTiBSVEVfTUJVRl9GX1JYX0lQX0NLU1VNX1VO S05PV04gUlRFX01CVUZfRl9SWF9PVVRFUl9MNF9DS1NVTV9VTktOT1dOCiAgICAgCiAgICBQb3J0 IDAgcGt0LWxlbj05OCBuYi1zZWdzPTEKICAgICAgRVRIOiAgc3JjPURFOkQwOjA2OkFDOkRDOjNC IGRzdD01Njo0ODo0Rjo1Mzo1NDowMSB0eXBlPTB4MDgwMAogICAgICBJUFY0OiBzcmM9MTkyLjE2 OC4xMDEuMiBkc3Q9MTkyLjE2OC4xMDEuMyBwcm90bz0xIChJQ01QKQogICAgICBJQ01QOiBlY2hv IHJlcXVlc3Qgc2VxIGlkPTIzMAogICAgcG9ydCAwL3F1ZXVlIDA6IHNlbnQgMSBwYWNrZXRzCiAg ICAgIHNyYz01Njo0ODo0Rjo1Mzo1NDowMSAtIGRzdD1ERTpEMDowNjpBQzpEQzozQiAtIHBvb2w9 bWJfcG9vbF8wIC0gdHlwZT0weDA4MDAgLSBsZW5ndGg9OTggLSBuYl9zZWdzPTEgLSBzdyBwdHlw ZTogTDJfRVRIRVIgTDNfSVBWNCAgLSBsMl9sZW49MTQgLSBsM19sZW49MjAgLSBTZW5kIHF1ZXVl PTB4MAogICAgICBvbF9mbGFnczogUlRFX01CVUZfRl9UWF9MNF9OT19DS1NVTQogICAgIAogICAg VGVsbGluZyBjb3JlcyB0byBzdG9wLi4uCiAgICBXYWl0aW5nIGZvciBsY29yZXMgdG8gZmluaXNo Li4uCiAgICAgCiAgICAgIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gRm9yd2FyZCBzdGF0aXN0aWNz IGZvciBwb3J0IDAgIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KICAgICAgUlgtcGFja2V0czogMTUg ICAgICAgICAgICAgUlgtZHJvcHBlZDogMCAgICAgICAgICAgICBSWC10b3RhbDogMTUKICAgICAg VFgtcGFja2V0czogMTUgICAgICAgICAgICAgVFgtZHJvcHBlZDogMCAgICAgICAgICAgICBUWC10 b3RhbDogMTUKICAgICAgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQogICAgIAogICAgICArKysrKysrKysr KysrKysgQWNjdW11bGF0ZWQgZm9yd2FyZCBzdGF0aXN0aWNzIGZvciBhbGwgcG9ydHMrKysrKysr KysrKysrKysKICAgICAgUlgtcGFja2V0czogMTUgICAgICAgICAgICAgUlgtZHJvcHBlZDogMCAg ICAgICAgICAgICBSWC10b3RhbDogMTUKICAgICAgVFgtcGFja2V0czogMTUgICAgICAgICAgICAg VFgtZHJvcHBlZDogMCAgICAgICAgICAgICBUWC10b3RhbDogMTUKICAgICAgKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrKwogICAgIAogICAgRG9uZS4KICAgIHRlc3RwbWQ+CgoK --------------eEnOQcOba2z40BCh0cKmlsRG--