From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0CA1F43875; Wed, 17 Jan 2024 21:13:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C904C4014F; Wed, 17 Jan 2024 21:13:41 +0100 (CET) Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) by mails.dpdk.org (Postfix) with ESMTP id 94C134003C for ; Wed, 17 Jan 2024 21:13:39 +0100 (CET) Received: by mail-yb1-f176.google.com with SMTP id 3f1490d57ef6-da7ea62e76cso9790747276.3 for ; Wed, 17 Jan 2024 12:13:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705522419; x=1706127219; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=0wa84XW0gsOMFt3pGQPDeitsRv4yXGC/upD9TsxUZ+4=; b=dlRgo/uyVC6ZoPkmv+GVoHOQP1PfqmeFs4smdPQlw4VNiOjrxNUaJpo4hxNtwGRe5J 3B7/rnthnV4WcHBhuUeIxnQpkO1G9KG7GMtSxQ3E06VHf0zbDGnZuRsQhar/GT3bTny+ 3H/XlPV0jfboUYWI6olIBnWP3fBudT3NV26uaFSPSd1fBZiLpcA59qQ/eACYry/QbrKV 8zhgXwgGUxAvcVm7Gi5PjVtPZgMK+bi+zoxCBEYUlClpAhGjyYfdIpq8Biwov8ESzUSd f8pi+rr9Sq61sm9D/fSs/nJUp3PzoiWD+Y0bIe3rBn6QRVAcVYiJkbSQI/Z9LikZ0UxF piNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705522419; x=1706127219; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0wa84XW0gsOMFt3pGQPDeitsRv4yXGC/upD9TsxUZ+4=; b=lVmlr1t/XYy7ruA4O2YEe3pUS8ZoIohMwHCqGOaubbqqwgEb1v30Twn0xj3Vi3gDm8 d6oBdH6mwgWFIRlLjeK0swtjnhaW/b7hLgntgC+4Cju2FjdmEe5HiRT022z2nHmNbJc0 TTfgDAifM96zTvtl8pbj/HtvFUoIsDkm8qjqoije8j9E1fuoypbLVg+sHvJvC+Z7JXbA oyMTQ0FntaSliT8DVCY0aPdm6wzeiLoLClnbLVjOcLAQXdynp2+N3+KVL2PYJxipU2BW sshgMCsBYeSMCiOu33aWvez3ili7wiAOCsVjFxHkTZjl20t0jfM2lRBmWgdQ/u4ix/QN oGlg== X-Gm-Message-State: AOJu0YyKbm58bJ6HAcOHiHQI1Mbba3MbwG+J2hcrkG15+mSrIAx3wwKd Jr6UdHwPK+kL5/d3DTMGSmGeNF2PmBqrHAuPAnQ= X-Google-Smtp-Source: AGHT+IFdq0M1PBx+hz8CSxt4DwcBjv/66Pb3HQxXFYnWo8Vr+C4jQFWSmB3UpB4tqmCGMvbh+9SD5D6/EeYFX6mXv5Y= X-Received: by 2002:a25:8e0a:0:b0:dc2:303b:db2f with SMTP id p10-20020a258e0a000000b00dc2303bdb2fmr2096276ybl.101.1705522418843; Wed, 17 Jan 2024 12:13:38 -0800 (PST) MIME-Version: 1.0 References: <20240117195228.423261-1-kumaraparamesh92@gmail.co> <20240117201014.425906-1-kumaraparamesh92@gmail.com> In-Reply-To: <20240117201014.425906-1-kumaraparamesh92@gmail.com> From: kumaraparameshwaran rathinavel Date: Thu, 18 Jan 2024 01:43:28 +0530 Message-ID: Subject: Re: [PATCH v9] gro: fix reordering of packets in GRO layer To: hujiayu.hu@foxmail.com Cc: dev@dpdk.org, Kumara Parameshwaran Content-Type: multipart/alternative; boundary="000000000000ee85d9060f29e155" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000ee85d9060f29e155 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Jan 18, 2024 at 1:40=E2=80=AFAM Kumara Parameshwaran < kumaraparamesh92@gmail.com> wrote: > In the current implementation when a packet is received with > special TCP flag(s) set, only that packet is delivered out of order. > There could be already coalesced packets in the GRO table > belonging to the same flow but not delivered. > This fix makes sure that the entire segment is delivered with the > special flag(s) set which is how the Linux GRO is also implemented > > Signed-off-by: Kumara Parameshwaran > Co-authored-by: Kumara Parameshwaran > --- > If the received packet is not a pure ACK packet, we check if > there are any previous packets in the flow, if present we indulge > the received packet also in the coalescing logic and update the > flags > of the last recived packet to the entire segment which would avoi= d > re-ordering. > > Lets say a case where P1(PSH), P2(ACK), P3(ACK) are received in > burst mode, > P1 contains PSH flag and since it does not contain any prior > packets in the flow > we copy it to unprocess_packets and P2(ACK) and P3(ACK) are merge= d > together. > In the existing case the P2,P3 would be delivered as single > segment first and the > unprocess_packets will be copied later which will cause > reordering. With the patch > copy the unprocess packets first and then the packets from the GR= O > table. > > Testing done > The csum test-pmd was modified to support the following > GET request of 10MB from client to server via test-pmd (static ar= p > entries added in client > and server). Enable GRO and TSO in test-pmd where the packets > recived from the client mac > would be sent to server mac and vice versa. > In above testing, without the patch the client observerd > re-ordering of 25 packets > and with the patch there were no packet re-ordering observerd. > > v2: > Fix warnings in commit and comment. > Do not consider packet as candidate to merge if it contains > SYN/RST flag. > > v3: > Fix warnings. > > v4: > Rebase with master. > > v5: > Adding co-author email > v6: > Address review comments from the maintainer to restructure the > code > and handle only special flags PSH,FIN > > v7: > Fix warnings and errors > > v8: > Fix warnings and errors > > v9: > Fix commit message > > lib/gro/gro_tcp.h | 11 ++++++++ > lib/gro/gro_tcp4.c | 67 +++++++++++++++++++++++++++++----------------- > 2 files changed, 54 insertions(+), 24 deletions(-) > > diff --git a/lib/gro/gro_tcp.h b/lib/gro/gro_tcp.h > index d926c4b8cc..137a03bc96 100644 > --- a/lib/gro/gro_tcp.h > +++ b/lib/gro/gro_tcp.h > @@ -187,4 +187,15 @@ is_same_common_tcp_key(struct cmn_tcp_key *k1, struc= t > cmn_tcp_key *k2) > return (!memcmp(k1, k2, sizeof(struct cmn_tcp_key))); > } > > +static inline void > +update_tcp_hdr_flags(struct rte_tcp_hdr *tcp_hdr, struct rte_mbuf *pkt) > +{ > + struct rte_tcp_hdr *merged_tcp_hdr; > + > + merged_tcp_hdr =3D rte_pktmbuf_mtod_offset(pkt, struct rte_tcp_hd= r > *, pkt->l2_len + > + pkt->l3_len); > + merged_tcp_hdr->tcp_flags |=3D tcp_hdr->tcp_flags; > + > +} > + > #endif > diff --git a/lib/gro/gro_tcp4.c b/lib/gro/gro_tcp4.c > index 6645de592b..8af5a8d8a9 100644 > --- a/lib/gro/gro_tcp4.c > +++ b/lib/gro/gro_tcp4.c > @@ -126,6 +126,7 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, > uint32_t item_idx; > uint32_t i, max_flow_num, remaining_flow_num; > uint8_t find; > + uint32_t item_start_idx; > > /* > * Don't process the packet whose TCP header length is greater > @@ -139,13 +140,6 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, > tcp_hdr =3D (struct rte_tcp_hdr *)((char *)ipv4_hdr + pkt->l3_len= ); > hdr_len =3D pkt->l2_len + pkt->l3_len + pkt->l4_len; > > - /* > - * Don't process the packet which has FIN, SYN, RST, PSH, URG, EC= E > - * or CWR set. > - */ > - if (tcp_hdr->tcp_flags !=3D RTE_TCP_ACK_FLAG) > - return -1; > - > /* trim the tail padding bytes */ > ip_tlen =3D rte_be_to_cpu_16(ipv4_hdr->total_length); > if (pkt->pkt_len > (uint32_t)(ip_tlen + pkt->l2_len)) > @@ -183,6 +177,7 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, > if (tbl->flows[i].start_index !=3D INVALID_ARRAY_INDEX) { > if (is_same_tcp4_flow(tbl->flows[i].key, key)) { > find =3D 1; > + item_start_idx =3D tbl->flows[i].start_in= dex; > break; > } > remaining_flow_num--; > @@ -190,28 +185,52 @@ gro_tcp4_reassemble(struct rte_mbuf *pkt, > } > > if (find =3D=3D 0) { > - sent_seq =3D rte_be_to_cpu_32(tcp_hdr->sent_seq); > - item_idx =3D insert_new_tcp_item(pkt, tbl->items, > &tbl->item_num, > - tbl->max_item_num, > start_time, > - INVALID_ARRAY_INDEX, > sent_seq, ip_id, > - is_atomic); > - if (item_idx =3D=3D INVALID_ARRAY_INDEX) > + /* > + * Add new flow to the table only if contains ACK flag > with data. > + * Do not add any packets with additional tcp flags to th= e > GRO table > + */ > + if (tcp_hdr->tcp_flags =3D=3D RTE_TCP_ACK_FLAG) { > + sent_seq =3D rte_be_to_cpu_32(tcp_hdr->sent_seq); > + item_idx =3D insert_new_tcp_item(pkt, tbl->items, > &tbl->item_num, > + tbl->max_item_num= , > start_time, > + > INVALID_ARRAY_INDEX, sent_seq, ip_id, > + is_atomic); > + if (item_idx =3D=3D INVALID_ARRAY_INDEX) > + return -1; > + if (insert_new_flow(tbl, &key, item_idx) =3D=3D > + INVALID_ARRAY_INDEX) { > + /* > + * Fail to insert a new flow, so delete t= he > + * stored packet. > + */ > + delete_tcp_item(tbl->items, item_idx, > &tbl->item_num, > + INVALID_ARRAY_INDEX); > + return -1; > + } > + return 0; > + } else { > return -1; > - if (insert_new_flow(tbl, &key, item_idx) =3D=3D > - INVALID_ARRAY_INDEX) { > - /* > - * Fail to insert a new flow, so delete the > - * stored packet. > - */ > - delete_tcp_item(tbl->items, item_idx, > &tbl->item_num, INVALID_ARRAY_INDEX); > + } > + } else { > + /* > + * Any packet with additional flags like PSH,FIN should b= e > processed > + * and flushed immediately. > + * Hence marking the start time to 0, so that the packets > will be flushed > + * immediately in timer mode. > + */ > + if (tcp_hdr->tcp_flags & > (RTE_TCP_ACK_FLAG|RTE_TCP_PSH_FLAG|RTE_TCP_FIN_FLAG)) { > + if (tcp_hdr->tcp_flags !=3D RTE_TCP_ACK_FLAG) > + tbl->items[item_start_idx].start_time =3D= 0; > + return process_tcp_item(pkt, tcp_hdr, tcp_dl, > tbl->items, > + tbl->flows[i].start_index= , > + &tbl->item_num, > tbl->max_item_num, > + ip_id, is_atomic, > start_time); > + } else { > return -1; > } > - return 0; > } > > - return process_tcp_item(pkt, tcp_hdr, tcp_dl, tbl->items, > tbl->flows[i].start_index, > - &tbl->item_num, > tbl->max_item_num, > - ip_id, is_atomic, > start_time); > + return -1; > } > > /* > -- > 2.34.1 > >> Please ignore. > --000000000000ee85d9060f29e155 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: base64 PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PGJyPjwvZGl2Pjxicj48ZGl2IGNsYXNzPSJn bWFpbF9xdW90ZSI+PGRpdiBkaXI9Imx0ciIgY2xhc3M9ImdtYWlsX2F0dHIiPk9uIFRodSwgSmFu IDE4LCAyMDI0IGF0IDE6NDDigK9BTSBLdW1hcmEgUGFyYW1lc2h3YXJhbiAmbHQ7PGEgaHJlZj0i bWFpbHRvOmt1bWFyYXBhcmFtZXNoOTJAZ21haWwuY29tIj5rdW1hcmFwYXJhbWVzaDkyQGdtYWls LmNvbTwvYT4mZ3Q7IHdyb3RlOjxicj48L2Rpdj48YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVv dGUiIHN0eWxlPSJtYXJnaW46MHB4IDBweCAwcHggMC44ZXg7Ym9yZGVyLWxlZnQ6MXB4IHNvbGlk IHJnYigyMDQsMjA0LDIwNCk7cGFkZGluZy1sZWZ0OjFleCI+PGRpdj5JbiB0aGUgY3VycmVudCBp bXBsZW1lbnRhdGlvbiB3aGVuIGEgcGFja2V0IGlzIHJlY2VpdmVkIHdpdGg8YnI+DQpzcGVjaWFs IFRDUCBmbGFnKHMpIHNldCwgb25seSB0aGF0IHBhY2tldCBpcyBkZWxpdmVyZWQgb3V0IG9mIG9y ZGVyLjxicj4NClRoZXJlIGNvdWxkIGJlIGFscmVhZHkgY29hbGVzY2VkIHBhY2tldHMgaW4gdGhl IEdSTyB0YWJsZTxicj4NCmJlbG9uZ2luZyB0byB0aGUgc2FtZSBmbG93IGJ1dCBub3QgZGVsaXZl cmVkLjxicj4NClRoaXMgZml4IG1ha2VzIHN1cmUgdGhhdCB0aGUgZW50aXJlIHNlZ21lbnQgaXMg ZGVsaXZlcmVkIHdpdGggdGhlPGJyPg0Kc3BlY2lhbCBmbGFnKHMpIHNldCB3aGljaCBpcyBob3cg dGhlIExpbnV4IEdSTyBpcyBhbHNvIGltcGxlbWVudGVkPGJyPg0KPGJyPg0KU2lnbmVkLW9mZi1i eTogS3VtYXJhIFBhcmFtZXNod2FyYW4gJmx0OzxhIGhyZWY9Im1haWx0bzprdW1hcmFwYXJhbWVz aDkyQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmt1bWFyYXBhcmFtZXNoOTJAZ21haWwuY29t PC9hPiZndDs8YnI+DQpDby1hdXRob3JlZC1ieTogS3VtYXJhIFBhcmFtZXNod2FyYW4gJmx0Ozxh IGhyZWY9Im1haWx0bzprcmF0aGluYXZlbEBtaWNyb3NvZnQuY29tIiB0YXJnZXQ9Il9ibGFuayI+ a3JhdGhpbmF2ZWxAbWljcm9zb2Z0LmNvbTwvYT4mZ3Q7PGJyPg0KLS0tPGJyPg0KwqAgwqAgwqAg wqAgSWYgdGhlIHJlY2VpdmVkIHBhY2tldCBpcyBub3QgYSBwdXJlIEFDSyBwYWNrZXQsIHdlIGNo ZWNrIGlmPGJyPg0KwqAgwqAgwqAgwqAgdGhlcmUgYXJlIGFueSBwcmV2aW91cyBwYWNrZXRzIGlu IHRoZSBmbG93LCBpZiBwcmVzZW50IHdlIGluZHVsZ2U8YnI+DQrCoCDCoCDCoCDCoCB0aGUgcmVj ZWl2ZWQgcGFja2V0IGFsc28gaW4gdGhlIGNvYWxlc2NpbmcgbG9naWMgYW5kIHVwZGF0ZSB0aGUg ZmxhZ3M8YnI+DQrCoCDCoCDCoCDCoCBvZiB0aGUgbGFzdCByZWNpdmVkIHBhY2tldCB0byB0aGUg ZW50aXJlIHNlZ21lbnQgd2hpY2ggd291bGQgYXZvaWQ8YnI+DQrCoCDCoCDCoCDCoCByZS1vcmRl cmluZy48YnI+DQo8YnI+DQrCoCDCoCDCoCDCoCBMZXRzIHNheSBhIGNhc2Ugd2hlcmUgUDEoUFNI KSwgUDIoQUNLKSwgUDMoQUNLKcKgIGFyZSByZWNlaXZlZCBpbiBidXJzdCBtb2RlLDxicj4NCsKg IMKgIMKgIMKgIFAxIGNvbnRhaW5zIFBTSCBmbGFnIGFuZCBzaW5jZSBpdCBkb2VzIG5vdCBjb250 YWluIGFueSBwcmlvciBwYWNrZXRzIGluIHRoZSBmbG93PGJyPg0KwqAgwqAgwqAgwqAgd2UgY29w eSBpdCB0byB1bnByb2Nlc3NfcGFja2V0cyBhbmQgUDIoQUNLKSBhbmQgUDMoQUNLKSBhcmUgbWVy Z2VkIHRvZ2V0aGVyLjxicj4NCsKgIMKgIMKgIMKgIEluIHRoZSBleGlzdGluZyBjYXNlIHRoZcKg IFAyLFAzIHdvdWxkIGJlIGRlbGl2ZXJlZCBhcyBzaW5nbGUgc2VnbWVudCBmaXJzdCBhbmQgdGhl PGJyPg0KwqAgwqAgwqAgwqAgdW5wcm9jZXNzX3BhY2tldHMgd2lsbCBiZSBjb3BpZWQgbGF0ZXIg d2hpY2ggd2lsbCBjYXVzZSByZW9yZGVyaW5nLiBXaXRoIHRoZSBwYXRjaDxicj4NCsKgIMKgIMKg IMKgIGNvcHkgdGhlIHVucHJvY2VzcyBwYWNrZXRzIGZpcnN0IGFuZCB0aGVuIHRoZSBwYWNrZXRz IGZyb20gdGhlIEdSTyB0YWJsZS48YnI+DQo8YnI+DQrCoCDCoCDCoCDCoCBUZXN0aW5nIGRvbmU8 YnI+DQrCoCDCoCDCoCDCoCBUaGUgY3N1bSB0ZXN0LXBtZCB3YXMgbW9kaWZpZWQgdG8gc3VwcG9y dCB0aGUgZm9sbG93aW5nPGJyPg0KwqAgwqAgwqAgwqAgR0VUIHJlcXVlc3Qgb2YgMTBNQiBmcm9t IGNsaWVudCB0byBzZXJ2ZXIgdmlhIHRlc3QtcG1kIChzdGF0aWMgYXJwIGVudHJpZXMgYWRkZWQg aW4gY2xpZW50PGJyPg0KwqAgwqAgwqAgwqAgYW5kIHNlcnZlcikuIEVuYWJsZSBHUk8gYW5kIFRT TyBpbiB0ZXN0LXBtZCB3aGVyZSB0aGUgcGFja2V0cyByZWNpdmVkIGZyb20gdGhlIGNsaWVudCBt YWM8YnI+DQrCoCDCoCDCoCDCoCB3b3VsZCBiZSBzZW50IHRvIHNlcnZlciBtYWMgYW5kIHZpY2Ug dmVyc2EuPGJyPg0KwqAgwqAgwqAgwqAgSW4gYWJvdmUgdGVzdGluZywgd2l0aG91dCB0aGUgcGF0 Y2ggdGhlIGNsaWVudCBvYnNlcnZlcmQgcmUtb3JkZXJpbmcgb2YgMjUgcGFja2V0czxicj4NCsKg IMKgIMKgIMKgIGFuZCB3aXRoIHRoZSBwYXRjaCB0aGVyZSB3ZXJlIG5vIHBhY2tldCByZS1vcmRl cmluZyBvYnNlcnZlcmQuPGJyPg0KPGJyPg0KdjI6IDxicj4NCsKgIMKgIMKgIMKgIEZpeCB3YXJu aW5ncyBpbiBjb21taXQgYW5kIGNvbW1lbnQuPGJyPg0KwqAgwqAgwqAgwqAgRG8gbm90IGNvbnNp ZGVyIHBhY2tldCBhcyBjYW5kaWRhdGUgdG8gbWVyZ2UgaWYgaXQgY29udGFpbnMgU1lOL1JTVCBm bGFnLjxicj4NCjxicj4NCnYzOjxicj4NCsKgIMKgIMKgIMKgIEZpeCB3YXJuaW5ncy48YnI+DQo8 YnI+DQp2NDo8YnI+DQrCoCDCoCDCoCDCoCBSZWJhc2Ugd2l0aCBtYXN0ZXIuPGJyPg0KPGJyPg0K djU6PGJyPg0KwqAgwqAgwqAgwqAgQWRkaW5nIGNvLWF1dGhvciBlbWFpbDxicj4NCnY2Ojxicj4N CsKgIMKgIMKgIMKgIEFkZHJlc3MgcmV2aWV3IGNvbW1lbnRzIGZyb20gdGhlIG1haW50YWluZXIg dG8gcmVzdHJ1Y3R1cmUgdGhlIGNvZGUgPGJyPg0KwqAgwqAgwqAgwqAgYW5kIGhhbmRsZSBvbmx5 IHNwZWNpYWwgZmxhZ3MgUFNILEZJTjxicj4NCjxicj4NCnY3Ojxicj4NCsKgIMKgIMKgIMKgIEZp eCB3YXJuaW5ncyBhbmQgZXJyb3JzPGJyPg0KPGJyPg0Kdjg6PGJyPg0KwqAgwqAgwqAgwqAgRml4 IHdhcm5pbmdzIGFuZCBlcnJvcnM8YnI+DQo8YnI+DQp2OTo8YnI+DQrCoCDCoCDCoCDCoCBGaXgg Y29tbWl0IG1lc3NhZ2UgPGJyPg0KPGJyPg0KwqBsaWIvZ3JvL2dyb190Y3AuaMKgIHwgMTEgKysr KysrKys8YnI+DQrCoGxpYi9ncm8vZ3JvX3RjcDQuYyB8IDY3ICsrKysrKysrKysrKysrKysrKysr KysrKysrKysrLS0tLS0tLS0tLS0tLS0tLS08YnI+DQrCoDIgZmlsZXMgY2hhbmdlZCwgNTQgaW5z ZXJ0aW9ucygrKSwgMjQgZGVsZXRpb25zKC0pPGJyPg0KPGJyPg0KZGlmZiAtLWdpdCBhL2xpYi9n cm8vZ3JvX3RjcC5oIGIvbGliL2dyby9ncm9fdGNwLmg8YnI+DQppbmRleCBkOTI2YzRiOGNjLi4x MzdhMDNiYzk2IDEwMDY0NDxicj4NCi0tLSBhL2xpYi9ncm8vZ3JvX3RjcC5oPGJyPg0KKysrIGIv bGliL2dyby9ncm9fdGNwLmg8YnI+DQpAQCAtMTg3LDQgKzE4NywxNSBAQCBpc19zYW1lX2NvbW1v bl90Y3Bfa2V5KHN0cnVjdCBjbW5fdGNwX2tleSAqazEsIHN0cnVjdCBjbW5fdGNwX2tleSAqazIp PGJyPg0KwqAgwqAgwqAgwqAgcmV0dXJuICghbWVtY21wKGsxLCBrMiwgc2l6ZW9mKHN0cnVjdCBj bW5fdGNwX2tleSkpKTs8YnI+DQrCoH08YnI+DQo8YnI+DQorc3RhdGljIGlubGluZSB2b2lkPGJy Pg0KK3VwZGF0ZV90Y3BfaGRyX2ZsYWdzKHN0cnVjdCBydGVfdGNwX2hkciAqdGNwX2hkciwgc3Ry dWN0IHJ0ZV9tYnVmICpwa3QpPGJyPg0KK3s8YnI+DQorwqAgwqAgwqAgwqBzdHJ1Y3QgcnRlX3Rj cF9oZHIgKm1lcmdlZF90Y3BfaGRyOzxicj4NCis8YnI+DQorwqAgwqAgwqAgwqBtZXJnZWRfdGNw X2hkciA9IHJ0ZV9wa3RtYnVmX210b2Rfb2Zmc2V0KHBrdCwgc3RydWN0IHJ0ZV90Y3BfaGRyICos IHBrdC0mZ3Q7bDJfbGVuICs8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBwa3QtJmd0 O2wzX2xlbik7PGJyPg0KK8KgIMKgIMKgIMKgbWVyZ2VkX3RjcF9oZHItJmd0O3RjcF9mbGFncyB8 PSB0Y3BfaGRyLSZndDt0Y3BfZmxhZ3M7PGJyPg0KKzxicj4NCit9PGJyPg0KKzxicj4NCsKgI2Vu ZGlmPGJyPg0KZGlmZiAtLWdpdCBhL2xpYi9ncm8vZ3JvX3RjcDQuYyBiL2xpYi9ncm8vZ3JvX3Rj cDQuYzxicj4NCmluZGV4IDY2NDVkZTU5MmIuLjhhZjVhOGQ4YTkgMTAwNjQ0PGJyPg0KLS0tIGEv bGliL2dyby9ncm9fdGNwNC5jPGJyPg0KKysrIGIvbGliL2dyby9ncm9fdGNwNC5jPGJyPg0KQEAg LTEyNiw2ICsxMjYsNyBAQCBncm9fdGNwNF9yZWFzc2VtYmxlKHN0cnVjdCBydGVfbWJ1ZiAqcGt0 LDxicj4NCsKgIMKgIMKgIMKgIHVpbnQzMl90IGl0ZW1faWR4Ozxicj4NCsKgIMKgIMKgIMKgIHVp bnQzMl90IGksIG1heF9mbG93X251bSwgcmVtYWluaW5nX2Zsb3dfbnVtOzxicj4NCsKgIMKgIMKg IMKgIHVpbnQ4X3QgZmluZDs8YnI+DQorwqAgwqAgwqAgwqB1aW50MzJfdCBpdGVtX3N0YXJ0X2lk eDs8YnI+DQo8YnI+DQrCoCDCoCDCoCDCoCAvKjxicj4NCsKgIMKgIMKgIMKgIMKgKiBEb24mIzM5 O3QgcHJvY2VzcyB0aGUgcGFja2V0IHdob3NlIFRDUCBoZWFkZXIgbGVuZ3RoIGlzIGdyZWF0ZXI8 YnI+DQpAQCAtMTM5LDEzICsxNDAsNiBAQCBncm9fdGNwNF9yZWFzc2VtYmxlKHN0cnVjdCBydGVf bWJ1ZiAqcGt0LDxicj4NCsKgIMKgIMKgIMKgIHRjcF9oZHIgPSAoc3RydWN0IHJ0ZV90Y3BfaGRy ICopKChjaGFyICopaXB2NF9oZHIgKyBwa3QtJmd0O2wzX2xlbik7PGJyPg0KwqAgwqAgwqAgwqAg aGRyX2xlbiA9IHBrdC0mZ3Q7bDJfbGVuICsgcGt0LSZndDtsM19sZW4gKyBwa3QtJmd0O2w0X2xl bjs8YnI+DQo8YnI+DQotwqAgwqAgwqAgwqAvKjxicj4NCi3CoCDCoCDCoCDCoCAqIERvbiYjMzk7 dCBwcm9jZXNzIHRoZSBwYWNrZXQgd2hpY2ggaGFzIEZJTiwgU1lOLCBSU1QsIFBTSCwgVVJHLCBF Q0U8YnI+DQotwqAgwqAgwqAgwqAgKiBvciBDV1Igc2V0Ljxicj4NCi3CoCDCoCDCoCDCoCAqLzxi cj4NCi3CoCDCoCDCoCDCoGlmICh0Y3BfaGRyLSZndDt0Y3BfZmxhZ3MgIT0gUlRFX1RDUF9BQ0tf RkxBRyk8YnI+DQotwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqByZXR1cm4gLTE7PGJyPg0KLTxicj4N CsKgIMKgIMKgIMKgIC8qIHRyaW0gdGhlIHRhaWwgcGFkZGluZyBieXRlcyAqLzxicj4NCsKgIMKg IMKgIMKgIGlwX3RsZW4gPSBydGVfYmVfdG9fY3B1XzE2KGlwdjRfaGRyLSZndDt0b3RhbF9sZW5n dGgpOzxicj4NCsKgIMKgIMKgIMKgIGlmIChwa3QtJmd0O3BrdF9sZW4gJmd0OyAodWludDMyX3Qp KGlwX3RsZW4gKyBwa3QtJmd0O2wyX2xlbikpPGJyPg0KQEAgLTE4Myw2ICsxNzcsNyBAQCBncm9f dGNwNF9yZWFzc2VtYmxlKHN0cnVjdCBydGVfbWJ1ZiAqcGt0LDxicj4NCsKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIGlmICh0YmwtJmd0O2Zsb3dzW2ldLnN0YXJ0X2luZGV4ICE9IElOVkFMSURfQVJS QVlfSU5ERVgpIHs8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBpZiAo aXNfc2FtZV90Y3A0X2Zsb3codGJsLSZndDtmbG93c1tpXS5rZXksIGtleSkpIHs8YnI+DQrCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBmaW5kID0gMTs8YnI+ DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpdGVtX3N0 YXJ0X2lkeCA9IHRibC0mZ3Q7Zmxvd3NbaV0uc3RhcnRfaW5kZXg7PGJyPg0KwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgYnJlYWs7PGJyPg0KwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgfTxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIHJlbWFpbmluZ19mbG93X251bS0tOzxicj4NCkBAIC0xOTAsMjggKzE4NSw1 MiBAQCBncm9fdGNwNF9yZWFzc2VtYmxlKHN0cnVjdCBydGVfbWJ1ZiAqcGt0LDxicj4NCsKgIMKg IMKgIMKgIH08YnI+DQo8YnI+DQrCoCDCoCDCoCDCoCBpZiAoZmluZCA9PSAwKSB7PGJyPg0KLcKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgc2VudF9zZXEgPSBydGVfYmVfdG9fY3B1XzMyKHRjcF9oZHIt Jmd0O3NlbnRfc2VxKTs8YnI+DQotwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpdGVtX2lkeCA9IGlu c2VydF9uZXdfdGNwX2l0ZW0ocGt0LCB0YmwtJmd0O2l0ZW1zLCAmYW1wO3RibC0mZ3Q7aXRlbV9u dW0sPGJyPg0KLcKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgdGJsLSZndDttYXhfaXRlbV9udW0sIHN0YXJ0X3RpbWUs PGJyPg0KLcKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgSU5WQUxJRF9BUlJBWV9JTkRFWCwgc2VudF9zZXEsIGlwX2lk LDxicj4NCi3CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoGlzX2F0b21pYyk7PGJyPg0KLcKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgaWYgKGl0ZW1faWR4ID09IElOVkFMSURfQVJSQVlfSU5ERVgpPGJyPg0KK8KgIMKgIMKg IMKgIMKgIMKgIMKgIMKgLyo8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgKiBBZGQgbmV3 IGZsb3cgdG8gdGhlIHRhYmxlIG9ubHkgaWYgY29udGFpbnMgQUNLIGZsYWcgd2l0aCBkYXRhLjxi cj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAqIERvIG5vdCBhZGQgYW55IHBhY2tldHMgd2l0 aCBhZGRpdGlvbmFsIHRjcCBmbGFncyB0byB0aGUgR1JPIHRhYmxlPGJyPg0KK8KgIMKgIMKgIMKg IMKgIMKgIMKgIMKgICovPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgaWYgKHRjcF9oZHIt Jmd0O3RjcF9mbGFncyA9PSBSVEVfVENQX0FDS19GTEFHKSB7PGJyPg0KK8KgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgc2VudF9zZXEgPSBydGVfYmVfdG9fY3B1XzMyKHRjcF9oZHIt Jmd0O3NlbnRfc2VxKTs8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBp dGVtX2lkeCA9IGluc2VydF9uZXdfdGNwX2l0ZW0ocGt0LCB0YmwtJmd0O2l0ZW1zLCAmYW1wO3Ri bC0mZ3Q7aXRlbV9udW0sPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgdGJsLSZndDtt YXhfaXRlbV9udW0sIHN0YXJ0X3RpbWUsPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg SU5WQUxJRF9BUlJBWV9JTkRFWCwgc2VudF9zZXEsIGlwX2lkLDxicj4NCivCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoGlzX2F0b21pYyk7PGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgaWYgKGl0ZW1faWR4ID09IElOVkFMSURfQVJSQVlfSU5ERVgpPGJyPg0KK8KgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgcmV0dXJuIC0xOzxicj4N CivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoGlmIChpbnNlcnRfbmV3X2Zsb3co dGJsLCAmYW1wO2tleSwgaXRlbV9pZHgpID09PGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgSU5WQUxJRF9BUlJBWV9JTkRFWCkgezxicj4NCivCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoC8qPGJyPg0KK8KgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgICogRmFpbCB0byBpbnNl cnQgYSBuZXcgZmxvdywgc28gZGVsZXRlIHRoZTxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAqIHN0b3JlZCBwYWNrZXQuPGJyPg0KK8KgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgICovPGJyPg0KK8KgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgZGVsZXRlX3RjcF9pdGVtKHRi bC0mZ3Q7aXRlbXMsIGl0ZW1faWR4LCAmYW1wO3RibC0mZ3Q7aXRlbV9udW0sPGJyPg0KK8KgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgSU5W QUxJRF9BUlJBWV9JTkRFWCk7PGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgcmV0dXJuIC0xOzxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoH08YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqByZXR1 cm4gMDs8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqB9IGVsc2Ugezxicj4NCsKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIHJldHVybiAtMTs8YnI+DQotwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqBpZiAoaW5zZXJ0X25ld19mbG93KHRibCwgJmFtcDtrZXksIGl0ZW1faWR4KSA9 PTxicj4NCi3CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoElOVkFMSURfQVJSQVlf SU5ERVgpIHs8YnI+DQotwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAvKjxicj4N Ci3CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAqIEZhaWwgdG8gaW5zZXJ0IGEg bmV3IGZsb3csIHNvIGRlbGV0ZSB0aGU8YnI+DQotwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgKiBzdG9yZWQgcGFja2V0Ljxicj4NCi3CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCovPGJyPg0KLcKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgZGVs ZXRlX3RjcF9pdGVtKHRibC0mZ3Q7aXRlbXMsIGl0ZW1faWR4LCAmYW1wO3RibC0mZ3Q7aXRlbV9u dW0sIElOVkFMSURfQVJSQVlfSU5ERVgpOzxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoH08 YnI+DQorwqAgwqAgwqAgwqB9IGVsc2Ugezxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoC8q PGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgICogQW55IHBhY2tldCB3aXRoIGFkZGl0aW9u YWwgZmxhZ3MgbGlrZSBQU0gsRklOIHNob3VsZCBiZSBwcm9jZXNzZWQ8YnI+DQorwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgKiBhbmQgZmx1c2hlZCBpbW1lZGlhdGVseS48YnI+DQorwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgKiBIZW5jZSBtYXJraW5nIHRoZSBzdGFydCB0aW1lIHRvIDAsIHNvIHRo YXQgdGhlIHBhY2tldHMgd2lsbCBiZSBmbHVzaGVkPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKg IMKgICogaW1tZWRpYXRlbHkgaW4gdGltZXIgbW9kZS48YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgKi88YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpZiAodGNwX2hkci0mZ3Q7dGNw X2ZsYWdzICZhbXA7IChSVEVfVENQX0FDS19GTEFHfFJURV9UQ1BfUFNIX0ZMQUd8UlRFX1RDUF9G SU5fRkxBRykpIHs8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpZiAo dGNwX2hkci0mZ3Q7dGNwX2ZsYWdzICE9IFJURV9UQ1BfQUNLX0ZMQUcpPGJyPg0KK8KgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgdGJsLSZndDtpdGVtc1tpdGVt X3N0YXJ0X2lkeF0uc3RhcnRfdGltZSA9IDA7PGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgcmV0dXJuIHByb2Nlc3NfdGNwX2l0ZW0ocGt0LCB0Y3BfaGRyLCB0Y3BfZGws IHRibC0mZ3Q7aXRlbXMsPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgdGJsLSZndDtmbG93c1tpXS5zdGFy dF9pbmRleCw8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAmYW1wO3RibC0mZ3Q7aXRlbV9udW0sIHRibC0m Z3Q7bWF4X2l0ZW1fbnVtLDxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoGlwX2lkLCBpc19hdG9taWMsIHN0 YXJ0X3RpbWUpOzxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoH0gZWxzZSB7PGJyPg0KwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgcmV0dXJuIC0xOzxicj4NCsKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIH08YnI+DQotwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqByZXR1cm4gMDs8 YnI+DQrCoCDCoCDCoCDCoCB9PGJyPg0KPGJyPg0KLcKgIMKgIMKgIMKgcmV0dXJuIHByb2Nlc3Nf dGNwX2l0ZW0ocGt0LCB0Y3BfaGRyLCB0Y3BfZGwsIHRibC0mZ3Q7aXRlbXMsIHRibC0mZ3Q7Zmxv d3NbaV0uc3RhcnRfaW5kZXgsPGJyPg0KLcKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgJmFtcDt0YmwtJmd0O2l0ZW1f bnVtLCB0YmwtJmd0O21heF9pdGVtX251bSw8YnI+DQotwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpcF9pZCwgaXNf YXRvbWljLCBzdGFydF90aW1lKTs8YnI+DQorwqAgwqAgwqAgwqByZXR1cm4gLTE7PGJyPg0KwqB9 PGJyPg0KPGJyPg0KwqAvKjxicj4NCi0tIDxicj4NCjIuMzQuMTxicj48L2Rpdj4NCiZndDsmZ3Q7 IFBsZWFzZSBpZ25vcmUuIDxicj4NCjwvYmxvY2txdW90ZT48L2Rpdj48L2Rpdj4NCg== --000000000000ee85d9060f29e155--