From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1BA53A09ED for ; Sat, 12 Nov 2022 18:46:52 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9BF6440041; Sat, 12 Nov 2022 18:46:51 +0100 (CET) Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by mails.dpdk.org (Postfix) with ESMTP id B56E94003F for ; Sat, 12 Nov 2022 18:46:50 +0100 (CET) Received: by mail-pg1-f172.google.com with SMTP id b62so6872499pgc.0 for ; Sat, 12 Nov 2022 09:46:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=zXy2Rsk4DMtBXzdGApSmEasjexGyt4uIMpc7Cl8wO6s=; b=hBymE+KOO0DgwjoybTix2YbA9di1WP1jYu2QbOgMJi5ITpZE27cnefYsw+q9TM94Pc ABYnHBaIwyehWRsPFqqt+lV5YQh1iIf+iSTqaLSZ3Pw5nCijwa7Jed7Mj6w6KVCdvdxt Gr7VB8WxRKWDwtxmrKT2vQi7sjAzfmt7qJ4MJQtg93Eqr3NPOd6dFiguMwVoxWNREI2l YVEFuWac7YuEv7nTZ9MPJHLKdtjpRovqLRDhtgz+Lwb8g+46uWTXO4CjucSR72fHe2pI fSArAjgxxtL1fD0w0VtPruSsqHHgFMeJKLuY0iq0Rc0xEm7f09oFNB5tWF4WDS61rAef dYuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zXy2Rsk4DMtBXzdGApSmEasjexGyt4uIMpc7Cl8wO6s=; b=exSflmSzxXFAgp5ZO0HktTYsPFThh7AuR5NKc/NlilxAV23XTm666EN1fv+cNGQzoO akbS7BCynO44b4t8xzIiTt7qhuXNfn8aK+hKpkCNdxn6XJphLmxgyx1ueWoSMWgYeBKc PwCoRO9J2YoLb+oiPP8l7EHWb9tuS1Du7QgyD4a/gJkbyms302aChTsh76xooYhUlTpj tBNNLciEO13EenWhvgGlPRTX31ZOWNoBhrfXgQSxXHtrnPCXAJARv8LAX1Gy13scWOSA vOxD6z2vZnDRCLVhSe+nE4yjOUzjUMx9gB93hSRPUSgGAuc5pV++gnpkB0DLA/Hhc09z 5tqQ== X-Gm-Message-State: ANoB5pkzrpdm8ZWxxuzZD/3ZmJwJiMHEorbkYo1py/7FRWP+R286veZX y7h2htHy22Q3fI7FOtm6uKe6VrwmtPG+J6z4GG4= X-Google-Smtp-Source: AA0mqf4npZ+FkT6nE20wwnc5jNOs7WeKfjCVGha85wiDTfWHfexcj+KfGIZk0sjRs5CgOhC5zHtKeMbjZ7fr28sIdXA= X-Received: by 2002:a05:6a00:88c:b0:56b:e1d8:e7a1 with SMTP id q12-20020a056a00088c00b0056be1d8e7a1mr7685405pfj.28.1668275209499; Sat, 12 Nov 2022 09:46:49 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Boris Ouretskey Date: Sat, 12 Nov 2022 19:46:37 +0200 Message-ID: Subject: Re: DPDK server packets with AF_PACKET driver are not received by the client connected to the same virtual bridge. To: "J.J. Mars" Cc: users@dpdk.org Content-Type: multipart/mixed; boundary="0000000000004029fc05ed49973f" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000004029fc05ed49973f Content-Type: multipart/alternative; boundary="0000000000004029fa05ed49973d" --0000000000004029fa05ed49973d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi I have narrowed down the problem of sending packets via the AF_PACKET/SOCK_RAW interface, which might be what DPDK does in coressonding PMD drivers. The problem does not seem to be related to DPDK codebase directly, but still can be interesting for the users who are trying to use DPDK with AF_PACKET driver inside a dockerized environment. To reiterate the issue: Any UDP message sent via AF_PACKET/SOCK_RAW socket on one interface on virtual bridge, is not received by application listening on the other interface attached to the same bridge, however strangely enough it can be seen in tcpdump listening on the receiving interface. The application I used based on the example here [ https://www.kernel.org/doc/html/latest/_sources/networking/packet_mmap.rst.= txt] but instead of receiving I added a loop for sending 5 pre-recorded L2 frames. I am attaching the small project to easily reproduce the issue (assuming the MAC addresses are preserved between machines by the docker because the L2 packet is pre-recorded) . ------ The environment is two dockers (basic AlmaOS 8.5 having only network utils installed) attached to the same bridge. The environment is set up with docker-compose. To reproduce the issue: - make dockers and applications using Makefile and start the dockers. *make docker-compose up* - in other window set up server in first docker: * docker exec -it client /bin/bash nc -l -4 -u 192.168.2.1 5997* - in other window set up tcpdump to see the incoming packets on 5997 port: * docker exec -it client /bin/bash tcpdump -vneA -i eth0 port 5997* - in other window send 5 messages via application then via netcat: * docker exec -it server /bin/bash /share/a.out eth0 hash printf "ok1\r\n" | nc -s 192.168.2.5 -p 5998 -u -4 192.168.2.1 5997* Observe that only packets sent by nc are received. Though there's no difference in L2 frame structure apart from identification and checksum (wrong for both cases). The question is how can I debug the kernel to understand what makes it drop the packet and not to pass it to the application (netcat in this case). On Wed, Nov 9, 2022 at 10:50 AM J.J. Mars wrote: > If the nic on docker 1 of echo client can receive replies, the reason > should not be PMD I think. > Did you make sure every field in every layer is completely the same > between eth0 in docker 1 and eth0 in docker 2, and all of them are expect= ed? > > Boris Ouretskey =E4=BA=8E2022=E5=B9=B411=E6=9C=889= =E6=97=A5=E5=91=A8=E4=B8=89 03:51=E5=86=99=E9=81=93=EF=BC=9A > >> >> >> Hi >> >> I am trying to run a docker based system in one VM, with some servers >> using the DPDK and facing a very strange issue. >> >> The relevant part of the system is the echo server which is built with >> DPDK library and runs in docker, connected to the virtual bridge (with >> AF_PACKET driver), and echo client built with regular sockets (java >> application). See the drawing below for visual details. >> >> The strange thing is that echo replies sent from DPDK echo server docker >> (Docker2), can be seen in tcpdump in the docker of echo client (Docker1) >> however the application (java client) does not receive it on socket leve= l. >> When the echo server runs as a socket application everything works. Seam= s >> like OS (linux) decide not to route the packet to the application for >> some reason. >> >> Some quick important notes. >> >> - DPDK layer starts ok and no EAL errors are reported. Its echo replies >> as mentioned before can clearly be seen when capturing traffic on eth0 i= n >> client docker. >> - It is not the issue of parsing the echo replies on application level. = I >> checked the strace logs and cannot see any traffic delivered to the >> application >> - I have compared the captures from working socket based echo server and >> MACs seam to be ok, also the packets sent from DPDK server are formed we= ll. >> - No logs for martian packets when enabled. >> >> I would like to focus the attention of dear DPDK experts on the followin= g >> question: is it possible that DPDK AF_PACKET driver intercepts both pack= ets >> destined to the virtual nic eth0 of Docker1 AND eth0 of Docker2, >> preventing it from being received by application? >> >> DPDK Version 18 >> >> VIRTUAL BOX VM (Alma Linux) >> >> Docker1 Docker2 >> +---------------+ +------------+ >> | | Request | | >> | | --------------> | DPDK | >> | Socket Echo | | Echo Server >> | Client | Reply | Server | >> | | <------------- | | >> --vdev=3Deth_af_packet0,iface=3Deth0,qdisc_bypass=3D1 >> | | | | >> | tcpdump -i eth0 +------+-----+ >> +-------+-------+ eth0 | >> | eth0 | >> | No IP | >> +------v--------------------------------------v-----+ >> | Virtual Docker Bridge | >> +---------------------------------------------------+ >> >> >> Thanks >> > --0000000000004029fa05ed49973d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi

I have narrowed down the problem of sending pack= ets via the AF_PACKET/SOCK_RAW interface, which might be what DPDK does in = coressonding PMD drivers. The problem does not seem to be related to DPDK c= odebase directly, but still can be interesting for the users who are trying= to use DPDK with AF_PACKET driver inside a dockerized environment.

= To reiterate the issue: Any UDP message sent via AF_PACKET/SOCK_RAW socket = on one interface on virtual bridge, is not received by application listenin= g on the other interface attached to the same bridge, however strangely eno= ugh it can be seen in tcpdump listening on the receiving interface. The app= lication I used based on the example here [https://www.kern= el.org/doc/html/latest/_sources/networking/packet_mmap.rst.txt] but ins= tead of receiving I added a loop for sending 5 pre-recorded L2 frames.
<= br>I am attaching the small project to easily reproduce the issue (assuming= the MAC addresses are preserved between machines by the docker because the= L2 packet is pre-recorded) .
------
The environment is two dockers (= basic AlmaOS 8.5 having only network utils installed) attached to the same = bridge. The environment is set up with docker-compose.

To reproduce = the issue:

- make dockers and applications using Makefile and start = the dockers.
make
docker-compose up

- in other window = set up server in first docker:
docker exec -it client /bin/bash
= nc -l -4 -u 192.168.2.1 5997

- in other window set up tcpdump to= see the incoming packets on 5997 port:
docker exec -it client /bin/= bash
tcpdump -vneA -i eth0 port 5997

- in other window send = =C2=A05 messages =C2=A0via application then via netcat:
docker exec= -it server /bin/bash
/share/a.out eth0 hash
printf "ok1\r\n&q= uot; | nc -s 192.168.2.5 -p 5998 -u -4 192.168.2.1 5997

Observe = that only packets sent by nc are received. Though there's no difference= in L2 frame structure apart from identification and checksum (wrong for bo= th cases).

The question is how can I debug the kernel to understand= what makes it drop the packet and not to pass it to the application (netca= t in this case).=C2=A0

On Wed, Nov 9, 2022 at 10:50 AM J.J. Mars <mars14850@gmail.com> wrote:
<= /div>
If the nic on docker 1 of echo client can receive replies, the reason sho= uld not be PMD I think.
Did you make sure every field in every la= yer is completely the same between eth0 in docker 1 and eth0 in docker 2, a= nd all of them are expected?

=
Boris Ouretskey <borisusun@gmail.com> =E4=BA= =8E2022=E5=B9=B411=E6=9C=889=E6=97=A5=E5=91=A8=E4=B8=89 03:51=E5=86=99=E9= =81=93=EF=BC=9A
=


Hi
I am trying to run a docker based system in one VM, with some servers= using the DPDK and facing a very strange issue.

The relevant part o= f the system is the echo server which is built with DPDK library and runs i= n docker, connected to the virtual bridge (with AF_PACKET driver), and echo= client built with regular sockets (java application). See the drawing belo= w for visual details.

The strange thing is that echo replies sent fr= om DPDK echo server docker (Docker2), can be seen in tcpdump in the docker = of echo client (Docker1) however the application (java client) does not rec= eive it on socket level. When the echo server runs as a socket application = everything works. Seams like OS (linux) decide not to route the packet to t= he application for some=C2=A0reason.

Some quick important notes.
- DPDK layer starts ok and no EAL errors are reported. Its echo=C2=A0r= eplies as mentioned before can clearly be seen when capturing traffic on et= h0 in client docker.
- It is not the is= sue of parsing the echo replies on application level. I checked the strace = logs and cannot see any traffic delivered to the application
- I have co= mpared the captures from working socket based echo server and MACs seam to = be ok, also the packets sent from DPDK server are formed well.
- No logs= for martian packets when enabled.

I would like to focus the attenti= on of dear DPDK experts on the following question: is it possible that DPDK= AF_PACKET driver intercepts both packets destined to =C2=A0the virtual nic= eth0 of Docker1 AND eth0 of Docker2, preventing it from being received by = application?

DPDK Version 18

VIRTUAL BOX VM (Alma Linux)
<= br>=C2=A0 =C2=A0 Docker1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Docker2
= +---------------+ =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 +------------+
| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 | =C2=A0 =C2=A0Request =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|
| =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 -------------->=C2=A0 =C2=A0 = =C2=A0| =C2=A0DPDK =C2=A0 =C2=A0 =C2=A0|
| Socket Echo =C2=A0 | =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2= =A0Echo Server
| =C2=A0 Client =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Reply =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0Server =C2=A0 =C2=A0|<= br>| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 <--------= ----- =C2=A0 =C2=A0 =C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| --vd= ev=3Deth_af_packet0,iface=3Deth0,qdisc_bypass=3D1
| =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0|
| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tcpdump -i eth0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0+------+-----+
+-------+-------+ = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 eth0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 | eth0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 |
=C2=A0 =C2=A0 =C2=A0 =C2=A0 | =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 No IP =C2=A0|
=C2=A0+------v------------------------------= --------v-----+
=C2=A0| =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Virtua= l Docker Bridge =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 |=C2=A0+---------------------------------------------------+
=C2=A0
Thanks
--0000000000004029fa05ed49973d-- --0000000000004029fc05ed49973f Content-Type: application/x-gzip; name="test.tar.gz" Content-Disposition: attachment; filename="test.tar.gz" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_lae7yij60 H4sIAAPab2MAA+1ae2/bRhLPvyKQ7zBxmkByJYqUKMuR4+IUPxqjfgROgrbIBQJFLqWtVyRDruz4 Un/3zu6SEqmHox7OKYLbXxCKmp2Zndl57JIyJylvPnpYWIhutyM+7W7HKn7meGQ77Xa761iO4zyy bLvrdB9B54Htkpim3E0AHnnRKLzlZC3f18a/U3AR/8PIuyJJQNnDuCcCvLPjrI1/p+3k8W/brRbG v91tY/ytB7FmAf/n8T++vDgDl01cRsPp596u2XlsPDYu35+DHwbQuAUa4gIxBpggIWELg9N4lLg+ Ac9teCThNKCeixkFa3SQmLBGQhhxU1JkmbhXxHO9MYEGDgcJScdrNPzxCa7pBLxpwoClUQA8IQRu RoRDnE5o6kHKE9cjEE7cuBGiNRAS3uBRxFKAkT+Ea5eNEhr6wOhwPEVBNgzSxpRTZOBe7E8nsfhM SMxcnDdOoiknMEyoPyIZG425O2ToJhmqG2HswdkhNIc0bA5dZf1j45+O7SaQ9X+G6/9g1b9B/dvt Wf/vyP7fdixd/98E5pvXF+e/94BO3BEBkd+GgYXWK36X9z2j4sttAoZTynwwsVaDKPFII5lAQ5YZ vwXDECLIO/I8aPwqSrZx0cICp6GJhAjMZjp2E9J0Tawrw/CwF4TI3sh0pzyKAX74oZp9j1NouND4 JCajjCPFDT1M2SjZlxPW5qJoBvzXorQkKh1OIWcTDibBgumVgqxk/C7KfQmy/lV4HmwOWf/Ouvq3 rXbbzuq/Yztq/7eQpOv/G+ApDT02xS38Zcp9nwTm+CejRMNtcolGo0US7qijBdpt2uS3MUmXyTcu 5cvUVNTTCjqNPM7K5GlI0YoyTZ5fmjQYED4myZqx2F2eA4sXhwTNMJrbcJy4EwJVpwtDDHdag+2m gSnCqQdehKcQmIYpHYXEBw+7AcRXvP3B6X6EffhiWJ+tVh2sz468epa4urviquiWPb8XPDid+coU eCWmWSnSkVdFkaNOJ78X4hJHphJXDC05y3AoVcl7xyqIy6ttK/Enpvkv/KfEgxdiyG+vMb5Tps9m N/PZCz7aXXHdIYV7f26D5efioTnJxXel/q687gTyKl1o27kIXl3IIMSn0ZWtpO/2DAxSMvU4iDRy fT8Z4NajUmogvpI0RZ6nmOE0JHD2++Dw6O27wVn/wBLahPGrBm016KwcbKlBz1o12FaD7u6qQee+ OTvZoG2U806m27ZPrqlHBiEm6V4+TkMOgRvirjQQFbeKTv09TPkgxOngTf/gl6N3g+P++cX7d8ZT yEwokSuVir27Zmzwuv/2daVirRs+fYXCxlMS+jQompISPo0HKibV64j6NeOLIZwVgyRJ6hD4WEYZ Q/94oNTW4e3FwS+Dy/6vdRhzXIrq0bvXgzeD/ulprbYnFawIPWOlIYoPFZ/EdW82Y7YybjLKaAFU 0YCXYNWwlCuCFKNVUVLdUiZt4WySnKAjSQhHv50In09O318eKRV38johE3S1+hxnw5ytQ0r/Q6Kg il8L9nrxraCY+F8Gsw6FyGZsODuuh+x/aBmuw8nFwc8nxyfnh0e/1UHor81NF8yrbC8K/V0PGCs6 wFhuP2NmytggcCeU3aKJs1iVhvFpEPPts3AhczSjFL3DJyZfOlddCCJs10AasDT7fd4KdZt62dye pwBaUp2VCvw5+yLqCV6+BHunVgoKLo8wNIq5iszF6SxbS8VQNyrSlOfzqWYezUkLni3n32y2DXzD dmgYlTyE5RZYDGd5RJhQwaZ6IkMmHqzxaRy3xpsoucpSUzTaSlnqa3EWGvuKFRgJR3y8RsfYxWHU IEq7f3p0vreSa1XCreLDh3YeeRGDTKNsFtj+hTmHeOClITalKATstuucEvcfLLGtF/eK1dNJXnuB 176Ht7XA27qHt73A276H11ngde7h7SzwdlT6GIXkCnDTuMs7uGjYs11mnBDXX+rhqn0Xu3xt3m4Z nVCO47Y48y91XJXU5DMV9ZRJ3Ywpw5OYFGw04KdCucu9cDgNPtg7lvUxK4lsG8m+qTqVdooKReZZ 5uN9La+jxWZSqjshLStubl2x4HIdqp0IBDEehDnWFvdxneqw9XPEQR065eS96jO/Bv8Ot9baotyF Z7hONdjfl0tTWal6rmpEeEz9at5FlDlL/M/8HlwSj9BrPLvKKGSWpauUeCxKyTwW0nerJvLBKGVE itv8Uj5UNs6GzlczoVIxKvdkAlb0WzQhX2RRzRWhTdjFIxl6cUSfrbf4UpNtcHHD2a4tNcu8Uy4d L2o1aetivqhJN8oY5MgCJKTyJMFISOGUERJXbbzPIlq5Jx5iPcUTfFXc4G7i1bOz4jZ+icsFWlcV khPofPmFIDzZh/ZsaQsZJE9nW+9T8R4InqVwcv7u6PK4f3AEX8ZuOv6TDe9kEokJsWtuuAGLaZ+I s9AkrkrJ1kecRSjcqmV5UNyG91ccQ7PVYClZo40NN9J1+qqgaa3/4VUY3YSZIpCKPjxLP85db81c XxX6ueeFsx6aIkXtj4phfgrZn9UkPMcHggCRsUQJVCmOW3tAMRFt/Pjxx/mpgWJFiityIOtVNbcp vaHcG0MVh+ad1EWHrV6WsM1mucPLZKwUazzXJeUadm+hAsR8G3fMu5XeOGVv5JMDNpxpmsmKtwd4 rpGk0qJm2Yabyx1++6dfrHwnkO//1KvMhhdNYuwy5q07Yf/LOb7y/t9ud/Pffztt20a63ek4ln7/ 9y1wTZIUj8I92GqbL7ZwbyeJaE1pT+w8HqO4N/VUs5Y/AmSvumUHiEKOuw5Rz6+9jFkOYeO8xh17 RPDIgXsnkcRxlPIlTlRWYPHcWGy+PXUKhQacY4fuH56dnOcEPMfL2+uITSfCSsgGspfzPfUhydnT y4zJJwzPPsntAAdyGvoVXzv5jt8D+0XLtHd2TbyKBRCrQRLJvOj/ovuK1Vjj/dz5OV/J941dL3u+ yvEFv1e6vc7rjmIwijqWFfiJIPSyX0ULpAE+os7WG9uJqTqLmakzlYCpVkIu5aCgg8buZBasbAqf BO6U8ZnKMKCjefAakE6HwqyCC1az5cwYAEYuJzfubZGj1XG+zx9rHgCy/6vftR5sjs3//qfjdLu2 /P1nx9Z//6OhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhoaGhobGIvwAGCAHU AFAAAA== --0000000000004029fc05ed49973f--