From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9AE2241DE6; Mon, 6 Mar 2023 12:17:55 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7BD1840E50; Mon, 6 Mar 2023 12:17:55 +0100 (CET) Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by mails.dpdk.org (Postfix) with ESMTP id DFDC940A8A for ; Mon, 6 Mar 2023 12:17:53 +0100 (CET) Received: by mail-pj1-f46.google.com with SMTP id m20-20020a17090ab79400b00239d8e182efso12787783pjr.5 for ; Mon, 06 Mar 2023 03:17:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1678101473; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=MkmY6EXVf2Z4y260JuuGpsTJy/5ci+QDfoO+3viMq0E=; b=DVGpBPhlISK/LHle/fe34fSVtcXrD5cI8SDBrDKRgmj1E03LyRP+vomv3JGnr0UhFF JBOlJD49lvs8ndXbhaIfBSELYSQUds8dobuPOpjIjc9164A+7qXGjfcWj/D9Zj9gZ0mz EXDwaE1Gw9LIQo0YJsKESwGej0eAX0iwlCCqg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678101473; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=MkmY6EXVf2Z4y260JuuGpsTJy/5ci+QDfoO+3viMq0E=; b=LUhu/+wu9vnqDKU7CLh5QOlqCI7g2ABySytIh0VopiQ+yiY8//rChM+wot/PEqztOI Eyj7bWENxOuxc8kUbmw1dFly+rfz9I0lJAG4QaqxQplL4SxQiYHkcYOiR2XmJdMQj0LN X0M8SbyKDJO5lPDRkqb5/m1dJufDqd23vEPJ80m+MR8EsR4cr379yKOtXs5TnRqhr+3M /xdnMqKCT6apaRRQoFyHYH++HiNMLw92u2j0Tlw8s8VomL3S/JUtEJEWl2jH1ZuRke49 nBSaybpIdd9lUErnpuik3HerwZNOEcgcTbs0GAUG8gbF3iW6M34B9dCp604RVvqyDBkL J7dw== X-Gm-Message-State: AO0yUKV1uc4VS7uYLlDLQrl/JPnbcQ5qyCKa4Ssfm1SFtC717s8dTP/7 xIRfp06ne7479Uq/Eq2kbMzakHlkuM13I1eiWgcv4A== X-Google-Smtp-Source: AK7set+KaybvGOZOLlXsRUAyprUdfc59tne6fwY/ntCfsj4hf6iG69jOyGh05j4ZJM4AdYP9ymoV3CrtBeQe6+WU9ko= X-Received: by 2002:a17:902:ab45:b0:19b:e4c:2039 with SMTP id ij5-20020a170902ab4500b0019b0e4c2039mr4320100plb.4.1678101472987; Mon, 06 Mar 2023 03:17:52 -0800 (PST) MIME-Version: 1.0 References: <20230220060839.1267349-1-ashok.k.kaladi@intel.com> <20230220060839.1267349-2-ashok.k.kaladi@intel.com> <4786db4b-63dc-5329-522d-77eb58d4cff4@huawei.com> <20230221090053.14d653bf@hermes.local> <3cd97a71-b32f-b33b-dce1-46fabad182f6@huawei.com> <54fbf4e55cd44477b1e956f98a7a3c50@huawei.com> <3dc3b3d3-c80f-a361-8780-b1b3e48d843e@yandex.ru> <52296fe2-d9f6-1a24-e577-e5271a69a053@amd.com> <5cbf53cb272d42b994eb12b337466986@huawei.com> In-Reply-To: <5cbf53cb272d42b994eb12b337466986@huawei.com> From: Ajit Khaparde Date: Mon, 6 Mar 2023 03:17:36 -0800 Message-ID: Subject: Re: [PATCH 2/2] ethdev: fix race condition in fast-path ops setup To: Konstantin Ananyev Cc: Ruifeng Wang , Ferruh Yigit , Konstantin Ananyev , "dev@dpdk.org" , Fengchengwen , Honnappa Nagarahalli , Stephen Hemminger , nd Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="00000000000037b24105f63972e3" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --00000000000037b24105f63972e3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Mar 6, 2023 at 2:33=E2=80=AFAM Konstantin Ananyev wrote: > > > > > > -----Original Message----- > > > From: Ferruh Yigit > > > Sent: Saturday, March 4, 2023 1:19 AM > > > To: Konstantin Ananyev ; dev@dpdk.org= ; fengchengwen > > > ; Konstantin Ananyev ; Honnappa > > > Nagarahalli ; Stephen Hemminger ; > > > Ruifeng Wang ; Ajit Khaparde (ajit.khaparde@bro= adcom.com) > > > > > > Subject: Re: [PATCH 2/2] ethdev: fix race condition in fast-path ops = setup > > > > > > On 2/26/2023 5:22 PM, Konstantin Ananyev wrote: > > > > > > > >>>>>>>>>>> If ethdev enqueue or dequeue function is called during > > > >>>>>>>>>>> eth_dev_fp_ops_setup(), it may get pre-empted after setti= ng > > > >>>>>>>>>>> the function pointers, but before setting the pointer to = port data. > > > >>>>>>>>>>> In this case the newly registered enqueue/dequeue functio= n > > > >>>>>>>>>>> will use dummy port data and end up in seg fault. > > > >>>>>>>>>>> > > > >>>>>>>>>>> This patch moves the updation of each data pointers befor= e > > > >>>>>>>>>>> updating corresponding function pointers. > > > >>>>>>>>>>> > > > >>>>>>>>>>> Fixes: c87d435a4d79 ("ethdev: copy fast-path API into > > > >>>>>>>>>>> separate > > > >>>>>>>>>>> structure") > > > >>>>>>>>>>> Cc: stable@dpdk.org > > > >>>>>>>> > > > >>>>>>>> Why is something calling enqueue/dequeue when device is not > > > >>>>>>>> fully > > > >>>>>> started. > > > >>>>>>>> A correctly written application would not call rx/tx burst > > > >>>>>>>> until after ethdev start had finished. > > > >>>>>>> > > > >>>>>>> Please refer the eb0d471a894 (ethdev: add proactive error > > > >>>>>>> handling mode), when driver recover itself, the application m= ay > > > >>>>>>> still invoke > > > >>>>>> enqueue/dequeue API. > > > >>>>>> > > > >>>>>> Right now DPDK ethdev layer *does not* provide synchronization > > > >>>>>> mechanisms between data-path and control-path functions. > > > >>>>>> That was a deliberate deisgn choice. If we want to change that > > > >>>>>> rule, then I suppose we need a community consensus for it. > > > >>>>>> I think that if the driver wants to provide some sort of error > > > >>>>>> recovery procedure, then it has to provide some synchronizatio= n > > > >>>>>> mechanism inside it between data-path and control-path functio= ns. > > > >>>>>> Actually looking at eb0d471a894 (ethdev: add proactive error > > > >>>>>> handling mode), and following patches I wonder how it creeped = in? > > > >>>>>> It seems we just introduced a loophole for race condition with > > > >>>>>> this approach... > > > >>>> > > > >>>> Could you try to describe the specific scenario of loophole ? > > > >>> > > > >>> Ok, as I understand the existing mechanism: > > > >>> > > > >>> When PMD wants to start a recovery it has to: > > > >>> - invoke > > > >>> rte_eth_dev_callback_process(RTE_ETH_EVENT_ERR_RECOVERING); > > > >>> That supposed to call user provided callback. After callback = is > > > >>> finished PMD assumes > > > >>> that user is aware that recovery is about to start and should > > > >>> make some precautions. > > > >>> - when recovery is finished it invokes another callback: > > > >>> RTE_ETH_EVENT_RECOVERY_(SUCCESS/FAILED). After that user eithe= r > > > >>> can continue to > > > >>> use port or have to treat is as faulty. > > > >>> > > > >>> The idea is ok in principle, but there is a problem. > > > >>> > > > >>> lib/ethdev/rte_ethdev.h: > > > >>> /** Port recovering from a hardware or firmware error= . > > > >>> * If PMD supports proactive error recovery, > > > >>> * it should trigger this event to notify application > > > >>> * that it detected an error and the recovery is being s= tarted. > > > >>> > > > >>> <<< !!!!! > > > >>> * Upon receiving the event, the application should not > > > >>> invoke any control path API > > > >>> * (such as rte_eth_dev_configure/rte_eth_dev_stop...) > > > >>> until receiving > > > >>> * RTE_ETH_EVENT_RECOVERY_SUCCESS or > > > >>> RTE_ETH_EVENT_RECOVERY_FAILED event. > > > >>> * The PMD will set the data path pointers to dummy > > > >>> functions, > > > >>> * and re-set the data path pointers to non-dummy functi= ons > > > >>> * before reporting RTE_ETH_EVENT_RECOVERY_SUCCESS event= . > > > >>> <<< !!!!! > > > >>> > > > >>> That part is just wrong I believe. > > > >>> It should be: > > > >>> Upon receiving the event, the application should not invoke any > > > >>> *both control and data-path* API until receiving > > > >>> RTE_ETH_EVENT_RECOVERY_SUCCESS or RTE_ETH_EVENT_RECOVERY_FAILED > > > >>> event. > > > >>> Resetting data path pointers to dummy functions by PMD *before* > > > >>> invoking rte_eth_dev_callback_process(RTE_ETH_EVENT_ERR_RECOVERIN= G); > > > >>> introduces a race-condition with data-path threads, as such threa= d > > > >>> could already be inside RX/TX function or can already read RX/TX > > > >>> function/data pointers and be about to use them. > > > >> > > > >> Current practices: the PMDs already add some delay after set Rx/Tx > > > >> callback to dummy, and plus the DPDK worker thread is busypolling, > > > >> the probability of occurence in reality is zero. But in theoretica= lly > > > >> exist the above race-condition. > > > > > > > > > > > > Adding delay might make a problem a bit less reproducible, but it > > > > doesn't fix it. > > > > The bug is still there. > > > > > > > > > > > >> > > > >>> And right now rte_ethdev layer doesn't provide any mechanism to > > > >>> check it or wait when they'll finish, etc. > > > >> > > > >> Yes > > > >> > > > >>> > > > >>> So, probably the simplest way to fix it with existing DPDK design= : > > > >>> - user level callback RTE_ETH_EVENT_ERR_RECOVERING should return > > > >>> only after it ensures that *all* > > > >>> application threads (and processes) stopped using either contr= ol > > > >>> or data-path functions for that port > > > >> > > > >> Agree > > > >> > > > >>> (yes it means that application that wants to use this feature = has > > > >>> to provide its own synchronization mechanism > > > >>> around data-path functions (RX/TX) that it is going to use). > > > >>> - after that PMD is safe to reset rte_eth_fp_ops[] values to dumm= y ones. > > > >>> > > > >>> And message to all PMD developers: > > > >>> *please stop updating rte_eth_fp_ops[] on your own*. > > > >>> That's a bad practice and it is not supposed to do things that wa= y. > > > >>> There is a special API provided for these purposes: > > > >>> eth_dev_fp_ops_reset(), eth_dev_fp_ops_setup(), so use it. > > > >> > > > >> This two function is in private.h, so it should be expose to publi= c > > > >> header file. > > > > > > > > You mean we need to move these functions declarations into ethdev_d= river.h? > > > > If so, then yes, I think we probably do. > > > > > > > > > > > > > > > > > What about making slightly different version available to drivers, wh= ich only updates > > > function pointers, but not 'fpo->rxq' / 'fpo->txq'. > > > > > > This way driver can switch to between dummy and real burst function w= ithout worrying Rx/Tx > > > queue validity. > > > > > > @Chengwen, @Ruifeng, can this solve the issue for relaxed memory orde= ring systems? > > > > Yes, updating only function pointers removes the synchronization requir= ement between function > > pointer and qdata. > > Lads, that wouldn't work anyway. > The race between recovery procedure and data-path persists: > Recovery still has no idea is at given moment any thread doing RX/TX or n= ot, and there is no > way for it to know when such thread will finish. > We do need some synchronization mechanism between control(recovery) and d= ata-path threads. > I believe it is unavoidable. +1 > > > > > > > > > > > > > >>> > > > >>> BTW, I don't see any implementation for > > > >>> RTE_ETH_EVENT_ERR_RECOVERING within either testpmd or any other > > > >>> example apps. > > > >>> Am I missing something? > > > >> > > > >> Currently it just promote the event. > > > > > > > > > > > > Ok, can I suggest then to add a proper usage for into in testpmd? > > > > It looks really strange that we add new feature into ethdev (and 2 > > > > PMDs), but didn't provide any way for users to test it. > > > > > > > >> > > > >>> If not, then probably it could be a good starting point - let's > > > >>> incorporate it inside testpmd (new forwarding engine probably) so > > > >>> everyone can test/try it. > > > >>> > > > >>> * It means that the application cannot send or receive = any > > > >>> packets > > > >>> * during this period. > > > >>> * @note Before the PMD reports the recovery result, > > > >>> * the PMD may report the RTE_ETH_EVENT_ERR_RECOVERING > > > >>> event again, > > > >>> * because a larger error may occur during the recovery. > > > >>> */ > > > >>> RTE_ETH_EVENT_ERR_RECOVERING, > > > >>> > > > >>>>>> It probably needs to be either deprecated or reworked. > > > >>>>> Looking at the commit, it does not say anything about the data > > > >>>>> plane functions which probably means, the error recovery is > > > >>>> happening within the data plane thread. What happens to other da= ta > > > >>>> plane threads that are polling the same port on which the error > > > >>>> recovery is happening? > > > >>>> > > > >>>> The commit log says: "the PMD sets the data path pointers to dum= my > > > >>>> functions". > > > >>>> > > > >>>> So the data plane threads will receive non-packet and send zero > > > >>>> with port which in error recovery. > > > >>>> > > > >>>>> > > > >>>>> Also, the commit log says that while the error recovery is unde= r > > > >>>>> progress, the application should not call any control plane API= s. > > > >>>>> Does > > > >>>> that mean, the application has to check for error condition ever= y > > > >>>> time it calls a control plane API? > > > >>>> > > > >>>> If application has not register event > > > >>>> (RTE_ETH_EVENT_ERR_RECOVERING) callback, it could calls control > > > >>>> plane API, but it will return failed. > > > >>>> If application has register above callback, it can wait for > > > >>>> recovery result, or direct call without wait but this will retur= n failed. > > > >>>> > > > >>>>> > > > >>>>> The commit message also says that "PMD makes sure the control p= ath > > > >>>>> operations failed with retcode -EBUSY". It does not say how it > > > >>>> does this. But, any communication from the PMD thread to control > > > >>>> plane thread may introduce race conditions if not done correctly= . > > > >>>> > > > >>>> First there are no PMD thread, do you mean eal-intr-thread ? > > > >>>> > > > >>>> As for this question, you can see PMDs which already implement i= t, > > > >>>> they both provides mutual exclusion protection. > > > >>>> > > > >>>>> > > > >>>>>> > > > >>>>>>> > > > >>>>>>>> > > > >>>>>>>> Would something like this work better? > > > >>>>>>>> > > > >>>>>>>> Note: there is another bug in current code. The check for li= nk > > > >>>>>>>> state interrupt and link_ops could return -ENOTSUP and leave > > > >>>>>>>> device in > > > >>>>>> indeterminate state. > > > >>>>>>>> The check should be done before calling PMD. > > > >>>>>>>> > > > >>>>>>>> diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev= .c > > > >>>>>>>> index > > > >>>>>>>> 0266cc82acb6..d6c163ed85e7 100644 > > > >>>>>>>> --- a/lib/ethdev/rte_ethdev.c > > > >>>>>>>> +++ b/lib/ethdev/rte_ethdev.c > > > >>>>>>>> @@ -1582,6 +1582,14 @@ rte_eth_dev_start(uint16_t port_id) > > > >>>>>>>> return 0; > > > >>>>>>>> } > > > >>>>>>>> > > > >>>>>>>> + if (dev->data->dev_conf.intr_conf.lsc =3D=3D 0 && > > > >>>>>>>> + dev->dev_ops->link_update =3D=3D NULL) { > > > >>>>>>>> + RTE_ETHDEV_LOG(INFO, > > > >>>>>>>> + "Device with port_id=3D%"PRIu16" link up= date > > > >>>>>>>> +not > > > >>>>>> supported\n", > > > >>>>>>>> + port_id); > > > >>>>>>>> + return -ENOTSUP; > > > >>>>>>>> + } > > > >>>>>>>> + > > > >>>>>>>> ret =3D rte_eth_dev_info_get(port_id, &dev_info); > > > >>>>>>>> if (ret !=3D 0) > > > >>>>>>>> return ret; > > > >>>>>>>> @@ -1591,9 +1599,7 @@ rte_eth_dev_start(uint16_t port_id) > > > >>>>>>>> eth_dev_mac_restore(dev, &dev_info); > > > >>>>>>>> > > > >>>>>>>> diag =3D (*dev->dev_ops->dev_start)(dev); > > > >>>>>>>> - if (diag =3D=3D 0) > > > >>>>>>>> - dev->data->dev_started =3D 1; > > > >>>>>>>> - else > > > >>>>>>>> + if (diag !=3D 0) > > > >>>>>>>> return eth_err(port_id, diag); > > > >>>>>>>> > > > >>>>>>>> ret =3D eth_dev_config_restore(dev, &dev_info, port_id= ); @@ > > > >>>>>>>> -1611,16 > > > >>>>>>>> +1617,18 @@ rte_eth_dev_start(uint16_t port_id) > > > >>>>>>>> return ret; > > > >>>>>>>> } > > > >>>>>>>> > > > >>>>>>>> - if (dev->data->dev_conf.intr_conf.lsc =3D=3D 0) { > > > >>>>>>>> - if (*dev->dev_ops->link_update =3D=3D NULL) > > > >>>>>>>> - return -ENOTSUP; > > > >>>>>>>> - (*dev->dev_ops->link_update)(dev, 0); > > > >>>>>>>> - } > > > >>>>>>>> - > > > >>>>>>>> /* expose selection of PMD fast-path functions */ > > > >>>>>>>> eth_dev_fp_ops_setup(rte_eth_fp_ops + port_id, dev); > > > >>>>>>>> > > > >>>>>>>> + /* ensure state is set before marking device ready */ > > > >>>>>>>> + rte_smp_wmb(); > > > >>>>>>>> + > > > >>>>>>>> rte_ethdev_trace_start(port_id); > > > >>>>>>>> + > > > >>>>>>>> + /* Update current link state */ > > > >>>>>>>> + if (dev->data->dev_conf.intr_conf.lsc =3D=3D 0) > > > >>>>>>>> + (*dev->dev_ops->link_update)(dev, 0); > > > >>>>>>>> + > > > >>>>>>>> return 0; > > > >>>>>>>> } > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> . > > > >>>>>>>> > > > >>>>> > > > > > --00000000000037b24105f63972e3 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIIQdgYJKoZIhvcNAQcCoIIQZzCCEGMCAQExDzANBglghkgBZQMEAgEFADALBgkqhkiG9w0BBwGg gg3NMIIFDTCCA/WgAwIBAgIQeEqpED+lv77edQixNJMdADANBgkqhkiG9w0BAQsFADBMMSAwHgYD VQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMzETMBEGA1UEChMKR2xvYmFsU2lnbjETMBEGA1UE AxMKR2xvYmFsU2lnbjAeFw0yMDA5MTYwMDAwMDBaFw0yODA5MTYwMDAwMDBaMFsxCzAJBgNVBAYT AkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMTEwLwYDVQQDEyhHbG9iYWxTaWduIEdDQyBS MyBQZXJzb25hbFNpZ24gMiBDQSAyMDIwMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA vbCmXCcsbZ/a0fRIQMBxp4gJnnyeneFYpEtNydrZZ+GeKSMdHiDgXD1UnRSIudKo+moQ6YlCOu4t rVWO/EiXfYnK7zeop26ry1RpKtogB7/O115zultAz64ydQYLe+a1e/czkALg3sgTcOOcFZTXk38e aqsXsipoX1vsNurqPtnC27TWsA7pk4uKXscFjkeUE8JZu9BDKaswZygxBOPBQBwrA5+20Wxlk6k1 e6EKaaNaNZUy30q3ArEf30ZDpXyfCtiXnupjSK8WU2cK4qsEtj09JS4+mhi0CTCrCnXAzum3tgcH cHRg0prcSzzEUDQWoFxyuqwiwhHu3sPQNmFOMwIDAQABo4IB2jCCAdYwDgYDVR0PAQH/BAQDAgGG MGAGA1UdJQRZMFcGCCsGAQUFBwMCBggrBgEFBQcDBAYKKwYBBAGCNxQCAgYKKwYBBAGCNwoDBAYJ KwYBBAGCNxUGBgorBgEEAYI3CgMMBggrBgEFBQcDBwYIKwYBBQUHAxEwEgYDVR0TAQH/BAgwBgEB /wIBADAdBgNVHQ4EFgQUljPR5lgXWzR1ioFWZNW+SN6hj88wHwYDVR0jBBgwFoAUj/BLf6guRSSu TVD6Y5qL3uLdG7wwegYIKwYBBQUHAQEEbjBsMC0GCCsGAQUFBzABhiFodHRwOi8vb2NzcC5nbG9i YWxzaWduLmNvbS9yb290cjMwOwYIKwYBBQUHMAKGL2h0dHA6Ly9zZWN1cmUuZ2xvYmFsc2lnbi5j b20vY2FjZXJ0L3Jvb3QtcjMuY3J0MDYGA1UdHwQvMC0wK6ApoCeGJWh0dHA6Ly9jcmwuZ2xvYmFs c2lnbi5jb20vcm9vdC1yMy5jcmwwWgYDVR0gBFMwUTALBgkrBgEEAaAyASgwQgYKKwYBBAGgMgEo CjA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3dy5nbG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5LzAN BgkqhkiG9w0BAQsFAAOCAQEAdAXk/XCnDeAOd9nNEUvWPxblOQ/5o/q6OIeTYvoEvUUi2qHUOtbf jBGdTptFsXXe4RgjVF9b6DuizgYfy+cILmvi5hfk3Iq8MAZsgtW+A/otQsJvK2wRatLE61RbzkX8 9/OXEZ1zT7t/q2RiJqzpvV8NChxIj+P7WTtepPm9AIj0Keue+gS2qvzAZAY34ZZeRHgA7g5O4TPJ /oTd+4rgiU++wLDlcZYd/slFkaT3xg4qWDepEMjT4T1qFOQIL+ijUArYS4owpPg9NISTKa1qqKWJ jFoyms0d0GwOniIIbBvhI2MJ7BSY9MYtWVT5jJO3tsVHwj4cp92CSFuGwunFMzCCA18wggJHoAMC AQICCwQAAAAAASFYUwiiMA0GCSqGSIb3DQEBCwUAMEwxIDAeBgNVBAsTF0dsb2JhbFNpZ24gUm9v dCBDQSAtIFIzMRMwEQYDVQQKEwpHbG9iYWxTaWduMRMwEQYDVQQDEwpHbG9iYWxTaWduMB4XDTA5 MDMxODEwMDAwMFoXDTI5MDMxODEwMDAwMFowTDEgMB4GA1UECxMXR2xvYmFsU2lnbiBSb290IENB IC0gUjMxEzARBgNVBAoTCkdsb2JhbFNpZ24xEzARBgNVBAMTCkdsb2JhbFNpZ24wggEiMA0GCSqG SIb3DQEBAQUAA4IBDwAwggEKAoIBAQDMJXaQeQZ4Ihb1wIO2hMoonv0FdhHFrYhy/EYCQ8eyip0E XyTLLkvhYIJG4VKrDIFHcGzdZNHr9SyjD4I9DCuul9e2FIYQebs7E4B3jAjhSdJqYi8fXvqWaN+J J5U4nwbXPsnLJlkNc96wyOkmDoMVxu9bi9IEYMpJpij2aTv2y8gokeWdimFXN6x0FNx04Druci8u nPvQu7/1PQDhBjPogiuuU6Y6FnOM3UEOIDrAtKeh6bJPkC4yYOlXy7kEkmho5TgmYHWyn3f/kRTv riBJ/K1AFUjRAjFhGV64l++td7dkmnq/X8ET75ti+w1s4FRpFqkD2m7pg5NxdsZphYIXAgMBAAGj QjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBSP8Et/qC5FJK5N UPpjmove4t0bvDANBgkqhkiG9w0BAQsFAAOCAQEAS0DbwFCq/sgM7/eWVEVJu5YACUGssxOGhigH M8pr5nS5ugAtrqQK0/Xx8Q+Kv3NnSoPHRHt44K9ubG8DKY4zOUXDjuS5V2yq/BKW7FPGLeQkbLmU Y/vcU2hnVj6DuM81IcPJaP7O2sJTqsyQiunwXUaMld16WCgaLx3ezQA3QY/tRG3XUyiXfvNnBB4V 14qWtNPeTCekTBtzc3b0F5nCH3oO4y0IrQocLP88q1UOD5F+NuvDV0m+4S4tfGCLw0FREyOdzvcy a5QBqJnnLDMfOjsl0oZAzjsshnjJYS8Uuu7bVW/fhO4FCU29KNhyztNiUGUe65KXgzHZs7XKR1g/ XzCCBVUwggQ9oAMCAQICDAzZWuPidkrRZaiw2zANBgkqhkiG9w0BAQsFADBbMQswCQYDVQQGEwJC RTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTExMC8GA1UEAxMoR2xvYmFsU2lnbiBHQ0MgUjMg UGVyc29uYWxTaWduIDIgQ0EgMjAyMDAeFw0yMjA5MTAwODE4NDVaFw0yNTA5MTAwODE4NDVaMIGW MQswCQYDVQQGEwJJTjESMBAGA1UECBMJS2FybmF0YWthMRIwEAYDVQQHEwlCYW5nYWxvcmUxFjAU BgNVBAoTDUJyb2FkY29tIEluYy4xHDAaBgNVBAMTE0FqaXQgS3VtYXIgS2hhcGFyZGUxKTAnBgkq hkiG9w0BCQEWGmFqaXQua2hhcGFyZGVAYnJvYWRjb20uY29tMIIBIjANBgkqhkiG9w0BAQEFAAOC AQ8AMIIBCgKCAQEArZ/Aqg34lMOo2BabvAa+dRThl9OeUUJMob125dz+jvS78k4NZn1mYrHu53Dn YycqjtuSMlJ6vJuwN2W6QpgTaA2SDt5xTB7CwA2urpcm7vWxxLOszkr5cxMB1QBbTd77bXFuyTqW jrer3VIWqOujJ1n+n+1SigMwEr7PKQR64YKq2aRYn74ukY3DlQdKUrm2yUkcA7aExLcAwHWUna/u pZEyqKnwS1lKCzjX7mV5W955rFsFxChdAKfw0HilwtqdY24mhy62+GeaEkD0gYIj1tCmw9gnQToc K+0s7xEunfR9pBrzmOwS3OQbcP0nJ8SmQ8R+reroH6LYuFpaqK1rgQIDAQABo4IB2zCCAdcwDgYD VR0PAQH/BAQDAgWgMIGjBggrBgEFBQcBAQSBljCBkzBOBggrBgEFBQcwAoZCaHR0cDovL3NlY3Vy ZS5nbG9iYWxzaWduLmNvbS9jYWNlcnQvZ3NnY2NyM3BlcnNvbmFsc2lnbjJjYTIwMjAuY3J0MEEG CCsGAQUFBzABhjVodHRwOi8vb2NzcC5nbG9iYWxzaWduLmNvbS9nc2djY3IzcGVyc29uYWxzaWdu MmNhMjAyMDBNBgNVHSAERjBEMEIGCisGAQQBoDIBKAowNDAyBggrBgEFBQcCARYmaHR0cHM6Ly93 d3cuZ2xvYmFsc2lnbi5jb20vcmVwb3NpdG9yeS8wCQYDVR0TBAIwADBJBgNVHR8EQjBAMD6gPKA6 hjhodHRwOi8vY3JsLmdsb2JhbHNpZ24uY29tL2dzZ2NjcjNwZXJzb25hbHNpZ24yY2EyMDIwLmNy bDAlBgNVHREEHjAcgRphaml0LmtoYXBhcmRlQGJyb2FkY29tLmNvbTATBgNVHSUEDDAKBggrBgEF BQcDBDAfBgNVHSMEGDAWgBSWM9HmWBdbNHWKgVZk1b5I3qGPzzAdBgNVHQ4EFgQUbrcTuh0mr2qP xYdtyDgFeRIiE/gwDQYJKoZIhvcNAQELBQADggEBALrc1TljKrDhXicOaZlzIQyqOEkKAZ324i8X OwzA0n2EcPGmMZvgARurvanSLD3mLeeuyq1feCcjfGM1CJFh4+EY7EkbFbpVPOIdstSBhbnAJnOl aC/q0wTndKoC/xXBhXOZB8YL/Zq4ZclQLMUO6xi/fFRyHviI5/IrosdrpniXFJ9ukJoOXtvdrEF+ KlMYg/Deg9xo3wddCqQIsztHSkR4XaANdn+dbLRQpctZ13BY1lim4uz5bYn3M0IxyZWkQ1JuPHCK aRJv0SfR88PoI4RB7NCEHqFwARTj1KvFPQi8pK/YISFydZYbZrxQdyWDidqm4wSuJfpE6i0cWvCd u50xggJtMIICaQIBATBrMFsxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNh MTEwLwYDVQQDEyhHbG9iYWxTaWduIEdDQyBSMyBQZXJzb25hbFNpZ24gMiBDQSAyMDIwAgwM2Vrj 4nZK0WWosNswDQYJYIZIAWUDBAIBBQCggdQwLwYJKoZIhvcNAQkEMSIEIP6AwEkjH3fqX1OMxlYt 2R5cqyjopwXYzxAiu6Z75wsnMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkF MQ8XDTIzMDMwNjExMTc1M1owaQYJKoZIhvcNAQkPMVwwWjALBglghkgBZQMEASowCwYJYIZIAWUD BAEWMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzALBgkqhkiG9w0BAQowCwYJKoZIhvcNAQEHMAsG CWCGSAFlAwQCATANBgkqhkiG9w0BAQEFAASCAQCLNNNVLGzPqqvTVzgsU08DfArjQOxV8Xttqkyx csaHNUHl+D/Z2cmxu6pCp8O2KieIJRAYEtUOJAzyQ6bSY8HNqmEAfEzo5nXUZyeKM+e/M5/y7Hgk nS/HXM4EyQOmKLoLG15LL93j8OL88QWQ/+gXJGgmQe8vE19xvAG9/i8YHsKzTrBQ/g44RrX6MM3P ksSRhZ2CTa7WGI/wKqzd078FaTGjKqzwjdeiVvrgnVoDo82AnSEQzN4LW8kyY61rNFt/XSthiPQ/ 8DDrS9lFSuOM4AigV2OMXcDLKlfVWD84KDe4rOqk8agdh8y5FgWfOq21LnAXr9iJqZUurs8loUKd --00000000000037b24105f63972e3--