From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D77C3A0547; Wed, 29 Sep 2021 18:01:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C2CB4410ED; Wed, 29 Sep 2021 18:01:11 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mails.dpdk.org (Postfix) with ESMTP id 6DAA1410EA for ; Wed, 29 Sep 2021 18:01:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1632931269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RLQ95FzGPeS8Y7XoqyN4/QXz9B6qNhz3P1/scejRCpE=; b=J8kR6ZSEms3+1jvk+2JnixnrW5eXNBNOEMPkJvlz2VchtQ91bvnc+U/ox7ebZ0giko/1/I 5gIxJPLRfoo17jq9w09QRZxpXtKIh1t/yTkBxS4K/yeBDxLyqPA2ol/4z/8PmXNOpwt7Uv nLBbnrr9slvPziQeGaxkVOm7vm94YtM= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-578-noJ8uIloMOuEZC7WqNN6yA-1; Wed, 29 Sep 2021 12:01:08 -0400 X-MC-Unique: noJ8uIloMOuEZC7WqNN6yA-1 Received: by mail-wm1-f72.google.com with SMTP id y142-20020a1c7d94000000b0030cdc76dedeso2904495wmc.5 for ; Wed, 29 Sep 2021 09:01:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:cc:references:from:subject:in-reply-to :content-transfer-encoding; bh=RLQ95FzGPeS8Y7XoqyN4/QXz9B6qNhz3P1/scejRCpE=; b=HR4NoJwrdMTVgVVCi0jNnsWaGAF17WK0uGbyD4qXMD2AxAknSzzZHbEZlJ/N0jcHg+ azoiMYk66yCx3UZ8BDRSn5ONGv66ZIvFjMKf3cB2Kzz4RRV2O5dhB8DTNru2JIA7dklV hn1HLd3MVtbnRBex8IEWFF+jos3NPf2YZiYrztjlwTreEhcPRipHPsx08kDClTfsv6GX x6X891v7rSPRnsv12qIKfVAZv6FuJctQPK/x+UEZLgfPwn/fmafQ23qonK9Dx1F5yJtw pn9R6T4WIVJfuHM50nnRe77oRHrpONuJLQq//NJkccofuL0ov8X8uNFqgQV1bnQnHiv/ NmDQ== X-Gm-Message-State: AOAM532KP1dpVBHh395/Mgppo1yw/xd05Y4BMk7L9N6jxYSquPC9Q21t G+wGw3dcfCnrf8S693maQ79GN4CZcqHkNX5/C3akw4QPsAeDRwRp33eKRZw9SiHPKpz9h75ohYw v1aA= X-Received: by 2002:adf:9d47:: with SMTP id o7mr810726wre.50.1632931267122; Wed, 29 Sep 2021 09:01:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyJDpi0DWKBaK+AzaJ65LRKG256Nm/auWDPuCD9GJLFhciw+wqOJIh/ykDgRY52A39AZ8n7JA== X-Received: by 2002:adf:9d47:: with SMTP id o7mr810673wre.50.1632931266807; Wed, 29 Sep 2021 09:01:06 -0700 (PDT) Received: from [192.168.0.36] ([78.19.105.235]) by smtp.gmail.com with ESMTPSA id f123sm2209387wmf.30.2021.09.29.09.01.05 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 29 Sep 2021 09:01:06 -0700 (PDT) Message-ID: Date: Wed, 29 Sep 2021 17:01:05 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 To: Bruce Richardson Cc: dev@dpdk.org, ciara.power@intel.com, anatoly.burakov@intel.com, stable@dpdk.org, David Marchand References: <20210915141030.23514-1-bruce.richardson@intel.com> <20210924161842.2879019-1-bruce.richardson@intel.com> <6ea20d4e-a7dd-afcb-3ca1-ffc023114d72@redhat.com> From: Kevin Traynor In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=ktraynor@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH v2] telemetry: fix "in-memory" process socket conflicts X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 29/09/2021 16:31, Bruce Richardson wrote: > On Wed, Sep 29, 2021 at 04:24:06PM +0100, Bruce Richardson wrote: >> On Wed, Sep 29, 2021 at 03:54:48PM +0100, Kevin Traynor wrote: >>> On 29/09/2021 14:32, Bruce Richardson wrote: >>>> On Wed, Sep 29, 2021 at 01:28:53PM +0100, Kevin Traynor wrote: >>>>> Hi Bruce, >>>>> >>>>> On 24/09/2021 17:18, Bruce Richardson wrote: >>>>>> When DPDK is run with --in-memory mode, multiple processes can run >>>>>> simultaneously using the same runtime dir. This leads to each process >>>>>> removing another process' telemetry socket as it started up, giving >>>>>> unexpected behaviour. >>>>>> >>>>>> This patch changes that behaviour to first check if the existing socket >>>>>> is active. If not, it's an old socket to be cleaned up and can be >>>>>> removed. If it is active, telemetry initialization fails and an error >>>>>> message is printed out giving instructions on how to remove the error; >>>>>> either by using file-prefix to have a different runtime dir (and >>>>>> therefore socket path) or by disabling telemetry if it not needed. >>>>>> >>>>> >>>>> telemetry is enabled by default but it may not be used by the application. >>>>> Hitting this issue will cause rte_eal_init() to fail which will probably >>>>> stop or severely limit the application. >>>>> >>>>> So it could change a working application to a non-working one (albeit one >>>>> that doesn't interfere with other process' sockets). >>>>> >>>>> Can it just print a warning that telemetry will not be enabled and continue >>>>> so it's not returning an rte_eal_init failure? >>>>> >>>> >>>> For a backported fix, yes, that would probably be better behaviour, but for >>>> the latest branch, I think returning error and having the user explicitly >>>> choose the resolution they want to occur is best. I'll see about doing a >>>> separate backport patch for 20.11. >>>> >>> >>> But this is a runtime message dependent on runtime environment. The user may >>> not have access or know how to change eal parameters. >> >> True. But on the other hand, this problem only occurs with non-default EAL >> parameters anyway, so someone must have configured this with the >> --in-memory flag. >> >>> >>> In the case where the application doesn't care about telemetry, they have >>> gone from not having telemetry to rte_eal_init() failing, which probably has >>> severe consequence. >>> >> >> Yes, I agree, which I why I would suggest that for any backport of this >> fix, the error be made non-fatal as you suggest. [Having looked into it, >> having it as a non-fatal error is rather awkward, so it may be best just >> left unfixed and the current behaviour documented as known-issue]. >> >> However, for any application being updated and rebuilt against 21.11, I >> would have thought it reasonable to flag this as an error, as any such >> application would require revalidation anyway. >> >>> I could maybe agree if telemetry was default disable and the application had >>> set the --telemetry flag indicating that they want/need it. As it is, it >>> feels like it's possibly a worse outcome for the user. >>> >> >> Perhaps, but I believe the only case of there being an issue would be where: >> 1) a user who cannot modify the EAL parameters >> 2) runs an application which has been updated and rebuilt against 21.11 >> 3) where that application is hard-coded to use in-memory mode and >> 4) has never been verified with two or more instances of that running? That's a reasonable point that if it has in-memory hardcoded you might expect it to be tested with two or more, and if it's not hardcoded, it is added by the user so they are able to set eal params. I still see an extra step for the user but I agree if they can set eal params then it is a lot less impactful. For OVS, a user could update the dpdk-extra ovsdb entry for the additional eal flags. >> Or am I missing something here? >> > > Let me also go back to the drawing board on the solution here a bit, and > see if I can come up with something better. If I can find a reasonable way > to make it so that we can always create a socket in in-memory mode, despite > other processes running, it would sidestep this problem completely. Not > sure if it's possible, but let me see if I can come up with some ideas. > [One idea I did try is using abstract sockets on linux, but with those we > lose out on the permissions/protection we get from having a filesystem > path, so were a no-go for me because of that] > ok, thanks Bruce. I think you got the concerns anyway. I suppose a part of it goes back to: telemetry is default, but does that imply that it is required and dpdk should error out if it is not available or not. Kevin. > /Bruce >