From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DCC5942B9B; Thu, 25 May 2023 17:28:00 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 69E5840DF8; Thu, 25 May 2023 17:28:00 +0200 (CEST) Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by mails.dpdk.org (Postfix) with ESMTP id E599C40DDB for ; Thu, 25 May 2023 17:27:58 +0200 (CEST) Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1afa6afcf4fso14634425ad.0 for ; Thu, 25 May 2023 08:27:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20221208.gappssmtp.com; s=20221208; t=1685028478; x=1687620478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=P9vVTKcqYl7MI+6zXIfZyz+bDBM4V862ZY+Ody4VlfQ=; b=WvNdeZ6tIAE3leLPP2/hwRUTFlPUOz6nc+xnp9z8EIimbGbYTeIBs+0Hjr0lIfWJXB ZonwgidyeCrhBbOJ3Hs2h/rU+3UiVknQINBHR5MijoEEXE+9Z2gP2QaWC/5li8sRLAjT I/ZlB8gGgU+hJOE0djfw8IYUBUR8Q/6ZoTrNcQIIrnqxx1laHyXHuvkkoAb4HSXB0le6 LjOLlonb8rSNWmFa+d9C8y6fb7PdgFvRyJaedaM6641ZIskPiEif8Ix/nQmABf79ceoX aHa1LkJWMXAHHX8NzcH48jxc9h6SDARwaCcIyvjEdEXeiyo8PQtBKFBn5yWFsH+uRGpl W66A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685028478; x=1687620478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P9vVTKcqYl7MI+6zXIfZyz+bDBM4V862ZY+Ody4VlfQ=; b=CpYfWOKXH52lGkDb2nhAbC/4ZPpPruk5GGYwbAo/HDwjsNG3rNwoFqu+6u1BTgNczz iOeVzu3k2QHkyHGQbxYqRKrGj5F9uOK25y1+U4iSdm2XWYqF3jIFZigUNY41weaVk+yA A+CoHQsnNjoMUl9XlSOQaqbV73Tw5nQaxKsgH35BfYfRZF7FEtokSm5XLPRcpV5TbqNJ AeDHH8M1i22Dwfnrm9+5wLCbVAj1Xyb+wmsj/4ZtasYkD8YpdRTQoMgRyuiRrEOoYbhG YktlNULN7hF0Q5dqBwBEDvNYd2z9Xrz5uHTUdxDWOF65se4W43SW5uRV9faKVzsUkqch vBhw== X-Gm-Message-State: AC+VfDz7VnI3qnxFn3Z+pzjRtx1iWrIqbCtb5INwYAifQ9fcSbgjN2fz K3HcRJd2VzgLxJOhCkAuDUfFNA== X-Google-Smtp-Source: ACHHUZ5XpBcSbnJNiWzY8I+kH3mWvWO3H0SgZIVw3Us8KKyNde+4DL84+68ghoS0kpu/0E6GA10AVw== X-Received: by 2002:a17:902:760f:b0:1af:dc52:bf6a with SMTP id k15-20020a170902760f00b001afdc52bf6amr1885011pll.54.1685028478146; Thu, 25 May 2023 08:27:58 -0700 (PDT) Received: from hermes.local (204-195-120-218.wavecable.com. [204.195.120.218]) by smtp.gmail.com with ESMTPSA id t8-20020a170902b20800b001aafdf8063dsm1563720plr.157.2023.05.25.08.27.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 08:27:57 -0700 (PDT) Date: Thu, 25 May 2023 08:27:55 -0700 From: Stephen Hemminger To: Slava Ovsiienko Cc: Erez Ferber , "dev@dpdk.org" , Matan Azrad , Raslan Darawsheh , "stable@dpdk.org" Subject: Re: [PATCH] common/mlx5: adjust fork call with the new kernel API Message-ID: <20230525082756.630ab7f6@hermes.local> In-Reply-To: References: <20230524120140.416144-1-erezf@nvidia.com> <20230524075013.3c2f7b6e@hermes.local> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, 25 May 2023 08:10:03 +0000 Slava Ovsiienko wrote: > > -----Original Message----- > > From: Stephen Hemminger > > Sent: Wednesday, May 24, 2023 5:50 PM > > To: Erez Ferber > > Cc: dev@dpdk.org; Slava Ovsiienko ; Matan Azrad > > ; Raslan Darawsheh ; > > stable@dpdk.org > > Subject: Re: [PATCH] common/mlx5: adjust fork call with the new kernel = API > >=20 > > On Wed, 24 May 2023 15:01:40 +0300 > > wrote: > > =20 > > > From: Erez Ferber > > > > > > While doing process fork() the operating system remaps all the parent > > > process's memory to the address space of the child process and > > > activates the Copy-on-Write mechanics - it duplicates physical pages > > > once memory writing happens in the child process. Sometimes memory > > > duplication is not allowed - for example, if the page contains > > > hardware queue descriptors. To handle similar issues the rdma-core > > > library should be prepared for forking. > > > > > > The ibv_fork_init() prepares the library to track all the related > > > memory and prevent it from forking using madvise() system API. This > > > approach allows fork, but not all the memory is forked to the child > > > process and, application should care not to touch pages where the > > > parent application allocated the rdma-core objects. > > > > > > The newer kernels propose an option of copy-on-fork for DMA pages and > > > tracking all the memory and disabling it for the forking is no longer > > > needed. The new API routine ibv_is_fork_initialized() should be > > > involved to decide if library initialization for forking is required. > > > > > > Fixes: 0e83b8e536 ("net/mlx5: move rdma-core calls to separate file") > > > Cc: stable@dpdk.org > > > Signed-off-by: Erez Ferber =20 > > =20 > Hi, >=20 > > I don't think DPDK applications should fork(), and lots other parts of = the > > shared huge pages will break if an application does this. =20 >=20 > I agree - application should not, we have the secondary/primary processes= approach. > Nonetheless, we have the real use case - DPDK application does fork() and= works well. > Without mlx5 PMD =F0=9F=98=8A. With mlx5 it ran into some troubles. Now w= e have the solution. The problem is you are allowing fork(). And many other libraries may break. Imagine a DPDK library which has some local mutex and hugepages. If two forked processes use it then the locks won't work and hugepage data will be corrupted.