From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vs1-f66.google.com (mail-vs1-f66.google.com [209.85.217.66]) by dpdk.org (Postfix) with ESMTP id E18082BD3 for ; Fri, 29 Mar 2019 10:51:10 +0100 (CET) Received: by mail-vs1-f66.google.com with SMTP id d8so450111vsp.2 for ; Fri, 29 Mar 2019 02:51:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uEaktBFmoTEwK0ZVs+FpYXQnRxM7dx2sOnh+r/z4Joo=; b=cVv4kruMXLu+57R1xFlzK+35FOqoW+BYq/DqYs2ztpXBVR5DFWu+H+3pi2hkKeAZNE ZeQ2Wv7lSNnlUlptybVny24HgAydONPImiLvCIkqDogpP/gFAgEbCczsValbH1/he3zV q+w1x/jhmpLL+oyH5qP5W8JzxsGeHXl0zxgAmo0Sa5znlnk4IjK+mfxt5C+pQXO8pidH mKU6eVdAgo4DCGAg/P+LLr4BFia/m8UMat2KGCT+4L/kOx/75ycRpRQDyfWFXA3bGglQ cDBoriVNhRcaWBrDdWXbeB/+K5mhNbWDQBr/Xmf/JpsBRq2g8jR/RIYkkUEYx52bcFvH Vexw== X-Gm-Message-State: APjAAAUr2g8HjkMmv+Cd1Rf0BfWwQRTz6290lktlYWw4P96FuaTYiwYJ uIJoawHOl0bFSdWvdQCryxnGVA2iQJir3NbnyxOcaw== X-Google-Smtp-Source: APXvYqwuCH5wTzre8tJ487OMQuc+Y0dlq01enC5Dygab5Q/DOFaANLbBl++c1lqoJgI9b/pbuW1Vz5PJI8L9FpjneG8= X-Received: by 2002:a67:a446:: with SMTP id p6mr8966671vsh.198.1553853070229; Fri, 29 Mar 2019 02:51:10 -0700 (PDT) MIME-Version: 1.0 References: <07f664c33ddedaa5dcfe82ecb97d931e68b7e33a.1550855529.git.anatoly.burakov@intel.com> In-Reply-To: <07f664c33ddedaa5dcfe82ecb97d931e68b7e33a.1550855529.git.anatoly.burakov@intel.com> From: David Marchand Date: Fri, 29 Mar 2019 10:50:58 +0100 Message-ID: To: Anatoly Burakov Cc: dev , John McNamara , Marko Kovacevic , iain.barker@oracle.com, edwin.leung@oracle.com, Thomas Monjalon Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH] eal: add option to not store segment fd's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2019 09:51:11 -0000 On Fri, Feb 22, 2019 at 6:12 PM Anatoly Burakov wrote: > Due to internal glibc limitations [1], DPDK may exhaust internal > file descriptor limits when using smaller page sizes, which results > in inability to use system calls such as select() by user > applications. > > While the problem can be worked around using --single-file-segments > option, it does not work if --legacy-mem mode is also used. Add a > (yet another) EAL flag to disable storing fd's internally. This > will sacrifice compability with Virtio with vhost-backend, but > at least select() and friends will work. > > [1] https://mails.dpdk.org/archives/dev/2019-February/124386.html Sorry, I am a bit lost and I never took the time to look in the new memory allocation system. This gives the impression that we are accumulating workarounds, between legacy-mem, single-file-segments, now no-seg-fds. Iiuc, everything revolves around the need for per page locks. Can you summarize why we need them? Thanks. -- David Marchand From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 38322A05D3 for ; Fri, 29 Mar 2019 10:51:23 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id B985E378B; Fri, 29 Mar 2019 10:51:15 +0100 (CET) Received: from mail-vs1-f66.google.com (mail-vs1-f66.google.com [209.85.217.66]) by dpdk.org (Postfix) with ESMTP id E18082BD3 for ; Fri, 29 Mar 2019 10:51:10 +0100 (CET) Received: by mail-vs1-f66.google.com with SMTP id d8so450111vsp.2 for ; Fri, 29 Mar 2019 02:51:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=uEaktBFmoTEwK0ZVs+FpYXQnRxM7dx2sOnh+r/z4Joo=; b=cVv4kruMXLu+57R1xFlzK+35FOqoW+BYq/DqYs2ztpXBVR5DFWu+H+3pi2hkKeAZNE ZeQ2Wv7lSNnlUlptybVny24HgAydONPImiLvCIkqDogpP/gFAgEbCczsValbH1/he3zV q+w1x/jhmpLL+oyH5qP5W8JzxsGeHXl0zxgAmo0Sa5znlnk4IjK+mfxt5C+pQXO8pidH mKU6eVdAgo4DCGAg/P+LLr4BFia/m8UMat2KGCT+4L/kOx/75ycRpRQDyfWFXA3bGglQ cDBoriVNhRcaWBrDdWXbeB/+K5mhNbWDQBr/Xmf/JpsBRq2g8jR/RIYkkUEYx52bcFvH Vexw== X-Gm-Message-State: APjAAAUr2g8HjkMmv+Cd1Rf0BfWwQRTz6290lktlYWw4P96FuaTYiwYJ uIJoawHOl0bFSdWvdQCryxnGVA2iQJir3NbnyxOcaw== X-Google-Smtp-Source: APXvYqwuCH5wTzre8tJ487OMQuc+Y0dlq01enC5Dygab5Q/DOFaANLbBl++c1lqoJgI9b/pbuW1Vz5PJI8L9FpjneG8= X-Received: by 2002:a67:a446:: with SMTP id p6mr8966671vsh.198.1553853070229; Fri, 29 Mar 2019 02:51:10 -0700 (PDT) MIME-Version: 1.0 References: <07f664c33ddedaa5dcfe82ecb97d931e68b7e33a.1550855529.git.anatoly.burakov@intel.com> In-Reply-To: <07f664c33ddedaa5dcfe82ecb97d931e68b7e33a.1550855529.git.anatoly.burakov@intel.com> From: David Marchand Date: Fri, 29 Mar 2019 10:50:58 +0100 Message-ID: To: Anatoly Burakov Cc: dev , John McNamara , Marko Kovacevic , iain.barker@oracle.com, edwin.leung@oracle.com, Thomas Monjalon Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [PATCH] eal: add option to not store segment fd's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190329095058.OTREuxtyQW5REWy3c2Ij407f5K0mER4z-pLLgMqn4fA@z> On Fri, Feb 22, 2019 at 6:12 PM Anatoly Burakov wrote: > Due to internal glibc limitations [1], DPDK may exhaust internal > file descriptor limits when using smaller page sizes, which results > in inability to use system calls such as select() by user > applications. > > While the problem can be worked around using --single-file-segments > option, it does not work if --legacy-mem mode is also used. Add a > (yet another) EAL flag to disable storing fd's internally. This > will sacrifice compability with Virtio with vhost-backend, but > at least select() and friends will work. > > [1] https://mails.dpdk.org/archives/dev/2019-February/124386.html Sorry, I am a bit lost and I never took the time to look in the new memory allocation system. This gives the impression that we are accumulating workarounds, between legacy-mem, single-file-segments, now no-seg-fds. Iiuc, everything revolves around the need for per page locks. Can you summarize why we need them? Thanks. -- David Marchand