From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 5C8901DB8 for ; Fri, 29 Mar 2019 11:33:32 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 03:33:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="156917992" Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.103]) ([10.237.220.103]) by fmsmga004.fm.intel.com with ESMTP; 29 Mar 2019 03:33:29 -0700 To: David Marchand Cc: dev , John McNamara , Marko Kovacevic , iain.barker@oracle.com, edwin.leung@oracle.com, Thomas Monjalon References: <07f664c33ddedaa5dcfe82ecb97d931e68b7e33a.1550855529.git.anatoly.burakov@intel.com> From: "Burakov, Anatoly" Message-ID: <940ad1bd-8df5-5afb-e5e4-2f954a0a2686@intel.com> Date: Fri, 29 Mar 2019 10:33:28 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] eal: add option to not store segment fd's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2019 10:33:32 -0000 On 29-Mar-19 9:50 AM, David Marchand wrote: > On Fri, Feb 22, 2019 at 6:12 PM Anatoly Burakov > > wrote: > > Due to internal glibc limitations [1], DPDK may exhaust internal > file descriptor limits when using smaller page sizes, which results > in inability to use system calls such as select() by user > applications. > > While the problem can be worked around using --single-file-segments > option, it does not work if --legacy-mem mode is also used. Add a > (yet another) EAL flag to disable storing fd's internally. This > will sacrifice compability with Virtio with vhost-backend, but > at least select() and friends will work. > > [1] https://mails.dpdk.org/archives/dev/2019-February/124386.html > > > Sorry, I am a bit lost and I never took the time to look in the new > memory allocation system. > This gives the impression that we are accumulating workarounds, between > legacy-mem, single-file-segments, now no-seg-fds. Yep. I don't like this any more than you do, but i think there are users of all of these, so we can't just drop them willy-nilly. My great hope was that by now everyone would move on to use VFIO so legacy mem wouldn't be needed (the only reason it exists is to provide compatibility for use cases where lots of IOVA-contiguous memory is required, and VFIO cannot be used), but apparently that is too much to ask :/ > > Iiuc, everything revolves around the need for per page locks. > Can you summarize why we need them? The short answer is multiprocess. We have to be able to map and unmap pages individually, and for that we need to be sure that we can, in fact, remove a page because no one else uses it. We also need to store fd's because virtio with vhost-user backend needs them to work, because it relies on sharing memory between processes using fd's. > > Thanks. > > -- > David Marchand -- Thanks, Anatoly From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id D897BA05D3 for ; Fri, 29 Mar 2019 11:33:34 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AB8213256; Fri, 29 Mar 2019 11:33:33 +0100 (CET) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 5C8901DB8 for ; Fri, 29 Mar 2019 11:33:32 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 03:33:31 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="156917992" Received: from aburakov-mobl1.ger.corp.intel.com (HELO [10.237.220.103]) ([10.237.220.103]) by fmsmga004.fm.intel.com with ESMTP; 29 Mar 2019 03:33:29 -0700 To: David Marchand Cc: dev , John McNamara , Marko Kovacevic , iain.barker@oracle.com, edwin.leung@oracle.com, Thomas Monjalon References: <07f664c33ddedaa5dcfe82ecb97d931e68b7e33a.1550855529.git.anatoly.burakov@intel.com> From: "Burakov, Anatoly" Message-ID: <940ad1bd-8df5-5afb-e5e4-2f954a0a2686@intel.com> Date: Fri, 29 Mar 2019 10:33:28 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format="flowed" Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] eal: add option to not store segment fd's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190329103328.8z8xA-M0mcab0du34pyxYp5_ZmlXtKnFSF-sUTStXHg@z> On 29-Mar-19 9:50 AM, David Marchand wrote: > On Fri, Feb 22, 2019 at 6:12 PM Anatoly Burakov > > wrote: > > Due to internal glibc limitations [1], DPDK may exhaust internal > file descriptor limits when using smaller page sizes, which results > in inability to use system calls such as select() by user > applications. > > While the problem can be worked around using --single-file-segments > option, it does not work if --legacy-mem mode is also used. Add a > (yet another) EAL flag to disable storing fd's internally. This > will sacrifice compability with Virtio with vhost-backend, but > at least select() and friends will work. > > [1] https://mails.dpdk.org/archives/dev/2019-February/124386.html > > > Sorry, I am a bit lost and I never took the time to look in the new > memory allocation system. > This gives the impression that we are accumulating workarounds, between > legacy-mem, single-file-segments, now no-seg-fds. Yep. I don't like this any more than you do, but i think there are users of all of these, so we can't just drop them willy-nilly. My great hope was that by now everyone would move on to use VFIO so legacy mem wouldn't be needed (the only reason it exists is to provide compatibility for use cases where lots of IOVA-contiguous memory is required, and VFIO cannot be used), but apparently that is too much to ask :/ > > Iiuc, everything revolves around the need for per page locks. > Can you summarize why we need them? The short answer is multiprocess. We have to be able to map and unmap pages individually, and for that we need to be sure that we can, in fact, remove a page because no one else uses it. We also need to store fd's because virtio with vhost-user backend needs them to work, because it relies on sharing memory between processes using fd's. > > Thanks. > > -- > David Marchand -- Thanks, Anatoly