From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 01290326C for ; Tue, 27 Nov 2018 16:00:09 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 27 Nov 2018 07:00:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,287,1539673200"; d="scan'208";a="93685856" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by fmsmga008.fm.intel.com with ESMTP; 27 Nov 2018 07:00:08 -0800 Received: from fmsmsx118.amr.corp.intel.com ([169.254.1.160]) by fmsmsx104.amr.corp.intel.com ([169.254.3.128]) with mapi id 14.03.0415.000; Tue, 27 Nov 2018 07:00:07 -0800 From: "Wiles, Keith" To: dpdk-dev CC: "Richardson, Bruce" , "Yigit, Ferruh" , Thomas Monjalon , "Ananyev, Konstantin" Thread-Topic: [dpdk-dev] [RFC] DFS (DPDK Filesystem) Thread-Index: AQHUX1g+rIvd8hoH1Eq/csPHM63tj6Vkiy8A Date: Tue, 27 Nov 2018 15:00:07 +0000 Message-ID: <897126DE-FC79-48FF-9B35-3E262EBDF093@intel.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.251.129.67] Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [RFC] DFS (DPDK Filesystem) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Nov 2018 15:00:10 -0000 > On Oct 8, 2018, at 5:42 PM, Wiles, Keith wrote: >=20 > Hi Everyone, >=20 > I would like to request comments on DFS as I presented at the DPDK summit= in Ireland. >=20 > As we know DPDK can be difficult to manage at run time and as DPDK become= s more dynamic we need to address how to configure and monitor DPDK and DPD= K-based applications. A possible solution is to use FUSE file system suppor= t already in the Linux kernel through a DPDK library to expose a directory = structure for anyone to be able to monitor and control the app by reading/w= riting files. >=20 > Primarily DPDK runs on Linux based systems, where FUSE is well supported.= FUSE was added to FreeBSD 10 which appears to be the same design as Linux.= Windows does not support FUSE directly from MS, but a open source design h= as a FUSE based set of wrapper APIs to help port FUSE based file systems. >=20 > Today the only way to get information out of DPDK is to use log files or = a command line within DPDK to display the information. The information is n= ot easy for an external application to collect or monitor. Using DFS any ap= plication can read and write into this virtual file system (FUSE) and retri= eve information from DPDK/Application via a raw ASCII text file and/or JSON= formatted text. >=20 > Data read from DFS can be static or dynamic e.g. version/copyright text w= ould be static, but port stats would be dynamic and generated when the user= requests the information. Writing new configuration data into DPDK can be = a simple number or text or a JSON data structure. Writing simple ASCII text= data is simple, but writing a long JSON or configuration file may some car= e, i.e. write a path to a file in DFS to the location of the configuration = and use that file path for the configuration. >=20 > DPDK command line is getting to be very complex and long, but we can elim= inate or greatly reduce the command line by using DFS to configure DPDK at = startup along with the application configuration. Having DPDK start up with= minimum resources and allow updating/creating a file in the DPDK file syst= em to configure DPDK would be easier. >=20 > The DFS is a FUSE filesystem anchored at /dpdk in the host file system, t= hen a directory is created under /dpdk for each DPDK instance or applicatio= n using - as a directory name format. Then it is up to DPDK/Appl= ication to create the directory structures, permissions and files within th= e directory mount point. >=20 > Currently I have a directory and files layout for DPDK, that needs more a= ttention as I was focusing on FUSE then the layout as it can be changed fai= rly easily. I can email the working documentation for DFS if needed please = send me a short email or I can post to the dev list. The docs also try to e= xplain the design and APIs used by the developer for creating files/directo= ries. The docs are still a work in progress and more attention needs to be = done as we progress. >=20 > DFS is just a library and not required to be used plus it does not modify= DPDK APIs or structures except for adding a few new dump routines for feat= ures that do not have one today. The file system or FUSE backend in DPDK is= a free floating thread at this time, but could be affinitized to a core if= required. >=20 > The FUSE protocol to the backend code (in DPDK) has a large number of ope= rations, but we have simplified the interface for developers to 5-6. The in= terface on top of libfuse3 is very simple and currently provides most of th= e needed APIs for managing the files and directories. The FUSE file system = with the added simpler API is dynamic and can be changed at runtime. The di= rectory/files can be constructed via APIs at startup or anytime after start= up. >=20 > I would like to put a repo someplace with DPDK/DFS say on GitHub to allow= others to play with the design in the short term. I also have a doc file f= or DFS if anyone wants to learn more information and will be in the repo as= well. If we agree I can then create a set of patches for DPDK. Well the 18.11 release is out the door, this one seemed pretty big and I lo= t of hard work was put into this one, thanks to everyone. It is time to try and get some feedback on this RFC. I know the code in the= repo is not finished, but I would like to get some folks to give it a try = and give feedback. I would like to submit the basic DFS code with some low = hanging items mostly readonly information for the 19.02 release. Here is the repo again you can look at and play with. The code is mostly re= ady for upstream but still more work needs to be done. I will try to update= the repo to the 18.11 release code based soon. https://github.com/pktgen/dfs-dpdk Thanks for giving this a review. >=20 > Regards, > Keith >=20 Regards, Keith