From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 77B471B195 for ; Tue, 9 Oct 2018 00:43:05 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Oct 2018 15:43:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,358,1534834800"; d="scan'208";a="90288676" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga003.jf.intel.com with ESMTP; 08 Oct 2018 15:42:51 -0700 Received: from fmsmsx113.amr.corp.intel.com (10.18.116.7) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 8 Oct 2018 15:42:51 -0700 Received: from fmsmsx117.amr.corp.intel.com ([169.254.3.34]) by FMSMSX113.amr.corp.intel.com ([169.254.13.112]) with mapi id 14.03.0319.002; Mon, 8 Oct 2018 15:42:51 -0700 From: "Wiles, Keith" To: dpdk-dev Thread-Topic: [RFC] DFS (DPDK Filesystem) Thread-Index: AQHUX1g+3LgzLEXi8EiFigRgY1m60g== Date: Mon, 8 Oct 2018 22:42:50 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.255.69.176] Content-Type: text/plain; charset="us-ascii" Content-ID: <0B5069CB9675CA44AB2FD47BC700EF35@intel.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [dpdk-dev] [RFC] DFS (DPDK Filesystem) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Oct 2018 22:43:06 -0000 Hi Everyone, I would like to request comments on DFS as I presented at the DPDK summit i= n Ireland. =20 As we know DPDK can be difficult to manage at run time and as DPDK becomes = more dynamic we need to address how to configure and monitor DPDK and DPDK-= based applications. A possible solution is to use FUSE file system support = already in the Linux kernel through a DPDK library to expose a directory st= ructure for anyone to be able to monitor and control the app by reading/wri= ting files. =20 Primarily DPDK runs on Linux based systems, where FUSE is well supported. F= USE was added to FreeBSD 10 which appears to be the same design as Linux. W= indows does not support FUSE directly from MS, but a open source design has= a FUSE based set of wrapper APIs to help port FUSE based file systems. =20 Today the only way to get information out of DPDK is to use log files or a = command line within DPDK to display the information. The information is not= easy for an external application to collect or monitor. Using DFS any appl= ication can read and write into this virtual file system (FUSE) and retriev= e information from DPDK/Application via a raw ASCII text file and/or JSON f= ormatted text. =20 Data read from DFS can be static or dynamic e.g. version/copyright text wou= ld be static, but port stats would be dynamic and generated when the user r= equests the information. Writing new configuration data into DPDK can be a = simple number or text or a JSON data structure. Writing simple ASCII text d= ata is simple, but writing a long JSON or configuration file may some care,= i.e. write a path to a file in DFS to the location of the configuration an= d use that file path for the configuration. =20 DPDK command line is getting to be very complex and long, but we can elimin= ate or greatly reduce the command line by using DFS to configure DPDK at st= artup along with the application configuration. Having DPDK start up with m= inimum resources and allow updating/creating a file in the DPDK file system= to configure DPDK would be easier. =20 The DFS is a FUSE filesystem anchored at /dpdk in the host file system, the= n a directory is created under /dpdk for each DPDK instance or application = using - as a directory name format. Then it is up to DPDK/Applic= ation to create the directory structures, permissions and files within the = directory mount point. =20 Currently I have a directory and files layout for DPDK, that needs more att= ention as I was focusing on FUSE then the layout as it can be changed fairl= y easily. I can email the working documentation for DFS if needed please se= nd me a short email or I can post to the dev list. The docs also try to exp= lain the design and APIs used by the developer for creating files/directori= es. The docs are still a work in progress and more attention needs to be do= ne as we progress. =20 DFS is just a library and not required to be used plus it does not modify D= PDK APIs or structures except for adding a few new dump routines for featur= es that do not have one today. The file system or FUSE backend in DPDK is a= free floating thread at this time, but could be affinitized to a core if r= equired. =20 The FUSE protocol to the backend code (in DPDK) has a large number of opera= tions, but we have simplified the interface for developers to 5-6. The inte= rface on top of libfuse3 is very simple and currently provides most of the = needed APIs for managing the files and directories. The FUSE file system wi= th the added simpler API is dynamic and can be changed at runtime. The dire= ctory/files can be constructed via APIs at startup or anytime after startup= . =20 I would like to put a repo someplace with DPDK/DFS say on GitHub to allow o= thers to play with the design in the short term. I also have a doc file for= DFS if anyone wants to learn more information and will be in the repo as w= ell. If we agree I can then create a set of patches for DPDK. Regards, Keith