From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 833451B1AC for ; Sat, 23 Sep 2017 00:07:55 +0200 (CEST) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 22 Sep 2017 15:07:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.42,427,1500966000"; d="scan'208";a="154506859" Received: from irsmsx151.ger.corp.intel.com ([163.33.192.59]) by fmsmga005.fm.intel.com with ESMTP; 22 Sep 2017 15:07:52 -0700 Received: from irsmsx103.ger.corp.intel.com ([169.254.3.49]) by IRSMSX151.ger.corp.intel.com ([169.254.4.108]) with mapi id 14.03.0319.002; Fri, 22 Sep 2017 23:07:51 +0100 From: "Singh, Jasvinder" To: Thomas Monjalon , "Dumitrescu, Cristian" CC: "dev@dpdk.org" , "Yigit, Ferruh" Thread-Topic: [dpdk-dev] [PATCH v4 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Thread-Index: AQHTMiY9CqOu+Qt9hUyrQoilE3G2Z6LBd1Eg Date: Fri, 22 Sep 2017 22:07:50 +0000 Message-ID: <54CBAA185211B4429112C315DA58FF6D33258E64@IRSMSX103.ger.corp.intel.com> References: <20170811124929.118564-2-jasvinder.singh@intel.com> <20170918091015.82824-1-jasvinder.singh@intel.com> <9843308.sSVeHNjL0n@xps> In-Reply-To: <9843308.sSVeHNjL0n@xps> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZTlhNTQ3MjQtMTZmMS00ZjFkLTllZTgtZTVlNjgwNGI5MTU2IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6IllFZ1RYdHVqVkRMeE5ldStHbzhDQzhXSm9JV3Y0MHdwMm9oRnMwYXI4K009In0= x-ctpclassification: CTP_IC dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v4 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Sep 2017 22:07:56 -0000 Hi Thomas, > -----Original Message----- > From: Thomas Monjalon [mailto:thomas@monjalon.net] > Sent: Wednesday, September 20, 2017 4:36 PM > To: Singh, Jasvinder ; Dumitrescu, Cristian > > Cc: dev@dpdk.org; Yigit, Ferruh > Subject: Re: [dpdk-dev] [PATCH v4 0/4] net/softnic: sw fall-back pmd for > traffic mgmt and others >=20 > Hi, >=20 > 18/09/2017 11:10, Jasvinder Singh: > > The SoftNIC PMD is intended to provide SW fall-back options for > > specific ethdev APIs in a generic way to the NICs not supporting those > features. >=20 > I agree it is important to have a solution in DPDK to better manage SW > fallbacks. One question is to know whether we can implement and maintain > many solutions. We probably must choose only one solution. >=20 > I have not read the code. I am just interested in the design for now. > I think it is a smart idea but maybe less convenient than calling fallbac= k from > ethdev API glue code. My opinion has not changed since v1. IMHO, Calling fallback from ethdev API glue code suffers from scalability i= ssue. Let's assume the scenario of having another sw fallback implementation for TM or its specific feature is available. Wha= t will be the approach when we already have something glued in TM API?=20 The softnic could be considered as a placeholder for adding and enabling mo= re features at any granularity in addition to having complete TM feature. =20 > Thanks for the detailed explanations. Let's discuss below. >=20 > [...] > > * RX/TX: The app reads packets from/writes packets to the "soft" port > > instead of the "hard" port. The RX and TX queues of the "soft" port a= re > > thread safe, as any ethdev. >=20 > "thread safe as any ethdev"? > I would say the ethdev queues are not thread safe. [Jasvinder] Agree. > [...] > > * Meets the NFV vision: The app should be (almost) agnostic about the N= IC > > implementation (different vendors/models, HW-SW mix), the app should > not > > require changes to use different NICs, the app should use the same AP= I > > for all NICs. If a NIC does not implement a specific feature, the HW > > should be augmented with SW to meet the functionality while still > > preserving the same API. >=20 > This goal could also be achieved by adding the SW capability to the API. > After getting capabilities of a hardware, the app could set the capabilit= y of > some driver features to "SW fallback". > So the capability would become a tristate: > - not supported > - HW supported > - SW supported >=20 > The unique API goal is failed if we must manage two ports, the HW port fo= r > some features and the softnic port for other features. > You explain it in A5 below. [Jasvinder] TM API is agnostic to underlying implementation and allows app= lications to implement solution on SW, HW or hybrid of HW and SW at any granularity, and on any nu= mber of devices depending upon the availability of features. No Restriction. Thus, managing and confi= guring number of devices (physical and virtual) using high level api is at the disposal of application level framework. When softnic d= evice is enabled, application sends and receives packet from soft device instead of hard device as soft device implements the featu= res missing in hard device. It deosn't mean that softnic device should hide the hard device. However, it doesn't restrict applicatio= n to communicate directly with hard device. If desired, application can bypass the softnic device and send tx packet straight to hard device th= rough the queues not used by soft device. > [...] > > Example: Create "soft" port for "hard" port "0000:04:00.1", enable the > > TM feature with default settings: > > --vdev 'net_softnic0,hard_name=3D0000:04:00.1,soft_tm=3Don' >=20 > So the app will use only the vdev net_softnic0 which will forward packets= to > 0000:04:00.1? > Can we say in this example that net_softnic0 owns 0000:04:00.1? > Probably not, because the config of the HW must be done separately (cf. Q= 5). > See my "ownership proposal": > http://dpdk.org/ml/archives/dev/2017-September/074656.html >=20 > The issue I see in this example is that we must define how to enable ever= y > features. It should be equivalent to defining the ethdev capabilities. > In this example, the option soft_tm=3Don is probably not enough fine-grai= n. > We could support some parts of TM API in HW and other parts in SW. >=20 [Jasvinder] - This is one instance where the complete hierarchical schedule= r is presented as software fallback. But the approach doesn't restrict to add more features (at any granularity) to soft= nic and enable them altogether naming arguments during device creation. > [...] > > Q3: Why not change the "hard" device (and keep a single device) instead= of > > creating a new "soft" device (and thus having two devices)? > > A3: This is not possible with the current librte_ether ethdev > > implementation. The ethdev->dev_ops are defined as constant structu= re, > > so it cannot be changed per device (nor per PMD). The new ops also > > need memory space to store their context data structures, which > > requires updating the ethdev->data->dev_private of the existing > > device; at best, maybe a resize of ethdev->data->dev_private could = be > > done, assuming that librte_ether will introduce a way to find out i= ts > > size, but this cannot be done while device is running. Other side > > effects might exist, as the changes are very intrusive, plus it lik= ely > > needs more changes in librte_ether. >=20 > Q3 is about calling SW fallback from the driver code, right? >=20 > We must not implement fallbacks in drivers because it would hide it to th= e > application. > If a feature is not available in hardware, the application can choose to = bypass > this feature or integrate the fallback in its own workflow. [Jasvinder]: Naturally, if hard device has the TM feature, then TM specific= ops which are invoked API function are implemented by its pmd. Similar approach has been followed in sw fallback solution as softnic port = that complements the hard device. =20 > > Q4: Why not call the SW fall-back dev_ops directly in librte_ether for > > devices which do not support the specific feature? If the device > > supports the capability, let's call its dev_ops, otherwise call the > > SW fall-back dev_ops. > > A4: First, similar reasons to Q&A3. This fixes the need to change > > ethdev->dev_ops of the device, but it does not do anything to fix t= he > > other significant issue of where to store the context data structur= es > > needed by the SW fall-back functions (which, in this approach, are > > called implicitly by librte_ether). > > Second, the SW fall-back options should not be restricted arbitrari= ly > > by the librte_ether library, the decision should belong to the app. > > For example, the TM SW fall-back should not be limited to only > > librte_sched, which (like any SW fall-back) is limited to a specifi= c > > hierarchy and feature set, it cannot do any possible hierarchy. If > > alternatives exist, the one to use should be picked by the app, not= by > > the ethdev layer. >=20 > Q4 is about calling SW callback from the API glue code, right? >=20 > We could summarize Q3/Q4 as "it could be done but we propose another > way". > I think we must consider the pros and cons of both approaches from a user > perspective. > I agree the application must decide which fallback to use. > We could propose one fallback in ethdev which can be enabled explicitly (= see > my tristate capabilities proposal above). [Jasvinder] As explained above as well, the approach of sticking sw solutio= n with API will create issue of scalability. How two different SW solution will coexist or = for instance N software solutions. That's why the softnic (virtual device) is proposed as an alternative which= could be extended to include and enable features. =20 > > Q5: Why is the app required to continue to configure both the "hard" an= d > > the "soft" devices even after the "soft" device has been created? W= hy > > not hiding the "hard" device under the "soft" device and have the > > "soft" device configure the "hard" device under the hood? > > A5: This was the approach tried in the V2 of this patch set (overlay > > "soft" device taking over the configuration of the underlay "hard" > > device) and eventually dropped due to increased complexity of havin= g > > to keep the configuration of two distinct devices in sync with > > librte_ether implementation that is not friendly towards such > > approach. Basically, each ethdev API call for the overlay device > > needs to configure the overlay device, invoke the same configuratio= n > > with possibly modified parameters for the underlay device, then res= ume > > the configuration of overlay device, turning this into a device > > emulation project. > > V2 minuses: increased complexity (deal with two devices at same tim= e); > > need to implement every ethdev API, even those not needed for the > scope > > of SW fall-back; intrusive; sometimes have to silently take decisio= ns > > that should be left to the app. > > V3 pluses: lower complexity (only one device); only need to impleme= nt > > those APIs that are in scope of the SW fall-back; non-intrusive (de= al > > with "hard" device through ethdev API); app decisions taken by the = app > > in an explicit way. >=20 > I think it is breaking what you call the NFV vision in several places. [Jasvinder] Mentioning nfv vision is about hiding heterogeneous implementat= ion (HW,SW, HW-SW hybrid) under the abstraction layer provided by TM API instead of restricting app to use API for specific port. >=20 > [...] > > 9. [rte_ring proliferation] Thread safety requirements for ethdev > > RX/TXqueues require an rte_ring to be used for every RX/TX queue > > of each "soft" ethdev. This rte_ring proliferation unnecessarily > > increases the memory footprint and lowers performance, especiall= y > > when each "soft" ethdev ends up on a different CPU core (ping-po= ng > > of cache lines). >=20 > I am curious to understand why you consider thread safety as a requiremen= t > for queues. No need to reply here, the question is already asked at the > beginning of this email ;)