From: "Singh, Jasvinder" <jasvinder.singh@intel.com>
To: "Yigit, Ferruh" <ferruh.yigit@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>,
"thomas@monjalon.net" <thomas@monjalon.net>,
"Lu, Wenzhuo" <wenzhuo.lu@intel.com>
Subject: Re: [dpdk-dev] [PATCH v8 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others
Date: Tue, 10 Oct 2017 19:09:19 +0000 [thread overview]
Message-ID: <54CBAA185211B4429112C315DA58FF6D33268C47@IRSMSX103.ger.corp.intel.com> (raw)
In-Reply-To: <1396e222-0c2b-bc8e-1773-174fc199de70@intel.com>
> -----Original Message-----
> From: Yigit, Ferruh
> Sent: Tuesday, October 10, 2017 7:31 PM
> To: Singh, Jasvinder <jasvinder.singh@intel.com>; dev@dpdk.org
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>;
> thomas@monjalon.net; Lu, Wenzhuo <wenzhuo.lu@intel.com>
> Subject: Re: [PATCH v8 0/5] net/softnic: sw fall-back pmd for traffic mgmt
> and others
>
> On 10/10/2017 11:18 AM, Jasvinder Singh wrote:
> > The SoftNIC PMD is intended to provide SW fall-back options for
> > specific ethdev APIs in a generic way to the NICs not supporting those
> features.
> >
> > Currently, the only implemented ethdev API is Traffic Management (TM),
> > but other ethdev APIs such as rte_flow, traffic metering & policing,
> > etc can be easily implemented.
> >
> > Overview:
> > * Generic: The SoftNIC PMD works with any "hard" PMD that implements
> the
> > ethdev API. It does not change the "hard" PMD in any way.
> > * Creation: For any given "hard" ethdev port, the user can decide to
> > create an associated "soft" ethdev port to drive the "hard" port. The
> > "soft" port is a virtual device that can be created at app start-up
> > through EAL vdev arg or later through the virtual device API.
> > * Configuration: The app explicitly decides which features are to be
> > enabled on the "soft" port and which features are still to be used from
> > the "hard" port. The app continues to explicitly configure both the
> > "hard" and the "soft" ports after the creation of the "soft" port.
> > * RX/TX: The app reads packets from/writes packets to the "soft" port
> > instead of the "hard" port. The RX and TX queues of the "soft" port are
> > thread safe, as any ethdev.
> > * Execution: The "soft" port is a feature-rich NIC implemented by the CPU,
> > so the run function of the "soft" port has to be executed by the CPU in
> > order to get packets moving between "hard" port and the app.
> > * Meets the NFV vision: The app should be (almost) agnostic about the NIC
> > implementation (different vendors/models, HW-SW mix), the app should
> not
> > require changes to use different NICs, the app should use the same API
> > for all NICs. If a NIC does not implement a specific feature, the HW
> > should be augmented with SW to meet the functionality while still
> > preserving the same API.
> >
> > Traffic Management SW fall-back overview:
> > * Implements the ethdev traffic management API (rte_tm.h).
> > * Based on the existing librte_sched DPDK library.
> >
> > Example: Create "soft" port for "hard" port "0000:04:00.1", enable the
> > TM feature with default settings:
> > --vdev 'net_softnic0,hard_name=0000:04:00.1,soft_tm=on'
> >
> > Q1: Why generic name, if only TM is supported (for now)?
> > A1: The intention is to have SoftNIC PMD implement many other (all?)
> > ethdev APIs under a single "ideal" ethdev, hence the generic name.
> > The initial motivation is TM API, but the mechanism is generic and can
> > be used for many other ethdev APIs. Somebody looking to provide SW
> > fall-back for other ethdev API is likely to end up inventing the same,
> > hence it would be good to consolidate all under a single PMD and have
> > the user explicitly enable/disable the features it needs for each
> > "soft" device.
> >
> > Q2: Are there any performance requirements for SoftNIC?
> > A2: Yes, performance should be great/decent for every feature, otherwise
> > the SW fall-back is unusable, thus useless.
> >
> > Q3: Why not change the "hard" device (and keep a single device) instead of
> > creating a new "soft" device (and thus having two devices)?
> > A3: This is not possible with the current librte_ether ethdev
> > implementation. The ethdev->dev_ops are defined as constant structure,
> > so it cannot be changed per device (nor per PMD). The new ops also
> > need memory space to store their context data structures, which
> > requires updating the ethdev->data->dev_private of the existing
> > device; at best, maybe a resize of ethdev->data->dev_private could be
> > done, assuming that librte_ether will introduce a way to find out its
> > size, but this cannot be done while device is running. Other side
> > effects might exist, as the changes are very intrusive, plus it likely
> > needs more changes in librte_ether.
> >
> > Q4: Why not call the SW fall-back dev_ops directly in librte_ether for
> > devices which do not support the specific feature? If the device
> > supports the capability, let's call its dev_ops, otherwise call the
> > SW fall-back dev_ops.
> > A4: First, similar reasons to Q&A3. This fixes the need to change
> > ethdev->dev_ops of the device, but it does not do anything to fix the
> > other significant issue of where to store the context data structures
> > needed by the SW fall-back functions (which, in this approach, are
> > called implicitly by librte_ether).
> > Second, the SW fall-back options should not be restricted arbitrarily
> > by the librte_ether library, the decision should belong to the app.
> > For example, the TM SW fall-back should not be limited to only
> > librte_sched, which (like any SW fall-back) is limited to a specific
> > hierarchy and feature set, it cannot do any possible hierarchy. If
> > alternatives exist, the one to use should be picked by the app, not by
> > the ethdev layer.
> >
> > Q5: Why is the app required to continue to configure both the "hard" and
> > the "soft" devices even after the "soft" device has been created? Why
> > not hiding the "hard" device under the "soft" device and have the
> > "soft" device configure the "hard" device under the hood?
> > A5: This was the approach tried in the V2 of this patch set (overlay
> > "soft" device taking over the configuration of the underlay "hard"
> > device) and eventually dropped due to increased complexity of having
> > to keep the configuration of two distinct devices in sync with
> > librte_ether implementation that is not friendly towards such
> > approach. Basically, each ethdev API call for the overlay device
> > needs to configure the overlay device, invoke the same configuration
> > with possibly modified parameters for the underlay device, then resume
> > the configuration of overlay device, turning this into a device
> > emulation project.
> > V2 minuses: increased complexity (deal with two devices at same time);
> > need to implement every ethdev API, even those not needed for the
> scope
> > of SW fall-back; intrusive; sometimes have to silently take decisions
> > that should be left to the app.
> > V3 pluses: lower complexity (only one device); only need to implement
> > those APIs that are in scope of the SW fall-back; non-intrusive (deal
> > with "hard" device through ethdev API); app decisions taken by the app
> > in an explicit way.
> >
> > Q6: Why expose the SW fall-back in a PMD and not in a SW library?
> > A6: The SW fall-back for an ethdev API has to implement that specific
> > ethdev API, (hence expose an ethdev object through a PMD), as opposed
> > to providing a different API. This approach allows the app to use the
> > same API (NFV vision). For example, we already have a library for TM
> > SW fall-back (librte_sched) that can be called directly by the apps
> > that need to call it outside of ethdev context (use-cases exist), but
> > an app that works with TM-aware NICs through the ethdev TM API would
> > have to be changed significantly in order to work with different
> > TM-agnostic NICs through the librte_sched API.
> >
> > Q7: Why have all the SW fall-backs in a single PMD? Why not develop
> > the SW fall-back for each different ethdev API in a separate PMD, then
> > create a chain of "soft" devices for each "hard" device? Potentially,
> > this results in smaller size PMDs that are easier to maintain.
> > A7: Arguments for single ethdev/PMD and against chain of ethdevs/PMDs:
> > 1. All the existing PMDs for HW NICs implement a lot of features under
> > the same PMD, so there is no reason for single PMD approach to break
> > code modularity. See the V3 code, a lot of care has been taken for
> > code modularity.
> > 2. We should avoid the proliferation of SW PMDs.
> > 3. A single device should be handled by a single PMD.
> > 4. People are used with feature-rich PMDs, not with single-feature
> > PMDs, so we change of mindset?
> > 5. [Configuration nightmare] A chain of "soft" devices attached to
> > single "hard" device requires the app to be aware that the N "soft"
> > devices in the chain plus the "hard" device refer to the same HW
> > device, and which device should be invoked to configure which
> > feature. Also the length of the chain and functionality of each
> > link is different for each HW device. This breaks the requirement
> > of preserving the same API while working with different NICs (NFV).
> > This most likely results in a configuration nightmare, nobody is
> > going to seriously use this.
> > 6. [Feature inter-dependecy] Sometimes different features need to be
> > configured and executed together (e.g. share the same set of
> > resources, are inter-dependent, etc), so it is better and more
> > performant to do them in the same ethdev/PMD.
> > 7. [Code duplication] There is a lot of duplication in the
> > configuration code for the chain of ethdevs approach. The ethdev
> > dev_configure, rx_queue_setup, tx_queue_setup API functions have to
> > be implemented per device, and they become
> meaningless/inconsistent
> > with the chain approach.
> > 8. [Data structure duplication] The per device data structures have to
> > be duplicated and read repeatedly for each "soft" ethdev. The
> > ethdev device, dev_private, data, per RX/TX queue data structures
> > have to be replicated per "soft" device. They have to be re-read for
> > each stage, so the same cache misses are now multiplied with the
> > number of stages in the chain.
> > 9. [rte_ring proliferation] Thread safety requirements for ethdev
> > RX/TXqueues require an rte_ring to be used for every RX/TX queue
> > of each "soft" ethdev. This rte_ring proliferation unnecessarily
> > increases the memory footprint and lowers performance, especially
> > when each "soft" ethdev ends up on a different CPU core (ping-pong
> > of cache lines).
> > 10.[Meta-data proliferation] A chain of ethdevs is likely to result
> > in proliferation of meta-data that has to be passed between the
> > ethdevs (e.g. policing needs the output of flow classification),
> > which results in more cache line ping-pong between cores, hence
> > performance drops.
> >
> > Cristian Dumitrescu (4):
> > Jasvinder Singh (4):
> > net/softnic: add softnic PMD
> > net/softnic: add traffic management support
> > net/softnic: add TM capabilities ops
> > net/softnic: add TM hierarchy related ops
> >
> > Jasvinder Singh (1):
> > app/testpmd: add traffic management forwarding mode
>
> Series applied to dpdk-next-net/master, thanks.
>
> (Was getting same build error from previous version, fixed while applying
> please confirm the pushed commit.
> Also waiting for testpmd document to squash this set later)
I have pushed the documentation patch. Thanks.
next prev parent reply other threads:[~2017-10-10 19:09 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-26 18:11 [dpdk-dev] [PATCH 0/2] net/softnic: sw fall-back for traffic management Jasvinder Singh
2017-05-26 18:11 ` [dpdk-dev] [PATCH 1/2] net/softnic: add softnic PMD " Jasvinder Singh
2017-06-26 16:43 ` [dpdk-dev] [PATCH v2 0/2] net/softnic: sw fall-back " Jasvinder Singh
2017-06-26 16:43 ` [dpdk-dev] [PATCH v2 1/2] net/softnic: add softnic PMD " Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 1/4] net/softnic: add softnic PMD Jasvinder Singh
2017-09-05 14:53 ` Ferruh Yigit
2017-09-08 9:30 ` Singh, Jasvinder
2017-09-08 9:48 ` Ferruh Yigit
2017-09-08 10:42 ` Singh, Jasvinder
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 1/4] net/softnic: add softnic PMD Jasvinder Singh
2017-09-18 16:58 ` Singh, Jasvinder
2017-09-18 19:09 ` Thomas Monjalon
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 2/4] net/softnic: add traffic management support Jasvinder Singh
2017-09-25 1:58 ` Lu, Wenzhuo
2017-09-28 8:14 ` Singh, Jasvinder
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-06 16:59 ` [dpdk-dev] [PATCH v6 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-10-06 16:59 ` [dpdk-dev] [PATCH v6 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-10-09 20:18 ` Ferruh Yigit
2017-10-10 10:08 ` Singh, Jasvinder
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-10-11 23:18 ` Thomas Monjalon
2017-10-12 8:22 ` Singh, Jasvinder
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-10-10 18:24 ` Ferruh Yigit
2017-10-10 18:31 ` [dpdk-dev] [PATCH v8 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Ferruh Yigit
2017-10-10 19:09 ` Singh, Jasvinder [this message]
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-10-09 20:17 ` Ferruh Yigit
2017-10-10 10:07 ` Singh, Jasvinder
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-10-06 18:57 ` [dpdk-dev] [PATCH v6 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Ferruh Yigit
2017-10-09 11:32 ` Singh, Jasvinder
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 3/4] net/softnic: add TM capabilities ops Jasvinder Singh
2017-09-25 2:33 ` Lu, Wenzhuo
2017-09-28 8:16 ` Singh, Jasvinder
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 4/4] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-09-25 7:14 ` Lu, Wenzhuo
2017-09-28 8:39 ` Singh, Jasvinder
2017-09-20 15:35 ` [dpdk-dev] [PATCH v4 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Thomas Monjalon
2017-09-22 22:07 ` Singh, Jasvinder
2017-10-06 10:40 ` Dumitrescu, Cristian
2017-10-06 12:13 ` Thomas Monjalon
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 2/4] net/softnic: add traffic management support Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 3/4] net/softnic: add TM capabilities ops Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 4/4] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-09-08 17:08 ` [dpdk-dev] [PATCH v3 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Dumitrescu, Cristian
2017-06-26 16:43 ` [dpdk-dev] [PATCH v2 2/2] net/softnic: add traffic management ops Jasvinder Singh
2017-05-26 18:11 ` [dpdk-dev] [PATCH " Jasvinder Singh
2017-06-07 14:32 ` [dpdk-dev] [PATCH 0/2] net/softnic: sw fall-back for traffic management Thomas Monjalon
2017-06-08 13:27 ` Dumitrescu, Cristian
2017-06-08 13:59 ` Thomas Monjalon
2017-06-08 15:27 ` Dumitrescu, Cristian
2017-06-08 16:16 ` Thomas Monjalon
2017-06-08 16:43 ` Dumitrescu, Cristian
2017-07-04 23:48 ` Thomas Monjalon
2017-07-05 9:32 ` Dumitrescu, Cristian
2017-07-05 10:17 ` Thomas Monjalon
2017-08-11 15:28 ` Stephen Hemminger
2017-08-11 16:22 ` Dumitrescu, Cristian
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54CBAA185211B4429112C315DA58FF6D33268C47@IRSMSX103.ger.corp.intel.com \
--to=jasvinder.singh@intel.com \
--cc=cristian.dumitrescu@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=thomas@monjalon.net \
--cc=wenzhuo.lu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).