From: Ferruh Yigit <ferruh.yigit@intel.com>
To: Jasvinder Singh <jasvinder.singh@intel.com>, dev@dpdk.org
Cc: cristian.dumitrescu@intel.com, thomas@monjalon.net, wenzhuo.lu@intel.com
Subject: Re: [dpdk-dev] [PATCH v8 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others
Date: Tue, 10 Oct 2017 19:31:23 +0100 [thread overview]
Message-ID: <1396e222-0c2b-bc8e-1773-174fc199de70@intel.com> (raw)
In-Reply-To: <20171010101818.146523-1-jasvinder.singh@intel.com>
On 10/10/2017 11:18 AM, Jasvinder Singh wrote:
> The SoftNIC PMD is intended to provide SW fall-back options for specific
> ethdev APIs in a generic way to the NICs not supporting those features.
>
> Currently, the only implemented ethdev API is Traffic Management (TM),
> but other ethdev APIs such as rte_flow, traffic metering & policing, etc
> can be easily implemented.
>
> Overview:
> * Generic: The SoftNIC PMD works with any "hard" PMD that implements the
> ethdev API. It does not change the "hard" PMD in any way.
> * Creation: For any given "hard" ethdev port, the user can decide to
> create an associated "soft" ethdev port to drive the "hard" port. The
> "soft" port is a virtual device that can be created at app start-up
> through EAL vdev arg or later through the virtual device API.
> * Configuration: The app explicitly decides which features are to be
> enabled on the "soft" port and which features are still to be used from
> the "hard" port. The app continues to explicitly configure both the
> "hard" and the "soft" ports after the creation of the "soft" port.
> * RX/TX: The app reads packets from/writes packets to the "soft" port
> instead of the "hard" port. The RX and TX queues of the "soft" port are
> thread safe, as any ethdev.
> * Execution: The "soft" port is a feature-rich NIC implemented by the CPU,
> so the run function of the "soft" port has to be executed by the CPU in
> order to get packets moving between "hard" port and the app.
> * Meets the NFV vision: The app should be (almost) agnostic about the NIC
> implementation (different vendors/models, HW-SW mix), the app should not
> require changes to use different NICs, the app should use the same API
> for all NICs. If a NIC does not implement a specific feature, the HW
> should be augmented with SW to meet the functionality while still
> preserving the same API.
>
> Traffic Management SW fall-back overview:
> * Implements the ethdev traffic management API (rte_tm.h).
> * Based on the existing librte_sched DPDK library.
>
> Example: Create "soft" port for "hard" port "0000:04:00.1", enable the TM
> feature with default settings:
> --vdev 'net_softnic0,hard_name=0000:04:00.1,soft_tm=on'
>
> Q1: Why generic name, if only TM is supported (for now)?
> A1: The intention is to have SoftNIC PMD implement many other (all?)
> ethdev APIs under a single "ideal" ethdev, hence the generic name.
> The initial motivation is TM API, but the mechanism is generic and can
> be used for many other ethdev APIs. Somebody looking to provide SW
> fall-back for other ethdev API is likely to end up inventing the same,
> hence it would be good to consolidate all under a single PMD and have
> the user explicitly enable/disable the features it needs for each
> "soft" device.
>
> Q2: Are there any performance requirements for SoftNIC?
> A2: Yes, performance should be great/decent for every feature, otherwise
> the SW fall-back is unusable, thus useless.
>
> Q3: Why not change the "hard" device (and keep a single device) instead of
> creating a new "soft" device (and thus having two devices)?
> A3: This is not possible with the current librte_ether ethdev
> implementation. The ethdev->dev_ops are defined as constant structure,
> so it cannot be changed per device (nor per PMD). The new ops also
> need memory space to store their context data structures, which
> requires updating the ethdev->data->dev_private of the existing
> device; at best, maybe a resize of ethdev->data->dev_private could be
> done, assuming that librte_ether will introduce a way to find out its
> size, but this cannot be done while device is running. Other side
> effects might exist, as the changes are very intrusive, plus it likely
> needs more changes in librte_ether.
>
> Q4: Why not call the SW fall-back dev_ops directly in librte_ether for
> devices which do not support the specific feature? If the device
> supports the capability, let's call its dev_ops, otherwise call the
> SW fall-back dev_ops.
> A4: First, similar reasons to Q&A3. This fixes the need to change
> ethdev->dev_ops of the device, but it does not do anything to fix the
> other significant issue of where to store the context data structures
> needed by the SW fall-back functions (which, in this approach, are
> called implicitly by librte_ether).
> Second, the SW fall-back options should not be restricted arbitrarily
> by the librte_ether library, the decision should belong to the app.
> For example, the TM SW fall-back should not be limited to only
> librte_sched, which (like any SW fall-back) is limited to a specific
> hierarchy and feature set, it cannot do any possible hierarchy. If
> alternatives exist, the one to use should be picked by the app, not by
> the ethdev layer.
>
> Q5: Why is the app required to continue to configure both the "hard" and
> the "soft" devices even after the "soft" device has been created? Why
> not hiding the "hard" device under the "soft" device and have the
> "soft" device configure the "hard" device under the hood?
> A5: This was the approach tried in the V2 of this patch set (overlay
> "soft" device taking over the configuration of the underlay "hard"
> device) and eventually dropped due to increased complexity of having
> to keep the configuration of two distinct devices in sync with
> librte_ether implementation that is not friendly towards such
> approach. Basically, each ethdev API call for the overlay device
> needs to configure the overlay device, invoke the same configuration
> with possibly modified parameters for the underlay device, then resume
> the configuration of overlay device, turning this into a device
> emulation project.
> V2 minuses: increased complexity (deal with two devices at same time);
> need to implement every ethdev API, even those not needed for the scope
> of SW fall-back; intrusive; sometimes have to silently take decisions
> that should be left to the app.
> V3 pluses: lower complexity (only one device); only need to implement
> those APIs that are in scope of the SW fall-back; non-intrusive (deal
> with "hard" device through ethdev API); app decisions taken by the app
> in an explicit way.
>
> Q6: Why expose the SW fall-back in a PMD and not in a SW library?
> A6: The SW fall-back for an ethdev API has to implement that specific
> ethdev API, (hence expose an ethdev object through a PMD), as opposed
> to providing a different API. This approach allows the app to use the
> same API (NFV vision). For example, we already have a library for TM
> SW fall-back (librte_sched) that can be called directly by the apps
> that need to call it outside of ethdev context (use-cases exist), but
> an app that works with TM-aware NICs through the ethdev TM API would
> have to be changed significantly in order to work with different
> TM-agnostic NICs through the librte_sched API.
>
> Q7: Why have all the SW fall-backs in a single PMD? Why not develop
> the SW fall-back for each different ethdev API in a separate PMD, then
> create a chain of "soft" devices for each "hard" device? Potentially,
> this results in smaller size PMDs that are easier to maintain.
> A7: Arguments for single ethdev/PMD and against chain of ethdevs/PMDs:
> 1. All the existing PMDs for HW NICs implement a lot of features under
> the same PMD, so there is no reason for single PMD approach to break
> code modularity. See the V3 code, a lot of care has been taken for
> code modularity.
> 2. We should avoid the proliferation of SW PMDs.
> 3. A single device should be handled by a single PMD.
> 4. People are used with feature-rich PMDs, not with single-feature
> PMDs, so we change of mindset?
> 5. [Configuration nightmare] A chain of "soft" devices attached to
> single "hard" device requires the app to be aware that the N "soft"
> devices in the chain plus the "hard" device refer to the same HW
> device, and which device should be invoked to configure which
> feature. Also the length of the chain and functionality of each
> link is different for each HW device. This breaks the requirement
> of preserving the same API while working with different NICs (NFV).
> This most likely results in a configuration nightmare, nobody is
> going to seriously use this.
> 6. [Feature inter-dependecy] Sometimes different features need to be
> configured and executed together (e.g. share the same set of
> resources, are inter-dependent, etc), so it is better and more
> performant to do them in the same ethdev/PMD.
> 7. [Code duplication] There is a lot of duplication in the
> configuration code for the chain of ethdevs approach. The ethdev
> dev_configure, rx_queue_setup, tx_queue_setup API functions have to
> be implemented per device, and they become meaningless/inconsistent
> with the chain approach.
> 8. [Data structure duplication] The per device data structures have to
> be duplicated and read repeatedly for each "soft" ethdev. The
> ethdev device, dev_private, data, per RX/TX queue data structures
> have to be replicated per "soft" device. They have to be re-read for
> each stage, so the same cache misses are now multiplied with the
> number of stages in the chain.
> 9. [rte_ring proliferation] Thread safety requirements for ethdev
> RX/TXqueues require an rte_ring to be used for every RX/TX queue
> of each "soft" ethdev. This rte_ring proliferation unnecessarily
> increases the memory footprint and lowers performance, especially
> when each "soft" ethdev ends up on a different CPU core (ping-pong
> of cache lines).
> 10.[Meta-data proliferation] A chain of ethdevs is likely to result
> in proliferation of meta-data that has to be passed between the
> ethdevs (e.g. policing needs the output of flow classification),
> which results in more cache line ping-pong between cores, hence
> performance drops.
>
> Cristian Dumitrescu (4):
> Jasvinder Singh (4):
> net/softnic: add softnic PMD
> net/softnic: add traffic management support
> net/softnic: add TM capabilities ops
> net/softnic: add TM hierarchy related ops
>
> Jasvinder Singh (1):
> app/testpmd: add traffic management forwarding mode
Series applied to dpdk-next-net/master, thanks.
(Was getting same build error from previous version, fixed while
applying please confirm the pushed commit.
Also waiting for testpmd document to squash this set later)
next prev parent reply other threads:[~2017-10-10 18:31 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-26 18:11 [dpdk-dev] [PATCH 0/2] net/softnic: sw fall-back for traffic management Jasvinder Singh
2017-05-26 18:11 ` [dpdk-dev] [PATCH 1/2] net/softnic: add softnic PMD " Jasvinder Singh
2017-06-26 16:43 ` [dpdk-dev] [PATCH v2 0/2] net/softnic: sw fall-back " Jasvinder Singh
2017-06-26 16:43 ` [dpdk-dev] [PATCH v2 1/2] net/softnic: add softnic PMD " Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 1/4] net/softnic: add softnic PMD Jasvinder Singh
2017-09-05 14:53 ` Ferruh Yigit
2017-09-08 9:30 ` Singh, Jasvinder
2017-09-08 9:48 ` Ferruh Yigit
2017-09-08 10:42 ` Singh, Jasvinder
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 1/4] net/softnic: add softnic PMD Jasvinder Singh
2017-09-18 16:58 ` Singh, Jasvinder
2017-09-18 19:09 ` Thomas Monjalon
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 2/4] net/softnic: add traffic management support Jasvinder Singh
2017-09-25 1:58 ` Lu, Wenzhuo
2017-09-28 8:14 ` Singh, Jasvinder
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-06 16:59 ` [dpdk-dev] [PATCH v6 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-10-06 16:59 ` [dpdk-dev] [PATCH v6 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-10-09 20:18 ` Ferruh Yigit
2017-10-10 10:08 ` Singh, Jasvinder
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 1/5] net/softnic: add softnic PMD Jasvinder Singh
2017-10-11 23:18 ` Thomas Monjalon
2017-10-12 8:22 ` Singh, Jasvinder
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-10-10 10:18 ` [dpdk-dev] [PATCH v8 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-10-10 18:24 ` Ferruh Yigit
2017-10-10 18:31 ` Ferruh Yigit [this message]
2017-10-10 19:09 ` [dpdk-dev] [PATCH v8 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Singh, Jasvinder
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-10-09 12:58 ` [dpdk-dev] [PATCH v7 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-10-09 20:17 ` Ferruh Yigit
2017-10-10 10:07 ` Singh, Jasvinder
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 2/5] net/softnic: add traffic management support Jasvinder Singh
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-10-06 17:00 ` [dpdk-dev] [PATCH v6 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-10-06 18:57 ` [dpdk-dev] [PATCH v6 0/5] net/softnic: sw fall-back pmd for traffic mgmt and others Ferruh Yigit
2017-10-09 11:32 ` Singh, Jasvinder
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 3/5] net/softnic: add TM capabilities ops Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 4/5] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-09-29 14:04 ` [dpdk-dev] [PATCH v5 5/5] app/testpmd: add traffic management forwarding mode Jasvinder Singh
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 3/4] net/softnic: add TM capabilities ops Jasvinder Singh
2017-09-25 2:33 ` Lu, Wenzhuo
2017-09-28 8:16 ` Singh, Jasvinder
2017-09-18 9:10 ` [dpdk-dev] [PATCH v4 4/4] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-09-25 7:14 ` Lu, Wenzhuo
2017-09-28 8:39 ` Singh, Jasvinder
2017-09-20 15:35 ` [dpdk-dev] [PATCH v4 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Thomas Monjalon
2017-09-22 22:07 ` Singh, Jasvinder
2017-10-06 10:40 ` Dumitrescu, Cristian
2017-10-06 12:13 ` Thomas Monjalon
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 2/4] net/softnic: add traffic management support Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 3/4] net/softnic: add TM capabilities ops Jasvinder Singh
2017-08-11 12:49 ` [dpdk-dev] [PATCH v3 4/4] net/softnic: add TM hierarchy related ops Jasvinder Singh
2017-09-08 17:08 ` [dpdk-dev] [PATCH v3 0/4] net/softnic: sw fall-back pmd for traffic mgmt and others Dumitrescu, Cristian
2017-06-26 16:43 ` [dpdk-dev] [PATCH v2 2/2] net/softnic: add traffic management ops Jasvinder Singh
2017-05-26 18:11 ` [dpdk-dev] [PATCH " Jasvinder Singh
2017-06-07 14:32 ` [dpdk-dev] [PATCH 0/2] net/softnic: sw fall-back for traffic management Thomas Monjalon
2017-06-08 13:27 ` Dumitrescu, Cristian
2017-06-08 13:59 ` Thomas Monjalon
2017-06-08 15:27 ` Dumitrescu, Cristian
2017-06-08 16:16 ` Thomas Monjalon
2017-06-08 16:43 ` Dumitrescu, Cristian
2017-07-04 23:48 ` Thomas Monjalon
2017-07-05 9:32 ` Dumitrescu, Cristian
2017-07-05 10:17 ` Thomas Monjalon
2017-08-11 15:28 ` Stephen Hemminger
2017-08-11 16:22 ` Dumitrescu, Cristian
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1396e222-0c2b-bc8e-1773-174fc199de70@intel.com \
--to=ferruh.yigit@intel.com \
--cc=cristian.dumitrescu@intel.com \
--cc=dev@dpdk.org \
--cc=jasvinder.singh@intel.com \
--cc=thomas@monjalon.net \
--cc=wenzhuo.lu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).