From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id D511F1B1BF for ; Wed, 3 Jan 2018 13:12:28 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Jan 2018 04:12:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,501,1508828400"; d="scan'208";a="7025790" Received: from awalabdu-mobl.ger.corp.intel.com (HELO [163.33.228.178]) ([163.33.228.178]) by fmsmga002.fm.intel.com with ESMTP; 03 Jan 2018 04:12:24 -0800 To: Neil Horman , Remy Horton Cc: dev@dpdk.org, Wenzhuo Lu , Jingjing Wu References: <1510929733-7225-1-git-send-email-mohammad.abdul.awal@intel.com> <20171222145221.10474-1-remy.horton@intel.com> <20171226135406.GA15284@neilslaptop.think-freely.org> From: Mohammad Abdul Awal Message-ID: Date: Wed, 3 Jan 2018 12:12:24 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <20171226135406.GA15284@neilslaptop.think-freely.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US Subject: Re: [dpdk-dev] [PATCH v3 0/5] lib: add Port Representors X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Jan 2018 12:12:29 -0000 Hi Neil, On 26/12/2017 13:54, Neil Horman wrote: > On Fri, Dec 22, 2017 at 02:52:16PM +0000, Remy Horton wrote: >> Port Representors provide a logical presentation in DPDK of VF (virtual >> function) ports for the purposes of control and monitoring. Each port >> representor device represents a single VF and is associated with it's >> parent physical function (PF) PMD which provides the back-end hooks for >> the representor device ops and defines the control domain to which that >> port belongs. This allows to use existing DPDK APIs to monitor and control >> the port without the need to create and maintain VF specific APIs. >> >> +-----------------------------+ +---------------+ +---------------+ >> | Control Plane | | Data Plane | | Data Plane | >> | Application | | Application | | Application | >> +-----------------------------+ +---------------+ +---------------+ >> | eth dev api | | eth dev api | | eth dev api | >> +-----------------------------+ +---------------+ +---------------+ >> +-------+ +-------+ +-------+ +---------------+ +---------------+ >> | PF0 | | Port | | Port | | VF0 PMD | | VF0 PMD | >> | PMD <--+ Rep 0 | | Rep 1 | +---------------+ +------+--------+ >> | | | PMD | | PMD | | >> +---+--^+ +-------+ +-+-----+ | >> | | | | | >> | +----------------+ | | >> | | | >> | | | >> +--------------------------------+ | >> | | HW (logical view) | | | >> | --+------+ +-------+ +---+---+ | | >> | | PF | | VF0 | | VF1 | | | >> | | | | | | +----------------------------+ >> | +--------+ +-------+ +-------+ | >> | +----------------------------+ | >> | | VEB | | >> | +----------------------------+ | >> | +--------+ | >> | | Port | | >> | | 0 | | >> | +--------+ | >> +--------------------------------+ >> >> The figure above shows a deployment where the PF is bound to a DPDK control >> plane application which uses representor ports to manage the configuration and >> monitoring of it's VF ports. Each virtual function is represented in the >> application by a representor port PMD which enables control of the corresponding >> VF through eth dev APIs on the representor PMD such as: >> >> - void rte_eth_promiscuous_enable(uint8_t port_id); >> - void rte_eth_promiscuous_disable(uint8_t port_id); >> - void rte_eth_allmulticast_enable(uint8_t port_id); >> - void rte_eth_allmulticast_disable(uint8_t port_id); >> - int rte_eth_dev_mac_addr_add(uint8_t port, struct ether_addr *mac_addr, >> uint32_t pool); >> - int rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask); >> >> as well as monitoring through API's like >> >> - void rte_eth_link_get(uint8_t port_id, struct rte_eth_link *link); >> - int rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats); >> >> The port representor infrastructure is enabled through a single common, device >> independent, virtual PMD whos context is initialized and enabled through a >> broker instance running within the context of the physical function device >> driver. >> >> +-------------------------+ +-------------------------+ >> | rte_ethdev | | rte_ethdev | >> +-------------------------+ +-------------------------+ >> | Physical Function PMD | | Port Reperesentor PMD | >> | +-------------+ | | +---------+ +---------+ | >> | | Representor | | | | dev_data| | dev_ops | | >> | | Broker | | | +----+----+ +----+----+ | >> | | +---------+ | | +------|-----------|------+ >> | | | VF Port | | | | | >> | | | Context +------------------+ | >> | | +---------+ | | | >> | | +---------+ | | | >> | | | Handler +------------------------------+ >> | | | Ops | | | >> | | +---------+ | | >> | +-------------+ | >> +-------------------------+ >> >> Creation of representor ports can be achieved either through the --vdev EAL >> option or through the rte_vdev_init() API. Each port representor requires the >> BDF of it's parent PF and the Virtual Function ID of the port which the >> representor will support. During initialization of the representor PMD, it calls >> the broker API to register itself with the PF PMD and to get it's context >> configured which includes the setting up of it's context and ops function >> handlers. >> >> As the port representor model is based around the paradigm of using standard >> port based APIs, it will allow future expansion of functionality without the >> need to add new APIs. For example it should be possible to support configuration >> of egress QoS parameters using existing TM APIs by extending the port >> representor PMD/broker infrastructure. >> >> Changes in v2: >> * Rebased to DPDK 17.11 >> >> Changes in v3: >> * Removed RTE_ETH_DEV_REPRESENTOR_PORT define >> * Removed switch_domain from struct rte_eth_dev_info >> (to be reintroduced seperately when required). >> * Port Representor PMD functionality merged into main library >> * Removed functions from .map file that are not for use by applications >> * Some minor bugfixes (uninitalised variables & NULL terminators) >> * Use of SPDX licence headers in new files >> * Seperate headers for PMD and application >> * SPDX-License-Identifier in new files >> * Added test-pmd representor add/del commands >> >> >> Remy Horton (5): >> lib: add Port Representor library >> eal: add Port Representor command-line option >> drivers/net/i40e: add Port Representor functionality >> drivers/net/ixgbe: add Port Representor functionality >> app/test-pmd: add Port Representor commands >> >> app/test-pmd/cmdline.c | 88 ++++ >> config/common_base | 5 + >> drivers/net/i40e/Makefile | 1 + >> drivers/net/i40e/i40e_ethdev.c | 16 + >> drivers/net/i40e/i40e_ethdev.h | 1 + >> drivers/net/i40e/i40e_prep_ops.c | 495 +++++++++++++++++++++ >> drivers/net/i40e/i40e_prep_ops.h | 15 + >> drivers/net/i40e/rte_pmd_i40e.c | 47 ++ >> drivers/net/i40e/rte_pmd_i40e.h | 18 + >> drivers/net/ixgbe/Makefile | 1 + >> drivers/net/ixgbe/ixgbe_ethdev.c | 22 +- >> drivers/net/ixgbe/ixgbe_ethdev.h | 5 + >> drivers/net/ixgbe/ixgbe_prep_ops.c | 259 +++++++++++ >> drivers/net/ixgbe/ixgbe_prep_ops.h | 15 + >> lib/Makefile | 3 + >> lib/librte_eal/bsdapp/eal/eal.c | 6 + >> lib/librte_eal/common/eal_common_options.c | 1 + >> lib/librte_eal/common/eal_internal_cfg.h | 2 + >> lib/librte_eal/common/eal_options.h | 2 + >> lib/librte_eal/common/include/rte_eal.h | 8 + >> lib/librte_eal/linuxapp/eal/eal.c | 9 + >> lib/librte_representor/Makefile | 26 ++ >> lib/librte_representor/rte_port_representor.c | 326 ++++++++++++++ >> lib/librte_representor/rte_port_representor.h | 60 +++ >> .../rte_port_representor_driver.h | 138 ++++++ >> .../rte_port_representor_version.map | 8 + >> mk/rte.app.mk | 1 + >> 27 files changed, 1577 insertions(+), 1 deletion(-) >> create mode 100644 drivers/net/i40e/i40e_prep_ops.c >> create mode 100644 drivers/net/i40e/i40e_prep_ops.h >> create mode 100644 drivers/net/ixgbe/ixgbe_prep_ops.c >> create mode 100644 drivers/net/ixgbe/ixgbe_prep_ops.h >> create mode 100644 lib/librte_representor/Makefile >> create mode 100644 lib/librte_representor/rte_port_representor.c >> create mode 100644 lib/librte_representor/rte_port_representor.h >> create mode 100644 lib/librte_representor/rte_port_representor_driver.h >> create mode 100644 lib/librte_representor/rte_port_representor_version.map >> >> -- >> 2.9.5 >> >> > Same question as in V1, how does this patch mesh with the notion of port > ownership? The consensus from that thread was that, because the DPDK considers > itself lockless, that the application(s) managing/using a given port need to > enforce their own mutual exclusion. The design of this feature explicitly > creates an alias effect, in that the same hardware (a VF in this case), can be > accessed via two different port structures by two different applications without > being aware of the shared nature of the port. Given that one is in a vm, > co-ordination between applications is going to be awkward at best. How do you > plan to enforce mutual exclusion in this environment, given that you don't have > any shared memory between the vm and the host? Without the port representor model, it is possible that both PF and VF can have shared access of the VF hardware. Lets take an example of i40e drivers [1]. Application running in host can modify the hardware configuration using the provided APIs in the reference [1]. Applications running on the VF can modify the hardware configuration using mailbox mechanism and with privileged mode. With the port representor model, we still have the same shared access model. Only important difference is that, with port representor, we do not have to make those device specific private APIs public, as all can be accessed by eth_dev_ops. In an OVS like scenario, it is expected that the control application will have sole privilege to modify the hardware configuration. Currently, we cannot stop the application running on VF to modify the hardware (hence your concern is valid but still accepted scenario as in reference [1]). If we do not want to allow the applications running on VF to modify the hardware, one idea is to intercept the interrupts regarding the malibox requested coming from the VF, and redirect to the port representor model. But this is out of the scope of port representor itself. [1] http://dpdk.org/ml/archives/dev/2017-April/063642.html > Neil > Regards, Awal.