From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f50.google.com (mail-oi0-f50.google.com [209.85.218.50]) by dpdk.org (Postfix) with ESMTP id 8F051199AB for ; Fri, 8 Sep 2017 15:59:30 +0200 (CEST) Received: by mail-oi0-f50.google.com with SMTP id r20so15166512oie.0 for ; Fri, 08 Sep 2017 06:59:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=u2NGlD6ObrVmSAdVWHKlea+BqhF6UFxhwFwu1bEYAAI=; b=P/6dgvfLD5hD5uO5Z/YsvyLiyGiYSXzOO2C7XTHCcA4yG1bnPk/RQC/79lkerT9bmQ 6w6rcVsVC+34RulVbueHmHfDpL7zTHBq2K54aw8UFgEz4UPJvasSVi0Xlxj7KnEGAY7H K2BhQsxOPIBX8H5NcJtcNpraJ3g/y8QYu7ziqNRiHgYux+u1t4B4AVes/98ubUiN5U7L 8MIaUsbhmDNiAq1zyuIEowaaCCT1lNUtwWj9/tSm4+ALnKjwCYXcZ65Xk2+HQPnyTuS3 EXknlaVkND+AeVo58KHvR11lgl2Af376TKURvIfHAlqUK0qylJuToR4PTYtrOuKfVSsl IEew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=u2NGlD6ObrVmSAdVWHKlea+BqhF6UFxhwFwu1bEYAAI=; b=A+Q/UxIMd5CAHG0On8HTSY+FcxpI/r33lmDrbJ1KnnbZpmF8dkhYnoTuvrD5KnlnrM VeCMaTkem7CsnkyTiainfYcvDI7JunsLU5CvLbicy7zjtJmS+8Gece8D1cuwk9X2DF/g hzMuEOuH/Vhs69grNCWNA1gtgAEHhbDyjPYRZnblFuBf+/VqJVWJcAV/gBj4tiCI5pgd hCNMEcTunrvyKpwAIOcvLo++TKkh0ypDJj2NrJfaQXP2X5YumJfzqEQwgBkinCNaly3J IWVmop6sDbbjYoIZdXht5CXn8ZE40TES7PMHS9Zz+4M6ZpscJMIWdCfl1xiU5fYvrzj7 CwBQ== X-Gm-Message-State: AHPjjUjmah+hNgwSgmQcXPfYNbCAqtIvB55LrhygNJqFBjlCfKMakvhe sOwGAP1Vec5eZNp9+FTgiu/DySpzbWSr X-Google-Smtp-Source: AOwi7QArBZbzMywaAvuKj6YwrQ/jbnuQ9Rnn6P7rceLnR/0av+OdpNAgxSV5OdJDD+tgW2IPkqDFQsma5Cj8/H8o5YQ= X-Received: by 10.202.227.21 with SMTP id a21mr2871737oih.194.1504879169741; Fri, 08 Sep 2017 06:59:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.33.141 with HTTP; Fri, 8 Sep 2017 06:59:29 -0700 (PDT) In-Reply-To: References: <1504773339-21022-1-git-send-email-mohammad.abdul.awal@intel.com> From: Alejandro Lucero Date: Fri, 8 Sep 2017 14:59:29 +0100 Message-ID: To: Declan Doherty Cc: Mohammad Abdul Awal , dev , Remy Horton Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] [RFC 0/5] Port Representor for control and monitoring of VF devices X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 13:59:31 -0000 On Thu, Sep 7, 2017 at 2:13 PM, Declan Doherty wrote: > On 07/09/2017 11:01 AM, Alejandro Lucero wrote: > >> I understand this is the representor idea suiting Intel cards but it does >> not cover other possibilities. >> >> At least, Netronome and Mellanox require a representor not just for >> controlling a VF, but also for sending and receiving packets through the >> representor PMD. >> >> > Hey Alejandro, > > What we've proposed here doesn't preclude the support of a data path on > the representor pmd as the functionality which the PMD supports is > dependent on how the parent device running the representor broker > configures each representor instance. As you note in the case of our > current generation of IO devices supporting a data path doesn't make sense, > but I think this framework could support that functionality if required. > > Hi Declan, Tell me if I'm wrong, but this looks to me like kernel netdev ndo_set_vf_* functions, maybe with the idea of having more possibilities than those currently available with the kernel. The representor idea I refer to is completely different, and it is related to using SRIOV with OVS offload. It has other possibilities but this is the current main reason. Some references: https://www.slideshare.net/Netronome/using-agilio-smartnics-for-openstack-networking-acceleration https://netdevconf.org/1.2/slides/oct6/04_gerlitz_efraim_introduction_to_switchdev_sriov_offloads.pdf I sent an abstract for a presentation in next Dublin Users meeting, and >> discussing the representor idea was in the agenda. >> >> > I've also submitted an presentation which I got notification was accepted > this morning which is based on the RFC, so maybe we should collaborate on a > presentation if there is a large overlap? Unfortunately, I did not get my abstract approved. Maybe you could add some slide about this other view for the shake of discussion. > > > By the way, I remember there was reticence about adding control plane to >> DPDK. What is the current official DPDK position in this regard? >> >> > I also remember there was concerns about adding control plane APIs to > DPDK, hence the RFC, but I think the advantage of the representor port > approach is that there are no new public API being introduced, with only > back-end support in the PMDs which wish to support this functionality. > > I'd like to know what is the opinion of the DPDK community regarding the control path. > > >> >> On Thu, Sep 7, 2017 at 9:35 AM, Mohammad Abdul Awal < >> mohammad.abdul.awal@intel.com> wrote: >> >> Signed-off-by: Mohammad Abdul Awal >>> Signed-off-by: Remy Horton >>> Signed-off-by: Declan Doherty >>> --- >>> This RFC introduces a port representor based model for the control, >>> management >>> and monitoring of SR-IOV virtual function (VF) devices for control plane >>> applications which have bound the devices physical function. >>> >>> Port Representors are virtual poll mode drivers (PMD) which provide a >>> logical >>> representation in DPDK for VF ports for control and monitoring. Each port >>> representor PMD represents a single VF and is associated with it's parent >>> physical function (PF) PMD which provides the back-end hooks for the >>> representor >>> device ops and defines the control domain to which that port belongs.This >>> allows to use existing DPDK APIs to monitor and control the port without >>> the >>> need to create and maintain VF specific APIs. >>> >>> >>> +-----------------------------+ +---------------+ +---------------+ >>> | Control Plane | | Data Plane | | Data Plane | >>> | Application | | Application | | Application | >>> +-----------------------------+ +---------------+ +---------------+ >>> | eth dev api | | eth dev api | | eth dev api | >>> +-----------------------------+ +---------------+ +---------------+ >>> +-------+ +-------+ +-------+ +---------------+ +---------------+ >>> | PF0 | | Port | | Port | | VF0 PMD | | VF0 PMD | >>> | PMD <--+ Rep 0 | | Rep 1 | +---------------+ +------+--------+ >>> | | | PMD | | PMD | | >>> +---+--^+ +-------+ +-+-----+ | >>> | | | | | >>> | +----------------+ | | >>> | | | >>> | | | >>> +--------------------------------+ | >>> | | HW (logical view) | | | >>> | --+------+ +-------+ +---+---+ | | >>> | | PF | | VF0 | | VF1 | | | >>> | | | | | | +----------------------------+ >>> | +--------+ +-------+ +-------+ | >>> | +----------------------------+ | >>> | | VEB | | >>> | +----------------------------+ | >>> | +--------+ | >>> | | Port | | >>> | | 0 | | >>> | +--------+ | >>> +--------------------------------+ >>> >>> The figure above shows a deployment where the PF is bound to a DPDK >>> control >>> plane application which uses representor ports to manage the >>> configuration >>> and >>> monitoring of it's VF ports. Each virtual function is represented in the >>> application by a representor port PMD which enables control of the >>> corresponding >>> VF through eth dev APIs on the representor PMD such as: >>> >>> - void rte_eth_promiscuous_enable(uint8_t port_id); >>> - void rte_eth_promiscuous_disable(uint8_t port_id); >>> - void rte_eth_allmulticast_enable(uint8_t port_id); >>> - void rte_eth_allmulticast_disable(uint8_t port_id); >>> - int rte_eth_dev_mac_addr_add(uint8_t port, struct ether_addr >>> *mac_addr, >>> uint32_t pool); >>> - int rte_eth_dev_set_vlan_offload(uint8_t port_id, int offload_mask); >>> >>> as well as monitoring through API's like >>> >>> - void rte_eth_link_get(uint8_t port_id, struct rte_eth_link *link); >>> - int rte_eth_stats_get(uint8_t port_id, struct rte_eth_stats *stats); >>> >>> The port representor infrastructure is enabled through a single common, >>> device >>> independent, virtual PMD whos context is initialized and enabled through >>> a >>> broker instance running within the context of the physical function >>> device >>> driver. >>> >>> +-------------------------+ +-------------------------+ >>> | rte_ethdev | | rte_ethdev | >>> +-------------------------+ +-------------------------+ >>> | Physical Function PMD | | Port Reperesentor PMD | >>> | +-------------+ | | +---------+ +---------+ | >>> | | Representor | | | | dev_data| | dev_ops | | >>> | | Broker | | | +----+----+ +----+----+ | >>> | | +---------+ | | +------|-----------|------+ >>> | | | VF Port | | | | | >>> | | | Context +------------------+ | >>> | | +---------+ | | | >>> | | +---------+ | | | >>> | | | Handler +------------------------------+ >>> | | | Ops | | | >>> | | +---------+ | | >>> | +-------------+ | >>> +-------------------------+ >>> >>> Creation of representor ports can be achieved either through the --vdev >>> EAL >>> option or through the rte_vdev_init() API. Each port representor requires >>> the >>> BDF of it's parent PF and the Virtual Function ID of the port which the >>> representor will support. During initialization of the representor PMD, >>> it >>> calls >>> the broker API to register itself with the PF PMD and to get it's context >>> configured which includes the setting up of it's context and ops function >>> handlers. >>> >>> >>> As the port representor model is based around the paradigm of using >>> standard >>> port based APIs, it will allow future expansion of functionality without >>> the >>> need to add new APIs. For example it should be possible to support >>> configuration >>> of egress QoS parameters using existing TM APIs by extending the port >>> representor PMD/broker infrastructure. >>> >>> Mohammad Abdul Awal (5): >>> Implemented port representor broker infrastructure, created BDF to >>> port function. >>> added --enable-representor command line argument in EAL to load >>> representor broker infrastructure. >>> Implement port representor PMD >>> Enable port representor PMD and broker for fortville PMD driver >>> Enable port representor PMD and broker for ixgbe PMD driver. >>> >>> config/common_base | 5 + >>> drivers/net/Makefile | 2 + >>> drivers/net/i40e/Makefile | 1 + >>> drivers/net/i40e/i40e_ethdev.c | 17 + >>> drivers/net/i40e/i40e_prep_ops.c | 402 +++++++++ >>> drivers/net/i40e/i40e_prep_ops.h | 41 + >>> drivers/net/i40e/rte_pmd_i40e.c | 47 + >>> drivers/net/i40e/rte_pmd_i40e.h | 18 + >>> drivers/net/ixgbe/Makefile | 2 +- >>> drivers/net/ixgbe/ixgbe_ethdev.c | 33 +- >>> drivers/net/ixgbe/ixgbe_ethdev.h | 11 + >>> drivers/net/ixgbe/ixgbe_prep_ops.c | 279 ++++++ >>> drivers/net/ixgbe/ixgbe_prep_ops.h | 41 + >>> drivers/net/representor/Makefile | 51 ++ >>> drivers/net/representor/rte_eth_representor.c | 973 >>> +++++++++++++++++++++ >>> .../representor/rte_pmd_representor_version.map | 4 + >>> lib/librte_eal/bsdapp/eal/eal.c | 6 + >>> lib/librte_eal/common/eal_common_options.c | 1 + >>> lib/librte_eal/common/eal_internal_cfg.h | 2 + >>> lib/librte_eal/common/eal_options.h | 2 + >>> lib/librte_eal/common/include/rte_eal.h | 8 + >>> lib/librte_eal/linuxapp/eal/eal.c | 9 + >>> lib/librte_ether/Makefile | 2 + >>> lib/librte_ether/rte_ethdev.c | 93 ++ >>> lib/librte_ether/rte_ethdev.h | 26 + >>> lib/librte_ether/rte_ethdev_vdev.h | 37 +- >>> lib/librte_ether/rte_ether_version.map | 9 + >>> lib/librte_ether/rte_port_representor.c | 160 ++++ >>> lib/librte_ether/rte_port_representor.h | 289 ++++++ >>> mk/rte.app.mk | 1 + >>> 30 files changed, 2556 insertions(+), 16 deletions(-) >>> create mode 100644 drivers/net/i40e/i40e_prep_ops.c >>> create mode 100644 drivers/net/i40e/i40e_prep_ops.h >>> create mode 100644 drivers/net/ixgbe/ixgbe_prep_ops.c >>> create mode 100644 drivers/net/ixgbe/ixgbe_prep_ops.h >>> create mode 100644 drivers/net/representor/Makefile >>> create mode 100644 drivers/net/representor/rte_eth_representor.c >>> create mode 100644 drivers/net/representor/rte_ >>> pmd_representor_version.map >>> create mode 100644 lib/librte_ether/rte_port_representor.c >>> create mode 100644 lib/librte_ether/rte_port_representor.h >>> >>> -- >>> 2.7.4 >>> >>> >>> >> >