From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id E96272B86 for ; Thu, 2 Mar 2017 12:07:24 +0100 (CET) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga105.jf.intel.com with ESMTP; 02 Mar 2017 03:07:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,230,1484035200"; d="scan'208";a="231421852" Received: from unknown (HELO localhost.localdomain.sh.intel.com) ([10.239.128.150]) by fmsmga004.fm.intel.com with ESMTP; 02 Mar 2017 03:07:22 -0800 From: "Chen Jing D(Mark)" To: dev@dpdk.org Cc: cunming.liang@intel.com, gerald.rogers@intel.com, keith.wiles@intel.com, bruce.richardson@intel.com, "Chen Jing D(Mark)" Date: Thu, 2 Mar 2017 12:03:55 +0800 Message-Id: <1488427438-13646-4-git-send-email-jing.d.chen@intel.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1488427438-13646-1-git-send-email-jing.d.chen@intel.com> References: <1488427438-13646-1-git-send-email-jing.d.chen@intel.com> Subject: [dpdk-dev] [PATCH 3/6] prgdev: add bus probe and remove functions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Mar 2017 11:07:25 -0000 This patch adds 4 major functions: pci_probe/pci_remove, prgdev_allocate/prgdev_release to help drivers registering programmable devices. pci_probe functions helps drivers to allocate prgdev structures and trigger an initialization function that drivers provided. This function will be called when bus scan action performed or hotplug occured. prgdev_allocate will allocate a prgdev structures, but won't perform an intialization of the device. Comparing pci_probe function which will be triggered by bus scan, it can be called at any time by driver. Signed-off-by: Chen Jing D(Mark) Signed-off-by: Gerald Rogers --- lib/librte_prgdev/rte_prgdev.c | 233 ++++++++++++++++++++++++++++++++++++++++ lib/librte_prgdev/rte_prgdev.h | 228 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 461 insertions(+), 0 deletions(-) diff --git a/lib/librte_prgdev/rte_prgdev.c b/lib/librte_prgdev/rte_prgdev.c index e69de29..03465f9 100644 --- a/lib/librte_prgdev/rte_prgdev.c +++ b/lib/librte_prgdev/rte_prgdev.c @@ -0,0 +1,233 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016-2017 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "rte_prgdev.h" + +static struct rte_prgdev rte_prgdev_devices[RTE_PRGDEV_MAX_DEVS]; +static uint8_t nb_devs; + +static struct rte_prgdev *rte_prgdev_allocated(char *name); + +int +rte_prgdev_pci_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev) +{ + struct prgdev_driver *prgdrv; + struct rte_prgdev *prgdev; + char prgdev_name[PRGDEV_NAME_MAX_LEN]; + int retval; + + /* Not support secondary process to operate on prgdev */ + RTE_PRG_PRIMARY_PROC_OR_ERR_RET(-EINVAL); + + prgdrv = (struct prgdev_driver *)pci_drv; + if (prgdrv == NULL) + return -ENODEV; + + rte_eal_pci_device_name(&pci_dev->addr, prgdev_name, + sizeof(prgdev_name)); + + prgdev = rte_prgdev_allocate(prgdev_name); + if (prgdev == NULL) + return -1; + + prgdev->dev_data.dev_private = + rte_zmalloc_socket( + "prgdevprivate structure", + prgdrv->dev_private_size, + RTE_CACHE_LINE_SIZE, + rte_socket_id()); + + if (prgdev->dev_data.dev_private == NULL) + rte_panic("Cannot allocate memzone for private " + "device data"); + + prgdev->device = &pci_dev->device; + prgdev->driver = prgdrv; + + + /* Invoke PMD device initialization function */ + retval = (*prgdrv->prgdev_init)(prgdev); + if (retval == 0) + return 0; + + PRG_LOG_ERR("driver %s: prgdev_init(vendor_id=0x%x device_id=0x%x)" + " failed", pci_drv->driver.name, + (unsigned) pci_dev->id.vendor_id, + (unsigned) pci_dev->id.device_id); + + rte_free(prgdev->dev_data.dev_private); + + rte_prgdev_release(prgdev); + + return -ENXIO; +} + +int +rte_prgdev_pci_remove(struct rte_pci_device *pci_dev) +{ + struct prgdev_driver *prgdrv; + struct rte_prgdev *prgdev; + char prgdev_name[PRGDEV_NAME_MAX_LEN]; + int ret; + + if (pci_dev == NULL) + return -EINVAL; + + /* Not support secondary process to operate on prgdev */ + RTE_PRG_PRIMARY_PROC_OR_ERR_RET(-EINVAL); + + rte_eal_pci_device_name(&pci_dev->addr, prgdev_name, + sizeof(prgdev_name)); + + + prgdev = rte_prgdev_allocated(prgdev_name); + if (prgdev == NULL) + return -ENODEV; + + prgdrv = (struct prgdev_driver *)pci_dev->driver; + if (prgdrv == NULL) + return -ENODEV; + + /* Invoke device uninit function */ + if (*prgdrv->prgdev_uninit) { + ret = (*prgdrv->prgdev_uninit)(prgdev); + if (ret) + return ret; + } + + /* free prgdev device */ + rte_prgdev_release(prgdev); + + rte_free(prgdev->dev_data.dev_private); + + return 0; +} + +static uint8_t +prgdev_find_free_dev_slot(void) +{ + unsigned i; + + for (i = 0; i < RTE_PRGDEV_MAX_DEVS; i++) { + if (rte_prgdev_devices[i].attached == PRGDEV_DETACHED) + return i; + } + return RTE_PRGDEV_MAX_DEVS; +} + +static struct rte_prgdev * +rte_prgdev_allocated(char *name) +{ + unsigned i; + + for (i = 0; i < RTE_PRGDEV_MAX_DEVS; i++) { + if ((rte_prgdev_devices[i].attached == PRGDEV_ATTACHED) && + strcmp(rte_prgdev_devices[i].dev_data.name, name) == 0) + return &rte_prgdev_devices[i]; + } + return NULL; +} + +struct rte_prgdev * +rte_prgdev_allocate(char *name) +{ + uint8_t dev_id; + struct rte_prgdev *prg_dev; + + /* Not support secondary process to operate on prgdev */ + RTE_PRG_PRIMARY_PROC_OR_ERR_RET(NULL); + dev_id = prgdev_find_free_dev_slot(); + if (dev_id == RTE_PRGDEV_MAX_DEVS) { + PRG_LOG_DEBUG("Reached maximum number of prgdev numbers"); + return NULL; + } + + if (rte_prgdev_allocated(name) != NULL) { + PRG_LOG_DEBUG("Prgdev Device with name %s already allocated!\n", + name); + return NULL; + } + + prg_dev = &rte_prgdev_devices[dev_id]; + memset(prg_dev, 0, sizeof(*prg_dev)); + prg_dev->attached = PRGDEV_ATTACHED; + snprintf(prg_dev->dev_data.name, sizeof(prg_dev->dev_data.name), "%s", + name); + prg_dev->dev_data.devid = dev_id; + prg_dev->dev_data.numa_node = rte_socket_id(); + nb_devs++; + return prg_dev; +} + +int +rte_prgdev_release(struct rte_prgdev *prg_dev) +{ + /* Not support secondary process to operate on prgdev */ + RTE_PRG_PRIMARY_PROC_OR_ERR_RET(-EINVAL); + + if (prg_dev == NULL) + return -EINVAL; + + prg_dev->attached = PRGDEV_ATTACHED; + prg_dev->dev_ops = NULL; + prg_dev->driver = NULL; + nb_devs--; + return 0; +} + diff --git a/lib/librte_prgdev/rte_prgdev.h b/lib/librte_prgdev/rte_prgdev.h index 52f8a45..c25cfef 100644 --- a/lib/librte_prgdev/rte_prgdev.h +++ b/lib/librte_prgdev/rte_prgdev.h @@ -45,6 +45,9 @@ #include #include #include + +#define PRGDEV_NAME_MAX_LEN (64) + /* Logging Macros */ #define PRG_LOG_ERR(...) \ RTE_LOG(ERR, PRG, \ @@ -60,6 +63,231 @@ #define PRG_LOG_DEBUG(...) (void)0 #endif +/* + * reflect the device status + */ +enum rte_prgdev_stat { + RTE_PRG_STAT_UNKNOWN = 0, /** Device in a unkown state */ + RTE_PRG_STAT_READY, /** Device is ready to be programmed. */ + RTE_PRG_STAT_OPEN, /*Device is opened, waiting for programming */ + RTE_PRG_STAT_ERASING, /** Device is being programmed. */ + /** Device is performing functionalities and can't be programmed. */ + RTE_PRG_STAT_RUNNING, +}; + +/* Reflect the function block attributes */ +/* Block is readable */ +#define RTE_PRG_BLK_ATTR_RD 0x00000001 +/* Block is writable */ +#define RTE_PRG_BLK_ATTR_WR 0x00000002 +/* Block is readable and writable */ +#define RTE_PRG_FUNC_ATTR_RDWR (RTE_PRG_BLK_ATTR_RD & RTE_PRG_BLK_ATTR_WR) + +struct rte_prgdev_blk_info { + unsigned int size; /* the block size in bytes */ + unsigned int version; /* It's optional */ + unsigned int flags; /* Flags to indicate blk is readable/writable */ +}; + +#define MAX_SIGNATURE_LEN 256 +#define MAX_BLK_NUM 256 + +struct rte_prgdev; +struct rte_prgdev_info; + +/* + * reflect the firmware status after programming with new image + */ +enum rte_prg_fwstat { + RTE_PRGDEV_FWSTAT_OK = 0, /** Image are running well */ + RTE_PRGDEV_FWSTAT_ERR = -1, /** Image has error. */ + RTE_PRGDEV_FWSTAT_ERR_VALID = -2, /** Image is not valid. */ + RTE_PRGDEV_FWSTAT_ERR_CKSUM = -3, /** Image checksum is not correct. */ + RTE_PRGDEV_FWSTAT_ERR_LEN = -4, /** Image length is not correct. */ +}; + +/* +* prg_dev_init routine +* +* returns : 0 success, non zero failure. +*/ +typedef int (*prgdev_init_t)(struct rte_prgdev *prg_dev); + +/* +* prg_dev_uninit routine +* +* returns : 0 success, non zero failure. +*/ +typedef int (*prgdev_uninit_t)(struct rte_prgdev *prg_dev); + + +/**< @internal query infos of the device. */ +typedef int (*prgdev_get_info_t)(struct rte_prgdev *prg_dev, + struct rte_prgdev_info *info); + +/**< @internal Open device for programming, acquiring on-die image, etc. */ +typedef int (*prgdev_open_t)(struct rte_prgdev *prg_dev); + +/**< @internal Download image from host to programmable device. */ +typedef int (*prgdev_img_download_t)(struct rte_prgdev *prg_dev, + uint8_t *buffer_ptr, uint32_t buf_len); + +/**< @internal Upload image from programmable device to host. */ +typedef int (*prgdev_img_upload_t)(struct rte_prgdev *prg_dev, + uint8_t *buffer_ptr, uint32_t buf_len, uint32_t *act_len); + +/**< @internal Check if the downloaded image works in expected way. */ +typedef int (*prgdev_check_stat_t)(struct rte_prgdev *prg_dev, + enum rte_prg_fwstat *stat); + +/**< @internal free up resources or whatever to do to hardware. */ +typedef int (*prgdev_close_t)(struct rte_prgdev *prg_dev); + +/**< @internal bind personalities with drivers. */ +typedef int (*prgdev_bind_t)(struct rte_prgdev *prg_dev); + +/**< @internal unbind all personalities except prgdev from drivers. */ +typedef int (*prgdev_unbind_t)(struct rte_prgdev *prg_dev); + +/** prgdev device operations function pointer table */ +struct rte_prgdev_ops { + prgdev_get_info_t prg_infos_get; /**< Get device info. */ + prgdev_open_t prg_open; /**< Open device. */ + prgdev_img_download_t prg_download; /**< download Image. */ + prgdev_img_upload_t prg_upload; /**< upload device. */ + prgdev_check_stat_t prg_check_stat; /**< Get image status. */ + prgdev_close_t prg_close; /**< Open device. */ + prgdev_bind_t prg_bind; /**< bind drivers */ + prgdev_unbind_t prg_unbind; /**< unbind drivers */ +}; + +/** + * @internal + * The structure associated with a programmable device driver. + * + * - An *rte_pci_driver* structure (which must be the first field). + * + * - The *prgdev_init_t* function invoked for each matching PCI device. + * + * - The *prgdev_uninit_t* function invoked for each matching PCI device. + * + * - The size of the private data to allocate for each matching device. + */ +struct prgdev_driver { + struct rte_pci_driver pci_drv; /**< The PMD is also a PCI driver. */ + prgdev_init_t prgdev_init; /**< Device init function. */ + prgdev_uninit_t prgdev_uninit; /**< Device uninit function. */ + unsigned int dev_private_size; /**< Size of device private data. */ +}; + +struct rte_prgdev_info { + const char *driver_name; /**< Device Driver name. */ + /* Programable device HW version number. it's possible that app + * have dependency to HW version. + */ + struct rte_pci_device *pci_dev; /**< PCI information. */ + uint16_t hw_ver_major; /**< major version number */ + uint16_t hw_ver_minor; /**< minor version number */ + + /* A array to store hardware, firmware, running image info.Each device + * can define and interpret the info. For example, a device can define + * byte[3:0] as signature for on-die personality, byte[5:4] is the major + * version, byte[7:6] is minor version. if 0xFFEA1000 is a signature for + * virtio personality, major version is 0x0001, minor version is 0x0004. + * then signature can be defined : + * sig_num = 8; + * signature[7:0]= {00, 0x10, 0xEA, 0xFF, 0x00, 0x01, 0x00, 0x04}; + */ + unsigned int sig_num; /** < the valid signature length in bytes> */ + char signature[MAX_SIGNATURE_LEN]; + enum rte_prgdev_stat status; + + /**< number of blocks within device */ + unsigned int blk_num; + /**< block info */ + struct rte_prgdev_blk_info blk_info[MAX_BLK_NUM]; +}; + +struct rte_prgdev_data { + char name[PRGDEV_NAME_MAX_LEN]; + unsigned int devid; /**< Index to bound host interface, or 0 if none. */ + void *dev_private; /**< prgdev -specific private data */ + int numa_node; /**< NUMA node connection */ + const char *drv_name; /**< Driver name */ +}; + +enum { + PRGDEV_DETACHED = 0, + PRGDEV_ATTACHED +}; + +struct rte_prgdev { + struct rte_prgdev_data dev_data; + const struct prgdev_driver *driver; + struct rte_prgdev_ops *dev_ops; + /**< Supported features */ + uint64_t feature_flags; + struct rte_device *device; + uint8_t attached; /**< Flag indicating the device is attached */ +} __rte_cache_aligned; + +/* Macros to check for valid device */ +#define RTE_PRG_VALID_DEVID_OR_ERR_RET(dev_id, retval) do { \ + if (!rte_prgdev_is_valid_dev(dev_id)) { \ + PRG_LOG_DEBUG("Invalid dev_id=%d", dev_id); \ + return retval; \ + } \ +} while (0) + +#define RTE_PRG_FUNC_PTR_OR_ERR_RET(func, retval) do { \ + if ((func) == NULL) { \ + PRG_LOG_DEBUG("Function not supported\n"); \ + return retval; \ + } \ +} while (0) + +#define RTE_PRG_PRIMARY_PROC_OR_ERR_RET(retval) do { \ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) { \ + PRG_LOG_DEBUG("prgdev doesn't support secondary process"); \ + return retval; \ + } \ +} while (0) + +/** + * Wrapper for use by pci drivers as a .probe function to attach to a prgdev + * interface. + */ +int rte_prgdev_pci_probe(struct rte_pci_driver *pci_drv, + struct rte_pci_device *pci_dev); + +/** + * Wrapper for use by pci drivers as a .remove function to detach a prgdev + * interface. + */ +int rte_prgdev_pci_remove(struct rte_pci_device *pci_dev); + +/** + * @internal + * Allocates a new prgdev slot for an programmable device and returns the + * pointer to that slot for the driver to use. + * + * @param name: Unique identifier name for each Ethernet device + * @return + * - Slot in the rte_dev_devices array for a new device; + */ +struct rte_prgdev *rte_prgdev_allocate(char *name); + +/** + * @internal + * Release the specified prgdev device. + * + * @param eth_dev + * The *prg_dev* pointer is the address of the *rte_prgdev* structure. + * @return + * - 0 on success, negative on error + */ +int rte_prgdev_release(struct rte_prgdev *prg_dev); + #ifdef __cplusplus } #endif -- 1.7.7.6