From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-f175.google.com (mail-lb0-f175.google.com [209.85.217.175]) by dpdk.org (Postfix) with ESMTP id 5CCFFC492 for ; Thu, 28 Jan 2016 16:20:24 +0100 (CET) Received: by mail-lb0-f175.google.com with SMTP id dx2so25203074lbd.3 for ; Thu, 28 Jan 2016 07:20:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9sMLSvcC151QQE1W/UWijWUaMGQPYyuzH52Vegrx79M=; b=VqMva/SSfKargxGvI/sMyhNXoC232jw8tMrLwDcpl8V3IEiGnrMuZEJvU/KJ9ykjz1 +igaSM3nDk4L7CmJ5WHPOsTluKQnuPV62fJNQ3lDCaYxi3pgtTCASF5gMsLDNuNF7KRv FZntdWxpKUIo0i6TH7TNi9Ete0qbHtq+jUg1DSXy0iOxpJMBHUu117a+ZtrVbcvYVe4O BrTxTSdsGZvWfEmCf6GrOEr7HFz5IH25Jh3ix2aiQGpuxEZ7iSoNhXTEdeufcUVdl36U lts1Y/q9bzHbfpKtluR08ko/4X5uebvp/EeqTDZf2Wl6rK6bQt3OpEkyMbH/YxIe4EUt 2stQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9sMLSvcC151QQE1W/UWijWUaMGQPYyuzH52Vegrx79M=; b=LEJsmHwi7cXIZemTclc595IwZUZ1rMWBtIffBBpgK6UUu5LSKlzfC7X8ZEqiSGIPsA /MR+dIlbdBjCjqKsrN/ugVgohEtZ7cMGzf774WxMUd1H8dRWPiTVGWW07qVYbSL6vAVo QqucDQrJiXbtwErYqrmnwZoyd/1gYYvdnurblWmxmA2j4eTn9DQLZiQkM3Wk+Piavs+s BvfGlO52qXbFuimlrV2tIfDiY9Eebbxm+d1kcpZ56RgBcwNTnwaMe0jZ/gIrBUXTP8o6 vs4+HPpl0ogvEPSdQMwKtUTeNSizOBbF/kq9ju6r7KexHVNsX2KbsD/A0Z7sVeNyWtRf AurA== X-Gm-Message-State: AG10YOTtZu6dCcEkuf08BatCknS9BMAOh6vt2de6X27z+zS93ys3HGBX0u0YHbPGtDMqiQ== X-Received: by 10.112.147.1 with SMTP id tg1mr1131647lbb.119.1453994424112; Thu, 28 Jan 2016 07:20:24 -0800 (PST) Received: from anpa-dpdk-2.lab.semihalf.com (cardhu.semihalf.com. [213.17.239.108]) by smtp.gmail.com with ESMTPSA id qa9sm1471090lbb.40.2016.01.28.07.20.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Jan 2016 07:20:23 -0800 (PST) From: Jan Medala To: dev@dpdk.org Date: Thu, 28 Jan 2016 16:20:10 +0100 Message-Id: <1453994413-403-2-git-send-email-jan@semihalf.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1453994413-403-1-git-send-email-jan@semihalf.com> References: <1453994413-403-1-git-send-email-jan@semihalf.com> Cc: Evgeny Schemeilin , matua@amazon.com Subject: [dpdk-dev] [PATCH 1/4] Amazon ENA UIO driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 15:20:24 -0000 Amazon ENA device doesn't implement legacy interrupt which is required by default UIO. This driver introduces all necessary memory mappings in order to use ENA device. Signed-off-by: Evgeny Schemeilin Signed-off-by: Jan Medala Signed-off-by: Jakub Palider --- config/common_linuxapp | 5 + lib/librte_eal/common/include/rte_pci.h | 1 + lib/librte_eal/common/include/rte_pci_dev_ids.h | 16 ++ lib/librte_eal/linuxapp/Makefile | 3 + lib/librte_eal/linuxapp/eal/eal_pci.c | 4 + lib/librte_eal/linuxapp/ena_uio/Makefile | 55 +++++ lib/librte_eal/linuxapp/ena_uio/ena_uio_driver.c | 276 +++++++++++++++++++++++ 7 files changed, 360 insertions(+) create mode 100644 lib/librte_eal/linuxapp/ena_uio/Makefile create mode 100644 lib/librte_eal/linuxapp/ena_uio/ena_uio_driver.c diff --git a/config/common_linuxapp b/config/common_linuxapp index 74bc515..1777c4e 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -249,6 +249,11 @@ CONFIG_RTE_LIBRTE_CXGBE_DEBUG_TX=n CONFIG_RTE_LIBRTE_CXGBE_DEBUG_RX=n # +# Compile burst-oriented Amazon ENA PMD driver +# +CONFIG_RTE_EAL_ENA_UIO=y + +# # Compile burst-oriented Cisco ENIC PMD driver # CONFIG_RTE_LIBRTE_ENIC_PMD=y diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h index 334c12e..201a5a7 100644 --- a/lib/librte_eal/common/include/rte_pci.h +++ b/lib/librte_eal/common/include/rte_pci.h @@ -149,6 +149,7 @@ enum rte_kernel_driver { RTE_KDRV_VFIO, RTE_KDRV_UIO_GENERIC, RTE_KDRV_NIC_UIO, + RTE_KDRV_ENA_UIO, RTE_KDRV_NONE, }; diff --git a/lib/librte_eal/common/include/rte_pci_dev_ids.h b/lib/librte_eal/common/include/rte_pci_dev_ids.h index d088191..0600e03 100644 --- a/lib/librte_eal/common/include/rte_pci_dev_ids.h +++ b/lib/librte_eal/common/include/rte_pci_dev_ids.h @@ -152,6 +152,15 @@ #define RTE_PCI_DEV_ID_DECL_BNX2XVF(vend, dev) #endif +#ifndef RTE_PCI_DEV_ID_DECL_ENA +#define RTE_PCI_DEV_ID_DECL_ENA(vend, dev) +#endif + +#ifndef PCI_VENDOR_ID_AMAZON +/** Vendor ID used by Amazon devices */ +#define PCI_VENDOR_ID_AMAZON 0x1D0F +#endif + #ifndef PCI_VENDOR_ID_INTEL /** Vendor ID used by Intel devices */ #define PCI_VENDOR_ID_INTEL 0x8086 @@ -598,6 +607,12 @@ RTE_PCI_DEV_ID_DECL_VMXNET3(PCI_VENDOR_ID_VMWARE, VMWARE_DEV_ID_VMXNET3) RTE_PCI_DEV_ID_DECL_FM10KVF(PCI_VENDOR_ID_INTEL, FM10K_DEV_ID_VF) +/****************** Amazon devices ******************/ + +#define PCI_DEVICE_ID_ENA_VF 0xEC20 + +RTE_PCI_DEV_ID_DECL_ENA(PCI_VENDOR_ID_AMAZON, PCI_DEVICE_ID_ENA_VF) + /****************** Cisco VIC devices ******************/ #define PCI_DEVICE_ID_CISCO_VIC_ENET 0x0043 /* ethernet vnic */ @@ -656,6 +671,7 @@ RTE_PCI_DEV_ID_DECL_BNX2X(PCI_VENDOR_ID_BROADCOM, BNX2X_DEV_ID_57840_MF) */ #undef RTE_PCI_DEV_ID_DECL_BNX2X #undef RTE_PCI_DEV_ID_DECL_BNX2XVF +#undef RTE_PCI_DEV_ID_DECL_ENA #undef RTE_PCI_DEV_ID_DECL_EM #undef RTE_PCI_DEV_ID_DECL_IGB #undef RTE_PCI_DEV_ID_DECL_IGBVF diff --git a/lib/librte_eal/linuxapp/Makefile b/lib/librte_eal/linuxapp/Makefile index d9c5233..b293b1c 100644 --- a/lib/librte_eal/linuxapp/Makefile +++ b/lib/librte_eal/linuxapp/Makefile @@ -31,6 +31,9 @@ include $(RTE_SDK)/mk/rte.vars.mk +ifeq ($(CONFIG_RTE_EAL_ENA_UIO),y) +DIRS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += ena_uio +endif ifeq ($(CONFIG_RTE_EAL_IGB_UIO),y) DIRS-$(CONFIG_RTE_LIBRTE_EAL_LINUXAPP) += igb_uio endif diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index bc5b5be..b05a564 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -137,6 +137,7 @@ pci_map_device(struct rte_pci_device *dev) #endif break; case RTE_KDRV_IGB_UIO: + case RTE_KDRV_ENA_UIO: case RTE_KDRV_UIO_GENERIC: /* map resources for devices that use uio */ ret = pci_uio_map_resource(dev); @@ -161,6 +162,7 @@ pci_unmap_device(struct rte_pci_device *dev) RTE_LOG(ERR, EAL, "Hotplug doesn't support vfio yet\n"); break; case RTE_KDRV_IGB_UIO: + case RTE_KDRV_ENA_UIO: case RTE_KDRV_UIO_GENERIC: /* unmap resources for devices that use uio */ pci_uio_unmap_resource(dev); @@ -355,6 +357,8 @@ pci_scan_one(const char *dirname, uint16_t domain, uint8_t bus, if (!ret) { if (!strcmp(driver, "vfio-pci")) dev->kdrv = RTE_KDRV_VFIO; + else if (!strcmp(driver, "ena_uio")) + dev->kdrv = RTE_KDRV_ENA_UIO; else if (!strcmp(driver, "igb_uio")) dev->kdrv = RTE_KDRV_IGB_UIO; else if (!strcmp(driver, "uio_pci_generic")) diff --git a/lib/librte_eal/linuxapp/ena_uio/Makefile b/lib/librte_eal/linuxapp/ena_uio/Makefile new file mode 100644 index 0000000..10c4dc6 --- /dev/null +++ b/lib/librte_eal/linuxapp/ena_uio/Makefile @@ -0,0 +1,55 @@ +# +# BSD LICENSE +# +# Copyright (c) 2015-2016 Amazon.com, Inc. or its affiliates. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of copyright holder nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +# + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# module name and path +# +MODULE = ena_uio +MODULE_PATH = drivers/net/ena_uio + +# +# CFLAGS +# +MODULE_CFLAGS += -I$(SRCDIR) --param max-inline-insns-single=100 +MODULE_CFLAGS += -I$(RTE_OUTPUT)/include +MODULE_CFLAGS += -Winline -Wall -Werror +MODULE_CFLAGS += -include $(RTE_OUTPUT)/include/rte_config.h + +# +# all source are stored in SRCS-y +# +SRCS-y += ena_uio_driver.c + +include $(RTE_SDK)/mk/rte.module.mk diff --git a/lib/librte_eal/linuxapp/ena_uio/ena_uio_driver.c b/lib/librte_eal/linuxapp/ena_uio/ena_uio_driver.c new file mode 100644 index 0000000..8bf8a9e --- /dev/null +++ b/lib/librte_eal/linuxapp/ena_uio/ena_uio_driver.c @@ -0,0 +1,276 @@ +/*- +* BSD LICENSE +* +* Copyright (c) 2015-2016 Amazon.com, Inc. or its affiliates. +* All rights reserved. +* +* Redistribution and use in source and binary forms, with or without +* modification, are permitted provided that the following conditions +* are met: +* +* * Redistributions of source code must retain the above copyright +* notice, this list of conditions and the following disclaimer. +* * Redistributions in binary form must reproduce the above copyright +* notice, this list of conditions and the following disclaimer in +* the documentation and/or other materials provided with the +* distribution. +* * Neither the name of copyright holder nor the names of its +* contributors may be used to endorse or promote products derived +* from this software without specific prior written permission. +* +* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +*/ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "rte_pci_dev_ids.h" + +#define DRV_MODULE_NAME "ena_uio" +#define DRV_MODULE_VERSION "1.0" +#define DRV_MODULE_RELDATE "January 14, 2016" + +/** + Defines from uio_driver.h +*/ + +/* defines for uio_info->irq */ +#define UIO_IRQ_CUSTOM -1 +#define UIO_IRQ_NONE 0 + + +/* defines for uio_port->porttype */ +#define UIO_PORT_NONE 0 +#define UIO_PORT_X86 1 +#define UIO_PORT_GPIO 2 +#define UIO_PORT_OTHER 3 + +/* defines for muio_mem->memtype */ +#define UIO_MEM_NONE 0 +#define UIO_MEM_PHYS 1 +#define UIO_MEM_LOGICAL 2 +#define UIO_MEM_VIRTUAL 3 +#define UIO_MEM_PHYS_CACHEABLE 4 +#define UIO_MEM_PHYS_WC 5 + +#define UIO_MMAP_PHYSICAL_CACHE 0x1 +#define UIO_MMAP_PHYSICAL_WC 0x2 + + +MODULE_AUTHOR("Alex Matushevsky "); +MODULE_DESCRIPTION("UIO driver for Amazon ENA PCI cards"); +MODULE_LICENSE("GPL"); +MODULE_VERSION(DRV_MODULE_VERSION); + + +/* PCI BARs */ +#define ENA_REG_BAR 0 + + +typedef enum { + ENA = 0 +} board_t; + +static DEFINE_PCI_DEVICE_TABLE(ena_uio_pci_tbl) = { + { PCI_VENDOR_ID_AMAZON, PCI_DEVICE_ID_ENA_VF, + PCI_ANY_ID, PCI_ANY_ID, 0, 0, ENA}, + { 0, } +}; + +MODULE_DEVICE_TABLE(pci, ena_uio_pci_tbl); + + +/** + * A structure describing the private information for a uio device. + */ +struct rte_uio_pci_dev { + struct uio_info info; + struct pci_dev *pdev; + spinlock_t lock; /* spinlock for accessing PCI config space or msix data in multi tasks/isr */ +}; + + +static int +ena_uio_pci_irqcontrol(struct uio_info *info, s32 irq_state) +{ + return -1; +} + +static irqreturn_t +ena_uio_pci_irqhandler(int irq, struct uio_info *info) +{ + return IRQ_NONE; +} + +static int +ena_uio_pci_setup_iomem(struct pci_dev *dev, struct uio_info *info, + int n, int pci_bar, const char *name) +{ + unsigned long addr, len; + void *internal_addr; + + addr = pci_resource_start(dev, pci_bar); + len = pci_resource_len(dev, pci_bar); + + printk(KERN_INFO "addr len %lu f %lu\n", addr, len); + if (addr == 0 || len == 0) + return -1; + + internal_addr = ioremap(addr, len); + + printk(KERN_INFO "internal addr %p for %d len %lu \n", internal_addr, n, len); + if (internal_addr == NULL) { + + return -1; + } + info->mem[n].name = name; + info->mem[n].addr = addr; + info->mem[n].internal_addr = internal_addr; + info->mem[n].size = len; + info->mem[n].memtype = UIO_MEM_PHYS; + + return 0; +} + +/* Unmap previously ioremap'd resources */ +static void +ena_uio_pci_release_iomem(struct uio_info *info) +{ + int i; + for (i = 0; i < MAX_UIO_MAPS; i++) { + if (info->mem[i].internal_addr) + iounmap(info->mem[i].internal_addr); + } +} + +static int +ena_uio_pci_probe(struct pci_dev *dev, const struct pci_device_id *id) +{ + struct rte_uio_pci_dev *udev; + + udev = kzalloc(sizeof(struct rte_uio_pci_dev), GFP_KERNEL); + if (!udev) + return -ENOMEM; + + /* + * enable device: ask low-level code to enable I/O and + * memory + */ + if (pci_enable_device(dev)) { + printk(KERN_ERR "Cannot enable PCI device\n"); + goto fail_free; + } + + if (pci_set_dma_mask(dev, DMA_BIT_MASK(40))) { + printk(KERN_ERR "Cannot set DMA mask\n"); + goto fail_disable; + } + + if (pci_set_consistent_dma_mask(dev, DMA_BIT_MASK(40))) { + printk(KERN_ERR "Cannot set DMA mask\n"); + goto fail_disable; + } + + /* + * reserve device's PCI memory regions for use by this + * module + */ + if (pci_request_regions(dev, "ena_uio")) { + printk(KERN_ERR "Cannot request regions\n"); + goto fail_disable; + } + + /* enable bus mastering on the device */ + pci_set_master(dev); + + /* remap IO memory */ + if (ena_uio_pci_setup_iomem(dev, &udev->info, 0, ENA_REG_BAR, "bar0")) + goto fail_release_iomem; + + /* fill uio infos */ + udev->info.name = "ENA UIO"; + udev->info.version = "1.0"; + udev->info.handler = ena_uio_pci_irqhandler; + udev->info.irqcontrol = ena_uio_pci_irqcontrol; + udev->info.priv = udev; + udev->pdev = dev; + spin_lock_init(&udev->lock); + + /* irq is not supported */ + udev->info.irq_flags = 0; + udev->info.irq = -1; + + pci_set_drvdata(dev, udev); + + /* register uio driver */ + if (uio_register_device(&dev->dev, &udev->info)) + goto fail_release_iomem; + + return 0; + +fail_release_iomem: + ena_uio_pci_release_iomem(&udev->info); + + pci_release_regions(dev); +fail_disable: + pci_disable_device(dev); +fail_free: + kfree(udev); + + return -ENODEV; +} + + +static void +ena_uio_pci_remove(struct pci_dev *dev) +{ + struct uio_info *info = pci_get_drvdata(dev); + + if (info->priv == NULL) { + return; + } + + uio_unregister_device(info); + ena_uio_pci_release_iomem(info); + + pci_release_regions(dev); + pci_disable_device(dev); + pci_set_drvdata(dev, NULL); + kfree(info); +} + +static struct pci_driver ena_uio_pci_driver = { + .name = DRV_MODULE_NAME, + .id_table = ena_uio_pci_tbl, + .probe = ena_uio_pci_probe, + .remove = ena_uio_pci_remove, +}; + +static int __init ena_uio_init(void) +{ + return pci_register_driver(&ena_uio_pci_driver); +} + +static void __exit ena_uio_cleanup(void) +{ + pci_unregister_driver(&ena_uio_pci_driver); +} + +module_init(ena_uio_init); +module_exit(ena_uio_cleanup); -- 1.9.1