From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lf0-f65.google.com (mail-lf0-f65.google.com [209.85.215.65]) by dpdk.org (Postfix) with ESMTP id 080261B59E for ; Fri, 29 Jun 2018 14:17:28 +0200 (CEST) Received: by mail-lf0-f65.google.com with SMTP id m13-v6so6633408lfb.12 for ; Fri, 29 Jun 2018 05:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WPzKN/+Jm1syAx/3y0hWW2WWUwBF00KCZX8QjZjkls0=; b=w+I1+kPql7TIKyAESYIfB3qyX0IwQvKa8VSX1C23oT1Ah5kuSQmwfV4DCqd5mesnYb 6EiZg100kmckhGG/4CJNgJTzOkwJzF39+/9AoM8EYwn8Q2w1gC4lieU+80MYp8lVOWva xlv9JIuNAhEPbWwDKZZ4iX60z04iAcFCffOlnORvoD7C7CYHYzZ8RAIL/utga9FAUswb HUHOGZhi0MW8DtBZfWTN8jSZx6Hn6/JvbdHcrUvto3/agTOjkFnh6IRKQ7fB4ddm3MvD sJm9KDuk9zf5p5DNKfEqbS2PK7AOxhWkhOkePyfPut06hgV1E0N1/Xl1HZNJB2wmCPvT 5EKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WPzKN/+Jm1syAx/3y0hWW2WWUwBF00KCZX8QjZjkls0=; b=nykDUQvht99rU5FOAKkjxxDFvotS/NCJZug2ChgWwh8FbBLWOhwK2sd6uhPWtoiqv/ zWMKrexCsL8ifDLpzJriEzACK9RrlDu2IIyfaWP+OdAqjRyYdir6XIdDnxtFoAsmRpGM bxD/2WuuhCd490CqnVeq58imXEF2OESn9J9pPKFpyaXSXbaHlcive9+J7lX50Zv+r8wj 6GCdbC2ngApQVgw9IgULIK4GM/SRMndULp/TJwVW0RDksngnXh+vpsjGd0CieXPnhTON lp1FkJVEAtNpl/LJ3MQ1ctlfOtrPK3HjKgssDSghAyAQWZ910zK3EcF0rwBkSX41d5y3 Et4Q== X-Gm-Message-State: APt69E3JmxYDbC4mcREvJrDfK64C3gqg9SP6Jo4bq6iNUcaWQwik9xwD zMrtkgUrRhRBQiUSUAjLF8KsTO2FbBZCXw== X-Google-Smtp-Source: AAOMgpclpum9YUsoVzrFwhinQzYD+WJ3mGFR52FsPA+I+8KX2sp8QrOMDthF+XT4UfMmrDNr/GUxuQ== X-Received: by 2002:a19:6d11:: with SMTP id i17-v6mr9455605lfc.103.1530274647433; Fri, 29 Jun 2018 05:17:27 -0700 (PDT) Received: from rafalkozik.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id r21-v6sm1527333lja.92.2018.06.29.05.17.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 29 Jun 2018 05:17:26 -0700 (PDT) From: Rafal Kozik To: dev@dpdk.org Cc: mw@semihalf.com, mk@semihalf.com, gtzalik@amazon.com, evgenys@amazon.com, matua@amazon.com, igorch@amazon.com, thomas@monjalon.net, ferruh.yigit@intel.com, Kozik Date: Fri, 29 Jun 2018 14:17:16 +0200 Message-Id: <1530274637-10156-4-git-send-email-rk@semihalf.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1530274637-10156-1-git-send-email-rk@semihalf.com> References: <1530267873-7236-2-git-send-email-rk@semihalf.com> <1530274637-10156-1-git-send-email-rk@semihalf.com> Subject: [dpdk-dev] [PATCH v4 3/4] eal: enable WC during resources mapping X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2018 12:17:28 -0000 From: Kozik Write combining (WC) increases NIC performance by making better utilization of PCI bus, but cannot be used by all PMDs. It will be enabled only if RTE_PCI_DRV_WC_ACTIVATE will be set in drivers flags. For proper work also igb_uio driver must be loaded with wc_activate set to 1. When mapping PCI resources, firstly check if it supports WC and try to use it. In case of failure, it will fall-back to normal mode. Signed-off-by: Rafal Kozik Acked-by: Bruce Richardson --- drivers/bus/pci/linux/pci_uio.c | 47 ++++++++++++++++++++++++++++++----------- drivers/bus/pci/rte_bus_pci.h | 2 ++ 2 files changed, 37 insertions(+), 12 deletions(-) diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c index d423e4b..a7c1442 100644 --- a/drivers/bus/pci/linux/pci_uio.c +++ b/drivers/bus/pci/linux/pci_uio.c @@ -282,22 +282,19 @@ int pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx, struct mapped_pci_resource *uio_res, int map_idx) { - int fd; + int fd = -1; char devname[PATH_MAX]; void *mapaddr; struct rte_pci_addr *loc; struct pci_map *maps; + int wc_activate = 0; + + if (dev->driver != NULL) + wc_activate = dev->driver->drv_flags & RTE_PCI_DRV_WC_ACTIVATE; loc = &dev->addr; maps = uio_res->maps; - /* update devname for mmap */ - snprintf(devname, sizeof(devname), - "%s/" PCI_PRI_FMT "/resource%d", - rte_pci_get_sysfs_path(), - loc->domain, loc->bus, loc->devid, - loc->function, res_idx); - /* allocate memory to keep path */ maps[map_idx].path = rte_malloc(NULL, strlen(devname) + 1, 0); if (maps[map_idx].path == NULL) { @@ -309,11 +306,37 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx, /* * open resource file, to mmap it */ - fd = open(devname, O_RDWR); - if (fd < 0) { - RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", + if (wc_activate) { + /* update devname for mmap */ + snprintf(devname, sizeof(devname), + "%s/" PCI_PRI_FMT "/resource%d_wc", + rte_pci_get_sysfs_path(), + loc->domain, loc->bus, loc->devid, + loc->function, res_idx); + + if (access(devname, R_OK|W_OK) != -1) { + fd = open(devname, O_RDWR); + if (fd < 0) + RTE_LOG(INFO, EAL, "%s cannot be mapped. " + "Fall-back to non prefetchable mode.\n", + devname); + } + } + + if (!wc_activate || fd < 0) { + snprintf(devname, sizeof(devname), + "%s/" PCI_PRI_FMT "/resource%d", + rte_pci_get_sysfs_path(), + loc->domain, loc->bus, loc->devid, + loc->function, res_idx); + + /* then try to map resource file */ + fd = open(devname, O_RDWR); + if (fd < 0) { + RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", devname, strerror(errno)); - goto error; + goto error; + } } /* try mapping somewhere close to the end of hugepages */ diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 458e6d0..828acc5 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -135,6 +135,8 @@ struct rte_pci_bus { /** Device needs PCI BAR mapping (done with either IGB_UIO or VFIO) */ #define RTE_PCI_DRV_NEED_MAPPING 0x0001 +/** Device needs PCI BAR mapping with enabled write combining (wc) */ +#define RTE_PCI_DRV_WC_ACTIVATE 0x0002 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 /** Device driver supports device removal interrupt */ -- 2.7.4