From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) by dpdk.org (Postfix) with ESMTP id D7E138D4D for ; Fri, 29 Jun 2018 10:35:05 +0200 (CEST) Received: by mail-lj1-f194.google.com with SMTP id l12-v6so6671046lja.1 for ; Fri, 29 Jun 2018 01:35:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=r5IE+2py6+Lg1GCWXXxg3Th5Tj9MN7TwZmNviEByWy8=; b=Q2XQpH04lUgnOaz/sP6QiQU+5eS5O907hOiCLbYt65btW3PdgRRNCaq2yzdQAfR/rU jGXtpq1x0IVwIgI6Fb9M1Uv1fKVSnB+pZzi12E/unx+cFGVXBMnKGmvaUaalrYtNcQZr BHUVTlBfjshKP3fI66TPZRaMcNLs0niA6qDOvIHVnPJl3vKA+6MOT2XJbgmG9k9z0qTg PaSyaZU7rTYJiK6CJvx/SqCvH7G61kHf8GGi6BWEE1KYt+UfHlcT0RsBBhJoBYT2C3mI UVyA4mpFrbEIxwX0yZUN8P7bJRYo7/cDdQ+IX42ZedJ2E9P6cl+LxO9xT+sV/JLLQ9VQ owOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=r5IE+2py6+Lg1GCWXXxg3Th5Tj9MN7TwZmNviEByWy8=; b=O3QtBnSi64qVAQlASj3jxN+hl5sr+5jLCLfsgTlCun9ZHtugLhLS7gLHpCTqVQwmIK uj1uzBHYaokqLq3VDKNfl8jKhmNYafuWAiQmmfcoHBnRPvpakLjCRlB9iPyjlQVMyd+W RMbLwYD/a15g3gRGso2P5pfzEU3Yfy43qfbfXgJLKTyuxtuIjjItwLOYVuw0EtFDP4mM BAtC/p5Of9uRLEcJu3aKEEc1Jc7WNKCv+nqQZ/WrOh8HnpvkNrWqDOo85g7pstLW53Mq KOS00LMiCGtTxhM5VYR07dvG4UksECc3JaDkGlwZzqZmRq65cvuHfuK23WIGXYIDotj+ +9vQ== X-Gm-Message-State: APt69E1IuEjyg3WOvAlls/n3xyRwGKD26iQ4zgzwI5jeMtcRFg+ckCvd 705XNJvQxbIIjrNjMGmzDMHUGTTaBYyNKtAluSneeg== X-Google-Smtp-Source: AAOMgpdqzsNWm2DChRJGJdhQnK0r2PUkpmSwMel6KU7vAm7YV0P0lvKpW3o+/DHzAChtGf6SFiRVoH46w63Wm4q++eg= X-Received: by 2002:a2e:3a0e:: with SMTP id h14-v6mr7504386lja.20.1530261305400; Fri, 29 Jun 2018 01:35:05 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a19:4a95:0:0:0:0:0 with HTTP; Fri, 29 Jun 2018 01:35:04 -0700 (PDT) In-Reply-To: <356e2563-5b3e-510e-7a01-f735f6c33641@intel.com> References: <1530191753-18689-1-git-send-email-rk@semihalf.com> <1530191753-18689-2-git-send-email-rk@semihalf.com> <356e2563-5b3e-510e-7a01-f735f6c33641@intel.com> From: =?UTF-8?B?UmFmYcWCIEtvemlr?= Date: Fri, 29 Jun 2018 10:35:04 +0200 Message-ID: To: Ferruh Yigit Cc: dev@dpdk.org, Marcin Wojtas , =?UTF-8?Q?Micha=C5=82_Krawczyk?= , "Tzalik, Guy" , "Schmeilin, Evgeny" , "Matushevsky, Alexander" , "Chauskin, Igor" , Thomas Monjalon Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v2 1/4] igb_uio: add wc option X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2018 08:35:06 -0000 2018-06-28 16:32 GMT+02:00 Ferruh Yigit : > On 6/28/2018 2:15 PM, Rafal Kozik wrote: >> Write combining (WC) increases NIC performance by making better >> utilization of PCI bus, but cannot be use by all PMDs. >> >> To get internal_addr memory need to be mapped. But as memory could not be >> mapped twice: with and without WC, it should be skipped for WC. [1] >> >> To do not spoil other drivers that potentially could use internal_addr, >> parameter wc_activate adds possibility to skip it for those PMDs, >> that do not use it. >> >> [1] https://www.kernel.org/doc/ols/2008/ols2008v2-pages-135-144.pdf >> section 5.3 and 5.4 > > Hi Rafal, > > Thank you for more information but I have a few more question: > > - What do you mean "But as memory could not be mapped twice: with and without WC"? > > ioremap() maps the physical address for kernel usage, and via uio we are mapping > it to userspace, do you mean these two? > > - "internal_addr" is should be for kernel sage not for DPDK drivers which are in > the userspace, why it is a concern for us? > > - What happens if you don't update this code at all? Won't you able to map > device address into userspace? > I tested adding RTE_PCI_DRV_WC_ACTIVATE to i40e, on top of your patch, and able > to map without igb_uio update. > I am not able to understand need of the modification. > Hello Ferruh, I was not precisely. Memory could be mapped multiple time, but cannot be mapped with and without WC support simultaneously. When not setting wc_activate memory mapping work, but silently fall-back to non prefetchable mode. I perform measurements of writing speed. When parameter wc_activate was set I get 4.81 GB/s. Without this parameter result was 0.07 GB/s. Code used for testing is located here: gist.github.com/semihalf-kozik-rafal/327208cd52a2fac2d12250028becf9b3 Best regards, Rafal >> >> Signed-off-by: Rafal Kozik >> Acked-by: Bruce Richardson >> --- >> kernel/linux/igb_uio/igb_uio.c | 17 ++++++++++++++--- >> 1 file changed, 14 insertions(+), 3 deletions(-) >> >> diff --git a/kernel/linux/igb_uio/igb_uio.c b/kernel/linux/igb_uio/igb_uio.c >> index b3233f1..3382fb1 100644 >> --- a/kernel/linux/igb_uio/igb_uio.c >> +++ b/kernel/linux/igb_uio/igb_uio.c >> @@ -30,6 +30,7 @@ struct rte_uio_pci_dev { >> int refcnt; >> }; >> >> +static int wc_activate; >> static char *intr_mode; >> static enum rte_intr_mode igbuio_intr_mode_preferred = RTE_INTR_MODE_MSIX; >> /* sriov sysfs */ >> @@ -375,9 +376,13 @@ igbuio_pci_setup_iomem(struct pci_dev *dev, struct uio_info *info, >> len = pci_resource_len(dev, pci_bar); >> if (addr == 0 || len == 0) >> return -1; >> - internal_addr = ioremap(addr, len); >> - if (internal_addr == NULL) >> - return -1; >> + if (wc_activate == 0) { >> + internal_addr = ioremap(addr, len); >> + if (internal_addr == NULL) >> + return -1; >> + } else { >> + internal_addr = NULL; >> + } >> info->mem[n].name = name; >> info->mem[n].addr = addr; >> info->mem[n].internal_addr = internal_addr; >> @@ -650,6 +655,12 @@ MODULE_PARM_DESC(intr_mode, >> " " RTE_INTR_MODE_LEGACY_NAME " Use Legacy interrupt\n" >> "\n"); >> >> +module_param(wc_activate, int, 0); >> +MODULE_PARM_DESC(wc_activate, >> +"Activate support for write combining (WC) (default=0)\n" >> +" 0 - disable\n" >> +" other - enable\n"); >> + >> MODULE_DESCRIPTION("UIO driver for Intel IGB PCI cards"); >> MODULE_LICENSE("GPL"); >> MODULE_AUTHOR("Intel Corporation"); >> >