From: Ferruh Yigit <ferruh.yigit@intel.com>
To: "Rafał Kozik" <rk@semihalf.com>
Cc: dev@dpdk.org, "Marcin Wojtas" <mw@semihalf.com>,
"Michał Krawczyk" <mk@semihalf.com>,
"Tzalik, Guy" <gtzalik@amazon.com>,
"Schmeilin, Evgeny" <evgenys@amazon.com>,
"Matushevsky, Alexander" <matua@amazon.com>,
"Chauskin, Igor" <igorch@amazon.com>,
"Thomas Monjalon" <thomas@monjalon.net>
Subject: Re: [dpdk-dev] [PATCH v2 1/4] igb_uio: add wc option
Date: Fri, 29 Jun 2018 11:08:31 +0100 [thread overview]
Message-ID: <fa3e1d75-b87d-8c9b-2a90-0c5d0c3000e0@intel.com> (raw)
In-Reply-To: <CAMG3L49WS5ZsNZic=Wej_bPSdK7ca=Bw4EvSjXEnMgmAnLB9ww@mail.gmail.com>
On 6/29/2018 9:35 AM, Rafał Kozik wrote:
> 2018-06-28 16:32 GMT+02:00 Ferruh Yigit <ferruh.yigit@intel.com>:
>> On 6/28/2018 2:15 PM, Rafal Kozik wrote:
>>> Write combining (WC) increases NIC performance by making better
>>> utilization of PCI bus, but cannot be use by all PMDs.
>>>
>>> To get internal_addr memory need to be mapped. But as memory could not be
>>> mapped twice: with and without WC, it should be skipped for WC. [1]
>>>
>>> To do not spoil other drivers that potentially could use internal_addr,
>>> parameter wc_activate adds possibility to skip it for those PMDs,
>>> that do not use it.
>>>
>>> [1] https://www.kernel.org/doc/ols/2008/ols2008v2-pages-135-144.pdf
>>> section 5.3 and 5.4
>>
>> Hi Rafal,
>>
>> Thank you for more information but I have a few more question:
>>
>> - What do you mean "But as memory could not be mapped twice: with and without WC"?
>>
>> ioremap() maps the physical address for kernel usage, and via uio we are mapping
>> it to userspace, do you mean these two?
>>
>> - "internal_addr" is should be for kernel sage not for DPDK drivers which are in
>> the userspace, why it is a concern for us?
>>
>> - What happens if you don't update this code at all? Won't you able to map
>> device address into userspace?
>> I tested adding RTE_PCI_DRV_WC_ACTIVATE to i40e, on top of your patch, and able
>> to map without igb_uio update.
>> I am not able to understand need of the modification.
>>
>
> Hello Ferruh,
>
> I was not precisely. Memory could be mapped multiple time,
> but cannot be mapped with and without WC support simultaneously.
> When not setting wc_activate memory mapping work, but silently
> fall-back to non prefetchable mode.
How can I confirm this silently fall-back behavior, is there any log can I turn
on in kernel or anything from proc/sysfs?
>
> I perform measurements of writing speed.
> When parameter wc_activate was set I get 4.81 GB/s.
> Without this parameter result was 0.07 GB/s.
> Code used for testing is located here:
> gist.github.com/semihalf-kozik-rafal/327208cd52a2fac2d12250028becf9b3
>
> Best regards,
> Rafal
>
>>>
>>> Signed-off-by: Rafal Kozik <rk@semihalf.com>
>>> Acked-by: Bruce Richardson <bruce.richardson@intel.com>
>>> ---
>>> kernel/linux/igb_uio/igb_uio.c | 17 ++++++++++++++---
>>> 1 file changed, 14 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/kernel/linux/igb_uio/igb_uio.c b/kernel/linux/igb_uio/igb_uio.c
>>> index b3233f1..3382fb1 100644
>>> --- a/kernel/linux/igb_uio/igb_uio.c
>>> +++ b/kernel/linux/igb_uio/igb_uio.c
>>> @@ -30,6 +30,7 @@ struct rte_uio_pci_dev {
>>> int refcnt;
>>> };
>>>
>>> +static int wc_activate;
>>> static char *intr_mode;
>>> static enum rte_intr_mode igbuio_intr_mode_preferred = RTE_INTR_MODE_MSIX;
>>> /* sriov sysfs */
>>> @@ -375,9 +376,13 @@ igbuio_pci_setup_iomem(struct pci_dev *dev, struct uio_info *info,
>>> len = pci_resource_len(dev, pci_bar);
>>> if (addr == 0 || len == 0)
>>> return -1;
>>> - internal_addr = ioremap(addr, len);
>>> - if (internal_addr == NULL)
>>> - return -1;
>>> + if (wc_activate == 0) {
>>> + internal_addr = ioremap(addr, len);
>>> + if (internal_addr == NULL)
>>> + return -1;
>>> + } else {
>>> + internal_addr = NULL;
>>> + }
>>> info->mem[n].name = name;
>>> info->mem[n].addr = addr;
>>> info->mem[n].internal_addr = internal_addr;
>>> @@ -650,6 +655,12 @@ MODULE_PARM_DESC(intr_mode,
>>> " " RTE_INTR_MODE_LEGACY_NAME " Use Legacy interrupt\n"
>>> "\n");
>>>
>>> +module_param(wc_activate, int, 0);
>>> +MODULE_PARM_DESC(wc_activate,
>>> +"Activate support for write combining (WC) (default=0)\n"
>>> +" 0 - disable\n"
>>> +" other - enable\n");
>>> +
>>> MODULE_DESCRIPTION("UIO driver for Intel IGB PCI cards");
>>> MODULE_LICENSE("GPL");
>>> MODULE_AUTHOR("Intel Corporation");
>>>
>>
next prev parent reply other threads:[~2018-06-29 10:08 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-11 14:07 [dpdk-dev] [PATCH 0/4] support for write combining Rafal Kozik
2018-04-11 14:07 ` [dpdk-dev] [PATCH 1/4] igb_uio: add wc option Rafal Kozik
2018-06-27 16:34 ` Ferruh Yigit
2018-06-28 13:08 ` Rafał Kozik
2018-06-28 13:15 ` [dpdk-dev] [PATCH v2 0/4] support for write combining Rafal Kozik
2018-06-28 13:15 ` [dpdk-dev] [PATCH v2 1/4] igb_uio: add wc option Rafal Kozik
2018-06-28 14:32 ` Ferruh Yigit
2018-06-29 8:35 ` Rafał Kozik
2018-06-29 10:08 ` Ferruh Yigit [this message]
2018-06-29 10:24 ` [dpdk-dev] [PATCH v3 0/4] support for write combining Rafal Kozik
2018-06-29 10:24 ` [dpdk-dev] [PATCH v3 1/4] igb_uio: add wc option Rafal Kozik
2018-06-29 10:53 ` Ferruh Yigit
2018-06-29 12:17 ` [dpdk-dev] [PATCH v4 0/4] Support for write combining Rafal Kozik
2018-06-29 12:17 ` [dpdk-dev] [PATCH v4 1/4] igb_uio: add wc option Rafal Kozik
2018-06-29 13:11 ` Rafał Kozik
2018-06-29 13:20 ` Ferruh Yigit
2018-06-29 13:40 ` Ferruh Yigit
2018-06-29 12:17 ` [dpdk-dev] [PATCH v4 2/4] bus/pci: reference driver structure Rafal Kozik
2018-06-29 12:17 ` [dpdk-dev] [PATCH v4 3/4] eal: enable WC during resources mapping Rafal Kozik
2018-06-29 12:17 ` [dpdk-dev] [PATCH v4 4/4] net/ena: enable WC Rafal Kozik
2018-06-29 13:54 ` [dpdk-dev] [PATCH v5 0/4] Support for write combining Rafal Kozik
2018-06-29 13:54 ` [dpdk-dev] [PATCH v5 1/4] igb_uio: add wc option Rafal Kozik
2018-06-29 13:54 ` [dpdk-dev] [PATCH v5 2/4] bus/pci: reference driver structure Rafal Kozik
2018-06-29 13:54 ` [dpdk-dev] [PATCH v5 3/4] eal: enable WC during resources mapping Rafal Kozik
2018-06-29 13:54 ` [dpdk-dev] [PATCH v5 4/4] net/ena: enable WC Rafal Kozik
2018-06-29 14:26 ` [dpdk-dev] [PATCH v5 0/4] Support for write combining Ferruh Yigit
2018-06-29 22:13 ` Thomas Monjalon
2018-06-29 10:24 ` [dpdk-dev] [PATCH v3 2/4] bus/pci: reference driver structure Rafal Kozik
2018-06-29 10:24 ` [dpdk-dev] [PATCH v3 3/4] eal: enable WC during resources mapping Rafal Kozik
2018-06-29 10:24 ` [dpdk-dev] [PATCH v3 4/4] net/ena: enable WC Rafal Kozik
2018-06-28 13:15 ` [dpdk-dev] [PATCH v2 2/4] bus/pci: reference driver structure Rafal Kozik
2018-06-28 13:15 ` [dpdk-dev] [PATCH v2 3/4] eal: enable WC during resources mapping Rafal Kozik
2018-06-28 14:50 ` Ferruh Yigit
2018-06-29 8:58 ` Rafał Kozik
2018-06-29 9:05 ` Ferruh Yigit
2018-06-29 10:28 ` Rafał Kozik
2018-06-29 10:37 ` Ferruh Yigit
2018-06-28 13:15 ` [dpdk-dev] [PATCH v2 4/4] net/ena: enable WC Rafal Kozik
2018-07-02 6:52 ` Michał Krawczyk
2018-04-11 14:07 ` [dpdk-dev] [PATCH 2/4] bus/pci: reference driver structure Rafal Kozik
2018-06-27 16:36 ` Ferruh Yigit
2018-06-28 13:05 ` Rafał Kozik
2018-04-11 14:07 ` [dpdk-dev] [PATCH 3/4] eal: enable WC during resources mapping Rafal Kozik
2018-06-27 16:41 ` Ferruh Yigit
2018-06-28 13:06 ` Rafał Kozik
2018-04-11 14:07 ` [dpdk-dev] [PATCH 4/4] net/ena: enable WC Rafal Kozik
2018-06-27 16:11 ` Thomas Monjalon
2018-06-28 13:04 ` Rafał Kozik
2018-04-11 14:42 ` [dpdk-dev] [PATCH 0/4] support for write combining Bruce Richardson
2018-04-16 11:36 ` Rafał Kozik
2018-04-27 8:27 ` Rafał Kozik
2018-04-27 11:30 ` Bruce Richardson
2018-04-30 8:07 ` Rafał Kozik
2018-04-30 8:58 ` Bruce Richardson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fa3e1d75-b87d-8c9b-2a90-0c5d0c3000e0@intel.com \
--to=ferruh.yigit@intel.com \
--cc=dev@dpdk.org \
--cc=evgenys@amazon.com \
--cc=gtzalik@amazon.com \
--cc=igorch@amazon.com \
--cc=matua@amazon.com \
--cc=mk@semihalf.com \
--cc=mw@semihalf.com \
--cc=rk@semihalf.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).