From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 79D1CA04B5; Tue, 12 Jan 2021 09:23:41 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 37984140D21; Tue, 12 Jan 2021 09:23:41 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mails.dpdk.org (Postfix) with ESMTP id 5555A140D19 for ; Tue, 12 Jan 2021 09:23:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1610439819; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kOYxTYPJbHS3kZWEnscVNUr1al5Rhm0lQaQqC94Se3U=; b=UZLbG5hV2v9gUP3+xKFemIlOj379qYKM05IIelivZAZKebpwBKks33abkpGaJAJNW7T80T Y+x5x7JAc0AjVoh/6yFIuWM/IbUG0dbAqobTFp2dKvclNtC5/NIfHnUu4LEtuKcSKZ5vLI at76BX2bkr9oz3xZsL2D2hvKXZMTKWI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-460-GUoocDliOwaEZHkujaAyJw-1; Tue, 12 Jan 2021 03:23:35 -0500 X-MC-Unique: GUoocDliOwaEZHkujaAyJw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7D741107ACF8; Tue, 12 Jan 2021 08:23:34 +0000 (UTC) Received: from [10.36.110.24] (unknown [10.36.110.24]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3EB1F6A909; Tue, 12 Jan 2021 08:23:25 +0000 (UTC) To: =?UTF-8?B?6LCi5Y2O5LyfKOatpOaXtuatpOWIu++8iQ==?= , ferruh.yigit@intel.com Cc: dev@dpdk.org, anatoly.burakov@intel.com, david.marchand@redhat.com, zhihong.wang@intel.com, chenbo.xia@intel.com, grive@u256.net References: <68ecd941-9c56-4de7-fae2-2ad15bdfd81a@alibaba-inc.com> <1603381885-88819-1-git-send-email-huawei.xhw@alibaba-inc.com> <1603381885-88819-3-git-send-email-huawei.xhw@alibaba-inc.com> From: Maxime Coquelin Message-ID: <9dfad03c-d1db-5756-d222-2e9c2f8da65a@redhat.com> Date: Tue, 12 Jan 2021 09:23:23 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <1603381885-88819-3-git-send-email-huawei.xhw@alibaba-inc.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH v5 2/3] PCI: support MMIO in rte_pci_ioport_map/unap/read/write X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Title should be something like: "bus/pci: support MMIO in PCI ioport accessors On 10/22/20 5:51 PM, 谢华伟(此时此刻） wrote: > From: "huawei.xhw" > > If IO BAR, we get PIO address. > If MMIO BAR, we get mapped virtual address. > We distinguish PIO and MMIO by their address like how kernel does. > ioread/write8/16/32 is provided to access PIO/MMIO. > BTW, for virtio on arch other than x86, BAR flag indicates PIO but is mapped. No acronym in the commit message. Also, I am not sure to understand this comment. Does it means in the case of ARM for example, the IORESOURCE_IO flag is set but the base address is above PIO_MAX? > > Signed-off-by: huawei.xhw As in previous patch, we need your full name for the sign-off. > --- > drivers/bus/pci/linux/pci.c | 4 -- > drivers/bus/pci/linux/pci_uio.c | 123 ++++++++++++++++++++++++++-------------- > 2 files changed, 82 insertions(+), 45 deletions(-) > > diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c > index 0f38abf..0dc99e9 100644 > --- a/drivers/bus/pci/linux/pci.c > +++ b/drivers/bus/pci/linux/pci.c > @@ -715,8 +715,6 @@ int rte_pci_write_config(const struct rte_pci_device *device, > break; > #endif > case RTE_PCI_KDRV_IGB_UIO: > - pci_uio_ioport_read(p, data, len, offset); > - break; I think this part should be in patch 1. > case RTE_PCI_KDRV_UIO_GENERIC: > pci_uio_ioport_read(p, data, len, offset); > break; > @@ -736,8 +734,6 @@ int rte_pci_write_config(const struct rte_pci_device *device, > break; > #endif > case RTE_PCI_KDRV_IGB_UIO: > - pci_uio_ioport_write(p, data, len, offset); > - break; Same here. > case RTE_PCI_KDRV_UIO_GENERIC: > pci_uio_ioport_write(p, data, len, offset); > break; > diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c > index 01f2a40..c19382f 100644 > --- a/drivers/bus/pci/linux/pci_uio.c > +++ b/drivers/bus/pci/linux/pci_uio.c > @@ -379,14 +379,9 @@ > char buf[BUFSIZ]; > uint64_t phys_addr, end_addr, flags; > unsigned long base; > + bool iobar; > int i; > > - if (rte_eal_iopl_init() != 0) { > - RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n", > - __func__, dev->name); > - return -1; > - } > - > /* open and read addresses of the corresponding resource in sysfs */ > snprintf(filename, sizeof(filename), "%s/" PCI_PRI_FMT "/resource", > rte_pci_get_sysfs_path(), dev->addr.domain, dev->addr.bus, > @@ -408,15 +403,30 @@ > &end_addr, &flags) < 0) > goto error; > > - if (!(flags & IORESOURCE_IO)) { > - RTE_LOG(ERR, EAL, "%s(): bar resource other than IO is not supported\n", __func__); > + if (flags & IORESOURCE_IO) { > + iobar = 1; > + base = (unsigned long)phys_addr; > + RTE_LOG(INFO, EAL, "%s(): PIO BAR %08lx detected\n", __func__, base); > + } else if (flags & IORESOURCE_MEM) { > + iobar = 0; > + base = (unsigned long)dev->mem_resource[bar].addr; > + RTE_LOG(INFO, EAL, "%s(): MMIO BAR %08lx detected\n", __func__, base); > + } else { > + RTE_LOG(ERR, EAL, "%s(): unknown BAR type\n", __func__); > + goto error; > + } > + > + > + if (iobar && rte_eal_iopl_init() != 0) { > + RTE_LOG(ERR, EAL, "%s(): insufficient ioport permissions for PCI device %s\n", > + __func__, dev->name); > goto error; > } > - base = (unsigned long)phys_addr; > - RTE_LOG(INFO, EAL, "%s(): PIO BAR %08lx detected\n", __func__, base); > > - if (base > UINT16_MAX) > + if (iobar && (base > UINT16_MAX)) { > + RTE_LOG(ERR, EAL, "%s(): %08lx too large PIO resource\n", __func__, base); > goto error; > + } It looks like above check could be moved directly to (flags & IORESOURCE_IO) case, so iobar boolean is not needed. > > /* FIXME only for primary process ? */ > if (dev->intr_handle.type == RTE_INTR_HANDLE_UNKNOWN) { > @@ -517,6 +527,61 @@ > } > #endif > > +#define PIO_MAX 0x10000 > +static inline uint8_t ioread8(void *addr) > +{ > + uint8_t val; > + > + val = (uint64_t)(uintptr_t)addr >= PIO_MAX ? > + *(volatile uint8_t *)addr : > + inb((unsigned long)addr); > + > + return val; > +} > + > +static inline uint16_t ioread16(void *addr) > +{ > + uint16_t val; > + > + val = (uint64_t)(uintptr_t)addr >= PIO_MAX ? > + *(volatile uint16_t *)addr : > + inw((unsigned long)addr); > + > + return val; > +} > + > +static inline uint32_t ioread32(void *addr) > +{ > + uint32_t val; > + > + val = (uint64_t)(uintptr_t)addr >= PIO_MAX ? > + *(volatile uint32_t *)addr : > + inl((unsigned long)addr); > + > + return val; > +} > + > +static inline void iowrite8(uint8_t val, void *addr) > +{ > + (uint64_t)(uintptr_t)addr >= PIO_MAX ? > + *(volatile uint8_t *)addr = val : > + outb(val, (unsigned long)addr); > +} > + > +static inline void iowrite16(uint16_t val, void *addr) > +{ > + (uint64_t)(uintptr_t)addr >= PIO_MAX ? > + *(volatile uint16_t *)addr = val : > + outw(val, (unsigned long)addr); > +} > + > +static inline void iowrite32(uint32_t val, void *addr) > +{ > + (uint64_t)(uintptr_t)addr >= PIO_MAX ? > + *(volatile uint32_t *)addr = val : > + outl(val, (unsigned long)addr); > +} > + > void > pci_uio_ioport_read(struct rte_pci_ioport *p, > void *data, size_t len, off_t offset) > @@ -528,25 +593,13 @@ > for (d = data; len > 0; d += size, reg += size, len -= size) { > if (len >= 4) { > size = 4; > -#if defined(RTE_ARCH_X86) > - *(uint32_t *)d = inl(reg); > -#else > - *(uint32_t *)d = *(volatile uint32_t *)reg; > -#endif > + *(uint32_t *)d = ioread32((void *)reg); > } else if (len >= 2) { > size = 2; > -#if defined(RTE_ARCH_X86) > - *(uint16_t *)d = inw(reg); > -#else > - *(uint16_t *)d = *(volatile uint16_t *)reg; > -#endif > + *(uint16_t *)d = ioread16((void *)reg); > } else { > size = 1; > -#if defined(RTE_ARCH_X86) > - *d = inb(reg); > -#else > - *d = *(volatile uint8_t *)reg; > -#endif > + *d = ioread8((void *)reg); > } > } > } > @@ -562,25 +615,13 @@ > for (s = data; len > 0; s += size, reg += size, len -= size) { > if (len >= 4) { > size = 4; > -#if defined(RTE_ARCH_X86) > - outl_p(*(const uint32_t *)s, reg); > -#else > - *(volatile uint32_t *)reg = *(const uint32_t *)s; > -#endif > + iowrite32(*(const uint32_t *)s, (void *)reg); > } else if (len >= 2) { > size = 2; > -#if defined(RTE_ARCH_X86) > - outw_p(*(const uint16_t *)s, reg); > -#else > - *(volatile uint16_t *)reg = *(const uint16_t *)s; > -#endif > + iowrite16(*(const uint16_t *)s, (void *)reg); > } else { > size = 1; > -#if defined(RTE_ARCH_X86) > - outb_p(*s, reg); > -#else > - *(volatile uint8_t *)reg = *s; > -#endif > + iowrite8(*s, (void *)reg); > } > } > } >