From: Neil Horman <nhorman@tuxdriver.com>
To: Stephen Hemminger <stephen@networkplumber.org>
Cc: dev@dpdk.org, Stephen Hemminger <shemming@brocade.com>
Subject: Re: [dpdk-dev] [PATCH 1/4] pci: allow access to PCI config space
Date: Mon, 11 May 2015 08:54:54 -0400 [thread overview]
Message-ID: <20150511125454.GA8310@hmsreliant.think-freely.org> (raw)
In-Reply-To: <1431041135-6289-2-git-send-email-stephen@networkplumber.org>
On Thu, May 07, 2015 at 04:25:32PM -0700, Stephen Hemminger wrote:
> From: Stephen Hemminger <shemming@brocade.com>
>
> Some drivers need ability to access PCI config (for example for power
> management). This adds an abstraction to do this; only implemented
> on Linux, but should be possible on BSD.
>
> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
> ---
> lib/librte_eal/common/include/rte_pci.h | 28 +++++++++++++++
> lib/librte_eal/linuxapp/eal/eal_pci.c | 48 +++++++++++++++++++++++++
> lib/librte_eal/linuxapp/eal/eal_pci_init.h | 11 ++++++
> lib/librte_eal/linuxapp/eal/eal_pci_uio.c | 14 ++++++++
> lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 16 +++++++++
> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 ++
> 6 files changed, 119 insertions(+)
>
> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
> index 223d3cd..cea982a 100644
> --- a/lib/librte_eal/common/include/rte_pci.h
> +++ b/lib/librte_eal/common/include/rte_pci.h
> @@ -393,6 +393,34 @@ void rte_eal_pci_register(struct rte_pci_driver *driver);
> */
> void rte_eal_pci_unregister(struct rte_pci_driver *driver);
>
> +/**
> + * Read PCI config space.
> + *
> + * @param device
> + * A pointer to a rte_pci_device structure describing the device
> + * to use
> + * @param buf
> + * A data buffer where the bytes should be read into
> + * @param size
> + * The length of the data buffer.
> + */
> +int rte_eal_pci_read_config(const struct rte_pci_device *device,
> + void *buf, size_t len, off_t offset);
> +
> +/**
> + * Write PCI config space.
> + *
> + * @param device
> + * A pointer to a rte_pci_device structure describing the device
> + * to use
> + * @param buf
> + * A data buffer containing the bytes should be written
> + * @param size
> + * The length of the data buffer.
> + */
> +int rte_eal_pci_write_config(const struct rte_pci_device *device,
> + const void *buf, size_t len, off_t offset);
> +
I still think this needs a BSD implementation before we pull the whole thing in.
Only partially implementing infrastructure like this is bad practice, and will
lead to complicated build procedures (i.e. developers will require institutional
knoweldge to know that bnx2x can't build on BSD because someone still needs to
implement these functions on bsd. Even having them just return -ENOTSUPP would
be preferable, so that you got a proper run time error, rather than a build
break, though I don't think thats even necessecary, as pci passthrough is
possible in project like bhyve (implying user space pci access)
Neil
> #ifdef __cplusplus
> }
> #endif
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c
> index d2adc66..6d79a08 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c
> @@ -756,6 +756,54 @@ rte_eal_pci_close_one_driver(struct rte_pci_driver *dr __rte_unused,
> }
> #endif /* RTE_LIBRTE_EAL_HOTPLUG */
>
> +/* Read PCI config space. */
> +int rte_eal_pci_read_config(const struct rte_pci_device *device,
> + void *buf, size_t len, off_t offset)
> +{
> + const struct rte_intr_handle *intr_handle = &device->intr_handle;
> +
> + switch (intr_handle->type) {
> + case RTE_INTR_HANDLE_UIO:
> + return pci_uio_read_config(intr_handle, buf, len, offset);
> +
> +#ifdef VFIO_PRESENT
> + case RTE_INTR_HANDLE_VFIO_MSIX:
> + case RTE_INTR_HANDLE_VFIO_MSI:
> + case RTE_INTR_HANDLE_VFIO_LEGACY:
> + return pci_vfio_read_config(intr_handle, buf, len, offset);
> +#endif
> + default:
> + RTE_LOG(ERR, EAL,
> + "Unknown handle type of fd %d\n",
> + intr_handle->fd);
> + return -1;
> + }
> +}
> +
> +/* Write PCI config space. */
> +int rte_eal_pci_write_config(const struct rte_pci_device *device,
> + const void *buf, size_t len, off_t offset)
> +{
> + const struct rte_intr_handle *intr_handle = &device->intr_handle;
> +
> + switch (intr_handle->type) {
> + case RTE_INTR_HANDLE_UIO:
> + return pci_uio_write_config(intr_handle, buf, len, offset);
> +
> +#ifdef VFIO_PRESENT
> + case RTE_INTR_HANDLE_VFIO_MSIX:
> + case RTE_INTR_HANDLE_VFIO_MSI:
> + case RTE_INTR_HANDLE_VFIO_LEGACY:
> + return pci_vfio_write_config(intr_handle, buf, len, offset);
> +#endif
> + default:
> + RTE_LOG(ERR, EAL,
> + "Unknown handle type of fd %d\n",
> + intr_handle->fd);
> + return -1;
> + }
> +}
> +
> /* Init the PCI EAL subsystem */
> int
> rte_eal_pci_init(void)
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_init.h b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
> index aa7b755..c28e5b0 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_init.h
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_init.h
> @@ -68,6 +68,11 @@ void *pci_find_max_end_va(void);
> void *pci_map_resource(void *requested_addr, int fd, off_t offset,
> size_t size, int additional_flags);
>
> +int pci_uio_read_config(const struct rte_intr_handle *intr_handle,
> + void *buf, size_t len, off_t offs);
> +int pci_uio_write_config(const struct rte_intr_handle *intr_handle,
> + const void *buf, size_t len, off_t offs);
> +
> /* map IGB_UIO resource prototype */
> int pci_uio_map_resource(struct rte_pci_device *dev);
>
> @@ -86,6 +91,12 @@ int pci_vfio_enable(void);
> int pci_vfio_is_enabled(void);
> int pci_vfio_mp_sync_setup(void);
>
> +/* access config space */
> +int pci_vfio_read_config(const struct rte_intr_handle *intr_handle,
> + void *buf, size_t len, off_t offs);
> +int pci_vfio_write_config(const struct rte_intr_handle *intr_handle,
> + const void *buf, size_t len, off_t offs);
> +
> /* map VFIO resource prototype */
> int pci_vfio_map_resource(struct rte_pci_device *dev);
> int pci_vfio_get_group_fd(int iommu_group_fd);
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> index 2d1c69b..bbc59ed 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_uio.c
> @@ -58,6 +58,20 @@ EAL_REGISTER_TAILQ(rte_uio_tailq)
>
> #define OFF_MAX ((uint64_t)(off_t)-1)
>
> +int
> +pci_uio_read_config(const struct rte_intr_handle *intr_handle,
> + void *buf, size_t len, off_t offset)
> +{
> + return pread(intr_handle->uio_cfg_fd, buf, len, offset);
> +}
> +
> +int
> +pci_uio_write_config(const struct rte_intr_handle *intr_handle,
> + const void *buf, size_t len, off_t offset)
> +{
> + return pwrite(intr_handle->uio_cfg_fd, buf, len, offset);
> +}
> +
> static int
> pci_uio_set_bus_master(int dev_fd)
> {
> diff --git a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
> index aea1fb1..092a369 100644
> --- a/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
> +++ b/lib/librte_eal/linuxapp/eal/eal_pci_vfio.c
> @@ -77,6 +77,22 @@ EAL_REGISTER_TAILQ(rte_vfio_tailq)
> /* per-process VFIO config */
> static struct vfio_config vfio_cfg;
>
> +int
> +pci_vfio_read_config(const struct rte_intr_handle *intr_handle,
> + void *buf, size_t len, off_t offs)
> +{
> + return pread64(intr_handle->vfio_dev_fd, buf, len,
> + VFIO_GET_REGION_ADDR(VFIO_PCI_CONFIG_REGION_INDEX) + offs);
> +}
> +
> +int
> +pci_vfio_write_config(const struct rte_intr_handle *intr_handle,
> + const void *buf, size_t len, off_t offs)
> +{
> + return pwrite64(intr_handle->vfio_dev_fd, buf, len,
> + VFIO_GET_REGION_ADDR(VFIO_PCI_CONFIG_REGION_INDEX) + offs);
> +}
> +
> /* get PCI BAR number where MSI-X interrupts are */
> static int
> pci_vfio_get_msix_bar(int fd, int *msix_bar, uint32_t *msix_table_offset,
> diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
> index 7e850a9..494aae0 100644
> --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
> +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
> @@ -42,9 +42,11 @@ DPDK_2.0 {
> rte_eal_pci_dump;
> rte_eal_pci_probe;
> rte_eal_pci_probe_one;
> + rte_eal_pci_read_config;
> rte_eal_pci_register;
> rte_eal_pci_scan;
> rte_eal_pci_unregister;
> + rte_eal_pci_write_config;
> rte_eal_process_type;
> rte_eal_remote_launch;
> rte_eal_tailq_lookup;
> --
> 2.1.4
>
>
next prev parent reply other threads:[~2015-05-11 12:55 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-07 23:25 [dpdk-dev] [PATCH 0/4 v4] bnx2x: new poll mode driver Stephen Hemminger
2015-05-07 23:25 ` [dpdk-dev] [PATCH 1/4] pci: allow access to PCI config space Stephen Hemminger
2015-05-11 12:54 ` Neil Horman [this message]
[not found] ` <38426478085b4e779e18967cd1b6ae4f@BRMWP-EXMB11.corp.brocade.com>
2015-05-11 15:23 ` Stephen Hemminger
2015-05-11 15:37 ` Neil Horman
2015-05-11 17:30 ` Stephen Hemminger
2015-05-11 17:31 ` Stephen Hemminger
2015-05-12 9:56 ` Bruce Richardson
2015-05-07 23:25 ` [dpdk-dev] [PATCH 2/4] bnx2x: new poll mode driver (part1) Stephen Hemminger
2015-05-07 23:25 ` [dpdk-dev] [PATCH 3/4] bnx2x: new poll mode driver (part2) Stephen Hemminger
2015-05-07 23:25 ` [dpdk-dev] [PATCH 4/4] bnx2x: enable BNX2X poll mode driver Stephen Hemminger
-- strict thread matches above, loose matches on Subject: below --
2015-02-06 18:36 [dpdk-dev] [PATCH 0/4] Broadcom 10G NIC Poll Mode Driver Stephen Hemminger
2015-02-06 18:36 ` [dpdk-dev] [PATCH 1/4] pci: allow access to PCI config space Stephen Hemminger
2015-02-09 9:45 ` David Marchand
2015-02-10 23:49 ` Stephen Hemminger
[not found] ` <c058fdefd5564f3f96bd21593c9ea19a@BRMWP-EXMB11.corp.brocade.com>
2015-02-11 1:23 ` Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150511125454.GA8310@hmsreliant.think-freely.org \
--to=nhorman@tuxdriver.com \
--cc=dev@dpdk.org \
--cc=shemming@brocade.com \
--cc=stephen@networkplumber.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).