* [dpdk-dev] RES: RES: RES: RES: hw.nic_uio.bdfs
[not found] ` <!&!AAAAAAAAAAAuAAAAAAAAAD0Npi+x74NOlwmRBONHU/QBANlTnCJhprtFudq2LHCBs8EBACQA//8AABAAAACm+y6gpJugQ4PdPZ1Q7TTXAQAAAAA=@hotmail.com>
@ 2014-03-27 22:06 ` Fred Pedrisa
0 siblings, 0 replies; 2+ messages in thread
From: Fred Pedrisa @ 2014-03-27 22:06 UTC (permalink / raw)
To: 'Carew, Alan', 'Masaru Oki'; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 7032 bytes --]
Hi !
I've attached my contribution (the fixed source) by changing the behavior
the way Alan wanted :)
It is working and if you like to use it, this would be cool.
Sincerely,
Fred Pedrisa
-----Mensagem original-----
De: Fred Pedrisa [mailto:fredhps10@hotmail.com]
Enviada em: quinta-feira, 27 de março de 2014 17:28
Para: 'Carew, Alan'; 'Masaru Oki'
Cc: 'dev@dpdk.org'
Assunto: RES: [dpdk-dev] RES: RES: RES: hw.nic_uio.bdfs
Hello,
It just requires a small code change :), and it can work in the expected
way.
So you mean bdfs is to 'select' the only wanted devices yes ? May I change
my code proposition for you ?
Sincerely,
Fred Pedrisa
-----Mensagem original-----
De: Carew, Alan [mailto:alan.carew@intel.com] Enviada em: quinta-feira, 27
de março de 2014 17:12
Para: Fred Pedrisa; Masaru Oki
Cc: dev@dpdk.org
Assunto: [dpdk-dev] RES: RES: RES: hw.nic_uio.bdfs
Hi folks,
Just to clarify, the purpose of hw.nic_uio.bdfs is to remove NICs from
drivers not owned by nic_uio and add them to a pool of unbound devices.
As with the Linux equivalent driver(igb_uio), nic_uio will then attempt to
bind any unbound NICs and make them available for DPDK.
However, the Linux OS also has the ability to selectively bind/unbind
devices via sysfs, this is something we lack on FreeBSD.
I can see where it would be a problem when only a subset of unbound devices
is required for nic_uio and currently the only option would be to ensure the
original driver is loaded first.
The change below changes the purpose of hw.nic_uio.bdfs, i.e. blacklist
devices so that they are not used by nic_uio.
A better approach would be to make the variable an exclusive list of devices
to be used, this would then require all users to specify the exact devices
to be used.
Thanks,
Alan
-----Mensagem original-----
De: dev [mailto:dev-bounces at dpdk.org] Em nome de Fred Pedrisa Enviada em:
quarta-feira, 26 de março de 2014 04:22
Para: 'Masaru Oki'
Cc: dev at dpdk.org
Assunto: [dpdk-dev] RES: RES: hw.nic_uio.bdfs
Hello,
Here is my fix for probe code :
static int
nic_uio_probe (device_t dev)
{
int i, len;
char *remaining;
long bus = 0, device = 0, function = 0;
remaining = bdf_str;
len = strlen(remaining);
for (i = 0; remaining && len >= 5 && i < len;i+=6) {
if ( remaining[i + 1] == ':' && remaining[i + 3] == ':' ) {
bus = strtol(&remaining[i + 0],NULL,0);
device = strtol(&remaining[i + 2],NULL,0);
function = strtol(&remaining[i + 4],NULL,0);
if (dev != NULL) {
if (pci_get_bus(dev) == bus &&
pci_get_slot(dev) == device && pci_get_function(dev) == function) {
printf("nic_uio: success blocking
probe of : %ld:%ld:%ld!\n", bus, device, function);
return (ENXIO);
}
}
}
}
for (i = 0; i < NUM_DEVICES; i++)
if (pci_get_vendor(dev) == devices[i].vend &&
pci_get_device(dev) == devices[i].dev) {
device_set_desc(dev, "Intel(R) DPDK PCI Device");
return (BUS_PROBE_SPECIFIC);
}
return (ENXIO);
}
Now it is working as intended ;)
-----Mensagem original-----
De: dev [mailto:dev-bounces at dpdk.org] Em nome de Fred Pedrisa Enviada em:
quarta-feira, 26 de março de 2014 04:16
Para: 'Masaru Oki'
Cc: dev at dpdk.org
Assunto: [dpdk-dev] RES: hw.nic_uio.bdfs
Hello,
Yes, I am writing a fix for this too ;)
De: Masaru Oki [mailto:m-oki at stratosphere.co.jp] Enviada em:
quarta-feira,
26 de março de 2014 04:08
Para: Fred Pedrisa
Cc: dev at dpdk.org
Assunto: Re: [dpdk-dev] hw.nic_uio.bdfs
> By default nic_uio takes all the NICs for itself
Yes.
I think nic_uio_probe should check hw.nic_uio.bdfs.
2014-03-26 15:49 GMT+09:00 Fred Pedrisa <fredhps10 at hotmail.com>:
Hello,
By default nic_uio takes all the NICs for itself
So in theory, you needed an option to reserve some NIC ports to your system,
without DPDK taking it for itself
De: Masaru Oki [mailto:m-oki at stratosphere.co.jp] Enviada em:
quarta-feira,
26 de março de 2014 03:43
Para: Fred Pedrisa
Cc: dev at dpdk.org
Assunto: Re: [dpdk-dev] hw.nic_uio.bdfs
avoid??? want you hw.nic_uio.avoid_bdfs?
nic_uio behavior I guessed
1. detach kernel driver specified by hw.nic_uio.bdfs
2. attach nic_uio driver for all NICs not attached.
but 2. is not correct, I think.
2014-03-26 15:20 GMT+09:00 Fred Pedrisa <fredhps10 at hotmail.com>:
Hello,
You did not understand the purpose of that parameter, it is made to 'avoid'
nic_uio from pursuing the wanted NICs... so they are free to be used in the
system :)
Right now the code to handle it is wrong and I am trying to fix it myself.
De: Masaru Oki [mailto:m-oki at stratosphere.co.jp] Enviada em:
quarta-feira,
26 de março de 2014 03:16
Para: Fred Pedrisa
Cc: dev at dpdk.org
Assunto: Re: [dpdk-dev] hw.nic_uio.bdfs
Hi,
I tried with Intel version 1.6.0 and FreeBSD 9.2-RELEASE on VMware Player.
kldload nic_uio by hand, works fine.
But kldunload nic_uio only detach uio driver, don't re-attach kernel driver.
[oki@ ~]$ cat /boot/loader.conf
##############################################################
### User settings ##########################################
##############################################################
hw.contigmem.num_buffers=64
hw.contigmem.buffer_size=2097152
hw.nic_uio.bdfs="2:5:0,2:6:0"
contigmem_load="YES"
#nic_uio_load="YES"
[oki@ ~]$ pciconf -l | egrep '(em|uio)'
em0 at pci0:2:1:0: class=0x020000 card=0x075015ad chip=0x100f8086 rev=0x01
hdr=0x00
em1 at pci0:2:5:0: class=0x020000 card=0x075015ad chip=0x100f8086 rev=0x01
hdr=0x00
em2 at pci0:2:6:0: class=0x020000 card=0x075015ad chip=0x100f8086 rev=0x01
hdr=0x00
[oki@ ~]$ kenv hw.nic_uio.bdfs
2:5:0,2:6:0
[oki@ ~]$ sudo kldload nic_uio
Password:
[oki@ ~]$ pciconf -l | egrep '(em|uio)'
em0 at pci0:2:1:0: class=0x020000 card=0x075015ad chip=0x100f8086 rev=0x01
hdr=0x00
nic_uio0 at pci0:2:5:0: class=0x020000 card=0x075015ad chip=0x100f8086
rev=0x01 hdr=0x00
nic_uio1 at pci0:2:6:0: class=0x020000 card=0x075015ad chip=0x100f8086
rev=0x01 hdr=0x00
[oki@ ~]$ sudo kldunload nic_uio
[oki@ ~]$ pciconf -l | egrep '(em|uio)'
em0 at pci0:2:1:0: class=0x020000 card=0x075015ad chip=0x100f8086 rev=0x01
hdr=0x00
[oki@ ~]$
2014-03-26 14:35 GMT+09:00 Fred Pedrisa <fredhps10 at hotmail.com>:
Hi, guys.
This variable is not working as intended for FreeBSD :(
It does not dettach nic_uio from the wanted ports :/
[-- Attachment #2: nic_uio.c --]
[-- Type: text/plain, Size: 9924 bytes --]
/* -
* BSD LICENSE
*
* Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in
* the documentation and/or other materials provided with the
* distribution.
* * Neither the name of Intel Corporation nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
#include <sys/param.h> /* defines used in kernel.h */
#include <sys/module.h>
#include <sys/kernel.h> /* types used in module initialization */
#include <sys/conf.h> /* cdevsw struct */
#include <sys/bus.h> /* structs, prototypes for pci bus stuff and DEVMETHOD */
#include <sys/rman.h>
#include <sys/systm.h>
#include <sys/proc.h>
#include <machine/bus.h>
#include <dev/pci/pcivar.h> /* For pci_get macros! */
#include <dev/pci/pcireg.h> /* The softc holds our per-instance data. */
#include <vm/vm.h>
#include <vm/uma.h>
#include <vm/vm_object.h>
#include <vm/vm_page.h>
#include <vm/vm_pager.h>
#define MAX_BARS (PCIR_MAX_BAR_0 + 1)
struct nic_uio_softc {
device_t dev_t;
struct cdev *my_cdev;
int bar_id[MAX_BARS];
struct resource *bar_res[MAX_BARS];
u_long bar_start[MAX_BARS];
u_long bar_size[MAX_BARS];
};
/* Function prototypes */
static d_open_t nic_uio_open;
static d_close_t nic_uio_close;
static d_mmap_t nic_uio_mmap;
static d_mmap_single_t nic_uio_mmap_single;
static int nic_uio_probe(device_t dev);
static int nic_uio_attach(device_t dev);
static int nic_uio_detach(device_t dev);
static int nic_uio_shutdown(void);
static int nic_uio_modevent(module_t mod, int type, void *arg);
static struct cdevsw uio_cdevsw = {
.d_name = "nic_uio",
.d_version = D_VERSION,
.d_open = nic_uio_open,
.d_close = nic_uio_close,
.d_mmap = nic_uio_mmap,
.d_mmap_single = nic_uio_mmap_single,
};
static device_method_t nic_uio_methods[] = {
DEVMETHOD(device_probe, nic_uio_probe),
DEVMETHOD(device_attach, nic_uio_attach),
DEVMETHOD(device_detach, nic_uio_detach),
DEVMETHOD_END
};
struct device {
int vend;
int dev;
};
struct pci_bdf {
uint32_t bus;
uint32_t devid;
uint32_t function;
};
#define RTE_PCI_DEV_ID_DECL_EM(vend, dev) {vend, dev},
#define RTE_PCI_DEV_ID_DECL_IGB(vend, dev) {vend, dev},
#define RTE_PCI_DEV_ID_DECL_IGBVF(vend, dev) {vend, dev},
#define RTE_PCI_DEV_ID_DECL_IXGBE(vend, dev) {vend, dev},
#define RTE_PCI_DEV_ID_DECL_IXGBEVF(vend, dev) {vend, dev},
#define RTE_PCI_DEV_ID_DECL_VIRTIO(vend, dev) {vend, dev},
const struct device devices[] = {
#include <rte_pci_dev_ids.h>
};
#define NUM_DEVICES (sizeof(devices)/sizeof(devices[0]))
static devclass_t nic_uio_devclass;
DEFINE_CLASS_0(nic_uio, nic_uio_driver, nic_uio_methods, sizeof(struct nic_uio_softc));
DRIVER_MODULE(nic_uio, pci, nic_uio_driver, nic_uio_devclass, nic_uio_modevent, 0);
static char bdf_str[1024];
static int
nic_uio_mmap(struct cdev *cdev, vm_ooffset_t offset, vm_paddr_t *paddr,
int prot, vm_memattr_t *memattr)
{
*paddr = offset;
return (0);
}
static int
nic_uio_mmap_single(struct cdev *cdev, vm_ooffset_t *offset, vm_size_t size,
struct vm_object **obj, int nprot)
{
/*
* The BAR index is encoded in the offset. Divide the offset by
* PAGE_SIZE to get the index of the bar requested by the user
* app.
*/
unsigned bar = *offset/PAGE_SIZE;
struct nic_uio_softc *sc = cdev->si_drv1;
if (bar >= MAX_BARS)
return EINVAL;
if (sc->bar_res[bar] == NULL) {
sc->bar_id[bar] = PCIR_BAR(bar);
if (PCI_BAR_IO(pci_read_config(sc->dev_t, sc->bar_id[bar], 4)))
sc->bar_res[bar] = bus_alloc_resource_any(sc->dev_t, SYS_RES_IOPORT,
&sc->bar_id[bar], RF_ACTIVE);
else
sc->bar_res[bar] = bus_alloc_resource_any(sc->dev_t, SYS_RES_MEMORY,
&sc->bar_id[bar], RF_ACTIVE);
}
if (sc->bar_res[bar] == NULL)
return ENXIO;
sc->bar_start[bar] = rman_get_start(sc->bar_res[bar]);
sc->bar_size[bar] = rman_get_size(sc->bar_res[bar]);
device_printf(sc->dev_t, "Bar %u @ %lx, size %lx\n", bar,
sc->bar_start[bar], sc->bar_size[bar]);
*offset = sc->bar_start[bar];
*obj = vm_pager_allocate(OBJT_DEVICE, cdev, size, nprot, *offset,
curthread->td_ucred);
return 0;
}
int
nic_uio_open(struct cdev *dev, int oflags, int devtype, d_thread_t *td)
{
return 0;
}
int
nic_uio_close(struct cdev *dev, int fflag, int devtype, d_thread_t *td)
{
return 0;
}
static int
nic_uio_probe (device_t dev)
{
int i, len;
char *remaining;
long bus = 0, device = 0, function = 0;
remaining = bdf_str;
len = strlen(remaining);
for (i = 0; remaining && len >= 5 && i < len;i+=6) {
if ( remaining[i + 1] == ':' && remaining[i + 3] == ':' ) {
bus = strtol(&remaining[i + 0],NULL,0);
device = strtol(&remaining[i + 2],NULL,0);
function = strtol(&remaining[i + 4],NULL,0);
if (dev != NULL) {
if (pci_get_bus(dev) == bus && pci_get_slot(dev) == device && pci_get_function(dev) == function) {
for (i = 0; i < NUM_DEVICES; i++) {
if (pci_get_vendor(dev) == devices[i].vend && pci_get_device(dev) == devices[i].dev) {
printf("nic_uio: success probing the wanted device : %ld:%ld:%ld!\n", bus, device, function);
device_set_desc(dev, "Intel(R) DPDK PCI Device");
return (BUS_PROBE_SPECIFIC);
}
}
}
}
}
}
return (ENXIO);
}
static int
nic_uio_attach(device_t dev)
{
int i;
struct nic_uio_softc *sc;
sc = device_get_softc(dev);
sc->dev_t = dev;
sc->my_cdev = make_dev(&uio_cdevsw, device_get_unit(dev),
UID_ROOT, GID_WHEEL, 0600, "uio@pci:%u:%u:%u",
pci_get_bus(dev), pci_get_slot(dev), pci_get_function(dev));
if (sc->my_cdev == NULL)
return ENXIO;
sc->my_cdev->si_drv1 = sc;
for (i = 0; i < MAX_BARS; i++)
sc->bar_res[i] = NULL;
pci_enable_busmaster(dev);
return 0;
}
static int
nic_uio_detach(device_t dev)
{
int i;
struct nic_uio_softc *sc;
sc = device_get_softc(dev);
for (i = 0; i < MAX_BARS; i++)
if (sc->bar_res[i] != NULL) {
if (PCI_BAR_IO(pci_read_config(dev, sc->bar_id[i], 4)))
bus_release_resource(dev, SYS_RES_IOPORT, sc->bar_id[i],
sc->bar_res[i]);
else
bus_release_resource(dev, SYS_RES_MEMORY, sc->bar_id[i],
sc->bar_res[i]);
}
if (sc->my_cdev != NULL)
destroy_dev(sc->my_cdev);
return 0;
}
static void
nic_uio_load(void)
{
char *remaining;
long bus = 0, device = 0, function = 0;
int i, j, len;
device_t dev;
memset(bdf_str, 0, sizeof(bdf_str));
TUNABLE_STR_FETCH("hw.nic_uio.bdfs", bdf_str, sizeof(bdf_str));
remaining = bdf_str;
len = strlen(remaining);
for (i = 0; remaining && len >= 5 && i < len;i+=6) {
if ( remaining[i + 1] == ':' && remaining[i + 3] == ':' ) {
bus = strtol(&remaining[i + 0],NULL,0);
device = strtol(&remaining[i + 2],NULL,0);
function = strtol(&remaining[i + 4],NULL,0);
printf("nic_uio: trying to dettach %ld:%ld:%ld...\n", bus, device, function);
dev = pci_find_bsf(bus, device, function);
if (dev != NULL) {
for (j = 0; j < NUM_DEVICES; j++) {
if (pci_get_vendor(dev) == devices[j].vend && pci_get_device(dev) == devices[j].dev) {
printf("nic_uio: success in dettachment of : %ld:%ld:%ld!\n", bus, device, function);
device_detach(dev); }
}
}
}
}
}
static void
nic_uio_unload(void)
{
}
static int
nic_uio_shutdown(void)
{
return (0);
}
static int
nic_uio_modevent(module_t mod, int type, void *arg)
{
switch (type) {
case MOD_LOAD:
nic_uio_load();
break;
case MOD_UNLOAD:
nic_uio_unload();
break;
case MOD_SHUTDOWN:
nic_uio_shutdown();
break;
default:
break;
}
return (0);
}
^ permalink raw reply [flat|nested] 2+ messages in thread