Re: [dpdk-dev] Nehalem Intel Xeon X5506 architecture and PCIe NIC association to Numa node

[Bruce Richardson <bruce.richardson@intel.com>]

On Thu, Feb 08, 2018 at 04:27:36PM +0000, Huertas García, Víctor wrote:
> 
> Hi all,
> 
> After having tried many ways to make the PCIe NIC card appear associated to a numa node, I haven't been able to do it.
> That is, every time I try to look at which numa node belongs it always returns -1.
> 
> $ cat /sys/bus/pci/devices/0000\:04\:00.1/numa_node
> -1
> 
> $ cat /sys/bus/pci/devices/0000\:04\:00.0/numa_node
> -1
> 
> Using lstopo, I confirm that all PCI cards are "outside" of  any Numa node.
> 
> I have read in previous posted messages in dpdk-dev community that this is normal in Nehalem generation Xeon architecture and there is nothing I can do about it. Can somebody confirm this?

For that generation architecture, it is indeed expected. The NICs are
not directly connected to any NUMA node.

> If so, what implications could this have on packet capture and performance?

Unsurprisingly, it's the case that newer platforms will perform better,
as you are missing out on performance benefits from improved cores and
also features like Intel® DDIO [1].
However, what I/O throughput are you hoping to get from your system?
Depending on your requirements, what you have may be enough. Some people
use DPDK on lower-end platforms because that is all that they need. You
may also find the chart on slide 6 of [2] of use to see how the max
throughput of a platform has improved over time (and has improved
further since that chart was published).

> 
> Are the NICs available in my DPDK applications? Do I have to specifically "add" them by "-w 04:00.1 - w 04:00.0"?

Yes, your NICs will still be available, even without NUMA affinity, and
no, you should not need to explicitly whitelist them - though you can if
you want. So long as they are bound to a uio or vfio driver (e.g.
igb_uio or vfio-pci), they should be detected by DPDK EAL init and made
available to your app.

> Is RSS supported and usable from the DPDK application?

Yes, at least for Intel NICs, and probably most other DPDK-supported
NICs too.

> 
> Thanks a lot for your attention
> 
> Victor
> 

/Bruce

[1] https://www.intel.com/content/www/us/en/io/data-direct-i-o-technology.html
[2] https://dpdksummit.com/Archive/pdf/2016Germany/DPDK-2016-DPDK_FD_IO_Introduction.pdf

PS: This is a public list, so email disclaimers are rather pointless.
It's best if they can be removed from mails sent here.

> ________________________________
> 
> Este correo electrónico y, en su caso, cualquier fichero anexo al mismo, contiene información de carácter confidencial exclusivamente dirigida a su destinatario o destinatarios. Si no es vd. el destinatario indicado, queda notificado que la lectura, utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. En el caso de haber recibido este correo electrónico por error, se ruega notificar inmediatamente esta circunstancia mediante reenvío a la dirección electrónica del remitente.
> Evite imprimir este mensaje si no es estrictamente necesario.
> 
> This email and any file attached to it (when applicable) contain(s) confidential information that is exclusively addressed to its recipient(s). If you are not the indicated recipient, you are informed that reading, using, disseminating and/or copying it without authorisation is forbidden in accordance with the legislation in effect. If you have received this email by mistake, please immediately notify the sender of the situation by resending it to their email address.
> Avoid printing this message if it is not absolutely necessary.