Use of the 'virtio-user' exception path interface

Use of the 'virtio-user' exception path interface

[Patrick Mahan]

This is on yocto-linux 4.19.87 on X86_64.  DPDK (stable) 18.11.11

I am working on a project where I am trying to make use of the 'virtio-user' 
exception path and it (sort of) seems to be working.  The issue I am trying to 
understand is the following:

Background -

Per the how-to documentation 
(https://doc.dpdk.org/guides-25.07/howto/virtio_user_as_exception_path.html), I 
have called rte_eal_init() passing in the PCI address of my physical interfaces. 
I am also creating a couple of these "exception" interfaces with the following code -

     char portname[32], portargs[256];
     char macaddr[6] = { 0x00, 0x0c, 0x1b, 0x29, 0x29, 0x71 };
     uint16_t portid;

     /**
      * Create the first 'iflan0'
      */
     snprintf(portargs, 256, 
"path=/dev/vhost-net,queues=2,queue_size=1024,iface=iflan0,mac=%02x:%02x:%02x:%02x:%02x:%02x",
macaddr[0], macaddr[1], macaddr[2], macaddr[3], macaddr[4], macaddr[5]);

     /**
      * Call the hotplug layer
      */
     if (rte_eal_hotplug_add("vdev", "virt-user0", portargs) < 0) {
         fprintf(stderr, "[DPDK::hotplug] failed to create iflan0 (virtio-user0)\n");
         return -1;
     }

     /**
      * Create the second 'ifwan0'
      */
     macaddr[5]++;
     snprintf(portargs, 256, 
"path=/dev/vhost-net,queues=2,queue_size=1024,iface=ifwan0,mac=%02x:%02x:%02x:%02x:%02x:%02x",
macaddr[0], macaddr[1], macaddr[2], macaddr[3], macaddr[4], macaddr[5]);

     /**
      * Call the hotplug layer
      */
     if (rte_eal_hotplug_add("vdev", "virt-user1", portargs) < 0) {
         fprintf(stderr, "[DPDK::hotplug] failed to create iflan0 (virtio-user0)\n");
         return -1;
     }

     ...

     RTE_ETH_FOREACH_DEV(portid) {
         if (port_init(portid, bufPool, NRXQUEUES, NTXQUEUES) < 0) {
             fprintf(stderr, "[DPDK::port_init] failed to init DPDK port %u",
                     portid);
             return -1;
         }
     }

     ...

     /* Retrieve the ifindex of iflan0 and ifwan0 */
     unsigned int iflan0_index = if_nametoindex("iflan0");
     unsigned int ifwan0_index = if_nametoindex("ifwan0");

     fprintf(stdout, "[DPDK] Exception interface 'iflan0': %u", iflan0_index);
     fprintf(stdout, "[DPDK] Exception interface 'ifwan0': %u", ifwan0_index);

     ... /* onto the packet processing loops */

The interfaces are instantiated and those two logs messages are seen -

. . .
[DPDK] Exception interface 'iflan0': 58
[DPDK] Exception interface 'ifwan0': 59
. . .

However, when I check the list of kernel interfaces using 'ip link show', I find:

pmahan-dpdk-v1: # ip link show
. . .
66: iflan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT 
group default qlen 1000
     link/ether 00:0c:1b:29:29:71 brd ff:ff:ff:ff:ff:ff
67: ifwan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT 
group default qlen 1000
     link/ether 00:0c:1b:29:29:72 brd ff:ff:ff:ff:ff:ff

So, some point after I created and started these ports and obtain the original 
ifindexes, they were deleted and re-created.  I have some clue from the IKE 
kernel log messages:
...
Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 11[KNL] interface iflan0 activated
Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 09[KNL] interface ifwan0 activated
...
Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 09[KNL] interface iflan0 deactivated
Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 13[KNL] interface iflan0 deleted
Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 12[KNL] interface ifwan0 deactivated
Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 05[KNL] interface ifwan0 deleted

I never see another IKE log about those interfaces being activated again 
(possibly because they show as DOWN).

Short of crawling through the guts of the virtio-user support in vdev, I thought 
I would start asking questions here.

Also, is there any specific logging I can enabled to see if I can understand why 
this is happening?

Thanks for any help,

Patrick

Re: Use of the 'virtio-user' exception path interface

[Stephen Hemminger]

On Sat, 27 Sep 2025 18:41:58 -0700
Patrick Mahan <mahan@mahan.org> wrote:

> This is on yocto-linux 4.19.87 on X86_64.  DPDK (stable) 18.11.11
> 
> I am working on a project where I am trying to make use of the 'virtio-user' 
> exception path and it (sort of) seems to be working.  The issue I am trying to 
> understand is the following:
> 
> Background -
> 
> Per the how-to documentation 
> (https://doc.dpdk.org/guides-25.07/howto/virtio_user_as_exception_path.html), I 
> have called rte_eal_init() passing in the PCI address of my physical interfaces. 
> I am also creating a couple of these "exception" interfaces with the following code -
> 
>      char portname[32], portargs[256];
>      char macaddr[6] = { 0x00, 0x0c, 0x1b, 0x29, 0x29, 0x71 };
>      uint16_t portid;
> 
>      /**
>       * Create the first 'iflan0'
>       */
>      snprintf(portargs, 256, 
> "path=/dev/vhost-net,queues=2,queue_size=1024,iface=iflan0,mac=%02x:%02x:%02x:%02x:%02x:%02x",
> macaddr[0], macaddr[1], macaddr[2], macaddr[3], macaddr[4], macaddr[5]);
> 
>      /**
>       * Call the hotplug layer
>       */
>      if (rte_eal_hotplug_add("vdev", "virt-user0", portargs) < 0) {
>          fprintf(stderr, "[DPDK::hotplug] failed to create iflan0 (virtio-user0)\n");
>          return -1;
>      }
> 
>      /**
>       * Create the second 'ifwan0'
>       */
>      macaddr[5]++;
>      snprintf(portargs, 256, 
> "path=/dev/vhost-net,queues=2,queue_size=1024,iface=ifwan0,mac=%02x:%02x:%02x:%02x:%02x:%02x",
> macaddr[0], macaddr[1], macaddr[2], macaddr[3], macaddr[4], macaddr[5]);
> 
>      /**
>       * Call the hotplug layer
>       */
>      if (rte_eal_hotplug_add("vdev", "virt-user1", portargs) < 0) {
>          fprintf(stderr, "[DPDK::hotplug] failed to create iflan0 (virtio-user0)\n");
>          return -1;
>      }
> 
>      ...
> 
>      RTE_ETH_FOREACH_DEV(portid) {
>          if (port_init(portid, bufPool, NRXQUEUES, NTXQUEUES) < 0) {
>              fprintf(stderr, "[DPDK::port_init] failed to init DPDK port %u",
>                      portid);
>              return -1;
>          }
>      }
> 
>      ...
> 
>      /* Retrieve the ifindex of iflan0 and ifwan0 */
>      unsigned int iflan0_index = if_nametoindex("iflan0");
>      unsigned int ifwan0_index = if_nametoindex("ifwan0");
> 
>      fprintf(stdout, "[DPDK] Exception interface 'iflan0': %u", iflan0_index);
>      fprintf(stdout, "[DPDK] Exception interface 'ifwan0': %u", ifwan0_index);
> 
>      ... /* onto the packet processing loops */
> 
> The interfaces are instantiated and those two logs messages are seen -
> 
> . . .
> [DPDK] Exception interface 'iflan0': 58
> [DPDK] Exception interface 'ifwan0': 59
> . . .
> 
> However, when I check the list of kernel interfaces using 'ip link show', I find:
> 
> pmahan-dpdk-v1: # ip link show
> . . .
> 66: iflan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT 
> group default qlen 1000
>      link/ether 00:0c:1b:29:29:71 brd ff:ff:ff:ff:ff:ff
> 67: ifwan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT 
> group default qlen 1000
>      link/ether 00:0c:1b:29:29:72 brd ff:ff:ff:ff:ff:ff
> 
> So, some point after I created and started these ports and obtain the original 
> ifindexes, they were deleted and re-created.  I have some clue from the IKE 
> kernel log messages:
> ...
> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 11[KNL] interface iflan0 activated
> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 09[KNL] interface ifwan0 activated
> ...
> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 09[KNL] interface iflan0 deactivated
> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 13[KNL] interface iflan0 deleted
> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 12[KNL] interface ifwan0 deactivated
> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 05[KNL] interface ifwan0 deleted
> 
> I never see another IKE log about those interfaces being activated again 
> (possibly because they show as DOWN).
> 
> Short of crawling through the guts of the virtio-user support in vdev, I thought 
> I would start asking questions here.
> 
> Also, is there any specific logging I can enabled to see if I can understand why 
> this is happening?
> 
> Thanks for any help,
> 
> Patrick
> 

It maybe that your systems networking code (udev) in user space is reacting
to the new interface.

Re: Use of the 'virtio-user' exception path interface

[Patrick Mahan]

On 9/28/25 9:27 AM, Stephen Hemminger wrote:
> On Sat, 27 Sep 2025 18:41:58 -0700
> Patrick Mahan <mahan@mahan.org> wrote:
> 
>> This is on yocto-linux 4.19.87 on X86_64.  DPDK (stable) 18.11.11
>>
>> I am working on a project where I am trying to make use of the 'virtio-user'
>> exception path and it (sort of) seems to be working.  The issue I am trying to
>> understand is the following:
>>
>> Background -
>>
>> Per the how-to documentation
>> (https://doc.dpdk.org/guides-25.07/howto/virtio_user_as_exception_path.html), I
>> have called rte_eal_init() passing in the PCI address of my physical interfaces.
>> I am also creating a couple of these "exception" interfaces with the following code -
>>
>>       char portname[32], portargs[256];
>>       char macaddr[6] = { 0x00, 0x0c, 0x1b, 0x29, 0x29, 0x71 };
>>       uint16_t portid;
>>
>>       /**
>>        * Create the first 'iflan0'
>>        */
>>       snprintf(portargs, 256,
>> "path=/dev/vhost-net,queues=2,queue_size=1024,iface=iflan0,mac=%02x:%02x:%02x:%02x:%02x:%02x",
>> macaddr[0], macaddr[1], macaddr[2], macaddr[3], macaddr[4], macaddr[5]);
>>
>>       /**
>>        * Call the hotplug layer
>>        */
>>       if (rte_eal_hotplug_add("vdev", "virt-user0", portargs) < 0) {
>>           fprintf(stderr, "[DPDK::hotplug] failed to create iflan0 (virtio-user0)\n");
>>           return -1;
>>       }
>>
>>       /**
>>        * Create the second 'ifwan0'
>>        */
>>       macaddr[5]++;
>>       snprintf(portargs, 256,
>> "path=/dev/vhost-net,queues=2,queue_size=1024,iface=ifwan0,mac=%02x:%02x:%02x:%02x:%02x:%02x",
>> macaddr[0], macaddr[1], macaddr[2], macaddr[3], macaddr[4], macaddr[5]);
>>
>>       /**
>>        * Call the hotplug layer
>>        */
>>       if (rte_eal_hotplug_add("vdev", "virt-user1", portargs) < 0) {
>>           fprintf(stderr, "[DPDK::hotplug] failed to create iflan0 (virtio-user0)\n");
>>           return -1;
>>       }
>>
>>       ...
>>
>>       RTE_ETH_FOREACH_DEV(portid) {
>>           if (port_init(portid, bufPool, NRXQUEUES, NTXQUEUES) < 0) {
>>               fprintf(stderr, "[DPDK::port_init] failed to init DPDK port %u",
>>                       portid);
>>               return -1;
>>           }
>>       }
>>
>>       ...
>>
>>       /* Retrieve the ifindex of iflan0 and ifwan0 */
>>       unsigned int iflan0_index = if_nametoindex("iflan0");
>>       unsigned int ifwan0_index = if_nametoindex("ifwan0");
>>
>>       fprintf(stdout, "[DPDK] Exception interface 'iflan0': %u", iflan0_index);
>>       fprintf(stdout, "[DPDK] Exception interface 'ifwan0': %u", ifwan0_index);
>>
>>       ... /* onto the packet processing loops */
>>
>> The interfaces are instantiated and those two logs messages are seen -
>>
>> . . .
>> [DPDK] Exception interface 'iflan0': 58
>> [DPDK] Exception interface 'ifwan0': 59
>> . . .
>>
>> However, when I check the list of kernel interfaces using 'ip link show', I find:
>>
>> pmahan-dpdk-v1: # ip link show
>> . . .
>> 66: iflan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
>> group default qlen 1000
>>       link/ether 00:0c:1b:29:29:71 brd ff:ff:ff:ff:ff:ff
>> 67: ifwan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT
>> group default qlen 1000
>>       link/ether 00:0c:1b:29:29:72 brd ff:ff:ff:ff:ff:ff
>>
>> So, some point after I created and started these ports and obtain the original
>> ifindexes, they were deleted and re-created.  I have some clue from the IKE
>> kernel log messages:
>> ...
>> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 11[KNL] interface iflan0 activated
>> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 09[KNL] interface ifwan0 activated
>> ...
>> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 09[KNL] interface iflan0 deactivated
>> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 13[KNL] interface iflan0 deleted
>> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 12[KNL] interface ifwan0 deactivated
>> Sep 27 23:30:22 2025 pmahan-dpdk-v1 IKE: 05[KNL] interface ifwan0 deleted
>>
>> I never see another IKE log about those interfaces being activated again
>> (possibly because they show as DOWN).
>>
>> Short of crawling through the guts of the virtio-user support in vdev, I thought
>> I would start asking questions here.
>>
>> Also, is there any specific logging I can enabled to see if I can understand why
>> this is happening?
>>
>> Thanks for any help,
>>
>> Patrick
>>
> 
> It maybe that your systems networking code (udev) in user space is reacting
> to the new interface.

Thanks for the suggestion, but as far as I can tell, our udev rules only exist 
for USB devices, there are not any for virtual interfaces.

Continuing to investigate...

Thanks,

Patrick