Re: ixgbe rxq interrupt not working

[Rajasekhar Pulluru <pullururajasekhar@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4354 bytes --]

Hi Stephen,

ixgbe interfaces works correctly when not used with dpdk.

# cat /proc/interrupts | grep enp2s0f1
109:        436          0          0          0  IR-PCI-MSI 1050624-edge
   enp2s0f1-TxRx-0
110:          0        167          0          0  IR-PCI-MSI 1050625-edge
   enp2s0f1-TxRx-1
111:          0          0        306          0  IR-PCI-MSI 1050626-edge
   enp2s0f1-TxRx-2
112:          0          0          0        173  IR-PCI-MSI 1050627-edge
   enp2s0f1-TxRx-3
113:          0          0          1          0  IR-PCI-MSI 1050628-edge
   enp2s0f1

dmesg doesn't seem to offer any clue.

# dmesg | grep ixgbe
[    7.680989] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[    7.680996] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[    8.318606] ixgbe 0000:02:00.0: Multiqueue Enabled: Rx Queue count = 4,
Tx Queue count = 4 XDP Queue count = 0
[    8.477642] ixgbe 0000:02:00.0: MAC: 6, PHY: 27, PBA No: 000700-000
[    8.477654] ixgbe 0000:02:00.0: 00:e0:ed:db:33:69
[    8.539077] ixgbe 0000:02:00.0: Intel(R) 10 Gigabit Network Connection
[    8.888011] ixgbe 0000:02:00.1: Multiqueue Enabled: Rx Queue count = 4,
Tx Queue count = 4 XDP Queue count = 0
[    9.019755] ixgbe 0000:02:00.1: MAC: 6, PHY: 27, PBA No: 000700-000
[    9.019760] ixgbe 0000:02:00.1: 00:e0:ed:db:33:68
[    9.063320] ixgbe 0000:02:00.1: Intel(R) 10 Gigabit Network Connection

Thanks & Regards,
Rajasekhar

On Wed, Mar 8, 2023 at 11:14 PM Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Wed, 8 Mar 2023 22:54:12 +0530
> Rajasekhar Pulluru <pullururajasekhar@gmail.com> wrote:
>
> > No Honnappa.
> >
> > Thanks & Regards,
> > Rajasekhar
> >
> > On Wed, Mar 8, 2023 at 5:49 AM Honnappa Nagarahalli <
> > Honnappa.Nagarahalli@arm.com> wrote:
> >
> > >
> > >
> > > From: Rajasekhar Pulluru <pullururajasekhar@gmail.com>
> > > Sent: Tuesday, March 7, 2023 12:52 PM
> > > To: dev@dpdk.org
> > > Subject: ixgbe rxq interrupt not working
> > >
> > > Hi Team,
> > >
> > > Bringing-up dpdk-22.07 on an intel machine with 8 ports, 4 of them
> driven
> > > by igb and the rest of the 4 ports driven by ixgbe.
> > > [Honnappa]  Do you have packets crossing between the 2 drivers?
> > >
> > >
> > > I am following the below sequence to initialize these ports:
> > >
> > > dev_conf.intr_conf.lsc = 1; //Enable link state change interrupt
> > > dev_conf.intr_conf.rxq = 1; //Enable RX Queue Interrupt
> > > dev_conf.rxmode.mq_mode = RTE_ETH_MQ_RX_NONE;
> > > dev_conf.rxmode.offloads = 0;
> > > dev_conf.txmode.mq_mode = RTE_ETH_MQ_TX_NONE;
> > > dev_conf.txmode.offloads = 0;
> > >
> > > rte_eth_dev_configure
> > > rte_eth_rx_queue_setup
> > > rte_eth_tx_queue_setup
> > > rte_eth_dev_start
> > > data = port_id << CHAR_BIT | queue_id;
> > > rte_eth_dev_rx_intr_ctl_q(port_id, queue_id, RTE_EPOLL_PER_THREAD,
> > > RTE_INTR_EVENT_ADD, (void *)((uintptr_t)data));
> > > rte_eth_dev_rx_intr_enable(port_id, queue_id);
> > >
> > > And then main loop repeats the below:
> > >
> > > rte_epoll_wait(RTE_EPOLL_PER_THREAD, event, 1, timeout /* 200micro-sec
> > > */); /* ignore return value */
> > > rte_eth_dev_rx_intr_disable(port_id, queue_id);
> > > rte_eth_rx_burst(port_id, queue_id, pkts, num_pkts);
> > > rte_eth_dev_rx_intr_enable(port_id, queue_id);
> > >
> > > The code is same for all the ports, igb ports are able to come-up and
> rx
> > > packets, where-as the ixgbe ports are not able to rx packets at all.
> > > cat /proc/interrupts dumps vfio-msix counters for ixgbe as 0, where-as
> > > it's non-zero for igb.
> > > If I don't use/enable rxq interrupt for ixgbe (and remove epoll wait,
> > > interrupt enable/disable from while loop) and simply poll for
> > > rte_eth_rx_burst in a loop, ixgbe ports are able to rx packets.
> > >
> > > What could be wrong here? Appreciate any help.
> > >
> > > I would also like to know if there's an asynchronous rxq interrupt
> > > notification to the application instead of rte_epoll_wait (and sleep).
> > >
> > > Thanks & Regards,
> > > Rajasekhar
> > >
>
> Does the device work as expected when not used with DPDK?
> I.e does the kernel driver handle it correctly.
>
>
> Also check the kernel dmesg log, for any relevant info.
> There maybe VFIO or other overlap involved.
>

[-- Attachment #2: Type: text/html, Size: 5743 bytes --]