From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0328741E33; Wed, 8 Mar 2023 18:24:27 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E538C410F9; Wed, 8 Mar 2023 18:24:26 +0100 (CET) Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by mails.dpdk.org (Postfix) with ESMTP id C9779410EE for ; Wed, 8 Mar 2023 18:24:25 +0100 (CET) Received: by mail-pl1-f180.google.com with SMTP id ky4so18358881plb.3 for ; Wed, 08 Mar 2023 09:24:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678296265; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=dFpWEVX8zwx5Fu+HVY5prLbPtJmAKpSZQyLmgzzmiBE=; b=WWUjyLl/fGsGJR5722oZlOXt6H+HyYv+ch9YCowx3LZiWCjNAEwsy80+8tMLH0Dzlk SfkEVYC0P6KFhbuSxoEJc05765Ha3JZSkYWIK+raHUojCB4xx7QZVnrnwW/bdMJsTQOb n2hY3+r9kKFLMTHLnk9KUL2TppDafu0SIXFY8WXcymJHc1r9nzJynhh5LWjlDZ9+uFTu Uh38abmaDjbanbWQy//EwKd4fFa/dQLNc4PZdKMVi4DU0bOFR7QNv82eZfIh54bNiP9Q qd7QCaDPJO1lXjOyN0VVKhkM0nPO7qOf8jecLxiaiVWbYANlECMLfH41WCQpGqbRz/nS 0t+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678296265; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dFpWEVX8zwx5Fu+HVY5prLbPtJmAKpSZQyLmgzzmiBE=; b=wSQJMRzBEYDX1P5sq3eD8JUFL7rWHcdlPsZQD0fjgdRu10dCQQ8DRiCXTcUpi/nUJs 9TZ6q0R+I2vMHlQwhRtMZgD3vADkKv29yB7Pkkv8QMZicavslQ/H/IOAFZHs0at25HMY zLkiQt14pzCu2snUTPi2+0c3PbXFc0xQ66JoWmI+0gAkiCZ/967RBpPIYVoT3xfCFv13 Fiu1NWZxJ3P3wdv8kx/JtLnvT6vWq0i9cZf5y/Eg91keVoJRwRhvZFliOSqexMwmGh9L oVBlhRHlj+oMZ8j98Rw+NnaUpH77rse4YQi/ZUIoEdhtGTE6Pvn/e4MLb1fx7sD0gOgx K5JA== X-Gm-Message-State: AO0yUKVUoFS2ntdmVoNn/EI0Kfvxe91DtpBldvTroyWFVWqG4yIizbLh hhp/Wzna6Im55zJPkgs7afQcORb+456UFpV7tRb0hSWT X-Google-Smtp-Source: AK7set94KTB1LhEbyqz4eJ/83c60DmIOPKEIGDQpuIji7ja7vOxJRIinJqUS7CR8J/Dyv/uup7cWpvBMYRwLNObMaTM= X-Received: by 2002:a17:902:784a:b0:199:482e:6f93 with SMTP id e10-20020a170902784a00b00199482e6f93mr9744540pln.2.1678296264929; Wed, 08 Mar 2023 09:24:24 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Rajasekhar Pulluru Date: Wed, 8 Mar 2023 22:54:12 +0530 Message-ID: Subject: Re: ixgbe rxq interrupt not working To: Honnappa Nagarahalli Cc: "dev@dpdk.org" , nd Content-Type: multipart/alternative; boundary="000000000000b2f75f05f666ccbf" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --000000000000b2f75f05f666ccbf Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable No Honnappa. Thanks & Regards, Rajasekhar On Wed, Mar 8, 2023 at 5:49=E2=80=AFAM Honnappa Nagarahalli < Honnappa.Nagarahalli@arm.com> wrote: > > > From: Rajasekhar Pulluru > Sent: Tuesday, March 7, 2023 12:52 PM > To: dev@dpdk.org > Subject: ixgbe rxq interrupt not working > > Hi Team, > > Bringing-up dpdk-22.07 on an intel machine with 8 ports, 4 of them driven > by igb and the rest of the 4 ports driven by ixgbe. > [Honnappa] Do you have packets crossing between the 2 drivers? > > > I am following the below sequence to initialize these ports: > > dev_conf.intr_conf.lsc =3D 1; //Enable link state change interrupt > dev_conf.intr_conf.rxq =3D 1; //Enable RX Queue Interrupt > dev_conf.rxmode.mq_mode =3D RTE_ETH_MQ_RX_NONE; > dev_conf.rxmode.offloads =3D 0; > dev_conf.txmode.mq_mode =3D RTE_ETH_MQ_TX_NONE; > dev_conf.txmode.offloads =3D 0; > > rte_eth_dev_configure > rte_eth_rx_queue_setup > rte_eth_tx_queue_setup > rte_eth_dev_start > data =3D port_id << CHAR_BIT | queue_id; > rte_eth_dev_rx_intr_ctl_q(port_id, queue_id, RTE_EPOLL_PER_THREAD, > RTE_INTR_EVENT_ADD, (void *)((uintptr_t)data)); > rte_eth_dev_rx_intr_enable(port_id, queue_id); > > And then main loop repeats the below: > > rte_epoll_wait(RTE_EPOLL_PER_THREAD, event, 1, timeout /* 200micro-sec > */); /* ignore return value */ > rte_eth_dev_rx_intr_disable(port_id, queue_id); > rte_eth_rx_burst(port_id, queue_id, pkts, num_pkts); > rte_eth_dev_rx_intr_enable(port_id, queue_id); > > The code is same for all the ports, igb ports are able to come-up and rx > packets, where-as the ixgbe ports are not able to rx packets at all. > cat /proc/interrupts dumps vfio-msix counters for ixgbe as 0, where-as > it's non-zero for igb. > If I don't use/enable rxq interrupt for ixgbe (and remove epoll wait, > interrupt enable/disable from while loop) and simply poll for > rte_eth_rx_burst in a loop, ixgbe ports are able to rx packets. > > What could be wrong here? Appreciate any help. > > I would also like to know if there's an asynchronous rxq interrupt > notification to the application instead of rte_epoll_wait (and sleep). > > Thanks & Regards, > Rajasekhar > --000000000000b2f75f05f666ccbf Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
No Honnappa.=C2=A0

Thanks & Regards= ,
Rajasekhar

On Wed, Mar 8, 2023 at 5:49=E2=80=AFAM Honnap= pa Nagarahalli <Honnappa= .Nagarahalli@arm.com> wrote:


From: Rajasekhar Pulluru <pullururajasekhar@gmail.com>
Sent: Tuesday, March 7, 2023 12:52 PM
To: dev@dpdk.org
Subject: ixgbe rxq interrupt not working

Hi Team,

Bringing-up dpdk-22.07 on an intel machine with 8 ports, 4 of them driven b= y igb and the rest of the 4 ports driven by ixgbe.
[Honnappa]=C2=A0 Do you have packets crossing between the 2 drivers?


I am following the below sequence to=C2=A0initialize these ports:

dev_conf.intr_conf.lsc =3D 1; //Enable link state change interrupt
dev_conf.intr_conf.rxq =3D 1; //Enable RX Queue Interrupt
dev_conf.rxmode.mq_mode =3D RTE_ETH_MQ_RX_NONE;
dev_conf.rxmode.offloads =3D 0;
dev_conf.txmode.mq_mode =3D RTE_ETH_MQ_TX_NONE;
dev_conf.txmode.offloads =3D 0;

rte_eth_dev_configure
rte_eth_rx_queue_setup
rte_eth_tx_queue_setup
rte_eth_dev_start
data =3D port_id << CHAR_BIT | queue_id;
rte_eth_dev_rx_intr_ctl_q(port_id, queue_id, RTE_EPOLL_PER_THREAD, RTE_INTR= _EVENT_ADD, (void *)((uintptr_t)data));
rte_eth_dev_rx_intr_enable(port_id, queue_id);

And then main loop repeats the below:

rte_epoll_wait(RTE_EPOLL_PER_THREAD, event, 1, timeout /* 200micro-sec */);= /* ignore return value */
rte_eth_dev_rx_intr_disable(port_id, queue_id);
rte_eth_rx_burst(port_id, queue_id, pkts, num_pkts);
rte_eth_dev_rx_intr_enable(port_id, queue_id);

The code is same for all the ports, igb ports are able to come-up and rx pa= ckets, where-as the ixgbe ports are not able to rx packets at all.
cat /proc/interrupts dumps vfio-msix counters for ixgbe as 0, where-as it&#= 39;s non-zero for igb.
If I don't use/enable rxq interrupt for ixgbe (and remove epoll wait, i= nterrupt enable/disable from while loop) and simply poll for rte_eth_rx_bur= st in a loop, ixgbe ports are able to rx packets.

What could be wrong here? Appreciate any help.

I would also like to know if there's an asynchronous rxq interrupt noti= fication to the application instead of rte_epoll_wait (and sleep).

Thanks & Regards,
Rajasekhar
--000000000000b2f75f05f666ccbf--