From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4E337A0C57 for ; Mon, 1 Nov 2021 09:56:41 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 44B3F40E28; Mon, 1 Nov 2021 09:56:41 +0100 (CET) Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by mails.dpdk.org (Postfix) with ESMTP id 4581440DF6 for ; Mon, 1 Nov 2021 09:56:40 +0100 (CET) Received: by mail-pj1-f51.google.com with SMTP id nh10-20020a17090b364a00b001a69adad5ebso717109pjb.2 for ; Mon, 01 Nov 2021 01:56:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chillysky-com.20210112.gappssmtp.com; s=20210112; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IEl0f8A1VkyxL76NX0/O+DaVlqJrRjVrnIWLYgdseOY=; b=Wfu1/DHfIRWuPwY0qkLy6FLCw+eQDMolVmfMI0tQWmwOmOkrClXNiZ1Ij3WpQl07K/ HWBQ5gphMe+FlQv4d3jh8hGqCDSH+r/2+Ookp72KOE8Jb1fnpvgCWhTQ0LwcuhM8DRSP zCTxa0ecp33+/sJmoBMvTGqPG0rWjZp/NtM2hb0sHKhJDJPmbWMBFA2OXy1wryKOQZ9k 9KSrgWni3+S7ZXyJsDur6pEFNaQK62E4CgiE+/ps+CtKZKhQy6oIdILShsOibXaHw8UG M3PlwIdzOt9DxNFkj/Uy1psovuOy7U5jDPZavpgNTrXJCQ/L/QYkxE64DUxKbzgdfLWf mZCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IEl0f8A1VkyxL76NX0/O+DaVlqJrRjVrnIWLYgdseOY=; b=xZF6YTC0ww49bDHR6TTgIYILZHJxwqUYFuJuFoHEkg+eqGSkTzNOYAJfctj76n12C6 7wutjMBnN/EsBqePudpgGFQ5xC2/bmkkxyTbaphpc+8GGTPKzCGApEnZM5hje6aLmjiF h7XRp4WtRrWHUCGOV+HTdXjvcUrSdtyLYnsMhd/eyZu4pdrbmwsmJufoUqal9jLcBiUE Og+9IxuukRsYfzM+E5iuGNdSgpZFWMRWpNy5w282WAZ40Spy/ZCDk4aEgZ5wWVdR0yZ/ KupSbARFmmWU1ONOP9V0q27oahCLA3nIh/LpUfSjBe+0p5UKDQe/nUhzKnlzWFNrvv2G GDug== X-Gm-Message-State: AOAM533QNdtf9tohyR639A2lWTATYGqoZI6PHqOGg1J2zRFpj+p6w+0G YJ/jExjT5/+wePEOFvMENWWj8g== X-Google-Smtp-Source: ABdhPJxhDrr6e4VR7deE4jfVkEZ+06e2BzIxC8uTTuUtw2qN0stsvclcbUoXei1p8mUx/A1Xlw1IbQ== X-Received: by 2002:a17:90a:b786:: with SMTP id m6mr36777130pjr.30.1635756999250; Mon, 01 Nov 2021 01:56:39 -0700 (PDT) Received: from smtpclient.apple (DN160vrd000d6kb000000000009c.stanford.edu. [2607:f6d0:d:3516::12c]) by smtp.gmail.com with ESMTPSA id e15sm14157969pfc.134.2021.11.01.01.56.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Nov 2021 01:56:38 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: Enable RX interrupts From: Jack Humphries In-Reply-To: <20211101104648.3771e1c4@sovereign> Date: Mon, 1 Nov 2021 01:56:37 -0700 Cc: users@dpdk.org, Haiyue Wang Content-Transfer-Encoding: quoted-printable Message-Id: <8B6D83B4-DF17-4053-88D1-404CBCBB584F@chillysky.com> References: <73CAB2FD-77CD-4C84-BE8B-F100A1CA55F3@chillysky.com> <20211101104648.3771e1c4@sovereign> To: Dmitry Kozlyuk X-Mailer: Apple Mail (2.3693.20.0.1.32) X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi Dmitry, Thanks for the quick response! Yes, I was calling = rte_eth_dev_rx_intr_enable =E2=80=94 I just tried re-arming interrupts = before each call to epoll and my initial experiment shows that fixes the = issue. You are right that l3fwd-power does the re-arming as well. If possible, = it would be helpful to have a comment in that example code about the = re-arming since the impression I got from quickly looking at the = l3fwd-power code is that interrupts are turned off because the app wants = to poll for a threshold of time before =E2=80=9Cgiving up=E2=80=9D and = turning interrupts back on again. I=E2=80=99m happy to open a pull = request to add this comment, too, if you prefer. Thanks again for your help! Jack > On Nov 1, 2021, at 12:46 AM, Dmitry Kozlyuk = wrote: >=20 > 2021-10-31 22:50 (UTC-0700), Jack Humphries: >> Hi folks, >>=20 >> Hope all are well. I=E2=80=99m trying to enable interrupts in DPDK so = that my network receive thread can sleep on an epoll until packets = arrive. I am using the ixgbe kernel driver and igb_uio userspace driver = with an Intel 82599ES 10Gbps NIC. >>=20 >> I'm doing roughly the following to enable the interrupts, but the = epoll never indicates that packets have arrived. The thread only handles = packets when the epoll times out. I don't even see interrupts arrive = from the device when monitoring /proc/interrupts. >>=20 >> port_conf.intr_conf.rxq =3D 1; >>=20 >> ... >>=20 >> CHECK_EQ(rte_eth_dev_rx_intr_ctl_q(kPort, kQueue, = RTE_EPOLL_PER_THREAD, >> RTE_INTR_EVENT_ADD, nullptr), >> 0);=20 >> CHECK_EQ(rte_eth_dev_rx_intr_enable(kPort, kQueue), 0); >>=20 >> ... >>=20 >> rte_epoll_event event; >> while (true) { >> int n =3D rte_epoll_wait(RTE_EPOLL_PER_THREAD, &event, = /*maxevents=3D*/1, >> /*timeout=3D*/1000); >> if (n =3D=3D 0) { >> // Timeout expired. >> } else { >> // Received RX interrupt. >> } >> } >>=20 >> Given that I don't see anything coming through in /proc/interrupts, I = am going to start digging through the ixgbe driver. However, I wanted to = ask here first to see if my setup is missing anything obvious. I based = it closely on the l3fwd-power example, though I haven't been able to get = that example up and running yet since my NIC only has one port plugged = in right now and the example requires two (COVID building restrictions). >>=20 >> Thanks, >> Jack Humphries >=20 > Hi Jack, >=20 > Have you called rte_eth_dev_rx_intr_enable(port_id, queue_id)? >=20 > Not sure about ixgbe, but vmxnet3 also requires rearming interrupts = with it > on each iteration before rte_epoll_wait(). > This is what l3fwd-power does by the way (see turn_on_off_intr() = function). >=20 > + ixgbe maintainer just in case.