From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B75542BF0 for ; Mon, 5 Jun 2023 17:02:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B63AB42C4D; Mon, 5 Jun 2023 17:02:16 +0200 (CEST) Received: from mail-yb1-f177.google.com (mail-yb1-f177.google.com [209.85.219.177]) by mails.dpdk.org (Postfix) with ESMTP id C9E27406B8 for ; Fri, 2 Jun 2023 12:14:11 +0200 (CEST) Received: by mail-yb1-f177.google.com with SMTP id 3f1490d57ef6-bad041bf313so1951644276.0 for ; Fri, 02 Jun 2023 03:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685700851; x=1688292851; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=2A+1oTjfwiy52Vxm1G9MjMj2rAm/1clKkBOlY6NJJUA=; b=ZTfQx6lnK57jpVqFoDMSvYYF9Dr2blJvNYL3FMYpMx8tFI4dR4vEN7dfth7Sk4RcQb 4qmy+/yl5ZFu5wQUsXOYeYrRDQNxHhPfoBZ1EliK5fvc99IMCHb/u4ebaUe6N78xFtfK G6Kx6dSkPtqt6YvxyuT4dwMzSNeX5Ng+eihHRuzevCFPrQDm1vIq1ovnWOLJ9VzfTluH +/7M0h/w+VqeoUOxpEKC6r1eF/b9BpQ8lDuBA5F/gYSz2C4Hom92TtN2t3qndbWmsPqg R9/iTTaK5tKok64bb151xsJb/v9XUqAQcdVpjyfFjlxwIbMlEkdQKW8b3kScPysQg9+d 0HrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685700851; x=1688292851; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2A+1oTjfwiy52Vxm1G9MjMj2rAm/1clKkBOlY6NJJUA=; b=ezIX1CBLXqTklNwgY+wcV7Ar3xs9n6UrVPUCCIXNirTrGiECIPPlkOqTO3JR2rLbmN vM8OubTB5C5wwutyjeC/s+MKGxiyEZlC4t+2I5FDecXO0T2YKThpLJ5iCu01qbC7IngY NQAsVDl308VzSxChzvEbF2SlWL05dEeVRc19M3xStaN7VFwdHfqGnTzsZ256xxSKAjal ZIfEGFq7Vxk30HFV8VNlNsYOPP8onHwsKawLq+/cOx7Vt4H0lxEhzbvADBDDYXOBI4Vb HxgAYNZOTu/kLHJ0eKUp6Q/kHvPuaXByAyjbihLIeG3Xv1uBZQYKpvLmujq0c+R/fwFc XQzQ== X-Gm-Message-State: AC+VfDxGg2kKRG/PDA/CIVgfOOkYI8nNY02f9obkWPngbx3LXX/9Tlo0 50bW67pzFVL+jeI8FvNAbpGKBTNxZb5Xmvj2RN0orcaI53c= X-Google-Smtp-Source: ACHHUZ7mlDcc/L6p1lyb1L4jC7L7F910yKD6SHX02D30+I6UhHznJV7YkwuPr8wV6sRMRY1RfokcvFrFS6hi7BUL89Y= X-Received: by 2002:a25:7453:0:b0:bab:fccc:2c0d with SMTP id p80-20020a257453000000b00babfccc2c0dmr2613316ybc.6.1685700850961; Fri, 02 Jun 2023 03:14:10 -0700 (PDT) MIME-Version: 1.0 References: <20230530092429.00e68897@hermes.local> In-Reply-To: <20230530092429.00e68897@hermes.local> From: Fengkai Sun Date: Fri, 2 Jun 2023 18:14:00 +0800 Message-ID: Subject: Re: Scheduling of multiple RX/TX queues on a single port To: Stephen Hemminger Cc: users@dpdk.org Content-Type: multipart/alternative; boundary="0000000000006b52cd05fd22d0f7" X-Mailman-Approved-At: Mon, 05 Jun 2023 17:02:13 +0200 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --0000000000006b52cd05fd22d0f7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Stephen, Thank you very much for your reply! After using 'transmit scheduling' as the keyword, much more useful results now show up. For others who are also interested in how queues are scheduled, I would recommend reading this paper (it's open access): https://www.usenix.org/system/files/nsdi19-stephens.pdf -- Thanks, Fengkai On Wed, May 31, 2023 at 12:24=E2=80=AFAM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Mon, 29 May 2023 23:02:45 +0800 > Fengkai Sun wrote: > > > Hi list, > > > > I'm curious how DPDK programs the NIC to receive/transmit packets when > > there are multiple queues on a single port. > > > > As for RX, the answer might be clear. > > The NIC can only receive a packet once at a time, since the cable only > > outputs one signal (0 or 1) at a time (correct me if I'm wrong). > > Therefore the NIC can receive a packet, check it's information, and > finally > > put in into the right queue via some policies, e.g. RSS, all > sequentially. > > > > However, it confuses me when it comes to TX. > > As there are multiple TX queues on the same port, the NIC must decide > which > > queue to get packets from when it's idle. > > This is where scheduling lies. How does the NIC select the queue? > > Round-Robin? Does it have to enforce fairness among the queues? > > > > I'm wondering where I can find some documentation on this issue. Thank > you! > > Transmit scheduling is up to the hardware (not DPDK). > Generally I assume it is round-robin > but there maybe cases like priority queues (like DCB) or large packets > with segment offload. > --0000000000006b52cd05fd22d0f7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Stephen,

Thank you very much for you= r reply!

After using 'transmit scheduling'= as the keyword, much more useful results now show up.

=
For others who are also interested in how queues are scheduled, I woul= d recommend reading this paper (it's open access):=C2=A0https://www.usenix.org= /system/files/nsdi19-stephens.pdf


--
Thanks,
Fengkai

On Wed, May 31, 2023 at 12:2= 4=E2=80=AFAM Stephen Hemminger <stephen@networkplumber.org> wrote:
On Mon, 29 May 2023 23:02:45 +0800
Fengkai Sun <q= cloud1014@gmail.com> wrote:

> Hi list,
>
> I'm curious how DPDK programs the NIC to receive/transmit packets = when
> there are multiple queues on a single port.
>
> As for RX, the answer might be clear.
> The NIC can only receive a packet once at a time, since the cable only=
> outputs one signal (0 or 1) at a time (correct me if I'm wrong). > Therefore the NIC can receive a packet, check it's information, an= d finally
> put in into the right queue via some policies, e.g. RSS, all sequentia= lly.
>
> However, it confuses me when it comes to TX.
> As there are multiple TX queues on the same port, the NIC must decide = which
> queue to get packets from when it's idle.
> This is where scheduling lies. How does the NIC select the queue?
> Round-Robin? Does it have to enforce fairness among the queues?
>
> I'm wondering where I can find some documentation on this issue. T= hank you!

Transmit scheduling is up to the hardware (not DPDK).
Generally I assume it is round-robin
but there maybe cases like priority queues (like DCB) or large packets
with segment offload.
--0000000000006b52cd05fd22d0f7--