From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 26B1EA04A2 for ; Sun, 16 Jan 2022 22:27:25 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B5AFE41143; Sun, 16 Jan 2022 22:27:24 +0100 (CET) Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by mails.dpdk.org (Postfix) with ESMTP id DE02140683 for ; Sun, 16 Jan 2022 22:27:22 +0100 (CET) Received: by mail-wm1-f46.google.com with SMTP id p18so16501582wmg.4 for ; Sun, 16 Jan 2022 13:27:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=xrfWjY7K9zOA+3edNB3CciDd7YZMYGY8E1F9jlANDXI=; b=MghUPRl1wgdJNGCK02A9/rXQOSJDg129SquVesrkpK+/kFkDADUaQfI4XMpf/MH3J4 UYgLKtt1zPLd0gqmDGQUgWTF5d76y+9cVMANhqhZxCq/9MASjfEKGe3mKBCiiymOIxeQ j2g+oxe7j/MokT3T8n51urwcbi0TOcCFOcl/yKIoZIPW4RwfQkOZ5LywcB0VePFksHW4 UkhxSZntY0e6KNcxMcQxbdIM27QYRUy2kIylY01q3X8ZC1mKYE2q4k/ElcsQMwKXPsa+ 4GfX1A3qaFLB8+NWpPZFic10jSi94rlga9iFqS2hD8Uai33uFKCEPt0+7RHzXMuJJzNV Vquw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=xrfWjY7K9zOA+3edNB3CciDd7YZMYGY8E1F9jlANDXI=; b=VQmKfJLlJDx4pCYFYy1Pp1AFKDX+ubIDJra/sTSMxRpBeRZVeXbKXzaRiWR6Y/KM0j fh3eL6ATSsc62xO0fOCzIinDF2rFWshtVS2Q8L4nTADtBYj54qi0zZGNUGEMxTmiLP80 OX9o3D9DnoCEd1OTWO42UxstrXrtm2Dcw4FIW2fIyLwIp0Sfp4e+/ik8hjPkzeWZm7m9 roY1aILpQ9cJUJjolmBifboGR3gUzul+u3W9Fe9hIswEQXPb4HxSlsus7ZCo5Y57kD+U wNWiLMYyBGvJ11RMo1ec0fRxnJzGDen7jiiLizXDerOpGc2f1siu8/435D3Lw8/A7Vrm hjew== X-Gm-Message-State: AOAM532xbUFYiUP2TbsVz8zyfsBi8B+XLGyhjt2jEdtYTIcGisWw7ULm cEuy1YdmqP8NYPW7WqcqV2iGsgwP6+smW5gLGDnWSyJJ15s= X-Google-Smtp-Source: ABdhPJxDSDlgwXU5Yt/2DZleH/7qSk7QX9bzirrJNK68QVn/hhv6/Fp7QszlX981BFuNJmh0ssxB8YP42tFhMsLG13Q= X-Received: by 2002:a05:600c:6013:: with SMTP id az19mr11315943wmb.53.1642368442184; Sun, 16 Jan 2022 13:27:22 -0800 (PST) MIME-Version: 1.0 From: fwefew 4t4tg <7532yahoo@gmail.com> Date: Sun, 16 Jan 2022 16:27:11 -0500 Message-ID: Subject: Question: DPDK thread model To: users@dpdk.org Content-Type: multipart/alternative; boundary="00000000000096374505d5b9b3d4" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --00000000000096374505d5b9b3d4 Content-Type: text/plain; charset="UTF-8" I completed code to initialize an AWS ENA adapter with RX, TX queues. With this work in hand, DPDK creates one thread pinned to the right core as per the --lcores argument. So far so good. The DPDK documentation and example code is fairly clear here. What's not as clear is how RX packets are handled. As far as I can tell the canonical way to deal with RX packets is running 'rte_eth_add_rx_callback' for each RXQ. This allows one to process each received packet (for a given RXQ) via a provided callback in the same lcore/hardware-thread that DPDK created for me. As such, there is no need to create additional threads. Correct? Furthermore, I hope the mbufs the callback gets somehow correspond to mbufs associated with the RX descriptors provided to the RXQs so there's no need for copying packets after the NIC receives them before the callback acts on it. As far as I can this hope is ill-founded.. A lot of DPDK code I've seen allocates more mbufs per RXQ than the number of RX descriptors. To me this seems to imply DPDK's RXQ threads put copies of the received-off-the-wire-packets into a copy for delivery to app code. TX is less clear to me. For TX there seems to be no way to transmit packets (burst or otherwise) without creating another thread --- that is, another thread beyond what DPDK makes for me. This other thread must at the appropriate time prepare mbufs and call 'rte_eth_tx_burst' on the correct TXQ. DPDK seems to want to keep its thread for it's own work. Yes, DPDK provides 'rte_eth_add_tx_callback' but that only works after the mbufs have been created and told to transmit, which is after the fact of creation. Putting this together, DPDK requires me to create new threads unlike RX. Correct? While creating additional threads for TX is not the end of the world, I do not want the DPDK TX thread to copy mbufs; I want zero-copy. Here, then, I gather DPDK's TXQ thread takes the mbufs the helper TX thread provides in the 'rte_eth_tx_burst' call and provides them to the TXQS descriptors so they go out on the wire without copying. Is this correct? Now, it's worth pointing out here that 'rte_eth_tx_queue_setup' unlike the RX equivalent does not accept a mempool. So in addition to the above points, those additional TX helper threads (those which call rte_eth_tx_burst) will need to arrange for its own mempool. That's not hard to do, but I just want confirmation. Thanks. --00000000000096374505d5b9b3d4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I completed code to initialize an AWS ENA adapter with RX,= TX queues. With this work in hand, DPDK creates one thread pinned to the r= ight core as per the --lcores argument. So far so good.
The DPDK docume= ntation and example code is fairly=C2=A0clear here.
=C2=A0
What's not as clear is how RX packets are handled. As far as I can tel= l the canonical way to deal with RX packets is running 'rte_eth_add_rx_= callback' for each RXQ. This allows one to process each received packet= (for a given RXQ) via a provided callback in the same lcore/hardware-threa= d that DPDK created for me. As such, there is no need to create additional = threads. Correct?=C2=A0

Furthermore, I hope the mb= ufs the callback gets somehow correspond=C2=A0to mbufs associated with the = RX descriptors provided to the RXQs so there's no need for copying pack= ets after the NIC receives them before the callback acts on it. As far as I= can this=C2=A0hope is ill-founded.. A lot of DPDK code I've seen alloc= ates more mbufs per RXQ than the number of RX descriptors. To me this seems= to imply DPDK's RXQ threads put copies of the received-off-the-wire-pa= ckets into a copy for delivery to app code.=C2=A0

= TX is less clear=C2=A0to me.

For TX there seems to be no = way to transmit packets (burst or otherwise) without creating another threa= d --- that is, another thread beyond what DPDK makes for me. This other thr= ead must at the appropriate time prepare=C2=A0mbufs and call 'rte_eth_t= x_burst' on the correct TXQ. DPDK seems to want to keep its thread for = it's own work. Yes, DPDK provides 'rte_eth_add_tx_callback' but= that only works after the mbufs have been created and told to transmit, wh= ich is after the fact of creation. Putting this together, DPDK requires me = to create new threads unlike RX. Correct?

While cr= eating additional threads for TX is not the end of the world, I do not want= the DPDK TX thread to copy mbufs; I want zero-copy. Here, then, I gather D= PDK's TXQ thread takes the mbufs the helper TX thread provides in the &= #39;rte_eth_tx_burst' call and provides them to the TXQS descriptors so= they go out on the wire without copying. Is this correct?=C2=A0
=
Now, it's worth pointing out here that 'rte_eth_tx_q= ueue_setup' unlike=C2=A0the RX equivalent does not accept a mempool.=C2= =A0 So in addition to the above points, those additional TX helper threads = (those which call rte_eth_tx_burst) will need to arrange for its own mempoo= l. That's not hard to do, but I just want confirmation.

<= /div>
Thanks.
--00000000000096374505d5b9b3d4--