From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C384445C75; Tue, 5 Nov 2024 19:58:44 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 603934027E; Tue, 5 Nov 2024 19:58:44 +0100 (CET) Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by mails.dpdk.org (Postfix) with ESMTP id BA19340156 for ; Tue, 5 Nov 2024 19:58:42 +0100 (CET) Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-720d5ada03cso4693731b3a.1 for ; Tue, 05 Nov 2024 10:58:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1730833121; x=1731437921; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=R12HvfkF5VS3ntDiEJFaUejGvpzc9aUg0VS9pBR/WSQ=; b=aEj5puo4IGCv9sNv4o83J8zqJid5vNPbDY0GcOJtBfO8WITSu4Amz8j+nseYp0d70a cW+WgvuZdqoiEVonTxnfIWljT2nttPNLb0M6UgXH1YoisPAJKiv71qMMVBhZ6CZs/tKw Q89tkIPzB+C4vPUaKUlgK3PB1kEZ9+Tc9JB6eWH/rSHXnCk0luxyFkg2kQpCYyEifBCN ZksMAn/GrAADH5bn4htAMNFylA/mTeZF5LVV7b40O+1FvpD5xinmYthR8LQRIWFFkL4V Ia+K0PnNyOXvUiSLpiw5LkhmwQ2ydcuzIwKTffnnJx9t1+jhgnKZoBK74157OrKYvPyz VWBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730833121; x=1731437921; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=R12HvfkF5VS3ntDiEJFaUejGvpzc9aUg0VS9pBR/WSQ=; b=CtJdbfNF/TFf1MwXEMT1q2TWRg2xQG8C/XsJ17cENtX++urwqezIPBCa3R93OxcAMf vpwiuppZuc49KGJwXR7mZ2VfVwJcsthpW+f8BEBRpQbSr+KCI0ZoCnNSZzfRaakRyajq Y/1lhotmq71dlcGpVP2IMvr87a3NGmk+NoVzvhNy1gn7mzqnhTiA33zbeR8lxZqOYOQG pefXTXzTDJLPvYdVO1EQyk5QXsyImtXzVvCGh4G5oc7f8sFJTtJYSoEqjmshGPxAZkfI Tfy+FwJCq5Heqe5cinYq9tSrxl6MMOTkLNZZXKY56LeoMVmqVinytZevqOTpdTYIHHcY ttEA== X-Gm-Message-State: AOJu0YwCbScZzb+e326pdWOA0ARcK/b3EX30wGZTslVqGl2Zb1X7Xnz0 jfNaAVA/PZIZ/QahCvO7UurFOGyq1cWYbOn550Gi+lkkrU9T0VBC4zhPi+O1GTTw+sFCxI49aMF S6vQ= X-Google-Smtp-Source: AGHT+IE6PkSzQ9upAW/+h4ySdbTPAHT9Hedbca4pydRvsTnIEYA317qAxi/TyyqLq6zBbpN/6H/78w== X-Received: by 2002:a05:6a20:748c:b0:1db:e4c3:99f0 with SMTP id adf61e73a8af0-1dbe4c39a83mr6838240637.32.1730833121554; Tue, 05 Nov 2024 10:58:41 -0800 (PST) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-720bc2e95a6sm10269992b3a.145.2024.11.05.10.58.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Nov 2024 10:58:41 -0800 (PST) Date: Tue, 5 Nov 2024 10:58:39 -0800 From: Stephen Hemminger To: Morten =?UTF-8?B?QnLDuHJ1cA==?= Cc: Subject: Re: RFC - Tap io_uring PMD Message-ID: <20241105105839.36c85e8f@hermes.local> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35E9F86A@smartserver.smartshare.dk> References: <20241030145644.0b97f23c@hermes.local> <98CBD80474FA8B44BF855DF32C47DC35E9F858@smartserver.smartshare.dk> <20241031173450.26cdb54c@hermes.local> <98CBD80474FA8B44BF855DF32C47DC35E9F86A@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Sat, 2 Nov 2024 23:28:49 +0100 Morten Br=C3=B8rup wrote: > > > > > > > > Probably the hardest part of using io_uring is figuring out how to > > > > collect > > > > completions. The simplest way would be to handle all completions rx= =20 > > and =20 > > > > tx > > > > in the rx_burst function. =20 > > > > > > Please don't mix RX and TX, unless explicitly requested by the =20 > > application through the recently introduced "mbuf recycle" feature. > >=20 > > The issue is Rx and Tx share a single fd and ioring for completion is > > per fd. > > The implementation for ioring came from the storage side so initially > > it was for fixing > > the broken Linux AIO support. > >=20 > > Some other devices only have single interrupt or ring shared with rx/tx > > so not unique. > > Virtio, netvsc, and some NIC's. > >=20 > > The problem is that if Tx completes descriptors then there needs to be > > locking > > to prevent Rx thread and Tx thread overlapping. And a spin lock is a > > performance buzz kill. =20 >=20 > Brainstorming a bit here... > What if the new TAP io_uring PMD is designed to use two io_urings per por= t, one for RX and another one for TX on the same TAP interface? > This requires that a TAP interface can be referenced via two file descrip= tors (one fd for the RX io_uring and another fd for the TX io_uring), e.g. = by using dup() to create the additional file descriptor. I don't know if th= is is possible, and if it works with io_uring. There a couple of problems with multiple fd's. - multiple fds pointing to same internal tap queue are not going to get c= ompleted separately. - when multi-proc is supported, limit of 253 fd's in Unix domain IPC come= s into play - tap does not support tx only fd for queues. If fd is queue of tap, rece= ive fan out will go to it. If DPDK was more flexible, harvesting of completion could be done via anoth= er thread but that is not general enough to work transparently with all applications. Existing TAP device plays wit= h SIGIO, but signals are slower.