From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6860B45EBC; Mon, 16 Dec 2024 10:58:24 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F36424025F; Mon, 16 Dec 2024 10:58:23 +0100 (CET) Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by mails.dpdk.org (Postfix) with ESMTP id 9E97F40144 for ; Mon, 16 Dec 2024 10:58:21 +0100 (CET) Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2efe25558ddso2718473a91.2 for ; Mon, 16 Dec 2024 01:58:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734343101; x=1734947901; darn=dpdk.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=s3VISLzs0OM6HsNTP9ky+qVyIg07a1adO/2S2H5SmLA=; b=R7WPiM2J+TtcEiIj4/Sud8H2OouYcve+MLPQG4h76FNaEj1cGAJ8SclF13aDGblSaM Iegd8HxRP+zeiztY3lDwNoV/G31Jg2hkILoDHkVondqEjBaAlo+/atyDiCezplDBOP04 vC8TqgpEnFLU5zqjb/K6SQbXwkNmEKZtRjHOhmMg6bkbRJeHnaZChZ0GzJxBehxcL5sy V8cjqVLqIgI/I7Hdpu5d+Ns7Te+VVr12WWmlgB9ixkGcMfZHgimg8YU9oQ3va/5mVC+v S0L3Aq7dLtO9ToTQ0rFeLXPWO/pDZyNKmviwaHpHsZbJTNzrDUprTlKMyzSf+ZoYLpvS gZhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734343101; x=1734947901; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=s3VISLzs0OM6HsNTP9ky+qVyIg07a1adO/2S2H5SmLA=; b=t2tEZPfAPVuPPygpkX7cZOHBIhm9FrhijBVLHCOrIUruf/6CL1lsazpIiVKcte2Pcc e0DlfKRdC7BiE5iuBxftTITMRNFOxk9dzIfwy3PblhEES0XPh9eLaKpocnNuKW4L5pN9 tixAgeVmHARMmk13SJ2SmEJhej0DAbbfYGQj4RM8EDkEk/6FmyE627POHokXumrJ94IZ s8Fp8siZWPmKbx7uUjGR5VVRIQnHgceXKYYT+FZ668mnpulW9XtoK1Da2QWYfy5p3wHt iPgJr/tlnSnldL9XtFjrtQlAzJpg3m8vrh2zHmEzZMcsms3xebF4enS1Ono/2DhAgg// Gbdg== X-Forwarded-Encrypted: i=1; AJvYcCVPzPtxshUHSZFYXjeTkyt1sNHf2kupwO1KNMnUb48XJRUEfOii3Z5yQDG/Fkghis4ZFf0=@dpdk.org X-Gm-Message-State: AOJu0Ywua6UTqL83SH7ioiUYkpIEhe5Uzo8VkMRN8fHUis4eTjw09U/N kWa+HNsBG0P8nYexwny5ld8+4j3JRDklVf4QchRjHNHnfCLlw3OiJaf1OxUYmcTneidAR8O6Z0e mddajw9KIMKdDbhg+QAwXJL32Nug= X-Gm-Gg: ASbGncsRHgAA6EGcgmpfelMPNXGC+FkSiIfr5Pw4AW9Le+2vfaZyr6/CJLpcMEuJ4lS Ucx7j4EP+0i1zpA7mnF7a9X8S458faZVM/c5ueaPX X-Google-Smtp-Source: AGHT+IFnNu+oL9WO6PDZQlyZxFvX+d5zGKoOuTJJuIDa5WoT8C/X4X6Qb8KLOsjq2ru27JKO0ezF80NYJRQTYUrcPxE= X-Received: by 2002:a17:90b:4a4c:b0:2ee:ee5e:42fb with SMTP id 98e67ed59e1d1-2f28fb6f027mr16916704a91.13.1734343100632; Mon, 16 Dec 2024 01:58:20 -0800 (PST) MIME-Version: 1.0 References: <20241212080442.1628366-1-tudor.cornea@gmail.com> <20241212091102.20422f95@hermes.local> In-Reply-To: <20241212091102.20422f95@hermes.local> From: Tudor Cornea Date: Mon, 16 Dec 2024 11:58:09 +0200 Message-ID: Subject: Re: [PATCH] net/af_packet: allow disabling packet fanout To: Stephen Hemminger Cc: linville@tuxdriver.com, ferruh.yigit@amd.com, andrew.rybchenko@oktetlabs.ru, dev@dpdk.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > Controlling fanout more is a good idea but not sure what this patch > is trying to do with it. I am maintaining a DPDK application which should run on a large number of setups. Unfortunately, I do not have a lot of control over the environment the application runs on (e.g kernel). The problem I was trying to solve is that the Af_Packet PMD seems to fail to initialize properly on some setups. I managed to reproduce the issue with testpmd. sudo ./x86_64-native-linuxapp-gcc/app/dpdk-testpmd \ -l 0-3 \ -m 1024 \ --no-huge \ --no-shconf \ --no-pci \ --vdev=net_af_packet0,iface=eth1,blocksz=4096,framesz=2048,framecnt=512,qpairs=1,qdisc_bypass=0 \ --vdev=net_af_packet1,iface=eth2,blocksz=4096,framesz=2048,framecnt=512,qpairs=1,qdisc_bypass=0 \ -- \ -i Error: Unknown device type. EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL: Detected static linkage of DPDK EAL: Selected IOVA mode 'VA' AFPACKET: rte_pmd_init_internals(): net_af_packet0: could not set PACKET_FANOUT on AF_PACKET socket for eth1:Invalid argument VDEV_BUS: vdev_probe(): failed to initialize net_af_packet0 device AFPACKET: rte_pmd_init_internals(): net_af_packet1: could not set PACKET_FANOUT on AF_PACKET socket for eth1:Invalid argument VDEV_BUS: vdev_probe(): failed to initialize net_af_packet1 device EAL: Bus (vdev) probe failed. testpmd: No probed ethernet devices It seems that the call to PACKET_FANOUT setsockopt failed with EINVAL. I debugged the issue further. It seems that the root cause is that when the interface is down (e.g eth1), after the bind operation succeeds, the PACKET_FANOUT setsockopt will fail. This is a behavior of AF_Packet in Linux which I'm not sure is that well documented. It seems to be fixed in a recent Linux Kernel commit. I re-compiled the kernel with the change, and did not see the issue afterwards. Commit 2cee3e6e2e4b74bec96694169f01cd3feec1f264 af_packet: allow fanout_add when socket is not RUNNING [1] https://github.com/torvalds/linux/commit/2cee3e6e2e4b74bec96694169f01cd3feec1f264 I was trying to find a solution for my application to run on setups which don't have the kernel patch. Introducing a devarg to control when packet_fanout is enabled seemed like a possible way around the issue. As a second solution around the issue, I was also thinking of using the PACKET_FANOUT setsockopt only when nb_queues > 1. Conceptually, I think it would make some sense, because fanout doesn't really help when having a single socket. We would need more sockets in order to load balance incoming packets. And this would still allow me to control the behavior from the existing 'qpairs' devarg. The idea would be something along these lines if (nb_queues > 1) { rc = setsockopt(qsockfd, SOL_PACKET, PACKET_FANOUT, &fanout_arg, sizeof(fanout_arg)); /* ... */ } Would a patch using this idea be considered more suitable ? > - DPDK minimum kernel version is now 4.19 so no point in worrying about > backward compatibility. According to man page for packet, fanout > was added in 3.1 kernel. > > - It would be useful to allow application to control fanout in more detail. This is a great idea. Would introducing a new devarg, (e.g 'fanout_mode') be the proper way to allow the application to customize fanout in more detail ? --vdev=net_af_packet0,iface=eth1,blocksz=4096,framesz=2048,framecnt=512,qpairs=1,fanout_mode=[fanout_hash|fanout_cpu|fanout_rnd|fanout_qm]