From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D0B5145584; Thu, 4 Jul 2024 23:03:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BA82940DCD; Thu, 4 Jul 2024 23:03:19 +0200 (CEST) Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by mails.dpdk.org (Postfix) with ESMTP id C87DA402A7 for ; Thu, 4 Jul 2024 23:03:18 +0200 (CEST) Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-52ea34ffcdaso474007e87.1 for ; Thu, 04 Jul 2024 14:03:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720126998; x=1720731798; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=xTHNs8t+p+D7gHzNb4G7OAEhUxZcjmaYTRJAQJYhr4Q=; b=Z5UHXcYVNxFWXDYEVGVGhSBA+NJD/2VjDqvOnQruGAue8Iq5NIetmpD/FhQ/fc1xZF 2GjPgiqtoQyzRYxvk2CTbAmQ1klINPq2hYfCywGaD38ZyjoaG7IJQKwLEonzp+Sg6zhw cGuBGzEqw9Vy9iUdifUK20i7p1Uk2vvFE52y3fzIej4I49zplCloahzQMdlPff4atHHc ppHWYpEr4P72wSPbxTa+ONiyt/V5Wz7cQSMh0da6m2lL+jnsWnCz9mFN604XRLH+2NQ9 haYoEZ2NcnZ5yUUII0DG5th+F61aHsDTScvri5tfZl7pzSf+6JRsaE8bpy+mUzOMSkx1 ADIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720126998; x=1720731798; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xTHNs8t+p+D7gHzNb4G7OAEhUxZcjmaYTRJAQJYhr4Q=; b=Tf5+FgjvCT8oHyvYK3T5AYW5jDaVWH2uP384vdP6YDnChP92dmbqfAJ0x8xjjyvh99 UtKBnGWAvSwj7EuTfJQP5D7eNAIlW1Wec2/O/l6rIkeAw+WducP891rvNkJvmkI/pNqi q+TaG3YBgRXF+k8tADkeesDweiU6qhN+WWfyyN0vIVePaSS2UB/+npq0W5HFQBVzazhL cve6kzA6NVxZjNVcMs5DKNQ4UyakSMEP+j711JFO3D46VYWsdgqT+w6jEcy/NxeHSwg0 8On7DCQGrB+FZIqcZkq3PgB4RXqycngBopSMvOHImjabjMM16MNrF2Oq9s63A2vK+sjW oZhA== X-Gm-Message-State: AOJu0Ywwv8/7Z5HLMlZnluE7Babe36oRhuz5jc6v+jM91UU4OSkmka7s OPp5zeF/Mq2CXHlT7rjqN9nwhT8FSzGPprKLiETdqblM+pm4lA9A X-Google-Smtp-Source: AGHT+IHHtfCVmMZI8hf+ccTCeTLzZkmmY1pbpdnvqjg3neAfkWvvoJVntcsG8t1cnAZVQeSu5Jhr1Q== X-Received: by 2002:ac2:4545:0:b0:52e:9c0a:3522 with SMTP id 2adb3069b0e04-52ea06bb604mr1813125e87.44.1720126997865; Thu, 04 Jul 2024 14:03:17 -0700 (PDT) Received: from sovereign (broadband-109-173-110-33.ip.moscow.rt.ru. [109.173.110.33]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-52e9d607d7esm411991e87.295.2024.07.04.14.03.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Jul 2024 14:03:16 -0700 (PDT) Date: Fri, 5 Jul 2024 00:03:15 +0300 From: Dmitry Kozlyuk To: =?UTF-8?B?TcOhcmlv?= Kuka Cc: dev@dpdk.org, orika@nvidia.com, bingz@nvidia.com, viktorin@cesnet.cz Subject: Re: Hairpin Queues Throughput ConnectX-6 Message-ID: <20240705000315.710f2491@sovereign> In-Reply-To: <95896e0b-648c-40ee-986e-2f4c6d0bb29a@cesnet.cz> References: <3d746dbc-330e-403f-b87f-bf495cac3437@cesnet.cz> <20240625032224.45b65339@sovereign> <82d8f67c-3b0b-46c2-a94b-8457d0c602c2@cesnet.cz> <95896e0b-648c-40ee-986e-2f4c6d0bb29a@cesnet.cz> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org 2024-07-04 13:08 (UTC+0200), M=C3=A1rio Kuka: [...] > So I can't achieve my goal: traffic from the hairpin queues is not=20 > dropped if the CPU queue is overloaded. > Any idea how to achieve this in example 4? > What is the problem, full packet buffers/memory in the device that are=20 > shared between the hairpin and CPU queues? >=20 > Any guidance or suggestions on how to achieve this would be greatly=20 > appreciated. So you want priority traffic to use a dedicated HW buffer pool. Good news: QoS is the mechanism to do it. Bad news: flow rules cannot be used to determine priority, so you need to mark packets with VLAN PCP or IPv4 DSCP. I've reproduced your results roughly with 60 Mpps @ PCP 0 (10 Mpps with MAC for hairpin, 50 Mpps with MAC for normal RxQ), then switched to 10 Mpps @ PCP 0 + 50 Mpps @ PCP 1 and this solved the issue (and so does 10 Mpps @ PCP 1 + 50 Mpps @ PCP 0). I expected the need to tune --buffer_size and --prio2buffer with mlnx_qos [= 1], but it appears to be unnecessary. No idea why this works and what buffer size is used for PCP 1. [1]: https://enterprise-support.nvidia.com/s/article/mlnx-qos